Demosaicing method and circuit based on raw data for edge detection

Abstract

This invention discloses a demosaic method and circuit based on raw data edge detection. The method includes: performing edge detection on the raw data to determine whether there are pixels located on edges; the raw data includes green, blue, and red pixels, and the raw data is distributed in a nine-square grid; if there are pixels located on edges in the raw data, the edge direction of the detected edge pixels in the raw data is determined; and demosaic processing is performed on edge pixels and non-edge pixels separately. This invention distinguishes pixels located on edges from pixels located elsewhere by performing edge detection on the raw data. For pixels on edges, after determining the direction of the edge pixels, demosaic processing is performed on the center pixels of the nine-square grid to restore the edge pixels, thereby eliminating the jagged edges in the demosaiced image, avoiding color blurring of pixels on edges, and improving the image edge restoration effect.

Description

Technical Field

[0001] This invention relates to the field of electronic technology, and in particular to a method and circuit for demosaicing based on raw data edge detection. Background Technology

[0002] Most current consumer digital cameras obtain pixel information for captured images by overlaying a Color Filter Array (CFA) layer on top of the image sensor. The resulting raw image data is called raw data. In raw data, each pixel contains only the pixel value of one of the three primary colors: red, green, and blue. The process of processing raw data to recover the missing pixel data and obtain an RGB three-channel color image is called demosaic. Because the demosaic process directly affects the quality of the color images captured by digital cameras, it is crucial for both simple image capture and digital image processing tasks such as image recognition and image enhancement.

[0003] Currently, both consumer-grade and industrial digital cameras typically employ traditional interpolation de-mosaic algorithms. While these algorithms are relatively simple to calculate, fast to process, and consume less power, they are less effective at restoring details such as image edges.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a demosaic method and circuit based on raw data for edge detection, so as to solve the problem that the traditional interpolation demosaic algorithm has poor image edge restoration effect.

[0006] The technical solution of the present invention is as follows:

[0007] A demosaic method based on raw data edge detection includes:

[0008] Edge detection is performed on the raw data to determine whether there are pixels located on the edges; wherein, the raw data includes green pixels, blue pixels and red pixels, and the raw data is distributed in a nine-square grid;

[0009] If there are pixels on the edge in the raw data, then determine the edge direction of the detected edge pixels in the raw data;

[0010] Demosaic processing is performed on edge pixels and non-edge pixels separately.

[0011] In the above technical solution, the present invention performs edge detection on raw data to distinguish pixels located on the edge from pixels located elsewhere. For non-edge pixels, a traditional linear interpolation method is used for demosaicing. For pixels on the edge, after determining the direction of the edge pixels, the center pixel of the nine-grid is demosaiced to restore the edge pixels. In this way, the jagged edges in the demosaiced image can be eliminated, and the colors of pixels on the edge are prevented from becoming blurred, thereby improving the image edge restoration effect.

[0012] In a further embodiment of the present invention, the step of performing edge detection on the raw data to determine whether there are pixels located on the edge includes:

[0013] When the center pixel of the 3x3 grid is a red pixel, obtain the maximum and minimum pixel values ​​of the green pixels, and obtain the maximum and minimum pixel values ​​of the blue pixels;

[0014] The green pixels include a first green pixel, a second green pixel, a third green pixel, and a fourth green pixel, wherein the first green pixel and the third green pixel are distributed opposite to each other, and the second green pixel and the fourth green pixel are distributed opposite to each other; the blue pixels include a first blue pixel, a second blue pixel, a third blue pixel, and a fourth blue pixel, wherein the first blue pixel and the second blue pixel are distributed opposite to each other on both sides of the first green pixel, and the third blue pixel and the fourth blue pixel are distributed opposite to each other on both sides of the third green pixel;

[0015] Calculate the difference between the maximum and minimum pixel values ​​of the green pixels, and calculate the difference between the maximum and minimum pixel values ​​of the blue pixels;

[0016] If the difference between the maximum and minimum values ​​of the green pixel is greater than one-quarter of the maximum value of the green pixel, then it is determined that there is a pixel located on the edge among the green pixels.

[0017] If the difference between the maximum and minimum values ​​of the blue pixel is greater than one-quarter of the maximum value of the blue pixel, then it is determined that there is a pixel located on the edge among the blue pixels.

[0018] In a further embodiment of the present invention, the step of performing edge detection on the raw data and determining whether there are pixels located on the edge further includes:

[0019] When the center pixel of the 3x3 grid is a blue pixel, obtain the maximum and minimum pixel values ​​of the green pixels, and obtain the maximum and minimum pixel values ​​of the red pixels;

[0020] The red pixel includes a first red pixel, a second red pixel, a third red pixel, and a fourth red pixel. The first red pixel and the second red pixel are distributed opposite each other on both sides of the first green pixel, and the third red pixel and the fourth red pixel are distributed opposite each other on both sides of the third green pixel.

[0021] Calculate the difference between the maximum and minimum pixel values ​​of the green pixels, and calculate the difference between the maximum and minimum pixel values ​​of the red pixels;

[0022] If the difference between the maximum and minimum values ​​of the green pixel is greater than one-quarter of the maximum value of the green pixel, then it is determined that there is a pixel located on the edge among the green pixels.

[0023] If the difference between the maximum and minimum values ​​of the red pixel is greater than one-quarter of the maximum value of the red pixel, then it is determined that there is a pixel located on the edge among the red pixels.

[0024] In a further embodiment of the present invention, the step of determining the edge direction of the detected edge pixels in the raw data if there are pixels located on the edge includes:

[0025] Compare the absolute value of the difference between the pixel value of the first green pixel and the pixel value of the third green pixel with the absolute value of the difference between the pixel value of the second green pixel and the pixel value of the fourth green pixel;

[0026] If the absolute value of the difference between the pixel value of the first green pixel and the pixel value of the third green pixel is greater than the absolute value of the difference between the pixel value of the second green pixel and the pixel value of the fourth green pixel, then the direction of the edge pixels in the green pixel is determined to be horizontal; otherwise, the direction of the edge pixels in the green pixel is determined to be vertical.

[0027] Compare the absolute value of the difference between the pixel value of the first blue pixel and the pixel value of the third blue pixel with the absolute value of the difference between the pixel value of the second blue pixel and the pixel value of the fourth blue pixel;

[0028] If the absolute value of the difference between the pixel value of the first blue pixel and the pixel value of the third blue pixel is greater than the absolute value of the difference between the pixel value of the second blue pixel and the pixel value of the fourth blue pixel, then the edge direction of the blue pixel is the diagonal direction of the lower left and upper right; otherwise, the edge direction of the blue pixel is the diagonal direction of the upper left and lower right.

[0029] In a further embodiment of the present invention, the step of determining the edge direction of the detected edge pixels in the raw data if there are pixels located on the edge includes:

[0030] Compare the absolute value of the difference between the pixel value of the first green pixel and the pixel value of the third green pixel with the absolute value of the difference between the pixel value of the second green pixel and the pixel value of the fourth green pixel;

[0031] If the absolute value of the difference between the pixel value of the first green pixel and the pixel value of the third green pixel is greater than the absolute value of the difference between the pixel value of the second green pixel and the pixel value of the fourth green pixel, then the direction of the edge pixels in the green pixel is determined to be horizontal; otherwise, the direction of the edge pixels in the green pixel is determined to be vertical.

[0032] Compare the absolute value of the difference between the pixel value of the first red pixel and the pixel value of the third red pixel with the absolute value of the difference between the pixel value of the second red pixel and the pixel value of the fourth red pixel;

[0033] If the absolute value of the difference between the pixel value of the first red pixel and the pixel value of the third red pixel is greater than the absolute value of the difference between the pixel value of the second red pixel and the pixel value of the fourth red pixel, then the edge direction of the red pixel is the diagonal direction of the lower left and upper right; otherwise, the edge direction of the red pixel is the diagonal direction of the upper left and lower right.

[0034] A further provision of the present invention includes the step of performing demosaic processing on edge pixels and non-edge pixels respectively, comprising:

[0035] Calculate the first average of the pixel values ​​of the green pixels;

[0036] When the direction of the edge pixels in the green pixel is horizontal, calculate the difference between the pixel value of the first green pixel, the pixel value of the third green pixel, and the first average value.

[0037] Compare the difference between the pixel value of the first green pixel and the first average value with the difference between the pixel value of the third green pixel and the first average value. If the difference between the pixel value of the first green pixel and the first average value is greater than the difference between the pixel value of the third green pixel and the first average value, then the first green pixel is a pixel outside the object; otherwise, the third green pixel is a pixel outside the object.

[0038] When the direction of the edge pixels in the green pixel is vertical, calculate the difference between the pixel value of the second green pixel, the pixel value of the fourth green pixel, and the first average value.

[0039] Compare the difference between the second green pixel value and the first average value with the difference between the fourth green pixel value and the first average value. If the difference between the second green pixel value and the first average value is greater than the difference between the fourth green pixel value and the first average value, then the second green pixel is a pixel outside the object; otherwise, the fourth green pixel is a pixel outside the object.

[0040] A further provision of the present invention includes the step of performing demosaic processing on edge pixels and non-edge pixels respectively, comprising:

[0041] Calculate the second average of the pixel values ​​of the blue pixels;

[0042] When the edge direction of the blue pixel is the diagonal direction from the lower left to the upper right, calculate the difference between the pixel value of the first blue pixel and the second average value and the difference between the pixel value of the third blue pixel.

[0043] Compare the difference between the pixel value of the first blue pixel and the second average value with the difference between the pixel value of the third blue pixel and the second average value. If the difference between the pixel value of the first blue pixel and the second average value is greater than the difference between the pixel value of the third blue pixel and the second average value, then the first blue pixel is a pixel outside the object; otherwise, the third blue pixel is a pixel outside the object.

[0044] When the edge direction of the blue pixel is the upper left and lower right diagonal direction, calculate the difference between the pixel value of the second blue pixel and the second average value and the difference between the pixel value of the fourth blue pixel.

[0045] Compare the difference between the pixel value of the second blue pixel and the second average value with the difference between the pixel value of the fourth blue pixel and the second average value. If the difference between the pixel value of the second blue pixel and the second average value is greater than the difference between the pixel value of the fourth blue pixel and the second average value, then the second blue pixel is a pixel outside the object; otherwise, the fourth blue pixel is a pixel outside the object.

[0046] A further provision of the present invention includes the step of performing demosaic processing on edge pixels and non-edge pixels respectively, comprising:

[0047] Calculate the third average of the pixel values ​​of the red pixels;

[0048] When the edge direction of the red pixel is the diagonal direction from the lower left to the upper right, calculate the difference between the pixel value of the first red pixel and the third average value, and the difference between the pixel value of the third red pixel and the third average value, respectively.

[0049] Compare the difference between the pixel value of the first red pixel and the third average value with the difference between the pixel value of the third red pixel and the third average value. If the difference between the pixel value of the first red pixel and the third average value is greater than the difference between the pixel value of the third red pixel and the third average value, then the first red pixel is a pixel outside the object; otherwise, the third red pixel is a pixel outside the object.

[0050] When the edge direction of the red pixel is the upper left and lower right diagonal direction, calculate the difference between the pixel value of the second red pixel and the third average value, and the difference between the pixel value of the fourth red pixel and the third average value.

[0051] Compare the difference between the pixel value of the second red pixel and the third average value with the difference between the pixel value of the fourth red pixel and the third average value. If the difference between the pixel value of the second red pixel and the third average value is greater than the difference between the pixel value of the fourth red pixel and the third average value, then the second red pixel is a pixel outside the object; otherwise, the fourth red pixel is a pixel outside the object.

[0052] A further provision of the present invention includes the step of performing demosaic processing on edge pixels and non-edge pixels respectively, comprising:

[0053] The green channel pixel value of the center pixel of the nine-grid is calculated based on the first average value and the pixel value of the pixels outside the object.

[0054] The blue channel pixel value is recovered from the center pixel of the 3x3 grid based on the second average value and the pixel values ​​of pixels outside the object.

[0055] A further provision of the present invention includes the step of performing demosaic processing on edge pixels and non-edge pixels respectively, comprising:

[0056] The green channel pixel value of the center pixel of the nine-grid is calculated based on the first average value and the pixel value of the pixels outside the object.

[0057] The red channel pixel value of the center pixel of the 3x3 grid is calculated based on the third average value and the pixel values ​​of pixels outside the object.

[0058] Based on the same inventive concept, the present invention also provides a demosaic circuit for edge detection based on raw data, used to implement the demosaic method for edge detection based on raw data as described above, comprising:

[0059] The raw data nine-square grid composition module is used to construct the 3x3 raw data nine-square grid required for each calculation.

[0060] The calculation module is used to calculate the maximum, minimum, and average values ​​of the input pixel data;

[0061] The edge pixel recovery module is used to perform pixel edge detection based on the maximum, minimum and average values ​​of the input pixel count, and to perform de-mosaic processing on the center pixels of the nine-grid based on the detection results.

[0062] The de-mosaic pixel value output module is used to output the pixel value of the center pixel of the 3x3 grid after de-mosaicing.

[0063] In a further embodiment of the present invention, the raw data 3x3 grid construction module includes: a row buffer and a register group, wherein the row buffer and the register group are connected.

[0064] In a further embodiment of the present invention, the calculation module includes: a first calculation unit, a second calculation unit, a third calculation unit, a fourth calculation unit, and a first average value calculation unit;

[0065] The first calculation unit is connected to the register group and is used to calculate the maximum value of the green pixel;

[0066] The second calculation unit is connected to the register group and is used to calculate the maximum value of the blue pixel;

[0067] The third calculation unit is connected to the register group and is used to calculate the minimum pixel value of the green pixel;

[0068] The fourth calculation unit is connected to the register group and is used to calculate the minimum pixel value of the blue pixel;

[0069] The first average value calculation unit is connected to the register group and is used to calculate the first average value of the pixel values ​​of the green pixels and the second average value of the pixel values ​​of the blue pixels.

[0070] In a further embodiment of the present invention, the calculation unit further includes: a fifth calculation unit, a sixth calculation unit, and a second average value calculation unit;

[0071] The fifth calculation unit is connected to the register group and is used to calculate the maximum value of the red pixel.

[0072] The sixth calculation unit is connected to the register group and is used to calculate the minimum pixel value of the red pixel;

[0073] The second average value calculation unit is connected to the register group and is used to calculate the third average value of the pixel value of the red pixel.

[0074] In a further embodiment of the present invention, the edge pixel recovery module includes: a first edge pixel recovery unit, a second edge pixel recovery unit, a third edge pixel recovery unit, and a fourth edge pixel recovery unit;

[0075] The first edge pixel recovery unit is connected to the first calculation unit, the third calculation unit and the first average value calculation unit respectively, and is used to perform pixel edge detection based on the maximum value of the green pixel, the minimum value of the green pixel and the first average value of the green pixel, and to perform de-mosaic processing on the red center pixel of the nine-square grid based on the detection result.

[0076] The second edge pixel recovery unit is connected to the second calculation unit, the fourth calculation unit and the first average value calculation unit respectively, and is used to perform pixel edge detection based on the maximum value of the blue pixel, the minimum value of the blue pixel and the second average value of the blue pixel, and to perform de-mosaic processing on the red center pixel of the nine-square grid based on the detection result.

[0077] The third edge pixel recovery unit is connected to the first calculation unit, the third calculation unit and the first average value calculation unit respectively, and is used to perform pixel edge detection based on the maximum value of the green pixel, the minimum value of the green pixel and the first average value of the green pixel, and to perform de-mosaic processing on the blue center pixel of the nine-grid according to the detection result.

[0078] The fourth edge pixel recovery unit is connected to the fifth calculation unit, the sixth calculation unit and the second average value calculation unit, respectively, and is used to perform pixel edge detection based on the maximum value of the red pixel, the minimum value of the red pixel and the third average value of the red pixel, and to perform de-mosaic processing on the blue center pixel of the nine-grid according to the detection results. Attached Figure Description

[0079] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0080] Figure 1 This is a flowchart illustrating the demosaic method based on raw data for edge detection in this invention.

[0081] Figure 2 This is a distribution chart of four types of raw data for the input pixel nine-grid in Bayer CFA.

[0082] Figure 3 This is a schematic diagram illustrating the edge direction determination of the input pixel nine-grid in this invention, with the blue pixel as the center pixel. Figure 1 .

[0083] Figure 4 This is a schematic diagram illustrating the edge direction determination of the input pixel nine-grid in this invention, with the blue pixel as the center pixel. Figure 2 .

[0084] Figure 5 This is a schematic diagram illustrating the edge direction determination of the input pixel nine-grid in this invention, with the red pixel as the center pixel. Figure 1 .

[0085] Figure 6 This is a schematic diagram illustrating the edge direction determination of the input pixel nine-grid in this invention, with the red pixel as the center pixel. Figure 2 .

[0086] Figure 7 This is a framework diagram of the demosaic circuit for edge detection based on raw data in this invention.

[0087] Figure 8 This is a framework diagram of the demosaic circuit based on raw data for edge detection, with the red pixel as the center pixel in the input pixel nine-grid.

[0088] Figure 9 yes Figure 8 A schematic diagram of the structure of the first edge pixel recovery unit.

[0089] The following are the markings in the attached diagram: 1. Raw data nine-grid composition module; 2. Calculation module; 21. First calculation unit; 22. Second calculation unit; 23. Third calculation unit; 24. Fourth calculation unit; 25. First average value calculation unit; 3. Edge pixel recovery module; 31. First edge pixel recovery unit; 32. Second edge pixel recovery unit; 4. De-mosaic pixel value output module. Detailed Implementation

[0090] This invention provides a method and circuit for demosaicing based on raw data edge detection. To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0091] In the implementation methods and claims, unless otherwise specified in the text, the terms "a," "an," "the," and "the" may also include plural forms. If the embodiments of the present invention involve descriptions of "first," "second," etc., such descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features.

[0092] It should be further understood that the term "comprising" as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, "connected" or "coupled" as used herein can include wireless connections or wireless coupling. The term "and / or" as used herein includes all or any unit and all combinations of one or more associated listed items.

[0093] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0094] Furthermore, the technical solutions of the various embodiments can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0095] The inventors discovered that current digital cameras, whether for consumer or industrial use, typically employ traditional interpolation demosaic algorithms. While these algorithms are computationally simple, fast, and consume little power, they are poor at restoring details such as image edges. Although some more effective demosaic algorithms have been proposed in recent years, these algorithms are mostly computationally complex and consume significant hardware resources, making them difficult to apply to scenarios requiring low cost and real-time performance.

[0096] To address the aforementioned technical problems, this invention provides a demosaic method and circuit based on raw data edge detection. By performing edge detection on the raw data, pixels located on edges are distinguished from pixels in other positions. For non-edge pixels, traditional linear interpolation is used for demosaicing. For edge pixels, after determining the direction of the edge pixels, demosaicing is performed on the center pixel of the 3x3 grid to restore the edge pixels. This eliminates the jagged edges present in the demosaiced image and prevents the colors of edge pixels from becoming blurred, thereby improving the edge restoration effect. Furthermore, this invention only requires a single 3x3 register grid centered on the pixel to be restored when identifying edges. This requires only two rows of buffers and a few individual registers in hardware, maintaining good demosaic performance while reducing hardware resource requirements and power consumption.

[0097] Please also refer to Figures 1 to 6 The present invention provides a preferred embodiment of a demosaic method based on raw data for edge detection.

[0098] like Figure 1 As shown, the present invention provides a demosaic method based on raw data edge detection, comprising the following steps:

[0099] S100. Perform edge detection on the raw data to determine whether there are pixels located on the edge; wherein, the raw data includes green pixels, blue pixels and red pixels, and the raw data is distributed in a nine-square grid;

[0100] Among them, the input raw data contains, for example, Figure 2 The four raw data distribution scenarios shown depict input raw data distributed in a 3x3 raw data grid, with each grid containing 9 pixels. The first and second scenarios are also relevant. Figure 2 When the input data is distributed in a grid (a and b), i.e., with the green pixel as the center pixel, edge detection is not performed on the input grid. However, in the third and fourth grid distributions... Figure 2 The nine-grid distribution of input data (including c and d types) only requires edge detection on the raw input data when the center pixel is either a red or blue pixel. This invention processes one nine-grid distribution of input data in one computation cycle.

[0101] Please see Figures 1 to 4 Step S100 includes:

[0102] S101. When the center pixel of the 3x3 grid is a red pixel, obtain the maximum and minimum pixel values ​​of the green pixels, and obtain the maximum and minimum pixel values ​​of the blue pixels.

[0103] The green pixels include a first green pixel, a second green pixel, a third green pixel, and a fourth green pixel, wherein the first green pixel and the third green pixel are distributed opposite to each other, and the second green pixel and the fourth green pixel are distributed opposite to each other; the blue pixels include a first blue pixel, a second blue pixel, a third blue pixel, and a fourth blue pixel, wherein the first blue pixel and the second blue pixel are distributed opposite to each other on both sides of the first green pixel, and the third blue pixel and the fourth blue pixel are distributed opposite to each other on both sides of the third green pixel;

[0104] S102. Calculate the difference between the maximum and minimum pixel values ​​of the green pixel, and calculate the difference between the maximum and minimum pixel values ​​of the blue pixel;

[0105] S103. If the difference between the maximum and minimum values ​​of the green pixel is greater than one-quarter of the maximum value of the green pixel, then it is determined that there is a pixel located on the edge among the green pixels.

[0106] If the difference between the maximum and minimum values ​​of the blue pixel is greater than one-quarter of the maximum value of the blue pixel, then it is determined that there is a pixel located on the edge among the blue pixels.

[0107] Specifically, when the red pixel is the center pixel of the 3x3 grid, the maximum and minimum values ​​of the green pixels are obtained, namely the maximum value Max1 and minimum value Min1 of the first green pixel G1, the second green pixel G2, the third green pixel G3, and the fourth green pixel G4, and the maximum value Max2 and minimum value Min2 of the first blue pixel B1, the second blue pixel B2, the third blue pixel B3, and the fourth blue pixel B4, and expressed by the following formula:

[0108] Max1=Max(G1,G2,G3,G4); Max2=Max(B1,B2,B3,B4);

[0109] Min1=Min(G1,G2,G3,G4); Min2=Min(B1,B2,B3,B4);

[0110] Then, the difference Range1 between the maximum value Max1 and the minimum value Min1 of the green pixel is calculated, and the difference Range2 between the maximum value Max2 and the minimum value Min2 of the blue pixel is calculated, and expressed by the following formula:

[0111] Range1 = Max1 – Min1;

[0112] Range2 = Max2 – Min2;

[0113] When performing edge detection on the input first green pixel G1, second green pixel G2, third green pixel G3 and fourth green pixel G4, if Range1>Max1 / 4, it is determined that there are pixels located on the edge among the first green pixel G1, second green pixel G2, third green pixel G3 and fourth green pixel G4; otherwise, it is determined that none of the first green pixel G1, second green pixel G2, third green pixel G3 and fourth green pixel G4 are located on the edge.

[0114] Next, edge detection is performed on the first blue pixel B1, the second blue pixel B2, the third blue pixel B3, and the fourth blue pixel B4. If Range2 > Max2 / 4, it is determined that there are pixels on the edge among the first green pixel G1, the second green pixel G2, the third green pixel G3, and the fourth green pixel G4. Otherwise, it is determined that none of the first green pixel G1, the second green pixel G2, the third green pixel G3, and the fourth green pixel G4 are located on the edge.

[0115] Step S100 further includes:

[0116] S104. When the center pixel of the 3x3 grid is a blue pixel, obtain the maximum and minimum pixel values ​​of the green pixels, and obtain the maximum and minimum pixel values ​​of the red pixels.

[0117] The red pixel includes a first red pixel, a second red pixel, a third red pixel, and a fourth red pixel. The first red pixel and the second red pixel are distributed opposite each other on both sides of the first green pixel, and the third red pixel and the fourth red pixel are distributed opposite each other on both sides of the third green pixel.

[0118] S105. Calculate the difference between the maximum and minimum pixel values ​​of the green pixel, and calculate the difference between the maximum and minimum pixel values ​​of the red pixel;

[0119] S106. If the difference between the maximum and minimum values ​​of the green pixel is greater than one-quarter of the maximum value of the green pixel, then it is determined that there is a pixel located on the edge among the green pixels.

[0120] If the difference between the maximum and minimum values ​​of the red pixel is greater than one-quarter of the maximum value of the red pixel, then it is determined that there is a pixel located on the edge among the red pixels.

[0121] Specifically, when the 3x3 grid is centered on a blue pixel, the maximum and minimum values ​​of the green pixels are obtained, namely, the maximum value Max1 and minimum value Min1 of the first green pixel G1, the second green pixel G2, the third green pixel G3, and the fourth green pixel G4, and the maximum value Max3 and minimum value Min3 of the first red pixel R1, the second red pixel R2, the third red pixel R3, and the fourth red pixel R4, and expressed by the following formula:

[0122] Max1=Max(G1,G2,G3,G4); Max3=Max(R1,R2,R3,R4);

[0123] Min1=Min(G1,G2,G3,G4); Min3=Min(R1,R2,R3,R4);

[0124] Then, the difference Range1 between the maximum value Max1 and the minimum value Min1 of the green pixel is calculated, and the difference Range3 between the maximum value Max3 and the minimum value Min3 of the red pixel is calculated, and expressed by the following formula:

[0125] Range1 = Max1 – Min1;

[0126] Range3 = Max3 – Min3;

[0127] When performing edge detection on the input first green pixel G1, second green pixel G2, third green pixel G3, and fourth green pixel G4, if Range1 > Max1 / 4, it is determined that there are pixels located on the edge among the first green pixel G1, second green pixel G2, third green pixel G3, and fourth green pixel G4; otherwise, it is determined that none of the first green pixel G1, second green pixel G2, third green pixel G3, and fourth green pixel G4 are located on the edge.

[0128] Next, edge detection is performed on the input first red pixel R1, second red pixel R2, third red pixel R3 and fourth red pixel R4. If Range3 > Max3 / 4, it is determined that there are pixels located on the edge among the first red pixel R1, second red pixel R2, third red pixel R3 and fourth red pixel R4. Otherwise, it is determined that the first red pixel R1, second red pixel R2, third red pixel R3 and fourth red pixel R4 are not located on the edge.

[0129] S200. If there are pixels on the edge in the raw data, determine the edge direction of the detected edge pixels in the raw data.

[0130] Please see Figures 1 to 4 Wherein, step S200 includes the following steps:

[0131] S201. Compare the absolute value of the difference between the pixel value of the first green pixel and the pixel value of the third green pixel with the absolute value of the difference between the pixel value of the second green pixel and the pixel value of the fourth green pixel.

[0132] S202. If the absolute value of the difference between the pixel value of the first green pixel and the pixel value of the third green pixel is greater than the absolute value of the difference between the pixel value of the second green pixel and the pixel value of the fourth green pixel, then the direction of the edge pixels in the green pixel is determined to be horizontal; otherwise, the direction of the edge pixels in the green pixel is determined to be vertical.

[0133] Specifically, when an edge pixel is detected in the input green pixel, the absolute value of the difference between the pixel value of the first green pixel G1 and the pixel value of the third green pixel G3 is compared with the absolute value of the difference between the pixel value of the second green pixel G2 and the pixel value of the fourth green pixel G4. If the absolute value of the difference between the pixel value of the first green pixel G1 and the pixel value of the third green pixel G3 is greater than the absolute value of the difference between the pixel value of the second green pixel G2 and the pixel value of the fourth green pixel G4, i.e., |G1-G3|>|G2-G4|, then the direction of the edge pixel in the green pixel is determined to be horizontal. If the absolute value of the difference between the pixel value of the first green pixel G1 and the pixel value of the third green pixel G3 is less than the absolute value of the difference between the pixel value of the second green pixel G2 and the pixel value of the fourth green pixel G4, i.e., |G1-G3|<|G2-G4|, then the direction of the edge pixel in the green pixel is determined to be vertical.

[0134] S203. Compare the absolute value of the difference between the pixel value of the first blue pixel and the pixel value of the third blue pixel with the absolute value of the difference between the pixel value of the second blue pixel and the pixel value of the fourth blue pixel.

[0135] S204. If the absolute value of the difference between the pixel value of the first blue pixel and the pixel value of the third blue pixel is greater than the absolute value of the difference between the pixel value of the second blue pixel and the pixel value of the fourth blue pixel, then the edge direction of the blue pixel is the diagonal direction of the lower left and upper right; otherwise, the edge direction of the blue pixel is the diagonal direction of the upper left and lower right.

[0136] Specifically, if an edge pixel is detected among the input blue pixels, the absolute value of the difference between the pixel value of the first blue pixel B1 and the pixel value of the third blue pixel B3 is compared with the absolute value of the difference between the pixel value of the second blue pixel B2 and the pixel value of the fourth blue pixel B4. If the absolute value of the difference between the pixel value of the first blue pixel B1 and the pixel value of the third blue pixel B3 is greater than the absolute value of the difference between the pixel value of the second blue pixel B2 and the pixel value of the fourth blue pixel B4, then the edge direction in the blue pixel is the diagonal direction from bottom left to top right, i.e., |B1-B3| > |B2-B4|. If the absolute value of the difference between the pixel value of the first blue pixel B1 and the pixel value of the third blue pixel B3 is less than the absolute value of the difference between the pixel value of the second blue pixel B2 and the pixel value of the fourth blue pixel B4, i.e., |B1-B3| < |B2-B4|, then the edge direction in the blue pixel is the diagonal direction from top left to bottom right.

[0137] Please see Figure 1 , Figure 2 and combined Figure 5 and Figure 6 Step S200 further includes the following steps;

[0138] S205. Compare the absolute value of the difference between the pixel value of the first green pixel and the pixel value of the third green pixel with the absolute value of the difference between the pixel value of the second green pixel and the pixel value of the fourth green pixel.

[0139] S206. If the absolute value of the difference between the pixel value of the first green pixel and the pixel value of the third green pixel is greater than the absolute value of the difference between the pixel value of the second green pixel and the pixel value of the fourth green pixel, then the direction of the edge pixels in the green pixel is determined to be horizontal; otherwise, the direction of the edge pixels in the green pixel is determined to be vertical.

[0140] Specifically, when the blue pixel in the 3x3 grid is the center pixel, and when an edge pixel is detected among the input green pixels, the absolute value of the difference between the pixel value of the first green pixel G1 and the pixel value of the third green pixel G3 is compared with the absolute value of the difference between the pixel value of the second green pixel G2 and the pixel value of the fourth green pixel G4. If the absolute value of the difference between the pixel value of the first green pixel G1 and the pixel value of the third green pixel G3 is greater than the absolute value of the difference between the pixel value of the second green pixel G2 and the pixel value of the fourth green pixel G4, then... If the absolute value of the difference between the pixel values ​​of the four green pixels G4, i.e., |G1-G3| > |G2-G4|, then the direction of the edge pixels in the green pixels is determined to be horizontal. If the absolute value of the difference between the pixel value of the first green pixel G1 and the pixel value of the third green pixel G3 is less than the absolute value of the difference between the pixel value of the second green pixel G2 and the pixel value of the fourth green pixel G4, i.e., |G1-G3| < |G2-G4|, then the direction of the edge pixels in the green pixels is determined to be vertical.

[0141] S207. Compare the absolute value of the difference between the pixel value of the first red pixel and the pixel value of the third red pixel with the absolute value of the difference between the pixel value of the second red pixel and the pixel value of the fourth red pixel.

[0142] S208. If the absolute value of the difference between the pixel value of the first red pixel and the pixel value of the third red pixel is greater than the absolute value of the difference between the pixel value of the second red pixel and the pixel value of the fourth red pixel, then the edge direction of the red pixel is the diagonal direction of the lower left and upper right; otherwise, the edge direction of the red pixel is the diagonal direction of the upper left and lower right.

[0143] Specifically, when an edge pixel is detected among the input red pixels, the absolute value of the difference between the pixel value of the first red pixel R1 and the pixel value of the third red pixel R3 is compared with the absolute value of the difference between the pixel value of the second red pixel R2 and the pixel value of the fourth red pixel R4. If the absolute value of the difference between the pixel value of the first red pixel R1 and the pixel value of the third red pixel R3 is greater than the absolute value of the difference between the pixel value of the second red pixel R2 and the pixel value of the fourth red pixel R4, i.e., |R1-R3|>|R2-R4|, then the edge direction of the red pixel is the diagonal direction from left to right. If the absolute value of the difference between the pixel value of the first red pixel R1 and the pixel value of the third red pixel R3 is less than the absolute value of the difference between the pixel value of the second red pixel R2 and the pixel value of the fourth red pixel R4, i.e., |R1-R3|<|R2-R4|, then the edge direction of the red pixel is the diagonal direction from left to right.

[0144] S300: Perform demosaic processing on edge pixels and non-edge pixels respectively.

[0145] Step S300 includes:

[0146] S301. Calculate the first average value of the pixel values ​​of the green pixels;

[0147] Specifically, the first average value of the green pixel's pixel value, Aver1, is (G1 + G2 + G3 + G4) / 4.

[0148] S302. When the direction of the edge pixels in the green pixels is horizontal, calculate the difference between the pixel value of the first green pixel, the pixel value of the third green pixel, and the first average value.

[0149] S303. Compare the difference between the first green pixel value and the first average value with the difference between the pixel value of the third green pixel and the first average value. If the difference between the first green pixel value and the first average value is greater than the difference between the pixel value of the third green pixel and the first average value, then the first green pixel is a pixel outside the object; otherwise, the third green pixel is a pixel outside the object.

[0150] Specifically, when the direction of the edge pixel in the green pixel is detected to be horizontal, the differences between the pixel value of the first green pixel G1, the pixel value of the third green pixel G3, and the first average value Aver1 are calculated respectively. The differences between the pixel value of the first green pixel G1 and the first average value Aver1 are compared with the differences between the pixel value of the third green pixel G3 and the first average value Aver1. If the difference between the pixel value of the first green pixel G1 and the first average value Aver1 is greater than the difference between the pixel value of the third green pixel G3 and the first average value Aver1, then the first green pixel G1 is a pixel outside the object. If the difference between the pixel value of the third green pixel G3 and the first average value Aver1 is less than the difference between the pixel value of the third green pixel G3 and the first average value Aver1, then the third green pixel G3 is a pixel outside the object.

[0151] S304. When the direction of the edge pixels in the green pixels is vertical, calculate the difference between the pixel value of the second green pixel, the pixel value of the fourth green pixel, and the first average value.

[0152] S305. Compare the difference between the pixel value of the second green pixel and the first average value with the difference between the pixel value of the fourth green pixel and the first average value. If the difference between the pixel value of the second green pixel and the first average value is greater than the difference between the pixel value of the fourth green pixel and the first average value, then the second green pixel is a pixel outside the object; otherwise, the fourth green pixel is a pixel outside the object.

[0153] Specifically, when the direction of the edge pixels in the green pixels is vertical, the differences between the pixel value of the second green pixel G2, the pixel value of the fourth green pixel G4, and the first average value Aver1 are calculated respectively. The differences between the pixel value of the second green pixel G2 and the first average value Aver1 are compared with the differences between the pixel value of the fourth green pixel G4 and the first average value Aver1. If the difference between the pixel value of the second green pixel G2 and the first average value Aver1 is greater than the difference between the pixel value of the fourth green pixel G4 and the first average value Aver1, then the second green pixel G2 is a pixel outside the object. If the difference between the pixel value of the second green pixel G2 and the first average value Aver1 is less than the difference between the pixel value of the fourth green pixel G4 and the first average value Aver1, then the fourth green pixel G4 is a pixel outside the object.

[0154] S306. Calculate the second average value of the pixel values ​​of the blue pixels;

[0155] Specifically, the second average value of the blue pixel's pixel value, Aver2, is (B1 + B2 + B3 + B4) / 4.

[0156] S307. When the edge direction of the blue pixel is the lower left and upper right diagonal direction, calculate the difference between the pixel value of the first blue pixel and the second average value and the difference between the pixel value of the third blue pixel and the second average value respectively.

[0157] S308. Compare the difference between the pixel value of the first blue pixel and the second average value with the difference between the pixel value of the third blue pixel and the second average value. If the difference between the pixel value of the first blue pixel and the second average value is greater than the difference between the pixel value of the third blue pixel and the second average value, then the first blue pixel is a pixel outside the object; otherwise, the third blue pixel is a pixel outside the object.

[0158] Specifically, when the edge direction of the blue pixel is the diagonal direction from the lower left to the upper right, the difference between the pixel value of the first blue pixel B1 and the second average value Aver2, and the difference between the pixel value of the third blue pixel B3 and the second average value are calculated respectively. The magnitudes of the differences between the pixel values ​​of the first blue pixel B1 and the second average value Aver2, and the differences between the pixel values ​​of the third blue pixel B3 and the second average value Aver2 are compared. If the difference between the pixel value of the first blue pixel B1 and the second average value Aver2 is greater than the difference between the pixel value of the third blue pixel B3 and the second average value Aver2, then the first blue pixel B1 is a pixel outside the object. If the difference between the pixel value of the first blue pixel B1 and the second average value Aver2 is less than the difference between the pixel value of the third blue pixel B3 and the second average value Aver2, then the third blue pixel B3 is a pixel outside the object.

[0159] S309. When the edge direction of the blue pixel is the upper left and lower right diagonal direction, calculate the difference between the pixel value of the second blue pixel and the second average value and the difference between the pixel value of the fourth blue pixel.

[0160] S310. Compare the difference between the pixel value of the second blue pixel and the second average value with the difference between the pixel value of the fourth blue pixel and the second average value. If the difference between the pixel value of the second blue pixel and the second average value is greater than the difference between the pixel value of the fourth blue pixel and the second average value, then the second blue pixel is a pixel outside the object; otherwise, the fourth blue pixel is a pixel outside the object.

[0161] Specifically, when the edge direction of the blue pixel is a diagonal direction from the top left to the bottom right, the difference between the pixel value of the second blue pixel B2 and the second average value Aver2 is calculated, and the difference between the pixel value of the fourth blue pixel B4 is calculated. The difference between the pixel value of the second blue pixel B2 and the second average value Aver2 is compared with the difference between the pixel value of the fourth blue pixel B4 and the second average value Aver2. If the difference between the pixel value of the second blue pixel B2 and the second average value Aver2 is greater than the difference between the pixel value of the fourth blue pixel B4 and the second average value Aver2, then the second blue pixel B2 is a pixel outside the object. If the difference between the pixel value of the second blue pixel B2 and the second average value Aver2 is less than the difference between the pixel value of the fourth blue pixel B4 and the second average value Aver2, then the fourth blue pixel B4 is a pixel outside the object.

[0162] Step S300 further includes the following step:

[0163] S311. Calculate the third average value of the pixel values ​​of the red pixels;

[0164] Specifically, the third average value of the red pixel's pixel value, Aver3, is (R1 + R2 + R3 + R4) / 4.

[0165] S312. When the edge direction of the red pixel is the diagonal direction of the lower left and upper right, calculate the difference between the pixel value of the first red pixel and the third average value and the difference between the pixel value of the third red pixel and the third average value respectively.

[0166] S313. Compare the difference between the pixel value of the first red pixel and the third average value with the difference between the pixel value of the third red pixel and the third average value. If the difference between the pixel value of the first red pixel and the third average value is greater than the difference between the pixel value of the third red pixel and the third average value, then the first red pixel is a pixel outside the object; otherwise, the third red pixel is a pixel outside the object.

[0167] S314. When the edge direction of the red pixel is the upper left and lower right diagonal direction, calculate the difference between the pixel value of the second red pixel and the third average value and the difference between the pixel value of the fourth red pixel and the third average value respectively.

[0168] S315. Compare the difference between the pixel value of the second red pixel and the third average value with the difference between the pixel value of the fourth red pixel and the third average value. If the difference between the pixel value of the second red pixel and the third average value is greater than the difference between the pixel value of the fourth red pixel and the third average value, then the second red pixel is a pixel outside the object; otherwise, the fourth red pixel is a pixel outside the object.

[0169] Specifically, when the input 3x3 grid is centered on a blue pixel, and the edge direction of the red pixels is the diagonal direction from the bottom left to the top right, the difference between the pixel value of the first red pixel R1 and the third average value Aver3 is calculated, and the difference between the pixel value of the third red pixel R3 and the third average value Aver3 is compared. If the difference between the pixel value of the first red pixel R1 and the third average value Aver3 is greater than the difference between the pixel value of the third red pixel R3 and the third average value Aver3, then the first red pixel R1 is a pixel outside the object; otherwise, the third red pixel R3 is a pixel outside the object.

[0170] When the edge direction of the red pixel is a diagonal direction from the top left to the bottom right, calculate the difference between the pixel value of the second red pixel R1 and the third average value Aver3, and the difference between the pixel value of the fourth red pixel R4 and the third average value. Compare the difference between the pixel value of the second red pixel R2 and the third average value Aver3 with the difference between the pixel value of the fourth red pixel R4 and the third average value Aver3. If the difference between the pixel value of the second red pixel R2 and the third average value Aver3 is greater than the difference between the pixel value of the fourth red pixel R4 and the third average value Aver3, then the second red pixel R2 is a pixel outside the object; otherwise, the fourth red pixel R4 is a pixel outside the object.

[0171] Step S400 includes the following steps:

[0172] S401. Calculate the green channel pixel value of the center pixel of the nine-grid based on the first average value and the pixel value of the pixels outside the object.

[0173] S402. Calculate the blue channel pixel value of the center pixel of the nine-grid based on the second average value and the pixel value of the pixels outside the object.

[0174] Specifically, when the input 3x3 grid has a red pixel as the center pixel and the edge pixels are detected as green pixels, and the direction of the edge pixels in the green pixels is vertical, after determining the pixels located outside the object's edge, the pixel centered on the red pixel is de-mosaiced using the following formula, taking the first green pixel G1 as an example of a pixel outside the object:

[0175] P_recovered_G=(4×Aver1–P_outside1) / 3;

[0176] Wherein, P_recovered_G is the green channel pixel value recovered by the center pixel, P_outside1 is the pixel value of the first green pixel G1, and Aver1 is the average value of the first green pixel G1, the second green pixel G2, the third green pixel G3, and the fourth green pixel G4.

[0177] The following example illustrates the situation using the following pixel values: first green pixel G1 (120), second green pixel G2 (185), third green pixel G3 (180), and fourth green pixel G4 (175). We can see that the minimum green pixel value (Min1) is 120, the maximum value (Max1) is 185, and the first average value (Aver1) is 165. The difference between the maximum value (Max1) and the minimum value (Min1) (Range1) is greater than Max1 / 4, indicating the presence of edge pixels. Furthermore, the pixel value of the first green pixel G1 is different from that of the third green pixel G4. If the difference in pixel value of green pixel G3 is greater than the difference in pixel value of second green pixel G2 and fourth green pixel G4, then the edge pixels of the green pixels are horizontal. If the difference in pixel value of first green pixel G1 and first average value Aver1 is greater than the difference in pixel value of third green pixel G3 and first average value Aver1, then the first green pixel G1 is a pixel value located outside the object. The green channel pixel value P_recovered_G recovered by the center pixel is (4×165–120) / 3 = 180.

[0178] When a 3x3 grid is input with a red pixel as the center pixel, and blue pixels are detected as edge pixels, and the direction of the edge pixels in the blue pixels is from bottom left to top right, after determining that the pixels are outside the object's edge, the pixel centered on the red pixel is de-mosaiced using the following formula, taking the second blue pixel B2 as an example of a pixel outside the object:

[0179] P_recovered_B=(4×Aver2–P_outside2) / 3;

[0180] Where P_recovered_B is the blue channel pixel value recovered from the center pixel, P_outside2 is the pixel value of the second blue pixel B2, and Aver2 is the average value of the first blue pixel B1, the second blue pixel B2, the third blue pixel B3, and the fourth blue pixel B4.

[0181] It should be noted that if there are no pixels located at the edge in either the green or blue input pixels, then:

[0182] P_recovered_G=Aver1; P_recovered_B=Aver2.

[0183] Step S400 includes the following steps:

[0184] S403. Calculate the green channel pixel value of the center pixel of the nine-grid based on the first average value and the pixel value of the pixels outside the object.

[0185] S404. Calculate the red channel pixel value recovered from the center pixel of the nine-grid based on the third average value and the pixel values ​​of the pixels outside the object.

[0186] Specifically, when the input 3x3 grid has a blue pixel as the center pixel and the edge pixels are detected as green pixels, and the direction of the edge pixels in the green pixels is horizontal, after determining the pixels located outside the object's edge, the pixel centered on the red pixel is de-mosaiced using the following formula, taking the first green pixel G1 as an example of a pixel outside the object:

[0187] P_recovered_G=(4×Aver1–P_outside1) / 3;

[0188] Wherein, P_recovered_G is the green channel pixel value recovered by the center pixel, P_outside1 is the pixel value of the first green pixel G1, and Aver1 is the average value of the first green pixel G1, the second green pixel G2, the third green pixel G3, and the fourth green pixel G4.

[0189] When a 3x3 grid is input with a blue pixel as the center pixel, and red pixels are detected as edge pixels, and the direction of the edge pixels in the red pixels is from bottom left to top right, after determining that the pixels are outside the object's edge, the pixel centered on the blue pixel is de-mosaiced using the following formula, taking the second red pixel B2 as an example of a pixel outside the object:

[0190] P_recovered_R=(4×Aver3–P_outside3) / 3;

[0191] Where P_recovered_R is the blue channel pixel value recovered from the center pixel, P_outside3 is the pixel value of the second red pixel R2, and Aver3 is the average value of the first red pixel R1, the second red pixel R2, the third red pixel R3, and the fourth red pixel R4.

[0192] It should be noted that if there are no pixels located at the edge in either the green or blue input pixels, then:

[0193] P_recovered_G=Aver1; P_recovered_R=Aver3.

[0194] In the above technical solution, the present invention performs edge detection on raw data to distinguish pixels located on the edge from pixels located elsewhere. For non-edge pixels, a traditional linear interpolation method is used for demosaicing. For pixels on the edge, after determining the direction of the edge pixels, the center pixel of the nine-grid is demosaiced to restore the edge pixels. In this way, the jagged edges in the demosaiced image can be eliminated, and the colors of pixels on the edge are prevented from becoming blurred, thereby improving the edge restoration effect of the image and thus improving the quality of the output color image.

[0195] Please see Figure 7 In some embodiments, the present invention also provides a demosaic circuit for edge detection based on raw data, used to implement the demosaic method for edge detection based on raw data as described above, including: a raw data 3x3 grid construction module 1, a calculation module 2, an edge pixel recovery module 3, and a demosaic pixel value output module 4. The raw data 3x3 grid construction module 1 is used to construct a 3x3 raw data 3x3 grid required for each calculation; the calculation module 2 is used to calculate the maximum, minimum, and average values ​​of the input pixel data; the edge pixel recovery module 3 is used to perform pixel edge detection based on the maximum, minimum, and average values ​​of the input pixels, and to perform demosaic processing on the center pixel of the 3x3 grid based on the detection results; the demosaic pixel value output module 4 is used to output the pixel value after demosaicing the center pixel.

[0196] In some embodiments, the raw data grid construction module 1 includes: a row buffer and a register group, wherein the row buffer and the register group are connected.

[0197] Specifically, the raw data 3x3 grid construction module 1 includes a row buffer for storing the pixels (line0, line1) of the two rows of input images and a register group consisting of 6 registers, each register storing one pixel, which together with the 3 pixels of the external input (Data_in) constitute the raw data 3x3 grid required for each calculation.

[0198] In some embodiments, when the input pixel grid is centered on a red pixel, the calculation module 2 includes: a first calculation unit, a second calculation unit, a third calculation unit, a fourth calculation unit, and a first average value calculation unit. The first calculation unit is connected to the register group and is used to calculate the maximum value of the green pixel; the second calculation unit is connected to the register group and is used to calculate the maximum value of the blue pixel; the third calculation unit is connected to the register group and is used to calculate the maximum value of the green pixel; the fourth calculation unit is connected to the register group and is used to calculate the minimum value of the blue pixel; the first average value calculation unit is connected to the register group and is used to calculate a first average value of the green pixel values ​​and a second average value of the blue pixel values.

[0199] In some embodiments, when the input pixel grid is centered on a blue pixel, the calculation unit 2 further includes a fifth calculation unit, a sixth calculation unit, and a second average value calculation unit. The fifth calculation unit is connected to the register group and is used to calculate the maximum value of the red pixel; the sixth calculation unit is connected to the register group and is used to calculate the minimum value of the red pixel; the second average value calculation unit is connected to the register group and is used to calculate the third average value of the red pixel.

[0200] In some embodiments, the edge pixel recovery module 3 includes: a first edge pixel recovery unit, a second edge pixel recovery unit, a third edge pixel recovery unit, and a fourth edge pixel recovery unit. When the input pixel 3x3 grid has red pixels as the center pixels and green pixels as the edge pixels, the first edge pixel recovery unit is connected to the first calculation unit, the third calculation unit, and the first average value calculation unit, respectively, and is used to perform pixel edge detection based on the maximum value of the green pixels, the minimum value of the green pixels, and the first average value of the green pixels, and to perform de-mosaic processing on the red center pixels of the 3x3 grid based on the detection results. When the input pixel 3x3 grid has red pixels as the center pixels and blue pixels as the edge pixels, the second edge pixel recovery unit is connected to the second calculation unit, the fourth calculation unit, and the first average value calculation unit, respectively, and is used to perform pixel edge detection based on the maximum value of the blue pixels, the minimum value of the blue pixels, and the second average value of the blue pixels, and to perform de-mosaic processing on the red center pixels of the 3x3 grid based on the detection results.

[0201] When the input pixel 3x3 grid has a blue pixel as the center and green pixels as the edge pixels, the third edge pixel recovery unit is connected to the first calculation unit, the third calculation unit, and the first average value calculation unit, respectively. It performs pixel edge detection based on the maximum value, minimum value, and first average value of the green pixel values, and then performs de-mosaic processing on the blue center pixel of the 3x3 grid based on the detection results. When the input pixel 3x3 grid has a blue pixel as the center and red pixels as the edge pixels, the fourth edge pixel recovery unit is connected to the fifth calculation unit, the sixth calculation unit, and the second average value calculation unit, respectively. It performs pixel edge detection based on the maximum value, minimum value, and third average value of the red pixel values, and then performs de-mosaic processing on the blue center pixel of the 3x3 grid based on the detection results.

[0202] Please see Figure 8 and Figure 9 , Figure 8 This is a framework diagram of a demosaic circuit based on raw data, with the red pixel as the center pixel in a nine-pixel input pixel grid.

[0203] The calculation module 2 includes a first calculation unit 21 and a second calculation unit 22. The first calculation unit 21 and the second calculation unit 22 have the same structure, consisting of several comparators and data selectors (MUX). They are used to select the maximum value from four input data and output it, and to select the larger value based on the results of pairwise comparisons of previous data for the next round of comparison. The calculation module 2 also includes a third calculation unit 23 and a fourth calculation unit 24. The third calculation unit 23 and the fourth calculation unit 24 have the same structure, consisting of several comparators and data selectors (MUX). They are used to select the minimum value from four data and output it, and to select the smaller value based on the results of pairwise comparisons of previous data for the next round of comparison. The calculation module 2 also includes a first average value calculation unit 25, which consists of several dividers (DIV2) and adders (ADD). The first average value calculation unit 25 can calculate the average of the four input values, specifically the average pixel value Aver1 for green pixels and the average pixel value Aver2 for blue pixels. The output data of the calculation module 2 is stored in registers.

[0204] The edge pixel recovery module 3 includes a first edge pixel recovery unit 31 and a second edge pixel recovery unit 32. The first edge pixel recovery unit 31 and the second edge pixel recovery unit 32 have the same structure, each consisting of two multipliers MUL4, several absolute value calculators SUB, dividers (DIV3, DIV4), a comparator, and a data selector MUX. Specifically, the multiplier MUL4 multiplies the input data by 4, the absolute value calculator SUB subtracts two data and outputs the absolute value of the subtraction result, the divider DIV3 divides the data by 3, and the divider DIV4 divides the data by 4. The comparator compares the data size and outputs 0 or 1 based on the comparison result. The data selector MUX selects the de-mosaiced edge pixels to be output based on the input data, i.e., it is used to perform edge detection and de-mosaic on the center pixels of the 3x3 grid based on the detection results.

[0205] The demosaic pixel value output module 4 includes a data selector MUX, which is used to output the pixel value of the center pixel of the nine-grid after demosaicing based on the input pixel coordinates and other values.

[0206] It should be noted that the principle of the demosaic circuit based on raw data for edge detection with the red pixel as the center of the input pixel nine-grid is the same as that of the demosaic circuit based on raw data for edge detection with the blue pixel as the center of the input pixel nine-grid, and will not be repeated here.

[0207] In the above technical solution, the present invention only requires a 3x3 register grid centered on the pixel to be restored when identifying edges. This can be achieved in hardware with only two rows of cache and a few individual registers. While maintaining a good de-mosaic effect, it can reduce the demand for hardware resources and power consumption, thereby reducing costs.

[0208] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A demosaic method based on raw data edge detection, characterized in that, include: Edge detection is performed on the raw data to determine whether there are pixels located on the edges; wherein, the raw data includes green pixels, blue pixels and red pixels, and the raw data is distributed in a nine-square grid; If there are pixels on the edge in the raw data, then determine the edge direction of the detected edge pixels in the raw data; Perform demosaic processing on edge pixels and non-edge pixels separately; The steps of performing demosaic processing on edge pixels and non-edge pixels separately include: Calculate the first average of the pixel values ​​of the green pixels; When the direction of the edge pixels in the green pixel is horizontal, calculate the difference between the pixel value of the first green pixel, the pixel value of the third green pixel, and the first average value. When the direction of the edge pixels in the green pixel is vertical, calculate the difference between the pixel value of the second green pixel, the pixel value of the fourth green pixel, and the first average value. Calculate the second average of the pixel values ​​of the blue pixels; When the edge direction of the blue pixel is the diagonal direction from the lower left to the upper right, calculate the difference between the pixel value of the first blue pixel and the second average value, and the difference between the pixel value of the third blue pixel and the second average value, respectively. When the edge direction of the blue pixel is the upper left and lower right diagonal direction, calculate the difference between the pixel value of the second blue pixel and the second average value and the difference between the pixel value of the fourth blue pixel and the second average value respectively. Calculate the third average of the pixel values ​​of the red pixels; When the edge direction of the red pixel is the diagonal direction from the lower left to the upper right, calculate the difference between the pixel value of the first red pixel and the third average value and the difference between the pixel value of the third red pixel. When the edge direction of the red pixel is the upper left and lower right diagonal direction, calculate the difference between the pixel value of the second red pixel and the third average value, and the difference between the pixel value of the fourth red pixel and the third average value. The green channel pixel value of the center pixel of the nine-grid is calculated based on the first average value and the pixel value of the pixels outside the object. The blue channel pixel value of the center pixel of the 3x3 grid is calculated based on the second average value and the pixel value of the pixels outside the object. The step of performing demosaic processing on edge pixels and non-edge pixels separately further includes: The green channel pixel value of the center pixel of the nine-grid is calculated based on the first average value and the pixel value of the pixels outside the object. The red channel pixel value of the center pixel of the 3x3 grid is calculated based on the third average value and the pixel values ​​of pixels outside the object.

2. The demosaic method based on raw data edge detection according to claim 1, characterized in that, The step of performing edge detection on the raw data to determine whether there are pixels located on the edge includes: When the center pixel of the 3x3 grid is a red pixel, obtain the maximum and minimum pixel values ​​of the green pixels, and obtain the maximum and minimum pixel values ​​of the blue pixels; The green pixels include a first green pixel, a second green pixel, a third green pixel, and a fourth green pixel, wherein the first green pixel and the third green pixel are distributed opposite to each other, and the second green pixel and the fourth green pixel are distributed opposite to each other; the blue pixels include a first blue pixel, a second blue pixel, a third blue pixel, and a fourth blue pixel, wherein the first blue pixel and the second blue pixel are distributed opposite to each other on both sides of the first green pixel, and the third blue pixel and the fourth blue pixel are distributed opposite to each other on both sides of the third green pixel; Calculate the difference between the maximum and minimum pixel values ​​of the green pixels, and calculate the difference between the maximum and minimum pixel values ​​of the blue pixels; If the difference between the maximum and minimum values ​​of the green pixel is greater than one-quarter of the maximum value of the green pixel, then it is determined that there is a pixel located on the edge among the green pixels. If the difference between the maximum and minimum values ​​of the blue pixel is greater than one-quarter of the maximum value of the blue pixel, then it is determined that there is a pixel located on the edge among the blue pixels.

3. The demosaic method based on raw data for edge detection according to claim 2, characterized in that, The step of performing edge detection on the raw data to determine whether there are pixels located on the edge further includes: When the center pixel of the 3x3 grid is a blue pixel, obtain the maximum and minimum pixel values ​​of the green pixels, and obtain the maximum and minimum pixel values ​​of the red pixels; The red pixel includes a first red pixel, a second red pixel, a third red pixel, and a fourth red pixel. The first red pixel and the second red pixel are distributed opposite each other on both sides of the first green pixel, and the third red pixel and the fourth red pixel are distributed opposite each other on both sides of the third green pixel. Calculate the difference between the maximum and minimum pixel values ​​of the green pixels, and calculate the difference between the maximum and minimum pixel values ​​of the red pixels; If the difference between the maximum and minimum values ​​of the green pixel is greater than one-quarter of the maximum value of the green pixel, then it is determined that there is a pixel located on the edge among the green pixels. If the difference between the maximum and minimum values ​​of the red pixel is greater than one-quarter of the maximum value of the red pixel, then it is determined that there is a pixel located on the edge among the red pixels.

4. The demosaic method based on raw data edge detection according to claim 2, characterized in that, If the raw data contains pixels located on edges, the step of determining the edge direction of the detected edge pixels in the raw data includes: Compare the absolute value of the difference between the pixel value of the first green pixel and the pixel value of the third green pixel with the absolute value of the difference between the pixel value of the second green pixel and the pixel value of the fourth green pixel; If the absolute value of the difference between the pixel value of the first green pixel and the pixel value of the third green pixel is greater than the absolute value of the difference between the pixel value of the second green pixel and the pixel value of the fourth green pixel, then the direction of the edge pixels in the green pixel is determined to be horizontal; otherwise, the direction of the edge pixels in the green pixel is determined to be vertical. Compare the absolute value of the difference between the pixel value of the first blue pixel and the pixel value of the third blue pixel with the absolute value of the difference between the pixel value of the second blue pixel and the pixel value of the fourth blue pixel; If the absolute value of the difference between the pixel value of the first blue pixel and the pixel value of the third blue pixel is greater than the absolute value of the difference between the pixel value of the second blue pixel and the pixel value of the fourth blue pixel, then the edge direction of the blue pixel is the diagonal direction of the lower left and upper right; otherwise, the edge direction of the blue pixel is the diagonal direction of the upper left and lower right.

5. The demosaic method based on raw data for edge detection according to claim 3, characterized in that, If the raw data contains pixels located on edges, the step of determining the edge direction of the detected edge pixels in the raw data includes: Compare the absolute value of the difference between the pixel value of the first green pixel and the pixel value of the third green pixel with the absolute value of the difference between the pixel value of the second green pixel and the pixel value of the fourth green pixel; If the absolute value of the difference between the pixel value of the first green pixel and the pixel value of the third green pixel is greater than the absolute value of the difference between the pixel value of the second green pixel and the pixel value of the fourth green pixel, then the direction of the edge pixels in the green pixel is determined to be horizontal; otherwise, the direction of the edge pixels in the green pixel is determined to be vertical. Compare the absolute value of the difference between the pixel value of the first red pixel and the pixel value of the third red pixel with the absolute value of the difference between the pixel value of the second red pixel and the pixel value of the fourth red pixel; If the absolute value of the difference between the pixel value of the first red pixel and the pixel value of the third red pixel is greater than the absolute value of the difference between the pixel value of the second red pixel and the pixel value of the fourth red pixel, then the edge direction of the red pixel is the diagonal direction of the lower left and upper right; otherwise, the edge direction of the red pixel is the diagonal direction of the upper left and lower right.

6. The demosaic method based on raw data for edge detection according to claim 4, characterized in that, The steps of performing demosaic processing on edge pixels and non-edge pixels separately include: Compare the difference between the pixel value of the first green pixel and the first average value with the difference between the pixel value of the third green pixel and the first average value. If the difference between the pixel value of the first green pixel and the first average value is greater than the difference between the pixel value of the third green pixel and the first average value, then the first green pixel is a pixel outside the object; otherwise, the third green pixel is a pixel outside the object. Compare the difference between the pixel value of the second green pixel and the first average value with the difference between the pixel value of the fourth green pixel and the first average value. If the difference between the pixel value of the second green pixel and the first average value is greater than the difference between the pixel value of the fourth green pixel and the first average value, then the second green pixel is a pixel outside the object; otherwise, the fourth green pixel is a pixel outside the object.

7. The demosaic method based on raw data for edge detection according to claim 6, characterized in that, The steps of performing demosaic processing on edge pixels and non-edge pixels separately include: Compare the difference between the pixel value of the first blue pixel and the second average value with the difference between the pixel value of the third blue pixel and the second average value. If the difference between the pixel value of the first blue pixel and the second average value is greater than the difference between the pixel value of the third blue pixel and the second average value, then the first blue pixel is a pixel outside the object; otherwise, the third blue pixel is a pixel outside the object. The difference between the pixel value of the second blue pixel and the second average value is compared with the difference between the pixel value of the fourth blue pixel and the second average value. If the difference between the pixel value of the second blue pixel and the second average value is greater than the difference between the pixel value of the fourth blue pixel and the second average value, then the first blue pixel is a pixel outside the object; otherwise, the third blue pixel is a pixel outside the object.

8. The demosaic method based on raw data edge detection according to claim 6, characterized in that, The steps of performing demosaic processing on edge pixels and non-edge pixels separately include: Compare the difference between the first red pixel value and the third average value with the difference between the pixel value of the third red pixel and the third average value. If the difference between the pixel value of the first red pixel and the third average value is greater than the difference between the pixel value of the third red pixel and the third average value, then the first red pixel is a pixel outside the object; otherwise, the third red pixel is a pixel outside the object. Compare the difference between the pixel value of the second red pixel and the third average value with the difference between the pixel value of the fourth red pixel and the third average value. If the difference between the pixel value of the second red pixel and the third average value is greater than the difference between the pixel value of the fourth red pixel and the third average value, then the second red pixel is a pixel outside the object; otherwise, the fourth red pixel is a pixel outside the object.

9. A demosaic circuit for edge detection based on raw data, used to implement the demosaic method for edge detection based on raw data as described in any one of claims 1-8, characterized in that, include: The raw data nine-square grid composition module is used to construct the 3x3 raw data nine-square grid required for each calculation. The calculation module, connected to the raw data nine-square grid module, is used to calculate the maximum, minimum and average values ​​of the input pixel data; An edge pixel recovery module, connected to the calculation module, is used to perform pixel edge detection based on the maximum, minimum and average values ​​of the input pixel count, and to perform de-mosaic processing on the center pixels of the nine-grid based on the detection results. The demosaic pixel value output module is connected to the edge pixel recovery module and is used to output the pixel value of the center pixel of the nine-grid after demosaicing.

10. The demosaic circuit for edge detection based on raw data according to claim 9, characterized in that, The raw data 3x3 grid module includes: a row buffer and a register group, wherein the row buffer and the register group are connected.

11. The demosaic circuit for edge detection based on raw data according to claim 10, characterized in that, The calculation module includes: a first calculation unit, a second calculation unit, a third calculation unit, a fourth calculation unit, and a first average value calculation unit; The first calculation unit is connected to the register group and is used to calculate the maximum value of the green pixel; The second calculation unit is connected to the register group and is used to calculate the maximum value of the blue pixel; The third calculation unit is connected to the register group and is used to calculate the minimum pixel value of the green pixel; The fourth calculation unit is connected to the register group and is used to calculate the minimum pixel value of the blue pixel; The first average value calculation unit is connected to the register group and is used to calculate the first average value of the pixel values ​​of the green pixels and the second average value of the pixel values ​​of the blue pixels.

12. The demosaic circuit for edge detection based on raw data according to claim 10, characterized in that, The calculation unit further includes: a fifth calculation unit, a sixth calculation unit, and a second average value calculation unit; The fifth calculation unit is connected to the register group and is used to calculate the maximum value of the red pixel. The sixth calculation unit is connected to the register group and is used to calculate the minimum pixel value of the red pixel; The second average value calculation unit is connected to the register group and is used to calculate the third average value of the pixel value of the red pixel.

13. The demosaic circuit for edge detection based on raw data according to claim 9, characterized in that, The edge pixel recovery module includes: a first edge pixel recovery unit, a second edge pixel recovery unit, a third edge pixel recovery unit, and a fourth edge pixel recovery unit; The first edge pixel recovery unit is connected to the first calculation unit, the third calculation unit and the first average value calculation unit respectively, and is used to perform pixel edge detection based on the maximum value of the green pixel, the minimum value of the green pixel and the first average value of the green pixel, and to perform de-mosaic processing on the red center pixel of the nine-square grid based on the detection result. The second edge pixel recovery unit is connected to the second calculation unit, the fourth calculation unit and the first average value calculation unit respectively, and is used to perform pixel edge detection based on the maximum value of the blue pixel, the minimum value of the blue pixel and the second average value of the blue pixel, and to perform de-mosaic processing on the red center pixel of the nine-square grid based on the detection result. The third edge pixel recovery unit is connected to the first calculation unit, the third calculation unit and the first average value calculation unit respectively, and is used to perform pixel edge detection based on the maximum value of the green pixel, the minimum value of the green pixel and the first average value of the green pixel, and to perform de-mosaic processing on the blue center pixel of the nine-grid according to the detection result. The fourth edge pixel recovery unit is connected to the fifth calculation unit, the sixth calculation unit and the second average value calculation unit, respectively, and is used to perform pixel edge detection based on the maximum value of the red pixel, the minimum value of the red pixel and the third average value of the red pixel, and to perform de-mosaic processing on the blue center pixel of the nine-grid according to the detection results.

Images