An interpolation method for image color restoration based on FPGA

By using sliding matrix and gradient edge calculation methods in FPGA, a new color reduction formula was established, which solved the problem of color distortion in the human abdominal environment in the prior art, and achieved clearer image edges and details Reserved, suitable for endoscopic systems.

CN114155166BActive Publication Date: 2024-11-05QINGDAO NOVELBEAM TECH
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202111474556.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-06
Publication Date
2024-11-05
Estimated Expiration
2041-12-06

AI Technical Summary

Technical Problem

The existing edge-oriented adaptive interpolation method is prone to severe color distortion when interpolation of red and blue colors in specific scenarios, especially in the human abdominal environment, and is not suitable for use in endoscopic systems and cannot be effective. Preserve and enhance image details.

Method used

Using the image color reduction method based on FPGA, by constructing a sliding matrix sequence and gradient edge calculation formula, a new color reduction formula is established based on the gradient size and direction of the non-reconstructed color components of the surrounding pixels of the reconstruction pixel, and the traditional edge-oriented interpolation method is abandoned. Implement interpolation reduction of green, red and blue components.

Benefits of technology

It effectively eliminates the problem of pseudo-color at the edge of the image in traditional methods, and the image edges are clearer and sharper, retaining and enhancing detailed information. It is suitable for endoscopic systems, achieving richer image detail restoration, and real-time pipeline processing is realized in FPGA.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114155166B_ABST
    Figure CN114155166B_ABST
Patent Text Reader

Abstract

The present invention relates to an interpolation method for image color restoration based on FPGA. After obtaining edge information of an original BAYER image based on an adaptive interpolation method, a new color restoration formula is established according to an actual working environment and the color correlation of each channel. In the process of missing color reconstruction, the missing color component in the BAYER image is restored according to the gradient size and direction of non-reconstructed color components of pixels around the reconstructed pixel and the gradient relationship of the non-reconstructed color components at the reconstructed pixel. The restored color component is made close to the side with a small gradient change along the edge direction and more different from the side with a large gradient change along the edge direction, so that the details contained in other colors are retained and enhanced. The color restored by interpolation according to the new color restoration formula eliminates the problem of false color at the edge of the image in the traditional method at the place where the color changes violently, and the image edge is clearer and sharper.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to an interpolation method for image color restoration based on FPGA, belonging to the technical field of image processing. Background Art

[0002] At present, the data format output by color COMS image sensors is generally in BAYER format. Compared with the RGB data format, the BAYER format image will lack two colors at each pixel position. In order to obtain a full-color image, an interpolation method is generally used to restore the other two colors missing at each pixel position. The interpolation restoration methods currently used in the industry are mainly linear interpolation methods and adaptive interpolation methods guided by image edge information. Linear interpolation is to restore the missing color by selecting the mean of the surrounding same colors according to the color arrangement of the BAYER image. In 1997, Adams and Hamilton discovered that the number of green pixels is twice the number of red and blue pixels, which will contain more edge information of the original image, and invented the adaptive interpolation method.

[0003] The basic principle of the adaptive interpolation method is to first design the edge detection operator that best suits the product application scenario and detects the data gradient in the RGB color space, and then interpolate the green component in the correct direction according to the calculation result of the edge detection operator. The reconstruction of the red and blue components is based on the edge information of the reconstructed green component, and then the corresponding algorithm is used to complete the interpolation reconstruction. The adaptive interpolation algorithm effectively solves the problem that the edge of the image obtained by the linear interpolation algorithm is blurred or jagged after normal imaging or scaling.

[0004] However, edge-oriented interpolation algorithms have problems in certain scenarios and in the human abdominal cavity. For example, when interpolating the color at the edge, especially when interpolating the missing red and blue, it is easy to cause serious color distortion, and the green component inside the abdominal cavity is very small compared to the red component. Many details inside the abdominal cavity are more present in the red component, which makes the adaptive interpolation method unsuitable for application in endoscope systems. Summary of the invention

[0005] In view of the above deficiencies of the prior art, the present invention proposes an interpolation method for image color restoration based on FPGA. After obtaining the edge information of the original BAYER image based on the adaptive interpolation method, this method abandons the traditional methods of various edge-directed interpolation algorithms for color interpolation at the edges, but establishes a new color restoration formula according to the actual working environment and the color correlation of each channel. During the reconstruction of the missing colors, the missing color components in the BAYER image are restored according to the gradient magnitude and direction of the non-reconstructed color components of the surrounding pixels of the reconstructed pixel and the gradient relationship of the non-reconstructed color components at the reconstructed pixel.

[0006] To achieve the above object, the present invention provides an interpolation method for BAYER data color restoration based on FPGA. For the convenience of description, the size of the BAYER image to be processed is defined as MXN, and the data bit width is Z. It is characterized by including the following steps:

[0007] Step S1: Construct a sliding matrix sequence for restoring the green component G of the BAYER image. Specifically, a matrix of (2k + 1) X (2k + 1) is used, where k is a positive integer, 1 < k < M and 1 < k < N. The parity of the image row and column is used as the basis for distinguishing the color at the restoration position. When the original BAYER color component at the position (i, j) of the matrix is the green component G i,j no restoration operation is performed. When restoring the green component, the matrix data reading format can be divided into the following two cases:

[0008] ① When the original BAYER color component at the position (i, j) of the matrix is R i,j (blue component), restore the missing green component G at this position i,j , and its matrix data reading format is:

[0009] data(2k + 1)… datak… data1 L1 R(i - k, j - k) R(i - k, j) R(i - k, j + k) … … … … Lk R(i, j - k) R(i, j) R(i, j + k) … … … … L(2k + 1) R(i + k, j - k) R(i + k, j) R(i + k, j + k)

[0010] ② When the original BAYER color component at the position (i, j) of the matrix is B i,j (blue component), restore the missing green component G at this position i,j , and its matrix data reading format is:

[0011] data(2k + 1)… datak… data1 L1 B(i - k, j - k) B(i - k, j) B(i - k, j + k) … … … … Lk B(i, j - k) B(i, j) B(i, j + k) … … … … L(2k + 1) B(i + k, j - k) B(i + k, j) B(i + k, j + k) .

[0012] Step S2: Calculate the gradient magnitude in the horizontal or vertical direction at the position where the green component needs to be restored. The gradient edge calculation formula at this position is:

[0013]

[0014] where m = {1, 2, 3…, k}, m is a positive integer and 2m - 1 ≤ k, is the influence factor of other surrounding color components on the reconstructed green component, f(x) is the pixel data at (i,j), when calculating the horizontal edge gradient, f(x±2m) and f(x±2m±1) represent the pixel data at (i,j) with a left-right offset of 2m or (2m±1) coordinates, when calculating the vertical edge gradient, f(x) is the pixel data at (i,j), f(x±2m) and f(x±2m±1) represent the pixel data at (i,j) with a left-right offset of 2m or (2m±1) coordinates.

[0015] Step S3: select the interpolation direction guided by the calculation result of the gradient edge calculation formula, and determine the green component interpolation restoration rule according to the gradient size and direction of the non-green components of the surrounding pixels of the reconstructed pixel and the gradient relationship of the non-green components at the reconstructed pixel. Specifically, according to the direction of the gradient edge at the (i, j) coordinate obtained by the calculation of the gradient edge calculation formula, the green component is interpolated and restored along the edge direction. G i,j Represents the restored green component. When reconstructing the green component in the horizontal direction, f(x) is the pixel data at (i, j), and f(x±2m) and f(x±2m±1) represent the pixel data at the coordinates of (i, j) offset by 2m to the left or right or (2m±1). When reconstructing the green component in the vertical direction, f(x) is the pixel data at (i, j), and f(x±2m) and f(x±2m±1) represent the pixel data at the coordinates of (i, j) offset by 2m to the top or (2m±1). The interpolation restoration rule follows the following calculation formula:

[0016] ① When and or and hour,

[0017]

[0018] ② When and or and hour,

[0019]

[0020] Step S4: Fuse the green component obtained in Step 3 with the original BAYER image, and the image data bit width becomes 3Z. Based on the fused image data, construct a sliding matrix required to restore the red component and the blue component. Specifically, construct a (2h + 1)×(2h + 1) matrix from the fused image data, where h is a positive integer, 1 < k < M and 1 < k < N. The matrix data reading format can be divided into the following four cases:

[0021] ① When there is only a green component at the position (i, j) of the matrix, and the other color components in the vertical direction are red components, and the other color components in the horizontal direction are blue components, the matrix data reading format is:

[0022]

[0023]

[0024] ② When there is only a green component at the position (i, j) of the matrix, and the other color components in the vertical direction are blue components, and the other color components in the horizontal direction are red components, the matrix data reading format is:

[0025] data(2h + 1)… datah… data1 L1 G(i - h, j - h) GB(i - h, j) G(i - h, j + h) … … … … Lh RG(i, j - h) G(i, j) RG(i, j + h) … … … … L(2h + 1) G(i + h, j - h) GB(i + h, j) G(i + h, j + h)

[0026] ③ When there are green and blue components at the position (i, j) of the matrix, the matrix data reading format is:

[0027] data(2h + 1)… datah… data1 L1 RG(i - h, j - h) G(i - h, j) RG(i - h, j + h) … … … … Lh G(i, j - h) GB(i, j) G(i, j + h) … … … … L(2h + 1) RG(i + h, j - h) G(i + h, j) RG(i + h, j + h)

[0028] ④ When there are green and red components at the position (i, j) of the matrix, the matrix data reading format is:

[0029] data(2h + 1)… datah… data1 L1 GB(i - h, j - h) G(i - h, j) GB(i - h, j + h) … … … … Lh G(i, j - h) RG(i, j) G(i, j + h) … … … … L(2h + 1) GB(i + h, j - h) G(i + h, j) GB(i + h, j + h) .

[0030] Step S5: Restore the red component R i,j and the blue component B i,j at the positions of only the G i,j green component of the original BAYER image. Specifically, the matrix reading format is as shown in the first and second matrix reading formats in Step S4, and the interpolation restoration rule follows the following calculation formula:

[0031] (1) When restoring the red component at the position (i, j) of the matrix, where R i,j represents the reconstructed red component, f(x) is the red component at (i, j), g(x) is the green component at (i, j), and the reconstruction formula is as follows:

[0032] ① When and or p = 1hgx + 2p - 1 < p = 1hgx - 2p + 1 and when

[0033]

[0034] ② When and or p = 1hgx + 2p - 1 < p = 1hgx - 2p + 1 and when

[0035]

[0036] (2) When restoring the blue component at the matrix (i, j) position, where B i,j represents the reconstructed red component, f(x) is the blue component at (i, j), g(x) is the green component at (i, j), and the reconstruction formula is as follows:

[0037] ③ When and or and when

[0038]

[0039] ④ When and or p = 1hgx + 2p - 1 < p = 1hgx - 2p + 1 and when

[0040]

[0041] where p = {1, 2, 3…, h}, p is a positive integer and 2p - 1 ≤ h, is the influence factor of the surrounding green component on the reconstructed color component. When restoring the color in the horizontal direction, f(x) is the color component at (i, j) that is the same as the restored color. f(x±2p) and f(x±2p±1) represent the color component at (i, j) that is offset 2p to the left or right or (2p±1) coordinates that is the same as the restored color. When restoring the color in the vertical direction, f(x) is the pixel data at (i, j). f(x±2p) and f(x±2p±1) represent the color component at (i, j) that is offset 2p up or down or (2p±1) coordinates that is the same as the restored color. When restoring the color in the horizontal direction, g(x) is the green component at (i, j). g(x±2p) and g(x±2p±1) represent the color component at (i, j) that is offset 2p to the left or right or (2p±1) coordinates that is the same as the restored color. When restoring the color along the vertical direction, g(x) is the green component at (i, j), g(x±2p) and g(x±2p±1) represent the green component at the coordinates (i, j) offset 2p or (2p±1) above or below.

[0042] Step S6: Only R i,j The red component position restores the blue component B i,j Or only B i,j The blue component position restores the red component B i,j , it is necessary to calculate the gradient magnitude in the 45° and 135° directions of the position. Specifically, the matrix readout format is as shown in the third and fourth matrix readout formats in step S4. The gradient magnitude calculation rule follows the following calculation formula:

[0043]

[0044]

[0045] Where h>m>0, m is a positive integer, Δf 45° (i, j) and Δf 45° (i, j) are the edge gradient information at 45° and 135° respectively. and They are the influence factors of other color components in the 45° direction and 135° surrounding on the reconstructed color components, f(i, j) is the color component to be restored at (i, j), f(i+2m-1, j-2m+1) and f(i-2m+1, j+2m-1) represent the color components that are the same as the restored color components at the (i, j) coordinates offset by (2m±1) in the 45° direction. f(i+2m-1, j-2m+1) and f(i-2m+1, j+2m-1) represent the color components that are the same as the restored color components at the (i, j) coordinates offset by (2m±1) in the 135° direction, g(i, j) is the green component at (i, j), g(i+2m-1, j-2m+1) and g(i-2m+1, j+2m-1) represent the green components at the (i, j) coordinates offset by 2m or (2m±1) in the 45° direction. g(i+2m-1, j-2m+1) and g(i-2m+1,j+2m-1) represent the green component at the coordinate (i,j) offset by 135° in the direction of (2m±1).

[0046] Step S7: According to the 45° and 135° directional gradient results, determine the direction of the image gradient at the restored position, and perform color reconstruction according to the direction of the edge. i,j Represents the last two missing colors in the image. According to the color category that needs to be reconstructed, the corresponding color component can be calculated by substituting the color component of the position in the formula. The reconstruction formula is as follows:

[0047] ① When color reconstruction is performed along the 45° direction, and |(h-1)2*gi,jm=1hgi+2m-1,j+2m-1|>=|(h-1)2*fx-m=1hgi-2m+1,j-2m+1| or and hour,

[0048]

[0049] ② When color reconstruction is performed along the 45° direction, and |(h-1)2*gi,jm=1hgi+2m-1,j+2m-1|<|(h-1)2*fx-m=1hgi-2m+1,j-2m+1| or and hour,

[0050]

[0051] ③ When color reconstruction along the 135° direction, and |(h-1)2*gi,jm=1hgi+2m-1,j-2m+1|>=|(h-1)2*fx-m=1hgi-2m+1,j+2m-1| or when and hour,

[0052]

[0053] ④ When color reconstruction is performed along the 135° direction, and |(h-1)2*gi,jm=1hgi+2m-1,j-2m+1|<|(h-1)2*fx-m=1hgi-2m+1,j+2m-1| or when and hour,

[0054]

[0055] Step S8: Perform data threshold constraints on the interpolated color components. Specifically, when the value of one or more color components among the green component, red component, and blue component of RGB is greater than the threshold 2 Z When -1, the value is 2 Z -1, below threshold 2 Z When -1, the original value is taken, and then data fusion is performed to output RGB image data.

[0056] The present invention makes the restored color component close to the side with a smaller gradient change along the edge direction, and more different from the side with a larger gradient change along the edge direction, so that the details contained in other colors are retained and enhanced. The color restored by interpolation according to the new color restoration formula eliminates the problem of false color appearing at the edge of the image in the traditional method where the color changes dramatically. At the same time, the edge of the image is clearer and sharper. It can restore richer details when applied to the endoscope camera system, and can realize real-time pipeline processing in FPGA. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The present invention is a flowchart of an interpolation method for color restoration of a BAYER image based on FPGA. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present invention more clearly explained, the present invention is further described in detail below in conjunction with specific embodiments.

[0059] In the present invention, the BAYER data collected by the medical electronic endoscope and the high-definition CMOS image sensor chip will be used. The data is transmitted in the form of 12-bit bit width, 1920X1080 high-definition image, and the original BAYER data arranged in RGGB sequence in the form of HDMI external synchronization. The data output by the front-end sensor is sent to the FPGA chip for image processing via the SERDES transmission protocol to start image interpolation and restoration processing. This implementation example has exactly the same steps as the content of the invention. To avoid repetition, only the key data are listed:

[0060] In the image size MXN, M is 1920, N is 1080, and Z is 12. Specifically, when reconstructing the green component, the original BAYER data needs to go through step S1, step S2, and step S3, the k value is 2, and the m value is 1, then the matrix size of (2k+1)X(2k+1) is selected as a 5X5 matrix, and the values ​​of k and m are brought into the edge detection operator formula and according to the edge calculation results, the pixel coordinates along the edge direction in the matrix are selected for interpolation to finally restore all the missing green components in the image. In the process of reconstructing and restoring the green component, the method of the present invention will judge the high and low directions of other colors in the pixel coordinates along the edge direction and the gradient relationship with the color corresponding to the reconstructed pixel, and then use a special color restoration formula to make the reconstructed green component close to the side with small gradient change and more different from the side with large gradient change, so that the details contained in other colors are retained and enhanced. This makes the green component of the reconstruction less jagged at the edges than bilinear interpolation, and at the same time, compared with the ordinary adaptive interpolation method, the reconstructed green component avoids large edges while presenting richer and clearer details along the edge direction.

[0061] In step S4, the green component is obtained and image fused with the original BAYER image. At this time, the image data bit width to be restored becomes 36 bits, and the unreconstructed color component is replaced by 0. In step S4 and step S5, h takes the value of 1, p takes the value of 1, and m takes the value of 1. The fused data is used to construct a 3X3 matrix processing window. The red color component and the blue color component are located in different positions in the surrounding coordinates at the color restoration point, and the gradient change direction and size of the green component color in the pixels in the surrounding coordinates used for restoration and the green component color at the reconstructed pixel are used. The red component and the blue component of the green component sampling point in the original BAYER are calculated and restored using their respective color restoration formulas to obtain the red component and the blue component missing at the green component position in the original BAYER image. At this time, the reconstructed red component or blue component is close to the side with a small gradient change of the surrounding color, and is more different from the side with a large gradient change. This makes the details presented by the reconstructed red component and blue component rich and clear.

[0062] After completing step 3 in step S6 and step S7, only one red component or blue component pixel point is missing for color restoration. The h value is 1, and the m value is 1. Substitute the values ​​of h and m into the 45° and 135° edge detection operator formulas and select the colors in the pixel coordinates along the edge direction in the 3X3 matrix for interpolation restoration based on the edge calculation results, and finally restore the missing red component and blue component. In the process of reconstructing and restoring the missing components, the method of the present invention newly adds the judgment of the high and low directions of the green component gradient in the pixel coordinates along the edge direction and the gradient relationship with the green component color at the reconstructed pixel, and uses a special color restoration formula, so that the red component and blue component reconstructed in this step are close to the side with a small gradient change, and are more different from the side with a large gradient change.

[0063] In step S8, the interpolated color components are first constrained by data thresholds. Specifically, when the value of one or more of the green, red, and blue components of RGB is greater than or equal to the threshold value 4095, the value is 4095. When it is lower than the threshold value 4095, the original value is taken. After data fusion, high-definition 1080pRGB image data is finally output with a data bit width of 36 bits.

[0064] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An interpolation method based on FPAG image color restoration, characterized in that: including the following steps, Step S1: Construct a sequence of sliding matrices for restoring the green component G of the BAYER image. Specifically, a (2k + 1)×(2k + 1) matrix is used, where k is a positive integer, 1 < k < M and 1 < k < N. The parity of the image rows and columns is used as the basis for distinguishing the color at the restoration position. When the original BAYER color component at the position (i, j) of the matrix is the green component G i,j no restoration operation is performed. When performing the restoration of the green component, the matrix data reading format can be divided into the following two cases: ① When the original BAYER color component of the matrix (i, j) position is R i,j (red component) restores the missing green component G i,j , the matrix data readout format is: ②When the original BAYER color component at the matrix (i, j) position is B i,j (blue component) restores the missing green component G i,j , the matrix data read-out format is: ; Step S2: Calculate the gradient magnitude in the horizontal or vertical direction at the position where the green component needs to be restored. The gradient edge calculation formula at this position is: Where m = {1, 2, 3..., k}, m is a positive integer and 2m-1≤k, is the influence factor of other surrounding color components on the reconstructed green component, f(x) is the pixel data at (i, j), when calculating the horizontal edge gradient, f(x±2m) and f(x±2m±1) represent the pixel data at (i, j) with a left-right offset of 2m or (2m±1) coordinates, when calculating the vertical edge gradient, f(x) is the pixel data at (i, j), f(x±2m) and f(x±2m±1) represent the pixel data at (i, j) with a left-right offset of 2m or (2m±1) coordinates; Step S3: select the interpolation direction guided by the calculation result of the gradient edge calculation formula, and determine the green component interpolation restoration rule according to the gradient size and direction of the non-green components of the surrounding pixels of the reconstructed pixel and the gradient relationship of the non-green components at the reconstructed pixel. Specifically, according to the direction of the gradient edge at the (i, j) coordinate obtained by the calculation of the gradient edge calculation formula, the green component is interpolated and restored along the edge direction. G i,j Represents the restored green component. When reconstructing the green component along the horizontal direction, f(x) is the pixel data at (i, j). f(x±2m) and f(x±2m±1) represent the pixel data at (i, j) with a left-right offset of 2m or (2m±1). When reconstructing the green component along the vertical direction, f(x) is the pixel data at (i, j). f(x±2m) and f(x±2m±1) represent the pixel data at (i, j) with a left-right offset of 2m or (2m±1). The interpolation restoration rule follows the following calculation formula: ① When and or and hour, ② When and or and hour, Step S4: Fuse the green component obtained in Step 3 with the original BAYER image. The bit width of the image data becomes 3Z. Based on the fused image data, construct the sliding matrix required to restore the red and blue components. Specifically, construct a (2h + 1)×(2h + 1) matrix from the fused image data, where h is a positive integer, 1 < k < M and 1 < k < N. The matrix data reading format can be divided into the following four cases: ① When there is only a green component at the matrix (i, j) position, and the other color components in the vertical direction are red components, and the other color components in the horizontal direction are blue components, its matrix data reading format is: ② When there is only a green component at the matrix (i, j) position, and the other color components in the vertical direction are blue components, and the other color components in the horizontal direction are red components, its matrix data reading format is: ③ When there are green and blue components at the matrix (i, j) position, its matrix data reading format is: ④ When there are green and red components at the matrix (i, j) position, its matrix data reading format is: ; Step S5: Only G i,j The green component position restores the red component R i,j and blue component B i,j Specifically, the matrix readout format is as shown in the first and second matrix readout formats in step S4, and the interpolation restoration rule follows the following calculation formula: (1) When restoring the red component at the matrix (i, j), R i,j represents the reconstructed red component, f(x) is the red component at (i, j), g(x) is the green component at (i, j), and the reconstruction formula is as follows: ① When and or and hour, ② When and or and hour, (2) When the blue component is restored at the matrix (i, j), B i,j represents the reconstructed red component, f(x) is the blue component at (i, j), g(x) is the green component at (i, j), and the reconstruction formula is as follows: ③When and or and hour, ④When and or and hour, Where p = {1, 2, 3..., h}, p is a positive integer and 2p-1≤h, is the influence factor of the surrounding green component on the reconstructed color component. When the color is restored in the horizontal direction, f(x) is the color component at (i, j) that is the same as the restored color. f(x±2p) and f(x±2p±1) represent the color component at (i, j) that is offset 2p left and right or (2p±1) coordinates. When the color is restored in the vertical direction, f(x) is the pixel data at (i, j). f(x±2p) and f(x±2p±1) represent the color component at (i, j) that is offset 2p up and down or (2p±1) coordinates. The color component at (2p±1) coordinates is the same as the restored color; when the color is restored in the horizontal direction, g(x) is the green component at (i,j), and g(x±2p) and g(x±2p±1) represent the green component at (i,j) coordinates offset by 2p to the left or right or (2p±1) coordinates; when the color is restored in the vertical direction, g(x) is the green component at (i,j), and g(x±2p) and g(x±2p±1) represent the green component at (i,j) coordinates offset by 2p to the top or bottom or (2p±1) coordinates; Step S6: Only R i,j The red component position restores the blue component B i,j Or only B i,j The blue component position restores the red component B i,j , it is necessary to calculate the gradient magnitude in the 45° and 135° directions of the position. Specifically, the matrix readout format is as shown in the third and fourth matrix readout formats in step S4. The gradient magnitude calculation rule follows the following calculation formula: Where h>m>0, m is a positive integer, Δf 45° (i, j) and Δf 45° (i, j) are the edge gradient information at 45° and 135° respectively. and are the influence factors of other color components in the 45° direction and 135° surrounding on the reconstructed color components, f(i, j) is the color component to be restored at (i, j), f(i+2m-1, j-2m+1) and f(i-2m+1, j+2m-1) represent the color components at the coordinates (i, j) offset by (2m±1) in the 45° direction and the same as the restored color components; f(i + 2m - 1, j - 2m + 1) and f(i - 2m + 1, j + 2m - 1) represent the color components that are the same as the restored color components at the coordinates offset by (2m ± 1) in the 135° diagonal direction of the (i, j) coordinates. g(i, j) is the green component at (i, j). g(i + 2m - 1, j - 2m + 1) and g(i - 2m + 1, j + 2m - 1) respectively represent the green components at the coordinates offset by 2m or (2m ± 1) in the 45° diagonal direction of the (i, j) coordinates; g(i + 2m - 1, j - 2m + 1) and g(i - 2m + 1, j + 2m - 1) represent the green components at the coordinates offset by (2m ± 1) in the 135° diagonal direction of the (i, j) coordinates; Step S7: According to the 45° and 135° directional gradient results, determine the direction of the image gradient at the restored position, and perform color reconstruction according to the direction of the edge. i,j Represents the last two missing colors in the image. According to the color category that needs to be reconstructed, the corresponding color component can be calculated by substituting the color component of the position in the formula. The reconstruction formula is as follows: ① When color reconstruction is performed along the 45° direction, and or and hour, ② When color reconstruction is performed along the 45° direction, and or and hour, ③ When color reconstruction is performed along the 135° direction, and or when and hour, ④ When color reconstruction along the 135° direction, and or when and hour, Step S8: Perform data threshold constraints on the interpolated color components. Specifically, when the value of one or more color components among the green component, red component, and blue component of RGB is greater than the threshold 2 Z When -1, the value is 2 Z -1, below threshold 2 Z When -1, the original value is taken, and then data fusion is performed to output RGB image data; The interpolation method for color restoration based on the FPAG image makes the restored color components closer to the side with smaller gradient change along the edge direction and more different from the side with larger gradient change along the edge direction. At the same time, the edges of the image are clearer and sharper. When applied to the endoscopic imaging system, it can restore richer details and can achieve real-time pipeline processing in the FPGA.

Citation Information

Patent Citations

  • Fine definition demosaicking interpolation method based on FPGA platform

    CN105578160A

  • Bayer format image interpolation method based on improved edge orientation

    CN108024100A

  • Image color recovery method and device, computer readable storage medium and terminal

    CN108734668A

  • A Bayer image restoration method

    CN109285121A

  • Interpolation method for generating pixel color

    US20060078229A1