An FPGA-based Image Detail Enhancement Method

Abstract

The present invention discloses an image detail enhancement method based on FPGA, which includes the following steps: S1: Obtain the RGB image to be processed; S2: Denoise the RGB image to obtain the denoised low-frequency RGB image; S3: Sharpen the RGB image to enhance the details of the RGB image and obtain the enhanced RGB image; S4: Subtract and fuse the RGB image and the low-frequency RGB image according to weights to obtain the high-frequency RGB image; S5: Superimpose the high-frequency RGB image and the enhanced RGB image according to weights to obtain the fused RGB image; While using Laplace to highlight specific features, the present invention then uses Gaussian filtering to obtain the low-frequency image, subtract it from the original image according to weights, and then superimpose the images according to weights. When highlighting details, the noise in the entire image can also be effectively suppressed, thereby improving the quality of the overall image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly relates to an image detail enhancement method based on FPGA. Background Art

[0002] Detail enhancement is one of the core technologies in the field of image processing, which involves multiple aspects such as texture enhancement and detail restoration of images. Detail enhancement processing is an important technology for enhancing image edges and details. The purpose of detail enhancement processing is to enhance the target boundaries and details in the image to make them clearer. Through the enhancement processing, the visual effect of the image can be improved, especially suitable for scenarios where specific features need to be highlighted.

[0003] However, although the general detail enhancement algorithms will highlight specific features, they will also increase the noise of the overall image, making the noise points of the whole image more.

[0004] The Chinese invention patent with the publication number of CN110942431B discloses an image detail enhancement method, belonging to the technical field of image processing, including the following steps: separating the original blurred image into a low-resolution component and a high-resolution component; performing histogram equalization on the low-resolution component; converting the high-resolution component into a new frequency component L-1; fusing the low-resolution component after histogram equalization processing and the new frequency component L-1 to obtain a new image. This patent enhances the image details by separating into high-resolution and low-resolution components, performing histogram equalization on the low-resolution component and then fusing it with the high-resolution according to weights, but this will cause color distortion of the image.

[0005] The Chinese invention patent with the publication number of CN105869132B discloses an infrared image detail enhancement method, including the following steps: S1. obtaining a first detail layer image N1 by high-pass filtering; S2. obtaining a first smoothing layer image P2 by low-pass filtering and calculating a second detail layer image N2; S3. performing noise suppression and detail enhancement on the second detail layer image N2 based on the first detail layer image N1 to obtain a third detail layer image NE2; S4. performing stretching and histogram enhancement on the first smoothing layer image P2 to obtain a second smoothing layer image PE2; S5. synthesizing the third detail layer image NE2 and the second smoothing layer image PE2. This patent enhances the image details by obtaining multiple smoothing layers and detail layers through high-pass filtering, low-pass filtering, image stretching and histogram enhancement of the image and then fusing them, but when the image passes through high-pass filtering, some details of the image are actually lost, which is not conducive to the retention of image details.

[0006] A Chinese invention patent with the publication number CN104182939B discloses a method for enhancing the details of medical imaging images. The method includes the following steps: performing Gaussian pyramid decomposition on each layer of the image to obtain the next layer of the image and the high-frequency information of the current layer; respectively calculating the average value of the coefficients of the N*N template around each pixel of the high-frequency image and the low-frequency image to obtain the statistical matrix of the information area of the i-th layer of the image; repeating the above steps for layer-by-layer decomposition until the required decomposition level is reached; enhancing each layer of the high-frequency image and the low-frequency image in a certain manner to obtain the enhanced image; adding the enhanced low-frequency and high-frequency images to obtain the output image of the current layer, and upsampling the output image to obtain the next input image until the top layer image is reconstructed. This patent obtains multiple layers of images in the form of a Gaussian pyramid, extracts high-frequency information, and superimposes them to form a new image. However, essentially, the high-frequency image also contains noise. This method effectively suppresses the noise part in the high-frequency image. Summary of the Invention

[0007] The purpose of the present invention is to provide an image detail enhancement method based on FPGA to overcome the deficiencies in the prior art.

[0008] To achieve the above purpose, the present invention provides the following technical solutions:

[0009] The present application discloses an image detail enhancement method based on FPGA, including the following steps:

[0010] S1: Obtain the RGB image to be processed;

[0011] S2: Perform denoising processing on the RGB image to obtain the denoised low-frequency RGB image;

[0012] S3: Perform image sharpening processing on the RGB image to enhance the details of the RGB image and obtain the enhanced RGB image;

[0013] S4: Subtract and fuse the RGB image and the low-frequency RGB image according to the weight to obtain the high-frequency RGB image;

[0014] S5: Superimpose the high-frequency RGB image and the enhanced RGB image according to the weight to obtain the fused RGB image.

[0015] Preferably, the denoising processing of the RGB image in S2 is implemented by Gaussian filtering. Through Gaussian filtering, the entire image is filtered by convolving the entire image with a Gaussian kernel, thereby suppressing the noise of the entire image.

[0016] Preferably, the Gaussian filtering in S2 includes the following: for each pixel point in the RGB image, taking it as the center, obtaining the weighted average of the gray values of all pixels within its 3*3 area as the gray value of the center point, where the weight calculation includes: the closer the point, the greater the weight, and the farther the point, the smaller the weight.

[0017] Preferably, the image sharpening process in S3 is implemented by Laplacian enhancement. Through Laplacian enhancement, the entire image is enhanced by convolving the entire image with a Laplacian convolution kernel, thereby completing the detail enhancement of the entire image.

[0018] Preferably, the Laplacian enhancement in S3 includes the following: for the pixel points of the RGB image, when the gray value of the central pixel in the neighborhood is lower than the average gray value of other pixels in its neighborhood, reduce the gray value of the central pixel; otherwise, increase the gray value of the central pixel to achieve image sharpening.

[0019] Preferably, S4 includes the following sub-steps:

[0020] S41: Obtain coefficient one and coefficient two;

[0021] S42: Multiply the value of each pixel point in the RGB image by coefficient one to obtain intermediate value one;

[0022] S43: Multiply the value of each pixel point in the low-frequency RGB image by coefficient two to obtain intermediate value two;

[0023] S44: Subtract intermediate value two from intermediate value one for each pixel point to obtain a new image, namely the high-frequency RGB image.

[0024] Preferably, S5 includes the following sub-steps:

[0025] S51: Obtain coefficient three and coefficient four;

[0026] S52: Multiply the value of each pixel point in the high-frequency RGB image by coefficient three to obtain intermediate value three;

[0027] S53: Multiply the value of each pixel point in the enhanced RGB image by coefficient four to obtain intermediate value four;

[0028] S54: Add intermediate value three and intermediate value four for each pixel point to obtain a new image, namely the fused RGB image.

[0029] The present application also discloses an FPGA-based image detail enhancement device, including a system-on-chip bus controller, a detail enhancement circuit, a system-on-chip bus, an HDMI interface, an MIPI interface, a memory, and a non-volatile memory. An executable code is stored in the memory. When the one or more processors execute the executable code, it is used to implement the above-mentioned FPGA-based image detail enhancement method.

[0030] Advantages of the present invention:

[0031] (1) While the present invention uses Laplace to highlight specific features, it then uses Gaussian filtering to obtain a low-frequency image and subtracts it from the original image according to weights, and then superimposes the images according to weights. When highlighting details, the noise in the entire image can also be effectively suppressed, thereby improving the quality of the overall image.

[0032] The features and advantages of the present invention will be described in detail through embodiments in conjunction with the accompanying drawings. Description of the drawings

[0033] Figure 1 is a flowchart of an FPGA-based image detail enhancement method of the present invention;

[0034] Figure 2 is a schematic diagram of the device of the present invention. Detailed implementation manners

[0035] To make the purpose, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below through the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessarily confusing the concept of the present invention.

[0036] Refer to Figure 1 , an embodiment of the present invention provides an FPGA-based image detail enhancement method, including the following content:

[0037] S1: Obtain the RGB image to be processed;

[0038] Obtain the RGB image to be processed through an image acquisition device or read it from a storage.

[0039] S2: Denoise the RGB image;

[0040] For each pixel point in the RGB image, determine a specified area centered on it. In a feasible embodiment, the specified area is 3*3 in size;

[0041] According to the weight calculation method, weights are assigned to each pixel. The closer to the center, the greater the weight, and the farther away, the smaller the weight. The gray values of all pixels within the 3*3 area are weighted and averaged to obtain the new gray value of the center point, thereby realizing the filtering and denoising process of the entire image.

[0042] Among them, the denoising process for RGB images includes high-pass filtering and low-pass filtering. However, the low-pass filtering method is prone to losing details. Therefore, the high-pass filtering method is preferably used.

[0043] High-pass filtering means filtering the entire image by convolving the image with a Gaussian kernel, thereby suppressing some noise in the entire image.

[0044] In a feasible embodiment, the Gaussian kernel adopts a 3*3 matrix operation template, and the Gaussian kernel coefficients are as follows:

[0045]

[0046] Among them, the obtained denoised low-frequency RGB image means that after high-pass filtering, not only denoising is performed, but also some high-frequency signals of the image are removed.

[0047] S3: Perform image sharpening on the RGB image;

[0048] For each pixel of the RGB image, perform convolution operation using the Laplacian convolution kernel. According to the rule of Laplacian enhancement, when the gray value of the central pixel in the neighborhood is lower than the average gray value, its gray value is reduced, and vice versa, its gray value is increased, thereby realizing the detail enhancement of the entire image.

[0049] In a feasible embodiment, the Laplacian convolution kernel is a 5*5 matrix operation template, and the Laplacian convolution kernel is as follows:

[0050]

[0051] S4: Subtract and fuse the RGB image and the low-frequency RGB image;

[0052] The value of each pixel obtained by multiplying the value of each pixel of the RGB image by coefficient one k1 and subtracting the value of each pixel of the low-frequency RGB image multiplied by coefficient two k2 is used as a new image, that is, the high-frequency RGB image.

[0053] Coefficient one k1 is used to represent the weight of the RGB image. The larger coefficient one k1 is, the more high-frequency signal components the obtained high-frequency RGB image will have, and the more noise will also be. The value range of coefficient one k1 is 1 to 2; coefficient two k2 is used to represent the weight of the denoised low-frequency RGB image. The larger coefficient two k2 is, the less noise the obtained high-frequency RGB image will have, and the fewer high-frequency signal components will also be. The value range of coefficient two k2 is 0 to 1

[0054] S5: Superimpose and fuse the high-frequency RGB image and the enhanced RGB image;

[0055] The value of each pixel in the high-frequency RGB image is multiplied by coefficient k3, and the value of each pixel obtained by multiplying each pixel in the enhanced RGB image by coefficient k4 is used as the value of each pixel in a new image, that is, the fused RGB image;

[0056] Coefficient three k3 is used to represent the weight of the high-frequency RGB image. The larger coefficient three k3 is, the less noise there is in the resulting fused RGB image; coefficient four k4 is used to represent the weight of the enhanced RGB image. The larger coefficient four k4 is, the better the enhancement effect of the resulting fused RGB image. Specifically, increasing both coefficient three k3 and coefficient four k4 can enhance the image, but increasing coefficient four k4 has a more significant enhancement effect, but more noise will be generated.

[0057] An embodiment of the image detail enhancement device based on FPGA of the present invention can be applied to any device with data processing capabilities. Such a device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful device, it is formed by the processor of any device with data processing capabilities reading the corresponding computer program instructions in the non-volatile memory into the memory for operation. From a hardware perspective, as Figure 2 shown, it is a hardware structure diagram of any device with data processing capabilities where the image detail enhancement device based on FPGA of the present invention is located. In addition to Figure 2 the on-chip system bus controller, detail enhancement circuit, on-chip system bus, HDMI interface, MIPI interface, memory, and non-volatile memory shown, usually according to the actual functions of the device with data processing capabilities where the device in the embodiment is located, other hardware may also be included, which will not be elaborated here. The implementation processes of the functions and roles of each unit in the above device are specifically detailed in the implementation processes of the corresponding steps in the above method, which will not be elaborated here.

[0058] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Those of ordinary skill in the art can understand and implement it without creative work.

[0059] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. An FPGA-based image detail enhancement method, characterized in that: It includes the following steps: S1: Obtain the RGB image to be processed; S2: Denoise the RGB image to obtain the denoised low-frequency RGB image; S3: Sharpen the RGB image to enhance the details of the RGB image and obtain the enhanced RGB image; S4: Subtract and fuse the RGB image and the low-frequency RGB image according to weights to obtain the high-frequency RGB image; S5: Superimpose the high-frequency RGB image and the enhanced RGB image according to weights to obtain the fused RGB image; In step S3, the image sharpening process is implemented by the Laplacian enhancement method. Through the Laplacian enhancement method, the entire image is enhanced by convolving the entire image with a Laplacian convolution kernel, thereby completing the enhancement of the details of the entire image.

2. The image detail enhancement method based on FPGA according to claim 1, characterized in that: In step S2, the denoising process of the RGB image is implemented by the Gaussian filtering method. Through the Gaussian filtering method, the entire image is filtered by convolving the entire image with a Gaussian kernel, thereby suppressing the noise of the entire image.

3. The method for enhancing image details based on FPGA according to claim 2, wherein: The Gaussian filtering in step S2 includes the following: For each pixel point in the RGB image, a region is delimited with it as the center, and the weighted average of all pixel gray values in this region is obtained as the gray value of the center point. The weight calculation includes: The closer the point is, the greater the weight, and the farther the point is, the smaller the weight.

4. The method for enhancing image details based on FPGA according to claim 1, characterized in that: The Laplacian enhancement in step S3 includes the following: For the pixel points of the RGB image, when the gray value of the central pixel in the neighborhood is lower than the average gray value of other pixels in its neighborhood, the gray value of the central pixel is reduced; otherwise, the gray value of the central pixel is increased to implement the image sharpening process.

5. The image detail enhancement method based on FPGA according to claim 1, wherein: Step S4 includes the following sub-steps: S41: Obtain coefficient one and coefficient two, which respectively represent the weights of the RGB image and the low-frequency RGB image; S42: Multiply the value of each pixel point of the RGB image by coefficient one to obtain intermediate value one; S43: Multiply the value of each pixel point of the low-frequency RGB image by coefficient two to obtain intermediate value two; S44: Subtract intermediate value two from intermediate value one of each pixel point to obtain a new image, that is, the high-frequency RGB image.

6. The method for enhancing image details based on FPGA according to claim 5, characterized in that: Coefficient one in step S41 is used to represent the weight of the RGB image. The larger coefficient one is, the more high-frequency signal components the obtained high-frequency RGB image will have, and the more noise there will be. The value range of coefficient one is 1 to 2; coefficient two is used to represent the weight of the denoised low-frequency RGB image. The larger coefficient two is, the less noise the obtained high-frequency RGB image will have, and the fewer high-frequency signal components there will be. The value range of coefficient two is 0 to 1.

7. The image detail enhancement method based on FPGA according to claim 1, wherein: Step S5 includes the following sub-steps: S51: Obtain coefficient three and coefficient four, which respectively represent the weights of the high-frequency RGB image and the enhanced RGB image; S52: Multiply the value of each pixel point of the high-frequency RGB image by coefficient three to obtain intermediate value three; S53: Multiply the value of each pixel point of the enhanced RGB image by coefficient four to obtain intermediate value four; S54: Add intermediate value three and intermediate value four of each pixel point to obtain a new image, that is, the fused RGB image.

8. The method for enhancing image details based on FPGA according to claim 7, characterized in that: The coefficient three in S51 is used to represent the weight of the high-frequency RGB image. The larger the coefficient three is, the less noise there is in the fused RGB image obtained; the coefficient four is used to represent the weight of the enhanced RGB image. The larger the coefficient four is, the better the enhancement effect of the fused RGB image obtained.

Images