Image detail enhancement method based on FPGA

Abstract

The invention discloses an image detail enhancement method based on an FPGA (Field Programmable Gate Array). The method comprises the following steps: S1, acquiring an RGB (Red, Green and Blue) image to be processed; s2, carrying out denoising processing on the RGB image to obtain a denoised low-frequency RGB image; s3, performing image sharpening processing on the RGB image, enhancing details of the RGB image, and obtaining an enhanced RGB image; s4, performing weight subtraction fusion on the RGB image and the low-frequency RGB image to obtain a high-frequency RGB image; s5, superposing the high-frequency RGB image and the enhanced RGB image according to the weight to obtain a fused RGB image; according to the method, specific features are highlighted through Laplacian, meanwhile, a low-frequency image obtained through Gaussian filtering is subtracted from an original image according to the weight, the low-frequency image and the original image are superposed according to the weight, noise points of the whole image can be effectively restrained when details are highlighted, and therefore the quality of the whole image is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image detail enhancement method based on FPGA. Background Art

[0002] Detail enhancement is one of the core technologies in the field of image processing, which involves multiple aspects such as image texture enhancement and detail recovery. Detail enhancement processing is an important technology for enhancing image edges and details. The purpose of detail enhancement processing is to enhance the target boundaries and details in the image to make them clearer. Through enhancement processing, the visual effect of the image can be improved, which is especially suitable for scenes that need to highlight specific features.

[0003] However, although the usual detail enhancement algorithm will highlight specific features, it will also increase the noise of the overall image, making the entire image noisier.

[0004] The Chinese invention patent with publication number CN110942431B discloses a method for enhancing image details, which belongs to the field of image processing technology and includes the following steps: separating the original blurred image into a low-resolution component and a high-resolution component; applying histogram equalization to the low-resolution component; converting the high-resolution component into a new frequency component L‑1; fusing the low-resolution component after histogram equalization with the new frequency component L‑1 to obtain a new image. This patent enhances image details by dividing the image into a high-resolution component and a low-resolution component, and then fusing the low-resolution component with histogram equalization and then fusing it with the high-resolution component according to weights, but this will cause color distortion of the image.

[0005] The Chinese invention patent with the publication number CN105869132B discloses a method for enhancing infrared image details, including the following steps: S1. High-pass filtering to obtain the first detail layer image N1; S2. Low-pass filtering to obtain the first smooth layer image P2, and calculating the second detail layer image N2; S3. Based on the first detail layer image N1, the second detail layer image N2 is subjected to noise suppression and detail enhancement to obtain the third detail layer image NE2; S4. The first smooth layer image P2 is stretched and histogram enhanced to obtain the second smooth layer image PE2; S5. The third detail layer image NE2 and the second smooth layer image PE2 are synthesized. The patent enhances the image details by obtaining multiple smooth layers and detail layers by high-pass filtering, low-pass filtering, image stretching and histogram enhancement, and then merging them to form an image. However, after the image is subjected to high-pass filtering, the image has actually lost some details, which is not conducive to the retention of image details.

[0006] The Chinese invention patent with publication number CN104182939B discloses a method for enhancing the details of medical imaging images, which includes the following steps: performing Gaussian pyramid decomposition on each layer of the image to obtain the next layer of the image and the high-frequency information of the current layer; respectively counting the coefficient averages of the N*N templates around each pixel of the high-frequency image and the low-frequency image to obtain the statistical matrix of the image information area of ​​the i-th layer; repeating the above steps to decompose layer by layer until the required decomposition level is reached; enhancing each layer of high-frequency image and low-frequency image in a certain way to obtain an enhanced image; adding the enhanced low-frequency and high-frequency images to obtain the output image of the current layer, and upsampling the output image to obtain the lower input image until the top layer image is reconstructed. The patent obtains multiple layers of images in the form of a Gaussian pyramid and extracts high-frequency information and then superimposes them to form a new image, but essentially the high-frequency image also contains noise. This method is an effective way to suppress the noise in the high-frequency image. Summary of the invention

[0007] The purpose of the present invention is to provide an image detail enhancement method based on FPGA to overcome the deficiencies in the prior art.

[0008] To achieve the above object, the present invention provides the following technical solutions: The present application discloses an image detail enhancement method based on FPGA, comprising the following steps: S1: Get the RGB image to be processed; S2: De-noising the RGB image to obtain a denoised low-frequency RGB image; S3: performing image sharpening processing on the RGB image to enhance the details of the RGB image and obtain an enhanced RGB image; S4: Subtract and fuse the RGB image and the low-frequency RGB image according to the weights to obtain a high-frequency RGB image; S5: Superimpose the high-frequency RGB image and the enhanced RGB image according to the weights to obtain a fused RGB image.

[0009] Preferably, the denoising process of the RGB image in S2 is implemented by Gaussian filtering. By Gaussian filtering, the entire image is filtered in the form of convolution of the entire image with a Gaussian kernel, thereby suppressing the noise of the entire image.

[0010] Preferably, the Gaussian filtering in S2 includes the following contents: for each pixel point in the RGB image, taking it as the center, obtain the weighted average of the grayscale values ​​of all pixels in its 3*3 area as the grayscale value of the center point, wherein the weight calculation includes: the closer the point, the greater the weight, and the farther the point, the smaller the weight.

[0011] Preferably, the image sharpening processing in S3 is implemented by a Laplace enhancement method, in which the entire image is enhanced by convolving the entire image with a Laplace convolution kernel, thereby completing detail enhancement of the entire image.

[0012] Preferably, the Laplace enhancement in S3 includes the following contents: for a pixel point of an RGB image, when the grayscale of a central pixel in a neighborhood is lower than the average grayscale of other pixels in its neighborhood, the grayscale of the central pixel is reduced; otherwise, the grayscale of the central pixel is increased to achieve image sharpening processing.

[0013] Preferably, S4 includes the following sub-steps: S41: Obtain coefficient one and coefficient two; S42: multiply the value of each pixel of the RGB image by coefficient 1 to obtain an intermediate value 1; S43: multiplying the value of each pixel of the low-frequency RGB image by coefficient 2 to obtain intermediate value 2; S44: Subtract the intermediate value 1 from the intermediate value 2 of each pixel to obtain a new image, that is, a high-frequency RGB image.

[0014] Preferably, S5 includes the following sub-steps: S51: Obtain coefficient three and coefficient four; S52: multiply the value of each pixel of the high-frequency RGB image by coefficient three to obtain intermediate value three; S53: multiplying the value of each pixel of the enhanced RGB image by a coefficient of four to obtain an intermediate value of four; S54: The intermediate value three and the intermediate value four of each pixel are added to obtain a new image, that is, a fused RGB image.

[0015] The present application also discloses an FPGA-based image detail enhancement device, including an on-chip system bus controller, a detail enhancement circuit, an on-chip system bus, an HDMI interface, a MIPI interface, a memory and a non-volatile memory, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned FPGA-based image detail enhancement method.

[0016] Beneficial effects of the present invention: (1) The present invention uses Laplace to highlight specific features, and then uses Gaussian filtering to obtain a low-frequency image and subtract it from the original image according to weights, and then superimposes the images according to the weights. When highlighting details, the noise of the entire image can also be effectively suppressed, thereby improving the quality of the overall image.

[0017] The features and advantages of the present invention will be described in detail through embodiments in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of an image detail enhancement method based on FPGA of the present invention; Figure 2 It is a schematic diagram of the device of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below through the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the scope of the present invention. In addition, in the following description, the description of known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0020] See also Figure 1 The embodiment of the present invention provides an image detail enhancement method based on FPGA, including the following contents: S1: Get the RGB image to be processed; Read the RGB image to be processed through the image acquisition device or from storage.

[0021] S2: denoising the RGB image; For each pixel in the RGB image, a designated area is determined with the pixel as the center. In a feasible embodiment, the designated area is 3*3 in size; According to the weight calculation method, a weight is assigned to each pixel point. The closer to the center, the greater the weight, and the farther away, the smaller the weight. The grayscale values ​​of all pixels in the 3*3 area are weighted averaged to obtain the new grayscale value of the center point, thereby achieving filtering and denoising of the entire image.

[0022] Among them, the denoising process for the RGB image includes high-pass filtering and low-pass filtering, but the low-pass filtering method is prone to lose details, so the high-pass filtering method is preferred.

[0023] High-pass filtering refers to filtering the entire image by convolving the entire image with a Gaussian kernel, thereby suppressing some noise in the entire image.

[0024] In a feasible embodiment, the Gaussian kernel uses a 3*3 matrix operation template, and the Gaussian kernel coefficient is: The denoised low-frequency RGB image means that after high-pass filtering, not only the denoising process is performed, but also part of the high-frequency signals of the image are removed.

[0025] S3: perform image sharpening processing on the RGB image; For each pixel of the RGB image, a convolution operation is performed using the Laplace convolution kernel. According to the Laplace enhancement rule, when the grayscale of the central pixel in the neighborhood is lower than the average grayscale, its grayscale is reduced, otherwise its grayscale is increased, thereby enhancing the details of the entire image.

[0026] In a feasible embodiment, the Laplace convolution kernel is a 5*5 matrix operation template, and the Laplace convolution kernel is: S4: Subtract and fuse the RGB image and the low-frequency RGB image; The value of each pixel point of the RGB image multiplied by the coefficient k1 minus the value of each pixel point of the low-frequency RGB image multiplied by the coefficient k2 is obtained as a new image, namely the high-frequency RGB image.

[0027] The coefficient k1 is used to represent the weight of the RGB image. The larger the coefficient k1 is, the more high-frequency signal components and noise the high-frequency RGB image will have. The value range of the coefficient k1 is 1~2. The coefficient k2 is used to represent the weight of the denoised low-frequency RGB image. The larger the coefficient k2 is, the less noise and high-frequency signal components the high-frequency RGB image will have. The value range of the coefficient k2 is 0~1. S5: superimpose and fuse the high-frequency RGB image with the enhanced RGB image; The value of each pixel of the high-frequency RGB image multiplied by the coefficient k3 plus the value of each pixel of the enhanced RGB image multiplied by the coefficient k4 is used as a new image, that is, the fused RGB image; The coefficient three k3 is used to represent the weight of the high-frequency RGB image. The larger the coefficient three k3 is, the less noise the fused RGB image will have. The coefficient four k4 is used to represent the weight of the enhanced RGB image. The larger the coefficient four k4 is, the better the enhancement effect of the fused RGB image will be. Specifically, increasing the coefficient three k3 and the coefficient four k4 can enhance the image, but increasing the coefficient four k4 has a more significant enhancement effect, but more noise will be generated.

[0028] An embodiment of an image detail enhancement device based on FPGA of the present invention can be applied to any device with data processing capability, and the device with data processing capability can be a device or apparatus such as a computer. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the processor of any device with data processing capability reads the corresponding computer program instructions in the non-volatile memory into the memory and runs them. From the hardware level, if Figure 2As shown in the figure, it is a hardware structure diagram of any device with data processing capability where an image detail enhancement device based on FPGA of the present invention is located. Figure 2 In addition to the system-on-chip bus controller, detail enhancement circuit, system-on-chip bus, HDMI interface, MIPI interface, memory and non-volatile memory shown, any device with data processing capability in the embodiment may also include other hardware according to the actual function of the device with data processing capability, which will not be described in detail. The implementation process of the functions and effects of each unit in the above device is specifically described in the implementation process of the corresponding steps in the above method, which will not be described in detail here.

[0029] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0030] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent substitution or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image detail enhancement method based on FPGA, characterized in that: The steps include: S1: Get the RGB image to be processed; S2: De-noising the RGB image to obtain a denoised low-frequency RGB image; S3: performing image sharpening processing on the RGB image to enhance the details of the RGB image and obtain an enhanced RGB image; S4: Subtract and fuse the RGB image and the low-frequency RGB image according to the weights to obtain a high-frequency RGB image; S5: Superimpose the high-frequency RGB image and the enhanced RGB image according to the weights to obtain a fused RGB image.

2. The method for enhancing image details based on FPGA as claimed in claim 1, characterized in that: The denoising process of the RGB image in S2 is implemented by Gaussian filtering. By using Gaussian filtering, the entire image is filtered in the form of convolution of the entire image with a Gaussian kernel, thereby suppressing the noise of the entire image.

3. The FPGA-based image detail enhancement method according to claim 2, characterized in that: The Gaussian filtering in S2 includes the following contents: for each pixel point in the RGB image, an area is delineated with it as the center, and the weighted average of the grayscale values ​​of all pixels in the area is obtained as the grayscale value of the center point, wherein the weight calculation includes: the closer the point, the greater the weight, and the farther the point, the smaller the weight.

4. The method for enhancing image details based on FPGA as claimed in claim 1, characterized in that: The image sharpening process in S3 is implemented by Laplace enhancement. By Laplace enhancement, the whole image is enhanced by convolving the whole image with a Laplace convolution kernel, thereby completing detail enhancement of the whole image.

5. The method for enhancing image details based on FPGA as claimed in claim 4, characterized in that: The Laplace enhancement in S3 includes the following contents: for a pixel point of an RGB image, when the grayscale of a central pixel in a neighborhood is lower than the average grayscale of other pixels in its neighborhood, the grayscale of the central pixel is reduced; otherwise, the grayscale of the central pixel is increased to achieve image sharpening processing.

6. The method for enhancing image details based on FPGA as claimed in claim 1, characterized in that: The S4 includes the following sub-steps: S41: Obtain coefficient 1 and coefficient 2, which respectively represent the weights of the RGB image and the low-frequency RGB image; S42: multiply the value of each pixel of the RGB image by coefficient 1 to obtain an intermediate value 1; S43: multiplying the value of each pixel of the low-frequency RGB image by coefficient 2 to obtain intermediate value 2; S44: Subtract the intermediate value 1 from the intermediate value 2 of each pixel to obtain a new image, that is, a high-frequency RGB image.

7. The FPGA-based image detail enhancement method according to claim 6, characterized in that: The coefficient one in S41 is used to represent the weight of the RGB image. The larger the coefficient one is, the more high-frequency signal components of the obtained high-frequency RGB image will be, and the more noise will be. The value range of the coefficient one is 1~2; the coefficient two is used to represent the weight of the denoised low-frequency RGB image. The larger the coefficient two is, the less noise of the obtained high-frequency RGB image will be, and the less high-frequency signal components will be. The value range of the coefficient two is 0~1.

8. The method for enhancing image details based on FPGA as claimed in claim 1, characterized in that: The S5 includes the following sub-steps: S51: Obtain coefficient three and coefficient four, which respectively represent weights of the high-frequency RGB image and the enhanced RGB image; S52: multiply the value of each pixel of the high-frequency RGB image by coefficient three to obtain intermediate value three; S53: multiplying the value of each pixel of the enhanced RGB image by a coefficient of four to obtain an intermediate value of four; S54: The intermediate value three and the intermediate value four of each pixel are added to obtain a new image, that is, a fused RGB image.

9. The method for enhancing image details based on FPGA as claimed in claim 8, characterized in that: The coefficient three in S51 is used to represent the weight of the high-frequency RGB image. The larger the coefficient three is, the less noise the fused RGB image will have. The coefficient four is used to represent the weight of the enhanced RGB image. The larger the coefficient four is, the better the enhancement effect of the fused RGB image will be.

Images