Image quality enhancement method and device based on FPGA chip

Abstract

According to the image quality enhancement method and device based on the FPGA chip, a CLAHE equalization algorithm, saturation improvement and oval face detection are adopted, a new image quality enhancement function is formed, an FPGA hardware architecture can receive videos with any resolution and frame rate specification from the outside, any related video interface is compatible, and the image quality enhancement method and device based on the FPGA chip can be applied to the field of video processing. HSV and YUV conversion is carried out on the RGB model of the video image, CLAHE contrast equalization processing is carried out respectively, and saturation processing is carried out after face detection. An FPGA chip is embedded in a video link of a common display, parameters of image quality enhancement contrast and parameters of saturation can be configured, an image quality enhancement algorithm is deployed in the FPGA chip, the contrast and the saturation of a video image after algorithm processing can be obviously improved, the fidelity of the face skin color can be ensured, and the method can be used for providing a new method for an intelligent cabin of an electric vehicle. The television in the consumption field has an image quality improvement effect.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method for enhancing image quality based on an FPGA chip, a device for enhancing image quality based on an FPGA chip, a computer device and a storage medium. Background Art

[0002] High dynamic contrast and high color gamut are important indicators of display screens. Ordinary car screens generally have high backlight brightness, but the dynamic contrast and color saturation are relatively lacking, and the display effect is often too ordinary. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a method for image quality enhancement based on an FPGA chip, an apparatus for image quality enhancement based on an FPGA chip, a computer device and a storage medium that overcome the above problems or at least partially solve the above problems.

[0004] To achieve the above object, an embodiment of the present invention proposes a method for image quality enhancement based on an FPGA chip, the method comprising:

[0005] Get a frame of RGB image data;

[0006] Convert the RGB image data into HSV model data and YCrCb model data respectively;

[0007] Calculate the pdf of the pixel points in the partition of the HSV model data;

[0008] Obtaining the cdf value of the pixel points in the partition by calculating the pdf of the pixel points in the partition;

[0009] Performing a spatial convolution smoothing filter operation on the cdf values ​​of the pixels in the partition to obtain and store the cdf value of the frame image, and extracting the cdf value of the previous frame;

[0010] Perform bilinear interpolation on the V channel data of the HSV model data and the cdf value of the previous frame to obtain interpolated V channel data;

[0011] Identify a face area of ​​the YCrCb model data, and adjust a nonlinear ratio of the S channel data of the face area to obtain adjusted S channel data;

[0012] The original H channel data, the interpolated V channel data and the adjusted S channel data of the HSV model data are combined into new HSV model data, the new HSV model data is converted into new RGB image data, and the new RGB image data is output.

[0013] Preferably, the step of calculating the pdf of the pixels in the partition of the HSV model data comprises:

[0014] Get the width and height data of each partition, and calculate the grayscale average value of each partition in the required interval under the width and height data;

[0015] The histogram is truncated by setting a value higher than the grayscale average; the histogram of the frame image is calculated on the grayscale level, and all the excess parts are evenly distributed to all grayscale levels. After the grayscale is truncated and redistributed, a new PDF is obtained.

[0016] Preferably, the step of calculating the cdf value of the pixel points in the partition by using the pdf of the pixel points in the partition comprises:

[0017] Cumulative statistics are performed on the PDF until the number of pixel statistics in the interval is completed, forming a cdf mapping lookup table, and the cdf values ​​of the pixel points in the partition are obtained according to the cdf mapping lookup table.

[0018] Preferably, performing a spatial convolution smoothing filter operation on the cdf values ​​of the pixels in the partition to obtain and store the cdf value of the frame image, and extracting the cdf value of the previous frame, comprises:

[0019] The matrix corresponding to the cdf value is expanded to obtain the expanded matrix, a 3x3 convolution kernel is used to perform spatial filtering to obtain the updated cdf function, weighted average is used for temporal filtering to smooth the cdf function between frames, the cdf value of the frame image is obtained and stored, and the cdf value of the previous frame is extracted.

[0020] Preferably, bilinear interpolation is performed on the V channel data according to the HSV model data and the cdf value of the previous frame to obtain the interpolated V channel data, including:

[0021] Each sub-region of the partition is divided into four smaller equal parts, and each smaller equal part forms an interpolation relationship with the other three adjacent regions;

[0022] The distance between a certain pixel point and the center of the distance area is obtained, and interpolation calculation is performed according to the output values ​​of the cdf mapping lookup table of the four areas corresponding to the distance and the brightness of the pixel point to obtain the interpolated V channel data.

[0023] Preferably, the identifying the face area of ​​the YCrCb model data, adjusting the nonlinear ratio of the S channel data of the face area, and obtaining the adjusted S channel data includes:

[0024] Determine the center point, establish an elliptical area around the central red dot, determine the elliptical area as the face area, obtain the pixel points falling into the face area, control the nonlinear proportion of the S channel data of the face area to increase, control the global linear proportion of the S channel data outside the face area to increase, and obtain the adjusted S channel data.

[0025] The embodiment of the present invention provides a device for enhancing image quality based on an FPGA chip, the device comprising:

[0026] An image data acquisition module is used to acquire a frame of RGB image data;

[0027] A conversion module, used for converting the RGB image data into HSV model data and YCrCb model data respectively;

[0028] A first calculation module, used to calculate the pdf of the pixel points in the partition of the HSV model data;

[0029] A second calculation module is used to calculate the cdf value of the pixel points in the partition by using the pdf of the pixel points in the partition;

[0030] A filtering module, used for performing a spatial convolution smoothing filtering operation on the cdf values ​​of the pixels in the partition, obtaining and storing the cdf value of the frame image, and extracting the cdf value of the previous frame;

[0031] An interpolation module, used for performing bilinear interpolation according to the V channel data of the HSV model data and the cdf value of the previous frame to obtain interpolated V channel data;

[0032] A nonlinear ratio module, used for identifying the face area of ​​the YCrCb model data, adjusting the nonlinear ratio of the S channel data of the face area, and obtaining the adjusted S channel data;

[0033] The composition module is used to combine the original H channel data, the interpolated V channel data and the adjusted S channel data of the HSV model data into new HSV model data, convert the new HSV model data into new RGB image data, and output the new RGB image data.

[0034] Preferably, the first calculation module includes:

[0035] The first acquisition submodule is used to obtain the width and height data of each partition, and calculate the grayscale average value within the interval of each partition under the width and height data;

[0036] The calculation submodule is used to set a value higher than the grayscale average to truncate the histogram; calculate the histogram of the frame image on the grayscale, and evenly distribute all the excess parts to all the grayscales. After the grayscale is truncated and redistributed, a new PDF is obtained.

[0037] An embodiment of the present invention discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned image quality enhancement method based on an FPGA chip when executing the computer program.

[0038] An embodiment of the present invention discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned image quality enhancement method based on an FPGA chip are implemented.

[0039] In an embodiment of the present invention, an algorithm for image quality enhancement is provided, which realizes the functions of contrast enhancement and saturation improvement. The algorithm adopts the CLAHE equalization algorithm plus saturation improvement plus elliptical face detection, which constitutes a new function of image quality enhancement and can meet the demand for image quality improvement of the intelligent cockpit display of electric vehicles. The FPGA hardware architecture of the image quality enhancement algorithm of the present invention can receive videos of any resolution and frame rate specifications from the outside, is compatible with any relevant video interface, is compatible with any 1-way, 2-way and 4-way input videos, performs HSV and YUV conversion on the RGB model of the video image, performs CLAHE contrast equalization processing respectively, and performs saturation processing after face detection. In the video link of an ordinary display, an FPGA chip is embedded, and the FPGA communicates with the MCU in the module to realize parameter configuration, and the parameters of image quality enhancement contrast and saturation can be configured. FPGA is compatible with videos of any interface and resolution and frame rate specifications. Any video enhanced by FPGA can display the enhanced effect in the display, and the enhancement algorithm can also be shielded by MCU configuration parameters, and the display can display the original video effect. An image quality enhancement algorithm is deployed in the FPGA chip. After being processed by the algorithm, the contrast and saturation of the video image can be significantly improved, and the skin color of the face can be preserved, which can bring image quality improvement effects to the smart cockpit of electric vehicles and consumer televisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1This is a system architecture diagram of an image quality enhancement system based on an FPGA chip according to an embodiment of the present invention;

[0042] Figure 2 This is a comparison diagram of an electric vehicle's onboard central control screen before and after image processing according to an embodiment of the present invention;

[0043] Figure 3 is a basic principle diagram of limiting contrast according to an embodiment of the present invention;

[0044] Figure 4 is a schematic diagram of a platform framework of an embodiment of the present invention;

[0045] Figure 5 It is a schematic diagram of a top-level architecture of an FPGA chip according to an embodiment of the present invention;

[0046] Figure 6 It is a work flow chart of an FPGA according to an embodiment of the present invention;

[0047] Figure 7 is a schematic diagram of a sub-region bilinear interpolation region in an embodiment of the present invention;

[0048] Figure 8 is a principle diagram of a face detection algorithm according to an embodiment of the present invention;

[0049] Fig. 9 It is a structural block diagram of an embodiment of an image quality enhancement device based on an FPGA chip according to an embodiment of the present invention;

[0050] Fig.10 The diagram is a diagram of the internal structure of a computer device according to an embodiment. DETAILED DESCRIPTION

[0051] In order to make the technical problems, technical solutions and beneficial effects solved by the embodiments of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0052] Reference Figure 1 , shows a flowchart of a method for enhancing image quality based on an FPGA chip according to an embodiment of the present invention, which may specifically include the following steps:

[0053] Step 101, obtaining a frame of RGB image data;

[0054] The image quality enhancement device in the embodiment of the present invention can be arranged in the cockpit of an electric vehicle to display high dynamic contrast video images for the driver and passengers, and provide users with a good visual experience. For example, the physical picture of the system device is as follows: Figure 2The screen shown is 14.6 inches, Figure 2 The a in represents the video image before processing. Figure 2 b in the figure indicates that the video image processed by the method can also be adapted to screens of other sizes with different resolution specifications, such as televisions in the consumer field. The algorithm is deployed on the FPGA to form an image processing hardware product, which can enhance the image quality of the external digital TV signal and input it into the TV to improve the display effect of the TV.

[0055] High dynamic contrast and high color gamut are important indicators of display screens. Ordinary car screens generally have high backlight brightness, but the dynamic contrast and color saturation are relatively poor, and the display effect often appears bland. In the smart cockpit that highlights the sense of technology and experience, rendering the screen display quality into a three-dimensional layered and colorful effect is one of the technical effects of this application.

[0056] In terms of pixel brightness, the greater the degree of brightness and darkness between pixels, the greater the contrast, while the closer the degree of brightness and darkness, the smaller the contrast. In order to improve the contrast of the picture, the work converts the RGB model into the HSV model and performs equalization processing on the pixel brightness value V to achieve a clear distinction between light and dark, and the details in the dark are clearly visible. Figure 2 The sub-image a in is the original image, Figure 2 The contrast of sub-image b in the figure is improved. For example, the peak has both bright and dark sides, showing a distinct effect of light and dark. This is because the contrast is improved. The image is divided into local nxn areas such as 4x4, 6x6 or 8x8 partitions, depending on the image resolution. Since it is partition processing rather than global processing, it is reflected as an adaptive function.

[0057] The equalization operation of this application is mainly to calculate the pdf (Probability Density Function) function of the pixel brightness in the statistical area, and further convert the pdf function into the cdf (Cumulative Distribution Function) function. The V of each pixel point is converted into the equalized V' through the mapping of the cdf function. After the HSV' is converted into RGB, the contrast of the image is improved. When the histogram shrinks in a narrow brightness range, the cdf function curve changes too steeply, causing a serious mutation from V to V' at a certain brightness threshold boundary. The corresponding picture effect is that it becomes very dark after equalization below a certain threshold brightness, such as very close to 0, and the brightness becomes very bright after exceeding the brightness threshold, such as very close to 255. In order to avoid this kind of mutation effect, the histogram is restricted, such as Figure 3 As shown, Figure 3Sub-image a in has a histogram. Since the brightness is too concentrated in a narrow interval, the peak is removed and evenly distributed to other brightness intervals to obtain a relatively smooth cdf function, such as Figure 3 As shown in sub-figure b in .

[0058] Saturation refers to the color of a pixel. It is generally believed that the purer the color, the higher the saturation. For example, the brighter the red, the higher the saturation, and the lighter the red, the lower the saturation. This work converts the RGB model into the HSV model and processes the saturation S. The color saturation of the picture is improved, showing a vivid effect. Figure 2 The sky in sub-image a is pale blue. Figure 2 The sub-image b in the figure shows the effect of saturation enhancement, and the sky appears more gorgeous in dark blue. At present, the saturation is mainly enhanced by a global linear method, multiplying each pixel by the preset k value. When k>1, the saturation is enhanced. In addition, RGB is converted to YUV, and the chroma channel UV is used to detect faces. Where there are faces, the saturation enhancement coefficient is reduced to keep the skin color of the face basically unchanged. Specifically, the UV interval where the face is located is obtained through empirical values. If the pixel value falls within this interval, the k value will remain at 1, and the k value outside this interval will remain at the preset value.

[0059] The algorithm of this application is finally deployed in the FPGA chip using the Verilog hardware description language. The FPGA chip is integrated into the display module. The display screen embedded with the FPGA chip is an independent device. The video interface of the device, such as the eDP and LVDS interfaces, is connected to the external video. After the FPGA image quality is enhanced and rendered, the expected effect can be displayed on the screen. The external video source includes but is not limited to PC. In the actual development process, Figure 4 The main control board is mainly used for interface conversion, converting external HDMI to LVDS, and of course it is also compatible with many different video interfaces.

[0060] The top-level architecture of the FPGA chip of the present invention is as follows Figure 5 As shown, the external video image enters the FPGA chip through two LVDS channels, and transmits the RGB pixels of the odd channel and the even channel respectively.

[0061] In order to process the contrast V and saturation S, the RGB to HSV model conversion is performed, and in order to perform face detection, the RGB to YUV model conversion is performed. The CLAHE algorithm only samples odd-path pixels, and the V plane converted from the odd-path image is partitioned into 8x8 to form 64 regions. The brightness V statistics of 0-255 are performed on each region to obtain the corresponding pdf, and the pdf of each region is processed and converted into a cdf mapping function. The cdf value in the 8x8 region is convolved using a 3x3 convolution kernel to obtain the updated 8x8 cdf functions. The specific implementation method will be described in detail below. In order to smooth the cdf value between video frames, the cdf values ​​of the corresponding regions of the previous and next frames are weighted averaged, and the weight value is adjustable. The specific implementation method will be described in detail below. The V channel of the current frame pixel is converted to V' through cdf, in which the even path is not calculated by the CLAHE algorithm, and the cdf value calculated by the odd path is directly shared. In the process of cdf value conversion, it is necessary to further use the bilinear interpolation method to finally convert V into V'. The specific steps are described as follows.

[0062] On the other hand, the facial skin color detection method of the present application first determines the center point of the facial skin color on the UV plane, and establishes an elliptical area around this center point. The pixels falling within this area can be considered as the facial skin color. The saturation enhancement in the area requires progressive proportional amplification rather than directly multiplying it with the preset saturation value.

[0063] The saturation enhancement module mainly multiplies the S channel of the pixel by the preset ratio value. However, if the pixel falls into the UV elliptical area of ​​the facial skin color, the corresponding ratio value will be smaller than the preset value. The specific output is determined by the facial skin color algorithm.

[0064] The FPGA workflow of this application is as follows Figure 6 As shown, first, a certain frame of RGB image data can be obtained, that is, the current image is read in through the video interface and the RGB pixels are decoded, as well as the corresponding line synchronization signal VS, field synchronization signal HS, and pixel valid signal DE. The video interface can be LVDS, or eDP, or it can be a single signal input. For example, for a resolution of 1920x1080, 1920 pixels are output in one HS cycle; if two signals are input, 960 pixels are output in one HS cycle, which is divided into odd and even channels. If four channels are input, 480 pixels are output in one HS cycle. This is the characteristic of different interfaces. The present invention is explained with the case of two inputs of the LVDS interface. The present invention only performs pdf and cdf analysis on one of the channels, such as the odd channel, and calculates the cdf of the other even channel with reference to the odd channel.

[0065] Step 102, converting the RGB image data into HSV model data and YCrCb model data respectively;

[0066] Furthermore, the RGB image data can be converted into HSV model data and YCrCb model data respectively; specifically, the RGB model is converted into the HSV model, and the conversion formula is as follows:

[0067]

[0068] On the other hand, the conversion formula for converting the RGB model to the YCrCb model is:

[0069] Y=0.299R+0.587G+0.114B

[0070] C b =-0.168R-0.3313G+0.5B+128

[0071] C r =0.5R-0.4187G-0.0813B+128 (4)

[0072] Step 103, calculating the pdf of the pixel points in the partition of the HSV model data;

[0073] In an embodiment of the present invention, the calculation of the pdf of the pixel points in the partition of the HSV model data includes: obtaining the width and height data of each partition, calculating the grayscale average value in each partition interval under the width and height data; setting a value higher than the grayscale average value to perform a truncation operation on the histogram; calculating the histogram of the frame image on the grayscale level, and evenly distributing all the excess parts to all the grayscale levels, and obtaining a new pdf after the grayscale levels are truncated and redistributed;

[0074] For example, to calculate the pdf of the pixels in the 8x8 partition of the image data, we need to calculate the grayscale histogram statistics of the pixels in each of the 64 regions, using V in HSV as the grayscale value, and counting the number of each grayscale level in the interval [0,255]. Input the contrast cutoff threshold ClipLimit, and calculate the average value of each grayscale level in each partition according to the video input size, as shown in the following formula. Assuming that the width and height of a partition are W and H respectively, the ideal histogram has the grayscale values ​​of these WxH pixels evenly distributed in the interval [0,255]. Therefore, the calculation formula for average is:

[0075]

[0076] Where W and H are the width and height of each partition in pixels. Therefore, the histogram is usually truncated with a value slightly larger than the average, and this value is represented by ClipThreshold:

[0077] ClipThreshold=round(ClipLimit×average) (6)

[0078] Where round() represents the rounding operation.

[0079] Count the histogram of the video frame in grayscale, and count the number of frames exceeding ClipThreshold as n e , where e represents the part that exceeds the average value. Since we need to divide all the excess parts evenly into all gray levels, we need to calculate n e The average value is denoted as n em : Traverse all gray levels, set the number of gray level statistics of the i-th gray level to n i , the gray level after truncation and redistribution is n' i , divided into two cases for processing:

[0080] The first case: n i <ClipThreshold,then n' i =n i +n em ;

[0081] The second case: n i ≥ ClipThreshold, then n' i =n i ;

[0082] In the embodiment of the present invention, after processing the pixels in the 8x8 partition, a new histogram is obtained, and the cdf is calculated according to the new histogram.

[0083] Step 104, obtaining the cdf value of the pixel points in the partition by calculating the pdf of the pixel points in the partition;

[0084] In a preferred embodiment of the present invention, the cdf value of the pixel points in the partition is obtained by calculating the pdf of the pixel points in the partition, including:

[0085] Cumulative statistics are performed on the PDF until the number of pixel statistics in the interval is completed, forming a cdf mapping lookup table, and the cdf values ​​of the pixel points in the partition are obtained according to the cdf mapping lookup table.

[0086] Specifically, the PDF histogram is obtained by truncation, and cumulative statistics are performed to obtain the number of pixel statistics in a total of 256 intervals from [0,0], [0,1], [0,2] to [0,255], forming a CDF mapping lookup table with the horizontal axis being [0,255] and the vertical axis being the cumulative number of pixel statistics. i,j,k ,i,j∈[1,8],k∈[0,255].

[0087] Step 105, performing a spatial convolution smoothing filter operation on the cdf values ​​of the pixels in the partition, obtaining and storing the cdf value of the frame image, and extracting the cdf value of the previous frame;

[0088] In a specific application, a spatial convolution smoothing filtering operation is performed on the cdf values ​​of the pixels in the partition to obtain and store the cdf value of the frame image, and the cdf value of the previous frame is extracted, including: performing an expansion operation on the matrix corresponding to the cdf value to obtain an expansion matrix, using a 3x3 convolution kernel to obtain spatial filtering to obtain an updated cdf function, using weighted averaging to perform temporal filtering, smoothing the cdf function between frames, obtaining and storing the cdf value of the frame image, and extracting the cdf value of the previous frame.

[0089] The 8x8 cdf value performs a spatial convolution smoothing filter operation. The cdf is essentially a lookup table expressed as

[0090] V'=f(V) (7)

[0091] Where V is the grayscale value, and its value range is V∈[0,255]; V' is the grayscale value of the mapped pixel, and its value range is V'∈[0,255]. Therefore, the cdf in each sub-region has a total of 256 lookup table addresses.

[0092] Assume that the lookup table for the 8x8 region is a matrix, where f i,j,k Represents the cdf lookup table function of the corresponding area, where i,j∈[1,8] represents the row and column numbers of the partition, and k∈[0,255].

[0093]

[0094] The cdf matrix is ​​expanded to the following matrix, and a 3x3 convolution kernel is used to perform spatial filtering to obtain the updated cdf function f i, ' j,k , where i,j∈[1,8], k∈[0,255].

[0095]

[0096] In order to smooth the cdf function between frames, weighted averaging is used for temporal filtering, which operates as follows:

[0097]

[0098] in Represents the cdf function of the current frame, represents the cdf function of the previous frame, 0<λ<1. In the present invention, λ=0.25. Generally, λ≤0.5 is required to prevent the drastic change of cdf caused by the change of horizontal content. In addition, for the friendliness of hardware design, λ must meet the binary calculation requirements, such as

[0099] Step 106, performing bilinear interpolation based on the V channel data of the HSV model data and the cdf value of the previous frame to obtain interpolated V channel data;

[0100] In actual application in the embodiment of the present invention, bilinear interpolation is performed on the V channel data according to the HSV model data and the cdf value of the previous frame to obtain the interpolated V channel data, including: dividing each sub-region of the partition into four small equal parts, and each small equal part forms an interpolation relationship with the other three adjacent regions; obtaining the distance between a certain pixel point and the center of the distance region, and performing interpolation calculation based on the output value of the cdf mapping lookup table of the four regions corresponding to the distance and the brightness of the pixel point to obtain the interpolated V channel data.

[0101] Furthermore, the cdf lookup table process of the V'=f(V) pixel brightness needs to be bilinearly interpolated to further smooth the contrast equalization effect. Figure 7 It is part of the 8x8 sub-region map. Each square represents a sub-region. In order to calculate the weight of bilinear interpolation, each sub-region is divided into four small equal parts. Each small equal part forms an interpolation relationship with the other three adjacent regions. For example, when point p falls on the small equal part of the upper left corner of the 10th region, the interpolation of this point is associated with sub-regions 1, 2 and 9. Similarly, if point p falls on the upper right corner, the interpolation of this point is associated with sub-regions 2, 3 and 11, and so on.

[0102] The equalized brightness of pixel P is as follows:

[0103]

[0104] Where V' p represents the brightness of pixel P after bilinear interpolation, f1(V p ), f2(V p ), f9(V p ), f 10 (V p) represents the output value of the cdf lookup table of the four pass regions corresponding to the brightness of point P, and x, y, s, and r represent the distance of point P from the center of the region in pixels.

[0105] When the pixel P point falls in the upper left corner of the 1 area, the upper right corner of the 8 area, the lower left corner of the 57 area, and the lower right corner of the 64 area, no bilinear interpolation is required, and V' P =f n (V P ),n=1,8,57,64calculate and get the output result of cdf.

[0106] When pixel P falls on an edge area other than the four corner points above, only one adjacent area is interpolated instead of three adjacent areas. For example, when pixel P falls on the upper left corner of area 2, the interpolation operation is

[0107]

[0108] But when P falls in the lower left corner of the 9 region, the interpolation operation is:

[0109]

[0110] Step 107, identifying the face area of ​​the YCrCb model data, adjusting the nonlinear ratio of the S channel data of the face area, and obtaining adjusted S channel data;

[0111] In an embodiment of the present invention, the identifying of the face area of ​​the YCrCb model data, adjusting the nonlinear ratio of the S channel data of the face area, and obtaining the adjusted S channel data include: determining a center point, establishing an elliptical area around the central red dot, determining the elliptical area as a face area, obtaining pixel points falling within the face area, controlling the nonlinear ratio of the S channel data of the face area to increase, and controlling the global linear ratio of the S channel data outside the face area to increase, to obtain the adjusted S channel data.

[0112] In the technical solution of this application, the purpose of facial skin color detection is to keep the facial skin color from being oversaturated and distorted. According to empirical values, it is believed that Figure 8 The central red dot in the figure is the skin color of the human face. An elliptical area is established around the central red dot. The pixels falling into this area can be considered as the skin color of the human face. To enhance the saturation in the area, gradual proportional amplification is required, and the amplification factor is relatively reduced compared to the outside of the ellipse.

[0113] The following formula is used to determine whether the saturation of a pixel falls within the ellipse;

[0114]

[0115] Where l is the distance between the pixel saturation and the central red dot.

[0116] C rmax =173, C rmin =133,C bmax =127, C bmin =77,k=1.

[0117] Saturation enhancement generally enlarges the S channel of the pixel point. For the pixel points falling outside the elliptical area, the corresponding S is increased in a global linear proportion.

[0118] S'=S×k c ,k c >1 (15)

[0119] For the pixel points falling into the elliptical area, the corresponding S is nonlinearly increased.

[0120] S'=S×l,1<l≤k c (16)

[0121] Step 108, the original H channel data, the interpolated V channel data and the adjusted S channel data of the HSV model data are combined into new HSV model data, the new HSV model data is converted into new RGB image data, and the new RGB image data is output.

[0122] Specifically in the embodiment of the present invention, the original H channel data, the interpolated V channel data and the adjusted S channel data of the HSV model data are combined into new HSV model data, the new HSV model data is converted into new RGB image data, and the new RGB image data is output;

[0123] Among them, the HSV to RGB model conversion formula is:

[0124]

[0125]

[0126] t=v×(1-(1-f)×s) (20)

[0127]

[0128] In an embodiment of the present invention, an algorithm for image quality enhancement is provided, which realizes the functions of contrast enhancement and saturation improvement. The algorithm adopts the CLAHE equalization algorithm plus saturation improvement plus elliptical face detection, which constitutes a new function of image quality enhancement and can meet the demand for image quality improvement of the intelligent cockpit display of electric vehicles. The FPGA hardware architecture of the image quality enhancement algorithm of the present invention can receive videos of any resolution and frame rate specifications from the outside, is compatible with any relevant video interface, is compatible with any 1-way, 2-way and 4-way input videos, performs HSV and YUV conversion on the RGB model of the video image, performs CLAHE contrast equalization processing respectively, and performs saturation processing after face detection. In the video link of an ordinary display, an FPGA chip is embedded, and the FPGA communicates with the MCU in the module to realize parameter configuration, and the parameters of image quality enhancement contrast and saturation can be configured. FPGA is compatible with videos of any interface and resolution and frame rate specifications. Any video enhanced by FPGA can display the enhanced effect in the display, and the enhancement algorithm can also be shielded by MCU configuration parameters, and the display can display the original video effect. An image quality enhancement algorithm is deployed in the FPGA chip. After being processed by the algorithm, the contrast and saturation of the video image can be significantly improved, and the skin color of the face can be preserved, which can bring image quality improvement effects to the smart cockpit of electric vehicles and consumer televisions.

[0129] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0130] Reference Fig. 9 , shows a structural block diagram of an embodiment of an image quality enhancement device based on an FPGA chip according to an embodiment of the present invention, which may specifically include the following modules:

[0131] The image data acquisition module 301 is used to acquire a frame of RGB image data;

[0132] A conversion module 302, used to convert the RGB image data into HSV model data and YCrCb model data respectively;

[0133] A first calculation module 303 is used to calculate the pdf of the pixel points in the partition of the HSV model data;

[0134] A second calculation module 304 is used to calculate the cdf value of the pixel points in the partition by using the pdf of the pixel points in the partition;

[0135] The filtering module 305 is used to perform a spatial convolution smoothing filtering operation on the cdf values ​​of the pixels in the partition, obtain the cdf value of the frame image and store it, and extract the cdf value of the previous frame;

[0136] An interpolation module 306 is used to perform bilinear interpolation based on the V channel data of the HSV model data and the cdf value of the previous frame to obtain interpolated V channel data;

[0137] A non-linear ratio module 307, used for identifying a face region of the YCrCb model data, adjusting a non-linear ratio of the S channel data of the face region, and obtaining adjusted S channel data;

[0138] The composition module 308 is used to combine the original H channel data, the interpolated V channel data and the adjusted S channel data of the HSV model data into new HSV model data, convert the new HSV model data into new RGB image data, and output the new RGB image data.

[0139] Preferably, the first calculation module includes:

[0140] The first acquisition submodule is used to obtain the width and height data of each partition, and calculate the grayscale average value within the interval of each partition under the width and height data;

[0141] The calculation submodule is used to set a value higher than the grayscale average to truncate the histogram; calculate the histogram of the frame image on the grayscale, and evenly distribute all the excess parts to all the grayscales. After the grayscale is truncated and redistributed, a new PDF is obtained.

[0142] Preferably, the second calculation module includes:

[0143] The lookup table submodule is used to perform cumulative statistics on the PDF until the number of pixel statistics in the interval is completed, forming a cdf mapping lookup table, and obtaining the cdf value of the pixel point in the partition according to the cdf mapping lookup table.

[0144] Preferably, the filtering module comprises:

[0145] The filtering submodule is used to perform an expansion operation on the matrix corresponding to the cdf value to obtain an expanded matrix, use a 3x3 convolution kernel to perform spatial filtering to obtain an updated cdf function, use weighted average to perform temporal filtering, smooth the cdf function between frames, obtain the cdf value of the frame image and store it, and extract the cdf value of the previous frame.

[0146] Preferably, the interpolation module comprises:

[0147] The interpolation submodule is used to divide each sub-region of the partition into four small equal parts, and each small equal part forms an interpolation relationship with the other three adjacent regions; obtain the distance between a certain pixel point and the center of the distance region, and perform interpolation calculation based on the output value of the cdf mapping lookup table of the four regions corresponding to the distance and the brightness of the pixel point to obtain the interpolated V channel data.

[0148] Preferably, the nonlinear proportional module comprises:

[0149] The adjustment submodule is used to determine the center point, establish an elliptical area around the central red dot, determine the elliptical area as the face area, obtain the pixel points falling into the face area, control the nonlinear proportion of the S channel data of the face area to increase, control the global linear proportion of the S channel data outside the face area to increase, and obtain the adjusted S channel data.

[0150] The above-mentioned FPGA chip-based image quality enhancement device can be used to execute the FPGA chip-based image quality enhancement method provided in any of the above-mentioned embodiments, and has corresponding functions and beneficial effects.

[0151] In one embodiment, a computer device is provided. The computer device may be an automotive electronic instrument panel device. The internal structure diagram thereof may be as follows: Fig.10 As shown. The automotive electronic instrument screen device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for enhancing image quality based on an FPGA chip is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0152] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0153] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, Figures 1 to 8 The steps described.

[0154] In one embodiment, a computer readable storage medium is provided on which a computer program is stored. When the computer program is executed by a processor, Figures 1 to 8 The steps described

[0155] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0156] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0157] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0158] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0160] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0161] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0162] The above is a detailed introduction to an image quality enhancement method based on an FPGA chip, an image quality enhancement device based on an FPGA chip, a computer device and a storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for image quality enhancement based on FPGA chip, characterized in that: The method comprises: Get a frame of RGB image data; Convert the RGB image data into HSV model data and YCrCb model data respectively; Calculate the pdf of the pixel points in the partition of the HSV model data; Obtaining the cdf value of the pixel points in the partition by calculating the pdf of the pixel points in the partition; Performing a spatial convolution smoothing filter operation on the cdf values ​​of the pixels in the partition to obtain and store the cdf value of the frame image, and extracting the cdf value of the previous frame; Perform bilinear interpolation on the V channel data of the HSV model data and the cdf value of the previous frame to obtain interpolated V channel data; Identify a face area of ​​the YCrCb model data, and adjust a nonlinear ratio of the S channel data of the face area to obtain adjusted S channel data; The original H channel data, the interpolated V channel data and the adjusted S channel data of the HSV model data are combined into new HSV model data, the new HSV model data is converted into new RGB image data, and the new RGB image data is output.

2. The image quality enhancement method based on FPGA chip according to claim 1, characterized in that: The step of calculating the pdf of the pixels in the partition of the HSV model data comprises: Get the width and height data of each partition, and calculate the grayscale average value of each partition in the required interval under the width and height data; The histogram is truncated by setting a value higher than the grayscale average; the histogram of the frame image is calculated on the grayscale level, and all the excess parts are evenly distributed to all grayscale levels. After the grayscale is truncated and redistributed, a new PDF is obtained.

3. The image quality enhancement method based on FPGA chip according to claim 1, characterized in that: The step of calculating the cdf value of the pixel points in the partition by using the pdf of the pixel points in the partition includes: Cumulative statistics are performed on the PDF until the number of pixel statistics in the interval is completed, forming a cdf mapping lookup table, and the cdf values ​​of the pixel points in the partition are obtained according to the cdf mapping lookup table.

4. The image quality enhancement method based on FPGA chip according to claim 1, characterized in that: The cdf value of the pixel points in the partition is subjected to a spatial convolution smoothing filter operation to obtain and store the cdf value of the frame image, and the cdf value of the previous frame is extracted, including: The matrix corresponding to the cdf value is expanded to obtain the expanded matrix, a 3x3 convolution kernel is used to perform spatial filtering to obtain the updated cdf function, weighted average is used for temporal filtering to smooth the cdf function between frames, the cdf value of the frame image is obtained and stored, and the cdf value of the previous frame is extracted.

5. The image quality enhancement method based on FPGA chip according to claim 1, characterized in that: The V channel data according to the HSV model data and the cdf value of the previous frame are bilinearly interpolated to obtain the interpolated V channel data, including: Each sub-region of the partition is divided into four smaller equal parts, and each smaller equal part forms an interpolation relationship with the other three adjacent regions; The distance between a certain pixel point and the center of the distance area is obtained, and interpolation calculation is performed according to the output values ​​of the cdf mapping lookup table of the four areas corresponding to the distance and the brightness of the pixel point to obtain the interpolated V channel data.

6. The image quality enhancement method based on FPGA chip according to claim 1, characterized in that: The step of identifying a face region of the YCrCb model data and adjusting a nonlinear ratio of the S channel data of the face region to obtain adjusted S channel data includes: Determine the center point, establish an elliptical area around the central red dot, determine the elliptical area as the face area, obtain the pixel points falling into the face area, control the nonlinear proportion of the S channel data of the face area to increase, control the global linear proportion of the S channel data outside the face area to increase, and obtain the adjusted S channel data.

7. An image quality enhancement device based on FPGA chip, characterized in that: The device comprises: An image data acquisition module is used to acquire a frame of RGB image data; A conversion module, used for converting the RGB image data into HSV model data and YCrCb model data respectively; A first calculation module, used to calculate the pdf of the pixel points in the partition of the HSV model data; A second calculation module is used to calculate the cdf value of the pixel points in the partition by using the pdf of the pixel points in the partition; A filtering module, used for performing a spatial convolution smoothing filtering operation on the cdf values ​​of the pixels in the partition, obtaining and storing the cdf value of the frame image, and extracting the cdf value of the previous frame; An interpolation module, used for performing bilinear interpolation according to the V channel data of the HSV model data and the cdf value of the previous frame to obtain interpolated V channel data; A nonlinear ratio module, used for identifying the face area of ​​the YCrCb model data, adjusting the nonlinear ratio of the S channel data of the face area, and obtaining the adjusted S channel data; The composition module is used to combine the original H channel data, the interpolated V channel data and the adjusted S channel data of the HSV model data into new HSV model data, convert the new HSV model data into new RGB image data, and output the new RGB image data.

8. The image quality enhancement device based on FPGA chip according to claim 7, characterized in that: The first calculation module includes: The first acquisition submodule is used to obtain the width and height data of each partition, and calculate the grayscale average value within the interval of each partition under the width and height data; The calculation submodule is used to set a value higher than the grayscale average to truncate the histogram; calculate the histogram of the frame image on the grayscale, and evenly distribute all the excess parts to all the grayscales. After the grayscale is truncated and redistributed, a new PDF is obtained.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the image quality enhancement method based on an FPGA chip according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image quality enhancement method based on an FPGA chip according to any one of claims 1 to 6 are implemented.

Images