A method and system for specifying a high dynamic image histogram

Abstract

The present invention discloses a method for specifying a histogram of a high-dynamic image, which is mainly used for enhanced processing and display of ultra-high-dynamic images in real-time image processing systems. The histogram specification algorithm can effectively compress the dynamic range of high-dynamic images, and can adjust the processing effect of the algorithm according to actual application requirements. However, directly performing histogram specification processing on ultra-high-dynamic images occupies too much memory, and existing conventional hardware processing systems usually find it difficult to meet memory and real-time requirements. The present invention proposes an improved method for this problem, which uses a two-level histogram specification method to implement a histogram specification algorithm while significantly reducing memory usage, while meeting the requirements for real-time video processing. The method of the present invention has the characteristics of fast computing speed and small memory usage, and can realize real-time ultra-high-dynamic image processing based on low-cost FPGA devices or embedded processors.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a high dynamic image histogram designation method and system. Background Art

[0002] High dynamic range images have a wide range of application scenarios, such as image recording and display of driving recorders under strong light at night, processing and display of high dynamic welding images by industrial welding robots, etc. The brightness dynamic range of the image is very large, which can reach 20 bits. However, most display devices can only display 8-bit images at present, so special algorithms are needed to process such data above 20 bits so that it can retain the visual effect of the original scene in the human eye. The image enhancement algorithm specified based on the image histogram has a good enhancement effect for high dynamic range images, but for ultra-wide dynamic images of 20 bits or higher, the histogram is too large, and the calculation and storage amount are very large. At present, general processors or FPGA chips are difficult to meet the requirements. Conventional histogram equalization algorithms can also be used for high dynamic range image enhancement, but for ultra-high dynamic images, histogram equalization also has the problem of large calculation and storage amount, and the histogram equalization algorithm is difficult to achieve good results in special application scenarios because its parameters cannot be adjusted. Some other existing high dynamic image enhancement technologies are generally complex in calculation and difficult to be applied in real time on existing ordinary embedded platforms. Since the histogram specified algorithm can specify the target histogram, the processing effect of the algorithm is easy to control and the effect is ideal when dealing with special application scenarios. Summary of the invention

[0003] The present invention proposes a real-time enhancement processing method for high-dynamic video images, including a histogram specification method for high-dynamic images and videos. The method has the characteristics of small memory occupation, low computational complexity, and the ability to adjust parameters according to application needs, and is suitable for real-time processing of high-dynamic video image histogram specification.

[0004] The present invention first discloses a method for specifying a high dynamic image histogram, comprising the following steps:

[0005] S1: Specify the target histogram

[0006] S2: Take the high-order histogram H of the high-order K1-bit statistical image of the input image f0 ;

[0007] S3: According to the specified target histogram and high-order histogram H f0 Calculate the grayscale mapping table M0 from the high K1-bit input image grayscale value to the output image grayscale value;

[0008] S4: Use grayscale mapping table M0 to map the high-order histogram H f0 Perform grayscale transformation to obtain the transformed histogram H g0 ;

[0009] S5: Find the transformed histogram H g0 With target histogram The n grayscale intervals with the largest differences are sorted from largest to smallest according to the difference size as G i ,i=1,2,…n;

[0010] S6: According to each grayscale interval G of the output image i , i = 1, 2, ... n endpoints, and grayscale mapping table M0, to obtain the high-order histogram H corresponding to the input image f0 The grayscale interval endpoints of ;

[0011] S7: The grayscale intervals G found i , i=1,2,…n, the corresponding input image grayscale values ​​are linearly transformed to K2 bits, and then the histogram H of the input image is counted respectively. f1 ,H f2 ,…,H fn ;

[0012] S8: For each grayscale interval G i , i=1,2,…n, calculate the grayscale mapping table specified by the histogram from the K2-bit input image grayscale value to the output image grayscale value, and according to the transformation relationship between the K2-bit input image grayscale value and the original input image grayscale, obtain the grayscale mapping table M for each grayscale interval from the original input image grayscale value to the output image grayscale value i ,i=1,2,…n;

[0013] S9: Using grayscale mapping table M i , i=0,1,2,…n, transform the grayscale value of each pixel of the input image to obtain the output image.

[0014] The present invention also discloses a method for specifying a secondary histogram of a high dynamic video image, comprising the following steps:

[0015] S11: Specify target histogram

[0016] S12: Take the high-order histogram H of the high-order K1-bit statistical image for each frame of input image f0 ;

[0017] S13: According to the specified target histogram and high-order histogram H f0 Calculate the grayscale mapping table M0 from the high K1-bit input image grayscale value to the output image grayscale value;

[0018] S14: Use grayscale mapping table M0 to map histogram H f0 Perform grayscale transformation to obtain the transformed histogram H g0 ;

[0019] S15: Find the transformed histogram H g0 With target histogram The n grayscale intervals with the largest differences are sorted from largest to smallest according to the difference size as G i ,i=1,2,…n;

[0020] S16: According to each grayscale interval G of the output image i , i = 1, 2, ... n endpoints, and grayscale mapping table M0, calculate the high-order histogram H corresponding to the input image f0 The grayscale interval endpoints of ;

[0021] S17: Using the grayscale intervals G found in the previous frame image i , i=1,2,…n corresponding to the grayscale value of the input image, the grayscale value of the current image is linearly transformed to K2 bits, and then the histogram H of the current input image is counted respectively f1 ,H f2 ,…,H fn ;

[0022] S18: For each grayscale interval G i , i=1,2,…n, calculate the grayscale mapping table specified by the histogram from the current K2-bit input image grayscale value to the output image grayscale value, and according to the transformation relationship between the K2-bit input image grayscale value and the original input image grayscale, obtain the grayscale mapping table M for each grayscale interval from the original input image grayscale value to the output image grayscale value i ,i=1,2,…n;

[0023] S19: Using the grayscale mapping table M obtained from the previous frame image i , i=1,2,…n, and the grayscale mapping table M0 obtained from the previous frame image, transform the grayscale value of each pixel of the current input image to obtain the output image.

[0024] For the above-mentioned method for specifying the histogram of high dynamic images and videos, preferably, the transformed histogram H g0 With target histogram The method for finding the n grayscale intervals with the largest difference is: Calculate the transformed histogram H g0 The values ​​in and the corresponding target histogram The length of the interval that is continuously less than a specified threshold is taken as the difference of the grayscale interval.

[0025] When the input image is an RGB color image, convert the image to HSI or HSV color space, process only the I or V component, and then convert it back to RGB color image. If the input image is a color space without brightness component, convert it to HSV or HSI color space and take the brightness component (V or I) of the image. For example, the formula for converting the V component to HSV is as follows:

[0026] V=max(R,G,B)

[0027] The formula for transferring to the I component of HSI is as follows:

[0028] I=(R+G+B) / 3

[0029] Among them, R, G, and B represent the components of each pixel of the high-dynamic image in the red, green, and blue channels respectively.

[0030] The present invention also discloses a high dynamic video image histogram designation system, comprising the following modules:

[0031] An image acquisition module, used for acquiring video image data;

[0032] An image output module, used for outputting enhanced video images;

[0033] The high dynamic image histogram specification module is used to perform high dynamic image enhancement processing on the image, including the following sub-modules:

[0034] 1) High-order histogram statistics module, used to obtain the high-order histogram H of each frame input image at the K1-bit high-order statistical f0 ;

[0035] 2) Grayscale mapping table calculation and histogram transformation module, used to calculate the grayscale mapping table according to the specified target histogram. and high-order histogram H f0 Calculate the grayscale mapping table M0 from the high K1-bit input image grayscale value to the output image grayscale value, and use the grayscale mapping table M0 to transform the histogram H f0 Perform grayscale transformation to obtain the transformed histogram H g0 ;

[0036] 3) Secondary enhanced grayscale interval search module, used to find the transformed histogram H g0 With target histogram The n grayscale intervals with the largest differences are sorted from largest to smallest according to the difference size as G i ,i=1,2,…n;

[0037] 4) A grayscale interval determination module for an input image, used to determine the grayscale intervals G of an output image according to the grayscale intervals G of the output image. i, i = 1, 2, ... n endpoints, and grayscale mapping table M0, calculate the high-order histogram H corresponding to the input image f0 The grayscale interval endpoints of ;

[0038] 5) Linear grayscale transformation and secondary histogram statistics module, used to use the grayscale intervals G found in the previous frame image i , i=1,2,…n corresponding to the grayscale value of the input image, the grayscale value of the current image is linearly transformed to K2 bits, and then the histogram H of the current input image is counted respectively f1 ,H f2 ,…,H fn ;

[0039] 6) A secondary grayscale mapping table determination module is used to determine each grayscale interval G according to the secondary histogram. i , i=1,2,…n, calculate the grayscale mapping table specified by the secondary histogram from the current K2-bit input image grayscale value to the output image grayscale value, and according to the transformation relationship between the K2-bit input image grayscale value and the original input image grayscale, obtain the grayscale mapping table M for each grayscale interval from the original input image grayscale value to the output image grayscale value i ,i=1,2,…n;

[0040] 7) Grayscale transformation module, used to use the grayscale mapping table M obtained from the previous frame image i , i=1,2,…n, and the grayscale mapping table M0 obtained from the previous frame image, transform the grayscale value of each pixel of the current input image to obtain the output image.

[0041] Furthermore, the high dynamic video image histogram designation system further includes an image transmission module, which includes the following submodules:

[0042] 1) An image transmission module, used to convert the image data from the image acquisition module into an FPD-Link III or GMSL serial signal and transmit the data;

[0043] 2) Image receiving module, used to receive the serial signal of FPD-Link III or GMSL mode from the image sending module, and input the high dynamic image enhancement module after deserialization;

[0044] 3) Transmission line, using a coaxial cable or a twisted pair cable, for transmitting serialized image data from the image sending module to the image receiving module.

[0045] Furthermore, the high dynamic image histogram specifying module also includes a MIPI-CSI2 decoding module, which is used to convert the received MIPI CSI data into pixel-by-pixel data.

[0046] Furthermore, the high dynamic image histogram designation module also includes a color conversion module, which is used to convert the Bayer format image obtained by the sensor into the HSI or HSV format, and convert the processed image from the HSI or HSV format to the RGB format.

[0047] Furthermore, the high dynamic image histogram specifying module is implemented using FPGA or SoC FPGA chip.

[0048] The beneficial effect of the present invention is that the method and system of the present invention occupy little memory, have low computational complexity, and have good processing effect. Real-time enhanced processing of ultra-high dynamic images can be achieved using low-cost FPGA hardware devices or embedded processors. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flow chart of an embodiment of the method for specifying a high dynamic image histogram of the present invention.

[0050] Figure 2 is an example of a specified target histogram.

[0051] Figure 3 is an example of the actual effect of processing on a welding image.

[0052] Figure 4 It is a flow chart of an embodiment of a method for specifying a high dynamic video image histogram of the present invention.

[0053] Figure 5 It is a structural diagram of an embodiment of the high dynamic video image histogram specifying system of the present invention.

[0054] Figure 6 It is a structural diagram of an embodiment of a high dynamic image histogram specifying module of the present invention.

[0055] Figure 7 It is a structural diagram of an embodiment of the high dynamic video image processing system of the present invention including an image transmission module. DETAILED DESCRIPTION

[0056] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation examples. The following examples are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0057] For an input high dynamic image, a specific histogram specification method embodiment flow chart is as follows Figure 1 As shown, the specific steps are as follows:

[0058] S1: Specify the target histogram

[0059] First, according to the application requirements, such as the brightness distribution of the image, specify the target histogram that the processed image wants to have. For example, for the convenience of display, the grayscale value range of the target histogram is 0 to 255. For high-dynamic image applications, if you are concerned about a certain grayscale interval of the image, specify the pixel points with fewer distributions in the grayscale interval, so that the interval can be expanded. Figure 2 The following is an example of a target histogram specified for an arc welding application. The target histogram is defined by a segmented polyline, where the brighter interval has fewer pixels but requires attention, so fewer pixel distributions are specified to give it a wider dynamic range. In actual use, histograms of different shapes can be specified according to different actual application requirements.

[0060] S2: Take the high-order histogram H of the high-order K1-bit statistical image of the input image f0 ;

[0061] Take the high K1 bits of the binary number of the brightness component V of the input image. For example, for a 20-bit original input image, take K1 equal to 13 to obtain the high bit V of the brightness component of the input image. h :

[0062] V h =V>>(K max -K1)

[0063] Where K max The number of binary quantization bits corresponding to the maximum grayscale in the input high dynamic image, or the number of binary quantization bits of the input image, such as K max = 20. The present invention assumes that K max >K1.

[0064] Then according to the brightness component V h Count the high-order histogram H of the input image f0 The specific formula is as follows:

[0065]

[0066] Where M and N are the width and height of the image respectively, and the delta function is defined as:

[0067]

[0068] When the input image is an RGB color image, the image is converted to HSI or HSV color space, and only the I or V component is processed. After the histogram is specified in the last step S9, H and S are used to convert it back to RGB or other formats.

[0069] S3: According to the specified target histogram and high-order histogram H f0Calculate the grayscale mapping table M0 from the high K1-bit input image grayscale value to the output image grayscale value;

[0070] First, calculate the high-order histogram H f0 and target histogram The cumulative histogram (i.e., cumulative probability) S f0 and S g0 , as shown in the following formula:

[0071]

[0072]

[0073] Where T is the total number of pixels in the image. Generally, if the target histogram is in the form of an analytical function, then the cumulative probability function can be obtained by integrating it, and then the cumulative histogram S can be obtained by taking the value of the cumulative probability function and normalizing it. g0 .

[0074] In the above formula, S f0 (l) and S g0 (k) Neither of them can be divided by T, and the result will remain unchanged.

[0075] Then for each k corresponding to S g0 (k), find the cumulative histogram S f0 and S g0 Difference|S f0 (l)-S g0 (k)|The smallest l, making the high-order histogram H f0 The gray level in is mapped to the transformed new gray level. During the mapping, the mapping relationship M0 from the histogram to the output result is recorded.

[0076] S4: Use grayscale mapping table M0 to map the histogram H f0 Perform grayscale transformation to obtain the transformed histogram H g0 ;

[0077] Use the grayscale mapping table M0 obtained in the previous step to transform the input histogram H f0 Transform and obtain the transformed histogram H g0 , some of the input grayscale values ​​are transformed to the same output grayscale value, and they are accumulated together. This step is called the first-level histogram specification.

[0078] S5: Find the transformed histogram H g0 With target histogram The n grayscale intervals with the largest differences are sorted from largest to smallest according to the difference size as G i ,i=1,2,…n;

[0079] According to the specified histogram and target histogram The difference size is used to find the grayscale interval. Since the histogram specifies that there is a grayscale merge, but the grayscale of the input image will not be split, resulting in a zero-value interval in the output histogram, the length of the zero-value interval can be used as the length of the transformed histogram H. g0 With target histogram The difference between , that is, find the n grayscale intervals with the longest continuous zero value in the output histogram to make the secondary histogram: G i ,i=1,2,...,n. That is, find a preset number of n zero-value intervals starting from the maximum length.

[0080] When searching for grayscale intervals, the intervals can also be determined based on the grayscale mapping table M0. The discontinuity of the mapped output value of M0 is the cause of the zero-value interval. Therefore, the increment of the discontinuous value minus 1 is the length of the zero-value interval on the output histogram.

[0081] In addition, because the grayscale of the original image is relatively concentrated, or the grayscale is relatively concentrated and is merged together when the primary histogram is specified, some grayscale levels of the specified histogram are very different from the corresponding grayscale levels in the target histogram. This part of the interval can be found to specify the secondary histogram. In this embodiment, the histogram H after the grayscale level j is transformed is calculated. g0 The value H g0 (j) and the corresponding target histogram The value in Ratio is their difference, and the length of the interval that is continuously less than a specified threshold is taken as the difference of the interval, then the interval corresponding to the n values ​​with the largest difference can be found: G i ,i=1,2,...,n, which is the transformed histogram H g0 With target histogram The n grayscale intervals with the largest difference.

[0082] The interval found can be remembered as:

[0083] G i :[G il ,G ir ],i=1,2,...,n

[0084] Among them G il , G ir Respectively represent the left and right endpoints of the interval on the histogram after the first-level histogram specifies the transformation, G ir Is a non-zero value.

[0085] S6: According to each grayscale interval G of the output image i , i = 1, 2, ... n endpoints, and grayscale mapping table M0, to obtain the high-order histogram H corresponding to the input image f0The grayscale interval endpoints of ;

[0086] Suppose some interval [G il ,G ir ], its G il and G ir Corresponding input histogram H f0 The minimum and maximum grayscale values ​​are V i,min 、V i,max Then we can first determine the interval endpoint G according to the grayscale mapping table M0 il and G ir The value in the high-order histogram of the corresponding input image. Since only the high K1 bits of the image are input when the first-level histogram is specified, that is, the high-order histogram H f0 It is to shift the gray value of the input image to the right (K max -K1) bit after the K1 bit is obtained by statistics, so it corresponds to H f0 By shifting the index to the left by the corresponding number of bits, the grayscale value range of the input image corresponding to the grayscale interval endpoint can be determined [V i,min ,V i,max ]:

[0087] V i,min =G il <<(K max -K1)

[0088] V i,max =((G ir +1)<<(K max -K1))-1

[0089] S7: The grayscale intervals G found i , i=1,2,…n, the corresponding input image grayscale values ​​are linearly transformed to K2 bits, and then the histogram H of the input image is counted respectively. f1 ,H f2 ,…,H fn ;

[0090] That is, to make a linear transformation on each grayscale interval of the original image so that the i-th interval [V i,min ,V i,max ] maps to K2 is the number of binary bits corresponding to the pixel value range after the linear transformation used to count the histogram, which determines the size of the histogram. This histogram is called a secondary histogram, and K2 is generally taken to be the same as K1. The specific linear transformation method is:

[0091]

[0092] Where V is [V i,min ,V i,max] interval. Then, the histogram H of y after grayscale transformation of each grayscale interval of the original image is counted respectively. f1 ,H f2 ,…,H fn .if Then the linear transformation of the above formula can be changed to:

[0093] y=VV i,min

[0094] That is, no grayscale compression is required.

[0095] Then, the histogram H of each grayscale interval is calculated for each interval after the transformation of y fi

[0096] S8: For each grayscale interval G i , i=1,2,…n, calculate the grayscale mapping table specified by the histogram from the K2-bit input image grayscale value to the output image grayscale value, and according to the transformation relationship between the K2-bit input image grayscale value and the original input image grayscale, obtain the grayscale mapping table M for each grayscale interval from the original input image grayscale value to the output image grayscale value i ,i=1,2,…n;

[0097] First, according to the histogram H of each grayscale interval obtained in the previous step fi , calculate the cumulative histogram S of each grayscale interval respectively fi , i=1,2,…n:

[0098]

[0099] Where T i is the total number of pixels in each grayscale interval.

[0100] Then calculate the cumulative histogram S of the target histogram corresponding to each grayscale interval gi :

[0101]

[0102] Where T i is the total number of pixels in each interval, G il and G ir They are the left and right endpoints of the interval on the transformed histogram, that is, the grayscale of the output image.

[0103] Above S fi (l) and S gi The formula for (k) can be written as i , does not affect the results.

[0104] Then for each k corresponding to S gi (k), find the cumulative histogram Sfi and S gi Difference|S fi (l)-S gi (k)|The smallest l, so that the histogram H of each grayscale interval fi The gray level in is mapped to the new gray level after transformation. When mapping, record the mapping relationship M from the histogram to the output result i , i=1,2,…n. This is the second-level histogram specification.

[0105] S9: Using grayscale mapping table M i , i=0,1,2,…n, transform the grayscale value of each pixel of the input image to obtain the output image.

[0106] When inputting a pixel, first determine whether its grayscale value belongs to the interval i=1,2,…n according to the grayscale interval of the input image corresponding to each recorded grayscale interval, and determine its corresponding grayscale mapping table. After determining that the pixel grayscale value belongs to a certain interval, the pixel grayscale value needs to be normalized to [V i,min ,V i,max ], the specific normalization method is shown in step S7. If the input pixel grayscale value does not belong to the interval i=1,2,…n, the input pixel grayscale value is taken as the high K1 bit and corresponds to the grayscale mapping table M0. Then, the obtained grayscale mapping table M i , i=0,1,2,…n, transform the grayscale value of each pixel of the input image to obtain the output image.

[0107] When the input image is an RGB color image, since the above process only processes the I or V component after conversion to the HSI or HSV color space, it can be converted back to an RGB color image by using H and S, and can also be converted into color images of other formats.

[0108] In this embodiment, if a conventional histogram designation algorithm is used to process a 20-bit ultra-high dynamic image, the histogram size is 1M units, and the memory required is 8MB, which not only occupies a large amount of memory, but also takes a long time to clear the histogram, making it difficult to achieve real-time video processing; if the method of the present invention is used, if the number of intervals is set to 8, K1 is set to 13, and K2 is set to 13, then the memory occupied is 360KB. Therefore, the method can realize real-time ultra-high dynamic image enhancement processing on a low-cost FPGA or embedded processor.

[0109] The processing effect of this embodiment on a welding image is as follows: Figure 3 As shown, the left side is the output result of the image when only the high bit is taken, the dark part of the image becomes completely black, and the details of the bright part are also lost; the right side is the processing effect of the present invention, the dark content of the image is completely preserved, and the details of the bright part are richer, achieving the purpose of high dynamic image enhanced display.

[0110] Embodiment 2:

[0111] For input high-dynamic video, the method shown in Example 1 can be used to process each frame of the image. However, the entire image needs to be traversed in the three stages of statistical high-order histogram, statistical secondary histogram of each grayscale interval, and histogram designation processing of the image. Not only does it need to store the entire image data, but the processing time is long, and real-time processing requires high processor performance. The present invention proposes a pipeline processing method for performing secondary histogram designation on video images. The high-order histogram of the previous frame image and the secondary histogram of the previous frame image are used to perform histogram designation processing on the current image. As the image is input, the high-order histogram, secondary histogram statistics, and histogram designation processing of the current image are completed simultaneously. There is no need to store image data. It is particularly suitable for FPGA or embedded processors to realize real-time processing, and has the characteristics of fast processing speed and very small delay. A specific implementation flow chart of secondary histogram designation of high-dynamic video is shown as follows. Figure 4 As shown, the specific process is as follows:

[0112] S11: Specify target histogram

[0113] For high dynamic image video processing, each frame image uses the same target histogram. The specific method of specifying the target histogram is the same as step S1. When the system is initialized, the target histogram can be calculated and input into the cache first.

[0114] S12: Take the high-order histogram H of the high-order K1-bit statistical image for each frame of input image f0 ;

[0115] High dynamic video inputs image data into the system frame by frame. Since it is difficult to store a frame of image data and process it directly in the hardware processing system, the video data can be processed in a pipeline by taking advantage of the similar content of adjacent frames of the video, and the result information calculated in the previous frame of the image is used for the processing of the next frame. First, the high K1-bit histogram H is counted for the first frame of the input system. f0 , the specific method is the same as S2.

[0116] It should be noted that before the histogram is counted, the memory used for the histogram should be cleared. For video, this can be done during the blanking period of the video. Because this is well known to everyone, it will not be explained hereafter.

[0117] S13: According to the specified target histogram and high-order histogram H f0 The histogram from the high K1-bit input image grayscale value to the output image grayscale value specifies the grayscale mapping table M0, and the specific method is the same as S3.

[0118] For high-dynamic image video processing, since the cumulative distribution of the target histogram and the high-order histogram of the input image are both monotonically increasing functions when performing the histogram calculation, only one cycle is required to complete the grayscale transformation during the frame blanking period. This step can be calculated during the frame blanking period of the first frame image.

[0119] S14: Use grayscale mapping table M0 to map histogram H f0 Perform grayscale transformation to obtain the transformed histogram H g0 ;

[0120] Use the grayscale mapping table M0 obtained in the previous step to transform the input histogram H f0 Transform and obtain the transformed histogram H g0 , some of the input grayscale values ​​are transformed to the same output grayscale value, and they are accumulated together. This step is called the first-level histogram specification.

[0121] S15: Find the transformed histogram H g0 With target histogram The n grayscale intervals with the largest differences are sorted from largest to smallest according to the difference size as G i ,i=1,2,…n; the specific method is the same as S5.

[0122] S16: According to each grayscale interval G of the output image i , i = 1, 2, ... n endpoints, and grayscale mapping table M0, calculate the high-order histogram H corresponding to the input image f0 The grayscale interval endpoints of ;

[0123] The specific calculation method is the same as S6. This step is also completed in the frame blanking period before the next frame image arrives.

[0124] S17: Using the grayscale intervals G found in the previous frame image i , i=1,2,…n corresponding to the grayscale value of the input image, the grayscale value of the current image is linearly transformed to K2 bits, and then the histogram H of the current input image is counted respectively f1 ,H f2 ,…,H fn ;

[0125] The method is basically the same as step S7, except that the [V i,min ,V i,max ] is from the previous frame image. In addition, step S12 is processed simultaneously with this step, i.e., the high-order histogram of the current input image is counted simultaneously.

[0126] S18: For each grayscale interval G i, i=1,2,…n, calculate the grayscale mapping table specified by the histogram from the current K2-bit input image grayscale value to the output image grayscale value, and according to the transformation relationship between the K2-bit input image grayscale value and the original input image grayscale, obtain the grayscale mapping table M for each grayscale interval from the original input image grayscale value to the output image grayscale value i ,i=1,2,…n;

[0127] This step is the same as step S8.

[0128] S19: Using the grayscale mapping table M obtained from the previous frame image i , i=1,2,…n, and the grayscale mapping table M0 obtained from the previous frame image, transform the grayscale value of each pixel of the current input image to obtain the output image.

[0129] The method of this step is the same as step S9, except that the grayscale mapping table comes from the statistical results of the previous frame and the previous frame image respectively.

[0130] After the above steps are processed in pipeline, the histogram specification result of the input high dynamic image is output, and the delay is only a few pixel clock cycles, which has good real-time performance.

[0131] Embodiment three:

[0132] The embodiment of the present invention provides Figure 5 The high dynamic video image histogram designation system shown includes: an image acquisition module 101, used to acquire video image data; an image output module 105, used to output the enhanced video image; and a high dynamic image histogram designation module 104, used to perform high dynamic image enhancement processing on the image according to the method described in Example 2.

[0133] The high dynamic image histogram specification module is as follows Figure 6 As shown, it includes the following sub-modules:

[0134] 1) A high-order histogram statistics module 201 is used to obtain a high-order histogram H of the high K1 bits for each frame of input image. f0 ;

[0135] 2) Grayscale mapping table calculation and histogram conversion module 202, used to calculate the grayscale mapping table and histogram according to the specified target histogram and high-order histogram H f0 Calculate the grayscale mapping table M0 from the high K1-bit input image grayscale value to the output image grayscale value, and use the grayscale mapping table M0 to transform the histogram H f0 Perform grayscale transformation to obtain the transformed histogram H g0 ;

[0136] 3) Secondary enhanced grayscale interval search module 203, used to find the transformed histogram H g0 With target histogram The n grayscale intervals with the largest differences are sorted from largest to smallest according to the difference size as G i ,i=1,2,…n;

[0137] 4) A grayscale interval determination module 204 for an input image is used to determine the grayscale intervals G of the output image according to the grayscale intervals G of the output image. i , i = 1, 2, ... n endpoints, and grayscale mapping table M0, calculate the high-order histogram H corresponding to the input image f0 The grayscale interval endpoints of ;

[0138] 5) Linear grayscale transformation and secondary histogram statistics module 205, used to use the grayscale intervals G found in the previous frame image i , i=1,2,…n corresponding to the grayscale value of the input image, the grayscale value of the current image is linearly transformed to K2 bits, and then the histogram H of the current input image is counted respectively f1 ,H f2 ,…,H fn ;

[0139] 6) A secondary grayscale mapping table determination module 206 is used to determine each grayscale interval G according to the secondary histogram. i , i=1,2,…n, calculate the grayscale mapping table specified by the secondary histogram from the current K2-bit input image grayscale value to the output image grayscale value, and according to the transformation relationship between the K2-bit input image grayscale value and the original input image grayscale, obtain the grayscale mapping table M for each grayscale interval from the original input image grayscale value to the output image grayscale value i ,i=1,2,…n;

[0140] 7) Grayscale transformation module 207, used to use the grayscale mapping table M obtained from the previous frame image i , i=1,2,…n, and the grayscale mapping table M0 obtained from the previous frame image, transform the grayscale value of each pixel of the current input image to obtain the output image.

[0141] Above Figure 5 The modules can be integrated together and installed in one housing, such as a high dynamic image camera. However, in some occasions, such as welding robot applications, it is hoped that the image acquisition and processing modules will be divided into two parts, and image transmission is required between them. Figure 7 The figure shows a working form including an image transmission module, wherein the image transmission module is shown in a dotted box. The image transmission module includes the following submodules:

[0142] 1) An image sending module 102, used to convert image data from the image acquisition module into an FPD-Link III or GMSL serial signal and send the data;

[0143] 2) An image receiving module 103 is used to receive a serial signal in FPD-Link III or GMSL mode from an image sending module, and input the serial signal into a high dynamic image histogram designation module after deserialization;

[0144] 3) Transmission line 106, which is a coaxial line or a twisted pair line, is used to transmit the serialized image data from the image sending module to the image receiving module.

[0145] The image sensor used by the image acquisition module 101 mostly uses the MIPI-CSI2 interface to output image data, that is, the image acquisition module 101 outputs video image data in the MIPI-CSI2 format. If the system includes an image transmission module, the image receiving module outputs video image data in the MIPI-CSI2 format accordingly. The MIPI-CSI2 signal is a differential serial signal. At this time, the high dynamic image enhancement module also includes a MIPI-CSI2 decoding module for converting the received MIPI-CSI2 data into pixel-by-pixel parallel data. And in the MIPI protocol, pixels higher than 8 bits are spliced ​​into one pixel through multiple bytes. The embodiment of the present invention uses a SoC FPGA chip to perform serial-to-parallel, byte-to-pixel and Bayer-to-RGB processing on the data to convert the original image data into RGB image data.

[0146] In this embodiment, the image acquisition module outputs data in MIPI-CSI2 format, the image sending module 102 serializes the data in MIPI-CSI2 format, and the image receiving module 103 deserializes the FPD-Link III or GMSL transmitted on the transmission line 106 into data in MIPI-CSI2 format.

[0147] If the image sensor outputs monochrome high dynamic image data, that is, the output is grayscale data, the high dynamic image histogram specifying module can directly process according to the method described in Example 2. However, if the image sensor outputs color high dynamic image data, which is generally in Bayer format, the high dynamic image histogram specifying module also includes a color conversion module for converting the Bayer format image obtained by the sensor into HSI or HSV format, and converting the processed image from HSI or HSV format to RGB format.

[0148] This embodiment uses FPGA to implement the high dynamic image histogram designation module. Although the high dynamic image histogram designation algorithm of the present invention can also be implemented based on a high-performance processor, a simple FPGA and SoC FPGA chip are the preferred solutions for implementing the high dynamic image histogram designation of the method of the present invention. The SoC FPGA chip integrates the CPU and FPGA, which is convenient for control and image output, and is the best implementation solution of the present invention.

[0149] The image output module 105 is used to output the processed results. This embodiment includes HDMI output and RJ45 network port output. The HDMI output can directly output the image to the display for display, and the network port output can transfer the video image data to the PC for real-time display and storage.

[0150] The MIPI interface has the problems of short transmission distance and complex lines, which is not conducive to the practical application of the system. Figure 7 The image transmission module shown in the figure uses a serial deserialization chip (DS90UB835 / DS90UB936) to package the image data into FPD-Link III signals and transmit the image data through a coaxial cable. The system can transmit up to 10 meters and only requires one coaxial cable. The power supply can also be achieved through the cable, and the circuit is very simple. The driver of the image serial deserialization chip can be developed using the I2C controller integrated in the SoC FPGA chip, and the CPU sends instructions to control the functions of the serial deserialization chip.

Claims

1. A method for specifying a high dynamic image histogram, characterized in that: The following steps are involved: S1: Specify the target histogram S2: Take the high-order histogram H of the high-order K1-bit statistical image of the input image f0 ; S3: According to the specified target histogram and high-order histogram H f0 Calculate the grayscale mapping table M0 from the high K1-bit input image grayscale value to the output image grayscale value; S4: Use grayscale mapping table M0 to map the high-order histogram H f0 Perform grayscale transformation to obtain the transformed histogram H g0 ; S5: Find the transformed histogram H g0 With target histogram The n grayscale intervals with the largest differences are sorted from largest to smallest according to the difference size as G i ,i=1,2,…n; S6: According to each grayscale interval G of the output image i , i = 1, 2, ... n endpoints, and grayscale mapping table M0, to obtain the high-order histogram H corresponding to the input image f0 The grayscale interval endpoints of ; S7: The grayscale intervals G found i , i=1,2,…n, the corresponding input image grayscale values ​​are linearly transformed to K2 bits, and then the histogram H of the input image is counted respectively f1 ,H f2 ,…,H fn ; S8: For each grayscale interval G i , i=1,2,…n, calculate the grayscale mapping table specified by the histogram from the K2-bit input image grayscale value to the output image grayscale value, and according to the transformation relationship between the K2-bit input image grayscale value and the original input image grayscale, obtain the grayscale mapping table M for each grayscale interval from the original input image grayscale value to the output image grayscale value i ,i=1,2,…n; S9: Using grayscale mapping table M i , i=0,1,2,…n, transform the grayscale value of each pixel of the input image to obtain the output image.

2. A method for specifying a secondary histogram of a high dynamic video, characterized in that The steps include: S11: Specify target histogram S12: Take the high-order histogram H of the high-order K1-bit statistical image for each frame of input image f0 ; S13: According to the specified target histogram and high-order histogram H f0 Calculate the grayscale mapping table M0 from the high K1-bit input image grayscale value to the output image grayscale value; S14: Use grayscale mapping table M0 to map histogram H f0 Perform grayscale transformation to obtain the transformed histogram H g0 ; S15: Find the transformed histogram H g0 With target histogram The n grayscale intervals with the largest differences are sorted from largest to smallest according to the difference size as G i ,i=1,2,…n; S16: According to each grayscale interval G of the output image i , i = 1, 2, ... n endpoints, and grayscale mapping table M0, calculate the high-order histogram H corresponding to the input image f0 The grayscale interval endpoints of ; S17: Using the grayscale intervals G found in the previous frame image i , i=1,2,…n corresponding to the grayscale value of the input image, the grayscale value of the current image is linearly transformed to K2 bits, and then the histogram H of the current input image is counted respectively f1 ,H f2 ,…,H fn ; S18: For each grayscale interval G i , i=1,2,…n, calculate the grayscale mapping table specified by the histogram from the current K2-bit input image grayscale value to the output image grayscale value, and according to the transformation relationship between the K2-bit input image grayscale value and the original input image grayscale, obtain the grayscale mapping table M for each grayscale interval from the original input image grayscale value to the output image grayscale value i ,i=1,2,…n; S19: Using the grayscale mapping table M obtained from the previous frame image i , i=1,2,…n, and the grayscale mapping table M0 obtained from the previous frame image, transform the grayscale value of each pixel of the current input image to obtain the output image.

3. The method for specifying a high dynamic image or video histogram according to claim 1 or 2, characterized in that: The transformed histogram H g0 With target histogram The method for finding the n grayscale intervals with the largest difference is: Calculate the transformed histogram H g0 The values ​​in and the corresponding target histogram The length of the interval that is continuously less than a specified threshold is taken as the difference of the grayscale interval.

4. The method for specifying a high dynamic image or video histogram according to claim 1 or 2, characterized in that: When the input image is an RGB color image, convert the image to HSI or HSV color space and only process the I or V component.

5. A high dynamic video image histogram specification system, characterized in that: Includes the following modules: An image acquisition module, used for acquiring video image data; An image output module, used for outputting enhanced video images; The high dynamic image histogram specification module is used to perform high dynamic image enhancement processing on the image, including the following sub-modules: 1) High-order histogram statistics module, used to obtain the high-order histogram H of each frame input image at the K1-bit high-order statistical f0 ; 2) Grayscale mapping table calculation and histogram transformation module, used to calculate the grayscale mapping table according to the specified target histogram. and high-order histogram H f0 Calculate the grayscale mapping table M0 from the high K1-bit input image grayscale value to the output image grayscale value, and use the grayscale mapping table M0 to transform the histogram H f0 Perform grayscale transformation to obtain the transformed histogram H g0 ; 3) Secondary enhanced grayscale interval search module, used to find the transformed histogram H g0 With target histogram The n grayscale intervals with the largest differences are sorted from largest to smallest according to the difference size as G i ,i=1,2,…n; 4) A grayscale interval determination module for an input image, used to determine the grayscale intervals G of an output image according to the grayscale intervals G of the output image. i , i = 1, 2, ... n endpoints, and grayscale mapping table M0, calculate the high-order histogram H corresponding to the input image f0 The grayscale interval endpoints of ; 5) Linear grayscale transformation and secondary histogram statistics module, used to use the grayscale intervals G found in the previous frame image i , i=1,2,…n corresponding to the grayscale value of the input image, the grayscale value of the current image is linearly transformed to K2 bits, and then the histogram H of the current input image is counted respectively f1 ,H f2 ,…,H fn ; 6) A secondary grayscale mapping table determination module is used to determine each grayscale interval G according to the secondary histogram. i , i=1,2,…n, calculate the grayscale mapping table specified by the secondary histogram from the current K2-bit input image grayscale value to the output image grayscale value, and according to the transformation relationship between the K2-bit input image grayscale value and the original input image grayscale, obtain the grayscale mapping table M for each grayscale interval from the original input image grayscale value to the output image grayscale value i ,i=1,2,…n; 7) Grayscale transformation module, used to use the grayscale mapping table M obtained from the previous frame image i , i=1,2,…n, and the grayscale mapping table M0 obtained from the previous frame image, transform the grayscale value of each pixel of the current input image to obtain the output image.

6. The high dynamic video image histogram specification system according to claim 5, characterized in that It also includes an image transmission module, which includes the following submodules: 1) Image sending module, used to convert the image data from the image acquisition module into FPD-LinkIII or GMSL serial signals and send the data; 2) Image receiving module, used to receive the serial signal of FPD-Link III or GMSL mode from the image sending module, and input the high dynamic image enhancement module after deserialization; 3) Transmission line, using a coaxial cable or a twisted pair cable, for transmitting serialized image data from the image sending module to the image receiving module.

7. The high dynamic video image histogram designation system according to claim 5, characterized in that The high dynamic image histogram specifying module also includes a MIPI-CSI2 decoding module, which is used to convert the received MIPI-CSI2 data into pixel-by-pixel data.

8. The high dynamic video image histogram designation system according to claim 5, characterized in that The high dynamic image histogram designation module also includes a color conversion module, which is used to convert the Bayer format image obtained by the sensor into the HSI or HSV format, and convert the processed image from the HSI or HSV format to the RGB format.

9. The high dynamic image histogram designated enhancement system according to claim 5, characterized in that The high dynamic image histogram designation module is implemented using an FPGA chip.

Images