Video processing method and device based on FPGA platform

Abstract

The application discloses a video processing method and device based on an FPGA platform. The method comprises the following steps: dividing each image into m*n block partitions, and obtaining a plurality of luminance components corresponding to each block partition; performing histogram statistics on the plurality of luminance components corresponding to each block partition in the i-th row to obtain a plurality of luminance histograms, storing the luminance histograms in a first memory, correcting the luminance histograms, calculating cumulative histograms, storing the cumulative histograms in a third memory, and clearing the first memory; when calculating the cumulative histograms corresponding to each block partition in the i-th row, performing histogram statistics on the plurality of luminance components corresponding to each block partition in the i+1-th row to obtain a plurality of luminance histograms, storing the luminance histograms in a second memory, correcting the luminance histograms, calculating cumulative histograms, storing the cumulative histograms in a fourth memory, and clearing the second memory; when a camera video stream enters the i+2-th row, time-division multiplexing the first memory and clearing the third memory; and when entering the i+3-th row, time-division multiplexing the second memory and clearing the fourth memory.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a video processing method and device based on an FPGA platform. BACKGROUND

[0002] Image signal processing as the core function of a camera is a key indicator for evaluating the pros and cons of a camera and has a large number of application scenarios in actual applications. In recent years, although the image signal processor (ISP) has greatly improved in terms of computing speed and imaging effect, it still cannot realize real-time processing of high-resolution area arrays and high-gray-scale images in actual application scenarios. With the ISP requiring more and more hardware resources and computing efficiency, how to better solve the real-time processing of the ISP has become an important problem that enterprises urgently need to solve.

[0003] The traditional video processing method based on the field-programmable gate array (FPGA) platform has a small number of area array partitions, but needs to use a dozen random access memories (RAMs) to respectively statistically store each partition, and there is a frame delay between the histogram statistical parameters and the video stream data, which leads to resource waste, increases code redundancy, and cannot realize detailed image contrast stretching.

[0004] SUMMARY

[0005] The purpose of the present application is to solve the technical problem of resource waste and code redundancy of the traditional video processing method based on the FPGA platform. To achieve the above purpose, the present application provides a video processing method and device based on an FPGA platform.

[0006] The present application provides a video processing method based on an FPGA platform, comprising:

[0007] obtaining a camera video stream, dividing each image in the camera video stream into m*n block partitions, and obtaining a plurality of luminance components corresponding to each block partition according to the m*n block partition data of each image, m and n being positive integers greater than 4;

[0008] performing histogram statistics on the plurality of luminance components corresponding to each block partition in the i-th row to obtain a plurality of luminance histograms corresponding to each block partition in the i-th row, and storing the plurality of luminance histograms corresponding to each block partition in the i-th row in a first memory;

[0009] a first contrast threshold is calculated according to the maximum and minimum values in the plurality of luminance histograms corresponding to each block partition in the i-th row, each of the luminance histograms is modified according to the first contrast threshold, a plurality of modified luminance histograms corresponding to each block partition in the i-th row is obtained, and an accumulated histogram corresponding to each block partition in the i-th row is calculated according to the plurality of modified luminance histograms, and the accumulated histogram corresponding to each block partition in the i-th row is stored in a third memory, and the stored data in the first memory is cleared;

[0010] when the camera video stream enters the i-th row, the plurality of luminance components corresponding to each block partition in the i-th row are histogram counted, a plurality of luminance histograms corresponding to each block partition in the i-th row is obtained, and the plurality of luminance histograms corresponding to each block partition in the i-th row is stored in the first memory;

[0011] a second contrast threshold is calculated according to the maximum and minimum values in the plurality of luminance histograms corresponding to each block partition in the i-th row, each of the luminance histograms is modified according to the first contrast threshold, a plurality of modified luminance histograms corresponding to each block partition in the i-th row is obtained, and an accumulated histogram corresponding to each block partition in the i-th row is calculated according to the plurality of modified luminance histograms, and the accumulated histogram corresponding to each block partition in the i-th row is stored in a third memory, and the stored data in the first memory is cleared;

[0012] when the camera video stream enters the i-th row, the plurality of luminance components corresponding to each block partition in the i-th row are histogram counted, a plurality of luminance histograms corresponding to each block partition in the i-th row is obtained, and the plurality of luminance histograms corresponding to each block partition in the i-th row is stored in the first memory;

[0013] when the camera video stream enters the i-th row, the plurality of luminance components corresponding to each block partition in the i-th row are histogram counted, a plurality of luminance histograms corresponding to each block partition in the i-th row is obtained, and the plurality of luminance histograms corresponding to each block partition in the i-th row is stored in the first memory;

[0014] the data processing process of each block partition in the i-th row, the data processing process of each block partition in the i+1-th row, the data processing process of each block partition in the i+2-th row, and the data processing process of each block partition in the i+3-th row are repeated, and the m×n block partitions are processed in real time.

[0015] wherein i is an integer.

[0016] The application provides a video processing device based on an FPGA platform, comprising:

[0017] a data acquisition module, configured to acquire a camera video stream, divide each image in the camera video stream into m*n block partitions, and obtain a plurality of luminance components corresponding to each block partition according to m*n block partition data of each image, wherein m and n are positive integers greater than 4;

[0018] a first row luminance histogram processing module, configured to perform histogram statistics on the plurality of luminance components corresponding to each block partition in the i-th row, obtain a plurality of luminance histograms corresponding to each block partition in the i-th row, and store the plurality of luminance histograms corresponding to each block partition in the i-th row in a first memory;

[0019] a first row cumulative histogram processing module, configured to calculate a first contrast threshold value according to maximum and minimum values in the plurality of luminance histograms corresponding to each block partition in the i-th row, correct each luminance histogram according to the first contrast threshold value, obtain a plurality of corrected luminance histograms corresponding to each block partition in the i-th row, calculate a cumulative histogram corresponding to each block partition in the i-th row according to the plurality of corrected luminance histograms, store the cumulative histogram corresponding to each block partition in the i-th row in a third memory, and clear the storage data of the first memory;

[0020] a second row luminance histogram processing module, configured to perform histogram statistics on the plurality of luminance components corresponding to each block partition in the i+1-th row while calculating the cumulative histogram corresponding to each block partition in the i-th row, obtain a plurality of luminance histograms corresponding to each block partition in the i+1-th row, and store the plurality of luminance histograms corresponding to each block partition in the i+1-th row in a second memory;

[0021] a second row cumulative histogram processing module, configured to calculate a second contrast threshold value according to maximum and minimum values in the plurality of luminance histograms corresponding to each block partition in the i+1-th row, correct each luminance histogram according to the second contrast threshold value, obtain a plurality of corrected luminance histograms corresponding to each block partition in the i+1-th row, calculate a cumulative histogram corresponding to each block partition in the i+1-th row according to the plurality of corrected luminance histograms, store the cumulative histogram corresponding to each block partition in the i+1-th row in a fourth memory, and clear the storage data of the second memory;

[0022] The third row data processing module is configured to, when the camera video stream enters an (i+2)th row, time-division multiplex the first memory and clear the storage data of the third memory, and sequentially perform histogram statistics, histogram correction and histogram accumulation on the plurality of luminance components corresponding to each partition in the (i+2)th row to obtain an accumulated histogram corresponding to each partition in the (i+2)th row, and store the accumulated histogram corresponding to each partition in the (i+2)th row into the third memory.

[0023] The fourth row data processing module is configured to, when the camera video stream enters an (i+3)th row, time-division multiplex the second memory and clear the storage data of the fourth memory, and perform histogram statistics, histogram correction and histogram accumulation on the plurality of luminance components corresponding to each partition in the (i+3)th row to obtain an accumulated histogram corresponding to each partition in the (i+3)th row, and store the accumulated histogram corresponding to each partition in the (i+3)th row into the fourth memory.

[0024] The repeated data processing module is configured to repeat the data processing process of each partition in the ith row, the data processing process of each partition in the (i+1)th row, the data processing process of each partition in the (i+2)th row and the data processing process of each partition in the (i+3)th row to perform real-time processing on the m*n partitions.

[0025] Wherein, i is an integer.

[0026] In the above-mentioned video processing method and device based on the FPGA platform, the image is divided into m*n partitions, the luminance components of each partition are extracted for histogram statistics, and the contrast threshold of each partition is calculated based on the maximum value and the minimum value of the luminance histogram. The luminance histogram of different partitions is corrected according to different contrast thresholds of different partitions to obtain a corrected luminance histogram. Then, the accumulated histogram of each partition is calculated according to the corrected luminance histogram. When storing and processing the video stream, the histogram and the accumulated histogram, the processing process of ping-pong operation and iterative loop is adopted, and the time-division multiplex method is combined, so that the resource utilization rate of the video processing method based on the FPGA platform is greatly improved, and the code redundancy is small. m and n of the m*n partitions are positive integers greater than 4, the processing process of multiple partitions realizes more detailed segmentation of the image with less resources, so that the method of the present application can stretch more delicately in the case of rich scene or serious light and dark differentiation, avoiding the problem of image stretching distortion caused by too few partitions. Compared with the traditional 4*4 block example of 16 rams for statistical storage of each partition, the resources are saved, and the code redundancy is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1is a step flow diagram of a video processing method based on an FPGA platform provided by the present application.

[0028] Figure 2 is a diagram of m*n block partition provided by the present application.

[0029] Figure 3 is a diagram of 2*2 block sub-partition formed on the basis of m*n block partition provided by the present application.

[0030] Figure 4 is a partial diagram shown in one embodiment provided by the present application. Figure 3

[0031] Figure 5 is a structural diagram of a video processing device based on an FPGA platform provided by the present application. DETAILED DESCRIPTION

[0032] The technical solutions of the present application are described in further detail below by means of the drawings and embodiments.

[0033] Please refer to Figure 1 , the present application provides a video processing method based on an FPGA platform, comprising:

[0034] S10, acquiring a camera video stream, dividing each image in the camera video stream into m*n block partitions, and obtaining a plurality of luminance components corresponding to each block partition according to the m*n block partition data of each image, m and n being positive integers greater than 4;

[0035] S20, performing histogram statistics on the plurality of luminance components corresponding to each block partition in the i-th row to obtain a plurality of luminance histograms corresponding to each block partition in the i-th row, and storing the plurality of luminance histograms corresponding to each block partition in the i-th row into a first memory;

[0036] S30, calculating a first contrast threshold according to the maximum value and the minimum value in the plurality of luminance histograms corresponding to each block partition in the i-th row, correcting each luminance histogram according to the first contrast threshold to obtain a plurality of corrected luminance histograms corresponding to each block partition in the i-th row, calculating an accumulated histogram corresponding to each block partition in the i-th row according to the plurality of corrected luminance histograms, storing the accumulated histogram corresponding to each block partition in the i-th row into a third memory, and clearing the storage data of the first memory;

[0037] S40, while calculating the accumulated histogram corresponding to each block partition in the i-th row, performing histogram statistics on the plurality of luminance components corresponding to each block partition in the i+1-th row to obtain a plurality of luminance histograms corresponding to each block partition in the i+1-th row, and storing the plurality of luminance histograms corresponding to each block partition in the i+1-th row into a second memory;

[0038] ​S50, calculating a second contrast threshold according to the maximum value and the minimum value in the multiple luminance histograms corresponding to each partition in the i+1th row, correcting each luminance histogram according to the second contrast threshold, obtaining multiple corrected luminance histograms corresponding to each partition in the i+1th row according to the multiple corrected luminance histograms, and calculating an accumulated histogram corresponding to each partition in the i+1th row according to the multiple corrected luminance histograms, and storing the accumulated histogram corresponding to each partition in the i+1th row into the fourth memory and clearing the storage data of the second memory;

[0039] S60, when the camera video stream enters the i+2th row, time-multiplexing the first memory and clearing the storage data of the third memory, and sequentially performing histogram statistics, histogram correction and histogram accumulation on the multiple luminance components corresponding to each partition in the i+2th row, obtaining an accumulated histogram corresponding to each partition in the i+2th row, and storing the accumulated histogram corresponding to each partition in the i+2th row into the third memory;

[0040] S70, when the camera video stream enters the i+3th row, time-multiplexing the second memory and clearing the storage data of the fourth memory, and performing histogram statistics, histogram correction and histogram accumulation on the multiple luminance components corresponding to each partition in the i+3th row, obtaining an accumulated histogram corresponding to each partition in the i+3th row, and storing the accumulated histogram corresponding to each partition in the i+3th row into the fourth memory;

[0041] S80, repeating the data processing process of each partition in the i th row, the data processing process of each partition in the i+1th row, the data processing process of each partition in the i+2th row and the data processing process of each partition in the i+3th row, and performing real-time processing on the m*n partitions;

[0042] wherein i is an integer.

[0043] In the embodiment, m and n can be positive integers greater than 4, and the specific values are not limited. In an embodiment, each image is 3840*2160 pixels, and is divided into 8*8 blocks, each block being 480*270. The RGB format corresponding to each partition can be converted into hsv format, and then the luminance component of each partition can be obtained.

[0044] Four memories are adopted, the first memory, the second memory, the third memory and the fourth memory, which are respectively ram1, ram2, ram3 and ram4. The first memory and the second memory are used to store a plurality of luminance histograms, and the third memory and the fourth memory are used to store a plurality of accumulated histograms. The memory depth of the first memory and the second memory is 256x8, and the width is 18, and the binary representation of 480x270 requires 18 bits wide. The memory depth of the third memory and the fourth memory is 256x8, and the width is 18, and the binary representation of 480x270 requires 18 bits wide.

[0045] Please refer to Figure 2 As shown in the figure, the i-th row can be the first row or the second row or the third row, etc. The histogram statistics of each block partition in the i-th row are performed, and the current frame image data is stored in the ddr memory, and the statistical data of the plurality of luminance histograms is stored in the ram1. Each block is 480x270 in size. For example, please refer to Figure 2 When the number of rows of the camera video data is counted to 270 rows, the luminance histogram statistics of the 1st row in the m*n block partition is completed, and the luminance histogram of the 1st row in the m*n block partition is stored in the ram1.

[0046] The maximum value E max and the minimum value E min of the plurality of luminance histograms in each block partition in the i-th row are calculated, and a first contrast threshold is obtained. The first contrast threshold = (480x70 / (E max -E min ), and 480x270 is the number of pixels of each block partition. According to the first contrast threshold, the luminance histogram is limited and corrected, and the correction process is iterated at least once, and a plurality of corrected luminance histograms corresponding to each block partition are obtained. For example Figure 2 In the figure, the first row corresponds to 8 block partitions, and corresponds to 8 accumulated histograms. The accumulated histogram corresponding to each block partition in the i-th row is stored in the ram3, and the ram1 is cleared.

[0047] At the same time of the execution of S30, in S40, the histogram statistics of the plurality of luminance components corresponding to each block partition of the i+1-th row are performed, and the video data is stored in the ddr, and the histogram statistical data is stored in the ram2. For example, please refer to Figure 2 When the number of rows of the camera video stream is counted to 270x2 rows, it is indicated that the histogram statistics of each block partition of the 2nd row is completed, and the luminance histogram of the 2nd row in the m*n block partition is stored in the ram2.

[0048] The maximum value A' and the minimum value B' of the plurality of luminance histograms of the statistics in each block partition in the i+1th row are calculated, and a second contrast threshold is obtained. The second contrast threshold = (480x70 / (E max ’-E min ’)) is obtained. The luminance histograms are limited and corrected according to the second contrast threshold, and the correction is iterated at least once, and a plurality of corrected luminance histograms corresponding to each block partition in the i+1th row are obtained. The plurality of corrected luminance histograms corresponding to each block partition in the i+1th row are accumulated respectively to obtain accumulated histograms, and one accumulated histogram corresponds to one block partition. For example Figure 2 , the 2nd row corresponds to 8 block partitions, and 8 accumulated histograms correspond to the 2nd row. The accumulated histograms corresponding to each block partition in the i+1th row are stored in the ram4, and the ram2 is cleared.

[0049] When the video stream reaches the end of the 270th row, the accumulated histograms are calculated using the histogram statistical information of the ram1, and are stored in the ram3, which requires 256x8 clock cycles. For example, referring to the table shown in Figure 2 , the accumulated histogram of the ID11 block partition is obtained in 256 clock cycles after the end of the 270th row of the video stream. The accumulated histogram of the ID21 block partition is obtained in 256 clock cycles after the end of the 270x2th row.

[0050] When the video stream reaches the 270x2+1th row, the 3rd row table is entered, which can also be understood as starting to enter the i+2th row (i=1, Figure 2 , the 3rd row in the table) of the m x n block partitions, the ram1 can be time-division multiplexed, and the luminance histogram information of the 3rd row (which can be understood as ID31 to ID38 shown in Figure 2 ) of the ram1 is counted. When the video stream reaches the end of the 270x2th row and is ready to start entering the 270x2+1th row, that is, to start entering the i+2th row (i=1, Figure 2 , the 3rd row in the table) of the m x n block partitions, the accumulated histograms in the ram3 are cleared, and when the video stream reaches the end of the 270x3th row, the ram3 is reused to store the accumulated histograms of the 3rd row (ID31 to ID38) of the m x n block partitions, which can also be understood as storing the accumulated histograms corresponding to each block partition in the i+2th row of the m x n block partitions.

[0051] When the video stream reaches the 270x3+1th row, the 4th row table is entered, which can also be understood as starting to enter the i+3th row (i=1, Figure 2 , the 4th row in the table) of the m x n block partitions, the ram2 can be time-division multiplexed, and the memory of the ram2 counts the 4th row (which can be understood as Figure 2The luminance histogram information of the 8 block partitions of the 1st row, the 8 block partitions of the 2nd row, the 8 block partitions of the 3rd row and the 8 block partitions of the 4th row in the table is cleared, and when the video stream reaches the end of the 270x4 row, the ram4 is reused to store the accumulated histogram of the 4th row (ID41-ID48) of the m x n block partitions, which can also be understood as storing the accumulated histogram corresponding to each block partition in the i+3th row of the m x n block partitions. Figure 2 The luminance histogram information of the 8 block partitions of the 1st row, the 8 block partitions of the 2nd row, the 8 block partitions of the 3rd row and the 8 block partitions of the 4th row in the table is cleared, and when the video stream reaches the end of the 270x4 row, the ram4 is reused to store the accumulated histogram of the 4th row (ID41-ID48) of the m x n block partitions, which can also be understood as storing the accumulated histogram corresponding to each block partition in the i+3th row of the m x n block partitions.

[0052] The data processing process of each block partition in the i+3th row is repeated, which can also be understood as repeating the data processing process of the 8 block partitions of the 1st row, the 8 block partitions of the 2nd row, the 8 block partitions of the 3rd row and the 8 block partitions of the 4th row in the table. Figure 2 The data processing process of each block partition in the i+3th row is repeated, which can also be understood as repeating the data processing process of the 8 block partitions of the 1st row, the 8 block partitions of the 2nd row, the 8 block partitions of the 3rd row and the 8 block partitions of the 4th row in the table.

[0053] The video processing method based on the FPGA platform provided in the present application divides an image into m x n block partitions, extracts the luminance components of each block partition for histogram statistics, and calculates the contrast threshold of each block partition based on the maximum and minimum values of the luminance histogram. According to different contrast thresholds of different partitions, the luminance histogram of different partitions is corrected to obtain a corrected luminance histogram. Furthermore, the accumulated histogram of each block partition is calculated according to the corrected luminance histogram. When storing and processing the video stream, the histogram and the accumulated histogram, a ping-pong operation and an iterative loop processing process are adopted, and a time division multiplexing method is combined, so that the resource utilization rate of the video processing method based on the FPGA platform is greatly improved, and the code redundancy is small. m and n of the m x n block partition are positive integers greater than 4. The processing process of multiple block partitions realizes more detailed segmentation of the image with less resources, so that the present application method can stretch more delicately in the case of rich scenes or serious light and dark differentiation, avoiding the problem of image stretching distortion caused by too few partitions. Compared with the traditional 4 x 4 block partitioning, 16 rams are used to store and count each partition, which saves resources and reduces code redundancy.

[0054] In one embodiment, S10, a camera video stream is obtained, each image in the camera video stream is divided into m x n block partitions, and a plurality of luminance components corresponding to each block partition are obtained according to the m x n block partition data of each image, including:

[0055] S110, obtaining a red intensity value, a green intensity value and a blue intensity value of each pixel of each image;

[0056] S120, obtaining a maximum intensity value and a minimum intensity value according to the red intensity value, the green intensity value and the blue intensity value of each pixel;

[0057] S130, converting a red-green-blue mode of each image into a hue-saturation-value mode according to the red intensity value, the green intensity value, the blue intensity value, the maximum intensity value and the minimum intensity value, obtaining a plurality of value components corresponding to each partition, and keeping a plurality of hue components and a plurality of saturation components corresponding to each partition in time synchronization.

[0058] In the embodiment, the red intensity value is denoted as r, the green intensity value is denoted as g, and the blue intensity value is denoted as b. The data r, g and b of the three channels of RGB of each pixel are counted to obtain a maximum intensity value max and a minimum intensity value min of the intensity value.

[0059] The red-green-blue mode of each image is converted into the hue-saturation-value mode according to the following formula, and the conversion formula is as follows:

[0060] When max = min, h = 0;

[0061] When max = r, g ≥ b,

[0062] When max = r, g < b,

[0063] When max = g,

[0064] When max = b,

[0065] When max = 0, s = 0;

[0066] When max = otherwise,

[0067] v = max.

[0068] Therefore, after the RGB format is converted into the hsv color domain through the steps in the embodiment, the value component v is subjected to histogram processing. Meanwhile, the h component and the s component are subjected to beat-making in synchronization, so that the h component, the s component and the value component v subjected to histogram processing are kept in time synchronization. Through the steps in the embodiment, the value component v of RGB is extracted, and the value component v is subjected to separate histogram processing, so that the image contrast is stretched more gently and delicately compared with the traditional method, and the resource utilization used in the data processing process is 1 / 3 of the traditional method.

[0069] In one embodiment, the histogram statistics of the plurality of luminance components corresponding to each block partition in the i-th row are calculated while the current frame of video data is stored in the DDR memory, and the histogram statistics are stored in the ram1.

[0070] In the histogram calculation code, a first input first output (FIFO) is instantiated with a depth of 8192 and a width of 8 bits. When the input video stream reaches 270x2+1 rows, the reading of the DDR is started. When the data remaining in the FIFO >= 3840, the reading of the FIFO is performed. When the data remaining in the FIFO reaches 3840x2, the reading of the DDR is stopped. When the data remaining in the FIFO <= 1920, the reading of the DDR is restarted. When the data remaining in the FIFO reaches 3840x2, the reading of the DDR is stopped. Further, the data read from the FIFO and the accumulated histogram are numerically calculated by an equalization formula for subsequent interpolation calculation.

[0071] Please refer to Figure 3 In one embodiment, the video processing method based on the FPGA platform further comprises:

[0072] S910, each block partition in the m x n block partition is divided into four, forming a 2x2 block sub-partition corresponding to each block partition, and each image in the camera video stream is divided into a 2m x 2n block sub-partition according to the 2x2 block sub-partition;

[0073] S920, based on the accumulated histogram corresponding to the first block partition in the first row, the accumulated histogram corresponding to the n-th block partition in the first row, the accumulated histogram corresponding to the first block partition in the m-th row, and the accumulated histogram corresponding to the n-th block partition in the m-th row in the m x n block partition, the vertex equalization value is calculated according to the equalization formula as the luminance value of each vertex sub-partition located in the 2m x 2n block sub-partition.

[0074] In this embodiment, each block partition in the m x n block partition is further refined and divided into a 2x2 block sub-partition, and each image is divided into a 2m x 2n block sub-partition.

[0075] Each vertex sub-partition located in the 2m x 2n block sub-partition can refer to the vertex area filled with four oblique lines in Figure 3 For each vertex sub-partition, the accumulated histogram corresponding to the first block partition in the first row, the accumulated histogram corresponding to the n-th block partition in the first row, the accumulated histogram corresponding to the first block partition in the m-th row, and the accumulated histogram corresponding to the n-th block partition in the m-th row in the m x n block partition are equalized. The equalization formula is

[0076] wherein A0=480x270 represents the number of pixels in each block partition, Emax The maximum value of the luminance histogram of the first block partition in the first row, the n-th block partition in the first row, the first block partition in the m-th row or the n-th block partition in the m-th row in the m x n block partition is summed up, and the accumulated histogram corresponding to the first block partition in the first row, the n-th block partition in the first row, the first block partition in the m-th row or the n-th block partition in the m-th row in the m x n block partition is summed up. A D is calculated from the maximum value of the luminance histogram of a block partition, the accumulated histogram and the number of pixels. The maximum value of the luminance histogram of each block partition and the accumulated histogram are different, and the D obtained is also different. The maximum value of the luminance histogram of the first block partition in the first row, the n-th block partition in the first row, the first block partition in the m-th row and the n-th block partition in the m-th row in the m x n block partition is calculated to obtain D as the luminance value of each vertex sub-partition located in the 2m x 2n block sub-partition, that is Figure 3 The luminance value of the vertex area filled with four diagonal lines in the middle.

[0077] In an embodiment, the video processing method based on the FPGA platform further comprises:

[0078] S930, based on the accumulated histogram corresponding to each block partition in the first column and the accumulated histogram corresponding to each block partition in the n-th column in the m x n block partition, the accumulated histogram of the column adjacent two block partitions is calculated according to the equalization formula to obtain the column one equalization value and the column two equalization value;

[0079] S940, obtaining the column one center point coordinate and the column two center point coordinate of the column adjacent two block partitions, calculating the column one vertical distance between the column one center point coordinate and the pixel point coordinate in the column boundary sub-partition of the adjacent 2m x 2n block sub-partition, and calculating the column two vertical distance between the column two center point coordinate and the pixel point coordinate in the adjacent column boundary sub-partition;

[0080] S950, linear interpolation calculation is performed according to the column one equalization value, the column two equalization value, the column one vertical distance and the column two vertical distance to obtain the luminance value of each column boundary sub-partition.

[0081] In this embodiment, each column boundary sub-partition is two columns of the left and right edges of the 2m x 2n block sub-partition, and does not include the four vertex sub-partitions, such as Figure 3 The point filling pattern in the middle. Each column boundary sub-partition belongs to a part of each block partition in the first column and each block partition in the n-th column in the m x n block partition. Each column boundary sub-partition has pixel points, such as Figure 4 The pixel point (m, n) in the point filling pattern area in the middle.

[0082] Taking the A and C points in the middle as an example, the column one equalization value D Figure 4 is calculated according to the equalization formula respectively.A with the column two equalization value D C The column one center point coordinate and the column two center point coordinate of the two adjacent column partition are Figure 4 The vertical distance between the column one center point coordinate A (x1, y1) and the pixel point (m, n) is calculated to obtain the column one vertical distance. The vertical distance between the column two center point coordinate C (x3, y3) and the pixel point (m, n) is calculated to obtain the column two vertical distance. The column one equalization value D A with the column two equalization value D C is calculated by linear interpolation, and the calculation result is taken as the luminance value of the first column boundary subdivision on the left side.

[0083] In this embodiment, the method steps are not limited to Figure 4 The examples of the pixel point (m, n), the point A (x1, y1), and the point C (x3, y3) can be applied to the calculation of the luminance value of each column boundary subdivision, and the calculation principle is the same as that described in the steps of this embodiment.

[0084] In one embodiment, the video processing method based on the FPGA platform further includes:

[0085] S960, based on the accumulated histogram corresponding to each block partition in the first row and the accumulated histogram corresponding to each block partition in the mth row in the m×n block partition, and the accumulated histogram of the two adjacent row partitions in the accumulated histogram, the row one equalization value and the row two equalization value are calculated according to the equalization formula;

[0086] S970, the row one center point coordinate and the row two center point coordinate of the two adjacent row partitions are obtained, the row one horizontal distance between the row one center point coordinate and the pixel point coordinate in the row boundary subdivision of the adjacent 2m×2n block subdivision is calculated, and the row two horizontal distance between the row two center point coordinate and the pixel point coordinate in the adjacent row boundary subdivision is calculated;

[0087] S980, linear interpolation calculation is performed according to the row one equalization value, the row two equalization value, the row one horizontal distance, and the row two horizontal distance to obtain the luminance value of each row boundary subdivision.

[0088] In this embodiment, each row boundary subdivision is two rows of the upper and lower edges of the 2m×2n block subdivision, and does not include the four vertex subdivisions, such as Figure 3 the grid-shaped filling pattern. Each row boundary subdivision belongs to a part of each block partition in the first row and each block partition in the mth row in the m×n block partition. The pixel point of each row boundary subdivision, such as Figure 4 the pixel point (j, k) in the grid-shaped filling pattern region.

[0089] In this embodiment, the pixel point (j, k) in the grid-shaped filling pattern region is taken as an example. Figure 4Take points A and B as examples, the row equalization value D is calculated respectively A in the embodiment, D A is the same as the column equalization value D A in the above embodiment, and is the same block partition) and the row two equalization value D B . The row one center point coordinate and the row two center point coordinate of the adjacent two block partitions are Figure 4 point A (x1, y1) (which is the same as the column one center point coordinate A (x1, y1), and is the same point) and point B (x2, y2). The horizontal distance between the row one center point coordinate A (x1, y1) and the pixel point (j, k) is calculated to obtain the row one horizontal distance. The horizontal distance between the row two center point coordinate B (x2, y2) and the pixel point (j, k) is calculated to obtain the row two horizontal distance. The linear interpolation calculation is performed according to the row one horizontal distance, the row two horizontal distance, and the row one equalization value D A and the row two equalization value D B , and the calculation result is taken as the luminance value of the first row boundary subdivision on the left side of the upper boundary.

[0090] In the embodiment, the method steps are not limited to Figure 4 the examples of the pixel point (j, k), the point A (x1, y1), and the point B (x2, y2) can be applied to the calculation of the luminance value of each row boundary subdivision, and the calculation principle is the same as that described in the steps of the embodiment.

[0091] In one embodiment, the video processing method based on the FPGA platform further includes:

[0092] S990, based on the adjacent four block partitions in the corresponding accumulated histogram of each block partition in the m×n block partition, the first same row equalization value, the second same row equalization value, the same column equalization value, and the different column equalization value of the adjacent four block partitions are calculated according to the equalization formula;

[0093] S991, the first same row center point coordinate, the second same row center point coordinate, the same column center point coordinate, and the different column center point coordinate of the adjacent four block partitions are obtained;

[0094] S992, the first same row horizontal distance and the second same row horizontal distance from the first same row center point coordinate and the second same row center point coordinate to the pixel point coordinate in the non-boundary subdivision of the 2m×2n block subdivision adjacent to the adjacent four block partitions are calculated;

[0095] S993, the first luminance value is obtained by performing the bilinear interpolation calculation according to the first same row equalization value, the second same row equalization value, the first same row horizontal distance, and the second same row horizontal distance;

[0096] S994, calculating the same column horizontal distance and the different column horizontal distance from the same column center point coordinate and the different column center point coordinate to the pixel point coordinate in the non-boundary subdivided area of the 2m*2n block subdivided area adjacent to the adjacent four block partitions;

[0097] S995, performing bilinear interpolation calculation according to the same column equalization value, the different column equalization value, the same column horizontal distance and the different column horizontal distance to obtain the second brightness value;

[0098] S996, calculating the same row vertical distance and the same column vertical distance from the first same row center point coordinate and the same column center point coordinate to the pixel point coordinate in the non-boundary subdivided area of the 2m*2n block subdivided area adjacent to the adjacent four block partitions;

[0099] S997, performing bilinear interpolation calculation according to the same row vertical distance, the same column vertical distance, the first brightness value and the second brightness value to obtain the brightness value of each non-boundary subdivided area;

[0100] S998, performing red-green-blue mode recovery on the brightness value of each vertex subdivided area, the brightness value of each column boundary subdivided area, the brightness value of each row boundary subdivided area, the brightness value of each non-boundary subdivided area of the 2m*2n block subdivided area corresponding to each image in the camera video stream, the plurality of hue components and the plurality of saturation components corresponding to each block partition to obtain the optimized camera video stream.

[0101] In the embodiment, each non-boundary subdivided area is other subdivided area after removing the upper and lower boundaries and the left and right boundaries of the 2m*2n block subdivided area, such as Figure 3 In the embodiment, each non-boundary subdivided area belongs to a part of each block partition in the m*n block partition, and thus the calculation is performed according to the accumulated histogram corresponding to each block partition in the m*n block partition. Each pixel point in the non-boundary subdivided area, such as Figure 4 In the embodiment, the pixel point (u, w) in the non-filled blank area.

[0102] Taking the A point, the B point, the C point and the D point in Figure 4 as an example for description, the black thick solid line is the partition boundary, the intersection point of the dashed line is called the interpolation center point, and the dashed line subdivides each block partition in the m*n block partition to form the 2m*2n block subdivided area.

[0103] The first same row center point coordinate, the second same row center point coordinate, the same column center point coordinate and the different column center point coordinate are Figure 4A point (x1, y1) (same as the column one center point coordinate A point (x1, y1), same point), B point (x2, y2) (same as the row two center point coordinate B point (x2, y2), same point), C point (x3, y3) (same as the column two center point coordinate C point (x3, y3), same point) and D point (x4, y4) are respectively used to perform the equalization processing of four sub-regions of the pixel points in the non-boundary sub-region.

[0104] According to the equalization formula and based on the accumulated histogram of the adjacent four sub-regions of a certain non-boundary sub-region, the first same row equalization value D A ” (in this embodiment, D A ” is the same as the column one equalization value D A in the above embodiment, the first same row equalization value D A ’ (in this embodiment, D B ’ is the same as the row two equalization value D B ’ in the above embodiment, the same column equalization value D B ’ (in this embodiment, D C ’ is the same as the column two equalization value D C ’ in the above embodiment, the different column equalization value D C ’. D

[0105] The horizontal distance between the first same row center point coordinate A point (x1, y1) and the pixel point (u, w) is calculated to obtain the first same row horizontal distance. The horizontal distance between the second same row center point coordinate B point (x2, y2) and the pixel point (u, w) is calculated to obtain the second same row horizontal distance. According to the first same row equalization value D A ”, the second same row equalization value D B ’, the first same row horizontal distance and the second same row horizontal distance, the bilinear interpolation calculation is performed to obtain the first brightness value Q1.

[0106] The horizontal distance between the same column center point coordinate C point (x3, y3) and the pixel point (u, w) is calculated to obtain the same column horizontal distance. The horizontal distance between the different column center point coordinate D point (x4, y4) and the pixel point (u, w) is calculated to obtain the different column horizontal distance. According to the same column equalization value D C ’, the different column equalization value D D , the same column horizontal distance and the different column horizontal distance, the bilinear interpolation calculation is performed to obtain the second brightness value Q2.

[0107] ​The vertical distance between the first same-row center point coordinate A (x1, y1) and the pixel point (u, w) is calculated to obtain a same-row vertical distance. The vertical distance between the same-column center point coordinate C (x3, y3) and the pixel point (u, w) is calculated to obtain a same-column vertical distance. The third brightness value Q3 is obtained by bilinear interpolation calculation according to the same-row vertical distance, the same-column vertical distance, the first brightness value and the second brightness value, and the third brightness value Q3 is taken as the brightness value of the first non-boundary subdivision area at the upper left corner.

[0108] In the embodiment, the method steps are not limited to Figure 4 The examples of the pixel point (u, w), the A point, the B point, the C point and the D point can be applied to the calculation of the brightness value of each non-boundary subdivision area, and the calculation principle is the same as that described in the steps of the embodiment.

[0109] The h component and the s component are synchronized to beat, so that the h component, the s component and the histogram-processed v component are kept synchronous in time, the h component, the s component and the histogram-processed v component are subjected to RGB restoration again, and the restored RGB data is subjected to video stream output.

[0110] The h component, the s component and the histogram-processed v component are subjected to RGB restoration again, and the formula is as follows:

[0111] p = v x (1-s), q = v x (1-f x s), t = v x (1-(1-f) x s);

[0112] The formula for each color vector (r, g, b) is as follows:

[0113]

[0114] In the video processing method based on the FPGA platform provided in the application, the accumulated histograms of each block partition in the m x n block partition are different and the maximum value E of the brightness histogram is max different, so that different results are obtained by using different partition equalization formulas for the same pixel value, the weight distribution is realized, and the problem of obvious boundary alienation caused by different partitions is solved. Moreover, in the video processing method based on the FPGA platform provided in the application, each vertex partition, each column boundary partition, each row boundary partition and each non-boundary partition located in the 2m x 2n block subdivision area are more finely divided, different optimization processing methods are used for different subdivision areas, the problem of obvious boundary alienation of the block histogram is compensated, and the distortion problem of boundary calculation is effectively avoided, so that the picture is more flat and soft.

[0115] The steps S10, S20, S30, S40, S50, S60, S70, S80, S910, S920, S930, S940, S950, S960, S970, S980, S990, S991, S992, S993, S994, S995, S996, S997, S998 in each of the above embodiments are repeatedly executed until the video stream processing is successful.

[0116] Therefore, compared with the traditional method, the video processing method based on the FPGA platform provided in the application reduces the delay in the processing process from the inter-frame delay to the inter-line delay, can realize real-time processing, and the video effect is more smooth and stable in the low frame frequency special scene.

[0117] Please refer to Figure 5 The application provides a video processing device 100 based on an FPGA platform. The video processing device 100 based on the FPGA platform comprises a data acquisition module 10, a first line luminance histogram processing module 20, a first line cumulative histogram processing module 30, a second line luminance histogram processing module 40, a second line cumulative histogram processing module 50, a third line data processing module 60, a fourth line data processing module 70, and a repeated data processing module 80. The data acquisition module 10 is used for acquiring a camera video stream, dividing each image in the camera video stream into m*n block partitions, and obtaining a plurality of luminance components corresponding to each block partition according to the m*n block partition data of each image, wherein m and n are positive integers greater than 4.

[0118] The first line luminance histogram processing module 20 is used for performing histogram statistics on the plurality of luminance components corresponding to each block partition in the i-th line, obtaining a plurality of luminance histograms corresponding to each block partition in the i-th line, and storing the plurality of luminance histograms corresponding to each block partition in the i-th line in a first memory. The first line cumulative histogram processing module 30 is used for calculating a first contrast threshold value according to the maximum value and the minimum value in the plurality of luminance histograms corresponding to each block partition in the i-th line, correcting each luminance histogram according to the first contrast threshold value, obtaining a plurality of corrected luminance histograms corresponding to each block partition in the i-th line, calculating a cumulative histogram corresponding to each block partition in the i-th line according to the plurality of corrected luminance histograms, storing the cumulative histogram corresponding to each block partition in the i-th line in a third memory, and clearing the storage data of the first memory.

[0119] The second row luminance histogram processing module 40 is configured to, when calculating the accumulated histogram corresponding to each block partition in the i-th row, perform histogram statistics on the plurality of luminance components corresponding to each block partition in the (i+1)-th row, to obtain a plurality of luminance histograms corresponding to each block partition in the (i+1)-th row, and store the plurality of luminance histograms corresponding to each block partition in the (i+1)-th row in the second memory. The second row accumulated histogram processing module 50 is configured to calculate a second contrast threshold according to the maximum value and the minimum value in the plurality of luminance histograms corresponding to each block partition in the (i+1)-th row, correct each luminance histogram according to the second contrast threshold, obtain a plurality of corrected luminance histograms corresponding to each block partition in the (i+1)-th row, calculate the accumulated histogram corresponding to each block partition in the (i+1)-th row according to the plurality of corrected luminance histograms, and store the accumulated histogram corresponding to each block partition in the (i+1)-th row in the fourth memory, and clear the storage data of the second memory.

[0120] The third row data processing module 60 is configured to, when the camera video stream enters the (i+2)-th row, time-division multiplex the first memory and clear the storage data of the third memory, and sequentially perform histogram statistics, histogram correction, and histogram accumulation on the plurality of luminance components corresponding to each block partition in the (i+2)-th row, to obtain the accumulated histogram corresponding to each block partition in the (i+2)-th row, and store the accumulated histogram corresponding to each block partition in the (i+2)-th row in the third memory. The fourth row data processing module 70 is configured to, when the camera video stream enters the (i+3)-th row, time-division multiplex the second memory and clear the storage data of the fourth memory, and perform histogram statistics, histogram correction, and histogram accumulation on the plurality of luminance components corresponding to each block partition in the (i+3)-th row, to obtain the accumulated histogram corresponding to each block partition in the (i+3)-th row, and store the accumulated histogram corresponding to each block partition in the (i+3)-th row in the fourth memory.

[0121] The repeated data processing module 80 is configured to repeat the data processing process of each block partition in the i-th row, the data processing process of each block partition in the (i+1)-th row, the data processing process of each block partition in the (i+2)-th row, and the data processing process of each block partition in the (i+3)-th row, to perform real-time processing on the m*n block partitions. Wherein, i is an integer.

[0122] In this embodiment, the related description of the data acquisition module 10 can refer to the related description of S10 in the above embodiment. The related description of the first row luminance histogram processing module 20 can refer to the related description of S20 in the above embodiment. The related description of the first row accumulated histogram processing module 30 can refer to the related description of S30 in the above embodiment. The related description of the second row luminance histogram processing module 40 can refer to the related description of S40 in the above embodiment. The related description of the second row accumulated histogram processing module 50 can refer to the related description of S50 in the above embodiment. The related description of the third row data processing module 60 can refer to the related description of S60 in the above embodiment. The related description of the fourth row data processing module 70 can refer to the related description of S70 in the above embodiment. The related description of the repeated data processing module 80 can refer to the related description of S80 in the above embodiment.

[0123] In one embodiment, the data acquisition module 10 comprises a red-green-blue intensity acquisition module, an intensity value comparison module, and a luminance component acquisition module. The red-green-blue intensity acquisition module is configured to acquire the red intensity value, the green intensity value, and the blue intensity value of each pixel of each image. The intensity value comparison module is configured to obtain the maximum intensity value and the minimum intensity value according to the red intensity value, the green intensity value, and the blue intensity value of each pixel. The luminance component acquisition module is configured to convert the red-green-blue mode of each image into the hue-saturation-luminance mode according to the red intensity value, the green intensity value, the blue intensity value, the maximum intensity value, and the minimum intensity value, to obtain a plurality of luminance components corresponding to each partition, and to keep the plurality of hue components and the plurality of saturation components corresponding to each partition in time synchronization.

[0124] In this embodiment, the related description of the red-green-blue intensity acquisition module can refer to the related description of S110 in the above embodiment. The related description of the intensity value comparison module can refer to the related description of S120 in the above embodiment. The related description of the luminance component acquisition module can refer to the related description of S130 in the above embodiment.

[0125] In one embodiment, the video processing device 100 based on the FPGA platform further comprises a sub-partition division module and a vertex sub-partition calculation module. The sub-partition division module is configured to divide each partition in the m×n block partition into four, to form a 2×2 block sub-partition corresponding to each partition, and to divide each image in the camera video stream into a 2m×2n block sub-partition according to the 2×2 block sub-partition. The vertex sub-partition calculation module is configured to calculate the vertex equalization value based on the accumulated histogram corresponding to the first partition in the first row, the accumulated histogram corresponding to the nth partition in the first row, the accumulated histogram corresponding to the first partition in the mth row, and the accumulated histogram corresponding to the nth partition in the mth row in the m×n block partition, respectively, according to the equalization formula, as the luminance value of each vertex sub-partition located in the 2m×2n block sub-partition.

[0126] In this embodiment, the related description of the subdivision division module can refer to the related description of S910 in the above embodiment. The related description of the vertex subdivision calculation module can refer to the related description of S920 in the above embodiment.

[0127] In one embodiment, the FPGA platform-based video processing device 100 further comprises a column equalization value acquisition module, a column vertical distance acquisition module, and a column boundary subdivision calculation module. The column equalization value acquisition module is configured to calculate a first column equalization value and a second column equalization value based on the accumulated histogram corresponding to each block partition in the first column and the accumulated histogram corresponding to each block partition in the nth column in the m*n block partition, and according to the equalization formula.

[0128] The column vertical distance acquisition module is configured to acquire the column one center point coordinate and the column two center point coordinate of the column adjacent two block partitions, calculate the column one vertical distance between the column one center point coordinate and the pixel point coordinate in the column boundary subdivision of the adjacent 2m*2n block subdivision, and calculate the column two vertical distance between the column two center point coordinate and the pixel point coordinate in the adjacent column boundary subdivision. The column boundary subdivision calculation module is configured to perform linear interpolation calculation according to the first column equalization value, the second column equalization value, the column one vertical distance, and the column two vertical distance, to obtain the luminance value of each column boundary subdivision.

[0129] In this embodiment, the related description of the column equalization value acquisition module can refer to the related description of S930 in the above embodiment. The related description of the column vertical distance acquisition module can refer to the related description of S940 in the above embodiment. The related description of the column boundary subdivision calculation module can refer to the related description of S950 in the above embodiment.

[0130] In one embodiment, the FPGA platform-based video processing device 100 further comprises a row equalization value acquisition module, a row horizontal distance acquisition module, and a row boundary subdivision calculation module. The row equalization value acquisition module is configured to calculate a first row equalization value and a second row equalization value based on the accumulated histogram corresponding to each block partition in the first row and the accumulated histogram corresponding to each block partition in the mth row in the m*n block partition, and according to the equalization formula. The row horizontal distance acquisition module is configured to acquire the row one center point coordinate and the row two center point coordinate of the row adjacent two block partitions, calculate the row one horizontal distance between the row one center point coordinate and the pixel point coordinate in the row boundary subdivision of the adjacent 2m*2n block subdivision, and calculate the row two horizontal distance between the row two center point coordinate and the pixel point coordinate in the adjacent row boundary subdivision. The row boundary subdivision calculation module is configured to perform linear interpolation calculation according to the first row equalization value, the second row equalization value, the row one horizontal distance, and the row two horizontal distance, to obtain the luminance value of each row boundary subdivision.

[0131] In this embodiment, the description of the row equalization value obtaining module can refer to the description of S960 in the above embodiment. The description of the row horizontal distance obtaining module can refer to the description of S970 in the above embodiment. The description of the row boundary subdivision calculation module can refer to the description of S980 in the above embodiment.

[0132] In one embodiment, the FPGA platform-based video processing device 100 further comprises a row-column equalization value obtaining module, a center point coordinate obtaining module, a first non-boundary horizontal distance obtaining module, a first brightness value obtaining module, a second non-boundary horizontal distance obtaining module, a second brightness value obtaining module, a non-boundary vertical distance obtaining module, a non-boundary brightness value obtaining module, and an optimized camera video stream obtaining module. The row-column equalization value obtaining module is configured to calculate, based on the accumulated histograms of the adjacent four block partitions in the accumulated histogram corresponding to each block partition in the m*n block partition, the first same-row equalization value, the second same-row equalization value, the same-column equalization value, and the different-column equalization value of the adjacent four block partitions according to an equalization formula.

[0133] The center point coordinate obtaining module is configured to obtain the first same-row center point coordinate, the second same-row center point coordinate, the same-column center point coordinate, and the different-column center point coordinate of the adjacent four block partitions. The first non-boundary horizontal distance obtaining module is configured to calculate the first same-row horizontal distance and the second same-row horizontal distance from the first same-row center point coordinate and the second same-row center point coordinate to the pixel point coordinates in the non-boundary subdivision of the 2m*2n block subdivision adjacent to the adjacent four block partitions. The first brightness value obtaining module is configured to perform bilinear interpolation calculation according to the first same-row equalization value, the second same-row equalization value, the first same-row horizontal distance, and the second same-row horizontal distance to obtain the first brightness value.

[0134] The second non-boundary horizontal distance obtaining module is configured to calculate the same-column horizontal distance and the different-column horizontal distance from the same-column center point coordinate and the different-column center point coordinate to the pixel point coordinates in the non-boundary subdivision of the 2m*2n block subdivision adjacent to the adjacent four block partitions. The second brightness value obtaining module is configured to perform bilinear interpolation calculation according to the same-column equalization value, the different-column equalization value, the same-column horizontal distance, and the different-column horizontal distance to obtain the second brightness value.

[0135] The non-boundary vertical distance obtaining module is configured to calculate the same-row vertical distance and the same-column vertical distance from the first same-row center point coordinate and the same-column center point coordinate to the pixel point coordinates in the non-boundary sub-partition of the 2m×2n block sub-partition adjacent to the adjacent four blocks of the sub-partition. The non-boundary luminance value obtaining module is configured to perform bilinear interpolation calculation according to the same-row vertical distance, the same-column vertical distance, the first luminance value and the second luminance value to obtain the luminance value of each non-boundary sub-partition. The optimized camera video stream obtaining module is configured to perform RGB mode recovery on the luminance value of each vertex sub-partition of the 2m×2n block sub-partition corresponding to each image in the camera video stream, the luminance value of each column boundary sub-partition, the luminance value of each row boundary sub-partition, the luminance value of each non-boundary sub-partition, the plurality of hue components and the plurality of saturation components corresponding to each block of the sub-partition to obtain the optimized camera video stream.

[0136] In the embodiment, the related description of the row and column equalization value obtaining module can refer to the related description of S990 in the above embodiment. The related description of the center point coordinate obtaining module can refer to the related description of S991 in the above embodiment. The related description of the first non-boundary horizontal distance obtaining module can refer to the related description of S992 in the above embodiment. The related description of the first luminance value obtaining module can refer to the related description of S993 in the above embodiment. The related description of the second non-boundary horizontal distance obtaining module can refer to the related description of S994 in the above embodiment. The related description of the second luminance value obtaining module can refer to the related description of S995 in the above embodiment. The related description of the non-boundary vertical distance obtaining module can refer to the related description of S996 in the above embodiment. The related description of the non-boundary luminance value obtaining module can refer to the related description of S997 in the above embodiment. The related description of the optimized camera video stream obtaining module can refer to the related description of S998 in the above embodiment.

[0137] In one embodiment, in the first row cumulative histogram processing module 30, the first contrast threshold is (480×270 / (E max -E min )). Wherein, 480×270 is the pixel number of each block of the sub-partition, E max is the maximum value in the plurality of luminance histograms corresponding to each block of the sub-partition, E min is the minimum value in the plurality of luminance histograms corresponding to each block of the sub-partition.

[0138] In the embodiment, the related description of the first contrast threshold in the first row cumulative histogram processing module 30 can refer to the related description in the above method.

[0139] In one embodiment, the FPGA platform-based video processing device 100 further comprises a video data acquisition module and a video output module. The video data acquisition module is configured to initialize and configure the camera through the FPGA, unpack the video packet data output by the camera, complete image acquisition, obtain the camera video stream, and process the camera video stream to the data acquisition module 10.

[0140] The data acquisition module 10, the first row luminance histogram processing module 20, the first row cumulative histogram processing module 30, the second row luminance histogram processing module 40, the second row cumulative histogram processing module 50, the third row data processing module 60, the fourth row data processing module 70, the repeated data processing module 80, the subdivision division module, the vertex subdivision calculation module, the column equalization value acquisition module, the column vertical distance acquisition module, the column boundary subdivision calculation module, the row equalization value acquisition module, the row horizontal distance acquisition module, the row boundary subdivision calculation module, the row and column equalization value acquisition module, the center point coordinate acquisition module, the first non-boundary horizontal distance acquisition module, the first luminance value acquisition module, the second non-boundary horizontal distance acquisition module, the second luminance value acquisition module, the non-boundary vertical distance acquisition module, the non-boundary luminance value acquisition module, and the optimized camera video stream acquisition module are processed on the FPGA.

[0141] The video output module is configured to output the optimized camera video stream formed after the FPGA processing, and perform imaging on the host computer.

[0142] The above detailed description further explains the purpose, technical solutions, and beneficial effects of the present application. It should be understood that the above detailed description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A video processing method based on an FPGA platform, characterized in that, include: Acquire the camera video stream, divide each image in the camera video stream into m×n partitions, and obtain multiple brightness components corresponding to each partition based on the m×n partition data of each image, where m and n are positive integers greater than 4; For the first i Histogram statistics are performed on the multiple brightness components corresponding to each partition in the row to obtain the first... i Multiple brightness histograms corresponding to each partition in the row, and the first... i Multiple brightness histograms corresponding to each partition in the row are stored in the first memory. According to the i The first contrast threshold is calculated by comparing the maximum and minimum values ​​in the multiple brightness histograms corresponding to each partition in the row. Each brightness histogram is then corrected based on this first contrast threshold to obtain the first... i For each partition in the row, there are multiple corrected brightness histograms corresponding to it. Based on these multiple corrected brightness histograms, the first value is calculated. i The cumulative histogram corresponding to each partition in the row, and the first... i The accumulated histogram corresponding to each partition in the row is stored in the third memory, and the data stored in the first memory is cleared. In calculating the first i When calculating the cumulative histogram corresponding to each partition in a row, for the th... i+ Histogram statistics are performed on the multiple brightness components corresponding to each partition in row 1 to obtain the first... i+ The first row contains multiple brightness histograms corresponding to each partition, and the first row contains multiple brightness histograms corresponding to each partition. i+ The multiple brightness histograms corresponding to each partition in row 1 are stored in the second memory. According to the i+ The second contrast threshold is calculated by combining the maximum and minimum values ​​from the multiple brightness histograms corresponding to each partition in row 1. Based on this second contrast threshold, each brightness histogram is corrected to obtain the result. i+ For each partition in row 1, there are multiple corrected brightness histograms. Based on these multiple corrected brightness histograms, the first value is calculated. i+ The cumulative histogram corresponding to each partition in row 1, and the first... i+ The accumulated histogram corresponding to each partition in row 1 is stored in the fourth memory, and the data stored in the second memory is cleared. When the camera video stream enters the... i+ In line 2, time-division multiplexing is performed on the first memory and the stored data in the third memory is cleared, and the second... i+ In the second row, the multiple brightness components corresponding to each partition are sequentially subjected to histogram statistics, histogram correction, and histogram accumulation to obtain the first... i+ The cumulative histogram corresponding to each partition in the second row, and the first row... i+ The cumulative histogram corresponding to each partition in the two rows is stored in the third memory; When the camera video stream enters the... i+ In line 3, time-division multiplexing is performed on the second memory and the stored data in the fourth memory is cleared, and the data in the third memory is... i+ Histogram statistics, histogram correction, and histogram summation are performed on the multiple brightness components corresponding to each partition in the 3rd row to obtain the first... i+ The cumulative histogram corresponding to each partition in the 3rd row, and the th row... i+ The cumulative histogram corresponding to each partition in the three rows is stored in the fourth memory. Repeat the first i Arriving at the i+ The data processing procedure for each partition in the three lines is to process m×n partitions in real time; in, i It is an integer.

2. The video processing method based on an FPGA platform according to claim 1, characterized in that, Acquire the camera video stream, divide each image in the camera video stream into m×n partitions, and based on the m×n partition data of each image, obtain multiple brightness components corresponding to each partition, including: Obtain the red, green, and blue intensity values ​​for each pixel in each image; The maximum and minimum intensity values ​​are obtained based on the red, green, and blue intensity values ​​of each pixel; Based on the red, green, blue, maximum, and minimum intensity values, the red-green-blue mode of each image is converted into a hue-saturation-brightness mode to obtain multiple brightness components corresponding to each partition, and the multiple hue components and multiple saturation components corresponding to each partition are kept synchronized in time.

3. The video processing method based on an FPGA platform according to claim 2, characterized in that, The method also includes: Each of the m×n partitions is divided into four subdivisions to form a 2×2 subdivision region corresponding to each partition. Based on the 2×2 subdivision regions, each image in the camera video stream is divided into 2m×2n subdivision regions. Based on the accumulated histograms corresponding to the first partition in the first row of the m×n block partition, the accumulated histograms corresponding to the nth partition in the first row, the accumulated histograms corresponding to the first partition in the mth row, and the accumulated histograms corresponding to the nth partition in the mth row, the vertex equalization values ​​are calculated according to the equalization formula, and used as the brightness values ​​of each vertex subdivision located in the 2m×2n block subdivision area.

4. The video processing method based on an FPGA platform according to claim 3, characterized in that, The method also includes: Based on the accumulated histograms of each partition in the first column of the m×n partition and the accumulated histograms of two adjacent partitions in the column of the accumulated histogram of each partition in the nth column, the equalization values ​​of column one and column two are calculated according to the equalization formula. Obtain the coordinates of the center point of column 1 and the center point of column 2 of two adjacent partitions. Calculate the vertical distance of the center point of column 1 to the pixel coordinates of the adjacent 2m×2n subdivision block in column 1, and calculate the vertical distance of the center point of column 2 to the pixel coordinates of the adjacent column boundary subdivision block in column 2. Linear interpolation is performed based on the equalization values ​​of column 1 and column 2, the vertical distance of column 1, and the vertical distance of column 2 to obtain the brightness value of each column boundary subdivision area.

5. The video processing method based on an FPGA platform according to claim 4, characterized in that, The method also includes: Based on the accumulated histograms of each partition in the first row of the m×n partition and the accumulated histograms of the two adjacent partitions in the middle row of the accumulated histograms of each partition in the m-th row, the equalization values ​​of row one and row two are calculated according to the equalization formula. Obtain the coordinates of the center point of row 1 and the center point of row 2 of two adjacent partitions. Calculate the horizontal distance of the center point of row 1 to the pixel coordinates of the adjacent 2m×2n subdivision block in the row boundary subdivision block. Calculate the horizontal distance of the center point of row 2 to the pixel coordinates of the adjacent row boundary subdivision block in the row 2. Linear interpolation is performed based on the equalization values ​​of row 1, row 2, the horizontal distance of row 1, and the horizontal distance of row 2 to obtain the brightness value of each row boundary subdivision.

6. The video processing method based on an FPGA platform according to claim 5, characterized in that, The method also includes: Based on the accumulated histograms of the four adjacent partitions in the accumulated histograms corresponding to each partition in the m×n partition, the first row equalization value, the second row equalization value, the same column equalization value, and the opposite column equalization value of the four adjacent partitions are calculated according to the equalization formula. Obtain the coordinates of the center point of the first row, the center point of the second row, the center point of the same column, and the center point of different columns of four adjacent partitions; Calculate the horizontal distances of the center point coordinates of the first row and the center point coordinates of the second row to the pixel coordinates of the non-boundary subdivision regions of the 2m×2n subdivision regions adjacent to the four adjacent partitions, respectively. The first brightness value is obtained by bilinear interpolation based on the first row equalization value, the second row equalization value, the first row horizontal distance, and the second row horizontal distance. Calculate the horizontal distances of the center point coordinates in the same column and the center point coordinates in different columns to the pixel coordinates in the non-boundary subdivision area of ​​the 2m×2n subdivision area adjacent to the four adjacent subdivisions, respectively, in the same column and in different columns. The second brightness value is obtained by bilinear interpolation based on the equalization values ​​in the same column, the equalization values ​​in different columns, the horizontal distance in the same column, and the horizontal distance in different columns. Calculate the vertical distances of the center point coordinates of the first row and the center point coordinates of the first column to the pixel coordinates of the non-boundary subdivision area of ​​the 2m×2n subdivision area adjacent to the four adjacent subdivisions, respectively, in the same row and column. Bilinear interpolation is performed based on the vertical distance in the same row, the vertical distance in the same column, the first brightness value, and the second brightness value to obtain the brightness value of each non-boundary subdivision area; The luminance values ​​of each vertex subdivision, each column boundary subdivision, each row boundary subdivision, and each non-boundary subdivision in the 2m×2n subdivision corresponding to each image in the camera video stream are restored using red-green-blue mode to obtain an optimized camera video stream.

7. The video processing method based on an FPGA platform according to claim 1, characterized in that, According to the i In the calculation of the first contrast threshold using the maximum and minimum values ​​from multiple brightness histograms corresponding to each partition in the row, the first contrast threshold is (480×270 / (E max - E min )); Where 480×270 represents the number of pixels in each partition, E max E represents the maximum value in the multiple brightness histograms corresponding to each partition. min This represents the minimum value among the multiple brightness histograms corresponding to each partition.

8. A video processing device based on an FPGA platform, characterized in that, include: The data acquisition module acquires the camera video stream, divides each image in the camera video stream into m×n partitions, and obtains multiple brightness components corresponding to each partition based on the m×n partition data of each image, where m and n are positive integers greater than 4. The first row of the brightness histogram processing module processes the... i Histogram statistics are performed on the multiple brightness components corresponding to each partition in the row to obtain the first... i Multiple brightness histograms corresponding to each partition in the row, and the first... i Multiple brightness histograms corresponding to each partition in the row are stored in the first memory. The first row of the cumulative histogram processing module, based on the... i The first contrast threshold is calculated by comparing the maximum and minimum values ​​in the multiple brightness histograms corresponding to each partition in the row. Each brightness histogram is then corrected based on this first contrast threshold to obtain the first... i For each partition in the row, there are multiple corrected brightness histograms corresponding to it. Based on these multiple corrected brightness histograms, the first value is calculated. i The cumulative histogram corresponding to each partition in the row, and the first... i The accumulated histogram corresponding to each partition in the row is stored in the third memory, and the data stored in the first memory is cleared. The second row of the brightness histogram processing module calculates the... i When calculating the cumulative histogram corresponding to each partition in a row, for the th... i+ Histogram statistics are performed on the multiple brightness components corresponding to each partition in row 1 to obtain the first... i+ The first row contains multiple brightness histograms corresponding to each partition, and the first row contains multiple brightness histograms corresponding to each partition. i+ The multiple brightness histograms corresponding to each partition in row 1 are stored in the second memory. The second row of the cumulative histogram processing module, based on the... i+ The second contrast threshold is calculated by combining the maximum and minimum values ​​from the multiple brightness histograms corresponding to each partition in row 1. Based on this second contrast threshold, each brightness histogram is corrected to obtain the result. i+ For each partition in row 1, there are multiple corrected brightness histograms. Based on these multiple corrected brightness histograms, the first value is calculated. i+ The cumulative histogram corresponding to each partition in row 1, and the first... i+ The accumulated histogram corresponding to each partition in row 1 is stored in the fourth memory, and the data stored in the second memory is cleared. The third data processing module, when the camera video stream enters the... i+ In line 2, time-division multiplexing is performed on the first memory and the stored data in the third memory is cleared, and the second... i+ In the second row, the multiple brightness components corresponding to each partition are sequentially subjected to histogram statistics, histogram correction, and histogram accumulation to obtain the first... i+ The cumulative histogram corresponding to each partition in the second row, and the first row... i+ The cumulative histogram corresponding to each partition in the two rows is stored in the third memory; The fourth data processing module, when the camera video stream enters the... i+ In line 3, time-division multiplexing is performed on the second memory and the stored data in the fourth memory is cleared, and the data in the third memory is... i+ Histogram statistics, histogram correction, and histogram summation are performed on the multiple brightness components corresponding to each partition in the 3rd row to obtain the first... i+ The cumulative histogram corresponding to each partition in the 3rd row, and the th row... i + The cumulative histogram corresponding to each partition in the three rows is stored in the fourth memory. The repetitive data processing module repeats the first... i Arriving at the i+ The data processing procedure for each partition in the three lines is to process m×n partitions in real time; in, i It is an integer.

9. The video processing apparatus based on an FPGA platform according to claim 8, characterized in that, The data acquisition module includes: The red, green and blue intensity acquisition module is used to acquire the red intensity value, green intensity value and blue intensity value of each pixel in each image; The intensity value comparison module is used to obtain the maximum and minimum intensity values ​​based on the red, green, and blue intensity values ​​of each pixel; The brightness component acquisition module is used to convert the red-green-blue mode of each image into a hue-saturation-brightness mode based on the red intensity value, green intensity value, blue intensity value, maximum intensity value, and minimum intensity value, to obtain multiple brightness components corresponding to each partition, and to keep the multiple hue components and multiple saturation components corresponding to each partition synchronized in time.

10. The video processing apparatus based on an FPGA platform according to claim 9, Its characteristics are The device also includes: The subdivision module is used to divide each of the m×n partitions into four subdivisions, forming a 2×2 subdivision region corresponding to each partition, and based on the 2×2 subdivision regions, divide each image in the camera video stream into 2m×2n subdivision regions. The vertex subdivision calculation module is used to calculate the vertex equalization value according to the equalization formula based on the accumulated histograms corresponding to the first partition in the first row of the m×n block partitions, the accumulated histograms corresponding to the nth partition in the first row, the accumulated histograms corresponding to the first partition in the mth row, and the accumulated histograms corresponding to the nth partition in the mth row. These values ​​are used as the brightness values ​​of each vertex subdivision located in the 2m×2n block subdivision.

11. The video processing apparatus based on an FPGA platform according to claim 10, characterized in that... The device also includes: The column equalization value acquisition module is used to calculate the equalization values ​​of column one and column two based on the accumulated histograms of each partition in the first column of m×n partitions and the accumulated histograms of two adjacent partitions in the accumulated histograms of each partition in the nth column, according to the equalization formula. The column vertical distance acquisition module is used to acquire the coordinates of the center point of column 1 and the center point of column 2 of two adjacent partitions in a column, calculate the vertical distance of the center point of column 1 to the pixel coordinates of the adjacent 2m×2n subdivision block in column boundary subdivision, and calculate the vertical distance of the center point of column 2 to the pixel coordinates of the adjacent column boundary subdivision. The column boundary subdivision calculation module is used to perform linear interpolation calculations based on the equalization values ​​of column 1 and column 2, the vertical distance of column 1, and the vertical distance of column 2 to obtain the brightness value of each column boundary subdivision.

12. The video processing apparatus based on an FPGA platform according to claim 11, characterized in that... The device also includes: The row equalization value acquisition module is used to calculate the row one equalization value and the row two equalization value based on the accumulated histogram of each partition in the first row of the m×n partition and the accumulated histogram of the two adjacent partitions in the row of the accumulated histogram of each partition in the mth row, according to the equalization formula. The horizontal distance acquisition module is used to acquire the coordinates of the center point of row 1 and the center point of row 2 of two adjacent partitions, calculate the horizontal distance of the center point of row 1 to the pixel coordinates of the adjacent 2m×2n subdivision block in the row boundary subdivision block, and calculate the horizontal distance of the center point of row 2 to the pixel coordinates of the adjacent row boundary subdivision block. The row boundary subdivision calculation module is used to perform linear interpolation calculations based on the equalization values ​​of row 1, row 2, the horizontal distance of row 1, and the horizontal distance of row 2 to obtain the brightness value of each row boundary subdivision.

13. The video processing apparatus based on an FPGA platform according to claim 12, Its characteristics are The device also includes: The row and column equalization value acquisition module is fast. It is used to calculate the first row equalization value, the second row equalization value, the same column equalization value, and the opposite column equalization value of the four adjacent partitions based on the cumulative histogram of each partition in the m×n partition. The center point coordinate acquisition module is used to acquire the center point coordinates of the first row, the second row, the same column, and different columns of four adjacent partitions. The first non-boundary horizontal distance acquisition module is used to calculate the first and second row horizontal distances from the coordinates of the center point of the first row and the center point of the second row to the coordinates of the pixel points in the non-boundary subdivision area of ​​the 2m×2n subdivision area adjacent to the four adjacent partitions, respectively. The first brightness value acquisition module is used to perform bilinear interpolation calculation based on the first row equalization value, the second row equalization value, the first row horizontal distance, and the second row horizontal distance to obtain the first brightness value; The second non-boundary horizontal distance acquisition module is used to calculate the horizontal distances of the same column center point coordinates and the opposite column center point coordinates to the pixel coordinates in the non-boundary subdivision area of ​​the 2m×2n subdivision area adjacent to the four adjacent partitions, respectively, in the same column and opposite column. The second brightness value acquisition module is used to perform bilinear interpolation calculation based on the equalization values ​​in the same column, the equalization values ​​in different columns, the horizontal distance in the same column, and the horizontal distance in different columns to obtain the second brightness value. The non-boundary vertical distance acquisition module is used to calculate the vertical distances of the first row center point coordinates and the first column center point coordinates to the pixel coordinates in the non-boundary subdivision area of ​​the 2m×2n subdivision area adjacent to the four adjacent subdivisions, respectively, in the same row and column. The non-boundary brightness value acquisition module is used to perform bilinear interpolation calculations based on the vertical distance in the same row, the vertical distance in the same column, the first brightness value, and the second brightness value to obtain the brightness value of each non-boundary subdivision area; The camera video stream acquisition module is optimized to recover the luminance values ​​of each vertex subdivision, each column boundary subdivision, each row boundary subdivision, and each non-boundary subdivision of each 2m×2n subdivision corresponding to each image in the camera video stream using red-green-blue mode, thereby obtaining the optimized camera video stream.

14. The video processing apparatus based on an FPGA platform according to claim 8, characterized in that... In the first row of the cumulative histogram processing module, the first contrast threshold is (480×270 / (E max - E min )); in, 480×270 represents the number of pixels in each partition, E max E represents the maximum value in the multiple brightness histograms corresponding to each partition. min This represents the minimum value among the multiple brightness histograms corresponding to each partition.

Images