Video image real-time edge detection system based on FPGA

Abstract

The invention belongs to the field of digital image processing, and discloses an FPGA-based video image real-time edge detection system, which comprises an OV5640 image sensor, an image acquisition module, an image processing module, an image caching module, an image transmission control module and a display device. Wherein the OV5640 is responsible for collecting digital image data in real time; the image acquisition module is mainly used for completing initial configuration of an image sensor and receiving image data according to a data protocol format; the image processing module is used for realizing graying processing, filtering operation and edge detection operation of the image; the image caching module is used for caching image data and writing the cached data into an SDRAM (Synchronous Dynamic Random Access Memory); and the image transmission control module transmits the image data to display equipment for real-time display by means of a data transmission protocol. The invention aims to improve the performance of a video image edge detection system, improve the real-time performance and the operational capability of the system and reduce the occupancy rate of FPGA (Field Programmable Gate Array) logic resources.

Description

Technical Field

[0001] The invention relates to the field of digital image processing, and in particular to a video image real-time edge detection system based on FPGA. Background Art

[0002] Edges are areas in an image where the grayscale or structure changes significantly. By calculating the gradient of the grayscale distribution, the edges in the image can be determined. Edge detection is an image segmentation technology that aims to extract the edges and contours of objects in an image and highlight the details in the image. In practical applications, edge detection technology can be applied to weld detection, digital recognition, and recognition of road boundaries in unmanned driving. This technology plays an indispensable role in image processing, machine vision, feature detection, extraction, and other fields.

[0003] Edge detection is a basic technology in the field of digital image processing and machine vision. As image sensors and image data develop towards high resolution and high frame rate, the requirements for the real-time performance, processing speed, and processing effect of edge detection algorithms are gradually increasing. At present, edge detection algorithms are usually implemented on software platforms in a traditional way, and cannot be applied to scenarios with high requirements for real-time performance and power consumption. For example, conventional single-chip microcomputers, CPUs, etc., which are serial architectures, make it difficult to achieve real-time processing of high-frame-rate, high-resolution digital video streams, which can easily lead to frame loss, interlaced images, and long processing time. Therefore, how to improve real-time performance and reduce power consumption while ensuring the processing effect of edge detection algorithms is a key research issue in this field. Summary of the invention

[0004] In order to solve the above problems in this field, the present invention proposes a real-time edge detection system for video images based on FPGA, which realizes real-time edge detection of high frame rate and high resolution digital video streams, thereby meeting the real-time and reliability requirements of edge detection algorithms.

[0005] The technical solution of the present invention is as follows: A real-time edge detection system for video images based on FPGA, comprising:

[0006] OV5640 image sensor, image acquisition module, image processing module, image cache module, image transmission control module and display device;

[0007] The OV5640 image sensor is used to collect digital image data in real time and transmit the digital image data to the image acquisition module;

[0008] The image acquisition module is used to receive the digital image data collected by the OV5640 image sensor according to a specified data protocol format and transmit it to the image processing module;

[0009] The image processing module includes a grayscale processing module, a Gaussian filtering module and a Sobel edge detection module, which are used to complete grayscale conversion, noise reduction and edge detection of digital image data;

[0010] The image cache module implements image data caching by writing the digital image data processed by the image processing module into SDRAM;

[0011] The image transmission control module is used to transmit the digital image data of the image buffer module to the display device according to the data format supported by the display device;

[0012] The display device is used to display the processed image data in real time.

[0013] The specified data protocol format is the IIC protocol; the image acquisition module initializes and configures the OV5640 image sensor according to the IIC protocol, so that the OV5640 image sensor outputs image data in the RGB565 format; after the initialization is completed, the image acquisition module collects data according to the specified timing, and converts the 8-bit data into 16-bit RGB565 format data through the serial-to-parallel conversion circuit.

[0014] The specified timing is that the rising edge of the frame synchronization signal represents the start signal of each frame of the image, and the transmission of a frame of the image is completed between two rising edges; when the line synchronization signal becomes a high level, the OV5640 image sensor starts to output valid image data; in the RGB565 output format, each pixel point consists of 16 bits of data, and one pixel point is output in two pixel clock cycles.

[0015] The image acquisition module includes an IIC configuration module, an IIC driver module and a data acquisition module; the IIC configuration module is used to store the address and data of the register to be configured of the OV5640 image sensor, and configure the register through the case statement, and also complete the control of the initialization process; the IIC driver module is used to drive the IIC bus and provide a clock signal for the IIC configuration module;

[0016] The data acquisition module acquires image data according to a previously set data format.

[0017] In the image processing module, a grayscale processing module, a Gaussian filtering module and a Sobel edge detection module are sequentially connected, and a 3×3 pixel matrix buffer module is added.

[0018] The grayscale processing module converts the RGB565 format data into the Ycbcr format for the digital video system, retaining 8 bits of grayscale data;

[0019] The grayscale conversion formula is shifted and enlarged, and after the conversion is completed, it is shifted and reduced to obtain a grayscale image; the final grayscale value formula is shown in formula (1):

[0020] Y=(77R+150G+29B)>>8 (1)

[0021] Among them, Y represents the grayscale component in the YCbCr format; R, G, and B represent the red, green, and blue components in the RGB565 format respectively; ">>" represents a right shift operation, and ">>8" means shifting the result right by 8 bits.

[0022] The pixel matrix cache module is a dual-end RAM IP core, including RAM1 and RAM0, which are used to cache the pixel data of the first two rows respectively, and the latest written data is used as the current row; after the data of the current two rows are read, the data of the second row is written to the first row, and then the data of the current row is written to the second row, so as to realize the dynamic update of the data in the RAM.

[0023] The Gaussian filter module is used for:

[0024] Generate pixel data window;

[0025] Gaussian filtering is performed on the digital image data after grayscale conversion to suppress the noise signal in the image.

[0026] The pixel data window is convolved with the Gaussian filter module. Driven by the clock signal and the clock enable signal, the operation is completed in two clock cycles. The operation result is right-shifted by 4 bits in another clock cycle to obtain the final pixel value, which is then transmitted to the Sobel edge detection module.

[0027] The Sobel edge detection module is used to:

[0028] Set the convolution template in the X and Y directions;

[0029] Calculate the gradient value of the pixel point; first calculate the gradient components in the X and Y directions; calculate the sum of the squares of the gradient components in the X and Y directions; calculate the pixel point gradient size by calling the ALTSQRT IP core to perform square root;

[0030] The calculated gradient value is compared with the preset threshold to determine whether it is an edge; when the pixel point gradient size is less than the set threshold, the edge detection flag is set to 0 and the point is determined not to be an edge pixel; when the pixel point gradient size is greater than the set threshold, the point is determined to be an edge pixel and the edge detection flag is set to 1.

[0031] The beneficial effects of the present invention are as follows: The present invention provides a real-time edge detection system for video images based on FPGA, which can collect image information in real time through the OV5640 image sensor, and can realize the function of real-time edge detection with low power consumption and logic resource occupancy rate, and finally can display the final processing result in real time through the display device. The present invention effectively improves the real-time performance of the edge detection system and reduces the hardware resource occupancy rate, and can be applied to high-resolution, high-frame-rate, and high-real-time application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is an overall structural diagram of the real-time edge detection system for video images based on FPGA provided by the present invention;

[0033] Figure 2 This is the row-field timing diagram of the OV5640 image sensor;

[0034] Figure 3 It is a workflow diagram of the image processing module in the present invention;

[0035] Figure 4 It is a schematic diagram of the working principle of grayscale conversion;

[0036] Figure 5 It is a schematic diagram of the structural principle of the pixel matrix cache;

[0037] Figure 6 This is the overall workflow diagram of Sobel edge detection;

[0038] Figure 7 This is the functional structure diagram of the SDRAM controller module. DETAILED DESCRIPTION

[0039] The present invention will be further explained and illustrated in conjunction with the accompanying drawings.

[0040] like Figure 1 and Figure 3 As shown, the present invention relates to a real-time edge detection system for video images based on FPGA, comprising: an OV5640 image sensor, an image acquisition module, an image processing module, an image cache module, an image transmission control module and a display device. The image processing module comprises a grayscale processing module, a Gaussian filtering module and a Sobel edge detection module.

[0041] The OV5640 image sensor is used to collect digital image data in real time and transmit the data to the image acquisition module;

[0042] The image acquisition module is used to receive image data acquired by the OV5640 image sensor according to a specified data protocol format;

[0043] The image processing module includes a grayscale processing module, a Gaussian filtering module and a Sobel edge detection module, which are used to complete grayscale conversion, noise reduction and edge detection of the input image;

[0044] The image cache module includes an SDRAM FIFO control module and an SDRAM controller module, and implements image data caching by writing image data into SDRAM;

[0045] The image transmission control module is used to transmit the processed image data to the display device in a data format supported by the display device;

[0046] The display device is used to display the processed image data in real time.

[0047] The grayscale processing module is used to convert the data in RGB565 format into the Ycbcr format suitable for the digital video system, retaining 8 bits of grayscale data. The grayscale conversion formula is shifted and enlarged, and after the conversion is completed, it is shifted and reduced to obtain a grayscale image.

[0048] The Gaussian filter module is used to generate a pixel data window; perform Gaussian filtering on the input image data to suppress noise signals in the image;

[0049] The pixel data window is a 3×3 window; a pixel data window of corresponding size is generated by calling the RAM IP core;

[0050] The Sobel edge detection module is used to set the convolution template in the X direction and the Y direction; calculate the gradient value of the pixel point; compare the calculated gradient value with a preset threshold value to determine whether it is an edge.

[0051] The image cache module controls the processes of SDRAM power-on initialization, refresh, read and write operations through a three-stage linear state machine, and includes an SDRAM FIFO control module and an SDRAM controller module. The SDRAM FIFO control module encapsulates the SDRAM as a FIFO interface; the SDRAM controller module uses a state machine to control the working state of the SDRAM.

[0052] The SDRAM FIFO control module calls the asynchronous FIFO IP core of the Quartus platform; converts the row and column addresses of the SDRAM into 24-bit linear addresses; and opens two storage spaces of the same size in the SDRAM to cache image data to realize ping-pong operation.

[0053] The SDRAM controller module includes three submodules, namely, an SDRAM state control module, an SDRAM data module and an SDRAM instruction module.

[0054] The SDRAM state control module realizes the control of the SDRAM power-on initialization process and the control of the SDRAM refresh, read and write process respectively through two three-segment linear state machines.

[0055] The SDRAM instruction module receives the initialization state and working state information sent by the SDRAM state control module, and assigns instruction control signals corresponding to different states to the signal lines and address lines of the SDRAM according to the instruction table in the SDRAM data manual, and outputs control instructions to the SDRAM.

[0056] The SDRAM data module controls the bidirectional data bus of the SDRAM according to the status information sent by the SDRAM status control module. When in the read operation state, it stores the data read from the SDRAM; when in the write operation state, it outputs the data to be written to the SDRAM data bus.

[0057] In the present invention, the system first collects video image information through the OV5640 image sensor and transmits it to the image acquisition module. After the system is powered on, the image acquisition module initializes and configures the OV5640 according to the IIC protocol, so that the sensor outputs image data in the RGB565 format. After the initialization is completed, the image acquisition module Figure 2 The OV5640 shown collects data in timing and converts the 8-bit data into 16-bit RGB565 format data through a serial-to-parallel conversion circuit.

[0058] The structure diagram of the image processing module is as follows: Figure 3 As shown in Figure 1, it includes grayscale processing, pixel matrix cache, Gaussian filtering, Sobel edge detection and other links. First, the grayscale processing module follows Figure 4 The method shown converts the RGB565 format data into 8-bit grayscale data. Considering that Verilog does not support floating-point operations, in order to avoid occupying too many hardware resources, the grayscale value data is first expanded by 256 times, rounded to the nearest integer, and then right-shifted by 8 bits. The final grayscale value formula is shown in formula (1):

[0059] Y=(77R+150G+29B)>>8 (1)

[0060] Before performing Gaussian filtering or edge detection, the pixel matrix cache is implemented by calling the dual-end RAM IP core. The specific structural principle is as follows Figure 5 As shown. RAM1 and RAM0 are used to cache the pixel data of the first two rows respectively, and the data just written is used as the current row. After the data of the current two rows are read, the data of the second row is written to the first row, and then the data of the current row is written to the second row, so as to realize the dynamic update of the data in RAM.

[0061] In order to improve the accuracy of edge detection, a Gaussian filter is used to suppress noise signals before the edge detection. In the present invention, a Gaussian filter template with σ=0.8 is used, as shown in the following formula:

[0062]

[0063] By convolving the pixel matrix with the Gaussian filter template, we can get the following formula:

[0064]

[0065] Where f(x-1, y-1) to f(x+1, y+1) represent the pixels in the 3×3 pixel matrix. Driven by the clock signal and the clock enable signal, all operations in the brackets in the formula are completed in two clock cycles. Then, the operation result is shifted right by 4 bits in another clock cycle to obtain the final pixel value, which is then transmitted to the Sobel edge detection module.

[0066] The overall workflow of the Sobel edge detection module can be divided into five steps: Figure 6 shown.

[0067] First, calculate the gradient components in the X and Y directions; calculate the sum of the squares of the gradient components in the X and Y directions; calculate the size of the pixel gradient by calling the ALTSQRT IP core to perform square root; finally, compare the result with the preset threshold. When the result value is less than the set threshold, the edge detection flag is set to 0, and the point is determined not to be an edge pixel; when the result value is greater than the set threshold, the point is determined to be an edge pixel, and the edge detection flag is set to 1. In practical applications, the threshold should be moderate. If the value is too large, the image edge will be lost and discontinuous; if the value is too small, more pseudo edges will be formed.

[0068] The image cache module uses a state machine to complete the control of a series of operations such as SDRAM initialization, data reading and writing. The specific functional structure diagram is as follows Figure 7 shown.

[0069] The image transmission control module outputs corresponding control signals (such as backlight signals, horizontal and vertical synchronization signals, etc.) and 16-bit pixel data according to the display timing of the display device, and provides coordinate information and pixel data information corresponding to the pixel points.

[0070] In the present invention, FPGA is used as a development platform and Intel Cyclone IV series development board is used to perform real-time grayscale processing, Gaussian filtering and edge detection on a 50-frame digital video stream with a resolution of 640×480, and finally display it in real time through a display device.

Claims

1. A real-time edge detection system for video images based on FPGA, characterized in that: include: OV5640 image sensor, image acquisition module, image processing module, image cache module, image transmission control module and display device; The OV5640 image sensor is used to collect digital image data in real time and transmit the digital image data to the image acquisition module; The image acquisition module is used to receive the digital image data collected by the OV5640 image sensor according to a specified data protocol format and transmit it to the image processing module; The image processing module includes a grayscale processing module, a Gaussian filtering module and a Sobel edge detection module, which are used to complete grayscale conversion, noise reduction and edge detection of digital image data; The image cache module implements image data caching by writing the digital image data processed by the image processing module into SDRAM; The image transmission control module is used to transmit the digital image data of the image buffer module to the display device according to the data format supported by the display device; The display device is used to display the processed image data in real time.

2. The FPGA-based real-time edge detection system for video images according to claim 1, characterized in that: The specified data protocol format is the IIC protocol; the image acquisition module initializes and configures the OV5640 image sensor according to the IIC protocol, so that the OV5640 image sensor outputs image data in the RGB565 format; After initialization, the image acquisition module collects data according to the specified timing and converts the 8-bit data into 16-bit RGB565 format data through the serial-to-parallel conversion circuit.

3. The FPGA-based real-time edge detection system for video images according to claim 2, characterized in that: The specified timing is that the rising edge of the frame synchronization signal represents the start signal of each frame of the image, and the transmission of a frame of the image is completed between two rising edges; when the line synchronization signal becomes a high level, the OV5640 image sensor starts to output valid image data; in the RGB565 output format, each pixel point consists of 16 bits of data, and one pixel point is output in two pixel clock cycles.

4. The FPGA-based real-time edge detection system for video images according to claim 2, characterized in that: The image acquisition module includes an IIC configuration module, an IIC driver module and a data acquisition module; the IIC configuration module is used to store the address and data of the register to be configured of the OV5640 image sensor, and configure the register through the case statement, and also complete the control of the initialization process; the IIC driver module is used to drive the IIC bus and provide a clock signal for the IIC configuration module; The data acquisition module acquires image data according to a previously set data format.

5. The FPGA-based real-time edge detection system for video images according to claim 1, characterized in that: In the image processing module, a grayscale processing module, a Gaussian filtering module and a Sobel edge detection module are sequentially connected, and a 3×3 pixel matrix buffer module is added.

6. The FPGA-based real-time edge detection system for video images according to claim 5, characterized in that: The grayscale processing module converts the RGB565 format data into the Ycbcr format for the digital video system, retaining 8 bits of grayscale data; The grayscale conversion formula is shifted and enlarged, and after the conversion is completed, it is shifted and reduced to obtain a grayscale image; the final grayscale value formula is shown in formula (1): Y=(77R+150G+29B)>>8 (1) Among them, Y represents the grayscale component in the YCbCr format; R, G, and B represent the red, green, and blue components in the RGB565 format respectively; ">>" represents a right shift operation, and ">>8" means shifting the result right by 8 bits.

7. The FPGA-based real-time edge detection system for video images according to claim 5, characterized in that: The pixel matrix cache module is a dual-end RAM IP core, including RAM1 and RAM0, which are used to cache the pixel data of the first two rows respectively, and the latest written data is used as the current row; After the data of the current two rows are read, the data of the second row is written into the first row, and then the data of the current row is written into the second row, so as to realize the dynamic update of the data in the RAM.

8. The FPGA-based real-time edge detection system for video images according to claim 1, characterized in that: The Gaussian filter module is used for: Generate pixel data window; Gaussian filtering is performed on the digital image data after grayscale conversion to suppress the noise signal in the image.

9. The FPGA-based real-time edge detection system for video images according to claim 8, characterized in that: The pixel data window is convolved with the Gaussian filter module. Driven by the clock signal and the clock enable signal, the operation is completed in two clock cycles. The operation result is right-shifted by 4 bits in another clock cycle to obtain the final pixel value, which is then transmitted to the Sobel edge detection module.

10. The FPGA-based real-time edge detection system for video images according to claim 8, characterized in that: The Sobel edge detection module is used to: Set the convolution template in the X and Y directions; Calculate the gradient value of the pixel; first calculate the gradient components in the X and Y directions; Calculate the sum of the squares of the gradient components in the X and Y directions; perform square root extraction by calling the ALTSQRT IP core to calculate the pixel gradient size; The calculated gradient value is compared with the preset threshold to determine whether it is an edge; when the pixel point gradient size is less than the set threshold, the edge detection flag is set to 0 and the point is determined not to be an edge pixel; when the pixel point gradient size is greater than the set threshold, the point is determined to be an edge pixel and the edge detection flag is set to 1.

Images