Video image real-time edge detection system based on FPGA

Abstract

The invention belongs to the field of digital image processing, and discloses an FPGA-based video image real-time edge detection system, which comprises an OV5640 image sensor, an image acquisition module, an image processing module, an image caching module, an image transmission control module and a display device. Wherein the OV5640 is responsible for collecting digital image data in real time; the image acquisition module is mainly used for completing initial configuration of an image sensor and receiving image data according to a data protocol format; the image processing module is used for realizing graying processing, filtering operation and edge detection operation of the image; the image caching module is used for caching image data and writing the cached data into an SDRAM (Synchronous Dynamic Random Access Memory); and the image transmission control module transmits the image data to display equipment for real-time display by means of a data transmission protocol. The invention aims to improve the performance of a video image edge detection system, improve the real-time performance and the operational capability of the system and reduce the occupancy rate of FPGA (Field Programmable Gate Array) logic resources.

Description

Technical Field

[0001] This invention relates to the field of digital image processing, and more particularly to a real-time edge detection system for video images based on FPGA. Background Technology

[0002] Edges are regions in an image where there are significant changes in grayscale levels or structure. By calculating the gradient of the grayscale distribution, edges in an image can be identified. Edge detection is an image segmentation technique designed to extract the edges and contours of objects in an image, highlighting details. In practical applications, edge detection technology can be used for weld inspection, digital recognition, and road boundary recognition in autonomous driving, playing an indispensable role in image processing, machine vision, and feature detection and extraction.

[0003] Edge detection is a fundamental technology in digital image processing and machine vision. As image sensors and image data evolve towards higher resolution and higher frame rates, the demands on edge detection algorithms for real-time performance, processing speed, and processing quality are constantly increasing. Currently, edge detection algorithms are typically implemented on software platforms using traditional methods, making them unsuitable for scenarios with high requirements for real-time performance and power efficiency. For example, conventional microcontrollers and CPUs, with their serial architecture, struggle to handle high frame rates and high resolution digital video streams in real-time, easily leading to frame drops, image interlacing, and excessively long processing times. Therefore, improving real-time performance and reducing power consumption while maintaining the processing quality of edge detection algorithms is a key research challenge in this field. Summary of the Invention

[0004] To address the aforementioned problems in this field, this invention proposes a real-time edge detection system for video images based on FPGA, which enables real-time edge detection of high frame rate and high resolution digital video streams, thereby meeting the requirements for real-time performance and reliability of edge detection algorithms.

[0005] The technical solution of the present invention is as follows: A real-time edge detection system for video images based on FPGA, comprising:

[0006] OV5640 image sensor, image acquisition module, image processing module, image buffer module, image transmission control module, and display device;

[0007] The OV5640 image sensor is used to acquire digital image data in real time and transmit the digital image data to the image acquisition module;

[0008] The image acquisition module is used to receive digital image data acquired by the OV5640 image sensor according to a specified data protocol format, and transmit it to the image processing module.

[0009] The image processing module includes a grayscale processing module, a Gaussian filtering module, and a Sobel edge detection module, which are used to perform grayscale conversion, noise reduction, and edge detection on digital image data.

[0010] The image caching module caches image data by writing the digital image data processed by the image processing module into SDRAM;

[0011] The image transmission control module is used to transmit the digital image data from the image buffer module to the display device in accordance with the data format supported by the display device.

[0012] The display device is used to display the processed image data in real time.

[0013] The specified data protocol format is the IIC protocol; the image acquisition module initializes and configures the OV5640 image sensor according to the IIC protocol, so that the OV5640 image sensor outputs image data in RGB565 format; after initialization, the image acquisition module acquires data according to the specified timing sequence, and converts the 8-bit data into 16-bit RGB565 format data through a serial-to-parallel conversion circuit.

[0014] The specified timing is as follows: the rising edge of the frame synchronization signal represents the start signal of each frame of image, and the transmission of one frame of image is completed between two rising edges; when the line synchronization signal goes high, the OV5640 image sensor starts to output valid image data; in the RGB565 output format, each pixel consists of 16 bits of data, and one pixel is output in two pixel clock cycles.

[0015] The image acquisition module includes an IIC configuration module, an IIC driver module, and a data acquisition module. The IIC configuration module stores the addresses and data of the OV5640 image sensor's registers to be configured, configures the registers using case statements, and controls the initialization process. The IIC driver module drives the IIC bus and provides a clock signal to the IIC configuration module.

[0016] The data acquisition module acquires image data according to the previously set data format.

[0017] In the image processing module, a 3×3 pixel matrix buffer module is added between the grayscale processing module, the Gaussian filtering module, and the Sobel edge detection module, which are connected in sequence.

[0018] The grayscale processing module converts RGB565 format data into Ycbcr format for digital video systems, retaining 8 bits of grayscale data;

[0019] The grayscale conversion formula is shifted and amplified, and after conversion, it is shifted and reduced to obtain a grayscale image; the final grayscale value formula is shown in equation (1):

[0020] (1)

[0021] Where Y represents the grayscale component in the YCbCr format; R, G, and B represent the red, green, and blue components in the RGB565 format, respectively; ">>" indicates a right shift operation, and ">>8" means shifting the result 8 bits to the right.

[0022] The pixel matrix caching module is a dual-ended RAM IP core, including RAM1 and RAM0, which are used to cache the pixel data of the first two rows respectively, and the latest written data is used as the current row. After the data of the first two rows is read, the data of the second row is written to the first row, and then the data of the current row is written to the second row to realize the dynamic update of the data in RAM.

[0023] The Gaussian filter module is used for:

[0024] Generate pixel data window;

[0025] Gaussian filtering is applied to the grayscale-converted digital image data to suppress noise signals in the image.

[0026] The pixel data window is convolved with the Gaussian filter module. Driven by the clock signal and clock enable signal, the operation is completed in two clock cycles. Then, the result is shifted right by 4 bits in one clock cycle to obtain the final pixel value, which is then transmitted to the Sobel edge detection module.

[0027] The Sobel edge detection module is used for:

[0028] Set the convolution templates in the X and Y directions;

[0029] Calculate the gradient value of a pixel; first, calculate the gradient components in the X and Y directions; then calculate the sum of squares of the gradient components in the X and Y directions; finally, calculate the magnitude of the pixel gradient by taking the square root of the ALTSQRT IP core.

[0030] The calculated gradient value is compared with a preset threshold to determine whether it is an edge. When the gradient value of a pixel is less than the preset threshold, the edge detection flag is set to 0, and the pixel is determined not to be an edge pixel. When the gradient value of a pixel is greater than the preset threshold, the pixel is determined to be an edge pixel, and the edge detection flag is set to 1.

[0031] The beneficial effects of this invention are as follows: This invention provides a real-time edge detection system for video images based on FPGA. This system can acquire image information in real time using an OV5640 image sensor and achieve real-time edge detection with low power consumption and low logic resource usage. The final processing results can be displayed in real time via a display device. This invention effectively improves the real-time performance of the edge detection system and reduces hardware resource usage, making it suitable for applications requiring high resolution, high frame rates, and high real-time performance. Attached Figure Description

[0032] Figure 1 This is an overall structural diagram of the FPGA-based real-time edge detection system for video images provided by the present invention;

[0033] Figure 2 This is the line and field timing diagram of the OV5640 image sensor;

[0034] Figure 3 This is a flowchart of the image processing module in this invention;

[0035] Figure 4 This is a schematic diagram illustrating the working principle of grayscale conversion;

[0036] Figure 5 This is a schematic diagram illustrating the structural principle of a pixel matrix cache.

[0037] Figure 6 This is a flowchart of the overall workflow of Sobel edge detection;

[0038] Figure 7 This is a functional structure diagram of the SDRAM controller module. Detailed Implementation

[0039] The present invention will be further explained and described with reference to the accompanying drawings.

[0040] like Figure 1 and Figure 3 As shown, this invention relates to a real-time edge detection system for video images based on FPGA, comprising: an OV5640 image sensor, an image acquisition module, an image processing module, an image buffer module, an image transmission control module, and a display device. The image processing module includes a grayscale processing module, a Gaussian filtering module, and a Sobel edge detection module.

[0041] The OV5640 image sensor is used to acquire digital image data in real time and transmit the data to the image acquisition module.

[0042] The image acquisition module is used to receive image data acquired by the OV5640 image sensor in accordance with a specified data protocol format;

[0043] The image processing module includes a grayscale processing module, a Gaussian filtering module, and a Sobel edge detection module, which are used to perform grayscale conversion, noise reduction, and edge detection on the input image.

[0044] The image caching module includes an SDRAM FIFO control module and an SDRAM controller module, which caches image data by writing image data into SDRAM.

[0045] The image transmission control module is used to transmit the processed image data to the display device in accordance with the data format supported by the display device;

[0046] The display device is used to display the processed image data in real time.

[0047] The grayscale processing module converts RGB565 format data into Ycbcr format suitable for digital video systems, retaining 8 bits of grayscale data. The grayscale conversion formula is shifted and amplified, and then shifted and reduced again to obtain the grayscale image.

[0048] The Gaussian filtering module is used to generate a pixel data window; it performs Gaussian filtering on the input image data to suppress noise signals in the image.

[0049] The pixel data window is a 3×3 window; the corresponding pixel data window is generated by calling the RAM IP core;

[0050] The Sobel edge detection module is used to set convolution templates in the X and Y directions; calculate the gradient value of each pixel; and compare the calculated gradient value with a pre-set threshold to determine whether it is an edge.

[0051] The image buffer module controls the SDRAM power-on initialization, refresh, read / write operations, and other processes through a three-stage linear state machine. It includes an SDRAM FIFO control module and an SDRAM controller module. The SDRAM FIFO control module encapsulates the SDRAM as a FIFO interface; the SDRAM controller module uses a state machine to control the SDRAM's operating state.

[0052] The SDRAM FIFO control module calls the asynchronous FIFO IP core of the Quartus platform; converts the row and column addresses of the SDRAM into 24-bit linear addresses; and allocates two storage spaces of the same size in the SDRAM to cache image data, thereby realizing ping-pong operation.

[0053] The SDRAM controller module comprises three sub-modules: an SDRAM status control module, an SDRAM data module, and an SDRAM instruction module.

[0054] The SDRAM state control module uses two three-stage linear state machines to control the SDRAM power-on initialization process and the SDRAM refresh and read / write processes, respectively.

[0055] The SDRAM instruction module receives initialization and operating status information from the SDRAM status control module, and assigns instruction control signals corresponding to different states to the signal lines and address lines of the SDRAM according to the instruction table in the SDRAM datasheet, and outputs control instructions to the SDRAM.

[0056] The SDRAM data module controls the bidirectional data bus of the SDRAM based on the status information issued by the SDRAM status control module. When in read operation mode, it registers the data read from the SDRAM; when in write operation mode, it outputs the data to be written to the SDRAM data bus.

[0057] In this invention, the system first acquires video image information using an OV5640 image sensor and transmits it to the image acquisition module. After the system is powered on, the image acquisition module initializes and configures the OV5640 according to the IIC protocol, causing the sensor to output image data in RGB565 format. After initialization is complete, the image acquisition module proceeds as follows... Figure 2 The OV5640 timing data is shown, and the 8-bit data is converted into 16-bit RGB565 format data through a serial-to-parallel conversion circuit.

[0058] The structure diagram of the image processing module is as follows: Figure 3 As shown, it includes grayscale processing, pixel matrix caching, Gaussian filtering, and Sobel edge detection. First, the grayscale processing module follows... Figure 4 The method shown converts RGB565 format data into 8-bit grayscale data. Considering that Verilog does not support floating-point operations, in order to avoid consuming too many hardware resources, the grayscale data is first multiplied by 256, rounded to the nearest integer, and then shifted right by 8 bits. The final grayscale value formula is shown in equation (1):

[0059] (1)

[0060] Before performing Gaussian filtering or edge detection, a pixel matrix buffer is implemented by calling a dual-ended RAM IP core. The specific structural principle is as follows: Figure 5 As shown, RAM1 and RAM0 are used to cache the pixel data of the first two rows, respectively, with the data just written serving as the current row. After the data of the first two rows is read, the data of the second row is written to the first row, and then the data of the current row is written to the second row, thus realizing dynamic updating of the data in RAM.

[0061] To improve the accuracy of edge detection, a Gaussian filter is used to suppress noise signals beforehand. In this invention, a Gaussian filter template with σ=0.8 is used, as shown in the formula below:

[0062] (2)

[0063] Convolving the pixel matrix with the Gaussian filter template yields the following formula:

[0064] (3)

[0065] in ~ This represents the individual pixels in a 3×3 pixel matrix. Driven by the clock signal and clock enable signal, all calculations within the curly braces in the formula are completed in two clock cycles. Subsequently, the calculation result is shifted right by 4 bits in one clock cycle to obtain the final pixel value, which is then transmitted to the Sobel edge detection module.

[0066] The overall workflow of Sobel's edge detection module can be divided into 5 steps, such as... Figure 6 As shown.

[0067] First, the gradient components in the X and Y directions are calculated. Then, the sum of the squares of the X and Y gradient components is calculated. The square root of the gradient at each pixel is then calculated using the ALTSQRT IP kernel. Finally, the result is compared to a pre-set threshold. If the result is less than the threshold, the edge detection flag is set to 0, indicating that the pixel is not an edge pixel. If the result is greater than the threshold, the pixel is considered an edge pixel, and the edge detection flag is set to 1. In practical applications, the threshold should be appropriately set. If the threshold is too large, it will cause image edges to be lost or discontinuous; if the threshold is too small, it will create many false edges.

[0068] The image buffer module uses a state machine to control a series of operations such as SDRAM initialization and data reading / writing. The specific functional structure diagram is shown below. Figure 7 As shown.

[0069] The image transmission control module outputs corresponding control signals (such as backlight signals, horizontal and vertical synchronization signals, etc.) and 16-bit pixel data according to the display timing of the display device, and provides the coordinate information and pixel data information corresponding to the pixel.

[0070] In this invention, an FPGA is used as the development platform, and an Intel Cyclone IV series development board is used to perform real-time grayscale processing, Gaussian filtering, and edge detection on a 50-frame digital video stream with a resolution of 640×480. Finally, the video is displayed in real time through a display device.

Claims

1. A real-time edge detection system for video images based on FPGA, characterized in that, include: OV5640 image sensor, image acquisition module, image processing module, image buffer module, image transmission control module, and display device; The OV5640 image sensor is used to acquire digital image data in real time and transmit the digital image data to the image acquisition module; The image acquisition module is used to receive digital image data acquired by the OV5640 image sensor according to a specified data protocol format, and transmit it to the image processing module. The image processing module includes a grayscale processing module, a Gaussian filtering module, and a Sobel edge detection module, which are used to perform grayscale conversion, noise reduction, and edge detection on digital image data. The image caching module caches image data by writing the digital image data processed by the image processing module into SDRAM; The image transmission control module is used to transmit the digital image data from the image buffer module to the display device in accordance with the data format supported by the display device. The display device is used to display the processed image data in real time.

2. The FPGA-based real-time edge detection system for video images according to claim 1, characterized in that, The specified data protocol format is the IIC protocol; the image acquisition module initializes and configures the OV5640 image sensor according to the IIC protocol, so that the OV5640 image sensor outputs image data in RGB565 format; After initialization, the image acquisition module acquires data according to the specified timing and converts the 8-bit data into 16-bit RGB565 format data through a serial-to-parallel conversion circuit.

3. The FPGA-based real-time edge detection system for video images according to claim 2, characterized in that, The specified timing is as follows: the rising edge of the frame synchronization signal represents the start signal of each frame of image, and the transmission of one frame of image is completed between two rising edges; when the line synchronization signal goes high, the OV5640 image sensor starts to output valid image data; in the RGB565 output format, each pixel consists of 16 bits of data, and one pixel is output in two pixel clock cycles.

4. The FPGA-based real-time edge detection system for video images according to claim 2, characterized in that, The image acquisition module includes an IIC configuration module, an IIC driver module, and a data acquisition module. The IIC configuration module stores the addresses and data of the OV5640 image sensor's registers to be configured, configures the registers using case statements, and controls the initialization process. The IIC driver module drives the IIC bus and provides a clock signal to the IIC configuration module. The data acquisition module acquires image data according to the previously set data format.

5. The FPGA-based real-time edge detection system for video images according to claim 1, characterized in that, In the image processing module, a 3×3 pixel matrix buffer module is added between the grayscale processing module, the Gaussian filtering module, and the Sobel edge detection module, which are connected in sequence.

6. The FPGA-based real-time edge detection system for video images according to claim 5, characterized in that, The grayscale processing module converts RGB565 format data into Ycbcr format for digital video systems, retaining 8 bits of grayscale data; The grayscale conversion formula is shifted and amplified, and after conversion, it is shifted and reduced to obtain a grayscale image; the final grayscale value formula is shown in equation (1): （1）； Where Y represents the grayscale component in the YCbCr format; R, G, and B represent the red, green, and blue components in the RGB565 format, respectively; ">>" indicates a right shift operation, and ">>8" means shifting the result 8 bits to the right.

7. The FPGA-based real-time edge detection system for video images according to claim 5, characterized in that, The pixel matrix caching module is a dual-ended RAM IP core, including RAM1 and RAM0, which are used to cache the pixel data of the first two rows respectively, and the latest written data is used as the current row; After the data in the first two rows is read, the data in the second row is written into the first row, and then the data in the current row is written into the second row, so as to realize the dynamic update of data in RAM.

8. The FPGA-based real-time edge detection system for video images according to claim 1, characterized in that, The Gaussian filter module is used for: Generate pixel data window; Gaussian filtering is applied to the grayscale-converted digital image data to suppress noise signals in the image.

9. The FPGA-based real-time edge detection system for video images according to claim 8, characterized in that, The pixel data window is convolved with the Gaussian filter module. Driven by the clock signal and clock enable signal, the operation is completed in two clock cycles. Then, the result is shifted right by 4 bits in one clock cycle to obtain the final pixel value, which is then transmitted to the Sobel edge detection module.

10. The FPGA-based real-time edge detection system for video images according to claim 8, characterized in that, The Sobel edge detection module is used for: Set the convolution templates in the X and Y directions; Calculate the gradient value of each pixel; first, calculate the gradient components in the X and Y directions; Calculate the sum of squares of the gradient components in the X and Y directions; calculate the gradient magnitude of the pixel by taking the square root of the ALTSQRT IP kernel. The calculated gradient value is compared with a preset threshold to determine whether it is an edge. When the gradient value of a pixel is less than the preset threshold, the edge detection flag is set to 0, and the pixel is determined not to be an edge pixel. When the gradient value of a pixel is greater than the preset threshold, the pixel is determined to be an edge pixel, and the edge detection flag is set to 1.

Images