Image edge processing method and device based on FPGA acceleration

Abstract

The invention discloses an image edge processing method and device based on FPGA acceleration. The method comprises the steps that image preprocessing is carried out, image data obtained from an external interface are obtained, each pixel RGB value of the image data is converted into a gray value according to needs, and a converted gray image is stored in an FPGA; gaussian filtering processing is carried out on the grayscale image by using the parallel assembly line characteristic of the FPGA to obtain image data after smooth processing; calculating the gradient magnitude and direction of each pixel point of the image to estimate the edge intensity and direction of each pixel point; non-maximum suppression is completed by comparing the gradient magnitude of the current pixel with the gradient magnitudes of two adjacent pixels in the gradient direction; and optimizing dual-threshold processing by using a Tianying algorithm, automatically adjusting according to the features of the image to obtain an optimal threshold, and outputting an edge image. And the edge detection performance can be obviously improved. And the real-time requirement of high frame rate is met on the occasion of needing real-time or efficient image edge detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer image processing, and in particular is an image edge processing method and a device thereof based on FPGA acceleration. Background Art

[0002] In recent years, as new AI scenarios such as high-precision online industrial inspection (smart cameras), car autonomous driving, and new medical inspections continue to emerge, the demand for high-efficiency and low-power real-time image detection is growing. The Canny edge detection algorithm has the advantages of high accuracy, low error rate, simplicity, and high efficiency, and it is widely used in the field of computer vision.

[0003] The Eagle Optimization Algorithm is a new heuristic algorithm proposed by Laith Abualigah et al. in 2021, inspired by biological behavior. The search strategy of the Eagle Optimization Algorithm in the optimization process is developed to simulate the hunting behavior of the Eagle in the face of different prey. The hunting process of the Eagle maps the process of the algorithm finding the optimal solution from the candidate solutions. The capture of agile prey can reflect the global search ability of the algorithm, and the predation of slow-moving prey reflects the local exploration ability of the algorithm. Because the algorithm is easy to implement, has good scalability, and has certain global and local search capabilities, it can avoid the trouble of local optimal solutions. Used in the dual threshold processing step of the Canny algorithm, it can automatically adjust and obtain the optimal threshold according to the characteristics of the image, thereby retaining more edge information of the image.

[0004] For the current Canny edge algorithm processing pipeline, the real obstacle to its standardization and production implementation is the computing limitation, which leads to insufficient reasoning speed and makes it unable to meet the most basic real-time requirements. This requires an accelerator design architecture to speed up the reasoning speed of the image. However, the general-purpose hardware accelerator currently implemented by ASIC (application-specific integrated circuit) is not only difficult to design and has a long development cycle, but also may not be sufficient to meet the real-time requirements in terms of acceleration ratio. FPGA (field programmable gate array) is an integrated circuit that allows users to implement specific logical functions through programming. Unlike ASIC, FPGA can still be reprogrammed after manufacturing, so it has high flexibility. The important features of FPGA are parallelism and pipeline technology. Parallelism means that multiple operations can be performed simultaneously instead of sequentially. Pipeline parallelism means that through pipeline technology, FPGA can decompose a complex task into multiple stages, and each stage is processed in parallel, thereby increasing the processing speed. Compared with other typical parallel computing platforms such as multi-core CPUs and GPUs, FPGA has the characteristics of highly customizable parallel computing, high compatibility, and low power consumption. Summary of the invention

[0005] In order to solve the problem that the existing technology cannot meet the most basic real-time requirements, the present invention provides an image edge processing method and device based on FPGA acceleration, which are mainly used for new energy lithium battery detection and 3C industry visual detection, positioning and recognition functions. Based on the operation of FPGA parallel pipeline, the optimized Canny algorithm is accelerated to detect the edge of the image, which gives full play to the advantages of FPGA in pipeline design, meets the requirements of high efficiency and real-time performance of image processing, and the pipeline structure allows faster processing of image data with higher accuracy at lower power consumption, saving a lot of FPGA resources. Compared with the traditional Canny image processing algorithm, images are obtained faster and with higher accuracy.

[0006] The present invention is achieved by adopting the following technical solutions:

[0007] An image edge processing method based on FPGA acceleration includes the following steps:

[0008] Image preprocessing: obtaining image data from an external interface, converting the RGB value of each pixel of the image data into a grayscale value as required, and storing the converted grayscale image inside the FPGA;

[0009] The parallel pipeline characteristics of FPGA are used to perform Gaussian filtering on the grayscale image to obtain smoothed image data;

[0010] Calculate the gradient magnitude and direction of each pixel in the image to estimate the edge strength and direction at each pixel;

[0011] By comparing the gradient magnitude of the current pixel with the gradient magnitudes of the two adjacent pixels along the gradient direction, non-maximum suppression is completed;

[0012] The Eagle algorithm is used to optimize the dual threshold processing, automatically adjust the optimal threshold according to the characteristics of the image, and output the edge image.

[0013] Preferably, converting the RGB value of each pixel of the image data into a grayscale value as required comprises: the pixel RGB value conversion formula is as follows:

[0014] Gray=0.299*R+0.587*G+0.114*B.

[0015] Preferably, the use of the parallel pipeline characteristics of the FPGA to perform Gaussian filtering on the grayscale image includes: acquiring image data from the memory of the FPGA, and reading the current pixel and the neighboring pixels of the current pixel, convolving each pixel by multiplication and addition with the generated convolution kernel, normalizing the pixels of the image data to adapt to the output range, and outputting the processed image data to the memory.

[0016] Preferably, the calculation of the gradient magnitude and direction of each pixel of the image comprises: using an FPGA pipeline to perform Sobel operator calculation and gradient calculation parallel processing on multiple pixels of the image simultaneously;

[0017] Sobel operator calculation includes: In the Canny edge detection algorithm, the Sobel operator is used to calculate the horizontal gradient (G x ) and vertical direction (G y ) gradient, first calculate |G x ∣ and ∣G y The absolute value of ∣ discretizes the gradient direction into four main directions according to G x and G y The sign of determines the quadrant of the direction.

[0018] Preferably, the gradient calculation includes: performing convolution calculation on the sum of the squares of the horizontal gradient and the vertical gradient of the Sobel operator and the local area of ​​the image to obtain the gradient component of each pixel; the gradient component G x and G y After calculation, the gradient magnitude and gradient direction of each pixel in the image are calculated.

[0019] Preferably, the step of comparing the gradient amplitude of the current pixel with the gradient amplitudes of two adjacent pixels along the gradient direction comprises: using the parallelism of the FPGA to assign each pixel of the image to a different corresponding processing unit, calling a FIFO queue to organize the pixel data stream of the image data, and each corresponding processing unit is responsible for performing a comparison of one pixel: according to the gradient direction of the pixel, using a comparator module of the FPGA, performing a gradient amplitude comparison operation by selecting the gradient amplitude of the current pixel with the gradient amplitudes of two adjacent neighborhood pixels along the gradient direction;

[0020] If the current pixel is not a local maximum, the pixel gradient magnitude is set to zero; otherwise, if the current pixel is a local maximum, the gradient magnitude remains unchanged.

[0021] Preferably, the method of optimizing the dual threshold processing by using the Sky Eagle algorithm and automatically adjusting the optimal threshold according to the characteristics of the image includes:

[0022] The first individual of the population is randomly generated through the rand function, the initial number of iterations is preset, the relevant parameters of the Sky Eagle algorithm are set, and the initial population and initial parameters are randomly generated;

[0023] Take the objective function as the fitness function, calculate the fitness value of the initial population, and obtain the population with the highest fitness; set the double threshold boundary for the Canny algorithm to determine whether the population exceeds the boundary, and if not, update the population;

[0024] By simulating the four life and predation behaviors of the eagle in turn, calculating the fitness value of the updated population, updating the position of the function model, and obtaining the optimal solution X of the eagle population best (t) and the average value of the position vector X M (t);

[0025] Determine whether each population exceeds the boundary. When it does not exceed the boundary, update the current fitness value and the optimal threshold solution. When the termination condition is met, after the iteration is ended, the high threshold and low threshold corresponding to the individual with the highest current fitness value are output as the final optimal threshold combination. Use this high threshold and low threshold for double threshold detection and output the final edge image.

[0026] Preferably, the obtaining of the optimal solution of the eagle group and the average value of the position vector includes: determining the stage of the eagle individual of the eagle optimization algorithm, and the stage of the eagle individual includes:

[0027] Expanded exploration phase, by soaring vertically to expand the search range, locate birds in flight, and select the best feeding area;

[0028] During the narrowing search phase, when the eagle spots prey high in the sky, the flock of birds will spiral over the target prey in preparation for catching it;

[0029] In the expansion and development stage, when the Skyhawk finds prey and determines the prey area, it will take a vertical descent attack when it flies slowly at a low altitude to approach the prey or prepares to land and attack the prey. The position of the Skyhawk preparing for predation behavior is updated;

[0030] In the development phase, the Skyhawk approaches its prey and attacks it, adjusting its position to capture it based on the prey's random escape response.

[0031] Preferably, when the termination condition is met, ending the iteration includes: determining whether the current number of iterations t reaches T, if it reaches T, exiting the main loop, outputting the current optimal solution and the global optimal fitness value, otherwise going to the previous step to add 1 to the current number of iterations and continue iterating to determine the optimal solution.

[0032] The present invention also provides an image edge processing device based on FPGA acceleration, comprising:

[0033] The image preprocessing module is used to obtain the image data obtained from the external interface, convert the RGB value of each pixel of the image data into a grayscale value as required, and store the grayscale image converted by the grayscale conversion module inside the FPGA;

[0034] The Gaussian filter module is used to perform Gaussian filter processing on the grayscale image by utilizing the parallel pipeline characteristics of the FPGA to obtain smoothed image data;

[0035] The gradient calculation module is used to calculate the gradient magnitude and direction of each pixel in the image to estimate the edge strength and direction at each pixel;

[0036] A suppression module, used for comparing the gradient amplitude of the current pixel with the gradient amplitudes of two adjacent pixels along the gradient direction through a comparison module to complete non-maximum suppression;

[0037] The optimization module is used to optimize the dual threshold processing using the Eagle algorithm, automatically adjust the optimal threshold according to the characteristics of the image, and output the edge image.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention provides a roadmap for developing low-power and effective image edge processing solutions. The parallel pipeline technology of FPGA is used to accelerate the Canny algorithm processing architecture. In the system, the captured image data is transmitted from the camera to the FPGA pixel by pixel. Certain local operations, such as median or level filters, use some specific pipelines to accelerate the filtering process to improve processing efficiency when local processing of the image is required. Applying the Eagle algorithm to the dual threshold optimization in the Canny algorithm can significantly improve the performance of edge detection. Compared with the traditional Canny image processing algorithm, images are obtained faster and more accurately.

[0040] (1) High resolution: When Canny edge detection processes image details, especially small or weak edges, it may be affected by the threshold setting, resulting in reduced edge resolution. The Tianying optimization algorithm can automatically adjust and obtain the optimal threshold according to the characteristics of the image, and can consider more local and global information during the optimization process, thereby more finely adjusting the dual threshold processing process in the Canny algorithm, improving the edge detection accuracy of the details, and improving the resolution of image edge detection.

[0041] (2) Efficiency: The Canny edge detection algorithm usually contains multiple computationally intensive steps (such as Gaussian filtering, gradient calculation, and non-maximum suppression). Implementing multiple independent pipeline operations on FPGA can fully utilize its parallelism, thereby greatly improving the computing speed. FPGA can run at a high frequency, and each pipeline can process multiple tasks or data streams simultaneously, which can improve throughput and accelerate the processing of the Canny edge algorithm.

[0042] (3) Low latency: The pipeline structure of the FPGA enables each stage of the Canny algorithm to run continuously, and there is no need to wait while data flows through the pipeline. The pipeline structure can decompose complex image processing tasks into multiple stages, and each stage is processed simultaneously, thereby reducing the overall processing latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the image edge processing method based on FPGA acceleration of the present invention;

[0044] Figure 2 It is a block diagram of the FPGA acceleration module of the image edge processing method based on FPGA acceleration of the present invention;

[0045] Figure 3 It is a flow chart of the output optimal threshold value of the image edge processing method based on FPGA acceleration of the present invention;

[0046] Figure 4 It is a schematic diagram of an image edge processing device based on FPGA acceleration of the present invention.

[0047] Figure 5 A schematic diagram of an image to be optimized by the image edge processing method based on FPGA acceleration of the present invention;

[0048] Figure 6 This is a schematic diagram of the existing Canny algorithm processing;

[0049] Figure 7 It is a schematic diagram of an optimized processed image of the image edge processing method based on FPGA acceleration of the present invention. DETAILED DESCRIPTION

[0050] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0051] The Canny edge detection operator is a multi-level edge detection algorithm developed by John F. Canny in 1986. Canny edge detection is a technique for extracting useful structural information from different visual objects and greatly reducing the amount of data to be processed. It has been widely used in various computer vision systems. Use the variational method. The optimal function in the Canny detector is described by the sum of four exponential terms, which can be approximated by the first-order derivative of the Gaussian function. The Canny edge detection algorithm can be divided into the following 5 steps: 1) Use a Gaussian filter to smooth the image and filter out noise. 2) Calculate the gradient strength and direction of each pixel in the image. 3) Apply non-maximum suppression to eliminate stray responses caused by edge detection. 4) Apply double-threshold detection to determine real and potential edges. 5) Finally complete edge detection by suppressing isolated weak edges.

[0052] like Figure 1 As shown, the embodiment of the present application provides an image edge processing method based on FPGA acceleration, comprising the following steps:

[0053] S1, image preprocessing, obtaining image data obtained from an external interface, converting the RGB value of each pixel of the image data into a grayscale value as required, and storing the converted grayscale image inside the FPGA;

[0054] S2, using the parallel pipeline characteristics of FPGA to perform Gaussian filtering on the grayscale image to obtain smoothed image data;

[0055] S3, calculating the gradient magnitude and direction of each pixel of the image to estimate the edge strength and direction at each pixel;

[0056] S4, by comparing the gradient amplitude of the current pixel with the gradient amplitudes of two adjacent pixels along the gradient direction, non-maximum suppression is performed to eliminate the spurious response caused by edge detection;

[0057] S5. Use the Eagle algorithm to optimize the dual threshold processing, automatically adjust the optimal threshold according to the characteristics of the image, and output the edge image.

[0058] Preferably, in step S1, converting the RGB value of each pixel of the image data into a grayscale value as required comprises:

[0059] The pixel RGB value conversion formula is as follows,

[0060] Gray=0.299*R+0.587*G+0.114*B.

[0061] Preferably, step S2, using the parallel pipeline characteristics of FPGA to perform Gaussian filtering on the grayscale image includes: acquiring image data from the memory of FPGA, and reading the current pixel and its neighboring pixels, convolving each pixel by multiplication and addition with the generated convolution kernel, normalizing the pixels of the image data to adapt to the output range (such as 0 to 255), and outputting the processed image data to the memory. The convolution kernel is generated by the shifter as a 3*3 window convolution kernel and stored in the FPGA.

[0062] The purpose of Gaussian filtering is to filter out irrelevant information in the obtained grayscale image and reduce the impact of noise on edge detection. The result is a smoothed image. In Canny edge detection, the smoothed image is used to calculate the gradient and subsequent edge detection.

[0063] Gaussian filtering is accelerated by the parallel processing capability of FPGA and is suitable for real-time image processing. The formula of Gaussian function is as follows:

[0064]

[0065] Here, x and y are two-dimensional coordinates representing the horizontal and vertical distances relative to the center of the Gaussian kernel.

[0066] Preferably, step S3, calculating the gradient amplitude and direction of each pixel point of the image includes: using an FPGA pipeline to simultaneously perform Sobel operator calculation, gradient calculation, and comparator parallel processing on multiple pixels of the image; and based on the quantization operation of the gradient direction, making full use of the parallel processing capability of the FPGA to improve the processing speed.

[0067] Sobel operator calculation includes: In the Canny edge detection algorithm, the Sobel operator is used to calculate the horizontal gradient (G x ) and vertical direction (G y ) gradient, first calculate the absolute values ​​of |Gx| and |Gy|, discretize the gradient direction into 4 main directions, and determine the quadrant of the direction (for example, whether it is a 180° transition) based on the signs of Gx and Gy. The Sobel operator is:

[0068] Horizontal Gradient: Vertical Gradient:

[0069] Gradient calculation includes: convolution calculation based on the horizontal gradient and the square sum of the vertical gradient of the Sobel operator and the local area of ​​the image to obtain the gradient component of each pixel; the gradient component G x and G y After calculation, the gradient magnitude and gradient direction of each pixel of the image are calculated. For each pixel I(x,y) in the image, the convolution of the Sobel operator is calculated as:

[0070]

[0071] Among them, I(x+i,y+j) is the pixel value of the 3x3 neighborhood centered on the current pixel I(x,y) in the image, and K x (i,j) and K y (i, j) is the horizontal and vertical convolution kernel of the Sobel operator.

[0072] The traditional calculation method of gradient magnitude and gradient direction is as follows:

[0073] Gradient Amplitude:

[0074] Gradient direction:

[0075] like Figure 2As shown in the figure, in this process, FPGA synchronous pipeline design is used to design the hardware acceleration module of the Canny algorithm. The specific (FPGA) acceleration module design process, some modules in the figure such as reg, comparator, absolute value calculation, etc. are relatively simple and will not be described in detail here. The following mainly describes the basic structure and parameters of other modules: the function of the data synthesis module is to merge the gradient data of the two 9-bit horizontal gradient and vertical gradient into one 18-bit data, so as to facilitate the input to the gradient FIFO module for synchronous delay. The data decomposition module performs the inverse operation of the data synthesis module, calculates the absolute value of |Gx| and |Gy|, and combines the comparator and reg module. Both are implemented by the combinational logic circuit. Among them, the gradient FIFO depth is 325 and the bit width is 18 bits. The main function is to delay the two-way gradient data by 325 clock cycles to ensure that the output data can synchronously reach the non-maximum suppression module. The Eagle algorithm is used to optimize the dual threshold processing. The optimized dual threshold superposition gradient amplitude G is obtained. After comparison by the comparator, it enters the result output module to output the edge image. The gradient amplitude calculation module consists of a multiplier-accumulator.

[0076] In order to improve the calculation efficiency, the present invention only calculates the square sum of the horizontal gradient and the vertical gradient for the gradient amplitude, and does not perform square root operation. In the dual threshold operation, the gradient amplitude without square root is compared with the two thresholds after square, which not only saves FPGA resources but also improves the calculation speed, thereby accelerating image processing.

[0077] The calculation of arctan in FPGA is very complex and resource-consuming. Therefore, for the gradient direction, the present invention first calculates |G x ∣ and ∣G y The absolute value of |G is used to compare the size and discretize the gradient direction into four main directions: 0°: the gradient direction is close to the horizontal, |G y ∣<<∣G x ∣;

[0078] 45°: The gradient direction is between horizontal and vertical, |G y ∣≈∣G x ∣;

[0079] 90°: The gradient direction is close to vertical, |G x ∣<<∣G y ∣;

[0080] 135°: The gradient direction is between vertical and horizontal, |G y ∣>∣G x ∣, but does not meet the vertical condition. According to G x and G y The sign of determines the quadrant of the direction (e.g., whether it is a 180° transition).

[0081] The gradient is quantized into the following four directions according to the four directions of gradient discretization:

[0082] Horizontal edge (0°): When 0°≤θ<22.5° or 157.5°≤θ<202.5° or 337.5°≤θ<360°, the gradient direction is horizontal.

[0083] Diagonal edge (45°): When 22.5°≤θ<67.5° or 202.5°≤θ<247.5°, the direction is the 45° diagonal direction.

[0084] Vertical edge (90°): When 67.5°≤θ<112.5° or 247.5°≤θ<292.5, the direction is vertical.

[0085] Diagonal edge (135°): When 112.5°≤θ<157.5° or 292.5°≤θ<337.5, the direction is 135° diagonal. These directions correspond to horizontal, vertical, and diagonal edges in the image. During edge detection, this directional information ensures that the detected edges are more accurate and coherent.

[0086] The entire process above adopts FPGA pipeline design, and multiple pixels are calculated simultaneously. Figure 2 The gradient FIFO, Reg and delay unit in the module are to ensure that the data stream enters the lower-level module synchronously. The ultimate goal is to make all data streams reaching the non-maximum suppression module synchronous input to improve the processing speed.

[0087] Preferably, step S4, comparing the gradient amplitude of the current pixel with the gradient amplitudes of two adjacent pixels along the gradient direction, comprises: using the parallelism of the FPGA to assign each pixel of the image to a different corresponding processing unit, calling the FIFO queue to organize the pixel data stream of the image data, and then each corresponding processing unit is responsible for performing a comparison of one pixel: according to the gradient direction of the pixel, using the comparator module of the FPGA, performing a gradient amplitude comparison operation by selecting the gradient amplitude of the current pixel and the gradient amplitudes of two adjacent neighborhood pixels along the gradient direction;

[0088] For example: For a horizontal edge (0°), compare the gradient magnitudes of the current pixel with its left and right neighbors.

[0089] For vertical edges (90°), the gradient magnitudes of the current pixel and its upper and lower neighbors are compared.

[0090] For a 45° diagonal edge (45°), the gradient magnitude of the current pixel is compared with its upper left and lower right neighbors.

[0091] For a 135° diagonal edge (135°), the gradient magnitude of the current pixel is compared with its upper right and lower left neighbors.

[0092] If the current pixel is not a local maximum, the pixel gradient amplitude is set to zero; otherwise, if the current pixel is a local maximum (i.e., the gradient amplitude of a pixel point is larger than that of its two adjacent points in the gradient direction), the gradient amplitude remains unchanged.

[0093] The main purpose of non-maximum suppression is to refine the edge, suppress non-edge points in the gradient direction, and retain the pixels with the largest gradient intensity to eliminate the stray response caused by edge detection. That is, retain the real edge pixels from the calculated gradient amplitude and remove those pixels that have a large gradient amplitude but do not represent the edge. Use FIFO queues to organize data flow to ensure that each processing unit processes image data in sequence.

[0094] Preferably, step S5, the specific operations are as follows Figure 3 As shown. Using the Canny algorithm to optimize the dual threshold processing, the optimal threshold is adjusted according to the characteristics of the image, including:

[0095] The first individual of the population is randomly generated through the rand function, the initial number of iterations is preset to t = 1, the relevant parameters of the Eagle algorithm are set, and the initial population and initial parameters are randomly generated; the dimension is 2. The relevant parameters include the number of Eagle populations and the maximum number of iterations.

[0096] Use the objective function as the fitness function, calculate the fitness value of the initial population, and obtain the population with the highest fitness; set the double threshold boundary for the Canny algorithm, with the upper boundary as 255 and the lower boundary as 0, to determine whether the population exceeds the boundary. If not, update the population;

[0097] By simulating the four life and predation behaviors of the eagle in turn, calculating the fitness value of the updated population, updating the position of the function model, and obtaining the optimal solution X of the eagle population best (t) and the average value of the position vector X M (t);

[0098] Determine whether each population exceeds the boundary. If it does not exceed the boundary, update the current fitness value and the optimal threshold solution. When the termination condition is met, after the iteration is finished, output the high threshold and low threshold corresponding to the individual with the highest current fitness value as the final optimal threshold combination, use this high threshold and low threshold to perform double threshold detection, and output the final edge image. The individuals in this population are eagle individuals.

[0099] The dual thresholds are the high threshold and the low threshold. The high threshold determines whether pixels are strong edges. All pixels with gradient magnitudes greater than the high threshold are considered strong edges. The low threshold determines whether pixels are weak edges. All pixels with gradient magnitudes between the low threshold and the high threshold are considered weak edges. If all gradient magnitudes are less than the low threshold, the pixel is considered a non-edge and will not be retained. The high and low thresholds of the gradient magnitude are divided into strong edges and weak edges. Strong edges are identified as edges, and weak edges are judged as edges by connecting with strong edges.

[0100] The objective function includes the following two points: clarifying the optimization goal (specifically the choice of dual thresholds); defining the range of all feasible solutions and determining the parameter space of the problem, specifically the dual threshold boundary problem.

[0101] Preferably, when the termination condition is met, ending the iteration includes: determining whether the current number of iterations t reaches T, if so, exiting the main loop, outputting the current optimal solution and the global optimal fitness value, otherwise going to the previous step to add 1 to the current number of iterations t and continue iterating to determine the optimal solution.

[0102] Preferably, the calculation of the fitness value of the updated population and the position update of the function model include: judging the current number of iterations t and the value of the random number p, and determining the stage of the Eagle individual of the Eagle optimization algorithm to update the position. If t≤(2T / 3) and p≤0.5, the Eagle individual updates its position according to the expanded exploration stage X1; if t≤(2T / 3) and p>0.5, the Eagle individual updates its position according to the reduced search stage X2; if t>(2T / 3) and p≤0.5, the Eagle individual updates its position according to the expanded development stage X3; if t>(2T / 3) and p>0.5, the Eagle individual updates its position according to the reduced development stage X4.

[0103] Traditional fixed threshold methods are usually less adaptable to different image features (such as illumination, contrast, noise, etc.), while the Eagle algorithm can adjust the threshold adaptively based on the specific characteristics of the image, and the optimization effect may be better. By using the Eagle algorithm to optimize the dual thresholds and dynamically adjust the size of the two thresholds, it can adapt to different types of images.

[0104] Optimize and obtain the optimal solution X of the Eagle group best (t) and the average value of the position vector X M (t) including: determining the stage of the Eagle individual of the Eagle optimization algorithm, the stage of the Eagle individual including: expanding exploration stage (X1), narrowing search stage (X2), expanding development stage (X3), narrowing development stage (X4);

[0105] S51, the expanded exploration phase (X1), expands the search range by vertically soaring, determines the location of birds in flight, and selects the best predation area; its mathematical model is:

[0106]

[0107] Where t represents the number of optimizations performed by the current algorithm; T represents the total number of optimizations performed by the algorithm; X1(t+1) represents the next solution of t generated under the first hunting behavior (X1); X best (t) represents the optimal hunting position (optimal solution of the eagle group) under the current optimization number t; rand takes a random number in [0, 1]; X M (t) represents the average position of the solution at the tth iteration (the average value of the position vector); N represents the number of candidate solutions; dim represents the dimension of the optimized problem.

[0108] S52, in the narrowing search phase (X2), when the eagle finds prey in the sky, the eagle makes a spiral circle above the target prey to prepare for catching the target; this strategy of the eagle is called a short gliding attack with equal altitude flight, and its mathematical model is:

[0109] X2(t+1)=X best (t)*Levy(D)+X R (t)+(yx)*rand

[0110] Among them, x, y are parameters that control the shape of the spiral search, indicating the shape of the spiral flight; X2(t+1) represents the next solution of t generated under the second hunting behavior (X2); X R (t) is the random solution drawn in the range [1~N] at the i-th iteration; Levy (D) represents the Levy flight distribution strategy, and its distribution function is calculated as follows:

[0111]

[0112] Where S is 0.01; u and v can take any value in [0, 1]; β is 1.5; Γ(x) represents the gamma function.

[0113]

[0114] In the formula, Take 0.005; r1 takes the number of search cycles between [1,20]; D1 takes an integer between [1,dim].

[0115] S53, expansion and development stage (X3), when the eagle finds prey and determines the prey area, it will take a vertical descent attack when it flies slowly at a low altitude to approach the prey or prepares to land and attack the prey. The position of the eagle preparing for predation behavior is updated, and its mathematical model is:

[0116] X3(t+1)+(X best (t)-X M (t))*a-rand+((UB-LB)*rand+LB)*b

[0117] Among them, X3(t+1) represents the next solution generated under the third hunting behavior (X3); a and b represent development adjustment parameters, taking the smaller value of 0.1; LB is the lower boundary of the problem to be solved; UB is the upper boundary of the problem to be solved.

[0118] S54, narrowing the development stage (X4), the eagle approaches the prey and launches an attack, and adjusts its position to catch the prey according to the prey's random escape reaction; the mathematical model of hunting behavior is:

[0119] X4(t+1)=QF(t)*X best (t)-(G1*X(t)*rand)-G2*Levy(D)+rand*G1

[0120] Where X4(t+1) represents the next solution generated under the fourth hunting behavior (X4); QF(t) is the quality function value used to balance the search strategy, G1 represents the various movements of the eagle in the process of tracking prey, and G2 represents the slope between the position where the eagle starts flying and the position when it catches the prey. The calculation formula of the relevant parameters is as follows:

[0121]

[0122] G1=2*rand-1

[0123]

[0124] like Figure 4 As shown, the present invention also provides an image edge processing device based on FPGA acceleration, including: an image preprocessing module, used to obtain image data obtained from an external interface, convert the RGB value of each pixel of the image data into a grayscale value as required, and store the grayscale image converted by the grayscale conversion module inside the FPGA;

[0125] The Gaussian filter module is used to perform Gaussian filter processing on the grayscale image by utilizing the parallel pipeline characteristics of the FPGA to obtain smoothed image data;

[0126] The gradient calculation module is used to calculate the gradient amplitude and direction of each pixel of the image to estimate the edge strength and direction module at each pixel;

[0127] A suppression module, used for comparing the gradient amplitude of the current pixel with the gradient amplitudes of two adjacent pixels along the gradient direction through a comparison module to complete non-maximum suppression;

[0128] The optimization module is used to optimize the dual threshold processing using the Eagle algorithm, automatically adjust the optimal threshold according to the characteristics of the image, and output the edge image.

[0129] Preferably, the grayscale conversion module is used to convert the RGB value of each pixel of the image data into a grayscale value as required, including: the formula for converting the pixel RGB value into the grayscale value is as follows:

[0130] Gray=0.299*R+0.587*G+0.114*B.

[0131] Preferably, the use of the parallel pipeline characteristics of the FPGA to perform Gaussian filtering on the grayscale image includes: acquiring image data from the memory of the FPGA, and reading the current pixel and the neighboring pixels of the current pixel, convolving each pixel by multiplication and addition with the generated convolution kernel, normalizing the pixels of the image data to adapt to the output range, and outputting the processed image data to the memory.

[0132] Preferably, the module for calculating the gradient amplitude and direction of each pixel of the image includes: using an FPGA pipeline to simultaneously perform parallel processing of a Sobel operator calculation module, a gradient calculation module, and a comparator on multiple pixels of the image;

[0133] The Sobel operator calculation module includes: In the Canny edge detection algorithm, the Sobel operator is used to calculate the horizontal gradient (Gx) and vertical gradient (Gy) of the image. The absolute values ​​of |Gx| and |Gy| are first calculated, and the gradient direction is discretized into four main directions. The quadrant of the direction is determined according to the signs of Gx and Gy.

[0134] Preferably, the gradient calculation module includes: performing convolution calculation on the horizontal gradient and the square sum of the vertical gradient of the Sobel operator and the local area of ​​the image to obtain the gradient component of each pixel; after the gradient components Gx and Gy are calculated, the gradient amplitude and gradient direction of each pixel of the image are calculated.

[0135] Preferably, the comparison module comprises: using the parallelism of FPGA to allocate each pixel of the image to a different corresponding processing unit, calling the FIFO queue to organize the pixel data flow of the image data, each corresponding processing unit is responsible for performing a comparison of one pixel: according to the gradient direction of the pixel, using the comparator module of FPGA, performing a gradient amplitude comparison operation by selecting the gradient amplitude of the current pixel and the gradient amplitudes of two adjacent neighborhood pixels along the gradient direction;

[0136] If the pixel is not a local maximum, the pixel gradient value is set to zero; otherwise, if the pixel is a local maximum, the gradient magnitude remains unchanged.

[0137] Preferably, the module for automatically adjusting the optimal threshold value according to the characteristics of the image includes:

[0138] The first individual of the population is randomly generated through the rand function, the initial number of iterations is preset to t = 1, the relevant parameters of the Eagle algorithm are set, and the initial population and initial parameters are randomly generated; the dimension is 2. The relevant parameters include the number of Eagle populations and the maximum number of iterations.

[0139] Use the objective function as the fitness function, calculate the fitness value of the initial population, and obtain the population with the highest fitness; set the double threshold boundary for the Canny algorithm, with the upper boundary as 255 and the lower boundary as 0, to determine whether the population exceeds the boundary. If not, update the population;

[0140] By simulating the four life and predation behaviors of the eagle in turn, calculating the fitness value of the updated population, updating the position of the function model, and obtaining the optimal solution X of the eagle population best (t) and the average value of the position vector X M (t);

[0141] Determine whether each population exceeds the boundary. When it does not exceed the boundary, update the current fitness value and the optimal threshold solution. When the termination condition is met, after the iteration is ended, the high threshold and low threshold corresponding to the individual with the highest current fitness value are output as the final optimal threshold combination. Use this high threshold and low threshold for double threshold detection and output the final edge image.

[0142] Preferably, the obtaining of the optimal solution of the eagle group and the average value of the position vector includes: determining the stage of the eagle individual of the eagle optimization algorithm, and the stage module of the eagle individual includes: expanding the exploration stage (X1), reducing the search stage (X2), expanding the development stage (X3), and reducing the development stage (X4);

[0143] Expanded exploration phase, by soaring vertically to expand the search range, locate birds in flight, and select the best feeding area;

[0144] During the narrowing search phase, when the eagle spots prey high in the sky, the flock of birds will spiral over the target prey in preparation for catching it;

[0145] In the expansion and development stage, when the Skyhawk finds prey and determines the prey area, it will take a vertical descent attack when it flies slowly at a low altitude to approach the prey or prepares to land and attack the prey. The position of the Skyhawk preparing for predation behavior is updated;

[0146] In the development phase, the Skyhawk approaches its prey and attacks it, adjusting its position to capture it based on the prey's random escape response.

[0147] The present invention is verified by conducting simulation experiments on two different test images. Figure 5 , which can be clearly seen from the picture. Compared with the traditional Canny edge detection ( Figure 6 ), the improved algorithm of the present invention handles edge connectivity better ( Figure 7 ), which can obtain relatively complete edge contours and retain more edge information.

[0148] The present invention adopts the Eagle optimization algorithm in the dual threshold processing of the Canny algorithm to obtain the optimal threshold, and finally obtains a clearer image edge contour, and increases the processing speed of the Canny algorithm by 3.5 times, meeting the real-time requirements of high frame rate in situations where real-time or efficient image edge detection is required. It can significantly improve the performance of edge detection. Its advantages are reflected in global optimal search capability, adaptability, robustness, ability to handle multi-peak problems, and reduction of manual intervention. The Eagle algorithm provides strong support for threshold selection in edge detection, especially showing better adaptability and optimization effect when processing complex or diversified images.

[0149] In the description of the present invention, it is necessary to understand that the indicated orientation or positional relationship is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0150] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An image edge processing method based on FPGA acceleration, characterized in that: The following steps are involved: Image preprocessing: obtaining image data from an external interface, converting the RGB value of each pixel of the image data into a grayscale value as required, and storing the converted grayscale image inside the FPGA; The parallel pipeline characteristics of FPGA are used to perform Gaussian filtering on the grayscale image to obtain smoothed image data; Calculate the gradient magnitude and direction of each pixel in the image to estimate the edge strength and direction at each pixel; By comparing the gradient magnitude of the current pixel with the gradient magnitudes of the two adjacent pixels along the gradient direction, non-maximum suppression is completed; The Eagle algorithm is used to optimize the dual threshold processing, automatically adjust the optimal threshold according to the characteristics of the image, and output the edge image.

2. The method according to claim 1, characterized in that: The step of converting the RGB value of each pixel of the image data into a grayscale value as required includes: the pixel RGB value conversion formula is as follows: Gray=0.299*R+0.587*G+0.114*B.

3. The method according to claim 1, characterized in that: The method of performing Gaussian filtering on a grayscale image using the parallel pipeline characteristics of the FPGA includes: acquiring image data from the memory of the FPGA, and reading the current pixel and the neighboring pixels of the current pixel, convolving each pixel by multiplication and addition with the generated convolution kernel, normalizing the pixels of the image data to adapt to the output range, and outputting the processed image data to the memory.

4. The method according to claim 1, characterized in that: The calculation of the gradient magnitude and direction of each pixel of the image includes: using the FPGA pipeline characteristics to perform Sobel operator calculation and gradient calculation parallel processing on multiple pixels of the image at the same time; Sobel operator calculation includes: In the Canny edge detection algorithm, the Sobel operator is used to calculate the horizontal gradient (G x ) and vertical direction (G y ) gradient, first calculate |G x ∣ and ∣G y The absolute value of ∣ discretizes the gradient direction into four main directions according to G x and G y The sign of determines the quadrant of the direction.

5. The method according to claim 4, characterized in that: The gradient calculation includes: performing convolution calculation on the horizontal gradient and the square sum of the vertical gradient of the Sobel operator and the local area of ​​the image to obtain the gradient component of each pixel; the gradient component G x and G y After calculation, the gradient magnitude and gradient direction of each pixel in the image are calculated.

6. The method according to claim 1, characterized in that: The step of comparing the gradient magnitude of the current pixel with the gradient magnitudes of two adjacent pixels along the gradient direction comprises: Each pixel of the image is assigned to a different corresponding processing unit by using the parallelism of FPGA, and the FIFO queue is called to organize the pixel data flow of the image data. Each corresponding processing unit is responsible for performing a pixel comparison: according to the gradient direction of the pixel, the comparator module of FPGA is used to perform a gradient amplitude comparison operation by selecting the gradient amplitude of the current pixel and the gradient amplitude of the two adjacent neighboring pixels along the gradient direction; If the current pixel is not a local maximum, the pixel gradient magnitude is set to zero; otherwise, if the current pixel is a local maximum, the gradient magnitude remains unchanged.

7. The method according to claim 1, characterized in that: The method of optimizing the dual threshold processing by using the Sky Eagle algorithm and automatically adjusting the optimal threshold according to the characteristics of the image includes: The first individual of the population is randomly generated through the rand function, the initial number of iterations is preset, the relevant parameters of the Sky Eagle algorithm are set, and the initial population and initial parameters are randomly generated; Take the objective function as the fitness function, calculate the fitness value of the initial population, and obtain the population with the highest fitness; set the double threshold boundary for the Canny algorithm to determine whether the population exceeds the boundary, and if not, update the population; By simulating the four life and predation behaviors of the eagle in sequence, the fitness value of the updated population is calculated, the position of the function model is updated, and the optimal solution of the eagle population and the average value of the position vector are obtained; Determine whether each population exceeds the boundary. When it does not exceed the boundary, update the current fitness value and the optimal threshold solution. When the termination condition is met, after the iteration is ended, the high threshold and low threshold corresponding to the individual with the highest current fitness value are output as the final optimal threshold combination. Use this high threshold and low threshold for double threshold detection and output the final edge image.

8. The method according to claim 7, characterized in that: The step of obtaining the optimal solution of the eagle group and the average value of the position vector includes: determining the stage of the eagle individual of the eagle optimization algorithm, and the stage of the eagle individual includes: Expanded exploration phase, by soaring vertically to expand the search range, locate birds in flight, and select the best feeding area; In the narrowing search phase, when the eagle spots prey high in the sky, the flock of birds spirals over the target prey in preparation for catching it; In the expansion and development stage, when the Skyhawk finds prey and determines the prey area, it will take a vertical descent attack when it flies slowly at a low altitude to approach the prey or prepares to land and attack the prey. The position of the Skyhawk preparing for predation behavior is updated; In the development phase, the Skyhawk approaches its prey and attacks it, adjusting its position to capture it based on the prey's random escape response.

9. The method according to claim 7, characterized in that: When the termination condition is met, the iteration ends, including: judging whether the current number of iterations t reaches T, if it reaches T, exiting the main loop, outputting the current optimal solution and the global optimal fitness value, otherwise going to the previous step to add 1 to the current number of iterations and continue iterating to judge and find the optimal solution.

10. An image edge processing device based on FPGA acceleration, according to any one of the methods of claims 1-9, characterized in that: The image preprocessing module is used to obtain the image data obtained from the external interface, convert the RGB value of each pixel of the image data into a grayscale value as required, and store the grayscale image converted by the grayscale conversion module inside the FPGA; The Gaussian filter module is used to perform Gaussian filter processing on the grayscale image by utilizing the parallel pipeline characteristics of the FPGA to obtain smoothed image data; The gradient calculation module is used to calculate the gradient magnitude and direction of each pixel in the image to estimate the edge strength and direction at each pixel; A suppression module, used for comparing the gradient amplitude of the current pixel with the gradient amplitudes of two adjacent pixels along the gradient direction through a comparison module to complete non-maximum suppression; The optimization module is used to optimize the dual threshold processing using the Eagle algorithm, automatically adjust the optimal threshold according to the characteristics of the image, and output the edge image.

Images