An Extended Target Localization System for Unmanned Aerial Vehicles Based on FPGA Image Processing

Abstract

This invention discloses a UAV extended target localization system based on FPGA image processing, comprising an image preprocessing module, a template updating module, an edge matching module, an edge image caching module, and an edge template caching module. The image preprocessing module extracts edge information from the input image; the template updating module updates the edge template using an integral method; the edge matching module handles parallel matching operations; the edge image caching module temporarily caches the edge images; and the edge template caching module temporarily caches the edge templates. The template updating strategy based on integral edge templates proposed in this invention can gradually synchronize target attitude changes to the edge template, thereby enhancing the robustness of the template and achieving better localization results for UAV extended targets.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically a UAV extended target positioning system based on FPGA image processing. Background Technology

[0002] Unmanned aerial vehicle (UAV) target localization technology, as an important research direction in computer vision, plays an increasingly important role in various fields. During the localization of UAV targets, the target's flight attitude, size, and ambient lighting may undergo complex changes, posing significant challenges to the accuracy and real-time performance of target localization. Extended target localization is an important branch of target tracking. Extended targets refer to targets occupying more than two-thirds of the field of view. Typical scenarios in localizing extended targets include excessively large target size, changes in target attitude, and changes in ambient lighting. Excessively large target size leads to excessive computational load, resulting in poor real-time localization. Changes in target attitude and ambient lighting reduce localization accuracy. Among algorithms for UAV extended target localization, edge matching algorithms are widely used due to their low algorithm complexity and ease of hardware implementation. However, as the target's attitude and size change, the template may degenerate. To avoid the impact of template degradation on edge matching accuracy, a reasonable template update method is needed. Summary of the Invention

[0003] The purpose of this invention is to provide an extended target positioning system for unmanned aerial vehicles (UAVs) based on FPGA image processing in order to solve the problems mentioned above.

[0004] The technical solution adopted in this invention is as follows:

[0005] An extended target localization system for unmanned aerial vehicles based on FPGA image processing, the system comprising an image preprocessing module, an edge image caching module, a template updating module, an edge template caching module, and an edge matching module;

[0006] The output of the image preprocessing module is connected to the input of the edge image caching module and the input of the edge matching module, respectively; the output of the edge image caching module is connected to the input of the template update module; the output of the template update module is connected to the input of the edge template caching module; the output of the edge template caching module is connected to the input of the template update module and the input of the edge matching module, respectively; the output of the edge matching module is connected to the input of the template update module.

[0007] The image preprocessing module internally includes a median filtering module, a Sobel edge extraction module, a local mean binarization module, an erosion module, and a dilation module. The output of the median filtering module is connected to the input of the Sobel edge extraction module. The output of the Sobel edge extraction module is connected to the input of the local mean binarization module. The output of the local mean binarization module is connected to the input of the erosion module. The input of the erosion module is connected to the input of the dilation module. Each of the median filtering module, Sobel edge extraction module, local mean binarization module, erosion module, and dilation module has its own parallel hardware circuit.

[0008] Furthermore, the median filtering module uses a 3×3 square window to filter and reduce noise in the image, specifically filtering out salt-and-pepper noise. The parallelized hardware circuit of the median filtering module includes two FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to three shift registers, and the back end of the input image data stream is also connected to three shift registers. These nine shift registers form a 3×3 register window, and the data within this register window is used for median filtering.

[0009] Furthermore, the Sobel edge extraction module uses a 3×3 window to extract edges in four directions and superimposes the extracted edges in the four directions. The parallel hardware circuit of the Sobel edge extraction module includes two FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to three shift registers, and the back end of the input image data stream is also connected to three shift registers. These nine shift registers form a 3×3 register window, and the data in this register window is used for Sobel filtering operations.

[0010] Furthermore, the local mean binarization module determines the binarization result of the center pixel by calculating the average value of all pixels in a 5×5 area surrounding the center pixel. The parallel hardware circuit of the local mean binarization module includes four FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to five shift registers, and the back end of the input image data stream is also connected to five shift registers. These 25 shift registers form a 5×5 register window, and the data in this register window is used for the mean calculation operation.

[0011] Furthermore, the erosion module uses a 3×3 square window for erosion operations to remove isolated points on the binarized image. The parallel hardware circuit of the erosion module includes two FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to three shift registers, and the back end of the input image data stream is also connected to three shift registers. These nine shift registers form a 3×3 register window, and the data in this register window is used for the erosion operation.

[0012] Furthermore, the expansion module uses a 3×3 square window for expansion operations to increase the width of the edges; the parallel hardware circuit of the expansion module includes two FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to three shift registers, and the back end of the input image data stream is also connected to three shift registers. These nine shift registers form a 3×3 register window, and the data in this register window is used for the expansion operation.

[0013] Furthermore, the edge matching module uses the number of activation points that overlap between the edge template and the edge image to evaluate the correlation between the template and the target. The parallelized hardware circuit of the edge matching module includes 31 FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to 32 shift registers, and the back end of the input image data stream is also connected to 32 shift registers. These 1024 shift registers form a 32×32 register window, and the data in this register window is used for template matching operations.

[0014] Furthermore, the template update module updates the edge template using the edge image of each frame in an integral manner.

[0015] Furthermore, the edge image caching module temporarily caches the preprocessed binarized edge image; the temporarily cached binarized edge image is used for subsequent template update operations.

[0016] Furthermore, the edge template caching module implements temporary caching of edge templates; the cached edge templates are used for subsequent template update operations and edge matching operations.

[0017] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0018] (1) The present invention designs a parallel computing circuit for the edge matching module, which enables edge matching operations in multiple regions to be performed simultaneously, thereby improving the overall system speed. At the same time, the present invention uses edge features as matching features. Because edge features are not sensitive to changes in the surface illumination of the UAV target, the present invention can adapt to scenarios with drastic changes in outdoor illumination.

[0019] (2) The circuit structure designed in this invention can directly use the image data stream flowing in from the external image sensor to perform edge matching operation without caching the image of the current frame before the edge matching operation, thus reducing the target positioning delay of the UAV and saving FPGA storage resources.

[0020] (3) This invention updates the edge template using an integral method. This template update method can better adapt to changes in the target and environment, thus maintaining a stable and long-term tracking effect. At the same time, the template update strategy proposed in this invention can both incorporate the target information of the current frame and retain the information of the previous template, thereby ensuring the reliability of the template information during the tracking process and significantly improving the algorithm's adaptability to environmental changes.

[0021] (4) The edge matching operation and template update operation of this invention are computed in parallel. The edge matching operation can be performed immediately when the next frame of image data stream arrives, without waiting for the template update operation to complete. Therefore, the system's positioning frame rate depends entirely on the image frame rate of the external sensor and the maximum clock frequency that the edge matching module can operate at. This design makes the system more robust. Attached Figure Description

[0022] Figure 1 This is a system block diagram of the present invention;

[0023] Figure 2 This is a system block diagram of the image preprocessing module in this invention;

[0024] Figure 3 This is a circuit diagram for generating a point-by-point matching window in this invention;

[0025] Figure 4 This is a system block diagram of the template update module in this invention;

[0026] Figure 5 This is a circuit diagram of the integration module in this invention.

[0027] The module includes: image preprocessing module 1, edge image caching module 2, template update module 3, edge template caching module 4, edge matching module 5, median filtering module 6, Sobel edge extraction module 7, local mean binarization module 8, erosion module 9, and dilation module 10. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0029] refer to Figure 1 This invention discloses an extended target localization system for unmanned aerial vehicles (UAVs) based on FPGA image processing, comprising an image preprocessing module 1, an edge image caching module 2, a template updating module 3, an edge template caching module 4, and an edge matching module 5. The output of the image preprocessing module 1 is connected to the input of the edge image caching module 2 and the edge matching module 5. The output of the edge image caching module 2 is connected to the input of the template updating module 3. The output of the template updating module 3 is connected to the input of the edge template caching module 4. The output of the edge template caching module 4 is connected to the input of the template updating module 3 and the edge matching module 5. The output of the edge matching module 5 is connected to the input of the template updating module 3.

[0030] refer to Figure 2 The image preprocessing module 1 internally includes a median filtering module 6, a Sobel edge extraction module 7, a local mean binarization module 8, an erosion module 9, and a dilation module 10. The output of the median filtering module 6 is connected to the input of the Sobel edge extraction module 7. The output of the Sobel edge extraction module 7 is connected to the input of the local mean binarization module 8. The output of the local mean binarization module 8 is connected to the input of the erosion module 9. The input of the erosion module 9 is connected to the input of the dilation module 10.

[0031] The median filtering module 6 uses a 3×3 square window to filter and reduce noise in the image, aiming to filter out salt-and-pepper noise. The parallelized hardware circuit of the median filtering module includes two FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to three shift registers, and three shift registers are also connected to the back end of the input image data stream. These nine shift registers form a 3×3 register window, and the data in this register window is used for the median filtering operation.

[0032] The Sobel edge extraction module 7 uses four 3×3 windows to extract edges in four directions of the image and then superimposes the extracted edges. The parallel hardware circuit of the Sobel edge extraction module 7 includes two FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to three shift registers, and three shift registers are also connected to the back end of the input image data stream. These nine shift registers form a 3×3 register window, and the data in this register window is used for the Sobel filtering operation.

[0033] The traditional Sobel algorithm includes square and square root operations. FPGAs are not well-suited for handling square and square root operations because the circuitry for these operations requires extremely complex logic design, and currently there is no optimized circuit structure for square root operations. Designing square and square root circuitry within an FPGA not only consumes significant logic resources but also leads to excessive delay and power consumption in the Sobel edge filtering module, resulting in a degraded overall system performance. To avoid implementing square and square root operations in FPGAs, this invention improves the traditional Sobel algorithm. The improved Sobel algorithm achieves edge extraction while avoiding the square and square root operations of the original Sobel algorithm.

[0034] In order to effectively extract edge information in various directions of the target, this invention uses a total of four Sobel operators, as shown below.

[0035]

[0036]

[0037]

[0038]

[0039] S1 is used to extract the edge information of the target in the horizontal direction, S2 is used to extract the edge information of the target in the vertical direction, S3 is used to extract the edge information of the target in the main diagonal direction, and S4 is used to extract the edge information of the target in the secondary diagonal direction.

[0040] This invention convolves S1, S2, S3, and S4 with a 3×3 image window generated by a shift register to obtain the edge values ​​of the target in four directions. Assuming the obtained edge values ​​in the four directions are s1, s2, s3, and s4, the formula for calculating the total edge information of the target in all directions is shown in formula (1).

[0041]

[0042] E is the sum of edge information of a pixel in all directions.

[0043] The calculation formula shows that the improved Sobel algorithm only includes addition and subtraction operations, multiplication by 2, division by 4, and absolute value operations. FPGAs already have numerous optimized circuits that can implement addition and subtraction operations, ensuring the performance of the implemented circuit. Multiplication by 2 can be implemented by shifting the original data left by 1 bit, avoiding the need to synthesize a multiplication circuit. Division by 4 can be implemented by shifting the original data right by 2 bits, avoiding the need to synthesize a division circuit. The absolute value operation requires determining the sign of the data; if the data is positive, the original data is output directly; if the data is negative, all bits are inverted and then incremented by one. The improved Sobel algorithm avoids square and square root operations, thus significantly reducing the hardware implementation complexity of the algorithm.

[0044] The local mean binarization module 8 determines the binarization result of the center pixel by calculating the average value of all pixels in a 5×5 region surrounding the center pixel. The parallel hardware circuit of the local mean binarization module includes four FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to five shift registers, and the back end of the input image data stream is also connected to five shift registers. These 25 shift registers form a 5×5 register window, and the data in this register window is used for the mean calculation operation. The calculation formulas for local mean binarization are shown in formulas (2) and (3).

[0045]

[0046]

[0047] Among them, P i This represents the value of the center pixel and the 25 pixels surrounding it. AVE represents the average value of the center pixel and the 25 pixels surrounding it. C represents the pixel value of the center pixel. B represents the binarized value of the center pixel.

[0048] Equation (2) requires division. Currently, there is no optimized circuit structure for division in FPGA. If division is implemented directly in FPGA, it will consume a lot of logic resources and produce a large delay. In order to implement the division operation in Equation (2) in FPGA using only a simple circuit structure, this paper improves Equation (2). The improved formula is shown in Equation (4):

[0049]

[0050] As can be seen from formula (4), the denominator of the improved formula is no longer 25, but 1024. Furthermore, the improved formula introduces multiplication. The operation of dividing the data in formula (4) by 1024 can be achieved by shifting the data arithmetic to the right by 10 bits. For the multiplication operation in formula (4), there are currently many optimized circuit structures that can implement it, thus ensuring the performance of the multiplication operation in FPGA implementation.

[0051] During drone flights in the field, it is common to observe localized areas of strong illumination on the drone's surface while other areas are dimly lit. This is primarily due to localized reflections. If a global threshold binarization method is used, the dimly lit edge areas will disappear due to the influence of localized illumination. However, the local mean binarization method does not consider global features but performs binarization based on local region features. This method can effectively binarize even in dimly lit areas.

[0052] The erosion module uses a 3×3 square window for erosion operations. The parallel hardware circuit of the erosion module includes two FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to three shift registers, and three shift registers are also connected to the back end of the input image data stream. These nine shift registers form a 3×3 register window, and the data in this register window is used for the erosion operation.

[0053] The corrosion filter structural element used in this invention

[0054] The expansion module uses a 3×3 square window for expansion operations. The parallel hardware circuit of the expansion module includes two FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to three shift registers, and the back end of the input image data stream is also connected to three shift registers. These nine shift registers form a 3×3 register window, and the data in this register window is used for the expansion operation.

[0055] The dilation filter structure element used in this invention is

[0056] The dilation module is the last module in the image preprocessing module. The image data stream output by the dilation module is already a usable edge image data stream. The data stream output by the dilation module will be directly used in the edge matching module and the template update module.

[0057] The edge matching module implementation scheme is as follows:

[0058] A1. Select a point on the target as the target localization point. The target localization point is selected manually, generally on a key part of the drone where the location is to be maintained. The subsequent target localization algorithm can only be started after the localization point is selected. The manually selected initial localization point is also set as the target localization point on the edge template. The edge template is generated based on the first frame image and is continuously updated in subsequent image frames.

[0059] A2. On the edge template, pixels with a value greater than 6 are set as active points; pixels with a value less than or equal to 6 are set as inactive points. On the edge image, pixels with a value of 1 are set as active points; pixels with a value of 0 are set as active points. The pixel values ​​of the pixels on the edge template are integers between 0 and 15. The pixel values ​​of the pixels on the edge image are integers between 0 and 1.

[0060] A3. In the next frame of the video, move the edge template until the target's location on the edge template coincides with the target's location in the previous frame. The purpose of this step is to move the target on the edge template to the position where the target was in the previous frame.

[0061] A4. Move the edge template within a small range, specifically a 32×32 rectangular area centered on the previous frame's positioning point. At each moving position, calculate the number of activation points in the edge template that coincide with activation points in the edge image. Select the position with the largest number of coinciding activation points as the target positioning point for the current frame.

[0062] A5. Calculate the location matching rate for the current frame. The formula for calculating the location matching rate is shown below.

[0063]

[0064] (y,x)=(m,n)+(y pre ,x pre )-(y ini ,x ini )+(j,i) (6)

[0065] P(j,i)=∑e(m,n)×E(y,x) (7)

[0066]

[0067] T(m,n) represents the value of the edge template at coordinates (m,n). e(m,n) represents the binarized result of the edge template. (y pre ,x pre () represents the location point of the target in the previous frame. (y) ini ,x ini`(j,i)` represents the target's initial location point in the first frame. `(j,i)` represents the displacement of the edge template relative to the location point in the previous frame. `E(y,x)` represents the value of the edge image at coordinates (y,x). `P(j,i)` represents the number of activation points of the edge template that coincide with the activation points of the image edge. `R(j,i)` represents the matching rate when the template's displacement relative to the location point in the previous frame is (j,i). The location matching rate is used to measure the accuracy of the current frame's location. If the location matching rate is below 90%, it indicates that the template and target have failed to match. If the location matching rate is above 90%, it indicates that the template and target have successfully matched. In the case of template and target matching failure, an interrupt signal needs to be sent externally, and the location process should be exited, waiting for the selection of a new initial location point.

[0068] The template matching region of this invention is the entire image area, which means that this invention can locate very large targets or even targets that exceed the field of view, thereby meeting the positioning requirements of extended targets. At the same time, this invention uses edge features as matching features, and edge features are not sensitive to changes in illumination, so this invention can handle situations where the illumination on the target surface changes drastically.

[0069] More specifically, the hardware implementation scheme for the above edge matching module is as follows:

[0070] B1. A point-by-point matching unit of 32×32 window size for the edge image is generated using line buffering technology and shift register method, such as Figure 3 As shown, the generation of the point-to-point matching unit requires 31 FIFOs and a 32×32 rectangular register window. Each FIFO is followed by 32 registers, with no FIFOs preceding the first row of registers. The edge image data stream flows directly into the input terminals of the first row of registers and the first FIFO after exiting the dilation module.

[0071] B2. Count the rows and columns of the input edge image.

[0072] B3. Calculate the coordinates of the starting pixel of the edge template corresponding to the first pixel of the edge image in the current frame, thus realizing the operation in step A3. The formula for calculating the coordinates of the starting pixel of the edge template is shown in formula (9).

[0073] (y tini ,x xini )=(y efirst ,x efirst )+(y ini ,x ini )-(y pre ,x pre )-(16,16) (9)

[0074] Among them, (y tini,x tini ) represents the starting pixel coordinates of the edge template to be calculated. (y efirst ,x efirst ) represents the coordinates of the first pixel in the edge image, which is (0,0) here. ini ,x ini ) is the target's location point initialized in the first frame, which is the target's location point on the template. (y) pre ,x pre () is the target's location point calculated in the previous frame.

[0075] B4. When the pixels of the first edge image flow into the point-to-point matching unit described in B1, the (y) calculated in step B3 is read from the edge template cache module. tini ,x tini The edge template pixel value at the specified coordinates participates in the edge matching operation. For each subsequent incoming edge image pixel, the template pixel at the next address is read from the edge template cache module for edge matching. Because the template moves within a 32×32 range during edge matching, this invention generates 1024 counters in the FPGA. Each counter counts the number of overlapping active points at a given template movement position. Based on the row and column counts, it is determined whether the current edge image pixel is at the image boundary; pixels at the image boundary do not participate in the edge matching operation.

[0076] The template update module updates the edge template using the edge image of each frame in an integral manner. The implementation scheme of the template update module is shown below:

[0077] C1. Create an edge template based on the edge image of the first frame. Use the manually initialized target positioning points in the first frame as the target positioning points on the edge template. Because the target to be located in this invention is an extended target, and the target area occupies more than two-thirds of the field of view, this invention directly sets the size of the edge template to the size of the original image. The formula for creating the edge template is shown in formula (10).

[0078]

[0079] Where E(y,x) represents the pixel value of the edge image at coordinates (y,x), and T(y,x) represents the pixel value of the edge template at coordinates (y,x). Formula (10) shows that in the initial stage of target localization, the present invention uses the target edge of the first frame image to create a target template with the same shape as the original target edge.

[0080] C2. After the entire image data stream of the current frame flows into the edge matching module, the edge matching module can immediately calculate the positioning point of the current frame through parallelized hardware circuitry. Simultaneously, the edge image caching module has completed its edge image caching operation. After calculating the positioning point of the current frame, this invention moves the cached edge image of the current frame until the target positioning point of the current frame's edge image coincides with the target positioning point of the edge template. The purpose of this operation is to move the target on the edge image of the current frame to the position of the target on the edge template, in order to perform subsequent template update operations.

[0081] C3. Update the edge template using an integral method. The template update formula is shown in formula (11).

[0082]

[0083] Where T(m,n) represents the pixel value of the edge template at coordinates (m,n). upd (m,n) represents the pixel value of the updated edge template at image coordinates (m,n). E(m,n) represents the pixel value of the edge image at coordinates (m,n) after the edge image is moved in step C2.

[0084] C4. After completing the target localization for each frame, repeat steps C2 and C3.

[0085] The advantages of using an integral template in this invention are as follows: Compared with a binarized edge template, an integral template can retain historical template information while also timely incorporating current target pose information, thereby enhancing the robustness of the template.

[0086] like Figure 4 As shown, the FPGA hardware circuit implementation scheme for the above template update module is as follows:

[0087] D1. Set the data bit width of the edge template pixels to 4 bits, that is, the pixel value range of the edge template is an integer between 0 and 15.

[0088] D2. After the edge matching module calculates the target location points for the current frame, it sends a template creation or template update enable signal to the template update module. Simultaneously, a flag signal is used to indicate whether the current image is the first frame. If the flag is high, it indicates that the current image is the first frame; if the flag is low, it indicates that the current image is not the first frame. After system reset, the flag is at a high level. After receiving the first frame image data, the flag signal jumps to a low level. If the flag is high, a template creation operation is performed; if the flag is low, a template update operation is performed.

[0089] D3. Calculate the coordinates of the starting pixel of the edge image corresponding to the coordinates of the first pixel on the edge template, thereby realizing the operation of step C2. The formula for calculating the coordinates of the starting pixel of the edge image is shown in formula (12).

[0090] (y eini ,x eini )=(y tfirst ,x tfirst )+(y ini ,x ini )-(y cur ,x cur (12)

[0091] Where (y) eini ,x eini (y) represents the coordinates of the starting pixel in the calculated edge image. tfirst ,x tfirst () represents the coordinates of the first pixel on the edge template, with a value of (0,0). ini ,x ini ) is the target's location point in the first frame, which is the target's location point on the edge template. (y) cur ,x cur ) indicates the location point of the current frame output.

[0092] D4. After receiving the enable signal from the edge matching module, the read / write control circuit reads the pixel values ​​of the four edge templates starting from the address of the first pixel in the edge template cache module, i.e., reading 16 bits of data at a time. The read / write control circuit also reads the pixel values ​​of the four edge images starting from the starting pixel position in the edge image cache module, i.e., reading 4 bits of data at a time. The starting pixel position in the edge image cache module is calculated in step D3. The read pixel values ​​of the four edge templates and the pixel values ​​of the four edge images are simultaneously sent to the integration unit for parallel integration, i.e., updating the pixel points of the four edge templates within a single clock cycle. A single integration circuit is as follows: Figure 5 As shown, in order to achieve parallel integration operations, this invention will... Figure 5 The integrator circuit structure shown is replicated four times. Using parallel template matching speeds up the matching process, allowing the template matching module to access the latest templates promptly.

[0093] D5. The updated edge template value calculated by the integration operation is rewritten into the edge template cache module, thus completing a single template update operation. Then, the next set of edge templates and edge image data is read for integration update and the writing of the updated template value.

[0094] The edge image caching module temporarily caches the preprocessed binary edge image, which is then used for subsequent template update operations. This invention uses a dual-port BRAM to implement the edge image caching, with a cache size of 640×480 bits, determined by the image size. The dual-port BRAM has two read / write ports. The first port is responsible for writing image data output from the dilation module into the BRAM, so it always operates in write mode, with a write data width of 1 bit. The second port is responsible for reading edge image data from the BRAM and providing the read data to the template update module for template update operations, so it always operates in read mode, with a read data width of 4 bits.

[0095] To enable parallel update calculations of four pixels simultaneously in the edge template update module, the read data bit width of the BRAM in the edge image cache module is set to 4 bits. This means that the addresses of the pixels read from the edge image cache module at one time are 4n, 4n+1, 4n+2, and 4n+3, where n is a positive integer. However, if the pixel address (y) calculated in step D3... eini ,x eini If the value is not a multiple of 4, then it is impossible to obtain the 4 edge image pixels required for the update in a single read of the BRAM. The hardware circuit solution of this invention is as follows to address this problem.

[0096] E1. Calculate (y) eini ,x eini The pixel address addr corresponding to the BRAM is calculated using formula (13).

[0097] addr=y eini ×WIDTH+x eini (13)

[0098] Here, WIDTH represents the pixel width of the image.

[0099] E2. Assign the lower two bits of addr to the sel signal. The data width of the sel signal is 2 bits.

[0100] E3. Shift the arithmetic operation of addr two bits to the right and name it addr_shift. Use the addr_shift signal as the read address signal for the BRAM in the edge image buffer module.

[0101] E4. Read the 4-bit data at address addr_shift and the 4-bit data at address (addr_shift+1) from the BRAM. Then concatenate these two 4-bit data into 8-bit data. Assuming that the concatenated 8-bit data is data8, then data8 is as shown in formula (14).

[0102]

[0103] Where d(n) represents the value at address n in the BRAM, and this address is relative to the first read / write port with a read / write data bit width of 1.

[0104] E5. Select the template from data8 according to the sel signal to update the required 4-bit data data4, as shown in formula (15).

[0105]

[0106] E6. Before updating the four edge template pixels of the next group, read the 4-bit data d4 of the next read address of the edge image cache module, shift d4 to the right into the low bits of data8, and use step E5 to select the 4-bit edge image data data4 required for the next group of edge template updates from data8. The right shift formula is shown in formula (16).

[0107] data8 = {data8[3:0], d4} (16)

[0108] The circuit structure described above provides circuit support for the parallel template matching module, thereby enhancing the feasibility of the parallel template matching module.

[0109] An edge template storage module implements edge template caching operations. The cached edge templates are used for subsequent template update and edge matching operations. This invention uses a dual-port BRAM to implement the edge template caching module. The buffer size is 640×480×4 bits, determined by the image template size. The dual-port BRAM has two read / write ports. The first read / write port reads the edge template image and provides it to the edge matching module for edge matching calculations. The second read / write port reads the edge template before updating and writes the updated edge template.

[0110] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A UAV extended target localization system based on FPGA image processing, characterized in that: The system includes an image preprocessing module (1), an edge image caching module (2), a template update module (3), an edge template caching module (4), and an edge matching module (5). The output of the image preprocessing module (1) is connected to the input of the edge image caching module (2) and the input of the edge matching module (5), respectively; the output of the edge image caching module (2) is connected to the input of the template update module (3); the output of the template update module (3) is connected to the input of the edge template caching module (4); the output of the edge template caching module (4) is connected to the input of the template update module (3) and the input of the edge matching module (5), respectively; the output of the edge matching module (5) is connected to the input of the template update module (3); The image preprocessing module (1) includes a median filtering module (6), a Sobel edge extraction module (7), a local mean binarization module (8), an erosion module (9), and a dilation module (10). The output of the median filtering module (6) is connected to the input of the Sobel edge extraction module (7). The output of the Sobel edge extraction module (7) is connected to the input of the local mean binarization module (8). The output of the local mean binarization module (8) is connected to the input of the erosion module (9). The input of the erosion module (9) is connected to the input of the dilation module (10). The median filtering module (6), the Sobel edge extraction module (7), the local mean binarization module (8), the erosion module (9), and the dilation module (10) are each equipped with their own parallel hardware circuits. The template update module (3) is configured to update the edge template using the edge image of each frame in an integral manner; The edge image caching module (2) is configured to temporarily cache the binarized edge image of the preprocessed output; The temporarily cached binarized edge image is used for subsequent template update operations; The edge template caching module (4) is configured to implement temporary caching of edge templates; the cached edge templates are used for subsequent template update operations and edge matching operations.

2. The UAV extended target positioning system based on FPGA image processing according to claim 1, characterized in that: The median filtering module (6) uses a 3×3 square window to filter and reduce noise in the image, and is used to filter out salt and pepper noise in the image; The parallelized hardware circuit of the median filtering module (6) includes two FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to three shift registers, and the back end of the input image data stream is connected to three shift registers. These nine shift registers form a 3×3 register window, and the data in the register window is used for median filtering operations.

3. The UAV extended target positioning system based on FPGA image processing according to claim 1, characterized in that: The Sobel edge extraction module (7) uses a 3×3 window to extract edges in four directions and superimposes the extracted edges in the four directions. The parallel hardware circuit of the Sobel edge extraction module (7) includes two FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to three shift registers, and the back end of the input image data stream is connected to three shift registers. These nine shift registers form a 3×3 register window, and the data in the register window is used for Sobel filtering operations.

4. The UAV extended target positioning system based on FPGA image processing according to claim 1, characterized in that: The local mean binarization module (8) determines the binarization result of the center pixel by calculating the average value of all pixels in a 5×5 area around the center pixel; the parallel hardware circuit of the local mean binarization module (8) includes 4 FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to 5 shift registers, and the back end of the input image data stream is connected to 5 shift registers. These 25 shift registers form a 5×5 register window, and the data in the register window is used for mean calculation operations.

5. The UAV extended target positioning system based on FPGA image processing according to claim 1, characterized in that: The erosion module (9) uses a 3×3 square window to perform erosion operations to remove isolated points on the binarized image. The parallel hardware circuit of the erosion module (9) includes two FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to three shift registers, and the back end of the input image data stream is also connected to three shift registers. These nine shift registers form a 3×3 register window, and the data in the register window is used for the erosion operation.

6. The UAV extended target positioning system based on FPGA image processing according to claim 1, characterized in that: The expansion module (10) uses a 3×3 square window for expansion operation to increase the width of the edge; the parallel hardware circuit of the expansion module (10) includes two FIFOs, which perform row buffering operation on the input image data stream. The output of each FIFO is connected to three shift registers, and the back end of the input image data stream is connected to three shift registers. These nine shift registers form a 3×3 register window, and the data in the register window is used for expansion operation.

7. The UAV extended target positioning system based on FPGA image processing according to claim 1, characterized in that: The edge matching module (5) is configured to evaluate the correlation between the template and the target by using the number of activation points that coincide with the edge template and the edge image. The parallelized hardware circuit of the edge matching module (5) includes 31 FIFOs, which perform row buffering operations on the input image data stream. The output of each FIFO is connected to 32 shift registers, and the back end of the input image data stream is also connected to 32 shift registers. These 1024 shift registers form a 32×32 register window, and the data in the register window is used for template matching operations.

Images