Multi-light-spot centroid extraction method based on FPGA and system applicable to multi-light-spot centroid extraction method

Abstract

The invention relates to an FPGA-based multi-spot centroid extraction method and an applicable system thereof, and relates to the technical field of in-orbit laser communication links. The method comprises the following steps: step 1, image preprocessing; step 2, scanning line by line, and recording current line information; 3, run length merging, and current row information updating; and 4, judging whether one frame of image is traversed or not, and calculating the coordinates of the center of mass. According to the multi-light-spot centroid extraction method based on the FPGA and the system applicable to the method, the parallel processing advantage of the FPGA can be exerted, statistical information can be obtained by traversing the image once, two lines of binary image information are cached by utilizing a neighborhood pipeline processing method, resource occupation is reduced, and the running speed of the system is increased.

Description

Technical Field

[0001] The present invention relates to the technical field of on-orbit laser communication links, and in particular to an FPGA-based multi-spot centroid extraction method and an applicable system thereof. Background Art

[0002] High-precision extraction of a single laser spot typically uses the centroid method or surface fitting method based on image grayscale distribution. However, because the detector is affected by the complexity of the spatial light environment, the calculated centroid of the spot is inaccurate, which affects the coordinate extraction of the centroid.

[0003] Existing methods for distinguishing multiple centroids are all designed based on DSP and other algorithms. They all need to read an image, find the effective light spot and then extract it, resulting in a long response time, slow speed and inability to achieve real-time processing. Summary of the Invention

[0004] The present invention aims to solve the technical problems in the prior art and provides a multi-spot centroid extraction method based on FPGA and an applicable system thereof.

[0005] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0006] A multi-spot centroid extraction method based on FPGA, comprising the following steps:

[0007] Step 1: Image preprocessing;

[0008] FPGA obtains images through the CMOS camera. The collected images enter the algorithm processing. After passing through the Gaussian filter module, the image variance is calculated and threshold processing is performed to obtain a binary image.

[0009] Step 2: Scan line by line and record the current line information;

[0010] The binary image is divided into foreground and background, with the background being 0 and the foreground being 1. The image is scanned in a pipeline from left to right, treating each row of the binary image as a run, and recording the start and end coordinates, number, and assigned pre-marks of the run.

[0011] Step 3: Merge the runs and update the current row information;

[0012] By scanning line by line for judgment and comparison, the continuous foreground pixels belonging to the same object are given the same label and classified as a run;

[0013] Step 4: Determine whether a frame of image has been traversed and calculate the centroid coordinates;

[0014] After scanning the entire image once, the marking result is obtained; the run with the largest number of marks is the laser spot; the center of mass position is calculated through the coordinate values ​​corresponding to the marks in the memory.

[0015] In the above technical solution, step three is specifically as follows:

[0016] In a given binary image, the start of the Nth row run is determined to be two adjacent pixels 01, and the end of the run is determined to be two adjacent pixels 10;

[0017] Combine adjacent 1-value pixels into regions and use boundary information to describe connected domains;

[0018] When traversing the N+1th row, the same operation is performed to determine the starting and ending positions, assign marks, and determine whether it is connected to the Nth row. The judgment is mainly based on the eight-connected row coordinates:

[0019] X0≤x i+1 +1,

[0020] X1≥x i +1

[0021] Among them, X0 and X1 represent the starting and ending row coordinates of the current row respectively; i , x i+1 They respectively represent the starting and ending row coordinates of all runs in the previous row; when the above two formulas are satisfied at the same time, it is determined that the two runs are connected, and the temporary run mark of X0~X1 is changed to the equivalent pair mark of x0~x1 for storage.

[0022] In the above technical solution, in step 4, the process of calculating the center of mass coordinates is:

[0023] After completing the marking of all rows of a picture, obtain the largest number of run mark values ​​from the memory, as well as the start row coordinates, end row coordinates and ordinates of all runs corresponding to the mark value. The centroid calculation formula is:

[0024]

[0025]

[0026] Among them, X and Y are the center row and vertical coordinate point of the spot respectively, and S is the sum of all row coordinates in the previous row.

[0027] x i , x i+1 Respectively represent the starting and ending row coordinates of all runs in the previous row;

[0028] S i is the sum of all run row coordinates, y iis the vertical coordinate of all runs, and n is the number of all runs.

[0029] A system applicable to the above-mentioned FPGA-based multi-spot centroid extraction method includes: a CMOS image output module, a line buffer module, a filtering module, a threshold calculation module, a run-length storage and discrimination module, a connected domain discrimination module, and a centroid algorithm module connected in sequence;

[0030] The CMOS image output module is used to align the three rows of data in a frame of image and then output them;

[0031] The row buffer module is used to implement row buffer using three FIFOs;

[0032] The filtering module is used to perform Gaussian filtering calculations to obtain an image with Gaussian noise removed;

[0033] The threshold calculation module is used to calculate the mean and variance of the image after removing Gaussian noise to obtain the threshold value and output a binary image.

[0034] The run storage and discrimination module is provided with a first memory and a second memory, which is used to generate a write enable signal when a start signal is recognized, record the current starting row coordinate, ordinate and sequence mark value in the first memory, and start to accumulate the number of pixels contained in the run and record it in the second memory until a run end signal is determined, and record the current end row coordinate in the first memory;

[0035] The connected domain identification module is used to read image pixels from left to right row by row, identify all runs, and determine whether adjacent row runs are connected. If so, it updates the tag value information of the current row in the first memory and merges the counts of the same run tags in the second memory; if not, it assigns different tag values;

[0036] The centroid algorithm module is used to compare the pixel counts in the second memory in sequence to find the run mark value corresponding to the maximum count; read the row coordinates and vertical coordinates corresponding to the maximum mark value from the first memory to calculate the centroid coordinates.

[0037] The present invention has the following beneficial effects:

[0038] The FPGA-based multi-spot centroid extraction method and its applicable system of the present invention can give full play to the advantages of FPGA parallel processing, obtain statistical information by traversing the image once, and use the neighborhood pipelining method to cache two lines of binary image information, thereby reducing resource usage and improving the system's operating speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] Figure 1 This is a flow chart of the FPGA-based multi-spot centroid extraction method of the present invention.

[0041] Figure 2 Schematic diagram of labeled binary image.

[0042] Figure 3 Schematic diagram of the run-length merging principle.

[0043] Figure 4 It is a structural schematic diagram of a system applicable to the FPGA-based multi-spot centroid extraction method of the present invention.

[0044] Figure 5 Schematic diagram of the simulated test image.

[0045] Figure 6 This is a simulation timing diagram. DETAILED DESCRIPTION

[0046] The present invention will be described in detail below with reference to the accompanying drawings.

[0047] like Figure 1 As shown, the FPGA-based multi-spot centroid extraction method of the present invention includes the following steps:

[0048] Step 1: Image preprocessing;

[0049] The FPGA obtains images through the CMOS camera. The collected images enter the algorithm processing. After passing through the 3×3 Gaussian filter module, the image variance is calculated and the threshold is processed to obtain a binary image.

[0050] Step 2: Scan line by line and record the current line information;

[0051] like Figure 2 As shown, a single-frame binary image is divided into foreground and background, with the background assigned to 0 and the foreground to 1. Due to the hardware characteristics of FPGAs, real-time pipeline operations on images are typically performed on a row-by-row basis, preventing the complete image from being captured in one pass. Therefore, this design employs a left-to-right pipeline scan, treating each row of the binary image as a run. The run's start and end coordinates, number, and assigned pre-labels are recorded to prepare for feature extraction in the next step.

[0052] Step 3: Merge the runs and update the current row information;

[0053] like Figure 3 As shown, by scanning line by line for judgment and comparison, the continuous foreground pixels belonging to the same object are given the same label and classified into one run, distinguishing them from other discontinuous foreground pixels.

[0054] In step three, the basic principle of the run marking algorithm is:

[0055] In a given binary image, the start of a run in row N is determined by two adjacent pixels "01," and the end of a run is determined by two adjacent pixels "10." That is, when the current pixel is "0" and the next pixel in the same row is "1," it signals the start of a new run and is assigned a new temporary marker. Similarly, when the values ​​of the following pixels in the row transition from "1" to "0," the run ends. Adjacent "1" pixels are grouped into regions, and their boundaries are used to describe connected domains.

[0056] When traversing the N+1th row, the same operation is performed to determine the starting and ending positions, assign marks, and determine whether it is connected to the Nth row. The judgment is mainly based on the eight-connected row coordinates:

[0057] X0≤x i+1 +1,

[0058] X1≥x i +1

[0059] X0 and X1 represent the starting and ending coordinates of the current row. i , x i+1 Indicates the row coordinates of the starting and ending points of all runs in the previous row. When the above two formulas are satisfied at the same time, it is determined that the two runs are connected, and the temporary run mark of X0~X1 is changed to the equivalent pair mark of x0~x1 for storage.

[0060] Step 4: Determine whether a frame of image has been traversed and calculate the centroid coordinates;

[0061] After scanning the entire image once, the marking results are obtained. Image analysis for laser spot extraction is straightforward. The spot shape is relatively regular, stray spots and bad spots are smaller than normal spots, and there are no "U"-shaped or "E"-shaped connected areas that could affect algorithm performance. Therefore, the centroid is defined as the area with the largest area. In other words, the run with the most markings is the laser spot. The centroid position can be calculated using the coordinates corresponding to the markings in memory.

[0062] In step 4, the process of calculating the center of mass coordinates is:

[0063] After completing the marking of all rows of a picture, obtain the largest number of run mark values ​​from the memory, as well as the start row coordinates, end row coordinates and ordinates of all runs corresponding to the mark value. The centroid calculation formula is:

[0064]

[0065] Among them, X and Y are the center row and ordinate point of the spot respectively, and S is the sum of all row coordinates of the previous row;

[0066] x i , x i+1 Respectively represent the starting and ending row coordinates of all runs in the previous row;

[0067] S i is the sum of all run row coordinates, y i is the vertical coordinate of all runs, and n is the number of all runs.

[0068] like Figure 4 As shown, the system to which the FPGA-based multi-spot centroid extraction method of the present invention is applicable includes: a CMOS image output module, a line buffer module, a filtering module, a threshold calculation module, a run-length storage and discrimination module, a connected domain discrimination module, and a centroid algorithm module connected in sequence;

[0069] The FPGA logic implementation process in this system is as follows:

[0070] (1) Use FPGA internal storage resources to design line buffers for image pixels:

[0071] The CMOS image output module aligns the three rows of data in a frame and outputs them as a 3×3 window. The image size is 512×512, with an 8-bit pixel count.

[0072] The row buffer module uses three 1024-deep FIFOs to implement row cache. The first row of data is written into FIFO_1. After the first row is written, it is transferred to FIFO_2. And so on. Three FIFOs can store three rows of data.

[0073] When the fourth line arrives, the data is read together, the data is aligned, and the filtering module performs Gaussian filtering calculation to obtain an image with Gaussian noise removed;

[0074] The threshold calculation module calculates the mean and variance of the image obtained by removing Gaussian noise to obtain a threshold value, and outputs a binary image.

[0075] (2) The processed pixels are input into the connected domain discrimination module in groups of two.

[0076] In the run storage and discrimination module, a first memory ( Figure 4 RAM1 in the , with a width of 27 and a depth of 8192, records the run information; define a second memory ( Figure 4 RAM2 in the memory has a width of 16 and a depth of 50, and records the number of pixels in each run.

[0077] Table 1 First memory recording information

[0078]

[0079] When the start signal is identified, a write enable signal is generated, the current starting row coordinate, vertical coordinate and sequence mark value are recorded in the first memory, and the number of pixels contained in the run is counted cumulatively and recorded in the second memory until the run end signal is judged and the current end row coordinate is recorded in the first memory.

[0080] The connected domain identification module reads image pixels row by row from left to right, identifies all runs, and determines whether adjacent row runs are connected. If so, it updates the mark value information of the current row in the first memory and merges the counts of the same run marks in the second memory; if not, it assigns different mark values.

[0081] After the image traversal is completed, the memory information is immediately read. The pixel counts in the second memory are compared sequentially using the centroid algorithm module to find the run mark value corresponding to the maximum count, which is the light spot. The row coordinates and vertical coordinates corresponding to the maximum mark value are read from the first memory, and the centroid coordinates are calculated according to equations (1) and (2) respectively.

[0082] The present invention uses Microchip's SmartFution2 series FPGA as the hardware platform to mainly complete the CMOS camera driving, image acquisition and algorithm implementation. The Libero design suite is used as the development environment, and the Verilog hardware description language is used for algorithm module design and simulation. Figure 5 The test image shown is 512×512 in size and contains seven regularly shaped connected objects, simulating beacon light and other light spots caused by stray light, which account for 0.18% of the entire target surface. The maximum frame rate of the image is 258fps, the operating frequency is f=80Mhz, and the time consumption for calculating one image is 3.9ms. The simulation timing diagram is as follows: Figure 6 .

[0083] The extracted beacon light pixel count is 278, the marker value is 32, and the final calculated centroid coordinates are (212, 338). The centroid coordinates are calculated and output when the synchronization is set low in each frame.

[0084] By shifting the numerator and denominator in equations (1) and (2) to the left in the FPGA operation, sub-pixel subdivision can be achieved. The centroid operation retains two decimal places and the result is (212.72, 338.95). Verification is performed using MATLAB, and the centroid coordinates are calculated to be (212.726, 338.48), r = 0.475, and the resolution is more than 1 / 2 pixel.

[0085] The FPGA-based multi-spot centroid extraction method proposed in this paper is suitable for regional images with relatively regular connected areas. It can generate the centroid coordinates of the light spots with a single image scan. Experimental results demonstrate that the coordinate resolution is greater than 1 / 2 pixel, and the spot miss distance update frequency is 258Hz, meeting real-time requirements.

[0086] The FPGA-based multi-spot centroid extraction method of the present invention has a fixed processing time for each frame and is very suitable for realizing the FPGA stream processing function based on the characteristics of parallel computing.

[0087] The FPGA-based multi-spot centroid extraction method and its applicable system of the present invention can give full play to the advantages of FPGA parallel processing, obtain statistical information by traversing the image once, and use the neighborhood pipelining method to cache two lines of binary image information, thereby reducing resource usage and improving the system's operating speed.

[0088] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A multi-spot centroid extraction method based on FPGA, characterized in that: The following steps are involved: Step 1: Image preprocessing; FPGA obtains images through the CMOS camera. The collected images enter the algorithm processing. After passing through the Gaussian filter module, the image variance is calculated and threshold processing is performed to obtain a binary image. Step 2: Scan line by line and record the current line information; The binary image is divided into foreground and background, with the background being 0 and the foreground being 1. The image is scanned in a pipeline from left to right, treating each row of the binary image as a run, and recording the start and end coordinates, number, and assigned pre-marks of the run. Step 3: Merge the runs and update the current row information; By scanning line by line for judgment and comparison, the continuous foreground pixels belonging to the same object are given the same label and classified as a run; Step 4: Determine whether a frame of image has been traversed and calculate the centroid coordinates; After scanning the entire image once, the marking result is obtained; the run with the largest number of marks is the laser spot; the center of mass position is calculated through the coordinate values ​​corresponding to the marks in the memory.

2. The FPGA-based multi-spot centroid extraction method according to claim 1, characterized in that: Step three is as follows: In a given binary image, the start of the Nth row run is determined to be two adjacent pixels 01, and the end of the run is determined to be two adjacent pixels 10; Combine adjacent 1-value pixels into regions and use boundary information to describe connected domains; When traversing the N+1th row, the same operation is performed to determine the starting and ending positions, assign marks, and determine whether it is connected to the Nth row. The judgment is mainly based on the eight-connected row coordinates: X0≤x i+1 +1, X1≥x i +1 Among them, X0 and X1 represent the starting and ending row coordinates of the current row respectively; i , x i+1 They respectively represent the starting and ending row coordinates of all runs in the previous row; when the above two formulas are satisfied at the same time, it is determined that the two runs are connected, and the temporary run mark of X0~X1 is changed to the equivalent pair mark of x0~x1 for storage.

3. The FPGA-based multi-spot centroid extraction method according to claim 1, characterized in that: In step 4, the process of calculating the center of mass coordinates is: After completing the marking of all rows of a picture, obtain the largest number of run mark values ​​from the memory, as well as the start row coordinates, end row coordinates and ordinates of all runs corresponding to the mark value. The centroid calculation formula is: Among them, X and Y are the center row and vertical coordinate point of the spot respectively, and S is the sum of all row coordinates in the previous row. x i , x i+1 Respectively represent the starting and ending row coordinates of all runs in the previous row; S i is the sum of all run row coordinates, y i is the vertical coordinate of all runs, and n is the number of all runs.

4. A system applicable to the FPGA-based multi-spot centroid extraction method according to claim 1, characterized in that: The module comprises: a CMOS image output module, a line buffer module, a filtering module, a threshold calculation module, a run-length storage and discrimination module, a connected domain discrimination module, and a centroid algorithm module, which are connected in sequence; The CMOS image output module is used to align the three rows of data in a frame of image and then output them; The row buffer module is used to implement row buffer using three FIFOs; The filtering module is used to perform Gaussian filtering calculations to obtain an image with Gaussian noise removed; The threshold calculation module is used to calculate the mean and variance of the image after removing Gaussian noise to obtain the threshold value and output a binary image. The run storage and discrimination module is provided with a first memory and a second memory, which is used to generate a write enable signal when a start signal is recognized, record the current starting row coordinate, ordinate and sequence mark value in the first memory, and start to accumulate the number of pixels contained in the run and record it in the second memory until a run end signal is determined, and record the current end row coordinate in the first memory; The connected domain identification module is used to read image pixels from left to right row by row, identify all runs, and determine whether adjacent row runs are connected. If so, it updates the tag value information of the current row in the first memory and merges the counts of the same run tags in the second memory; if not, it assigns different tag values; The centroid algorithm module is used to compare the pixel counts in the second memory in sequence to find the run mark value corresponding to the maximum count; read the row coordinates and vertical coordinates corresponding to the maximum mark value from the first memory to calculate the centroid coordinates.

Images