A hardware implementation algorithm for image connected component labeling

Abstract

This invention relates to the field of image processing technology, and in particular to a hardware implementation algorithm for image connected component labeling. The hardware implementation architecture of the connected component labeling algorithm adopts a pixel-based approach for connected component labeling. The equivalence table structure uses a one-dimensional array. The computational architecture includes an initial labeling and equivalence table storage module, an equivalence table copying and updating module, a coordinate recording and updating module, and an adjacent row label storage and comparison module. Based on the characteristics of connected components in binary images, this invention performs copying and updating operations on the equivalence table according to the image behavior cycle, effectively reducing processing latency and forming a high-efficiency computational architecture for connected component labeling based on FPGA.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a real-time hardware processing algorithm for image connected component labeling based on FPGA. Background Technology

[0002] In fields such as image processing, object detection, and pattern recognition, feature extraction is an essential step. Therefore, connected component labeling algorithms, most commonly used for target candidate region detection and extraction, are widely employed. Compared to image processing algorithms like image filtering and thresholding that can be pipelined, connected component labeling algorithms, due to their higher complexity, often require more hardware resources and processing time.

[0003] FPGA is a programmable logic device with advantages such as low power consumption, low cost, flexible development, and high processing speed. FPGA can develop independent hardware circuit structures for each system, thus combining the characteristics of parallel structure and pipelined processing.

[0004] The key to meeting real-time requirements is to efficiently implement the connected component labeling algorithm, which addresses the real-time challenges of connected component labeling implementation. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a hardware implementation algorithm for image connected component labeling. Based on the characteristics of connected components in binary images, the algorithm performs operations such as copying and updating the equivalence table according to the image behavior cycle, effectively reducing processing latency and forming an efficient computing architecture for connected component labeling based on FPGA.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a hardware implementation algorithm for image connected component labeling. The hardware implementation architecture of the connected component labeling algorithm adopts a pixel-based approach for connected component labeling. The equivalence table structure uses a one-dimensional array. The computational architecture includes an initial labeling and equivalence table storage module, an equivalence table copying and updating module, a coordinate recording and updating module, and an adjacent row label storage and comparison module.

[0007] 1) Initial labeling and equivalence table storage module: a) Use a filtering template to perform initial labeling sliding window processing on the input binary image, assign a label value to each pixel with a value of "1", and store the detected equivalent labels in the equivalence table; b) Output the coordinates of pixels with a grayscale value of "1"; c) Detect and output the labels that appear in two adjacent rows during the sliding window process.

[0008] 2) Equivalence table copying and updating module: Copy the latest equivalence table E1 in a row cycle, and traverse and update E1 in the next row cycle of the image;

[0009] 3) Adjacent Row Tag Storage and Comparison Module: a) Store the tags of two adjacent rows that have appeared during the tagging process to obtain a tag set; b) Update the adjacent row tag set by obtaining the updated E1; c) By comparing the updated tag set, obtain the tag set {L} of the output region of the previous row in the two adjacent rows. O};

[0010] 4) Coordinate Recording and Update Module: a) Store the coordinates of pixels with a value of "1" during the marking process; b) Update the coordinate table according to the updated E1; c) According to the output marker {L O Extract the coordinates and pixel count of the corresponding connected component, calculate and output the centroid coordinates and radius of the connected component.

[0011] In some embodiments, the calculation process is as follows:

[0012] 1) First row cycle: A total of 4 operations need to be completed. The first is to mark the image data of row N-1, recording the equivalence information in the equivalence table in real time; the second is to detect the markers {L} that appeared in rows N-2 and N-1. N-2} and {L N-1 The third step is to copy the equivalence table and denote it as E1 while marking the N-1th row; the fourth step is to copy the coordinate table and denote it as C1 while marking the N-1th row.

[0013] 2) Second row cycle: Mark the Nth row of data. This requires three operations: first, traversing and updating the equivalence table E1; second, using the updated E1, storing the mark sets {L} of the (N-2)th and (N-1)th rows. N-2} and {L N-1} is updated to get {L N-2}' and {L N-2 The third method involves updating the coordinate table C1 using the updated E1 to obtain C1'.

[0014] 3) Third row cycle: Mark the (N+1)th row of data, and simultaneously update the mark set {L} N-2}' and {L N-1}' performs a label comparison to obtain {L N-2 The set of connected component labels that can be output from}' {L O};

[0015] 4) Fourth row cycle: Mark the data in the (N+2)th row, and simultaneously calculate and output the mark set {L}. O The centroid coordinates and radius of the connected components within the domain;

[0016] As can be seen from the above operations, when the image is marked with pixels in the (N+2)th row, the output region can be extended to the connected region that stops at the (N-2)th row. The output delay is at most 4 row cycles, starting from the end of the extended connected region.

[0017] In some embodiments, the process of pixel marking and equivalence table storage in the initial marking and equivalence table storage module is as follows:

[0018] Stage 1: Tag context modeling, using a preset window for tag context modeling;

[0019] Stage 2: Pixel-by-pixel marking. Based on the preset window size used for marking, a corresponding number of FIFOs are used to buffer row data, and a corresponding number of registers are used to buffer data in the filtering window.

[0020] Stage 3: Equivalence relation context modeling. To simplify equivalence relation judgment, a 2×2 window is used.

[0021] Stage 4.1: Equivalence relation judgment and storage. For the clock cycles of pixel judgment and assignment that need to be marked in the modeling process, that is, after the data stream flows through the preset window of stage 1, it is delayed by the same number of clock cycles of the same column of the preset window before it can directly flow into the window shown in stage 3 for sliding window. Therefore, no FIFO is needed for data buffering, and only 4 registers are needed to form the sliding window. When performing equivalence relation judgment, data in the 2×2 window is retrieved in pairs. If the pixel values ​​are the same (both are 1), but the mark values ​​are different, it means that there are marks with equivalence relation. This pair of marks is written into the equivalence table.

[0022] Stage 4.2: Detects markers appearing in adjacent rows. Using the sliding window shown in Stage 3, when {a,b} = {0,1} or {d,e} = {0,1} is detected, the marker values ​​for b and e are temporarily stored until the extension of the marker values ​​within the same row disappears. Analysis shows that it is not necessary to store all markers appearing in both rows when determining which markers have appeared in adjacent rows.

[0023] The markers in row N-1 that are connected to row N, such as 'b', do not need to be stored because the region represented by marker 'b' still extends into row N. Similarly, the markers in row N that are not connected to row N-1, such as 'c', also do not need to be stored because marker 'c' is a newly created marker and is not locally connected to row N-1. Therefore, only the markers in row N-1 that are not connected to row N, such as 'a', and the markers in row N that are connected to row N-1, such as 'd', need to be stored. This marker storage method reduces the computational load of comparing markers in subsequent adjacent rows.

[0024] In some embodiments, the equivalence table copying and updating module includes:

[0025] Since both equivalence table copying and updating are performed on a row-by-row basis, in order to construct a pipelined processing structure for equivalence table copying and updating operations, a "ping-pong operation" is used, which utilizes two RAMs to alternately copy and update the equivalence table.

[0026] flag=1: The row count of the current marked row is odd. RAM1 is used to store the copied equivalence table data, and RAM2 is used to update the equivalence table.

[0027] flag=0: The row count is even, RAM1 is used to update the equivalence table, and RAM2 is used to store the copied equivalence table data;

[0028] The equivalence table is copied and updated in an alternating and seamless switching manner as described above, realizing pipelined processing of equivalence table updates. The overall processing delay of the module is 1 row cycle.

[0029] In some embodiments, the main process of updating the state machine using the equivalence table is as follows:

[0030] a. When the state machine is in the IDLE state, it detects the start signal of a row and jumps to the S1 state;

[0031] b. In state S1, first determine whether the current read address is equal to the maximum label of the equivalence table;

[0032] c. If they are equal, it means that the equivalence table has been updated and jump to the IDLE state. If they are not equal, according to the value of flag, read the flag values ​​stored in RAM1 or RAM2 in ascending order of address, and then jump to S2.

[0033] d. In state S2, the read tag value label is compared with the address value addr. If they are equal, this tag is the root tag, and the process jumps to S1. If they are not equal, the process jumps to S3.

[0034] e. In state S3, read the data E(label) in RAM with label as the address, make addr = label and label = E(label), and check again whether label and address value addr are equal. If they are equal, write the current label value into the original address, and then jump to S1. If they are not equal, jump to S4.

[0035] f. In state S4, the judgment in S3 is repeated. If they are equal, jump to S1; otherwise, jump to S5.

[0036] g. In state S5, the latest label value read from the iteration is written to the initial address, and then the process jumps to S1;

[0037] After the above process, the equivalence table is updated, meaning that all labels within the same connected domain are replaced with their root labels.

[0038] In some embodiments, the storage, update, and comparison process of the adjacent row marker storage and comparison module is as follows:

[0039] Stage 1: Adjacent row labels are stored. When labeling the Nth row of the image data, the labels of the (N-1)th and Nth rows are detected by a sliding window. m and l n Store the data to form a tag set {L} N-1} and {L N};

[0040] Stage 2: Adjacent row markers are updated; {L} is adjusted based on the updated equivalence table E1. N-1} and {L N} is updated to get {L N-1}' and {L N To achieve a pipelined processing structure that stores and updates adjacent row tags, a pipelined architecture based on row processing cycles is constructed using "ping-pong operations".

[0041] Stage 3: Adjacent row tag comparison. After obtaining the root tags that have appeared in the (N-1)th row and the Nth row, it is necessary to determine which root tags appear in the (N-1)th row but not in the Nth row by comparing the tags. Register logic operations are used to replace the adjacent row tag traversal and matching.

[0042] In some embodiments, in stage 2 of the adjacent row tag storage and comparison module, the pipelined processing structure for storing and updating adjacent row tags is as follows:

[0043] flag=1: The row count of the row currently being marked is odd. FIFO1 and FIFO2 are used to store adjacent row tags, and FIFO3 and FIFO4 are used to update adjacent row tags.

[0044] flag=0: The row count of the currently marked row is even. FIFO3 and FIFO4 are used to store adjacent row tags, and FIFO1 and FIFO2 are used to update adjacent row tags.

[0045] In some embodiments, the register logic operation in the process stage 3 of the adjacent row mark storage and comparison module is as follows: two registers label_1 and label_2 with a depth equal to the width of the image column are used to represent the marks that have appeared in the N-1 row and the N row. For example, if mark 3 appears in the N-1 row, then label_1[3] = 1, and so on. Then, the corresponding bit of the same index in registers label_1 and label_2 is judged. If it is "10", it means that the mark corresponding to this index in the N-1 row can be output. Otherwise, it is not output. In this way, the time of the adjacent row mark matching operation can be limited to a fixed row cycle.

[0046] In some embodiments, the coordinate recording and updating module process involves constructing a coordinate table with the same structure as the equivalent table. The operations on the coordinate table include recording, updating, copying, and outputting. Each operation requires one RAM. Therefore, this module requires a total of 4 RAMs, which are referred to as RAM1, RAM2, RAM3, and RAM4, respectively.

[0047] The recording operation of the coordinate table is the same as that of the equivalent table, except that the saved values ​​are changed from markers to accumulated coordinate values;

[0048] The copying, updating, and output of the coordinate table are also performed in a row-by-row cycle and have a sequential relationship. In order to build a pipeline structure, the "ping-pong operation" is also used to make RAM2, RAM3, and RAM4 alternately copy, update, and output the coordinate table. The specific operation is similar to the equivalent table operation inside the above modules. After obtaining the outputtable tag from the adjacent tag storage and comparison module, the coordinate sum stored in RAM is read out using this tag as the address. Then, the centroid coordinates need to be calculated by division.

[0049] In some embodiments, in order to build a pipelined operation, a lookup table is built using ROM to perform division calculations, and the centroid coordinate calculation can be completed in one clock cycle.

[0050] Compared with the prior art, the beneficial effects of the present invention are: by analyzing the shape characteristics of connected components in the image and combining the update operation of the equivalence table in the connected component label calculation, based on the FPGA device architecture, a method of copying and updating the equivalence table in the image row cycle is adopted to design an FPGA-based connected component label calculation algorithm. Compared with the common calculation architecture, the calculation latency can be reduced from 1 frame cycle to 4 row cycles, effectively reducing the processing latency.

[0051] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. The embodiments of this application will provide a detailed description and understanding of the application. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the hardware architecture for connected component labeling in the technical solution of this invention;

[0053] Figure 2 This is a flowchart illustrating the connected component labeling calculation process in a specific embodiment of the present invention;

[0054] Figure 3 This is a diagram of the pixel marking and equivalence relation storage pipeline structure in a specific embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of the tagging context modeling in a specific embodiment of the present invention;

[0056] Figure 5 This is a schematic diagram of equivalence relation context modeling in a specific embodiment of the present invention;

[0057] Figure 6 This is a structural diagram of the equivalence table in a specific embodiment of the present invention;

[0058] Figure 7 This is a schematic diagram illustrating the connectivity of two adjacent rows of markers in a specific embodiment of the present invention;

[0059] Figure 8 This is a schematic diagram illustrating the ping-pong operation of copying and updating the equivalent table in the technical solution of this invention;

[0060] Figure 9 This is a schematic diagram of the state machine jump for updating the equivalence table in a specific embodiment of the present invention;

[0061] Figure 10 This is a flowchart illustrating the storage, update, and comparison calculation process for adjacent rows in a specific embodiment of the present invention.

[0062] Figure 11 This is a schematic diagram of the adjacent row copying and updating ping-pong operation in a specific embodiment of the present invention;

[0063] Figure 12 This is a schematic diagram illustrating the comparison of adjacent row marker traversal in a specific embodiment of the present invention;

[0064] Figure 13 This is a schematic diagram of the logical operation of the adjacent row marker register in a specific embodiment of the present invention;

[0065] Figure 14 This is a schematic diagram of the coordinate table structure in a specific embodiment of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] Example 1

[0068] This invention provides a technical solution: a hardware implementation algorithm for image connected component labeling. The hardware implementation architecture adopts a pixel-based approach for connected component labeling, and the equivalence table structure uses a one-dimensional array, such as... Figure 1 As shown, the computational architecture includes four modules: an initial tagging and equivalence table storage module, an equivalence table copying and updating module, a coordinate recording and updating module, and an adjacent row tag storage and comparison module.

[0069] 1) Initial tagging and equivalence table storage module:

[0070] a. Use a filtering template to perform initial labeling sliding window processing on the input binary image, assign a label value to each pixel with a value of "1", and store the detected equivalent labels in the equivalence table;

[0071] b. Output the coordinates of pixels with a grayscale value of "1"; c. Detect and output the markers that appear in two adjacent rows during the sliding window process.

[0072] 2) Equivalence table copying and updating module:

[0073] The latest equivalent table E1 is copied in the row cycle, and E1 is traversed and updated in the next row cycle of the image.

[0074] 3) Adjacent row marker storage and comparison module:

[0075] a. Store the two adjacent rows of tags that appear during the tagging process to obtain a tag set;

[0076] b. Obtain the updated E1 and update the adjacent row tag set;

[0077] c. By comparing the updated tag set, obtain the tag set {L} of the output region of the previous row in two adjacent rows. O}

[0078] 4) Coordinate recording and update module:

[0079] a. Store the coordinates of pixels with a value of "1" during the marking process;

[0080] b. Update the coordinate table based on the updated E1;

[0081] c. Based on the output marker {L O Extract the coordinates and pixel count of the corresponding connected component, calculate and output the centroid coordinates and radius of the connected component.

[0082] Example 2

[0083] Based on the architecture of Example 1, the algorithm calculation process is as follows: Figure 2 As shown.

[0084] 1) First row cycle: A total of 4 operations need to be completed. The first is to mark the image data of row N-1, recording the equivalence information in the equivalence table in real time; the second is to detect the markers {L} that appeared in rows N-2 and N-1. N-2} and {L N-1 The third is to copy the equivalence table and denote it as E1 while marking the N-1th row; the fourth is to copy the coordinate table and denote it as C1 while marking the N-1th row.

[0085] 2) Second row cycle: Mark the Nth row of data. This requires three operations: first, traversing and updating the equivalence table E1; second, using the updated E1, storing the mark sets {L} of the (N-2)th and (N-1)th rows. N-2} and {L N-1} is updated to get {L N-2}' and {L N-2 The third method involves updating the coordinate table C1 using the updated E1 to obtain C1'.

[0086] 3) Third row cycle: Mark the (N+1)th row of data, and simultaneously update the mark set {L} N-2}' and {L N-1}' performs a label comparison to obtain {L N-2 The set of connected component labels that can be output from}' {L O}

[0087] 4) Fourth row cycle: Mark the data in the (N+2)th row, and simultaneously calculate and output the mark set {L}. O The centroid coordinates and radius of the connected domain in the}.

[0088] As can be seen from the above operations, when the image is marked with pixels in the (N+2)th row, the output region can be extended to the connected region that stops at the (N-2)th row. The output delay is at most 4 row cycles, starting from the end of the extended connected region.

[0089] Example 3

[0090] The module design and optimization methods are explained in detail based on the process described in Example 2:

[0091] 1) Initial tagging and equivalence table storage module

[0092] The pipeline structure for pixel markers and equivalence table storage is as follows: Figure 3 As shown:

[0093] Stage 1: Marking context modeling. Using, for example... Figure 4 The window shown is used for labeling context modeling. Pixel e is the center pixel that needs to be labeled, and pixels a, b, c, and d are pixels that have already been labeled, with label values ​​l and l, respectively. a l b l c and l d .

[0094] Stage 2: Pixel-by-pixel marking. Since marking uses a 2×3 window, only one FIFO is needed to buffer row data, and five registers are used to buffer data in the filtering window.

[0095] Stage 3: Equivalence relation context modeling. To simplify equivalence relation determination, we use methods such as... Figure 5 The 2×2 window shown.

[0096] stage4.1: Equivalence relation determination and storage. e The judgment and assignment process requires a total of 3 clock cycles, which means the data stream needs to flow through... Figure 4 After the window is set, a further delay of 3 clock cycles allows for direct flow. Figure 5 The window shown uses a sliding window, so there is no need for FIFO data buffering; only four registers are needed to form the sliding window. When determining equivalence, data from the 2×2 window is retrieved in pairs. If the pixel values ​​are the same (both are 1), but the tag values ​​are different, it means that there are tags with an equivalence relationship, and this pair of tags is written into the equivalence table.

[0097] like Figure 6 As shown, the equivalence table is a one-dimensional array, implemented using the FPGA's internal BRAM. The equivalence tags are stored in the equivalence table as E(label_index) = label_array, where label_index represents the array index and label_array represents the array value, each representing an equivalence tag value. To facilitate tag merging, label_index ≥ label_array. This paper defines the equivalence table depth as the image column width, meaning the maximum tag value is equal to the image column width. This depth is sufficient for the pre-processed binary image and facilitates row-period updates to the equivalence table.

[0098] stage4.2: Detects markers appearing in two adjacent rows. (Using the same method...) Figure 5 The sliding window shown temporarily stores the flag values ​​of b and e when it detects {a,b} = {0,1} or {d,e} = {0,1}, until the extension of the flag values ​​within the same row disappears. Analysis revealed that it is not necessary to store all flags appearing in both adjacent rows when determining which flags have appeared in each pair. Figure 7 As shown:

[0099] The markers in row N-1 that are connected to row N, such as 'b', do not need to be stored because the region represented by marker 'b' still extends into row N. Similarly, the markers in row N that are not connected to row N-1, such as 'c', also do not need to be stored because marker 'c' is a newly created marker and is not locally connected to row N-1. Therefore, only the markers in row N-1 that are not connected to row N, such as 'a', and the markers in row N that are connected to row N-1, such as 'd', need to be stored. This marker storage method reduces the computational load of comparing markers in subsequent adjacent rows.

[0100] 2) Equivalence table copying and updating module

[0101] Since both equivalence table copying and updating are performed on a row-by-row basis, in order to construct a pipelined processing structure for equivalence table copying and updating operations, such as... Figure 8 As shown, this paper uses "ping-pong operation" to alternately copy and update the equivalence table using two RAMs.

[0102] flag=1: The row count of the current flag processing row is odd. RAM1 is used to store the copied equivalence table data, and RAM2 is used to update the equivalence table.

[0103] flag=0: The row count is even. RAM1 is used to update the equivalence table, and RAM2 is used to store the copied equivalence table data.

[0104] By performing equivalence table copying and updating in this alternating and seamless switching manner, a pipelined processing of equivalence table updates can be achieved, with the overall processing latency of the module being one row cycle.

[0105] The process of copying the equivalence table is relatively simple. Figure 9 This demonstrates the main process of updating the state machine using the equivalence table:

[0106] a. When the state machine is in the IDLE state, it detects the start signal of a row and jumps to the S1 state;

[0107] b. In state S1, first determine whether the current read address is equal to the maximum label of the equivalence table;

[0108] c. If they are equal, it means that the equivalence table has been updated and jump to the IDLE state. If they are not equal, according to the value of flag, read the flag values ​​stored in RAM1 or RAM2 in ascending order of address, and then jump to S2.

[0109] d. In state S2, the read tag value label is compared with the address value addr. If they are equal, this tag is the root tag, and the process jumps to S1. If they are not equal, the process jumps to S3.

[0110] e. In state S3, read the data E(label) in RAM with label as the address, make addr = label and label = E(label), and check again whether label and address value addr are equal. If they are equal, write the current label value into the original address, and then jump to S1. If they are not equal, jump to S4.

[0111] f. In state S4, the judgment in S3 is repeated. If they are equal, jump to S1; otherwise, jump to S5.

[0112] g. In state S5, the latest label value read from the iteration is written to the initial address, and then the process jumps to S1;

[0113] After the above process, the equivalence table is updated, meaning that all labels within the same connected domain are replaced with their root labels.

[0114] 3) Adjacent row marker storage and comparison module

[0115] like Figure 10 This demonstrates the process of storing, updating, and comparing adjacent row markers, with each stage performed on a row-by-row basis:

[0116] Stage 1: Adjacent row label storage. During the labeling of the Nth row of the image, a sliding window is used to detect the labels in rows N-1 and N. m and l n Store the data to form a tag set {L} N-1} and {L N}

[0117] Stage 2: Adjacent row marker update. Based on the updated equivalence table E1, update {L... N-1} and {L N} is updated to get {L N-1}' and {L N To implement a pipelined processing structure that stores and updates adjacent row tags, a pipelined architecture based on row-based processing cycles is constructed using "ping-pong operations," such as... Figure 11 As shown:

[0118] flag=1: The row count of the current row being marked is odd. FIFO1 and FIFO2 are used to store adjacent row tags, and FIFO3 and FIFO4 are used to update adjacent row tags.

[0119] flag=0: The row count of the currently marked row is even. FIFO3 and FIFO4 are used to store adjacent row tags, and FIFO1 and FIFO2 are used to update adjacent row tags.

[0120] Stage 3: Adjacent row tag comparison. After obtaining the root tags that appeared in row N-1 and row N, it is necessary to determine which root tags appeared in row N-1 but not in row N by tag comparison. The simplest method is as follows: Figure 11 Traversal matching is performed. However, traversal matching has the problem of uncertain computation time. If there are many tags between two adjacent rows, the traversal matching will take more than one row cycle, and then pipelined processing will not be possible. To address this issue, this paper uses register logic operations to replace the traversal matching of adjacent row tags.

[0121] like Figure 13 As shown, two registers, label_1 and label_2, with a depth equal to the column width of the image, are used to represent the tags that appear in the (N-1)th row and the Nth row. For example, if tag 3 appears in the (N-1)th row, then label_1[3] = 1, and so on. Then, the corresponding bit of the same index in registers label_1 and label_2 is judged. If it is "10", it means that the tag corresponding to this index in the (N-1)th row can be output. Otherwise, it is not output. In this way, the time of the adjacent row tag matching operation can be limited to a fixed row cycle.

[0122] 4) Coordinate Recording and Update Module

[0123] To calculate the centroid coordinates and radius of each region, it is necessary to accumulate the coordinates of the same connected region during the labeling process. For example... Figure 14 As shown, a coordinate table with the same structure as the equivalence table is constructed, also implemented using BRAM:

[0124] The operations on the coordinate table include recording, updating, copying, and outputting. Each operation requires one RAM, therefore this module requires a total of four RAMs, designated RAM1, RAM2, RAM3, and RAM4. The recording operation of the coordinate table is the same as that of the equivalent table, except that the stored values ​​are changed from markers to accumulated coordinate values. The copying, updating, and outputting of the coordinate table are also performed in a row-by-row cycle and have a sequential relationship. To construct a pipelined structure, a "ping-pong operation" is used to alternately copy, update, and output the coordinate table between RAM2, RAM3, and RAM4. The specific operation is similar to the equivalent table operations within the modules mentioned above. After obtaining the output marker from the adjacent marker storage and comparison module, the coordinate sum stored in RAM is read using this marker as the address. Then, division is used to calculate the centroid coordinates. To construct a pipelined operation, a lookup table is built in ROM for the division calculation, which completes the centroid coordinate calculation in one clock cycle.

[0125] This technical solution, based on the characteristics of connected components in binary images, performs operations such as copying and updating the equivalence table according to the image line cycle, forming an efficient computational architecture for connected component labeling based on FPGA. By analyzing the shape characteristics of connected components in images and combining the update operation of the equivalence table in connected component labeling computation, and based on the FPGA device architecture, a method of copying and updating the equivalence table according to the image line cycle is adopted to design an FPGA-based connected component labeling computation algorithm. Compared with common computational architectures, the computation latency can be reduced from 1 frame cycle to 4 line cycles, effectively reducing processing latency.

[0126] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0127] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

[0128] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A hardware implementation method for image connected component labeling, characterized in that: The image connected component labeling hardware implementation method adopts a pixel-based connected component labeling hardware architecture, which includes an initial labeling and equivalence table storage module, an equivalence table copying and updating module, a coordinate recording and updating module, and an adjacent row label storage and comparison module. Initial tagging and equivalence table storage module: a. Use a filtering template to perform initial labeling sliding window processing on the input binary image, assign a label value to each pixel with a pixel value of 1, and store the detected equivalent labels in an equivalence table. The structure of the equivalence table is implemented using a one-dimensional array. b. Output the coordinates of pixels with a grayscale value of 1; c. Detect and output the markers that appear in two adjacent rows during the sliding window process; Equivalence table copying and updating module: Copy the latest equivalence table E1 in a row cycle, and update E1 in the next row cycle of the image; Adjacent row marker storage and comparison module: a. Store the two adjacent rows of tags that appear during the tagging process to obtain a tag set; b. Obtain the updated E1 and update the adjacent row tag set; c. By comparing the updated tag set, obtain the tag set {Lo} of the output region of the previous row in the two adjacent rows; Coordinate recording and update module: a. Store the coordinates of pixels with a value of 1 during the marking process; b. Update the coordinate table based on the updated E1; c. Based on the output marker {Lo}, extract the coordinates and pixel count of the corresponding connected component, calculate and output the centroid coordinates and radius of the connected component.

2. The hardware implementation method for image connected component labeling according to claim 1, characterized in that: The computation process of the image connected component labeling hardware implementation method is executed in the following row-cycle order: The first row cycle: This requires four operations. The first operation is to label the (N-1)th row of image data, recording the equivalence information in the equivalence table in real time. The second operation is to detect the labels {L} that appeared in the (N-2)th and (N-1)th rows. N-2 } and {L N-1 The third operation is to copy the equivalence table and denote it as E1 while marking the N-1th row; the fourth operation is to copy the coordinate table and denote it as C1 while marking the N-1th row. The second row cycle: Marking the Nth row of data, which requires three operations. The first operation is to traverse and update the equivalence table E1; the second operation is to use the updated E1 to store the mark sets {L} of the (N-2)th and (N-1)th rows. N-2 } and {L N-1 } is updated to get {L N-2 }' and {L N-1 The third operation is to use the updated E1 to update the coordinate table C1 to obtain C1'; The third row cycle: Mark the (N+1)th row of data, and simultaneously update the mark set {L}. N-2 }' and {L N-1 }' performs a label comparison to obtain {L N-2 The set of connected component labels that can be output from}' {L O }; Fourth row cycle: Mark the data in the (N+2)th row, and simultaneously calculate and output the mark set {L}. O The centroid coordinates and radius of the connected components within the domain; Based on the above operations, when the image is marked with pixels in the (N+2)th row, the output region extension stops at the connected component in the (N-2)th row. Counting from the end of the connected component extension, the output delay is at most 4 row cycles.

3. A hardware implementation method for image connected component labeling according to claim 1 or 2, characterized in that: The pixel marking and equivalence table storage process in the initial marking and equivalence table storage module includes: stage 1: Marking context modeling, using a preset window to perform marking context modeling; Stage 2: Pixel-by-pixel marking. Based on the preset window size used for marking, a corresponding number of FIFOs are used to buffer row data, and a corresponding number of registers are used to buffer data in the filtering window. Stage 3: Equivalence relation context modeling. To simplify equivalence relation judgment, a 2×2 window is used. Stage 4.1: Equivalence relation judgment and storage. For the clock cycles of pixel judgment and assignment that need to be marked in the modeling process, that is, after the data stream flows through the preset window of stage 1, it is delayed by the same number of clock cycles of the same column of the preset window before it can directly flow into the window shown in stage 3 for sliding window. Therefore, no FIFO is needed for data buffering, and only 4 registers are needed to form the sliding window. When performing equivalence relation judgment, data in the 2×2 window is taken out in pairs. If the pixel values ​​are the same and are both 1, but the mark values ​​are different, it means that there are marks with equivalence relation. This pair of marks is written into the equivalence table. Stage 4.2: Detecting markers appearing in adjacent rows. Using the sliding window shown in Stage 3, when {a,b} = {0,1} or {d,e} = {0,1} is detected, the marker values ​​of b and e are temporarily stored until the extension of the marker values ​​within the same row disappears. Analysis reveals that it is not necessary to store all markers appearing in both rows when determining which markers have appeared in adjacent rows: markers in row N-1 that are connected to row N, such as b, do not need to be stored because the region represented by marker b still extends into row N; markers in row N that are not connected to row N-1, such as c, also do not need to be stored because marker c is a newly created marker and is not locally connected to row N-1. Therefore, it is only necessary to store markers in row N-1 that are not connected to row N, such as a, and markers in row N that are connected to row N-1, such as d. This marker storage method reduces the computational load for subsequent comparisons of markers in adjacent rows.

4. The hardware implementation method for image connected component labeling according to claim 1, characterized in that: The equivalence table copying and updating module: Since both equivalence table copying and updating are performed in a row-based cycle, in order to construct a pipelined processing structure for equivalence table copying and updating operations, a ping-pong operation is used, which utilizes two RAMs to alternately copy and update the equivalence table. flag=1: The row count of the current marked row is odd. RAM1 is used to store the copied equivalence table data, and RAM2 is used to update the equivalence table. flag=0: The row count is even, RAM1 is used to update the equivalence table, and RAM2 is used to store the copied equivalence table data; The equivalence table is copied and updated in an alternating and seamless switching manner as described above, realizing pipelined processing of equivalence table updates. The overall processing delay of the module is 1 row cycle.

5. The hardware implementation method for image connected component labeling according to claim 4, characterized in that: The process of updating the state machine using the equivalence table includes: a. When the state machine is in the IDLE state, it detects the start signal of a row and jumps to the S1 state; b. In state S1, first determine whether the current read address is equal to the maximum label of the equivalence table; c. If they are equal, it means that the equivalence table has been updated and jump to the IDLE state. If they are not equal, according to the value of flag, read the flag values ​​stored in RAM1 or RAM2 in ascending order of address, and then jump to S2. d. In state S2, the read tag value label is compared with the address value addr. If they are equal, this tag is the root tag, and the process jumps to S1. If they are not equal, the process jumps to S3. e. In state S3, read the data E(label) in RAM with label as the address, make addr = label and label = E(label), and check again whether label and address value addr are equal. If they are equal, write the current label value into the original address, and then jump to S1. If they are not equal, jump to S4. f. In state S4, the judgment in S3 is repeated. If they are equal, jump to S1; otherwise, jump to S5. g. In state S5, the latest label value read from the iteration is written into the initial address, and then jumps to S1. After the above process, the equivalence table is updated, that is, all labels in the same connected domain are replaced with its root label.

6. The hardware implementation method for image connected component labeling according to claim 1, characterized in that: The storage, update, and comparison process of the adjacent row marker storage and comparison module is as follows: Stage 1: Adjacent row labels are stored. When labeling the Nth row of image data, the labels of rows N-1 and N are detected by a sliding window. m and l n Store the data to form a tag set {L} N-1 } and {L N }; Stage 2: Adjacent row markers are updated; {L is adjusted based on the updated equivalence table E1. N-1 } and {L N } is updated to get {L N-1 }' and {L N To achieve a pipelined processing structure that stores and updates adjacent row tags, ping-pong operations are used to build a pipelined architecture with row processing cycles. Stage 3: Adjacent row tag comparison. After obtaining the root tags that have appeared in the (N-1)th row and the Nth row, it is necessary to determine which root tags appear in the (N-1)th row but not in the Nth row by comparing the tags. Register logic operations are used to replace the adjacent row tag traversal and matching.

7. The hardware implementation method for image connected component labeling according to claim 6, characterized in that: In the process stage 2 of the adjacent row tag storage and comparison module, the pipeline processing structure for storing and updating adjacent row tags is as follows: flag=1: The row count of the row currently being marked is odd. FIFO1 and FIFO2 are used to store adjacent row tags, and FIFO3 and FIFO4 are used to update adjacent row tags. flag=0: The row count of the currently marked row is even. FIFO3 and FIFO4 are used to store adjacent row tags, and FIFO1 and FIFO2 are used to update adjacent row tags.

8. The hardware implementation method for image connected component labeling according to claim 6, characterized in that: In stage 3 of the adjacent row marker storage and comparison module, the register logic operation is as follows: Two registers, label_1 and label_2, with a depth equal to the width of the image column, are used to represent the marks that have appeared in the (N-1)th and (N)th rows. For example, if mark 3 appears in the (N-1)th row, then label_1[3] = 1, and so on. Then, the corresponding bit of the same index in registers label_1 and label_2 is judged. If it is 10, it means that the mark corresponding to this index in the (N-1)th row can be output. Otherwise, it is not output. In this way, the time of the adjacent row mark matching operation can be limited to a fixed row cycle.

9. The hardware implementation method for image connected component labeling according to claim 1, characterized in that: The coordinate recording and updating module constructs a coordinate table with the same structure as the equivalent table. The operations on the coordinate table include recording, updating, copying, and outputting. Each operation requires one RAM, so this module requires a total of 4 RAMs, which are denoted as RAM1, RAM2, RAM3, and RAM4. The recording operation of the coordinate table is the same as that of the equivalent table, except that the saved value is changed from a marker to an accumulated coordinate value. The copying, updating, and output of the coordinate table are also performed in a row-by-row cycle and have a sequential relationship. In order to build a pipeline structure, ping-pong operation is also used to make RAM2, RAM3, and RAM4 alternately copy, update, and output the coordinate table. The specific operation is similar to the equivalent table operation inside the initial mark and equivalence table storage module, the equivalence table copying and updating module, and the adjacent row mark storage and comparison module. After obtaining the output mark from the adjacent mark storage and comparison module, the coordinate sum stored in RAM is read out using this mark as the address, and then the centroid coordinates are calculated by division.

10. A hardware implementation method for image connected component labeling according to claim 9, characterized in that: To build a pipelined operation, a lookup table is built using ROM to perform division calculations, and the centroid coordinates can be calculated in one clock cycle.

Images