A connected domain acquisition method based on a ZYNQ chip

Abstract

The application discloses a connected domain acquisition method based on a ZYNQ chip, pixel point information in an image is stored into an FPGA chip internal memory in the form of a data stream row by row, taking a starting pixel point of the image as a center pixel point, a 3*3 selection frame is used to select the pixel point information, and the pixel point information is sent to a register group I in sequence for storage and judgment of whether to be marked, and the coordinates and a mark number of a marked point are recorded; the selection is moved pixel by pixel, all pixel points in the image are marked respectively, and a union-find set mode is used to store the mark number and the pixel points into a DDR chip; an ARM chip traverses all the pixel points and the mark number stored in the DDR chip, classifies the pixel points and the mark number respectively according to the union-find set, and obtains all connected domains. The method uses the FPGA and the ARM in the ZYNQ for heterogeneous computing, effectively improves the acquisition speed of the connected domains, reduces the connected domain analysis time to the millisecond level, and can meet the high-tact detection demand.

Description

Technical Field

[0001] This invention relates to the field of connected component analysis, and more specifically to a method for obtaining connected components based on a ZYNQ chip. Background Technology

[0002] With the increasing demand for visual inspection in industrial manufacturing, higher requirements are being placed on inspection speed. Therefore, visual inspection equipment needs to accelerate its inspection cycle to keep pace with the manufacturing speed on the production line. Visual inspection involves image processing. To identify the location of the target object (the object under test or a defect), connected component analysis is typically performed. This involves marking the area where the target object is located and further analyzing the area of ​​the connected components, using this as an indicator to evaluate the target object.

[0003] Existing methods for connected component analysis involve extracting the contour coordinates and calculating the area of ​​connected components using a host computer or ARM architecture. These methods have the following problems:

[0004] 1. It is necessary to wait for all single-frame images to be acquired and stored in an operable storage space before related calculations can be performed; the computational load is large and the time consumption is long.

[0005] 2. The process of calculating the area of ​​a connected region needs to be performed separately after the contour coordinates of the connected region are extracted, which increases the processing time.

[0006] However, this method suffers from slow computation speed and long processing time, requiring image processing time to be on the order of seconds, which makes it difficult to meet the requirements of high-frequency detection. Summary of the Invention

[0007] To address the aforementioned technical issues, this invention provides a connected component acquisition method based on a ZYNQ chip. This method employs heterogeneous computing using the FPGA and ARM within the ZYNQ chip, effectively improving the speed of connected component acquisition and reducing the connected component analysis time to the millisecond level. Compared to existing processing methods based on host computers / ARM, the computation time is reduced by 100 times, meeting the requirements of high-frequency detection.

[0008] The technical solution of the present invention is as follows:

[0009] A method for obtaining connected components based on a ZYNQ chip, wherein the ZYNQ chip includes an FPGA chip, an ARM chip, and a DDR chip, and both the FPGA chip and the ARM chip can access the DDR chip, characterized in that:

[0010] The FPGA chip has an internal memory and a register group I. The internal memory is either a FIFO or a BRAM, and the register group I consists of 9 registers.

[0011] Step 1: The information of each pixel in the image is stored line by line in the internal memory in the form of a data stream. When the third row of pixel information is stored, the stored pixel information is selected using a 3×3 selection area with the starting pixel of the image as the center pixel and sent to register group I for storage and step A is executed. The 3×3 selection area is moved pixel by pixel. Each time it is moved, the pixel information in the selection area is sent to register group I for storage and step A is executed, until all pixels in the image have been used as the center pixel and step A has been executed.

[0012] Step A is as follows: using the judgment principle of the Two-Pass method, the center pixel in register group I is marked. If there is a marked pixel, its horizontal and vertical coordinates are recorded and a corresponding mark number is assigned to it.

[0013] Based on the connectivity between pixels with different marker numbers, a disjoint-set data structure is used to record the sets to which different marker numbers belong and store them in the DDR chip.

[0014] Step 2: The ARM chip traverses each pixel and tag number stored in the DDR chip, and classifies each pixel and tag number according to the disjoint set corresponding to different tag numbers, and stores each pixel and tag number belonging to the same category uniformly.

[0015] To facilitate obtaining the area of ​​the connected domain, preferably, the FPGA chip also has a register group II, which consists of another 9 registers.

[0016] A method for obtaining connected components based on a ZYNQ chip, characterized in that: the ZYNQ chip includes an FPGA chip, an ARM chip, and a DDR chip, wherein both the FPGA chip and the ARM chip can access the DDR chip.

[0017] The FPGA chip has an internal memory, register group I and register group II. The internal memory is either FIFO or BRAM. Register group I consists of 9 registers and register group II consists of another 9 registers.

[0018] Step 1: The information of each pixel in the image is stored line by line in the internal memory in the form of a data stream. When the third row of pixel information is stored, the stored pixel information is selected using a 3×3 selection area with the starting pixel of the image as the center pixel and sent to register group I for storage and step A is executed. The 3×3 selection area is moved pixel by pixel. Each time it is moved, the pixel information in the selection area is sent to register group I for storage and step A is executed, until all pixels in the image have been used as the center pixel and step A has been executed.

[0019] Step A is as follows: using the judgment principle of the Two-Pass method, the center pixel in register group I is marked. If there is a marked pixel, its horizontal and vertical coordinates are recorded and a corresponding mark number is assigned to it.

[0020] After processing in step A, the center pixel is stored line by line in the internal memory in the form of a data stream. When the third row of pixel information begins to be stored, the stored pixel information is selected using a 3×3 selection area with the starting pixel as the center pixel and sent to register group II for storage and step B is executed. The 3×3 selection area is moved pixel by pixel. Each time it is moved, the pixel information in the selection area is sent to register group II for storage and step B is executed, until all pixels have been used as the center pixel and step B has been executed.

[0021] Step B is as follows: If the center pixel has been marked in step A, the current mark number is recorded directly; otherwise, when the center pixel meets the grayscale / color threshold condition, it is marked, and the smallest mark number among the adjacent pixels to the left, upper left, upper, and upper right of the position is assigned to the mark number of the pixel and recorded.

[0022] Based on the connectivity between pixels with different marker numbers, a disjoint-set data structure is used to record the sets to which different marker numbers belong and store them in the DDR chip.

[0023] Step 2: The ARM chip iterates through all the pixels and markers stored in the DDR chip, and categorizes each pixel and marker according to the disjoint-set data structure corresponding to different markers. Pixels and markers belonging to the same category are stored together. Further, in Step 2, the number of markers belonging to the same category is counted and used as the area value of the connected component.

[0024] Furthermore, the selected area moves pixel by pixel in a left-to-right, top-to-bottom order.

[0025] Furthermore, in step A, the judgment principle of the Two-Pass method is as follows:

[0026] When the center pixel meets the grayscale / color threshold condition and there is an adjacent pixel above, to the left, to the right and below that does not meet the grayscale / color threshold condition, it is marked with an initial mark number of 1. The mark number of unmarked pixels is empty.

[0027] If none of the pixels to the left, upper left, above, and upper right of the marked pixel are marked, the current mark number is incremented by 1; otherwise, the smallest mark number among these four adjacent pixels is found and the value of the smallest mark number is assigned to the current mark number.

[0028] When there are different label numbers for the left, upper left, upper, and upper right adjacent pixels of a labeled pixel, record the disjoint set corresponding to the different label numbers.

[0029] Furthermore, in step one, the pixel information includes grayscale / color and the horizontal and vertical coordinates of the pixel.

[0030] Preferably, in step one, the stored pixel information is selected using a 3×3 selection area and sent to register group II / register group II in order from left to right and from top to bottom for storage.

[0031] In step two, to facilitate observation, the label numbers of all pixels belonging to the same category are unified to the same value, and they are marked in the image according to their horizontal and vertical coordinate information.

[0032] Furthermore, when the pixel at the edge of the image is taken as the center pixel, if the pixel information in the selected area is empty, zero padding is performed.

[0033] The method of the present invention has the following characteristics:

[0034] ① This method does not require waiting for all frames of image to be acquired and stored, resulting in higher real-time performance and faster speed. Specifically: During processing, image information is input into a 3×3 register group in the FPGA. Once the register group is filled, connected component labeling begins. As data flows in in real time, real-time image processing is completed, obtaining the label point information in different connected components. Subsequently, the ARM performs disjoint-set data structure processing, classifying each label point and distinguishing the label points in each connected component to complete the connected component analysis.

[0035] ② The method for marking pixels in register group I adopts the marking method used during the first traversal of the classic Two-Pass method. Based on this, this method improves upon the classic Two-Pass method by re-injecting the marked pixels into register group II. This process can mark pixels in the internal region of the connected component, so that the ARM does not need to calculate the contour area through the contour coordinates of the connected component during subsequent calculations, thereby reducing the computational load of the ARM and improving the speed of obtaining the area of ​​the connected component.

[0036] ③ The ZYNQ platform provides the feature that ARM and FPGA can access the same DDR through the bus. Based on this feature, this method stores the marking results of FPGA to DDR, and ARM calls the marking results in DDR to perform serial calculations and count the connected components to which each marking point belongs (the second traversal process of the classic Two-Pass method). This reduces the time consumption of data interaction between ARM and FPGA. ARM only needs to process a small number of marking points to obtain the desired results, which greatly reduces the amount of computation of ARM, improves detection efficiency, and enables it to meet the requirements of real-time and high-frequency detection. Attached Figure Description

[0037] Figure 1 A flowchart illustrating the method for obtaining connected components;

[0038] Figure 2 A schematic diagram of the process of marking step A;

[0039] Figure 3 A schematic diagram showing steps A and B after they have been marked.

[0040] Figure 4 This is a schematic diagram showing two connected components in an image.

[0041] Figure 5 This is a diagram illustrating the pixel-by-pixel movement of the selected area;

[0042] Figure 6 This is a schematic diagram of an FPGA parallel processing structure;

[0043] Figure 7 This is a schematic diagram of the disjoint-setup storage structure. Detailed Implementation

[0044] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0045] Example 1

[0046] A method for obtaining connected components based on a ZYNQ chip, wherein the ZYNQ chip includes an FPGA chip, an ARM chip, and a DDR chip, and both the FPGA chip and the ARM chip can access the DDR chip.

[0047] The FPGA chip has an internal memory and register group I. The internal memory is either FIFO or BRAM, and register group I consists of 9 registers.

[0048] Step 1: The information of each pixel in the image is stored line by line in the internal memory in the form of a data stream. When the information of the third row of pixels begins to be stored, the stored pixel information is selected using a 3×3 selection area with the starting pixel of the image as the center pixel and sent to register group I for storage in sequence. Step A is then executed. The 3×3 selection area is moved pixel by pixel. Each time it is moved, the pixel information within the selection area is sent to register group I for storage and Step A is executed, until all pixels in the image have been used as the center pixel and Step A has been executed.

[0049] Step A is as follows: Using the judgment principle of the Two-Pass method, mark the center pixel in register group I. If there is a marked pixel, record its horizontal and vertical coordinates and assign it the corresponding mark number.

[0050] Based on the connectivity between pixels with different marker numbers, a disjoint-set data structure is used to record the sets to which different marker numbers belong and store them in the DDR chip;

[0051] Step 2: The ARM chip iterates through each pixel and tag number stored in the DDR chip, and classifies each pixel and tag number according to the disjoint set corresponding to different tag numbers. Pixels and tag numbers belonging to the same category are stored together.

[0052] The preferred implementation is as follows: the selected area is moved pixel by pixel in the order from left to right and from top to bottom.

[0053] In step one, the pixel information includes grayscale / color and the horizontal and vertical coordinates of the pixel.

[0054] Specifically, in step A, the judgment principle of the Two-Pass method is as follows:

[0055] When the center pixel meets the grayscale / color threshold condition and there is an adjacent pixel above, to the left, to the right and below that does not meet the grayscale / color threshold condition, it is marked with an initial mark number of 1. The mark number of unmarked pixels is empty.

[0056] If none of the pixels to the left, upper left, above, and upper right of the marked pixel are marked, the current mark number is incremented by 1; otherwise, the smallest mark number among these four adjacent pixels is found and the value of the smallest mark number is assigned to the current mark number.

[0057] When there are different label numbers for the left, upper left, upper, and upper right adjacent pixels of a labeled pixel, record the disjoint set corresponding to the different label numbers.

[0058] The preferred implementation is as follows: In step one, the stored pixel information is selected using a 3×3 selection area and sent to register group II / register group II for storage in order from left to right and from top to bottom.

[0059] To facilitate observation, the label numbers of all pixels belonging to the same category are unified to the same value, and they are marked in the image according to their horizontal and vertical coordinate information.

[0060] like Figure 5 As shown, when the pixel at the edge of the image is taken as the center pixel, if the pixel information in the selected area is empty, zero padding is performed.

[0061] Example 2

[0062] A method for obtaining connected components based on a ZYNQ chip, wherein the ZYNQ chip includes an FPGA chip, an ARM chip, and a DDR chip, and both the FPGA chip and the ARM chip can access the DDR chip.

[0063] like Figure 6 As shown, the FPGA chip has internal memory, register group I and register group II. The internal memory is either FIFO or BRAM. Register group I consists of 9 registers and register group II consists of another 9 registers.

[0064] Step 1: The information of each pixel in the image is stored line by line into the internal memory in the form of a data stream. When the third line of pixel information begins to be stored, such as... Figure 5 As shown, the starting pixel of the image is used as the center pixel. A 3×3 selection area is used to select the stored pixel information and send it to register group I for storage in sequence. Step A is then executed. Specifically, the pixel information includes grayscale / color and the horizontal and vertical coordinates of the pixel.

[0065] In practice, when the pixel at the edge of the image is taken as the center pixel, if the pixel information in the selected area is empty, the pixel information will be padded with zeros.

[0066] The 3×3 selection area is moved pixel by pixel. Each time it moves, the pixel information within the selection box is sent to register group I for storage and step A is executed, until all pixels in the image have been treated as the center pixel and step A has been executed. Since the pixel information in the image is input into the internal memory sequentially from the top left corner to the bottom right corner, therefore, as follows... Figure 5 As shown, the selection area moves pixel by pixel in a left-to-right, top-to-bottom order. In this invention, up, down, left, and right are all conventional drawing directions. Furthermore, the information of the 9 pixels in the selection area is sent to register group II / register group II for storage in a left-to-right, top-to-bottom order.

[0067] Step A is as follows: Using the judgment principle of the Two-Pass method, mark the center pixel in register group I. If there is a marked pixel, record its horizontal and vertical coordinates and assign it the corresponding mark number.

[0068] In step A, not only the marker number is recorded, but also the coordinate information of the pixel is recorded;

[0069] Specifically, the judgment principle of the Two-Pass method (the judgment principle during the first traversal of the image) is as follows:

[0070] like Figure 2 As shown, when the center pixel meets the grayscale / color threshold condition (is a feature point) and there is an adjacent pixel (non-feature point) above, to the left, to the right, and below that does not meet the grayscale / color threshold condition, it is marked, and its initial mark number is 1. The mark number of unmarked pixels is empty. For example, if the input pixel information includes grayscale and the horizontal and vertical coordinates of the pixel, and the grayscale threshold condition is 100, when the grayscale of the center pixel is greater than 100, it is a feature point.

[0071] If none of the pixels to the left, upper left, above, and upper right of the marked pixel are marked, the current mark number is incremented by 1; otherwise, the smallest mark number among these four adjacent pixels is found and the value of the smallest mark number is assigned to the current mark number.

[0072] like Figure 3 As shown, when there are different label numbers (label number 1 and label number 2) between the left, upper left, upper, and upper right adjacent pixels of the labeled pixel, the disjoint set corresponding to the different label numbers is recorded.

[0073] Alternatively, to more conveniently obtain the area of ​​the connected domain, compared with the previous embodiment, in this embodiment, the FPGA chip also has a register group II, which consists of another 9 registers;

[0074] While marking in step A, the FPGA obtains the label numbers of pixels within the connected component in parallel using the following method:

[0075] like Figure 6 As shown, the center pixels processed in step A are stored line by line in the internal memory in the form of a data stream. Since the center pixels are marked pixel by pixel (i.e., the first selection is centered on the first pixel, the second pixel is centered after moving the selection pixel by pixel, the third pixel is centered after moving the selection pixel by pixel, and so on until the last pixel is centered after moving the selection pixel by pixel), the data stream at this time is still complete image data, except that some pixels are marked and some pixels are not marked.

[0076] When the third row of pixel information begins to be stored, the stored pixel information is selected using a 3×3 selection area with the starting pixel as the center pixel and sent to register group II for storage and step A is executed. The 3×3 selection area is moved pixel by pixel. Each time it is moved, the pixel information in the selection area is sent to register group II for storage and step B is executed, until all pixels have been used as the center pixel and step B has been executed.

[0077] Step B is as follows: If the center pixel has already been marked in step A, then directly record the current mark number; otherwise, when the center pixel meets the grayscale / color threshold condition, mark it, and assign the smallest mark number among the adjacent pixels to its left, upper left, upper, and upper right to the mark number of this pixel, and record it. In this step B, only the mark number can be stored, without storing the pixel coordinates, and the number of mark numbers in the connected component can be used as the number of pixels in the connected component area. However, in step A, both the mark number and the pixel coordinates were stored.

[0078] Based on the connectivity between pixels with different marker numbers (e.g.) Figure 4 ), using disjoint-set data structure (e.g. Figure 7 The disjoint-set data structure (a common data structure in China) records the sets to which different tags (including tags from both step A and step B) belong and stores them in the DDR chip.

[0079] The process of storing the above data into the DDR chip can be performed either after all pixels have been processed using steps A and B (after the entire image has been marked), or it can be performed synchronously during the execution of steps A and B (which can further speed up the processing). For example, after marking M pixels (M can be 4, 6, 10, or other preset numbers), the marking number and pixel coordinates are stored in the DDR chip using a disjoint-set data structure.

[0080] Step 2, as follows Figure 1 As shown, the ARM chip iterates through each pixel and its tag number stored in the DDR chip, and then uses a disjoint-set data structure (such as...) corresponding to each tag number. Figure 7 Each pixel and its label are categorized separately, and pixels and labels belonging to the same category (same connected component) are stored together.

[0081] Furthermore, the number of labels belonging to the same category can be counted and used as the area value of the connected region.

[0082] The results of the connected component analysis are summarized in the table below:

[0083]

[0084] For ease of observation, in practice, the label numbers of all pixels belonging to the same category can be unified to the same value, and they can be marked in the image according to their horizontal and vertical coordinate information.

[0085] The foregoing description of specific exemplary embodiments of the present invention is for illustrative and descriptive purposes. It is not intended to be exhaustive, nor to limit the invention to the precise forms disclosed; obviously, many changes and variations are possible in accordance with the foregoing teachings. The exemplary embodiments were chosen and described to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to implement and utilize various exemplary embodiments of the invention, as well as their different alternatives and modifications. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A method for acquiring connected components based on a ZYNQ chip, wherein the ZYNQ chip includes an FPGA chip, an ARM chip, and a DDR chip, wherein the FPGA chip and the ARM chip can both access the DDR chip, characterized in that: The FPGA chip internally includes an internal memory, register group I, and register group II. The internal memory is either a FIFO or a BRAM. Register group I consists of 9 registers, and register group II consists of another 9 registers. Step 1: The information of each pixel in the image is stored line by line in the internal memory in the form of a data stream. When the third row of pixel information begins to be stored, the starting pixel of the image is used as the center pixel. A 3×3 selection area is used to select the stored pixel information and sends it sequentially to register group I for storage and step A is executed. The 3×3 selection area is moved pixel by pixel. Each time it moves, the pixel information within the selection area is sent to register group I for storage and step A is executed, until all pixels in the image have been used as the center pixel and step A has been executed. Step A is as follows: using the judgment principle of the Two-Pass method, the center pixel in register group I is marked. If there is a marked pixel, its horizontal and vertical coordinates are recorded and a corresponding mark number is assigned to it. After processing in step A, the center pixel is stored line by line in the internal memory in the form of a data stream. When the third row of pixel information begins to be stored, the stored pixel information is selected using a 3×3 selection area with the starting pixel as the center pixel and sent to register group II for storage and step B is executed. The 3×3 selection area is moved pixel by pixel. Each time it is moved, the pixel information in the selection area is sent to register group II for storage and step B is executed, until all pixels have been used as the center pixel and step B has been executed. Step B is as follows: If the center pixel has been marked in step A, the current mark number is recorded directly; otherwise, when the center pixel meets the grayscale / color threshold condition, it is marked, and the smallest mark number among the adjacent pixels to the left, upper left, upper, and upper right of the position is assigned to the mark number of the pixel and recorded. Based on the connectivity between pixels with different marker numbers, a disjoint-set data structure is used to record the sets to which different marker numbers belong and store them in the DDR chip. Step 2: The ARM chip traverses each pixel and tag number stored in the DDR chip, and classifies each pixel and tag number according to the disjoint set corresponding to different tag numbers, and stores each pixel and tag number belonging to the same category uniformly.

2. The method for obtaining connected components based on a ZYNQ chip as described in claim 1, characterized in that: In step two, the number of labels belonging to the same category is counted and used as the area value of the connected region.

3. The method for obtaining connected components based on the ZYNQ chip as described in claim 1, characterized in that: The selection area is moved pixel by pixel in a left-to-right, top-to-bottom order.

4. The method for obtaining connected components based on a ZYNQ chip as described in claim 1, characterized in that: In step A, the judgment principle of the Two-Pass method is as follows: When the center pixel meets the grayscale / color threshold condition and there is an adjacent pixel above, to the left, to the right and below that does not meet the grayscale / color threshold condition, it is marked with an initial mark number of 1. The mark number of unmarked pixels is empty. If the pixels to the left, upper left, above, and upper right of a marked pixel are not marked, the current mark number is incremented by 1. Otherwise, find the smallest marker number among these four adjacent pixels and assign the value of the smallest marker number to the current marker number; When there are different label numbers for the left, upper left, upper, and upper right adjacent pixels of a labeled pixel, record the disjoint set corresponding to the different label numbers.

5. The method for obtaining connected components based on a ZYNQ chip as described in claim 1, characterized in that: In step one, the pixel information includes grayscale / color and the horizontal and vertical coordinates of the pixel.

6. The method for obtaining connected components based on a ZYNQ chip as described in claim 1, characterized in that: In step one, the stored pixel information is selected using a 3×3 selection area and sent to register group II / register group II in order from left to right and from top to bottom for storage.

7. The method for obtaining connected components based on a ZYNQ chip as described in claim 1, characterized in that: In step two, the label numbers of all pixels belonging to the same category are unified to the same value, and they are marked in the image according to their horizontal and vertical coordinate information.

8. The method for obtaining connected components based on a ZYNQ chip as described in claim 1, characterized in that: When the pixel at the edge of the image is taken as the center pixel, if the pixel information in the selected area is empty, zero padding is performed.

Images