Connected domain marking circuit, chip, processor and method

Abstract

The present application discloses a connected domain labeling circuit, chip, processor, and method, and relates to the field of chip technology. The connected domain labeling circuit includes m processing unit arrays and a label processing module; the m processing unit arrays are used to obtain an input pixel vector, and the width corresponding to the i-th sub-pixel vector in the input pixel vector is the same as the width of the i-th processing unit array in the m processing unit arrays; the i-th processing unit array is used to assign a label to the i-th sub-pixel vector to obtain a temporary label corresponding to each pixel in the i-th sub-pixel vector; the label processing module is used to merge the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain a final label corresponding to each pixel in the input pixel vector, and the final label is used to obtain a connected domain in the input pixel vector. The present application improves the labeling efficiency of the connected domain by jointly processing the input pixel vector through multiple processing unit arrays.

Description

Technical Field

[0001] The present application relates to the field of chip technology, and in particular to a connected domain marking circuit, chip, processor and method. Background Art

[0002] CCL (Connected Component Labeling) is a traditional CV (Computer Vision) algorithm that is widely used in image processing fields, such as foreground and background separation, image segmentation, object recognition, and OCR (Optical Character Recognition).

[0003] In related technologies, the CCL algorithm, which is inherently serial, is fundamentally parallelized by adopting a pyramid-structured processing unit array, thus realizing a pixel-level parallel CCL method. However, in actual applications, the processing delay of a single high-input-dimensional processing unit array is relatively high, and its corresponding processing complexity increases rapidly with the increase of parallel input pixels, thus affecting the overall performance. Summary of the Invention

[0004] The present invention provides a connected domain marking circuit, chip, processor, and method. The technical solution is as follows:

[0005] According to one aspect of an embodiment of the present application, a connected domain marking circuit is provided, the connected domain marking circuit comprising an array of m processing units and a label processing module, wherein the difference between the number of processing units corresponding to the nth row and the number of processing units corresponding to the n-1th row in the processing unit array is 1, and m and n are integers greater than 1;

[0006] The m processing unit arrays are used to obtain an input pixel vector, the input pixel vector including m sub-pixel vectors, the width corresponding to the i-th sub-pixel vector in the m sub-pixel vectors is the same as the width of the i-th processing unit array in the m processing unit arrays, the input pixel vector includes multiple consecutive pixels in the same row of the input image, and i is a positive integer;

[0007] The i-th processing unit array is used to assign a label to the i-th sub-pixel vector to obtain a temporary label corresponding to each pixel in the i-th sub-pixel vector;

[0008] The label processing module is used to merge the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain the final label corresponding to each pixel in the input pixel vector; wherein the final label is used to obtain the connected domain in the input pixel vector, each pixel belonging to the same connected domain in the input pixel vector has the same final label, and different connected domains in the input pixel vector correspond to different final labels.

[0009] According to one aspect of an embodiment of the present application, a chip is provided, wherein the processor includes the connected domain marking circuit as described above.

[0010] According to one aspect of an embodiment of the present application, a processor is provided, comprising the connected domain marking circuit as described above.

[0011] According to one aspect of an embodiment of the present application, a computer device is provided. The computer device includes a processor. The processor includes the connected domain marking circuit as described above.

[0012] According to one aspect of an embodiment of the present application, a connected domain labeling method based on a connected domain labeling circuit is provided. The connected domain labeling circuit includes an array of m processing units and a label processing module. The difference between the number of processing units corresponding to the nth row and the number of processing units corresponding to the n-1th row in the processing unit array is 1, and m and n are integers greater than 1. The method includes:

[0013] The m processing unit arrays obtain an input pixel vector, the input pixel vector including m sub-pixel vectors, the width corresponding to the i-th sub-pixel vector in the m sub-pixel vectors is the same as the width of the i-th processing unit array in the m processing unit arrays, the input pixel vector includes multiple consecutive pixels in the same row of the input image, and i is a positive integer;

[0014] The i-th processing unit array performs label assignment on the i-th sub-pixel vector to obtain a temporary label corresponding to each pixel in the i-th sub-pixel vector;

[0015] The label processing module merges the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain a final label corresponding to each pixel in the input pixel vector; wherein the final label is used to obtain a connected domain in the input pixel vector, each pixel belonging to the same connected domain in the input pixel vector has the same final label, and different connected domains in the input pixel vector correspond to different final labels.

[0016] The technical solutions provided in the embodiments of the present application can bring the following beneficial effects:

[0017] By adopting multiple processing unit arrays, labels are assigned to each sub-pixel unit in the same input pixel vector, and temporary labels corresponding to each pixel in the input pixel vector are obtained. The temporary labels corresponding to each pixel are then merged through the label processing module to obtain the final labels corresponding to each pixel. Compared with processing the input pixel vector through a processing unit array, the single processing unit array has many levels and high complexity. The present application is equivalent to splitting a high-input-dimensional processing unit array into multiple low-input-dimensional small processing unit arrays. Constrained by the architecture of the processing unit array, the small processing unit array has fewer levels (that is, fewer processing units are required) and lower complexity, which can effectively reduce the processing delay of the input pixel vector and thereby improve the labeling efficiency of the connected domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a schematic diagram of a processing unit array provided by one embodiment of the present application;

[0019] Figure 2 This is a schematic diagram of a connected domain marking circuit provided by an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a first type of processing unit provided by an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of a neighborhood labeling algorithm provided by one embodiment of the present application;

[0022] Figure 5 is a schematic diagram of the operation process of two processing unit arrays provided in one embodiment of the present application;

[0023] Figure 6 This is a schematic diagram of a connected domain labeling circuit provided by another embodiment of the present application applied to an image reconstruction scenario;

[0024] Figure 7 This is a flowchart of a connected domain marking method based on a connected domain marking circuit provided in one embodiment of the present application. DETAILED DESCRIPTION

[0025] Before introducing the embodiments of the present application, the relevant terms involved in the present application are first explained.

[0026] 1. Connected Component Labeling (CCL): Connected component labeling aims to find a unique label for each group of connected elements in the input data. This is done by topological clustering based on connectivity to detect connected regions in the input data. In the present embodiment, CCL may refer to the process of labeling connected regions in the input image. Connected regions in the input image can be implemented as connected regions consisting of foreground pixels.

[0027] 2. Pyramid array: refers to an array of processing units formed by arranging multiple processing units in a pyramid structure, which can be used to implement global search and marking of connected domains. The first row in the pyramid array includes the most processing units, i.e., the bottom of the pyramid array, and the last row in the pyramid array includes one processing unit, i.e., the vertex of the pyramid. Optionally, the pyramid array in the embodiment of the present application can also be referred to as a multi-dimensional input pyramid array, a pyramid processing unit array, an inverted pyramid array, etc. The CCL algorithm based on the pyramid array eliminates the data dependency between pixels in the CCL process, thereby realizing pixel-level parallel CCL.

[0028] 3. PE (Processing Element): A processing unit, which is the base element of the pyramid array and can be used to execute arithmetic logic, such as threshold segmentation logic, comparison logic, sorting logic, multiplexing logic, etc. Optionally, the processing unit in the embodiment of the present application can also be referred to as a processing node.

[0029] 4. FPGA (Field-Programmable Gate Array): A field-programmable gate array (FPGA) is a program-driven logic device, much like a microprocessor. Its control program is stored in memory and automatically loaded onto the chip for execution upon power-up. FPGAs emerged as a semi-specified circuit within the field of ASICs (Application Specific Integrated Circuits), enabling specific functions to be implemented through software coding.

[0030] 5. Clock cycle: This is the inverse of the clock frequency, also known as the oscillation period. The clock cycle is the most basic and smallest unit of time in a computer. During one clock cycle, the processor completes only one basic action (also known as a "micro-operation").

[0031] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0032] Artificial Intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0033] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI disciplines. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0034] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, autonomous driving, drones, digital twins, virtual humans, robots, artificial intelligence generated content (Artificial Intelligence Generated Content, AIGC), conversational interaction, smart medical care, intelligent customer service, game AI, virtual reality (VR), augmented reality (AR), etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0035] The technical solution of this application mainly relates to the development and design of AI processors in artificial intelligence technology, and mainly relates to connected domain marking circuits that can be used for connected domain marking.

[0036] In related technologies, one way to implement connected domain labeling is to construct a processing unit array. For example, multiple processing units are arranged in a pyramid structure to obtain a processing unit array. For example, the structural format of the processing unit array can refer to Figure 1The processing unit array 100 is a pyramid array with eight input dimensions (i.e., width). Eight processing units are arranged in parallel in the first row of the processing unit array 100, enabling it to simultaneously process eight pixels in parallel, e.g., each processing unit processes one pixel. Seven processing units are arranged in parallel in the second row of the processing unit array 100, with every two adjacent processing units in the first row connected to one processing unit in the second row. Six processing units are arranged in parallel in the third row of the processing unit array 100, with every two adjacent processing units in the second row connected to one processing unit in the third row. This continues in this order. Arranging the processing units in this manner creates a complete processing unit array.

[0037] For a processing unit array with an input dimension of P, the corresponding processing delay can be expressed as: (LP+LL) clock cycles; where LP is the time required for the processing unit array to process an input pixel vector. If the processing time of the processing unit is one clock cycle, then for a processing unit array with an input dimension of P, it has P rows of processing units. Since each row of processing units runs in parallel, the corresponding LP is P clock cycles. LL is the time required to organize the output (such as labels) of the processing unit array. As the input dimension increases, the more rows the processing unit array has, the more complex the processing logic becomes, and the corresponding processing delay of the processing unit array becomes higher and higher, which is not conducive to improving the computational performance of the CCL algorithm. At the same time, the number of processing units required for the processing unit array is also increasing, and the space complexity is increasing, which is not conducive to the miniaturization of hardware.

[0038] Furthermore, after processing a row of pixels in an input image (e.g., a small image with a small row width), the data in the processing unit array must be cleared before the next row of pixels in the input image can be processed using the processing unit array. Furthermore, the CCL for each row of pixels must use the labeling results from the previous row of pixels. Therefore, for an input image with a row width of W and a processing unit array with an input dimension of P, the CCL must satisfy ceil(W / P) >= (LP+LL) each time the processing unit array wraps around a row to ensure that the labeling results for the previous row of pixels are ready. Ceil(W / P) refers to the time required for the input port of the processing unit array to complete the input of a row of pixels, and ceil() is used to return the smallest integer greater than or equal to W / P. Therefore, each time the processing unit array wraps around a row, if ceil(W / P) < (LP+LL), it must wait for (LP+LL)-ceil(W / P) clock cycles. If the input image row width is fixed, the CCL computation performance implemented by the processing unit array will further degrade as the input dimension increases. It should be noted that for large images (i.e., images with larger row widths), since ceil(W / P) is much larger than (LP+LL), there is no need to wait when wrapping.

[0039] Please refer to Figure 2 , which shows a schematic diagram of the structure of a connected domain marking circuit provided by an embodiment of the present application. The connected domain marking circuit 200 includes an array of m processing units and a label processing module, where m is an integer greater than 1.

[0040] The m processing unit arrays are independent of each other, that is, each processing unit array processes its own input independently. Optionally, the m processing unit arrays have the same architecture, and the multiple processing units in each processing unit array are arranged in a pyramid structure, such that the difference between the number of processing units corresponding to the nth row and the number of processing units corresponding to the n-1th row in the processing unit array is 1, where n is an integer greater than 1.

[0041] Among them, the number of processing units refers to the number of processing units, such as for a certain processing unit array, the number of processing units corresponding to its nth row refers to the number of processing units included in the nth row of the processing unit array. Optionally, the input dimension of the processing unit array is the same as the number of rows of the processing unit array, such as for a processing unit array with 8 input dimensions, it includes 8 rows of processing units. For a single processing unit array, the corresponding number of processing units can be expressed as follows: N = n(n+1) / 2; where n is the input dimension of the processing unit array, then for a connected domain labeling circuit with m processing unit arrays, it has mN processing units. The input dimension (width) of the processing unit array can refer to the number of processing units corresponding to the first row of the processing unit array, which can determine how many pixels the processing unit array can process in parallel.

[0042] The processing unit array in the embodiment of the present application can be used to assign and transfer labels to pixels, which is the same as described in the above embodiment. The content not described in the embodiment of the present application can be referred to the above embodiment and will not be repeated here.

[0043] In one example, the m processing unit arrays are used to obtain an input pixel vector, where the input pixel vector includes m sub-pixel vectors, and a width corresponding to an i-th sub-pixel vector in the m sub-pixel vectors is the same as a width of an i-th processing unit array in the m processing unit arrays, where i is a positive integer;

[0044] The input pixel vector may refer to a vector composed of pixels in the input image. For example, the input pixel vector may include multiple consecutive pixels in the same row of the input image. In this way, the input pixel vector can be obtained by arranging the multiple consecutive pixels in the order in which they appear in the input image. The above-mentioned input image may refer to two-dimensional or multi-dimensional data (such as images, three-dimensional data with spatial information generated by lidar, etc.) to be marked as a connected domain. The embodiment of the present application does not limit the source of the input image.

[0045] In an embodiment of the present application, the pixel value of a pixel can be used to represent the pixel, and the above-mentioned input pixel vector can be represented by a pixel value vector composed of the pixel values ​​corresponding to each pixel included therein, and the arrangement order of the pixel values ​​corresponding to each pixel is the same as the arrangement order of each pixel. For example, since the pixel value of a pixel can be represented by a numerical value in a format such as grayscale, RGB (Red-Green-Blue), YUV (a color space), or binary, the above-mentioned input pixel vector can refer to a vector composed of numerical values ​​in a format such as grayscale, RGB, YUV, or binary, and the embodiment of the present application does not limit this.

[0046] A sub-pixel vector may be a pixel vector obtained by evenly dividing the input pixel vector according to the width of a single processing unit array. For example, if the width of the input pixel vector is 8 and the width of the processing unit array is 4, the input pixel vector may be divided into two sub-pixel vectors, each with a width of 4. The width of a pixel vector may refer to the number of pixels contained in the pixel vector.

[0047] Optionally, the total width of the m processing unit arrays is the same as the width of the input pixel vector, and each processing unit array is used to obtain its corresponding sub-pixel vector. For example, m processing unit arrays can be used to concurrently obtain their corresponding sub-pixel vectors within the same clock cycle, in the order in which the sub-pixel vectors appear in the input image. That is, the i-th processing unit array is used to obtain the i-th sub-pixel vector.

[0048] For example, for the input pixel vector: P1-P8, it can be divided into two sub-pixel vectors: P1-P4 and P5-P8, and the first processing unit array can be used to obtain the sub-pixel vector P1-P4, and the second processing unit array can be used to obtain the sub-pixel vector P5-P8; or the second processing unit array can be used to obtain the sub-pixel vector P1-P4, and the first processing unit array can be used to obtain the sub-pixel vector P5-P8. This embodiment of the present application is not limited to this.

[0049] Optionally, each processing unit array may acquire each pixel in the sub-pixel vector from left to right, or from right to left, and this is not limited in this embodiment of the present application. For example, for sub-pixel vectors P1-P4, the processing unit array may use the first processing unit on the right in the first row to acquire P1, the second processing unit on the right to acquire P2, the third processing unit on the right to acquire P3, and the fourth processing unit on the right to acquire P4. It should be noted that the order in which each processing unit array acquires pixels should correspond to the pixel order in the input pixel vector to preserve the adjacent relationship between pixels in the same row.

[0050] In one example, after m processing unit arrays obtain their respective sub-pixel vectors, the m processing unit arrays run in parallel to assign labels to the m sub-pixel vectors respectively, and obtain temporary labels corresponding to respective pixels in the m sub-pixel vectors.

[0051] That is, the m processing unit arrays acquire their respective inputs in parallel within the same clock cycle and process their respective inputs in parallel within the same clock cycle. This maintains parallel acceleration capabilities while reducing processing latency. For example, based on the above embodiment, for sub-pixel vectors P1-P4 and sub-pixel vectors P5-P8, while the first processing unit array processes each pixel in sub-pixel vectors P1-P4 in parallel, the second processing unit array also processes each pixel in sub-pixel vectors P5-P8 in parallel, thereby simultaneously obtaining temporary labels corresponding to P1-P8.

[0052] Optionally, after obtaining a temporary label corresponding to each pixel in the input pixel vector through the array of m processing units, the connected domain labeling circuit transmits the temporary label corresponding to each pixel to a label processing module. The label processing module is configured to merge the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain a final label corresponding to each pixel in the input pixel vector; wherein the final label is used to obtain a connected domain in the input pixel vector, wherein each pixel in the connected domain in the input pixel vector has the same final label. Optionally, the label processing module can obtain the final label through internal design, either through a single pass or through multiple passes after caching.

[0053] The temporary label merging in the embodiment of the present application refers to the process of unifying the temporary labels of each pixel in the connected domain corresponding to the input pixel vector. If there are multiple different temporary labels in the same connected domain, the temporary label with the smallest label value can be determined as the final label corresponding to the connected domain, that is, the final label corresponding to each pixel in the connected domain. The operation process of the label processing module will be specifically described below and will not be repeated here.

[0054] Optionally, after obtaining the connected domains in each input pixel vector corresponding to the input image, the connected domains in each input pixel vector can be merged based on the label to obtain the final connected domain corresponding to the input image, so as to complete the connected domain labeling of the input image.

[0055] When the total input dimension remains unchanged, the embodiment of the present application is equivalent to using multiple processing unit arrays with low input dimensions to replace a processing unit array with high input dimensions to achieve acceleration of CCL. Taking the use of N (N is 2^k) processing unit arrays to replace one processing unit array as an example, the processing delay of the input pixel vector can be reduced to: LP / N+log2(N)+LL clock cycles; wherein, log2(N) refers to the time required for the label processing module to merge the processing results of the m processing unit arrays. LP / N greatly reduces the overall processing delay of the input pixel vector, and the cost log2(N) paid is far less than the benefit. Therefore, the technical solution provided by the embodiment of the present application can bring about a significant improvement in the overall performance, which can effectively improve the efficiency of connected domain labeling.

[0056] For example, when the resolution of the input image is 16*16 (i.e., W=16), for a single 8-input dimensional processing unit array, the corresponding computational delay LP is 8 clock cycles, and the label processing delay LL is 4 clock cycles. Therefore, the corresponding processing delay of the single 8-input dimensional processing unit array is: 8+4=12 clock cycles, and the line break waiting delay is: 8+4-ceil(16 / 8)=10 clock cycles. However, after replacing the single 8-input dimensional processing unit array with two 4-input dimensional processing unit arrays, the processing delay can be reduced to: 4+1+4=9 clock cycles, and the line break waiting delay is reduced to: 8 / 2+log2(2)+4-ceil(16 / 8)=7 clock cycles. The line break waiting delay benefit is 3 clock cycles, and the number of idle clock cycles of the processing unit is reduced by 30%.

[0057] For another example, for a single 16-input processing unit array, the corresponding computational delay LP is 16 clock cycles, and the label processing delay LL is 4 clock cycles. Therefore, the corresponding processing delay of the single 16-input processing unit array is 16+4=20 clock cycles, and the line break waiting delay is 16+4-ceil(16 / 16)=19 clock cycles. However, after replacing the single 16-input processing unit array with four 4-input processing unit arrays, the processing delay can be reduced to 16 / 4+log2(4)+4=10 clock cycles, and the line break waiting delay can be reduced to 16 / 4+log2(4)+4-ceil(16 / 16)=9 clock cycles. The line break waiting delay benefit is 10 clock cycles, and the number of idle clock cycles of the processing unit is reduced by 53%.

[0058] As can be seen, the embodiments of the present application use multiple processing unit arrays with low input dimensions instead of a single processing unit array with high input dimensions to perform parallel processing on input pixel vectors, which can effectively reduce the delay of connected component labeling and thus improve the efficiency of connected component labeling, especially when the input dimension of the original single processing unit array is high.

[0059] In addition, when the total input dimension remains unchanged, by replacing a single high-input-dimensional processing unit array with multiple low-input-dimensional processing unit arrays, the number of processing units can be effectively reduced, thereby reducing the spatial complexity of the connected domain labeling circuit and saving the construction cost of the connected domain labeling circuit.

[0060] In one feasible example, if the width of the input pixel vector needs to be increased, the number of processing unit arrays can be adaptively increased to meet the data throughput requirement. This effectively increases the data throughput of the connected domain marking circuit while maintaining a certain processing efficiency. The connected domain marking circuit in the embodiment of the present application is easy to topologically and scalably adjust, and throughput and processing delay can be adjusted simply by adjusting the number of processing unit arrays.

[0061] The embodiments of the present application do not limit the number and input dimensions of the processing unit arrays in the connected domain marking circuit. These can be set and adjusted based on actual usage requirements. For example, the number and input dimensions of the processing unit arrays in the connected domain marking circuit can be set and adjusted based on real-time requirements or data throughput requirements to adapt to different scenarios. In one example, the connected domain marking circuit can be implemented as a chip that meets different real-time requirements or data throughput requirements, and users can select a chip based on their needs.

[0062] In a feasible example, multiple connected domain labeling circuits can be used to simultaneously label connected domains of multiple input images, thereby further speeding up the connected domain labeling and thereby improving the efficiency of the connected domain labeling.

[0063] To sum up, the technical solution provided by the embodiment of the present application adopts multiple processing unit arrays to assign labels to each sub-pixel unit in the same input pixel vector, obtain temporary labels corresponding to each pixel in the input pixel vector, and then merge the temporary labels corresponding to each pixel through the label processing module to obtain the final labels corresponding to each pixel. Compared with processing the input pixel vector through a processing unit array, the single processing unit array has many levels and high complexity. The present application is equivalent to splitting a high-input-dimensional processing unit array into multiple low-input-dimensional small processing unit arrays. Constrained by the architecture of the processing unit array, the small processing unit array has fewer levels (that is, fewer processing units are required) and lower complexity, which can effectively reduce the processing delay of the input pixel vector and thereby improve the labeling efficiency of the connected domain.

[0064] In addition, by using multiple processing unit arrays with low input dimensions instead of a single processing unit array with high input dimensions, parallel processing of input pixel vectors can effectively reduce the latency of connected component labeling, thereby improving the efficiency of connected component labeling, especially when the input dimension of the original single processing unit array is high.

[0065] The specific operation process of the processing unit array will be described below.

[0066] Since the processing method of each processing unit array is the same, the following will take the i-th processing unit array among the above-mentioned m processing unit arrays as an example to illustrate that the i-th processing unit array can be used to assign labels to the i-th sub-pixel vector to obtain temporary labels corresponding to each pixel in the i-th sub-pixel vector.

[0067] Among them, the i-th processing unit array can refer to any processing unit array in the m processing unit arrays, and the i-th sub-pixel vector is the input of the i-th processing unit array. The label in the embodiment of the present application is used to identify a connected domain, each connected domain corresponds to a unique label, and the pixels in each connected domain have the same label. The label value of the label can be an integer, such as the labels corresponding to the various connected domains in the input image are distributed in a continuous positive integer distribution, or they can be distributed in a discontinuous positive integer distribution. The temporary label can refer to the label that the processing unit array temporarily assigns to the pixel based on the existing information. As the information is updated, the temporary label will also be updated.

[0068] In one example, for each processing unit array, the first row of the processing unit array includes p first-type processing units, and the nth row includes p-n+1 second-type processing units, where p is an integer greater than 1.

[0069] The functions of the first type of processing units are different from those of the second type of processing units. The first type of processing units can be used for pixel acquisition, threshold segmentation, and label assignment, while the second type of processing units can be used for label transfer and cycle synchronization.

[0070] Exemplarily, the p first-class processing units in the above-mentioned i-th processing unit array are used to respectively obtain each pixel in the i-th sub-pixel vector, and each first-class processing unit is used to obtain a pixel in the i-th sub-pixel vector. The pixel in the embodiment of the present application is characterized by the pixel value corresponding to the pixel, that is, each first-class processing unit actually obtains the pixel value of the pixel, and the process can be completed within one clock cycle. Optionally, the p first-class processing units are also used to obtain a first threshold value, which can be preset according to demand. For example, if the element in the pixel input vector is a vector, the first threshold value is a vector; if the element in the pixel input vector is a scalar, the first threshold value is a scalar. The first threshold value is used to segment pixels.

[0071] After obtaining each pixel in the i-th sub-pixel vector, the p first-type processing units in the above-mentioned i-th processing unit array are used to perform threshold segmentation on each pixel in the i-th sub-pixel vector respectively to obtain binary data corresponding to each pixel in the i-th sub-pixel vector, and the binary data can be a first value or a second value.

[0072] The first numerical value and the second numerical value are different, and the first numerical value and the second numerical value can be integers. Optionally, the first numerical value can be used to indicate that the current pixel is a foreground pixel, such as 1, and the second numerical value can be used to indicate that the current pixel is a background pixel, such as 0. In the embodiment of the present application, the pixel value of the foreground pixel is greater than the first threshold value, and the pixel value of the background pixel is less than or equal to the first threshold value, then the embodiment of the present application may further include the following content:

[0073] The qth first-category processing unit among the p first-category processing units is further configured to, when the pixel value of the qth pixel is greater than the first threshold, determine the first value as the binary data corresponding to the qth pixel. Alternatively, the qth first-category processing unit among the p first-category processing units is further configured to, when the pixel value of the qth pixel is less than or equal to the first threshold, determine the second value as the binary data corresponding to the qth pixel. The qth first-category processing unit may be any first-category processing unit among the p first-category processing units, and q is a positive integer not greater than p.

[0074] For example, if the pixel values ​​of pixels P1 - P4 are 38, 21, 64, and 170, respectively, and the first threshold is 32, then the binary data of pixels P1 - P4 are 1, 0, 1, and 1, respectively.

[0075] In one example, reference Figure 3 , which shows a hardware structure diagram of a first type of processing unit provided by an embodiment of the present application. The first type of processing unit 300 may include a threshold segmentation module 301, a label comparison module 302 and a label sorting module 303.

[0076] The threshold segmentation process can be implemented by the threshold segmentation module 301. After obtaining the first threshold and the pixel value of the pixel, the threshold segmentation module 301 compares the first threshold and the pixel value to obtain binary data of the pixel, which is recorded as foreground-enable-o (abbreviated as en).

[0077] Optionally, when the technical solution provided in the embodiment of the present application is applied in a high-speed camera transmission scenario (i.e., during the transmission of pixels, the pixels are synchronously marked as connected domains), the threshold segmentation module 301 is also provided with an enable signal, which is the sum operation result of the FVAL (frame synchronization signal) and LVAL (line synchronization signal) corresponding to the high-speed camera. When the enable signal is 1, the threshold segmentation module 301 executes the threshold segmentation process.

[0078] Optionally, the p first-category processing units operate in parallel to simultaneously complete threshold segmentation for each pixel in the i-th sub-pixel vector, thereby improving the efficiency of threshold segmentation. The p first-category processing units in each processing unit array also operate in parallel to simultaneously complete threshold segmentation for each pixel in the m-th sub-pixel vector, thereby improving the efficiency of threshold segmentation.

[0079] After obtaining the binary data corresponding to each pixel in the i-th sub-pixel vector, the p first-type processing units in the above-mentioned i-th processing unit array are also used to assign labels to each pixel in the i-th sub-pixel vector based on the binary data and the reference labels corresponding to each pixel in the i-th sub-pixel vector, so as to obtain temporary labels corresponding to each pixel in the i-th sub-pixel vector.

[0080] Among them, the above-mentioned reference labels are related to the adjacent pixels of the pixel, and the distance between the adjacent pixels of the pixel is less than the distance threshold. Optionally, according to different neighborhood labeling algorithms, the current pixel may correspond to different numbers of adjacent pixels, and the embodiment of the present application does not limit the labeling method of adjacent pixels. For example, for the four-neighborhood labeling algorithm, the current pixel may correspond to 4 adjacent pixels, and the distance between the adjacent pixels and the current pixel is the minimum pixel distance. For the eight-neighborhood labeling algorithm, the current pixel may correspond to 8 adjacent pixels, and the distance between the adjacent pixels and the current pixel is less than or equal to times the minimum pixel distance, where the minimum pixel distance may refer to the distance between two adjacent pixels in the same row in the input image.

[0081] In an embodiment of the present application, since the labels of the pixels after the current pixel are unknown, for the four-neighborhood labeling algorithm, the adjacent pixels of the current pixel can be reduced to: the pixel directly above and the pixel directly to the left; for the eight-neighborhood labeling algorithm, the adjacent pixels of the current pixel can be reduced to: the pixel to the upper left, the pixel directly above, the pixel to the upper right, and the pixel to the directly left. In a feasible example, for the eight-neighborhood labeling algorithm, the adjacent pixels of the current pixel can also be reduced to: the pixel to the upper left, the pixel directly above, and the pixel to the directly left. This helps to reduce the computational complexity of label assignment (such as the complexity of the comparison logic and sorting logic of subsequent labels), thereby improving the efficiency of label assignment. For example, referring to Figure 4 For the four-neighborhood labeling algorithm, the current pixel 401 (i.e., p) corresponds to two adjacent pixels: b and d; for the eight-neighborhood labeling algorithm, the current pixel 401 (i.e., p) corresponds to four adjacent pixels: a, b, c, and d; for the optimized eight-neighborhood labeling algorithm, the current pixel 401 (i.e., p) corresponds to three adjacent pixels: a, b, and d.

[0082] A reference label is a label used for reference and can be used to determine a temporary label for the current label. In one example, the label of the adjacent pixel corresponding to the current pixel can be determined as the reference label for the current pixel. Taking the qth pixel as an example, the reference label determination process can also include the following.

[0083] 1. When the qth pixel is the first pixel in the input image, the qth pixel does not correspond to a reference label.

[0084] For the first pixel in the input image, there is no pixel before it, that is, the first pixel has no adjacent pixels, so the first pixel does not correspond to a reference label.

[0085] 2. When the qth pixel is not the first pixel in the input image and is located in the first row of the input image, the qth pixel corresponds to a reference label, and the label value of the reference label is the first numerical value or the second numerical value, and the first numerical value and the second numerical value are less than or equal to 1.

[0086] Optionally, since the first row of pixels does not have the previous row of pixels, it does not have adjacent pixels in the previous row, and since each first-class processing unit processes pixels individually, for the current pixel, the label of the pixel to its immediate left is unknown. At this time, binary data can be used as the label value to represent the label value of the parameter label.

[0087] For example, if the pixel to the immediate left is a foreground pixel, the first value can be determined as the label value of the reference label of the current pixel to indicate that there is a label to be inherited; if the pixel to the immediate left is a background pixel, the second value can be determined as the label value of the reference label of the current pixel to indicate that there is no label to be inherited.

[0088] 3. When the qth pixel is not located in the first row and the first column of the input image, the qth pixel corresponds to at least two reference labels, which are determined based on the final labels assigned to the adjacent pixels.

[0089] Optionally, refer to Figure 4 , for the four-neighborhood labeling algorithm, the final labels of b and d can be determined as the reference label for determining the current pixel 401. Among them, the label of d is unknown. If d is a foreground pixel, its corresponding reference label is assigned a value of 1; if d is a background pixel, its corresponding reference label is assigned a value of 0. In a feasible example, for pixels with unknown labels, their corresponding reference labels can be assigned a value of 1. In an embodiment of the present application, for background pixels, their corresponding final labels can be represented by 0 (i.e., the second numerical value), labels with label values ​​0 and 1 are not valid labels, and labels with label values ​​greater than 1 are valid labels. Valid labels can be used for label inheritance, which will be explained in detail below and will not be repeated here.

[0090] For the eight-neighborhood labeling algorithm, the final labels of a, b, c, and d can be determined as the reference label for determining the current pixel 401. Among them, the label of d is unknown. If d is a foreground pixel, its corresponding reference label is assigned a value of 1; if d is a background pixel, its corresponding reference label is assigned a value of 0.

[0091] For the optimized eight-neighborhood labeling algorithm, the final labels of a, b, and d can be determined as the reference label for determining the current pixel 401. Among them, the label of d is unknown. If d is a foreground pixel, its corresponding reference label is assigned a value of 1; if d is a background pixel, its corresponding reference label is assigned a value of 0.

[0092] The final label may refer to a unique label that can be assigned to the connected domain by the connected domain labeling circuit during the current processing.

[0093] 4. When the qth pixel is not located in the first row of the input image but in the first column of the input image, the qth pixel corresponds to at least one reference label, which is determined based on the final labels assigned to the adjacent pixels.

[0094] Since the pixels in the first column have no adjacent pixels to the left, the adjacent pixels to the left may be ignored. In some embodiments, the reference label corresponding to the adjacent pixels to the left may be assigned a value of 0. For example, Figure 4 For the four-neighborhood labeling algorithm, the final label of b can be determined as the reference label for determining the current pixel 401; for the eight-neighborhood labeling algorithm, the final labels of a, b and c can be determined as the reference label for determining the current pixel 401; for the optimized eight-neighborhood labeling algorithm, the final labels of a and b can be determined as the reference label for determining the current pixel 401.

[0095] In one example, taking the qth first-category processing unit as an example, the temporary label allocation process may be as follows:

[0096] The qth first-category processing unit among the above-mentioned p first-category processing units is used to obtain the number of valid labels corresponding to the qth pixel based on the reference label corresponding to the qth pixel when the binary data corresponding to the qth pixel in the i-th sub-pixel vector is the first numerical value. The valid label number refers to the number of valid labels, and the label value of the valid label is greater than the first numerical value. q is a positive integer not greater than p.

[0097] When the binary data corresponding to the qth pixel is the first value, the qth pixel is a foreground pixel, and the qth first-category processing unit needs to assign a label to the qth pixel. Optionally, the qth first-category processing unit may determine a reference label with a label value greater than 1 as a valid label, and then count the number of valid labels to obtain the number of valid labels.

[0098] The above-mentioned qth first-category processing unit is also used to determine the first numerical value as the label value of the temporary label of the qth pixel when the number of valid labels is equal to 0; or, when the number of valid labels is equal to 1, determine the valid label as the temporary label of the qth pixel; or, when the number of valid labels is greater than 1, determine the valid label with the smallest label value as the temporary label of the qth pixel.

[0099] When the number of valid labels is equal to 0, it indicates that the current pixel has no inheritable label, and the qth first-category processing unit can directly determine the first value (such as 1) as the label value of the temporary label of the qth pixel to indicate that the qth pixel needs to be assigned a new temporary label. Figure 4 For the current pixel 401, if a=b=0, d=0 / 1, and the binary data of p is 1, the label value of the temporary label of p is 1.

[0100] When the number of valid labels is equal to 1, it indicates that the current pixel has a label that can be inherited, and the qth first-class processing unit can directly determine the valid label as the temporary label of the qth pixel. Figure 4 For the current pixel 401, if a=6, b=0, d=0 / 1, and the binary data of p is 1, the label value of the temporary label of p is 6.

[0101] When the number of valid labels is greater than 1, it indicates that there may be conflicting labels. The qth first-class processing unit can determine the valid label with the smallest label value as the temporary label of the qth pixel. Figure 4 For the current pixel 401, if a=6, b=8, d=0 / 1, and the binary data of p is 1, then the label value of p's temporary label is 6. Here, relative to the label with a label value of 6, the label with a label value of 8 is a conflicting label (also called an equivalent label), that is, a and b are actually in the same connected domain. For another example, for the current pixel 401, if a=6, b=6, d=0 / 1, and the binary data of p is 1, the label values ​​of the two valid labels are the same, then the label value of p's temporary label can be 6.

[0102] In one example, reference Figure 3, the temporary label assignment process can be implemented by the label comparison module 302 in the first type of processing unit 300. After obtaining the three reference labels corresponding to the current pixel (such as a, b and d mentioned above), the first type of processing unit 300 compares the label values ​​of the three reference labels with 1 respectively to obtain the number of different valid labels corresponding to the current pixel, which is recorded as cft-num. The label comparison module 302 determines the valid label of the current pixel based on the reference labels, which is recorded as temple-label, and determines the temporary label (label-o) based on temple-label (optionally, this step can also be completed by the label sorting module 303). Optionally, the label comparison module 302 can be implemented by 6 parallel comparators, 2 MUXs (Multiplexers) and 1 CNT (Count).

[0103] Optionally, the p first-category processing units operate in parallel to simultaneously complete label assignment for each pixel in the i-th sub-pixel vector, thereby improving label assignment efficiency. The p first-category processing units in each processing unit array also operate in parallel to simultaneously complete label assignment for each pixel in the m-th sub-pixel vector, thereby improving label assignment efficiency.

[0104] In one example, after the qth first-category processing unit completes the label assignment for the qth pixel, when the binary data of the qth pixel is a first value (such as 1), the qth first-category processing unit is further used to determine conflicting labels from the valid labels when the number of valid labels is greater than 1, and for conflicting labels, there are equivalent labels; and sort the conflicting labels to obtain a conflicting label sequence.

[0105] The conflict label sequence is used to update the conflict label list, which is used to count conflict labels. Optionally, the qth first-category processing unit can sort the conflict labels in ascending order of label value to obtain a conflict label sequence. The label with the smallest label value in the conflict label sequence can be determined as the temporary label (label-o) of the qth pixel. The conflict labels in the conflict label list are also arranged in ascending order of label value.

[0106] Since conflicting labels may exist when the number of valid labels is greater than 1, the qth first-category processing unit can further determine whether the valid labels are the same. If the valid labels are different, conflicting labels exist, and the conflicting labels can be sorted. For example, based on the above embodiment, if a = 6 and b = 8, a and b are sorted to obtain a conflicting label sequence arranged as a and b.

[0107] For example, refer to Figure 3The label sorting process can be implemented by the label sorting module 303 in the first type of processing unit 300. The label sorting module 303 can be implemented by 3 comparators and 1 MUX. The label sorting module 303 can sort the maximum 3 different valid labels that may appear in the current clock cycle (i.e., temp-lable1 to temp-lable3 in the figure), and then use the valid label with the smallest label value as the temporary label of the current pixel, i.e., label-o, and output the conflict label sequence: cft-lable(1) to cft-lable(3).

[0108] Optionally, before enabling the label sorting module 303, the first type processing unit 300 also needs to determine whether the current pixel is a foreground pixel. If the current pixel is a foreground pixel, the first type processing unit 300 enables the label sorting module 303. If the current pixel is a background pixel, the first type processing unit 300 does not enable the label sorting module 303.

[0109] In one example, after obtaining the temporary labels and binary data corresponding to each pixel in the i-th sub-pixel vector, the above-mentioned i-th processing unit array is used to transmit the temporary labels and binary data corresponding to each pixel in the i-th sub-pixel vector to obtain the output corresponding to each second-type processing unit, which includes the temporary label and binary data.

[0110] The output of each first-type processing unit includes the temporary label and binary data of the pixel it processes. The output of each second-type processing unit includes the temporary label and binary data inherited from its two corresponding input processing units, which are the processing units in the row above the row where the second-type processing unit is located and are connected to the second-type processing unit.

[0111] Optionally, the i-th processing unit array may sequentially transmit the temporary label and the binary data through each row of the second type processing units, and the specific content may be as follows:

[0112] The first row of the second-category processing units in the i-th processing unit array is configured to transfer the outputs of the p first-category processing units in the i-th processing unit array to obtain the outputs of the first row of the second-category processing units. The first row of the second-category processing units is the second row of the processing units in the i-th processing unit array.

[0113] Optionally, for each second-type processing unit in the first row of second-type processing units, the second-type processing unit takes the outputs of the two input processing units as input, and takes the output of the two input processing units with the smaller label value as output.

[0114] For example, reference Figure 5For the second type of processing unit 511 in the processing unit array 510, its corresponding two input processing units are connected to it: the first type of processing unit 512 and the first type of processing unit 513. The output of the first type of processing unit 512 can be expressed as (8, 1), where 8 is the label value of the temporary label and 1 is binary data. The output of the first type of processing unit 513 can be expressed as (6, 1), and the output of the second type of processing unit 511 can be (6, 1).

[0115] Optionally, when the binary data included in the outputs of the two input processing units contain a second numerical value (ie, 0), the output of the current second-type processing unit may be directly represented as (0, 0).

[0116] When the binary data corresponding to the two input processing units are both the first value and there is a temporary label with a label value of 1, the output of the other input processing unit with a label value not 1 can be directly determined as the output of the current second-type processing unit.

[0117] The second-category processing unit in the t-th row in the i-th processing unit array is used to transfer the output of the second-category processing unit in the t-1-th row in the i-th processing unit array to obtain the output of the second-category processing unit in the t-th row, where t is a positive integer less than n.

[0118] The second-category processing unit in the tth row is the processing unit in the t+1th row of the i-th processing unit array. For each second-category processing unit in the tth row, the second-category processing unit takes the outputs of the two input processing units as input and takes the output of the second-category processing unit with the smaller label value as output.

[0119] Optionally, the second-type processing units in the same row run in parallel to complete label transfer simultaneously, thereby improving the efficiency of label transfer. The second-type processing units in the same row in each processing unit array also run in parallel to complete label transfer simultaneously, thereby improving the efficiency of label transfer.

[0120] After the last row of the second type of processing units in the i-th processing unit array obtains the output, the label transfer of the i-th sub-pixel vector can be completed.

[0121] In an example, the second type of processing unit is used to implement a two-or-one operation of the tags, and the second type of processing unit can be implemented by a MUX for two-or-one operation.

[0122] In some embodiments, reference Figure 5 , taking the process of two processing unit arrays with 4 input dimensions processing an input pixel vector with a width of 8 as an example, the operation process of the multi-processing unit array is explained.

[0123] like Figure 5 As shown, the connected domain marking circuit includes two processing unit arrays with 4 input dimensions: processing unit array 510 and processing unit array 520. The current input of the connected domain marking circuit is an input pixel vector with a width of 8. The input pixel vector includes pixels corresponding to pixel indices 0-7, and the pixel corresponding to pixel index -1 is the last pixel in the previous input pixel vector of the input pixel vector.

[0124] In an embodiment of the present application, since the input dimension of the processing unit array is 4, the input pixel vector can be divided into two sub-pixel vectors: a first sub-pixel vector composed of pixels corresponding to pixel indices 0-3, and a second sub-pixel vector composed of pixels corresponding to pixel indices 4-7.

[0125] Processing unit array 510 performs threshold segmentation on each pixel in the first sub-pixel vector, obtaining binary data of the first sub-pixel vector as 1011. Processing unit array 520 performs threshold segmentation on each pixel in the second sub-pixel vector, obtaining binary data of the second sub-pixel vector as 1100. Processing unit array 510 and processing unit array 520 operate in parallel.

[0126] The processing unit array 510 simultaneously assigns labels to each pixel in the first sub-pixel vector, and the processing unit array 520 simultaneously assigns labels to each pixel in the second sub-pixel vector. Let's say the eight pixels in the input pixel vector are P0-P7. For P0, its reference labels are 0, 0, 6, and 0 (known), so the temporary label of P0 can be 6. For P1, since it is a background pixel, the temporary label of P1 can be 0. For P2, its reference labels are 6, 0, 8, and 1 (unknown), so the temporary label of P2 can be 6. For P3, its reference labels are 0, 8, 0, and 1 (unknown), so the temporary label of P3 can be 8. For P4, its reference labels are 8, 0, 0, and 1 (unknown), so the temporary label of P4 can be 8. For P5, its reference labels are 0, 0, 9, and 1 (unknown), so the temporary label of P5 can be 9. For P6 and P7, since both are background pixels, the temporary labels of P6 and P7 can both be 0. In summary, the temporary labels of the eight pixels in the input pixel vector are: 6068 8900.

[0127] From left to right, the output of the first first-class processing unit in the processing unit array 510 can be represented as (6, 1), the output of the second first-class processing unit can be represented as (0, 0), the output of the third first-class processing unit can be represented as (6, 1), and the output of the fourth first-class processing unit can be represented as (8, 1).

[0128] From left to right, the output of the first first-class processing unit in the processing unit array 520 can be represented as (8, 1), the output of the second first-class processing unit can be represented as (9, 1), the output of the third first-class processing unit can be represented as (0, 0), and the output of the fourth first-class processing unit can be represented as (0, 0).

[0129] While processing unit array 510 propagates the temporary labels of each pixel in the first sub-pixel vector, processing unit array 520 propagates the temporary labels of each pixel in the second sub-pixel vector. For processing unit array 510, the first second-category processing unit in the second row, based on two inputs: (6, 1) and (0, 0), obtains an output of (0, 0); the second second-category processing unit in the second row, based on two inputs: (0, 0) and (6, 1), obtains an output of (0, 0); the third second-category processing unit in the second row, based on two inputs: (6, 1) and (8, 1), obtains an output of (6, 1). The first second-category processing unit in the third row, based on two inputs: (0, 0) and (0, 0), obtains an output of (0, 0); the second second-category processing unit in the third row, based on two inputs: (0, 0) and (6, 1), obtains an output of (0, 0). The first second-type processing unit in the fourth row obtains the output (0, 0) based on two inputs: (0, 0) and (0, 0).

[0130] For processing unit array 520, the first second-type processing unit in the second row obtains an output of (8, 1) based on two inputs: (8, 1) and (9, 1); the second second-type processing unit in the second row obtains an output of (0, 0) based on two inputs: (9, 1) and (0, 0); the third second-type processing unit in the second row obtains an output of (0, 0) based on two inputs: (0, 0) and (0, 0). The first second-type processing unit in the third row obtains an output of (0, 0) based on two inputs: (8, 1) and (0, 0); the second second-type processing unit in the third row obtains an output of (0, 0) based on two inputs: (0, 0) and (0, 0). The first second-type processing unit in the fourth row obtains an output of (0, 0) based on two inputs: (0, 0) and (0, 0).

[0131] In one example, when each row of the input image corresponds to multiple input pixel vectors, the above-mentioned m processing unit arrays process the multiple input pixel vectors in a pipeline manner; wherein, the m processing unit arrays obtain one input pixel vector per clock cycle, and when the xth input pixel vector is processed by the first row processing unit in the current clock cycle, the x+1th input pixel vector is input to the first row processing unit in the next clock cycle of the current clock cycle, where x is a positive integer.

[0132] Optionally, in an embodiment of the present application, the processing time of the first type of processing unit is related to the dimension of the input pixel vector (such as a positive correlation), then the processing time required for the first row of processing units is related to the dimension of the input pixel vector, and the processing time of the second type of processing unit is one clock cycle, then the processing time of each row of the second type of processing units is one clock cycle.

[0133] For example, taking the example that each row of the input image corresponds to 3 input pixel vectors, after the third clock cycle, the processing result of the first pixel vector by the processing unit in the first row of m processing units in the array can be obtained.

[0134] Among them, in the second clock cycle, the first row of processing units in the m processing unit array adds processing of a new input pixel vector (recorded as the second input pixel vector), and in the third clock cycle, the first row of processing units in the m processing unit array adds processing of a new input pixel vector again (recorded as the third input pixel vector).

[0135] In the fourth clock cycle, the processing results of the first row of processing units in the m processing unit array on the second input pixel vector can be obtained. In the fifth clock cycle, the processing results of the first row of processing units in the m processing unit array on the third input pixel vector can be obtained.

[0136] The same process continues until all three input pixel vectors are processed. For example, if the input dimension of the processing unit array is 3, then after three clock cycles, one input pixel vector can be processed per clock cycle, two input pixel vectors can be processed in four clock cycles, and three input pixel vectors can be processed in five clock cycles.

[0137] Optionally, for an input pixel vector, since its corresponding processing results are distributed in different clock cycles, in order to facilitate the subsequent label processing module, the embodiment of the present application further includes the following content:

[0138] The above-mentioned m processing unit arrays are also used to synchronize the outputs of the input pixel vector in different clock cycles to the last clock cycle corresponding to the input pixel vector, so as to obtain the outputs of the m processing unit arrays for the input pixel vector in the same clock cycle; wherein the output in the same clock cycle is transmitted to the above-mentioned label processing module.

[0139] For example, if the input pixel vector corresponds to a processing result within the first clock cycle, a processing result within the second clock cycle, and a processing result within the third clock cycle, then the processing result within the first clock cycle can be added to the processing result within the third clock cycle. This addition only increases the clock cycle and does not change the content. In addition, the processing result within the second clock cycle can be added to the processing result within the third clock cycle, thereby obtaining the processing result of the input pixel vector in the same clock cycle (i.e., the output of the input pixel vector in the same clock cycle). Optionally, this process can be implemented with the help of registers corresponding to the connected domain marking circuit.

[0140] The following will explain the specific operation process of the label processing module.

[0141] After obtaining the output of the input pixel vector in the same clock cycle, the label processing module is used to merge the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain the final labels corresponding to each pixel in the input pixel vector.

[0142] In one example, the label processing module may be implemented as a logic circuit, which may be used to implement functions such as isolated processing unit search, isolated processing unit merging, label merging, and label storage. These functions are combined to implement the label merging process. The embodiment of the present application may further include the following:

[0143] 1. Isolated processing unit search: The tag processing module is further configured to determine the isolated processing units corresponding to the m processing unit arrays according to the output of each processing unit in the m processing unit arrays.

[0144] The isolated processing unit refers to a processing unit that meets the isolation condition. For example, the isolation condition can be expressed as follows:

[0145]

[0146] Among them, I(r i , c j ) represents the judgment condition of the isolated processing unit. If I(r i , c j )=1, then the rth i Row and c j The processing unit of the column is an isolated processing unit. If I(r i , c j )=0, then the rth i Row and c j The processing unit of the column is not an isolated processing unit. i , c j ) represents the rth in the processing unit array iRow and c j The binary data of the column processing unit, b(r i , c j-1 ) represents the rth in the processing unit array i Row and c j-1 The binary data of the column processing unit, b(r i , c j+1 ) represents the rth in the processing unit array i Row and c j+1 The binary data of the processing unit of the column, n is the number of columns in each row in the processing unit array, and can be used to indicate that a processing unit is the last processing unit in a row.

[0147] That is, for each processing unit in the array of m processing units, the above-mentioned label processing module is also used to determine the processing unit as an isolated processing unit when the output of the processing unit includes a first value and the output of the adjacent processing unit corresponding to the processing unit in the row where the processing unit is located includes a second value.

[0148] Among them, the output of the processing unit includes binary data and a final label. The output of the processing unit includes a first value, which means that the binary data corresponding to the processing unit is the first value (such as 1). The adjacent processing unit of the processing unit refers to the processing unit adjacent to the processing unit.

[0149] Optionally, the isolated processing unit may be identified by identification data, and the identification data of the isolated processing unit may include a temporary label, and the row and column where the isolated processing unit is located.

[0150] For example, refer to Figure 5 For the second-type processing unit 511 in the processing unit array 510, its binary data is 1, and the binary data of the adjacent processing unit is 0. Therefore, the second-type processing unit 511 can be determined as an isolated processing unit, which is recorded as (6, 2, 3) (in order from left to right). For the first second-type processing unit in the second row of the processing unit array 520, its binary data is 1, and the binary data of the adjacent processing unit is 0. Therefore, this second-type processing unit is determined as an isolated processing unit, which is recorded as (8, 2, 5). Similarly, the first first-type processing unit in the first row of the processing unit array 510 is also an isolated processing unit, which is recorded as (6, 1, 1).

[0151] The row and column where an isolated processing unit is located may refer to its row and column in a large processing unit array that is fictitious and constructed based on m processing unit arrays, with the total input dimension of the m processing unit arrays as its input dimension. For example, if processing unit array 510 and processing unit array 520 are arranged in parallel and processing units are added in the gaps, the rows and columns where each processing unit is located can be determined.

[0152] It can be understood that the output of each processing unit in the above-mentioned m processing unit array may refer to outputs synchronized to the same clock cycle.

[0153] 2. Isolated processing unit merging: The above-mentioned label processing module is also used to merge the identification data of the isolated processing units corresponding to the m processing unit arrays according to the connectivity between each pixel in the input pixel vector, and obtain the merged identification data corresponding to the m processing unit arrays.

[0154] In the embodiment of the present application, the connectivity between pixels can be used to indicate whether the pixels are adjacent and connected. For example, if P1 and P2 in the input pixel vector are adjacent and both P1 and P2 are foreground pixels, then it can be determined that P1 and P2 are connected, that is, P1 and P2 are in the same connected domain. If P3 in the input pixel vector is connected to P2, then P3 is also connected to P1.

[0155] The merging of isolated processing units in the embodiment of the present application may refer to the merging of identification data corresponding to the isolated processing units.

[0156] In one example, the process of obtaining the merged identification data may be as follows:

[0157] 1) The label processing module is further used to construct a sub-processing unit array corresponding to the isolated point processing unit with the isolated processing unit as the vertex and the first row of the processing unit array as the bottom.

[0158] Optionally, the sub-processing unit array is constructed in a pyramid structure, so that the first type of processing unit corresponding to the isolated processing unit in the first row of the processing unit array can be determined, and then it can be directly determined that the foreground pixels processed by the first type of processing unit corresponding to the isolated processing unit in the first row of the processing unit array are in the same connected domain. For example, referring to Figure 5 , a sub-processing unit array 530 constructed with the second-type processing unit 511 as a vertex includes the first-type processing unit 512 and the first-type processing unit 513. The pixels processed by the first-type processing unit 512 are adjacent to the pixels processed by the first-type processing unit 513, and both are foreground pixels. Therefore, it can be determined that the pixels processed by the first-type processing unit 512 and the pixels processed by the first-type processing unit 513 are in the same connected domain, thereby obtaining the connected domain in the sub-pixel vector. Using the same method, the sub-processing unit array 550 in the processing unit array 510 and the sub-processing unit array 540 in the processing unit array 520 can also be obtained.

[0159] Thus, it can be seen that the identification data of the isolated processing unit can also be used to indicate the position and number of pixels in the connected domain to which it corresponds. For example, if the second type of processing unit 511 is an isolated processing unit and its identification data is (6, 2, 3), it can be seen that the pixels in the connected domain corresponding to the second type of processing unit 511 are two pixels starting from the third pixel in the input pixel vector, where column (3) can be used to indicate the position of the starting pixel in the connected domain, and row (2) can be used to indicate the number of pixels in the connected domain. For another example, (6, 1, 1) can be used to indicate that the first pixel in the input pixel vector belongs to a connected domain (corresponding to a sub-pixel vector), and (8, 2, 5) can be used to indicate that the fifth and sixth pixels in the input pixel vector belong to a connected domain (corresponding to a sub-pixel vector).

[0160] 2) For a first sub-processing unit array and a second sub-processing unit array corresponding to the m processing unit arrays, the label processing module is further configured to merge identification data of isolated processing units corresponding to the first sub-processing unit array with identification data of isolated processing units corresponding to the second sub-processing unit array if there are pixels in the first sub-processing unit array that are adjacent to pixels in the second sub-processing unit array, thereby obtaining merged identification data corresponding to the first sub-processing unit array and the second sub-processing unit array.

[0161] The first processing sub-unit array may refer to any processing sub-unit array corresponding to the m processing unit arrays, and the second sub-processing unit array may refer to any processing sub-unit array among the processing sub-unit arrays corresponding to the m processing unit arrays except the first processing sub-unit array.

[0162] Optionally, the existence of pixels in the first sub-processing unit array that are adjacent to pixels in the second sub-processing unit array means that among the pixels processed by the first sub-processing unit array, there are pixels that are connected to the pixels processed by the second sub-processing unit array. Figure 5 For the sub-processing unit array 530 in the processing unit array 510 and the sub-processing unit array 540 in the processing unit array 520, the vertex of the sub-processing unit array 530 is an isolated processing unit and is the last node in the second row of the processing unit array 510, and the vertex of the sub-processing unit array 540 is an isolated processing unit and is the first processing unit in the second row of the processing unit array 520. Then, there is connectivity between P3 and P4 in the input pixel vector, that is, the connected domain corresponding to the sub-processing unit array 530 and the connected domain corresponding to the sub-processing unit array 540 are actually the same connected domain, so the connected domain corresponding to the sub-processing unit array 530 and the connected domain corresponding to the sub-processing unit array 540 can be merged.

[0163] In an embodiment of the present application, merging representation data may be used to mark whether a connected domain in an input pixel vector needs to be merged. Optionally, the merging representation data may be determined based on the identification data of the first sub-processing unit array and the identification data of the second sub-processing unit array, the identification data with the smaller label value.

[0164] The identification data of the first sub-processing unit array is the identification data of the corresponding isolated processing unit, and the identification data of the second sub-processing unit array is the identification data of the corresponding isolated processing unit. The identification data with the smaller tag value of the included temporary tag can be determined as the combined identification data. For example, referring to Figure 5 , the identification data of the sub-processing unit array 530 is: (6, 2, 3), and the identification data of the sub-processing unit array 540 is: (8, 2, 5), then the merged identification data is: (6, 4, 3), that is, the second connected domain in the input pixel vector includes the four pixels P2-P5, and the label value of the final label of these four pixels can be 6.

[0165] Optionally, the merge identification data can also be used to indicate the positions of pixels included in the connected domain in the input pixel vector. For example, the above (3) can indicate that the starting pixel in the second connected domain is P2, and the above (4) indicates that the second connected domain includes 4 pixels.

[0166] It should be noted that if the label value of the temporary label included in one identification data is 1, and the label value of the temporary label included in the other identification data is not 1, the temporary label included in the other identification data can be determined as the temporary label included in the merged identification data. If the label value of the temporary label included in one identification data is 1, and the label value of the temporary label included in the other identification data is also 1, 1 can be determined as the label value of the temporary label included in the merged identification data to indicate that the connected domain corresponding to the merged identification data needs to be assigned a new label.

[0167] 3. Label merging: The label processing module is further configured to assign labels to the isolated processing units corresponding to the m processing unit arrays according to the merged identification data corresponding to the m processing unit arrays, and obtain the final labels corresponding to the pixels in the input pixel vector.

[0168] In the embodiment of the present application, label assignment to the isolated processing unit refers to label assignment to the connected domain in the input pixel vector, that is, label assignment to the pixels processed by the sub-processing unit array corresponding to the isolated processing unit.

[0169] In one example, the label processing module is further configured to assign a new label to the isolated processing unit when the temporary label corresponding to the isolated processing unit is a first value, to serve as a final label for pixels processed by the sub-processing unit array corresponding to the isolated processing unit.

[0170] For example, if the temporary label of each pixel in the current sub-pixel vector is 1 (that is, the corresponding pixels in the previous row are all background pixels), then there is only one isolated processing unit in the corresponding processing unit array, that is, the vertex of the processing unit array. The pixels in the current sub-pixel vector belong to the same connected domain, and the connected domain is a new connected domain. In this case, the label processing module can assign a new label to the connected domain as the final label of the pixels included in the connected domain.

[0171] Optionally, during the label assignment process, label management can be used to reduce the maximum number of labels required for each input pixel vector. By assigning labels to isolated processing nodes, the label assignment granularity can be converted from pixels to regions, thereby effectively reducing the number of labels.

[0172] Exemplarily, the connected domain marking circuit may also correspond to two label lists, the first label list being used to record used labels and unused labels, and the second label list being used to record new labels. If a conflicting label is released by the first label list, it can be added to the second label list to provide a new label for the subsequent label assignment process. For example, when a label does not appear until a row of pixels is processed, this means that the next row of pixels will not be connected to the area marked by the label. Therefore, this label will be released by the first label list and added to the second label list for future use. This will keep the quantization bit width of the label smaller, reduce the complexity of the label, and reduce the time for label comparison and merging, thereby improving the efficiency of connected domain marking.

[0173] When a new label is assigned to an isolated processing unit(s) and no existing label is inherited, the second label list is updated in the next clock cycle.

[0174] In one example, the second tag list can be implemented as a FIFO (First Input First Output) memory. For example, if the character length of the tag is 8 bits, the connected domain labeling circuit has 254 tags available (0 can be used to represent background pixels, and 1 can be used to indicate that a tag needs to be assigned). The use of each new tag will cause an element in the FIFO memory to be popped out. On the other hand, when a tag (such as a conflicting tag) is released from use, it will be pushed into the FIFO memory after the first tag list is refreshed.

[0175] Optionally, when there are sufficient labels (such as when processing small images with fewer pixels), a new label can be assigned to each foreground pixel in the input pixel vector, and then the new labels are merged according to connectivity to obtain the final label corresponding to each pixel in the input pixel vector.

[0176] In one example, the label processing module is also used to determine the temporary label included in the merged identification data of the isolated processing unit as the final label of the pixel processed by the sub-processing unit array corresponding to the isolated processing unit when the label value of the temporary label corresponding to the isolated processing unit is greater than the first numerical value.

[0177] Optionally, if the label value of the temporary label corresponding to the isolated processing unit is greater than the first value, it means that the connected domain corresponding to the sub-processing unit array corresponding to the isolated processing unit is connected to the connected domain in the previous row of pixels. Then, the pixels processed by the sub-processing unit array corresponding to the isolated processing unit can inherit the label corresponding to the connected domain in the previous row of pixels.

[0178] For two isolated processing units that need to merge identification data, the connected domains corresponding to the two isolated processing units are connected to each other. Then, the pixels corresponding to the two isolated processing units can inherit the labels corresponding to the connected domain in the previous row of pixels as their respective final labels.

[0179] For example, reference Figure 5 , the final labels of the pixels processed by the sub-processing unit array 530 and the sub-processing unit array 540 may inherit the label value 6. The label with a label value of 8 may be recorded as a conflicting label of the label with a label value of 6.

[0180] In one example, for an isolated processing unit that has not undergone identification data merging, the label processing module is further configured to, if the label value of the temporary label corresponding to the isolated processing unit is greater than a first value, determine the temporary label included in the identification data of the isolated processing unit as the final label for the pixels included in the sub-processing unit array corresponding to the isolated processing unit. Alternatively, if the label value of the temporary label corresponding to the isolated processing unit is equal to the first value, the label processing module is further configured to assign a new label to the pixels processed by the sub-processing unit array corresponding to the isolated processing unit as the final label.

[0181] For example, reference Figure 5For the sub-processing unit array 550, the final table of pixels processed by it can inherit the label value 6. Therefore, the final labels of each pixel in the input pixel vector are: 6066 6600. Among them, for the label with a label value of 6, the labels with label values ​​of 8 and 9 are both conflicting labels. The labels with label values ​​of 8 and 9 need to be replaced by the label with a label value of 6 and then adjusted to free labels for use in subsequent processes.

[0182] Optionally, after all pixels in the input image have been assigned a label, the labels of the pixels preceding the current input pixel vector may be uniformly replaced according to the conflicting label list, thereby determining each connected domain in the input image. Alternatively, when the current input pixel vector is processed, i.e., according to the conflicting label list, the labels of the pixels preceding the current input pixel vector may be replaced to release the conflicting labels, thereby enabling the reuse of label resources and ensuring sufficient label resources. This is not a limitation in the present embodiment of the application.

[0183] In one example, the label processing module is also used to construct a conflict label list based on the merge identification data corresponding to the m processing unit arrays and the conflict label sequence corresponding to the first type of processing unit in the processing unit array; wherein the conflict label list is used to merge labels after obtaining the final labels corresponding to each pixel in the input image, and obtain the merged labels corresponding to each pixel in the input image, and the pixels in the connected domain in the input image have the same merged label.

[0184] The conflicting label list is used to record conflicting labels. For example, if labels with values ​​6, 8, and 9 conflict with each other, then all labels with values ​​6, 8, and 9 can be replaced with the label with the smallest value among the three: label 6. This label with a value of 6 is called the merged label. In other words, for all conflicting labels in the same group, the label with the smallest value is the merged label.

[0185] Optionally, if the final label of a pixel does not have a conflicting label, the final label is retained for the pixel. After the label merging, the pixels in the input image can be divided based on the merged label or the final label to obtain the connected domain corresponding to the input image.

[0186] 4. Label storage: The connected domain labeling circuit in the embodiment of the present application also corresponds to a memory, and the label processing module can send the label of each pixel in the input image to the memory to complete the storage of the label of each pixel.

[0187] The specific process of obtaining the input pixel vector is described below.

[0188] In some embodiments, reference Figure 2The connected domain labeling circuit 200 further includes a pixel vector acquisition module. The pixel vector acquisition module is used to generate an input pixel vector.

[0189] Optionally, the pixel vector acquisition module is used to, when acquiring an input image, divide the input image row by row based on the total width of the m processing unit arrays to obtain a plurality of input pixel vectors.

[0190] The input image may be an image uploaded to the connected domain marking circuit. The embodiment of the present application does not limit the source of the input image. For example, it may refer to an image generated by an image acquisition device, such as a photo, video frame, drawing, etc.

[0191] After the connected domain labeling circuit obtains the input image through the pixel vector obtaining module, the pixel vector obtaining module divides the input image to obtain a plurality of input pixel vectors corresponding to the input image.

[0192] For example, for four processing unit arrays with four input dimensions, the pixel vector acquisition module can divide the input image row by row in units of 16 pixels. If the width of the input image is 32, each row of pixels can be divided into two input pixel vectors.

[0193] Optionally, when the width of the input pixel vector is less than the total width of the m processing unit arrays, the input pixel vector is padded with the second value. For example, based on the above embodiment, if the width of the input image is 30, each row of pixels can be divided into two input pixel vectors, and the last two elements of the second input pixel vector corresponding to each row are 0 (i.e., the pixel value is 0).

[0194] Optionally, the pixel vector acquisition module is used to, when acquiring the original input pixel vector, divide the original input pixel vector into units of a total width of the m processing unit arrays to obtain multiple input pixel vectors.

[0195] The raw input pixel vector can refer to a specific input pixel vector in the input image. For example, a high-speed camera can transmit multiple pixels in parallel per clock cycle, and these multiple pixels can be combined into a single raw input pixel vector. For example, in scenarios involving the measurement of high-speed moving targets, vision systems are often paired with high-speed cameras capable of parallel data transmission. A high-speed camera operating at full capacity can transmit eight pixels in parallel per clock cycle, providing 64 bits of data.

[0196] Optionally, if the width of the original input pixel vector is the same as the total input dimension of the m processing unit arrays, the original input pixel vector does not need to be divided; if the width of the original input pixel vector is greater than the total input dimension of the m processing unit arrays, the original input pixel vector is divided based on the total width of the m processing unit arrays; if the width of the original input pixel vector is less than the total width of the m processing unit arrays, the original input pixel vector is divided based on the total width of the m processing unit arrays, and the end of the input pixel vector with missing elements is padded with the second value.

[0197] In some embodiments, the connected domain labeling circuit provided in the embodiments of the present application can achieve CCL acceleration, which can be applied to any scenario requiring CCL, such as foreground and background separation, image segmentation, target recognition, OCR (Optical Character Recognition) recognition, and other scenarios.

[0198] For example, in an OCR recognition scenario based on deep learning, the connected domain marking circuit provided in the embodiment of the present application can be used to obtain the text area in the image to be recognized, as well as the coordinates, rotation angle and other information of the text area, to provide a basis for subsequent text recognition.

[0199] For another example, the connected domain labeling circuit provided in the embodiment of the present application can be applied to an image reconstruction scenario. In this scenario, the connected domain labeling circuit can be implemented as an FPGA chip or directly used in a chip. Figure 6 The scene includes a camera 601 for acquiring input images, an FPGA chip 602 for implementing CCL, and a display 603 for displaying the reconstructed image. Optionally, to reduce the operating pressure of the FPGA chip 602, a separate threshold segmentation device is provided to convert the input image into a binary image. The FPGA chip 602 only needs to process the binary image.

[0200] The FPGA chip 602 can be connected to the camera 601 in an embedded manner. The FPGA chip 602 can process pixels synchronously with pixel transmission and transmit the connected domain labeling results of the input image to the display 603 for display.

[0201] The following are examples of the method of the present application. For details not described in detail in the examples of the method of the present application, please refer to the above examples of the numerical sorting module.

[0202] Please refer to Figure 7, which shows a flowchart of a connected domain labeling method based on a connected domain labeling circuit according to one embodiment of the present application. This method is applied to the connected domain labeling circuit described above. The connected domain labeling circuit includes an array of m processing units and a label processing module. The difference between the number of processing units corresponding to the nth row and the number of processing units corresponding to the n-1th row in the processing unit array is 1, and m and n are integers greater than 1. The method may include at least one of the following steps 701 to 703.

[0203] In step 701, m processing unit arrays obtain an input pixel vector, which includes m sub-pixel vectors. The width corresponding to the i-th sub-pixel vector in the m sub-pixel vectors is the same as the width of the i-th processing unit array in the m processing unit arrays. The input pixel vector includes multiple consecutive pixels in the same row of the input image, and i is a positive integer.

[0204] In some embodiments, the connected domain marking circuit further includes a pixel vector acquisition module; the pixel vector acquisition module is configured to, when acquiring an input image, divide the input image row by row based on the total width of the m processing unit arrays to obtain a plurality of input pixel vectors;

[0205] Alternatively, the pixel vector acquisition module is used to divide the original input pixel vector into units of the total width of the m processing unit arrays to obtain multiple input pixel vectors when acquiring the original input pixel vector; wherein, when the width of the input pixel vector is less than the total width of the m processing unit arrays, the input pixel vector is padded with a second value.

[0206] In some embodiments, when each row of the input image corresponds to multiple input pixel vectors, m processing unit arrays process the multiple input pixel vectors in a pipeline manner; wherein the m processing unit arrays obtain one input pixel vector per clock cycle, and when the xth input pixel vector is processed by the first row processing unit in the current clock cycle, the x+1th input pixel vector is input to the first row processing unit in the next clock cycle of the current clock cycle, where x is a positive integer.

[0207] In step 702 , the i-th processing unit array performs label assignment on the i-th sub-pixel vector to obtain a temporary label corresponding to each pixel in the i-th sub-pixel vector.

[0208] In some embodiments, m processing unit arrays operate in parallel to assign labels to the m sub-pixel vectors respectively, and obtain temporary labels corresponding to respective pixels in the m sub-pixel vectors.

[0209] In some embodiments, the first row of the processing unit array includes p first-type processing units, and the nth row includes p-n+1 second-type processing units, where p is an integer greater than 1; the p first-type processing units in the ith processing unit array are configured to perform threshold segmentation on each pixel in the ith sub-pixel vector to obtain binary data corresponding to each pixel in the ith sub-pixel vector, where the binary data is a first value or a second value, and the p first-type processing units operate in parallel;

[0210] The p first-type processing units in the i-th processing unit array are further configured to assign a label to each pixel in the i-th sub-pixel vector based on the binary data and a reference label corresponding to each pixel in the i-th sub-pixel vector, to obtain a temporary label corresponding to each pixel in the i-th sub-pixel vector, where the reference label is associated with a neighboring pixel of the pixel, and a distance between the neighboring pixels of the pixel and the pixel is less than a distance threshold;

[0211] Each second-type processing unit in the i-th processing unit array is used to transmit the temporary label and binary data corresponding to each pixel in the i-th sub-pixel vector, and obtain the output corresponding to each second-type processing unit, which includes the temporary label and the binary data.

[0212] In some embodiments, the qth first-category processing unit among the p first-category processing units is configured to, when the binary data corresponding to the qth pixel in the i-th sub-pixel vector is the first value, obtain, based on the reference label corresponding to the qth pixel, a valid label number corresponding to the qth pixel, where the valid label number refers to the number of valid labels, the label value of the valid label being greater than the first value, and q is a positive integer not greater than p;

[0213] The qth first-category processing unit is further used to determine the first numerical value as the label value of the temporary label of the qth pixel when the number of valid labels is equal to 0; or, when the number of valid labels is equal to 1, determine the valid label as the temporary label of the qth pixel; or, when the number of valid labels is greater than 1, determine the valid label with the smallest label value as the temporary label of the qth pixel.

[0214] In some embodiments, when the qth pixel is the first pixel in the input image, the qth pixel does not correspond to a reference label;

[0215] Alternatively, when the qth pixel is not the first pixel in the input image and is located in the first row of the input image, the qth pixel corresponds to a reference label, and the label value of the reference label is the first value or the second value, and the first value and the second value are less than or equal to 1;

[0216] Alternatively, when the qth pixel is not located in the first row and the first column of the input image, the qth pixel corresponds to at least two reference labels, and the reference labels are determined based on the final labels assigned to the adjacent pixels;

[0217] Alternatively, when the qth pixel is not located in the first row of the input image but in the first column of the input image, the qth pixel corresponds to at least one reference label, and the reference label is determined based on the final label assigned to the adjacent pixels.

[0218] In some embodiments, the qth first-type processing unit is further configured to, when the number of valid tags is greater than 1, determine conflicting tags from the valid tags, where equivalent tags exist for the conflicting tags;

[0219] The conflicting tags are sorted to obtain a conflicting tag sequence; the conflicting tag sequence is used to update the conflicting tag list, and the conflicting tag list is used to count the conflicting tags.

[0220] In some embodiments, the first row of the second type processing unit in the i-th processing unit array is used to transfer the outputs of the p first type processing units in the i-th processing unit array to obtain the output of the first row of the second type processing unit;

[0221] The second-category processing unit in the t-th row of the i-th processing unit array is used to transfer the output of the second-category processing unit in the t-1-th row of the i-th processing unit array to obtain the output of the second-category processing unit in the t-th row, where t is a positive integer less than n; wherein the second-category processing unit takes the output of two input processing units as input and takes the output of the two input processing units with the smaller label value as output, and the input processing unit refers to the processing unit in the row above the row where the second-category processing unit is located and is connected to the second-category processing unit.

[0222] In some embodiments, the qth first-type processing unit among the p first-type processing units is further configured to, when a pixel value of the qth pixel is greater than a first threshold, determine the first value as binary data corresponding to the qth pixel;

[0223] Alternatively, the qth first-type processing unit among the p first-type processing units is further configured to, when the pixel value of the qth pixel is less than or equal to the first threshold, determine the second value as binary data corresponding to the qth pixel.

[0224] In some embodiments, the m processing unit arrays are also used to synchronize the outputs of the input pixel vector in different clock cycles to the last clock cycle corresponding to the input pixel vector, so as to obtain the outputs of the m processing unit arrays for the input pixel vector in the same clock cycle; wherein the outputs in the same clock cycle are transmitted to the label processing module.

[0225] In some embodiments, the processing time of the processing unit is one clock cycle.

[0226] In step 703, the label processing module merges the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain the final labels corresponding to each pixel in the input pixel vector; wherein the final labels are used to obtain the connected domain in the input pixel vector, and each pixel belonging to the same connected domain in the input pixel vector has the same final label, and different connected domains in the input pixel vector correspond to different final labels.

[0227] In some embodiments, the label processing module is further configured to determine, based on outputs of each processing unit in the m processing unit arrays, isolated processing units corresponding to the m processing unit arrays, respectively. The isolated processing units are processing units that meet an isolation condition. The identification data of the isolated processing units includes a temporary label, a row and a column where the isolated processing unit is located, and the output of the processing unit includes binary data and a final label.

[0228] The label processing module is further configured to merge identification data of the isolated processing units corresponding to the m processing unit arrays according to connectivity between pixels in the input pixel vector, to obtain merged identification data corresponding to the m processing unit arrays, the merged identification data being used to indicate positions of pixels included in the connected domain in the input pixel vector;

[0229] The label processing module is further used to assign labels to the isolated processing units corresponding to the m processing unit arrays according to the combined identification data corresponding to the m processing unit arrays, and obtain the final labels corresponding to each pixel in the input pixel vector.

[0230] In some embodiments, for each processing unit in the array of m processing units, the label processing module is also used to determine the processing unit as an isolated processing unit when the output of the processing unit includes a first value and the output of the adjacent processing unit corresponding to the processing unit in the row where the processing unit is located includes a second value.

[0231] In some embodiments, the tag processing module is further configured to construct a sub-processing unit array corresponding to the isolated processing unit with the isolated processing unit as the vertex and the first row of the processing unit array as the bottom;

[0232] For the first sub-processing unit array and the second sub-processing unit array corresponding to the m processing unit arrays, the label processing module is further used to merge the identification data of the isolated processing unit corresponding to the first sub-processing unit array with the identification data of the isolated processing unit corresponding to the second sub-processing unit array if there are pixels in the first sub-processing unit array adjacent to the pixels in the second sub-processing unit array, so as to obtain merged identification data corresponding to the first sub-processing unit array and the second sub-processing unit array; wherein the merged representation data is determined based on the identification data of the first sub-processing unit array and the identification data having the smaller label value among the identification data of the second sub-processing unit array.

[0233] In some embodiments, the label processing module is further configured to, when the temporary label corresponding to the isolated processing unit is the first value, assign a new label to the isolated processing unit to serve as a final label for pixels processed by the sub-processing unit array corresponding to the isolated processing unit;

[0234] Alternatively, the label processing module is also used to determine the temporary label included in the merged identification data of the isolated processing unit as the final label of the pixel processed by the sub-processing unit array corresponding to the isolated processing unit when the label value of the temporary label corresponding to the isolated processing unit is greater than the first numerical value.

[0235] In some embodiments, the label processing module is also used to construct a conflict label list based on the merge identification data corresponding to the m processing unit arrays and the conflict label sequence corresponding to the first type of processing unit in the processing unit array; wherein the conflict label list is used to merge labels after obtaining the final labels corresponding to each pixel in the input image, and obtain the merged labels corresponding to each pixel in the input image, and the pixels in the connected domain in the input image have the same merged label.

[0236] To sum up, the technical solution provided by the embodiment of the present application adopts multiple processing unit arrays to assign labels to each sub-pixel unit in the same input pixel vector, obtain temporary labels corresponding to each pixel in the input pixel vector, and then merge the temporary labels corresponding to each pixel through the label processing module to obtain the final labels corresponding to each pixel. Compared with processing the input pixel vector through a processing unit array, the single processing unit array has many levels and high complexity. The present application is equivalent to splitting a high-input-dimensional processing unit array into multiple low-input-dimensional small processing unit arrays. Constrained by the architecture of the processing unit array, the small processing unit array has fewer levels (that is, fewer processing units are required) and lower complexity, which can effectively reduce the processing delay of the input pixel vector and thereby improve the labeling efficiency of the connected domain.

[0237] An exemplary embodiment of the present application further provides a chip, which includes the connected domain marking circuit introduced in the above embodiment.

[0238] Optionally, the chip may refer to an FPGA chip, an ASIC chip, etc. implemented based on a connected domain marking circuit, which is not limited in the embodiments of the present application.

[0239] An exemplary embodiment of the present application further provides a processor, which includes the connected domain marking circuit introduced in the above embodiment.

[0240] Optionally, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), or PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor may also include an AI processor for processing computing operations related to machine learning.

[0241] The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory is used to store a computer program, which is configured to be executed by one or more processors to implement the above-mentioned connected domain labeling method.

[0242] An exemplary embodiment of the present application further provides a computer device, the computer device including a processor, the processor including the connected domain marking circuit introduced in the above embodiment.

[0243] Optionally, the processor is an AI processor, or other processor that requires the use of connected domain labels, which is not limited in this application.

[0244] Optionally, the computer device can be a server, or a terminal device such as a mobile phone, tablet computer, vehicle-mounted terminal, wearable device, smart home device, or any device that is applied to a processor, such as a robot or base station, and this application does not limit this.

[0245] It should be noted that this application can display a prompt interface, pop-up window or output voice prompt information before collecting the user's relevant data and during the process of collecting the user's relevant data. The prompt interface, pop-up window or voice prompt information is used to prompt the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining the user's relevant data after obtaining the user's confirmation operation on the prompt interface or pop-up window. Otherwise (that is, when the user's confirmation operation on the prompt interface or pop-up window is not obtained), the relevant steps of obtaining the user's relevant data are terminated, that is, the user's relevant data is not obtained. In other words, all user data collected by this application (including numerical vectors in vector registers) are processed strictly in accordance with the requirements of relevant national laws and regulations, and the informed consent or separate consent of the personal information subject is obtained only with the user's consent and authorization. Subsequent data use and processing behaviors are carried out within the scope of authorization of laws and regulations and the personal information subject, and the collection, use and processing of relevant user data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0246] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application do not limit this.

[0247] The above description is merely an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A connected domain marking circuit, characterized in that: The connected domain marking circuit includes an array of m processing units and a label processing module, wherein the difference between the number of processing units corresponding to the nth row and the number of processing units corresponding to the n-1th row in the processing unit array is 1, and m and n are integers greater than 1; The m processing unit arrays are used to obtain an input pixel vector, the input pixel vector including m sub-pixel vectors, the width corresponding to the i-th sub-pixel vector in the m sub-pixel vectors is the same as the width of the i-th processing unit array in the m processing unit arrays, the input pixel vector includes multiple consecutive pixels in the same row of the input image, and i is a positive integer; The i-th processing unit array is used to assign a label to the i-th sub-pixel vector to obtain a temporary label corresponding to each pixel in the i-th sub-pixel vector; The label processing module is used to merge the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain the final label corresponding to each pixel in the input pixel vector; wherein the final label is used to obtain the connected domain in the input pixel vector, each pixel belonging to the same connected domain in the input pixel vector has the same final label, and different connected domains in the input pixel vector correspond to different final labels.

2. The connected domain marking circuit according to claim 1, characterized in that: The m processing unit arrays run in parallel to respectively assign labels to the m sub-pixel vectors to obtain temporary labels corresponding to respective pixels in the m sub-pixel vectors.

3. The connected domain marking circuit according to claim 1, characterized in that: The first row of the processing unit array includes p first-type processing units, and the nth row includes p-n+1 second-type processing units, where p is an integer greater than 1; The p first-type processing units in the i-th processing unit array are configured to perform threshold segmentation on each pixel in the i-th sub-pixel vector to obtain binary data corresponding to each pixel in the i-th sub-pixel vector, where the binary data is a first value or a second value, and the p first-type processing units operate in parallel; The p first-type processing units in the i-th processing unit array are further configured to assign labels to respective pixels in the i-th sub-pixel vector based on the binary data and reference labels corresponding to respective pixels in the i-th sub-pixel vector, to obtain temporary labels corresponding to respective pixels in the i-th sub-pixel vector, wherein the reference labels are associated with adjacent pixels of the pixel, and a distance between the adjacent pixels of the pixel and the pixel is less than a distance threshold; Each second-type processing unit in the i-th processing unit array is used to transmit the temporary label and binary data corresponding to each pixel in the i-th sub-pixel vector, and obtain the output corresponding to each second-type processing unit, wherein the output includes the temporary label and the binary data.

4. The connected domain marking circuit according to claim 3, characterized in that: The qth first-category processing unit among the p first-category processing units is configured to, when the binary data corresponding to the qth pixel in the i-th sub-pixel vector is a first value, obtain, based on the reference label corresponding to the qth pixel, a valid label number corresponding to the qth pixel, where the valid label number refers to the number of valid labels, the label value of the valid label being greater than the first value, and q is a positive integer not greater than p; The qth first-category processing unit is further used to, when the number of valid labels is equal to 0, determine the first numerical value as the label value of the temporary label of the qth pixel; or, when the number of valid labels is equal to 1, determine the valid label as the temporary label of the qth pixel; or, when the number of valid labels is greater than 1, determine the valid label with the smallest label value as the temporary label of the qth pixel.

5. The connected domain marking circuit according to claim 4, characterized in that: In a case where the qth pixel is the first pixel in the input image, the qth pixel does not correspond to a reference label; or, When the qth pixel is not the first pixel in the input image and is located in the first row of the input image, the qth pixel corresponds to a reference label, and the label value of the reference label is the first value or the second value, and the first value and the second value are less than or equal to 1; or, When the qth pixel is not located in the first row and the first column of the input image, the qth pixel corresponds to at least two reference labels, and the reference labels are determined based on the final labels assigned to the adjacent pixels; or, When the qth pixel is not located in the first row of the input image but in the first column of the input image, the qth pixel corresponds to at least one reference label, and the reference label is determined based on the final label assigned to the adjacent pixel.

6. The connected domain marking circuit according to claim 4, characterized in that: The qth first-type processing unit is further configured to, when the number of valid tags is greater than 1, determine a conflicting tag from the valid tags, where an equivalent tag exists for the conflicting tag; Sorting the conflicting labels to obtain a conflicting label sequence; The conflicting label sequence is used to update a conflicting label list, and the conflicting label list is used to count the conflicting labels.

7. The connected domain marking circuit according to claim 3, characterized in that: The second-type processing units in the first row of the i-th processing unit array are configured to transfer outputs of the p first-type processing units in the i-th processing unit array to obtain outputs of the second-type processing units in the first row; The second-category processing unit in the t-th row of the ith processing unit array is configured to transfer the output of the second-category processing unit in the t-1-th row of the ith processing unit array to obtain the output of the second-category processing unit in the t-th row, where t is a positive integer less than n; Among them, the second type of processing unit takes the outputs of two input processing units as input and takes the output of the two input processing units with the smaller label value as output, and the input processing unit refers to the processing unit in the row above the row where the second type of processing unit is located and is connected to the second type of processing unit.

8. The connected domain marking circuit according to claim 1, characterized in that: The label processing module is further configured to determine, based on outputs of each processing unit in the m processing unit arrays, isolated processing units corresponding to the m processing unit arrays, wherein the isolated processing units refer to processing units that meet an isolation condition, and the identification data of the isolated processing units includes the temporary label, the row and column where the isolated processing unit is located, and the output of the processing unit includes binary data and the final label; The label processing module is further configured to merge identification data of isolated processing units corresponding to the m processing unit arrays according to connectivity between pixels in the input pixel vector to obtain merged identification data corresponding to the m processing unit arrays, wherein the merged identification data is used to indicate positions of pixels included in the connected domain in the input pixel vector; The label processing module is further configured to assign labels to the isolated processing units corresponding to the m processing unit arrays respectively according to the merged identification data corresponding to the m processing unit arrays respectively, to obtain final labels corresponding to the respective pixels in the input pixel vector.

9. The connected domain marking circuit according to claim 8, characterized in that: For each processing unit in the m processing unit array, the label processing module is also used to determine the processing unit as the isolated processing unit when the output of the processing unit includes the first numerical value corresponding to the binary data and the output of the adjacent processing unit corresponding to the processing unit in the row where the processing unit is located includes the second numerical value corresponding to the binary data.

10. The connected domain marking circuit according to claim 8, characterized in that: The tag processing module is further configured to construct a sub-processing unit array corresponding to the isolated processing unit with the isolated processing unit as the vertex and the first row of the processing unit array as the bottom; For a first sub-processing unit array and a second sub-processing unit array corresponding to the m processing unit arrays, the label processing module is further configured to, if there are pixels in the first sub-processing unit array that are adjacent to pixels in the second sub-processing unit array, merge identification data of isolated processing units corresponding to the first sub-processing unit array with identification data of isolated processing units corresponding to the second sub-processing unit array to obtain merged identification data corresponding to the first sub-processing unit array and the second sub-processing unit array; The combined identification data is determined based on the identification data of the first sub-processing unit array and the identification data of the second sub-processing unit array, which has a smaller tag value.

11. The connected domain marking circuit according to claim 8, characterized in that: The label processing module is further configured to, when the temporary label corresponding to the isolated processing unit is the first value corresponding to the binary data, assign a new label to the isolated processing unit to serve as a final label for pixels processed by the sub-processing unit array corresponding to the isolated processing unit; or, The label processing module is also used to determine the temporary label included in the merged identification data of the isolated processing unit as the final label of the pixel processed by the sub-processing unit array corresponding to the isolated processing unit when the label value of the temporary label corresponding to the isolated processing unit is greater than the first numerical value corresponding to the binary data.

12. The connected domain marking circuit according to claim 8, characterized in that: The label processing module is further configured to construct a conflict label list according to the merge identification data corresponding to the m processing unit arrays and the conflict label sequences corresponding to the first type of processing units in the processing unit array; Among them, the conflicting label list is used to merge labels after obtaining the final labels corresponding to each pixel in the input image, so as to obtain the merged labels corresponding to each pixel in the input image, and the pixels in the connected domain in the input image have the same merged label.

13. The connected domain marking circuit according to claim 3, characterized in that: The qth first-type processing unit among the p first-type processing units is further configured to, when a pixel value of the qth pixel in the i-th sub-pixel vector is greater than a first threshold, determine the first value as binary data corresponding to the qth pixel; or, The qth first-type processing unit among the p first-type processing units is also used to determine the second value as the binary data corresponding to the qth pixel when the pixel value of the qth pixel in the i-th sub-pixel vector is less than or equal to the first threshold.

14. The connected domain marking circuit according to claim 1, characterized in that: The connected domain marking circuit further includes a pixel vector acquisition module; The pixel vector acquisition module is configured to, when acquiring the input image, divide the input image row by row based on the total width of the m processing unit arrays to obtain a plurality of input pixel vectors; or, The pixel vector acquisition module is configured to, when acquiring an original input pixel vector, divide the original input pixel vector into units of the total width of the m processing unit arrays to obtain a plurality of the input pixel vectors; Wherein, when the width of the input pixel vector is smaller than the total width of the m processing unit arrays, the input pixel vector is padded with a second value.

15. The connected domain marking circuit according to claim 1, characterized in that: In a case where each row of the input image corresponds to a plurality of input pixel vectors, the m processing unit arrays process the plurality of input pixel vectors in a pipeline manner; Among them, the m processing unit arrays obtain an input pixel vector in each clock cycle. When the x-th input pixel vector is processed by the first row processing unit in the current clock cycle, the x+1-th input pixel vector is input to the first row processing unit in the next clock cycle of the current clock cycle, and x is a positive integer.

16. The connected domain marking circuit according to claim 15, characterized in that: The m processing unit arrays are further configured to synchronize outputs of the input pixel vector in different clock cycles to the last clock cycle corresponding to the input pixel vector, thereby obtaining outputs of the m processing unit arrays for the input pixel vector in the same clock cycle; The outputs in the same clock cycle are transmitted to the tag processing module.

17. The connected domain marking circuit according to claim 15, characterized in that: The processing time of the processing unit is one clock cycle.

18. A chip, characterized in that: The chip includes the connected domain marking circuit according to any one of claims 1 to 17.

19. A processor, characterized in that: The processor includes the connected domain marking circuit according to any one of claims 1 to 17.

20. A connected domain marking method based on a connected domain marking circuit, characterized in that: The connected domain marking circuit includes an array of m processing units and a label processing module, wherein the difference between the number of processing units corresponding to the nth row and the number of processing units corresponding to the n-1th row in the processing unit array is 1, and m and n are integers greater than 1; the method includes: The m processing unit arrays obtain an input pixel vector, the input pixel vector including m sub-pixel vectors, the width corresponding to the i-th sub-pixel vector in the m sub-pixel vectors is the same as the width of the i-th processing unit array in the m processing unit arrays, the input pixel vector includes multiple consecutive pixels in the same row of the input image, and i is a positive integer; The i-th processing unit array performs label assignment on the i-th sub-pixel vector to obtain a temporary label corresponding to each pixel in the i-th sub-pixel vector; The label processing module merges the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain a final label corresponding to each pixel in the input pixel vector; wherein the final label is used to obtain a connected domain in the input pixel vector, each pixel belonging to the same connected domain in the input pixel vector has the same final label, and different connected domains in the input pixel vector correspond to different final labels.

Images