Connected domain marking circuit, chip, processor and method

Abstract

The invention discloses a connected domain marking circuit, a chip, a processor and a method, and relates to the technical field of chips. The connected domain marking circuit comprises m processing unit arrays and a label processing module, the m processing unit arrays are used for acquiring an input pixel vector, and the width corresponding to the ith sub-pixel vector in the input pixel vector is the same as the width of the ith processing unit array in the m processing unit arrays; the ith processing unit array is used for carrying out label distribution on the ith sub-pixel vector to obtain temporary labels respectively corresponding to each pixel in the ith sub-pixel vector; the label processing module is used for merging the temporary labels corresponding to the pixels in the m sub-pixel vectors to obtain final labels corresponding to the pixels in the input pixel vector, and the final labels are used for obtaining connected domains in the input pixel vector. And the marking efficiency of the connected domain is improved.

Description

Technical Field

[0001] The present application relates to the field of chip technology, and in particular to a connected domain marking circuit, chip, processor and method. Background Art

[0002] CCL (Connected Component Labelling) is a traditional CV (Computer Vision) algorithm, which is widely used in the field of image processing, such as foreground and background separation, image segmentation, object recognition, OCR (Optical Character Recognition) recognition, etc.

[0003] In the related art, the CCL algorithm, which is essentially serial in nature, is fundamentally parallelized by adopting a pyramid-structured processing unit array, thereby realizing a pixel-level parallel CCL method. However, in actual applications, the processing delay of a single high-input-dimensional processing unit array is relatively high, and its corresponding processing complexity increases rapidly with the increase of parallel input pixels, thus affecting the overall performance. Summary of the invention

[0004] The embodiment of the present application provides a connected domain marking circuit, chip, processor and method. The technical solution is as follows:

[0005] According to one aspect of an embodiment of the present application, a connected domain marking circuit is provided, the connected domain marking circuit comprising an m processing unit array and a label processing module, the difference between the number of processing units corresponding to the nth row and the number of processing units corresponding to the n-1th row in the processing unit array is 1, and m and n are integers greater than 1;

[0006] The m processing unit arrays are used to obtain an input pixel vector, the input pixel vector includes m sub-pixel vectors, the width corresponding to the i-th sub-pixel vector in the m sub-pixel vectors is the same as the width of the i-th processing unit array in the m processing unit arrays, the input pixel vector includes a plurality of continuous pixels in the same row of the input image, and i is a positive integer;

[0007] The i-th processing unit array is used to assign labels to the i-th sub-pixel vector to obtain temporary labels corresponding to each pixel in the i-th sub-pixel vector;

[0008] The label processing module is used to merge the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain the final labels corresponding to each pixel in the input pixel vector; wherein the final label is used to obtain the connected domain in the input pixel vector, and each pixel belonging to the same connected domain in the input pixel vector has the same final label, and different connected domains in the input pixel vector correspond to different final labels.

[0009] According to one aspect of an embodiment of the present application, a chip is provided, wherein the processor includes the connected domain marking circuit as described above.

[0010] According to one aspect of an embodiment of the present application, a processor is provided, wherein the processor includes the connected domain marking circuit as described above.

[0011] According to one aspect of an embodiment of the present application, a computer device is provided. The computer device includes a processor. The processor includes the connected domain marking circuit as described above.

[0012] According to one aspect of an embodiment of the present application, a connected domain marking method based on a connected domain marking circuit is provided, wherein the connected domain marking circuit includes an m processing unit array and a label processing module, wherein the difference between the number of processing units corresponding to the nth row and the number of processing units corresponding to the n-1th row in the processing unit array is 1, and m and n are integers greater than 1; the method includes:

[0013] The m processing unit arrays acquire an input pixel vector, the input pixel vector includes m sub-pixel vectors, the width corresponding to the i-th sub-pixel vector in the m sub-pixel vectors is the same as the width of the i-th processing unit array in the m processing unit arrays, the input pixel vector includes a plurality of continuous pixels in the same row of the input image, and i is a positive integer;

[0014] The i-th processing unit array performs label assignment on the i-th sub-pixel vector to obtain temporary labels corresponding to each pixel in the i-th sub-pixel vector;

[0015] The label processing module merges the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain the final labels corresponding to each pixel in the input pixel vector; wherein the final label is used to obtain the connected domain in the input pixel vector, and each pixel belonging to the same connected domain in the input pixel vector has the same final label, and different connected domains in the input pixel vector correspond to different final labels.

[0016] The technical solution provided in the embodiments of the present application can bring the following beneficial effects:

[0017] By adopting multiple processing unit arrays, labels are assigned to each sub-pixel unit in the same input pixel vector, and temporary labels corresponding to each pixel in the input pixel vector are obtained. The temporary labels corresponding to each pixel are then merged through a label processing module to obtain final labels corresponding to each pixel. Compared with processing the input pixel vector through a processing unit array, a single processing unit array has many levels and high complexity. The present application is equivalent to splitting a processing unit array with a high input dimension into multiple small processing unit arrays with low input dimensions. Constrained by the architecture of the processing unit array, the small processing unit array has fewer levels (that is, fewer processing units are required) and lower complexity, which can effectively reduce the processing delay of the input pixel vector, thereby improving the marking efficiency of the connected domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a schematic diagram of a processing unit array provided by an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of a connected domain marking circuit provided by an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a first type of processing unit provided by an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of a neighborhood labeling algorithm provided by an embodiment of the present application;

[0022] Figure 5 is a schematic diagram of the operation process of two processing unit arrays provided in one embodiment of the present application;

[0023] Figure 6 is a schematic diagram of a connected domain labeling circuit provided by another embodiment of the present application applied to an image reconstruction scenario;

[0024] Figure 7 It is a flowchart of a connected domain marking method based on a connected domain marking circuit provided by an embodiment of the present application. DETAILED DESCRIPTION

[0025] Before introducing the embodiments of the present application, the relevant terms involved in the present application are first explained.

[0026] 1. CCL (Connected Component Labelling): Connected domain labeling, which aims to find a unique label for each group of connected elements in the input data, that is, to detect connected areas in the input data based on topological clustering of connectivity. In the embodiment of the present application, CCL may refer to the labeling process of connected domains in the input image, and the connected domains in the input image may be implemented as connected areas composed of foreground pixels.

[0027] 2. Pyramid array: refers to a processing unit array formed by multiple processing units arranged in a pyramid structure, which can be used to realize global search and marking of connected domains. The first row in the pyramid array includes the most processing units, that is, the bottom of the pyramid array, and the last row in the pyramid array includes one processing unit, that is, the vertex of the pyramid. Optionally, the pyramid array in the embodiment of the present application can also be referred to as a multidimensional input pyramid array, a pyramid processing unit array, an inverted pyramid array, etc. The CCL algorithm based on the pyramid array eliminates the data dependency between pixels in the CCL process, thereby realizing pixel-level parallel CCL.

[0028] 3. PE (Processing Element): a processing unit, which is a base element of a pyramid array, and can be used to execute operation logic, such as threshold segmentation logic, comparison logic, sorting logic, multi-way selection logic, etc. Optionally, the processing unit in the embodiment of the present application may also be referred to as a processing node.

[0029] 4. FPGA (Field-Programmable Gate Array): Field-Programmable Gate Array is a program-driven logic device, just like a microprocessor. Its control program is stored in the memory. After power is turned on, the program is automatically loaded into the chip for execution. FPGA appears as a semi-specified circuit in the field of ASIC (Application Specific Integrated Circuit), and specific functions can be realized through software coding.

[0030] 5. Clock cycle: The reciprocal of the clock frequency, also known as the oscillation period. The clock cycle is the most basic and smallest unit of time in a computer. In one clock cycle, the processor only completes one most basic action (i.e., "micro-operation").

[0031] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0032] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0033] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0034] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless cars, autonomous driving, drones, digital twins, virtual humans, robots, artificial intelligence generated content (Artificial Intelligence Generated Content, AIGC), conversational interaction, smart medical care, smart customer service, game AI, virtual reality (Virtual Reality, VR), augmented reality (Augmented Reality, AR), etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0035] The technical solution of the present application mainly relates to the development and design of AI processors in artificial intelligence technology, and mainly relates to a connected domain marking circuit that can be used for connected domain marking.

[0036] In the related art, one implementation of the connected domain labeling is to construct a processing unit array, such as arranging multiple processing units in a pyramid structure to obtain a processing unit array. For example, the structural format of the processing unit array can refer to Figure 1, the processing unit array 100 is a pyramid array with 8 input dimensions (i.e., width), and 8 processing units are deployed in parallel in the first row of the processing unit array 100, so that 8 pixels can be processed in parallel at the same time, such as each processing unit processes one pixel. 7 processing units are deployed in parallel in the second row of the processing unit array 100, and every two adjacent processing units in the first row of processing units are connected to one processing unit in the second row of processing units. 6 processing units are deployed in parallel in the third row of the processing unit array 100, and every two adjacent processing units in the second row of processing units are connected to one processing unit in the third row of processing units, and so on. By arranging the processing units according to this architecture, a complete processing unit array can be obtained.

[0037] For a processing unit array with an input dimension of P, the corresponding processing delay can be expressed as: (LP+LL) clock cycles; where LP is the time required for the processing unit array to process an input pixel vector. If the processing time of the processing unit is one clock cycle, then for a processing unit array with an input dimension of P, it has P rows of processing units. Since each row of processing units runs in parallel, the corresponding LP is P clock cycles. LL is the time required to sort out the output (such as labels) of the processing unit array. As the input dimension increases, the more rows the processing unit array has, the more complex the processing logic becomes, and the corresponding processing delay of the processing unit array becomes higher and higher, which is not conducive to improving the computing performance of the CCL algorithm. At the same time, the number of processing units required for the processing unit array is also increasing, and the spatial complexity is increasing, which is not conducive to the miniaturization of hardware.

[0038] In addition, after a row of pixels in the input image (such as a small image, i.e., an image with a small row width) is processed, the data in the processing unit array needs to be cleared, and then the processing unit array is used to process the next row of pixels in the input image, and the CCL of each row of pixels needs to use the label marking result of the previous row of pixels. Therefore, for an input image with a row width of W and a processing unit array with an input dimension of P, the processing unit array must satisfy ceil(W / P)>=(LP+LL) each time it changes rows, to ensure that the label marking result of the previous row of pixels is ready. Among them, ceil(W / P) refers to the time required for the input port of the processing unit array to complete the input of a row of pixels, and ceil() is used to return a minimum integer greater than or equal to W / P. In this way, each time the processing unit array changes rows, if ceil(W / P)<(LP+LL), it is necessary to wait for (LP+LL)-ceil(W / P) clock cycles. If the row width of the input image is fixed, the computing performance of the CCL implemented by the processing unit array will further decrease as the input dimension increases. It should be noted that for large images (i.e. images with larger row widths), since ceil(W / P) is much larger than (LP+LL), there is no need to wait when wrapping.

[0039] Please refer to Figure 2 , which shows a schematic diagram of the structure of a connected domain marking circuit provided by an embodiment of the present application. The connected domain marking circuit 200 includes m processing unit arrays and a label processing module, where m is an integer greater than 1.

[0040] The m processing unit arrays are independent of each other, that is, the m processing unit arrays process their respective inputs separately. Optionally, the m processing unit arrays have the same architecture, and the multiple processing units in each processing unit array are arranged in a pyramid structure, such as the difference between the number of processing units corresponding to the nth row and the number of processing units corresponding to the n-1th row in the processing unit array is 1, and n is an integer greater than 1.

[0041] Among them, the number of processing units refers to the number of processing units, such as for a certain processing unit array, the number of processing units corresponding to its nth row refers to the number of processing units included in the nth row of the processing unit array. Optionally, the input dimension of the processing unit array is the same as the number of rows of the processing unit array, such as for a processing unit array with 8 input dimensions, it includes 8 rows of processing units. For a single processing unit array, the corresponding number of processing units can be expressed as follows: N = n(n+1) / 2; wherein n is the input dimension of the processing unit array, then for a connected domain labeling circuit with m processing unit arrays, it has mN processing units. The input dimension (width) of the processing unit array may refer to the number of processing units corresponding to the first row of the processing unit array, which may determine how many pixels the processing unit array can process in parallel.

[0042] The processing unit array in the embodiment of the present application can be used to assign labels and transfer labels to pixels, which is the same as described in the above embodiment. The contents not described in the embodiment of the present application can be referred to the above embodiment and will not be repeated here.

[0043] In one example, the m processing unit arrays are used to obtain an input pixel vector, the input pixel vector includes m sub-pixel vectors, the width corresponding to the i-th sub-pixel vector in the m sub-pixel vectors is the same as the width of the i-th processing unit array in the m processing unit arrays, and i is a positive integer;

[0044] The input pixel vector may refer to a vector composed of pixels in the input image, such as the input pixel vector may include multiple consecutive pixels in the same row of the input image, so the input pixel vector may be obtained by arranging the multiple consecutive pixels in the order of their sequence in the input image. The above-mentioned input image may refer to two-dimensional or multi-dimensional data to be marked as a connected domain (such as images, three-dimensional data with spatial information generated by laser radar, etc.), and the embodiment of the present application does not limit the source of the input image.

[0045] In the embodiment of the present application, the pixel value of the pixel can be used to represent the pixel, and the above-mentioned input pixel vector can be represented by a pixel value vector composed of the pixel values ​​corresponding to each pixel included therein, and the arrangement order of the pixel values ​​corresponding to each pixel is the same as the arrangement order of each pixel. For example, since the pixel value of a pixel can be represented by a numerical value in a grayscale, RGB (Red-Green-Blue), YUV (a color space), binary, etc. format, the above-mentioned input pixel vector can refer to a vector composed of numerical values ​​in a grayscale, RGB, YUV, binary, etc. format, and the embodiment of the present application does not limit this.

[0046] The sub-pixel vector may be a pixel vector obtained by evenly dividing the input pixel vector according to the width of a single processing unit array. For example, when the width of the input pixel vector is 8 and the width of the processing unit array is 4, the input pixel vector may be divided into two sub-pixel vectors, each of which has a width of 4. The width of the pixel vector may refer to the number of pixels contained in the pixel vector.

[0047] Optionally, the total width of the m processing unit arrays is the same as the width of the input pixel vector, and each processing unit array is used to obtain the corresponding sub-pixel vector. Exemplarily, m processing unit arrays can be used to obtain the corresponding sub-pixel vectors in parallel in the same clock cycle according to the sequence of the sub-pixel vectors in the input image, that is, the i-th processing unit array is used to obtain the i-th sub-pixel vector.

[0048] For example, for the input pixel vector: P1-P8, it can be divided into two sub-pixel vectors: P1-P4 and P5-P8, and the first processing unit array can be used to obtain the sub-pixel vector P1-P4, and the second processing unit array can be used to obtain the sub-pixel vector P5-P8; the second processing unit array can also be used to obtain the sub-pixel vector P1-P4, and the first processing unit array can be used to obtain the sub-pixel vector P5-P8. The embodiment of the present application is not limited to this.

[0049] Optionally, for each processing unit array, it can obtain each pixel in the sub-pixel vector from left to right, or it can obtain each pixel in the sub-pixel vector from right to left, which is not limited in the embodiment of the present application. For example, for sub-pixel vectors P1-P4, the processing unit array can use the first processing unit on the right in the first row to obtain P1, the second processing unit on the right to obtain P2, the third processing unit on the right to obtain P3, and the fourth processing unit on the right to obtain P4. It should be noted that the order in which each processing unit array obtains pixels should correspond to the pixel order in the input pixel vector to preserve the adjacent relationship between pixels in the same row.

[0050] In one example, after the m processing unit arrays obtain their respective sub-pixel vectors, the m processing unit arrays run in parallel to assign labels to the m sub-pixel vectors respectively to obtain temporary labels corresponding to respective pixels in the m sub-pixel vectors.

[0051] That is, the m processing unit arrays obtain their corresponding inputs in parallel in the same clock cycle, and process their corresponding inputs in the same clock cycle, so that the parallel acceleration capability can be maintained while reducing the processing delay. For example, based on the above embodiment, for sub-pixel vectors P1-P4 and sub-pixel vectors P5-P8, while the first processing unit array processes each pixel in the sub-pixel vectors P1-P4 in parallel, the second processing unit array also processes each pixel in the sub-pixel vectors P5-P8 in parallel, so that the temporary labels corresponding to P1-P8 can be obtained at the same time.

[0052] Optionally, after the connected domain labeling circuit obtains the temporary labels corresponding to each pixel in the input pixel vector through the m processing unit arrays, the temporary labels corresponding to each pixel are transmitted to the label processing module, and the label processing module is used to merge the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain the final labels corresponding to each pixel in the input pixel vector; wherein the final labels are used to obtain the connected domain in the input pixel vector, and each pixel in the connected domain in the input pixel vector has the same final label. Optionally, the label processing module can obtain the final label by internal design through one or multiple traversals after caching.

[0053] The temporary label merging in the embodiment of the present application refers to: the process of unifying the temporary labels of each pixel in the connected domain corresponding to the input pixel vector. If there are multiple different temporary labels in the same connected domain, the temporary label with the smallest label value can be determined as the final label corresponding to the connected domain, that is, the final labels corresponding to each pixel in the connected domain. The operation process of the label processing module will be specifically described below and will not be repeated here.

[0054] Optionally, after obtaining the connected domains in each input pixel vector corresponding to the input image, the connected domains in each input pixel vector can be merged based on the label to obtain the final connected domain corresponding to the input image to complete the connected domain labeling of the input image.

[0055] In the case where the total input dimension remains unchanged, the embodiment of the present application is equivalent to using multiple processing unit arrays with low input dimensions to replace a processing unit array with high input dimensions to achieve acceleration of CCL. Taking the use of N (N is 2^k) processing unit arrays to replace one processing unit array as an example, the processing delay of the input pixel vector can be reduced to: LP / N+log2(N)+LL clock cycles; wherein log2(N) refers to the time required for the label processing module to merge the processing results of m processing unit arrays. LP / N greatly reduces the overall processing delay of the input pixel vector, but the cost log2(N) is far less than the benefit. Therefore, the technical solution provided by the embodiment of the present application can bring about a significant improvement in the overall performance, which can effectively improve the efficiency of connected domain labeling.

[0056] For example, when the resolution of the input image is 16*16 (i.e., W=16), for a single 8-input dimensional processing unit array, the corresponding calculation delay LP is 8 clock cycles, and the label processing delay LL is 4 clock cycles. Then, the corresponding processing delay of the single 8-input dimensional processing unit array is: 8+4=12 clock cycles, and the line break waiting delay is: 8+4-ceil(16 / 8)=10 clock cycles. After replacing the single 8-input dimensional processing unit array with two 4-input dimensional processing unit arrays, the processing delay can be reduced to: 4+1+4=9 clock cycles, and the line break waiting delay is reduced to: 8 / 2+log2(2)+4-ceil(16 / 8)=7 clock cycles. The line break waiting delay benefit is 3 clock cycles, and the number of idle clock cycles of the processing unit is reduced by 30%.

[0057] For another example, for a single 16-input dimensional processing unit array, the corresponding calculation delay LP is 16 clock cycles, and the label processing delay LL is 4 clock cycles. Then, the corresponding processing delay of the single 16-input dimensional processing unit array is: 16+4=20 clock cycles, and the line break waiting delay is: 16+4-ceil(16 / 16)=19 clock cycles. After replacing the single 16-input dimensional processing unit array with 4 4-input dimensional processing unit arrays, the processing delay can be reduced to: 16 / 4+log2(4)+4=10 clock cycles, and the line break waiting delay can be reduced to: 16 / 4+log2(4)+4-ceil(16 / 16)=9 clock cycles, the line break waiting delay benefit is 10 clock cycles, and the number of idle clock cycles of the processing unit is reduced by 53%.

[0058] It can be seen that the embodiment of the present application uses multiple processing unit arrays with low input dimensions to replace a single processing unit array with high input dimensions, and processes the input pixel vector in parallel, which can effectively reduce the delay of connected domain labeling and thus improve the efficiency of connected domain labeling, especially when the input dimension of the original single processing unit array is high.

[0059] In addition, when the total input dimension remains unchanged, by replacing a single high-input-dimensional processing unit array with multiple low-input-dimensional processing unit arrays, the number of processing units can be effectively reduced, thereby reducing the space complexity of the connected domain labeling circuit, thereby saving the construction cost of the connected domain labeling circuit.

[0060] In a feasible example, when the width of the input pixel vector needs to be increased, the number of processing unit arrays can be adaptively increased to meet the data throughput requirements, so that the data throughput of the connected domain marking circuit can be effectively increased while maintaining a certain processing efficiency. The connected domain marking circuit in the embodiment of the present application is easy to topologically and scale, and the throughput and processing delay can be adjusted by simply adjusting the number of processing unit arrays.

[0061] The embodiment of the present application does not limit the number and input dimension of the processing unit array in the connected domain marking circuit, which can be set and adjusted according to actual use requirements, such as the number and input dimension of the processing unit array in the connected domain marking circuit can be set and adjusted according to real-time requirements or data throughput requirements to suit different scenarios. In one example, the connected domain marking circuit can be implemented as a chip that meets different real-time requirements or data throughput requirements, and users can choose chips according to their own needs.

[0062] In a feasible example, multiple connected domain labeling circuits can be used to label connected domains of multiple input images at the same time, thereby further speeding up the connected domain labeling and thus improving the efficiency of the connected domain labeling.

[0063] To summarize, the technical solution provided by the embodiment of the present application adopts multiple processing unit arrays to assign labels to each sub-pixel unit in the same input pixel vector, obtain temporary labels corresponding to each pixel in the input pixel vector, and then merge the temporary labels corresponding to each pixel through the label processing module to obtain the final labels corresponding to each pixel. Compared with processing the input pixel vector through a processing unit array, the single processing unit array has many levels and high complexity. The present application is equivalent to splitting a processing unit array with a high input dimension into multiple small processing unit arrays with low input dimensions. Constrained by the architecture of the processing unit array, the small processing unit array has fewer levels (that is, fewer processing units are required) and lower complexity, which can effectively reduce the processing delay of the input pixel vector, thereby improving the marking efficiency of the connected domain.

[0064] In addition, by using multiple processing unit arrays with low input dimensions to replace a single processing unit array with high input dimensions, the input pixel vectors can be processed in parallel, which can effectively reduce the delay of connected domain labeling and thus improve the efficiency of connected domain labeling, especially when the input dimension of the original single processing unit array is high.

[0065] The specific operation process of the processing unit array will be described below.

[0066] Since the processing method of each processing unit array is the same, the i-th processing unit array among the above-mentioned m processing unit arrays will be used as an example for explanation below. The i-th processing unit array can be used to assign labels to the i-th sub-pixel vector to obtain temporary labels corresponding to each pixel in the i-th sub-pixel vector.

[0067] Among them, the i-th processing unit array can refer to any processing unit array in the m processing unit arrays, and the i-th sub-pixel vector is the input of the i-th processing unit array. The label in the embodiment of the present application is used to identify a connected domain, each connected domain corresponds to a unique label, and the pixels in each connected domain have the same label. The label value of the label can be an integer, such as the labels corresponding to each connected domain in the input image are distributed in a continuous positive integer distribution, or they can be distributed in a discontinuous positive integer distribution. The temporary label can refer to a label that the processing unit array temporarily assigns to a pixel based on existing information, and the temporary label will also be updated as the information is updated.

[0068] In one example, for each processing unit array, the first row of the processing unit array includes p first-type processing units, and the nth row includes p-n+1 second-type processing units, where p is an integer greater than 1.

[0069] The functions of the first type of processing units are different from those of the second type of processing units. The first type of processing units can be used for pixel acquisition, threshold segmentation, and label assignment, while the second type of processing units can be used for label transfer and cycle synchronization.

[0070] Exemplarily, the p first-category processing units in the above-mentioned i-th processing unit array are used to respectively obtain each pixel in the i-th sub-pixel vector, and each first-category processing unit is used to obtain a pixel in the i-th sub-pixel vector. The pixel in the embodiment of the present application is characterized by the pixel value corresponding to the pixel, that is, each first-category processing unit actually obtains the pixel value of the pixel, and the process can be completed within one clock cycle. Optionally, the p first-category processing units are also used to obtain a first threshold value, and the first threshold value can be preset according to demand. For example, if the element in the pixel input vector is a vector, then the first threshold value is a vector; if the element in the pixel input vector is a scalar, then the first threshold value is a scalar. The first threshold value is used to segment pixels.

[0071] After obtaining each pixel in the i-th sub-pixel vector, the p first-type processing units in the i-th processing unit array are used to perform threshold segmentation on each pixel in the i-th sub-pixel vector, respectively, to obtain binary data corresponding to each pixel in the i-th sub-pixel vector, and the binary data can be a first value or a second value.

[0072] The first value and the second value are different, and the first value and the second value can be integers. Optionally, the first value can be used to indicate that the current pixel is a foreground pixel, such as 1, and the second value can be used to indicate that the current pixel is a background pixel, such as 0. In the embodiment of the present application, the pixel value of the foreground pixel is greater than the first threshold value, and the pixel value of the background pixel is less than or equal to the first threshold value, then the embodiment of the present application can also include the following content:

[0073] The qth first-category processing unit among the p first-category processing units is further used to determine the first value as the binary data corresponding to the qth pixel when the pixel value of the qth pixel is greater than the first threshold. Alternatively, the qth first-category processing unit among the p first-category processing units is further used to determine the second value as the binary data corresponding to the qth pixel when the pixel value of the qth pixel is less than or equal to the first threshold. The qth first-category processing unit may refer to any first-category processing unit among the p first-category processing units, and q is a positive integer not greater than p.

[0074] For example, if the pixel values ​​of pixels P1 - P4 are 38, 21, 64, and 170, respectively, and the first threshold is 32, then the binary data of pixels P1 - P4 are 1, 0, 1, and 1, respectively.

[0075] In one example, reference Figure 3 , which shows a hardware structure diagram of a first type of processing unit provided by an embodiment of the present application. The first type of processing unit 300 may include a threshold segmentation module 301, a label comparison module 302 and a label sorting module 303.

[0076] The threshold segmentation process can be implemented by the above-mentioned threshold segmentation module 301. After obtaining the first threshold and the pixel value of the pixel, the threshold segmentation module 301 compares the first threshold and the pixel value to obtain binary data of the pixel, which is recorded as foreground-enable-o (en for short).

[0077] Optionally, when the technical solution provided in the embodiment of the present application is applied in a high-speed camera transmission scenario (i.e., during the transmission of pixels, the pixels are synchronously marked as connected domains), the threshold segmentation module 301 is also provided with an enable signal, which is the sum operation result of the FVAL (frame synchronization signal) and LVAL (line synchronization signal) corresponding to the high-speed camera. When the enable signal is 1, the threshold segmentation module 301 executes the threshold segmentation process.

[0078] Optionally, the p first-category processing units are operated in parallel to simultaneously complete the threshold segmentation of each pixel in the i-th sub-pixel vector, thereby improving the efficiency of the threshold segmentation. The p first-category processing units in each processing unit array are also operated in parallel to simultaneously complete the threshold segmentation of each pixel in the m sub-pixel vector, thereby improving the efficiency of the threshold segmentation.

[0079] After obtaining the binary data corresponding to each pixel in the i-th sub-pixel vector, the p first-type processing units in the i-th processing unit array are further used to assign labels to each pixel in the i-th sub-pixel vector based on the binary data and the reference labels corresponding to each pixel in the i-th sub-pixel vector, so as to obtain temporary labels corresponding to each pixel in the i-th sub-pixel vector.

[0080] Among them, the above-mentioned reference label is related to the adjacent pixels of the pixel, and the distance between the adjacent pixels of the pixel is less than the distance threshold. Optionally, according to different neighborhood labeling algorithms, the current pixel may correspond to different numbers of adjacent pixels, and the embodiment of the present application does not limit the labeling method of adjacent pixels. For example, for the four-neighborhood labeling algorithm, the current pixel may correspond to 4 adjacent pixels, and the distance between the adjacent pixels and the current pixel is the minimum pixel distance. For the eight-neighborhood labeling algorithm, the current pixel may correspond to 8 adjacent pixels, and the distance between the adjacent pixels and the current pixel is less than or equal to times the minimum pixel distance, where the minimum pixel distance may refer to the distance between two adjacent pixels in the same row in the input image.

[0081] In an embodiment of the present application, since the labels of the pixels after the current pixel are unknown, for the four-neighborhood labeling algorithm, the neighboring pixels of the current pixel can be reduced to: the pixel directly above and the pixel directly to the left; for the eight-neighborhood labeling algorithm, the neighboring pixels of the current pixel can be reduced to: the pixel to the upper left, the pixel directly above, the pixel to the upper right, and the pixel to the left. In a feasible example, for the eight-neighborhood labeling algorithm, the neighboring pixels of the current pixel can also be reduced to: the pixel to the upper left, the pixel directly above, and the pixel to the left. This helps to reduce the computational complexity of label allocation (such as the complexity of the comparison logic and sorting logic of subsequent labels), thereby improving the efficiency of label allocation. For example, refer to Figure 4 For the four-neighborhood labeling algorithm, the current pixel 401 (i.e., p) corresponds to two adjacent pixels: b and d; for the eight-neighborhood labeling algorithm, the current pixel 401 (i.e., p) corresponds to four adjacent pixels: a, b, c, and d; for the optimized eight-neighborhood labeling algorithm, the current pixel 401 (i.e., p) corresponds to three adjacent pixels: a, b, and d.

[0082] The reference label refers to a label used for reference, which can be used to determine the temporary label of the current label. In one example, the label of the adjacent pixel corresponding to the current pixel can be determined as the reference label of the current pixel. Taking the qth pixel as an example, the reference label determination process can also include the following content.

[0083] 1. When the qth pixel is the first pixel in the input image, the qth pixel does not correspond to a reference label.

[0084] For the first pixel in the input image, there is no pixel before it, that is, the first pixel has no adjacent pixels, so the first pixel does not correspond to a reference label.

[0085] 2. When the qth pixel is not the first pixel in the input image and is located in the first row of the input image, the qth pixel corresponds to a reference label, and the label value of the reference label is the first numerical value or the second numerical value, and the first numerical value and the second numerical value are less than or equal to 1.

[0086] Optionally, since the first row of pixels does not have the previous row of pixels, it does not have adjacent pixels in the previous row, and since each first-class processing unit processes pixels individually, for the current pixel, the label of the pixel to its immediate left is unknown. At this time, binary data can be used as the label value to represent the label value of the parameter label.

[0087] Exemplarily, if the pixel to the immediate left is a foreground pixel, the first numerical value may be determined as the label value of the reference label of the current pixel to indicate that there is a label to be inherited; if the pixel to the immediate left is a background pixel, the second numerical value may be determined as the label value of the reference label of the current pixel to indicate that there is no label to be inherited.

[0088] 3. When the qth pixel is not located in the first row and the first column of the input image, the qth pixel corresponds to at least two reference labels, which are determined based on the final labels assigned to the adjacent pixels.

[0089] Optionally, refer to Figure 4 For the four-neighborhood labeling algorithm, the final labels of b and d can be determined as the reference label for determining the current pixel 401. Among them, the label of d is unknown. If d is a foreground pixel, its corresponding reference label is assigned a value of 1; if d is a background pixel, its corresponding reference label is assigned a value of 0. In a feasible example, for pixels with unknown labels, their corresponding reference labels can be assigned a value of 1. In an embodiment of the present application, for background pixels, their corresponding final labels can be represented by 0 (i.e., the second numerical value), labels with label values ​​0 and 1 are not valid labels, and labels with label values ​​greater than 1 are valid labels. Valid labels can be used for label inheritance, which will be described in detail below and will not be repeated here.

[0090] For the eight-neighborhood labeling algorithm, the final labels of a, b, c, and d can be determined as the reference label for determining the current pixel 401. Among them, the label of d is unknown, if d is a foreground pixel, then the corresponding reference label is assigned 1; if d is a background pixel, then the corresponding reference label is assigned 0.

[0091] For the optimized eight-neighborhood labeling algorithm, the final labels of a, b, and d can be determined as the reference label for determining the current pixel 401. Among them, the label of d is unknown, if d is a foreground pixel, the corresponding reference label is assigned 1; if d is a background pixel, the corresponding reference label is assigned 0.

[0092] The final label may refer to a unique label that can be assigned to the connected domain by the connected domain labeling circuit in the current processing process.

[0093] 4. When the qth pixel is not located in the first row of the input image but in the first column of the input image, the qth pixel corresponds to at least one reference label, which is determined based on the final labels assigned to the adjacent pixels.

[0094] Since the pixels in the first column have no adjacent pixels to the left, the adjacent pixels to the left may be ignored. In some embodiments, the reference label corresponding to the adjacent pixels to the left may be assigned a value of 0. Figure 4 For the four-neighborhood labeling algorithm, the final label of b can be determined as the reference label for determining the current pixel 401; for the eight-neighborhood labeling algorithm, the final labels of a, b and c can be determined as the reference label for determining the current pixel 401; for the optimized eight-neighborhood labeling algorithm, the final labels of a and b can be determined as the reference label for determining the current pixel 401.

[0095] In an example, taking the qth first-category processing unit as an example, the allocation process of the temporary label may be as follows:

[0096] The qth first-category processing unit among the above-mentioned p first-category processing units is used to obtain the number of valid labels corresponding to the qth pixel according to the reference label corresponding to the qth pixel when the binary data corresponding to the qth pixel in the ith sub-pixel vector is the first numerical value. The valid label number refers to the number of valid labels, and the label value of the valid label is greater than the first numerical value, and q is a positive integer not greater than p.

[0097] When the binary data corresponding to the qth pixel is the first value, the qth pixel is a foreground pixel, and the qth first-category processing unit needs to assign a label to the qth pixel. Optionally, the qth first-category processing unit can determine a reference label with a label value greater than 1 as a valid label, and then count the number of valid labels to obtain the number of valid labels.

[0098] The above-mentioned qth first-category processing unit is also used to determine the first numerical value as the label value of the temporary label of the qth pixel when the number of valid labels is equal to 0; or, when the number of valid labels is equal to 1, determine the valid label as the temporary label of the qth pixel; or, when the number of valid labels is greater than 1, determine the valid label with the smallest label value as the temporary label of the qth pixel.

[0099] When the number of valid labels is equal to 0, it indicates that the current pixel has no inheritable label, and the qth first-category processing unit can directly determine the first value (such as 1) as the label value of the temporary label of the qth pixel to indicate that the qth pixel needs to be assigned a new temporary label. Figure 4 , for the current pixel 401, if a=b=0, d=0 / 1, and the binary data of p is 1, then the label value of the temporary label of p is 1.

[0100] When the number of valid labels is equal to 1, it indicates that the current pixel has a label that can be inherited, and the qth first-class processing unit can directly determine the valid label as the temporary label of the qth pixel. Figure 4 For the current pixel 401, if a=6, b=0, d=0 / 1, and the binary data of p is 1, the label value of the temporary label of p is 6.

[0101] When the number of valid labels is greater than 1, it indicates that there may be conflicting labels, and the qth first-class processing unit may determine the valid label with the smallest label value as the temporary label of the qth pixel. Figure 4 , for the current pixel 401, if a=6, b=8, d=0 / 1, and the binary data of p is 1, then the label value of the temporary label of p is 6. Among them, relative to the label with a label value of 6, the label with a label value of 8 is a conflicting label (also called an equivalent label), that is, a and b are actually in the same connected domain. For another example, for the current pixel 401, if a=6, b=6, d=0 / 1, and the binary data of p is 1, the label values ​​of the two valid labels are the same, then the label value of the temporary label of p can be 6.

[0102] In one example, reference Figure 3, the temporary label assignment process can be implemented by the label comparison module 302 in the first type processing unit 300. After obtaining the three reference labels corresponding to the current pixel (such as a, b and d mentioned above), the first type processing unit 300 compares the label values ​​of the three reference labels with 1 respectively, and obtains several different valid labels corresponding to the current pixel, which are recorded as cft-num. The label comparison module 302 determines the valid label of the current pixel based on the reference label, which is recorded as temple-lable, and determines the temporary label (lable-o) based on temple-lable (optionally, this step can also be completed by the label sorting module 303). Optionally, the label comparison module 302 can be implemented by 6 parallel comparators, 2 MUXs (Multiplexers) and 1 CNT (Count).

[0103] Optionally, the p first-category processing units are operated in parallel to simultaneously complete label assignment for each pixel in the i-th sub-pixel vector, thereby improving the efficiency of label assignment. The p first-category processing units in each processing unit array are also operated in parallel to simultaneously complete label assignment for each pixel in the m sub-pixel vector, thereby improving the efficiency of label assignment.

[0104] In one example, after the qth first-category processing unit completes the label assignment for the qth pixel, when the binary data of the qth pixel is a first value (such as 1), the qth first-category processing unit is further used to determine, when the number of valid labels is greater than 1, conflicting labels from the valid labels, for which there are equivalent labels; and sort the conflicting labels to obtain a conflicting label sequence.

[0105] The conflict label sequence is used to update the conflict label list, and the conflict label list is used to count the conflict labels. Optionally, the qth first-category processing unit can sort the conflict labels in the order of label values ​​from small to large to obtain a conflict label sequence. The label with the smallest label value in the conflict label sequence can be determined as the temporary label (label-o) of the qth pixel. The conflict labels in the conflict label list are also arranged in the order of label values ​​from small to large.

[0106] Since there may be conflicting labels when the number of valid labels is greater than 1, the qth first-class processing unit can further determine whether the valid labels are the same. If there are different valid labels, there are conflicting labels, and then the conflicting labels can be sorted. For example, based on the above embodiment, if a=6 and b=8, a and b are sorted to obtain a conflicting label sequence arranged as a and b.

[0107] For example, reference Figure 3The label sorting process can be implemented by the label sorting module 303 in the first type of processing unit 300, and the label sorting module 303 can be implemented by 3 comparators and 1 MUX. The label sorting module 303 can sort the maximum 3 different valid labels (i.e., temp-lable1 to temp-lable3 in the figure) that may appear in the current clock cycle, and then use the valid label with the smallest label value as the temporary label of the current pixel, i.e., label-o, and output the conflict label sequence: cft-lable(1) to cft-lable(3).

[0108] Optionally, before enabling the label sorting module 303, the first type of processing unit 300 also needs to determine whether the current pixel is a foreground pixel. If the current pixel is a foreground pixel, the first type of processing unit 300 enables the label sorting module 303. If the current pixel is a background pixel, the first type of processing unit 300 does not enable the label sorting module 303.

[0109] In one example, after obtaining the temporary labels and binary data corresponding to each pixel in the i-th sub-pixel vector, the i-th processing unit array is used to transmit the temporary labels and binary data corresponding to each pixel in the i-th sub-pixel vector to obtain the output corresponding to each second-type processing unit, which includes the temporary labels and binary data.

[0110] The output of each first-type processing unit includes the temporary label and binary data of the pixel processed by it. The output of each second-type processing unit includes the temporary label and binary data inherited from its corresponding two input processing units, which are processing units connected to the second-type processing unit in the previous row of the second-type processing unit.

[0111] Optionally, the i-th processing unit array may sequentially transmit temporary labels and binary data through the second-type processing units in each row, and the specific content may be as follows:

[0112] The first row of second-type processing units in the i-th processing unit array is used to transfer the outputs of p first-type processing units in the i-th processing unit array to obtain the outputs of the first row of second-type processing units, wherein the first row of second-type processing units is the second row of processing units in the i-th processing unit array.

[0113] Optionally, for each second-type processing unit in the first row of second-type processing units, the second-type processing unit takes the outputs of the two input processing units as input, and takes the output of the two input processing units with the smaller label value as output.

[0114] For example, refer to Figure 5, for the second type of processing unit 511 in the processing unit array 510, its corresponding two input processing units are connected thereto: the first type of processing unit 512 and the first type of processing unit 513. The output of the first type of processing unit 512 can be expressed as (8, 1), 8 is the label value of the temporary label, 1 is binary data, the output of the first type of processing unit 513 can be expressed as (6, 1), and the output of the second type of processing unit 511 can be (6, 1).

[0115] Optionally, when there is a second numerical value (ie, 0) in the binary data included in the outputs of the two input processing units, the output of the current second-type processing unit can be directly represented as (0, 0).

[0116] When the binary data corresponding to the two input processing units are both the first value and there is a temporary label with a label value of 1, the output of the other input processing unit with a label value not 1 can be directly determined as the output of the current second-type processing unit.

[0117] The second-category processing unit in the t-th row of the ith processing unit array is used to transfer the output of the second-category processing unit in the t-1th row of the ith processing unit array to obtain the output of the second-category processing unit in the t-th row, where t is a positive integer less than n.

[0118] The second-type processing unit in the tth row is the processing unit in the t+1th row in the i-th processing unit array. For each second-type processing unit in the tth row, the second-type processing unit takes the outputs of the two input processing units as input and takes the output of the two input processing units with the smaller label value as output.

[0119] Optionally, the second type processing units in the same row run in parallel to complete label transfer at the same time, thereby improving the efficiency of label transfer. The second type processing units in the same row in each processing unit array also run in parallel to complete label transfer at the same time, thereby improving the efficiency of label transfer.

[0120] After the last row of the second type of processing units in the i-th processing unit array obtains the output, the label transfer of the i-th sub-pixel vector can be completed.

[0121] In an example, the second type of processing unit is used to implement a two-or-one operation of the tag, and the second type of processing unit can be implemented by a MUX for two-or-one operation.

[0122] In some embodiments, reference Figure 5 , taking the process of two processing unit arrays with 4 input dimensions processing an input pixel vector with a width of 8 as an example, the operation process of the multi-processing unit array is explained.

[0123] like Figure 5 As shown, the connected domain marking circuit includes two processing unit arrays with 4 input dimensions: processing unit array 510 and processing unit array 520. The current input of the connected domain marking circuit is an input pixel vector with a width of 8. The input pixel vector includes pixels corresponding to pixel indexes 0-7, and the pixel corresponding to pixel index -1 is the last pixel in the previous input pixel vector of the input pixel vector.

[0124] In an embodiment of the present application, since the input dimension of the processing unit array is 4, the input pixel vector can be divided into two sub-pixel vectors: a first sub-pixel vector composed of pixels corresponding to pixel indices 0-3, and a second sub-pixel vector composed of pixels corresponding to pixel indices 4-7.

[0125] Processing unit array 510 performs threshold segmentation on each pixel in the first sub-pixel vector, and obtains binary data of the first sub-pixel vector as 1011. Processing unit array 520 performs threshold segmentation on each pixel in the second sub-pixel vector, and obtains binary data of the second sub-pixel vector as 1100. Processing unit array 510 and processing unit array 520 operate in parallel.

[0126] The processing unit array 510 simultaneously and separately assigns labels to each pixel in the first sub-pixel vector, and the processing unit array 520 simultaneously and separately assigns labels to each pixel in the second sub-pixel vector. The eight pixels in the input pixel vector are P0-P7. For P0, its reference labels are 0, 0, 6 and 0 (known), so the temporary label of P0 can be 6; for P1, since it is a background pixel, the temporary label of P1 can be 0; for P2, its reference labels are 6, 0, 8 and 1 (unknown), so the temporary label of P2 can be 6; for P3, its reference labels are 0, 8, 0 and 1 (unknown), so the temporary label of P3 can be 8; for P4, its reference labels are 8, 0, 0 and 1 (unknown), so the temporary label of P4 can be 8; for P5, its reference labels are 0, 0, 9 and 1 (unknown), so the temporary label of P5 can be 9; for P6 and P7, since both are background pixels, the temporary labels of P6 and P7 can be 0. In summary, the temporary labels of the eight pixels in the input pixel vector are: 6068 8900.

[0127] From left to right, the output of the first first-class processing unit in the processing unit array 510 can be represented as (6, 1), the output of the second first-class processing unit can be represented as (0, 0), the output of the third first-class processing unit can be represented as (6, 1), and the output of the fourth first-class processing unit can be represented as (8, 1).

[0128] From left to right, the output of the first first-class processing unit in the processing unit array 520 can be represented as (8, 1), the output of the second first-class processing unit can be represented as (9, 1), the output of the third first-class processing unit can be represented as (0, 0), and the output of the fourth first-class processing unit can be represented as (0, 0).

[0129] While processing unit array 510 transmits temporary labels of each pixel in the first sub-pixel vector, processing unit array 520 transmits temporary labels of each pixel in the second sub-pixel vector. For processing unit array 510, the first second-type processing unit in the second row obtains output (0, 0) based on two inputs: (6, 1) and (0, 0); the second second-type processing unit in the second row obtains output (0, 0) based on two inputs: (0, 0) and (6, 1); the third second-type processing unit in the second row obtains output (6, 1) based on two inputs: (6, 1) and (8, 1). The first second-type processing unit in the third row obtains output (0, 0) based on two inputs: (0, 0) and (0, 0); the second second-type processing unit in the third row obtains output (0, 0) based on two inputs: (0, 0) and (6, 1). The first second-type processing unit in the fourth row obtains the output (0, 0) based on two inputs: (0, 0) and (0, 0).

[0130] For the processing unit array 520, the first second-type processing unit in the second row obtains an output (8, 1) based on two inputs: (8, 1) and (9, 1); the second second-type processing unit in the second row obtains an output (0, 0) based on two inputs: (9, 1) and (0, 0); the third second-type processing unit in the second row obtains an output (0, 0) based on two inputs: (0, 0) and (0, 0). The first second-type processing unit in the third row obtains an output (0, 0) based on two inputs: (8, 1) and (0, 0); the second second-type processing unit in the third row obtains an output (0, 0) based on two inputs: (0, 0) and (0, 0). The first second-type processing unit in the fourth row obtains an output (0, 0) based on two inputs: (0, 0) and (0, 0).

[0131] In one example, when each row of the input image corresponds to multiple input pixel vectors, the above-mentioned m processing unit arrays process the multiple input pixel vectors in a pipeline manner; wherein the m processing unit arrays obtain one input pixel vector per clock cycle, and when the xth input pixel vector is processed by the first row processing unit in the current clock cycle, the x+1th input pixel vector is input to the first row processing unit in the next clock cycle of the current clock cycle, and x is a positive integer.

[0132] Optionally, in an embodiment of the present application, the processing time of the first type of processing unit is related to the dimension of the input pixel vector (such as a positive correlation), then the processing time required for the first row of processing units is related to the dimension of the input pixel vector, and the processing time of the second type of processing unit is one clock cycle, then the processing time of each row of the second type of processing units is one clock cycle.

[0133] For example, taking the example that each row of the input image corresponds to 3 input pixel vectors, after the third clock cycle, the processing result of the first pixel vector by the first row of processing units in the array of m processing units can be obtained.

[0134] Among them, in the second clock cycle, the first row of processing units in the m processing unit array adds processing of a new input pixel vector (recorded as the second input pixel vector), and in the third clock cycle, the first row of processing units in the m processing unit array adds processing of a new input pixel vector again (recorded as the third input pixel vector).

[0135] In the fourth clock cycle, the processing results of the first row of processing units in the m processing unit array on the second input pixel vector can be obtained. In the fifth clock cycle, the processing results of the first row of processing units in the m processing unit array on the third input pixel vector can be obtained.

[0136] And so on, until the above three input pixel vectors are all processed. For example, if the input dimension of the processing unit array is 3, then after 3 clock cycles, each clock cycle can complete the processing of 1 input pixel vector, 4 clock cycles can complete the processing of 2 input pixel vectors, and 5 clock cycles can complete the processing of the above three input pixel vectors.

[0137] Optionally, for an input pixel vector, since its corresponding processing results are distributed in different clock cycles, in order to facilitate the subsequent label processing module, the embodiment of the present application further includes the following content:

[0138] The above-mentioned m processing unit arrays are also used to synchronize the output of the input pixel vector in different clock cycles to the last clock cycle corresponding to the input pixel vector, so as to obtain the output of the m processing unit arrays for the input pixel vector in the same clock cycle; wherein the output in the same clock cycle is transmitted to the above-mentioned label processing module.

[0139] For example, if the input pixel vector corresponds to the processing result in the first clock cycle, the processing result in the second clock cycle and the processing result in the third clock cycle, the processing result in the first clock cycle can be added to the processing result in the third clock cycle, and the addition only increases the clock cycle and does not change the content, and the processing result in the second clock cycle is added to the processing result in the third clock cycle, so that the processing result of the input pixel vector in the same clock cycle can be obtained (that is, the output of the input pixel vector in the same clock cycle). Optionally, this process can be implemented with the help of the register corresponding to the connected domain marking circuit.

[0140] The specific operation process of the label processing module will be explained below.

[0141] After obtaining the output of the input pixel vector in the same clock cycle, the label processing module is used to merge the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain the final labels corresponding to each pixel in the input pixel vector.

[0142] In one example, the label processing module can be implemented as a logic circuit, which can be used to implement functions such as isolated processing unit search, isolated processing unit merging, label merging, and label storage. These functions are combined to implement the label merging process. The embodiment of the present application may also include the following contents:

[0143] 1. Isolated processing unit search: The above-mentioned label processing module is also used to determine the isolated processing units corresponding to the m processing unit arrays respectively according to the output of each processing unit in the m processing unit arrays.

[0144] The isolated processing unit refers to a processing unit that satisfies an isolation condition. Exemplarily, the isolation condition can be expressed as follows:

[0145]

[0146] Among them, I(r i , c j ) represents the judgment condition of the isolated processing unit. If I(r i , c j )=1, then the rth i Row and c j The processing unit of the column is an isolated processing unit. If I(r i , c j )=0, then the rth i Row and c j The processing unit of the column is not an isolated processing unit. i , c j ) represents the rth iRow and c j The binary data of the processing unit of the column, b(r i , c j-1 ) represents the rth i Row and c j-1 The binary data of the processing unit of the column, b(r i , c j+1 ) represents the rth i Row and c j+1 The binary data of the processing unit of the column, n is the number of columns in each row in the processing unit array, and can be used to indicate that a processing unit is the last processing unit in a row.

[0147] That is, for each processing unit in the m processing unit array, the above-mentioned label processing module is also used to determine the processing unit as an isolated processing unit when the output of the processing unit includes a first value and the output of the adjacent processing unit corresponding to the processing unit in the row where the processing unit is located includes a second value.

[0148] Among them, the output of the processing unit includes binary data and a final label, the output of the processing unit includes a first value, which means that the binary data corresponding to the processing unit is the first value (such as 1), and the adjacent processing unit of the processing unit refers to the processing unit adjacent to the processing unit.

[0149] Optionally, the isolated processing unit may be identified by identification data, and the identification data of the isolated processing unit may include a temporary label, and a row and a column where the isolated processing unit is located.

[0150] For example, reference Figure 5 , for the second type processing unit 511 in the processing unit array 510, its binary data is 1, and the binary data of the adjacent processing unit is 0, then the second type processing unit 511 can be determined as an isolated processing unit, which is recorded as (6, 2, 3) (in order from left to right). For the first second type processing unit in the second row of the processing unit array 520, its binary data is 1, and the binary data of the adjacent processing unit is 0, then the second type processing unit is determined as an isolated processing unit, which is recorded as (8, 2, 5). Similarly, it can be obtained that the first first type processing unit in the first row of the processing unit array 510 is also an isolated processing unit, which is recorded as (6, 1, 1).

[0151] The row and column where the isolated processing unit is located may refer to the row and column where the isolated processing unit is located in a large processing unit array that is fictitious based on m processing unit arrays and has a total input dimension of the m processing unit arrays as an input dimension. For example, if the processing unit array 510 and the processing unit array 520 are arranged in parallel and the processing units are added to the vacancies, the rows and columns where the individual processing units are located can be determined.

[0152] It can be understood that the output of each processing unit in the above m processing unit array may refer to the output synchronized to the same clock cycle.

[0153] 2. Isolated processing unit merging: The above-mentioned label processing module is also used to merge the identification data of the isolated processing units corresponding to the m processing unit arrays respectively according to the connectivity between each pixel in the input pixel vector, so as to obtain the merged identification data corresponding to the m processing unit arrays respectively.

[0154] In the embodiment of the present application, the connectivity between each pixel can be used to indicate whether the pixels are adjacent and connected. For example, if P1 and P2 in the input pixel vector are adjacent, and P1 and P2 are both foreground pixels, it can be determined that P1 and P2 are connected, that is, P1 and P2 are in the same connected domain. If P3 and P2 in the input pixel vector are connected, then P3 and P1 are also connected.

[0155] The merging of isolated processing units in the embodiment of the present application may refer to the merging of identification data corresponding to the isolated processing units.

[0156] In one example, the process of obtaining the merged identification data may be as follows:

[0157] 1) The label processing module is also used to construct a sub-processing unit array corresponding to the isolated point processing unit with the isolated processing unit as the vertex and the first row of the processing unit array as the bottom.

[0158] Optionally, the sub-processing unit array is constructed in accordance with a pyramid structure, so that the first type of processing unit corresponding to the isolated processing unit in the first row of the processing unit array can be determined, and then it can be directly determined that the foreground pixels processed by the first type of processing unit corresponding to the isolated processing unit in the first row of the processing unit array are in the same connected domain. Figure 5 , the sub-processing unit array 530 constructed with the second type processing unit 511 as the vertex includes the first type processing unit 512 and the first type processing unit 513. The pixel processed by the first type processing unit 512 is adjacent to the pixel processed by the first type processing unit 513, and both are foreground pixels. It can be determined that the pixel processed by the first type processing unit 512 and the pixel processed by the first type processing unit 513 are in the same connected domain, so the connected domain in the sub-pixel vector can be obtained. Using the same method, the sub-processing unit array 550 in the processing unit array 510 and the sub-processing unit array 540 in the processing unit array 520 can also be obtained.

[0159] It can be seen that the identification data of the isolated processing unit can also be used to indicate the position and number of pixels in the connected domain corresponding to it. For example, if the second type of processing unit 511 is an isolated processing unit and its identification data is (6, 2, 3), it can be seen that the pixels in the connected domain corresponding to the second type of processing unit 511 are two pixels starting from the third pixel in the input pixel vector, where the column (3) can be used to indicate the position of the starting pixel in the connected domain, and the row (2) can be used to indicate the number of pixels in the connected domain. For another example, (6, 1, 1) can be used to indicate that the first pixel in the input pixel vector belongs to a connected domain (corresponding to a sub-pixel vector), and (8, 2, 5) can be used to indicate that the fifth pixel and the sixth pixel in the input pixel vector belong to a connected domain (corresponding to a sub-pixel vector).

[0160] 2) For the first sub-processing unit array and the second sub-processing unit array corresponding to the m processing unit arrays, the label processing module is also used to merge the identification data of the isolated processing unit corresponding to the first sub-processing unit array with the identification data of the isolated processing unit corresponding to the second sub-processing unit array if there are pixels in the first sub-processing unit array adjacent to the pixels in the second sub-processing unit array, so as to obtain merged identification data corresponding to the first sub-processing unit array and the second sub-processing unit array.

[0161] The first processing sub-unit array may refer to any processing sub-unit array corresponding to the m processing unit arrays, and the second sub-processing unit array may refer to any processing sub-unit array among the processing sub-unit arrays corresponding to the m processing unit arrays except the first processing sub-unit array.

[0162] Optionally, the existence of pixels in the first sub-processing unit array that are adjacent to pixels in the second sub-processing unit array means that among the pixels processed by the first sub-processing unit array, there are pixels that are connected to the pixels processed by the second sub-processing unit array. Figure 5 For the sub-processing unit array 530 in the processing unit array 510 and the sub-processing unit array 540 in the processing unit array 520, the vertex of the sub-processing unit array 530 is an isolated processing unit and is the last node in the second row of the processing unit array 510, and the vertex of the sub-processing unit array 540 is an isolated processing unit and is the first processing unit in the second row of the processing unit array 520. Then, P3 and P4 in the input pixel vector are connected, that is, the connected domain corresponding to the sub-processing unit array 530 and the connected domain corresponding to the sub-processing unit array 540 are actually the same connected domain, so the connected domain corresponding to the sub-processing unit array 530 and the connected domain corresponding to the sub-processing unit array 540 can be merged.

[0163] In the embodiment of the present application, the merged representation data can be used to mark whether the connected domain in the input pixel vector needs to be merged. Optionally, the merged representation data can be determined based on the identification data of the first sub-processing unit array and the identification data with a smaller label value in the identification data of the second sub-processing unit array.

[0164] The identification data of the first sub-processing unit array is the identification data of the corresponding isolated processing unit, and the identification data of the second sub-processing unit array is the identification data of the corresponding isolated processing unit, which can be determined as the combined identification data according to the identification data with the smaller tag value of the included temporary tag. Figure 5 , the identification data of sub-processing unit array 530 is: (6, 2, 3), and the identification data of sub-processing unit array 540 is: (8, 2, 5), then the merged identification data is: (6, 4, 3), that is, the second connected domain in the input pixel vector includes the four pixels P2-P5, and the label value of the final label of these four pixels can be 6.

[0165] Optionally, the merge identification data can also be used to indicate the positions of pixels included in the connected domain in the input pixel vector. For example, the above (3) can indicate that the starting pixel in the second connected domain is P2, and the above (4) indicates that the second connected domain includes 4 pixels.

[0166] It should be noted that, when the label value of the temporary label included in one identification data is 1, and the label value of the temporary label included in the other identification data is not 1, the temporary label included in the other identification data can be determined as the temporary label included in the merged identification data. When the label value of the temporary label included in one identification data is 1, and the label value of the temporary label included in the other identification data is also 1, 1 can be determined as the label value of the temporary label included in the merged identification data to indicate that the connected domain corresponding to the merged identification data needs to be assigned a new label.

[0167] 3. Label merging: The label processing module is also used to assign labels to the isolated processing units corresponding to the m processing unit arrays according to the merged identification data corresponding to the m processing unit arrays, so as to obtain the final labels corresponding to each pixel in the input pixel vector.

[0168] In the embodiment of the present application, label assignment to the isolated processing unit refers to label assignment to the connected domain in the input pixel vector, that is, label assignment to the pixels processed by the sub-processing unit array corresponding to the isolated processing unit.

[0169] In one example, the label processing module is further used to assign a new label to the isolated processing unit when the temporary label corresponding to the isolated processing unit is a first value, so as to serve as a final label of pixels processed by the sub-processing unit array corresponding to the isolated processing unit.

[0170] For example, if the temporary label of each pixel in the current sub-pixel vector is 1 (that is, the corresponding previous row of pixels are all background pixels), there is only one isolated processing unit in the corresponding processing unit array, that is, the vertex of the processing unit array. The pixels in the current sub-pixel vector belong to the same connected domain, and the connected domain is a new connected domain. In this case, the label processing module can assign a new label to the connected domain as the final label of the pixels included in the connected domain.

[0171] Optionally, during the label allocation process, label management can be used to reduce the maximum number of labels required for each input pixel vector. By assigning labels to isolated processing nodes, the label allocation granularity can be converted from pixels to regions, thereby effectively reducing the number of labels.

[0172] Exemplarily, the connected domain marking circuit can also correspond to two label lists, the first label list is used to record used labels and unused labels, and the second label list is used to record new labels. If a conflicting label is released by the first label list, it can be added to the second label list to provide a new label for the subsequent label allocation process. For example, when a label does not appear until a row of pixels is processed, it means that the next row of pixels will not be connected to the area marked by the label. Therefore, this label will be released by the first label list and added to the second label list for future use. This will keep the quantization bit width of the label smaller, reduce the complexity of the label, and reduce the time for label comparison and merging, thereby improving the efficiency of connected domain marking.

[0173] When a new label is assigned to an isolated processing unit (or units) and no existing label is inherited, the second label list is updated in the next clock cycle.

[0174] In one example, the second label list can be implemented as a FIFO (First Input Fist Output) memory. For example, if the character length of the label is 8 bits, the connected domain marking circuit has 254 labels available (0 can be used to represent background pixels, and 1 can be used to indicate that a label needs to be assigned). The use of each new label will cause an element in the FIFO memory to be popped out. On the other hand, when a label (such as a conflicting label) is released from use, it will be pushed into the FIFO memory after the first label list is refreshed.

[0175] Optionally, when there are sufficient labels (such as when processing a small image with fewer pixels), a new label can be assigned to each foreground pixel in the input pixel vector, and then the new labels can be merged according to connectivity to obtain the final labels corresponding to each pixel in the input pixel vector.

[0176] In one example, the label processing module is also used to determine the temporary label included in the merged identification data of the isolated processing unit as the final label of the pixel processed by the sub-processing unit array corresponding to the isolated processing unit when the label value of the temporary label corresponding to the isolated processing unit is greater than the first numerical value.

[0177] Optionally, if the label value of the temporary label corresponding to the isolated processing unit is greater than the first value, it means that the connected domain corresponding to the sub-processing unit array corresponding to the isolated processing unit is connected to the connected domain in the previous row of pixels. Then, the pixels processed by the sub-processing unit array corresponding to the isolated processing unit can inherit the label corresponding to the connected domain in the previous row of pixels.

[0178] For two isolated processing units that need to merge identification data, the connected domains corresponding to the two isolated processing units are connected to each other, and the pixels corresponding to the two isolated processing units can inherit the labels corresponding to the connected domain in the previous row of pixels as their respective final labels.

[0179] For example, refer to Figure 5 , the final labels of the pixels processed by the sub-processing unit array 530 and the sub-processing unit array 540 may inherit the label value 6. The label with a label value of 8 may be recorded as a conflicting label of the label with a label value of 6.

[0180] In one example, for an isolated processing unit that has not undergone identification data merging, the label processing module is further used to determine the temporary label included in the identification data of the isolated processing unit as the final label of the pixels included in the sub-processing unit array corresponding to the isolated processing unit when the label value of the temporary label corresponding to the isolated processing unit is greater than a first value. Alternatively, the label processing module is further used to assign a new label as the final label to the pixels processed by the sub-processing unit array corresponding to the isolated processing unit when the label value of the temporary label corresponding to the isolated processing unit is equal to the first value.

[0181] For example, refer to Figure 5, for the sub-processing unit array 550, the final table of pixels processed by it can inherit the label value 6. Therefore, the final labels of each pixel in the input pixel vector are: 6066 6600. Among them, for the label with a label value of 6, the labels with label values ​​of 8 and 9 are conflicting labels, and the labels with label values ​​of 8 and 9 need to be replaced by the label with a label value of 6, and then adjusted to free labels for use in subsequent processes.

[0182] Optionally, after all pixels in the input image are assigned a label, the labels of the pixels before the current input pixel vector can be uniformly replaced according to the conflicting label list, thereby determining each connected domain in the input image. The labels of the pixels before the current input pixel vector can also be replaced when the current input pixel vector is processed, that is, according to the conflicting label list, to release the conflicting labels, so as to achieve the reuse of label resources, thereby ensuring the sufficiency of label resources, which is not limited in the embodiments of the present application.

[0183] In one example, the label processing module is also used to construct a conflicting label list based on the merge identification data corresponding to the m processing unit arrays and the conflicting label sequence corresponding to the first type of processing unit in the processing unit array; wherein the conflicting label list is used to merge labels after obtaining the final labels corresponding to each pixel in the input image, so as to obtain the merged labels corresponding to each pixel in the input image, and the pixels in the connected domain in the input image have the same merged labels.

[0184] The conflicting label list is used to record conflicting labels. For example, if labels with label values ​​of 6, 8, and 9 are conflicting labels, then all labels with label values ​​of 6, 8, and 9 can be replaced with the label with the smallest label value among the three: the label with a label value of 6. The label with a label value of 6 can be called a merged label. That is, for each conflicting label in the same group, the conflicting label with the smallest label value is the merged label.

[0185] Optionally, if the final label of a pixel does not have a conflicting label, the final label is retained for the pixel. After the labels are merged, the pixels in the input image can be divided based on the merged label or the final label to obtain the connected domain corresponding to the input image.

[0186] 4. Label storage: The connected domain labeling circuit in the embodiment of the present application also corresponds to a memory, and the label processing module can send the label of each pixel in the input image to the memory to complete the storage of the label of each pixel.

[0187] The specific process of obtaining the input pixel vector is described below.

[0188] In some embodiments, reference Figure 2The connected domain labeling circuit 200 further includes a pixel vector acquisition module. The pixel vector acquisition module is used to generate an input pixel vector.

[0189] Optionally, the pixel vector acquisition module is used to divide the input image row by row based on the total width of the m processing unit arrays to obtain multiple input pixel vectors when acquiring the input image.

[0190] The input image may be an image uploaded to the connected domain labeling circuit. The embodiment of the present application does not limit the source of the input image. For example, it may refer to an image generated by an image acquisition device, such as a photo, a video frame, a drawing, etc.

[0191] After the connected domain labeling circuit obtains the input image through the pixel vector obtaining module, the pixel vector obtaining module divides the input image to obtain a plurality of input pixel vectors corresponding to the input image.

[0192] For example, for four processing unit arrays with four input dimensions, the pixel vector acquisition module can divide the input image row by row in units of 16 pixels. If the width of the input image is 32, each row of pixels can be divided into two input pixel vectors.

[0193] Optionally, when the width of the input pixel vector is less than the total width of the m processing unit arrays, the input pixel vector is padded with the second value. For example, based on the above embodiment, if the width of the input image is 30, each row of pixels can be divided into 2 input pixel vectors, and the last two elements in the second input pixel vector corresponding to each row are 0 (i.e., the pixel value is 0).

[0194] Optionally, the pixel vector acquisition module is used to divide the original input pixel vector by the total width of the m processing unit arrays to obtain multiple input pixel vectors when acquiring the original input pixel vector.

[0195] The original input pixel vector may refer to an input pixel vector in the input image. For example, the high-speed camera has the data characteristic of transmitting multiple pixels in parallel in each clock cycle, and the multiple pixels can be combined into an original input pixel vector. For example, in the measurement scenario of high-speed moving targets, the visual system is usually equipped with a high-speed camera with parallel data transmission capability. The high-speed camera in full operation can transmit 8 pixels in parallel in one clock cycle, providing 64-bit data.

[0196] Optionally, if the width of the original input pixel vector is the same as the total input dimension of the m processing unit arrays, there is no need to divide the original input pixel vector; if the width of the original input pixel vector is greater than the total input dimension of the m processing unit arrays, the original input pixel vector is divided based on the total width of the m processing unit arrays; if the width of the original input pixel vector is less than the total width of the m processing unit arrays, the original input pixel vector is divided based on the total width of the m processing unit arrays, and the end of the input pixel vector with missing elements is padded with the second value.

[0197] In some embodiments, the connected domain labeling circuit provided in the embodiments of the present application can realize the acceleration of CCL, which can be applied to any scenario requiring CCL, such as foreground and background separation, image segmentation, target recognition, OCR (Optical Character Recognition) recognition and other scenarios.

[0198] For example, in an OCR recognition scenario based on deep learning, the connected domain marking circuit provided in the embodiment of the present application can be used to obtain the text area in the image to be recognized, as well as the coordinates, rotation angle and other information of the text area, to provide a basis for subsequent text recognition.

[0199] For another example, the connected domain labeling circuit provided in the embodiment of the present application can be applied to an image reconstruction scenario. In this scenario, the connected domain labeling circuit can be implemented as an FPGA chip, or directly used in a chip. Figure 6 The scene includes a camera 601 for input image acquisition, an FPGA chip 602 for implementing CCL, and a display 603 for displaying the reconstructed image. Optionally, in order to reduce the operating pressure of the FPGA chip 602, a threshold segmentation device is also separately set to convert the input image into a binary image, and the FPGA chip 602 only needs to process the binary image.

[0200] The FPGA chip 602 can be connected to the camera 601 in an embedded manner. The FPGA chip 602 can process pixels synchronously with pixel transmission and transmit the connected domain marking results of the input image to the display 603 for display.

[0201] The following is an embodiment of the method of the present application. For details not described in detail in the embodiment of the method of the present application, please refer to the above embodiment of the numerical sorting module.

[0202] Please refer to Figure 7, which shows a flow chart of a connected domain marking method based on a connected domain marking circuit provided by an embodiment of the present application, the method is applied to the connected domain marking circuit introduced above, the connected domain marking circuit includes m processing unit arrays and a label processing module, the difference between the number of processing units corresponding to the nth row and the number of processing units corresponding to the n-1th row in the processing unit array is 1, and m and n are integers greater than 1. The method may include at least one of the following steps 701 to 703.

[0203] Step 701, m processing unit arrays obtain an input pixel vector, which includes m sub-pixel vectors, and the width corresponding to the i-th sub-pixel vector in the m sub-pixel vectors is the same as the width of the i-th processing unit array in the m processing unit arrays. The input pixel vector includes multiple consecutive pixels in the same row of the input image, and i is a positive integer.

[0204] In some embodiments, the connected domain marking circuit further includes a pixel vector acquisition module; the pixel vector acquisition module is used to divide the input image row by row based on the total width of the m processing unit arrays to obtain a plurality of input pixel vectors when acquiring the input image;

[0205] Alternatively, the pixel vector acquisition module is used to divide the original input pixel vector into units of the total width of the m processing unit arrays to obtain multiple input pixel vectors when acquiring the original input pixel vector; wherein, when the width of the input pixel vector is smaller than the total width of the m processing unit arrays, the input pixel vector is padded with a second value.

[0206] In some embodiments, when each row of the input image corresponds to multiple input pixel vectors, m processing unit arrays process the multiple input pixel vectors in a pipeline manner; wherein the m processing unit arrays acquire one input pixel vector per clock cycle, and when the xth input pixel vector is processed by the first row processing unit in the current clock cycle, the x+1th input pixel vector is input to the first row processing unit in the next clock cycle of the current clock cycle, and x is a positive integer.

[0207] Step 702: The i-th processing unit array performs label assignment on the i-th sub-pixel vector to obtain temporary labels corresponding to each pixel in the i-th sub-pixel vector.

[0208] In some embodiments, m processing unit arrays operate in parallel to assign labels to the m sub-pixel vectors respectively, and obtain temporary labels corresponding to each pixel in the m sub-pixel vectors respectively.

[0209] In some embodiments, the first row of the processing unit array includes p first-type processing units, and the nth row includes p-n+1 second-type processing units, where p is an integer greater than 1; the p first-type processing units in the ith processing unit array are used to perform threshold segmentation on each pixel in the ith sub-pixel vector, respectively, to obtain binary data corresponding to each pixel in the ith sub-pixel vector, where the binary data is the first value or the second value, and the p first-type processing units operate in parallel;

[0210] The p first-type processing units in the i-th processing unit array are further used to assign labels to the pixels in the i-th sub-pixel vector according to the binary data and the reference labels corresponding to the pixels in the i-th sub-pixel vector, respectively, to obtain temporary labels corresponding to the pixels in the i-th sub-pixel vector, wherein the reference labels are related to the adjacent pixels of the pixel, and the distance between the adjacent pixels of the pixel and the pixel is less than the distance threshold;

[0211] Each second-type processing unit in the i-th processing unit array is used to transfer the temporary label and binary data corresponding to each pixel in the i-th sub-pixel vector, and obtain the output corresponding to each second-type processing unit, and the output includes the temporary label and the binary data.

[0212] In some embodiments, the qth first-category processing unit among the p first-category processing units is used to obtain the number of valid labels corresponding to the qth pixel according to the reference label corresponding to the qth pixel when the binary data corresponding to the qth pixel in the ith sub-pixel vector is the first value, where the number of valid labels refers to the number of valid labels, the label value of the valid label is greater than the first value, and q is a positive integer not greater than p;

[0213] The qth first-category processing unit is also used to determine the first numerical value as the label value of the temporary label of the qth pixel when the number of valid labels is equal to 0; or, when the number of valid labels is equal to 1, determine the valid label as the temporary label of the qth pixel; or, when the number of valid labels is greater than 1, determine the valid label with the smallest label value as the temporary label of the qth pixel.

[0214] In some embodiments, when the qth pixel is the first pixel in the input image, the qth pixel does not correspond to a reference label;

[0215] Alternatively, when the qth pixel is not the first pixel in the input image and is located in the first row of the input image, the qth pixel corresponds to a reference label, and the label value of the reference label is the first value or the second value, and the first value and the second value are less than or equal to 1;

[0216] Alternatively, when the qth pixel is not located in the first row and the first column of the input image, the qth pixel corresponds to at least two reference labels, and the reference labels are determined based on the final labels assigned to the adjacent pixels;

[0217] Alternatively, when the qth pixel is not located in the first row of the input image but in the first column of the input image, the qth pixel corresponds to at least one reference label, and the reference label is determined based on the final label assigned to the adjacent pixels.

[0218] In some embodiments, the qth first-type processing unit is further configured to determine conflicting tags from the valid tags when the number of valid tags is greater than 1, and for the conflicting tags, there are equivalent tags;

[0219] The conflicting labels are sorted to obtain a conflicting label sequence; wherein the conflicting label sequence is used to update the conflicting label list, and the conflicting label list is used to count the conflicting labels.

[0220] In some embodiments, the first row of the second type of processing unit in the i-th processing unit array is used to transfer the outputs of the p first type of processing units in the i-th processing unit array to obtain the outputs of the first row of the second type of processing units;

[0221] The second-category processing unit in the t-th row of the i-th processing unit array is used to transfer the output of the second-category processing unit in the t-1-th row of the i-th processing unit array to obtain the output of the second-category processing unit in the t-th row, where t is a positive integer less than n; wherein the second-category processing unit takes the output of two input processing units as input and takes the output of the two input processing units with the smaller label value as output, and the input processing unit refers to the processing unit in the row above the row where the second-category processing unit is located and is connected to the second-category processing unit.

[0222] In some embodiments, the qth first-type processing unit among the p first-type processing units is further configured to determine the first value as binary data corresponding to the qth pixel when the pixel value of the qth pixel is greater than the first threshold;

[0223] Alternatively, the qth first-type processing unit among the p first-type processing units is further configured to determine the second value as binary data corresponding to the qth pixel when the pixel value of the qth pixel is less than or equal to the first threshold.

[0224] In some embodiments, the m processing unit arrays are also used to synchronize the outputs of the input pixel vector in different clock cycles to the last clock cycle corresponding to the input pixel vector, so as to obtain the outputs of the m processing unit arrays for the input pixel vector in the same clock cycle; wherein the outputs in the same clock cycle are transmitted to the label processing module.

[0225] In some embodiments, the processing of the processing unit takes one clock cycle.

[0226] Step 703, the label processing module merges the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain the final labels corresponding to each pixel in the input pixel vector; wherein the final labels are used to obtain the connected domain in the input pixel vector, and each pixel belonging to the same connected domain in the input pixel vector has the same final label, and different connected domains in the input pixel vector correspond to different final labels.

[0227] In some embodiments, the label processing module is further used to determine the isolated processing units corresponding to the m processing unit arrays respectively according to the output of each processing unit in the m processing unit arrays, the isolated processing unit refers to a processing unit that meets the isolation condition, the identification data of the isolated processing unit includes a temporary label, the row and column where the isolated processing unit is located, and the output of the processing unit includes binary data and a final label;

[0228] The label processing module is also used to merge the identification data of the isolated processing units corresponding to the m processing unit arrays according to the connectivity between the pixels in the input pixel vector, so as to obtain the merged identification data corresponding to the m processing unit arrays, wherein the merged identification data is used to indicate the position of the pixels included in the connected domain in the input pixel vector;

[0229] The label processing module is also used to assign labels to the isolated processing units corresponding to the m processing unit arrays according to the combined identification data corresponding to the m processing unit arrays, so as to obtain the final labels corresponding to each pixel in the input pixel vector.

[0230] In some embodiments, for each processing unit in the array of m processing units, the label processing module is also used to determine the processing unit as an isolated processing unit when the output of the processing unit includes a first value and the output of the adjacent processing unit corresponding to the processing unit in the row where the processing unit is located includes a second value.

[0231] In some embodiments, the tag processing module is further used to construct a sub-processing unit array corresponding to the isolated point processing unit with the isolated processing unit as the vertex and the first row of the processing unit array as the bottom;

[0232] For the first sub-processing unit array and the second sub-processing unit array corresponding to the m processing unit arrays, the label processing module is also used to merge the identification data of the isolated processing unit corresponding to the first sub-processing unit array with the identification data of the isolated processing unit corresponding to the second sub-processing unit array if there are pixels in the first sub-processing unit array adjacent to the pixels in the second sub-processing unit array, so as to obtain merged identification data corresponding to the first sub-processing unit array and the second sub-processing unit array; wherein the merged representation data is determined based on the identification data of the first sub-processing unit array and the identification data with a smaller label value in the identification data of the second sub-processing unit array.

[0233] In some embodiments, the label processing module is further used to assign a new label to the isolated processing unit when the temporary label corresponding to the isolated processing unit is the first value, so as to serve as the final label of the pixel processed by the sub-processing unit array corresponding to the isolated processing unit;

[0234] Alternatively, the label processing module is also used to determine the temporary label included in the merged identification data of the isolated processing unit as the final label of the pixel processed by the sub-processing unit array corresponding to the isolated processing unit when the label value of the temporary label corresponding to the isolated processing unit is greater than the first numerical value.

[0235] In some embodiments, the label processing module is also used to construct a conflicting label list based on the merge identification data corresponding to the m processing unit arrays and the conflicting label sequence corresponding to the first type of processing unit in the processing unit array; wherein the conflicting label list is used to merge labels after obtaining the final labels corresponding to each pixel in the input image, so as to obtain the merged labels corresponding to each pixel in the input image, and the pixels in the connected domain in the input image have the same merged label.

[0236] To summarize, the technical solution provided by the embodiment of the present application adopts multiple processing unit arrays to assign labels to each sub-pixel unit in the same input pixel vector, obtain temporary labels corresponding to each pixel in the input pixel vector, and then merge the temporary labels corresponding to each pixel through the label processing module to obtain the final labels corresponding to each pixel. Compared with processing the input pixel vector through a processing unit array, the single processing unit array has many levels and high complexity. The present application is equivalent to splitting a processing unit array with a high input dimension into multiple small processing unit arrays with low input dimensions. Constrained by the architecture of the processing unit array, the small processing unit array has fewer levels (that is, fewer processing units are required) and lower complexity, which can effectively reduce the processing delay of the input pixel vector, thereby improving the marking efficiency of the connected domain.

[0237] An exemplary embodiment of the present application further provides a chip, wherein the chip includes the connected domain marking circuit introduced in the above embodiment.

[0238] Optionally, the chip may refer to an FPGA chip, an ASIC chip, etc. implemented based on a connected domain marking circuit, which is not limited in the embodiments of the present application.

[0239] An exemplary embodiment of the present application further provides a processor, wherein the processor includes the connected domain marking circuit introduced in the above embodiment.

[0240] Optionally, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor may also include an AI processor for processing computing operations related to machine learning.

[0241] The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory is used to store a computer program, which is configured to be executed by one or more processors to implement the above-mentioned connected domain labeling method.

[0242] An exemplary embodiment of the present application further provides a computer device, the computer device comprising a processor, and the processor comprising the connected domain marking circuit introduced in the above embodiment.

[0243] Optionally, the processor is an AI processor, or other processor that needs to use connected domain labels, which is not limited in this application.

[0244] Optionally, the computer device can be a server, or a terminal device such as a mobile phone, a tablet computer, a vehicle-mounted terminal, a wearable device, a smart home device, or any device that is applied to a processor, such as a robot or a base station, and this application does not limit this.

[0245] It should be noted that before collecting the user's relevant data and during the process of collecting the user's relevant data, this application can display a prompt interface, pop-up window or output voice prompt information. The prompt interface, pop-up window or voice prompt information is used to prompt the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining the user's relevant data after obtaining the user's confirmation operation on the prompt interface or pop-up window. Otherwise (that is, when the user's confirmation operation on the prompt interface or pop-up window is not obtained), the relevant steps of obtaining the user's relevant data are terminated, that is, the user's relevant data is not obtained. In other words, all user data collected by this application (including numerical vectors in vector registers) are processed strictly in accordance with the requirements of relevant national laws and regulations. The informed consent or separate consent of the subject of personal information is obtained only with the user's consent and authorization. The collection and subsequent data use and processing are carried out within the scope of authorization of laws and regulations and the subject of personal information, and the collection, use and processing of relevant user data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0246] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

[0247] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A connected domain labeling circuit, It is characterized in that The connected domain marking circuit comprises m processing unit arrays and a label processing module, the difference between the number of processing units corresponding to the nth row and the number of processing units corresponding to the n-1th row in the processing unit array is 1, and m and n are integers greater than 1; The m processing unit arrays are used to obtain an input pixel vector, the input pixel vector includes m sub-pixel vectors, the width corresponding to the i-th sub-pixel vector in the m sub-pixel vectors is the same as the width of the i-th processing unit array in the m processing unit arrays, the input pixel vector includes a plurality of continuous pixels in the same row of the input image, and i is a positive integer; The i-th processing unit array is used to assign labels to the i-th sub-pixel vector to obtain temporary labels corresponding to each pixel in the i-th sub-pixel vector; The label processing module is used to merge the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain the final labels corresponding to each pixel in the input pixel vector; wherein the final label is used to obtain the connected domain in the input pixel vector, and each pixel belonging to the same connected domain in the input pixel vector has the same final label, and different connected domains in the input pixel vector correspond to different final labels.

2. The connected domain marking circuit according to claim 1, It is characterized in that The m processing unit arrays run in parallel to respectively assign labels to the m sub-pixel vectors to obtain temporary labels corresponding to respective pixels in the m sub-pixel vectors.

3. The connected domain marking circuit according to claim 1, It is characterized in that The first row of the processing unit array includes p first-type processing units, and the nth row includes p-n+1 second-type processing units, where p is an integer greater than 1; The p first-type processing units in the i-th processing unit array are used to perform threshold segmentation on each pixel in the i-th sub-pixel vector to obtain binary data corresponding to each pixel in the i-th sub-pixel vector, where the binary data is a first value or a second value, and the p first-type processing units run in parallel; The p first-type processing units in the i-th processing unit array are further used to assign labels to the pixels in the i-th sub-pixel vector according to the binary data and reference labels corresponding to the pixels in the i-th sub-pixel vector, so as to obtain temporary labels corresponding to the pixels in the i-th sub-pixel vector, wherein the reference labels are related to adjacent pixels of the pixel, and the distance between the adjacent pixels of the pixel and the pixel is less than a distance threshold; Each second-type processing unit in the i-th processing unit array is used to transfer the temporary label and binary data corresponding to each pixel in the i-th sub-pixel vector, and obtain the output corresponding to each second-type processing unit, wherein the output includes the temporary label and the binary data.

4. The connected domain marking circuit according to claim 3, It is characterized in that The qth first-category processing unit among the p first-category processing units is configured to, when the binary data corresponding to the qth pixel in the ith sub-pixel vector is a first value, obtain the number of valid labels corresponding to the qth pixel according to the reference label corresponding to the qth pixel, where the number of valid labels refers to the number of valid labels, the label value of the valid label is greater than the first value, and q is a positive integer not greater than p; The qth first-category processing unit is also used to, when the number of valid labels is equal to 0, determine the first numerical value as the label value of the temporary label of the qth pixel; or, when the number of valid labels is equal to 1, determine the valid label as the temporary label of the qth pixel; or, when the number of valid labels is greater than 1, determine the valid label with the smallest label value as the temporary label of the qth pixel.

5. The connected domain marking circuit according to claim 4, It is characterized in that In the case where the qth pixel is the first pixel in the input image, the qth pixel does not correspond to a reference label; or, When the qth pixel is not the first pixel in the input image and is located in the first row of the input image, the qth pixel corresponds to a reference label, and the label value of the reference label is the first value or the second value, and the first value and the second value are less than or equal to 1; or, In a case where the qth pixel is not located in the first row of the input image and is not located in the first column of the input image, the qth pixel corresponds to at least two reference labels, and the reference labels are determined based on the final labels assigned to the adjacent pixels; or, In the case where the qth pixel is not located in the first row of the input image but in the first column of the input image, the qth pixel corresponds to at least one reference label, and the reference label is determined based on the final label assigned to the adjacent pixel.

6. The connected domain marking circuit according to claim 4, It is characterized in that The qth first-category processing unit is further configured to, when the number of valid tags is greater than 1, determine conflicting tags from the valid tags, and there are equivalent tags for the conflicting tags; Sorting the conflicting labels to obtain a conflicting label sequence; The conflicting label sequence is used to update the conflicting label list, and the conflicting label list is used to count the conflicting labels.

7. The connected domain marking circuit according to claim 3, It is characterized in that The first row of second-type processing units in the i-th processing unit array is used to transfer the outputs of the p first-type processing units in the i-th processing unit array to obtain the outputs of the first row of second-type processing units; The second-type processing unit in the t-th row of the ith processing unit array is used to transfer the output of the second-type processing unit in the t-1-th row of the ith processing unit array to obtain the output of the second-type processing unit in the t-th row, where t is a positive integer less than n; Among them, the second type of processing unit takes the outputs of two input processing units as input, and takes the output of the two input processing units with the smaller label value as output, and the input processing unit refers to the processing unit in the row above the row where the second type of processing unit is located and is connected to the second type of processing unit.

8. The connected domain marking circuit according to claim 1, It is characterized in that The label processing module is further used to determine the isolated processing units corresponding to the m processing unit arrays respectively according to the output of each processing unit in the m processing unit arrays, wherein the isolated processing unit refers to a processing unit that meets the isolation condition, and the identification data of the isolated processing unit includes the temporary label, the row and column where the isolated processing unit is located, and the output of the processing unit includes binary data and the final label; The label processing module is further used to merge the identification data of the isolated processing units respectively corresponding to the m processing unit arrays according to the connectivity between the pixels in the input pixel vector, so as to obtain the merged identification data respectively corresponding to the m processing unit arrays, wherein the merged identification data is used to indicate the position of the pixels included in the connected domain in the input pixel vector; The label processing module is also used to assign labels to the isolated processing units corresponding to the m processing unit arrays according to the combined identification data corresponding to the m processing unit arrays, so as to obtain the final labels corresponding to each pixel in the input pixel vector.

9. The connected domain marking circuit according to claim 8, It is characterized in that For each processing unit in the m processing unit array, the label processing module is also used to determine the processing unit as the isolated processing unit when the output of the processing unit includes the first value and the output of the adjacent processing unit corresponding to the processing unit in the row where the processing unit is located includes the second value.

10. The connected domain marking circuit according to claim 8, It is characterized in that The tag processing module is further used to construct a sub-processing unit array corresponding to the isolated point processing unit with the isolated processing unit as the vertex and the first row of the processing unit array as the bottom; For a first sub-processing unit array and a second sub-processing unit array corresponding to the m processing unit arrays, the label processing module is further used for merging identification data of isolated processing units corresponding to the first sub-processing unit array with identification data of isolated processing units corresponding to the second sub-processing unit array if there are pixels in the first sub-processing unit array adjacent to pixels in the second sub-processing unit array, so as to obtain merged identification data corresponding to the first sub-processing unit array and the second sub-processing unit array; The combined representation data is determined based on identification data of the first sub-processing unit array and identification data with a smaller tag value among the identification data of the second sub-processing unit array.

11. The connected domain marking circuit according to claim 8, It is characterized in that The label processing module is further used for assigning a new label to the isolated processing unit as a final label of pixels processed by the sub-processing unit array corresponding to the isolated processing unit when the temporary label corresponding to the isolated processing unit is the first value; or, The label processing module is also used to determine the temporary label included in the merged identification data of the isolated processing unit as the final label of the pixel processed by the sub-processing unit array corresponding to the isolated processing unit when the label value of the temporary label corresponding to the isolated processing unit is greater than the first value.

12. The connected domain marking circuit according to claim 8, It is characterized in that The label processing module is further used to construct a conflict label list according to the merge identification data respectively corresponding to the m processing unit arrays and the conflict label sequence corresponding to the first type of processing unit in the processing unit array; Among them, the conflicting label list is used to merge labels after obtaining the final labels corresponding to each pixel in the input image, so as to obtain the merged labels corresponding to each pixel in the input image, and the pixels in the connected domain in the input image have the same merged labels.

13. The connected domain marking circuit according to claim 3, It is characterized in that The qth first-type processing unit among the p first-type processing units is further configured to determine the first value as binary data corresponding to the qth pixel when the pixel value of the qth pixel is greater than a first threshold; or, The qth first-type processing unit among the p first-type processing units is further used to determine the second value as the binary data corresponding to the qth pixel when the pixel value of the qth pixel is less than or equal to the first threshold.

14. The connected domain marking circuit according to claim 1, It is characterized in that The connected domain marking circuit also includes a pixel vector acquisition module; The pixel vector acquisition module is used for, when acquiring the input image, dividing the input image row by row based on the total width of the m processing unit arrays to obtain a plurality of the input pixel vectors; or, The pixel vector acquisition module is used for, when acquiring an original input pixel vector, dividing the original input pixel vector by the total width of the m processing unit arrays as a unit to obtain a plurality of the input pixel vectors; Wherein, when the width of the input pixel vector is smaller than the total width of the m processing unit arrays, the input pixel vector is padded with a second value.

15. The connected domain marking circuit according to claim 1, It is characterized in that In the case where each row of the input image corresponds to a plurality of input pixel vectors, the m processing unit arrays process the plurality of input pixel vectors in a pipeline manner; Among them, the m processing unit arrays obtain an input pixel vector in each clock cycle. When the x-th input pixel vector is processed by the first row processing unit in the current clock cycle, the x+1-th input pixel vector is input into the first row processing unit in the next clock cycle of the current clock cycle, and x is a positive integer.

16. The connected domain marking circuit according to claim 15, It is characterized in that The m processing unit arrays are further used to synchronize the outputs of the input pixel vectors in different clock cycles to the last clock cycle corresponding to the input pixel vector, so as to obtain the outputs of the m processing unit arrays for the input pixel vector in the same clock cycle; The output under the same clock cycle is transmitted to the label processing module.

17. The connected domain marking circuit according to claim 15, It is characterized in that The processing time of the processing unit is one clock cycle.

18. A chip, It is characterized in that The chip comprises the connected domain marking circuit as claimed in any one of claims 1 to 17.

19. A processor, It is characterized in that The processor comprises the connected domain marking circuit according to any one of claims 1 to 17.

20. A connected domain marking method based on a connected domain marking circuit, It is characterized in that The connected domain marking circuit includes m processing unit arrays and a label processing module, the difference between the number of processing units corresponding to the nth row and the number of processing units corresponding to the n-1th row in the processing unit array is 1, and m and n are integers greater than 1; the method includes: The m processing unit arrays acquire an input pixel vector, the input pixel vector includes m sub-pixel vectors, the width corresponding to the i-th sub-pixel vector in the m sub-pixel vectors is the same as the width of the i-th processing unit array in the m processing unit arrays, the input pixel vector includes a plurality of continuous pixels in the same row of the input image, and i is a positive integer; The i-th processing unit array performs label assignment on the i-th sub-pixel vector to obtain temporary labels corresponding to each pixel in the i-th sub-pixel vector; The label processing module merges the temporary labels corresponding to each pixel in the m sub-pixel vectors to obtain the final labels corresponding to each pixel in the input pixel vector; wherein the final label is used to obtain the connected domain in the input pixel vector, and each pixel belonging to the same connected domain in the input pixel vector has the same final label, and different connected domains in the input pixel vector correspond to different final labels.

Images