Image connected domain labeling method, electronic device and storage medium

Abstract

The present application discloses a method for marking connected domains of an image, an electronic device and a storage medium, which processes the pixels in the image to be processed in parallel based on the computing unified device architecture CUDA; specifically, the foreground pixels and background pixels in the image are determined; for each row of pixels, each group of continuous foreground pixels in the row of pixels is determined, and each group of continuous foreground pixels is used as a sub-connected domain; for any first sub-connected domain and any second sub-connected domain in the image, if any first pixel in the first sub-connected domain is vertically adjacent to any second pixel in the second sub-connected domain, the first sub-connected domain and the second sub-connected domain are merged to obtain each connected domain; and the target label value of each connected domain is determined. The efficiency of connected domain marking is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image connected domain labeling method, electronic device and storage medium. Background Art

[0002] Connected Component Labeling (CCL) is a commonly used algorithm in the field of image processing. It is used to divide the pixels in an image into different connected domains according to their adjacent relationships. Each connected domain consists of a group of adjacent pixels and is assigned a unique number for identification. Connected Component Labeling has a wide range of applications in image processing and computer vision, and can achieve various tasks such as image segmentation, target detection and recognition, video analysis, feature extraction, and image restoration.

[0003] In the prior art, connected domain labeling is generally performed based on a central processing unit (CPU). This method is limited by the characteristics of CPU serial calculations, and it will be too time-consuming when processing large-sized images, making the efficiency of image connected domain labeling low. Summary of the invention

[0004] The present application provides an image connected domain labeling method, an electronic device and a storage medium, which are used to solve the problem of low efficiency of image connected domain labeling in the prior art.

[0005] In a first aspect, the present application provides a method for marking an image connected domain, the method comprising:

[0006] The pixels in the image to be processed are processed in parallel based on the Compute Unified Device Architecture (CUDA); the processing process includes:

[0007] Determine foreground pixels and background pixels in the image; for each row of pixels, determine each group of continuous foreground pixels in the row of pixels, and use each group of continuous foreground pixels as a sub-connected domain;

[0008] For any first sub-connected domain and any second sub-connected domain in the image, if any first pixel point in the first sub-connected domain is vertically adjacent to any second pixel point in the second sub-connected domain, the first sub-connected domain and the second sub-connected domain are merged to obtain each connected domain; and a target label value of each connected domain is determined.

[0009] Furthermore, after taking each group of continuous foreground pixel points as a sub-connected domain, the method further comprises:

[0010] For each sub-connected domain, determine the minimum index value of the pixels contained in the sub-connected domain; add a preset value to the minimum index value to obtain a first label value of the sub-connected domain;

[0011] Determining the target tag value of each connected domain includes:

[0012] For each of the connected domains, each of the sub-connected domains included in the connected domain is determined, and the minimum first label value in each of the sub-connected domains is determined as the target label value of the connected domain.

[0013] Furthermore, the method further comprises:

[0014] A background connected domain in the image is determined, wherein the background connected domain includes the background pixel points; and a label value of the background connected domain is determined to be 0.

[0015] Furthermore, after determining the target tag value of each connected domain, the method further includes:

[0016] For each connected domain, determine a first number of pixels included in the connected domain; and determine whether the first number is within a preset range of numbers. If so, keep the target mark value of the connected domain unchanged; if not, update the target mark value of the connected domain to 0.

[0017] Furthermore, after determining the target tag value of each connected domain, the method further includes:

[0018] For each of the connected domains, determine a structure corresponding to the connected domain; wherein the structure includes a target tag value, area information, and coordinate information in the image of the connected domain;

[0019] Based on the structures corresponding to the connected domains, an image processing task is performed; wherein the image processing task includes at least one of an image segmentation task, a target detection task, a target recognition task, a feature extraction task and an image restoration task.

[0020] Furthermore, based on the structures corresponding to the connected domains, performing image processing tasks includes:

[0021] Determine a first structure array according to the structures corresponding to the connected domains; wherein, for each structure, determine the storage position of the structure in the first structure array according to the target tag value included in the structure;

[0022] Perform image processing tasks according to the first structure array.

[0023] Furthermore, after determining the first structure array, the method further includes:

[0024] Determine a flag array corresponding to the first structure array; wherein the flag array includes flags of each storage position of the first structure array, wherein flag 1 represents that the storage position stores a structure, and flag 0 represents that the storage position does not store a structure;

[0025] For each structure in the first structure array, determine the storage position of the structure in the second structure array according to the sum of the flags before the structure in the flag array, and obtain the second structure array;

[0026] The second structure array is used as the updated first structure array, and an image processing task is performed according to the updated first structure array.

[0027] In a second aspect, the present application provides an image connected domain labeling device, the device comprising:

[0028] A first determination module is used to determine foreground pixels and background pixels in the image; for each row of pixels, determine each group of continuous foreground pixels in the row of pixels, and use each group of continuous foreground pixels as a sub-connected domain;

[0029] The second determination module is used to merge, for any first sub-connected domain and second sub-connected domain in the image, the first sub-connected domain and the second sub-connected domain to obtain respective connected domains if any first pixel point in the first sub-connected domain is vertically adjacent to any second pixel point in the second sub-connected domain; and determine the target mark value of each connected domain.

[0030] The first determination module is further used to determine, for each sub-connected domain, a minimum index value of pixels contained in the sub-connected domain; and add a preset value to the minimum index value to obtain a first label value of the sub-connected domain;

[0031] The second determination module is further configured to determine, for each connected domain, each sub-connected domain included in the connected domain, and determine the minimum first label value in each sub-connected domain as the target label value of the connected domain.

[0032] The second determination module is further used to determine a background connected domain in the image, wherein the background connected domain includes the background pixels; and determine that the mark value of the background connected domain is 0.

[0033] The second determination module is also used to determine, for each connected domain, a first number of pixel points contained in the connected domain; determine whether the first number is within a preset number range, if so, keep the target mark value of the connected domain unchanged, if not, update the target mark value of the connected domain to 0.

[0034] The second determination module is also used to determine, for each connected domain, a structure corresponding to the connected domain; wherein the structure includes a target mark value, area information and coordinate information in the image of the connected domain; and perform an image processing task based on the structure corresponding to each connected domain; wherein the image processing task includes at least one of an image segmentation task, a target detection task, a target recognition task, a feature extraction task and an image restoration task.

[0035] The second determination module is specifically used to determine a first structure array according to the structures corresponding to each connected domain; wherein, for each structure, the storage position of the structure in the first structure array is determined according to the target tag value included in the structure; and the image processing task is performed according to the first structure array.

[0036] The second determination module is also used to determine the flag array corresponding to the first structure array; wherein the flag array includes flags of each storage position of the first structure array, wherein flag 1 represents that the storage position stores a structure, and flag 0 represents that the storage position does not store a structure; for each structure in the first structure array, according to the sum of the previous flags of the structure in the flag array, determine the storage position of the structure in the second structure array to obtain the second structure array; use the second structure array as the updated first structure array, and perform image processing tasks according to the updated first structure array.

[0037] In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0038] Memory, used to store computer programs;

[0039] The processor is used to implement the method when executing the program stored in the memory.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the described method is implemented.

[0041] In a fifth aspect, the present application provides a computer program product, wherein the computer program product comprises an executable program, and the executable program is executed by a processor to implement the described method.

[0042] The present application provides a method for marking image connected domains based on CUDA, which processes pixels in the image to be processed in parallel based on CUDA. Specifically, the foreground pixels and background pixels in the image are first determined, and then the sub-connected domains of each row of pixels are determined; then the sub-connected domains with vertical adjacent relationships are merged to obtain each connected domain; and the target marking value of each connected domain is determined. Compared with the solution of marking connected domains based on CPU serial, the efficiency of connected domain marking is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A schematic diagram of the first image connected domain labeling process provided in this application;

[0045] Figure 2 A schematic diagram of the second image connected domain labeling process provided in this application;

[0046] Figure 3 A schematic diagram of the third image connected domain labeling process provided in this application;

[0047] Figure 4 A schematic diagram of the fourth image connected domain labeling process provided in this application;

[0048] Figure 5 Schematic diagram of the structure array update process provided by this application;

[0049] Figure 6 Schematic diagram of the tag generation process provided for this application;

[0050] Figure 7 Schematic diagram of the tag merging process provided for this application;

[0051] Figure 8 A schematic diagram of the process of marking and filling provided for this application;

[0052] Fig. 9 Schematic diagram of the connected domain feature extraction process provided in this application;

[0053] Fig.10A schematic diagram of the connected domain filtering process provided for this application;

[0054] Fig.11 Schematic diagram of the data continuous processing process provided for this application;

[0055] Fig.12 A schematic diagram of the structure of the image connected domain marking device provided in this application;

[0056] Fig.13 This is a schematic diagram of the electronic device structure provided in this application. DETAILED DESCRIPTION

[0057] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0058] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0059] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0060] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0061] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0063] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

[0064] Figure 1 This is a schematic diagram of the first image connected domain labeling process provided by this application. This application performs parallel processing on the pixels in the image to be processed based on the Compute Unified Device Architecture (CUDA), thereby realizing image connected domain labeling. Specifically, the following steps are included:

[0065] S101: determining foreground pixels and background pixels in the image; for each row of pixels, determining each group of continuous foreground pixels in the row of pixels, and using each group of continuous foreground pixels as a sub-connected domain;

[0066] S102: For any first sub-connected domain and any second sub-connected domain in the image, if any first pixel point in the first sub-connected domain is vertically adjacent to any second pixel point in the second sub-connected domain, merge the first sub-connected domain and the second sub-connected domain to obtain each connected domain; and determine the target mark value of each connected domain.

[0067] The image connected domain labeling method provided by the present application is applied to an electronic device, which can be a PC, a computer or other device, or a server. CUDA includes multiple thread bundles, each of which includes 32 threads. Each thread bundle processes the pixel points of the corresponding row in the image, and each thread processes a corresponding pixel point.

[0068] The electronic device first determines the foreground pixels and background pixels in the image. Optionally, the foreground pixels and background pixels in the image are determined by a binarization algorithm. Then, for each row of pixels, each group of continuous foreground pixels in the row of pixels is determined, and each group of continuous foreground pixels is used as a sub-connected domain. In this way, the sub-connected domain of each row can be determined. It should be noted that for a row of pixels, if there are no continuous foreground pixels in the row of pixels, there is no sub-connected domain in the row. Continuous foreground pixels refer to foreground pixels that are adjacent to each other on the left and right.

[0069] After determining the sub-connected domains of each row, any two sub-connected domains are respectively used as the first sub-connected domain and the second sub-connected domain. If any first pixel point in the first sub-connected domain is vertically adjacent to any second pixel point in the second sub-connected domain, the first sub-connected domain and the second sub-connected domain are merged. For example, if the pixel point a1 in the sub-connected domain A in the first row is vertically adjacent to the pixel point b1 in the sub-connected domain B in the second row, the sub-connected domain A and the sub-connected domain B are merged. It should be noted that, based on the above example, if the pixel point b10 in the sub-connected domain B in the second row is vertically adjacent to the pixel point C10 in the sub-connected domain C in the third row, the sub-connected domain A, the sub-connected domain B and the sub-connected domain C are merged. And so on, until all the vertically adjacent sub-connected domains are merged, each connected domain in the image is obtained. And determine the target label value of each connected domain. Optionally, different label values ​​can be assigned to each connected domain as the target label value of the connected domain. Preferably, since the label value of background pixels in the image is generally 0, different label values ​​can be assigned to each connected domain as the target non-zero label value of the connected domain. In this way, each connected domain can be distinguished from each other, and each connected domain can also be distinguished from the image background.

[0070] The present application provides a method for marking image connected domains based on CUDA, which processes pixels in the image to be processed in parallel based on CUDA. Specifically, the foreground pixels and background pixels in the image are first determined, and then the sub-connected domains of each row of pixels are determined; then the sub-connected domains with vertical adjacent relationships are merged to obtain each connected domain; and the target marking value of each connected domain is determined. Compared with the solution of marking connected domains based on CPU serial, the efficiency of connected domain marking is improved.

[0071] Figure 2 The second image connected domain labeling process provided in this application includes the following steps:

[0072] S201: determining foreground pixels and background pixels in the image; for each row of pixels, determining each group of continuous foreground pixels in the row of pixels, and using each group of continuous foreground pixels as a sub-connected domain;

[0073] S202: for each sub-connected domain, determining a minimum index value of pixels contained in the sub-connected domain; adding a preset value to the minimum index value to obtain a first label value of the sub-connected domain;

[0074] S203: For any first sub-connected domain and any second sub-connected domain in the image, if any first pixel point in the first sub-connected domain is vertically adjacent to any second pixel point in the second sub-connected domain, merge the first sub-connected domain and the second sub-connected domain to obtain respective connected domains;

[0075] S204: for each connected domain, determine each sub-connected domain included in the connected domain, and determine the minimum first label value in each sub-connected domain as the target label value of the connected domain.

[0076] Each pixel in the image has a unique index value. For example, if the image contains 100 pixels, the index values ​​of these 100 pixels are continuous values ​​from 0 to 99. The index value of the pixel in the upper left corner is the smallest, and the index value of the pixel in the lower right corner is the largest. Each group of continuous foreground pixels is taken as a sub-connected domain, that is, after determining each sub-connected domain in each row, for each sub-connected domain, first determine the minimum index value of the pixels contained in the sub-connected domain, that is, the index value of the first pixel from left to right in the sub-connected domain is the minimum index value. Then add the preset value to the minimum index value to obtain the first label value of the sub-connected domain. The preset value is a positive value, preferably, the preset value is 1. In this way, the first label value of each sub-connected domain in the image can be determined.

[0077] For any first sub-connected domain and any second sub-connected domain in the image, if any first pixel point in the first sub-connected domain is vertically adjacent to any second pixel point in the second sub-connected domain, the first sub-connected domain and the second sub-connected domain are merged to obtain each connected domain. For each connected domain, first determine each sub-connected domain contained in the connected domain. Since each connected domain is obtained by merging each sub-connected domain, each sub-connected domain contained in the connected domain can be determined. Then determine the minimum first label value in each sub-connected domain, and then determine the minimum first label value as the target label value of the connected domain. Thereby improving the accuracy of determining the target label value of the connected domain.

[0078] In the present application, the method further includes: determining a background connected domain in the image, wherein the background connected domain includes the background pixel points; and determining that the mark value of the background connected domain is 0.

[0079] Figure 3 The third image connected domain labeling process provided in this application includes the following steps:

[0080] S301: determining foreground pixels and background pixels in the image; for each row of pixels, determining each group of continuous foreground pixels in the row of pixels, and using each group of continuous foreground pixels as a sub-connected domain;

[0081] S302: for each sub-connected domain, determining a minimum index value of pixels contained in the sub-connected domain; adding a preset value to the minimum index value to obtain a first label value of the sub-connected domain;

[0082] S303: For any first sub-connected domain and any second sub-connected domain in the image, if any first pixel point in the first sub-connected domain is vertically adjacent to any second pixel point in the second sub-connected domain, merge the first sub-connected domain and the second sub-connected domain to obtain respective connected domains;

[0083] S304: for each connected domain, determining each sub-connected domain included in the connected domain, and determining the minimum first label value in each sub-connected domain as the target label value of the connected domain;

[0084] S305: For each connected domain, determine a first number of pixels included in the connected domain; determine whether the first number is within a preset range, if yes, keep the target mark value of the connected domain unchanged, if no, update the target mark value of the connected domain to 0.

[0085] In the present application, the preset number range includes a lower limit value and an upper limit value. For each connected domain, the first number of pixels contained in the connected domain is determined; if the first number of pixels is less than the lower limit value, the connected domain is considered to be too small. At this time, the connected domain is considered to be an interfering connected domain, and the interfering connected domain is filtered out, thereby improving the accuracy of the connected domain marking. If the first number of pixels is greater than the upper limit value, the connected domain is considered to be too large. At this time, the connected domain is considered to be an interfering connected domain, and the interfering connected domain is filtered out, thereby improving the accuracy of the connected domain marking. The process of filtering the interfering connected domain is to update the target mark value of the interfering connected domain to 0. That is, the interfering connected domain is updated to the background in the image.

[0086] For each connected domain, if the first number of pixels included in the connected domain is within a preset number range, the connected domain is considered not to be an interfering connected domain, and the target mark value of the connected domain remains unchanged. The present application filters the interfering connected domain according to the first number of pixels included in the connected domain and the preset number threshold, further improving the accuracy of the connected domain marking.

[0087] In the present application, after determining the target tag value of each connected domain, the method further includes:

[0088] For each of the connected domains, determine a structure corresponding to the connected domain; wherein the structure includes a target tag value, area information, and coordinate information in the image of the connected domain;

[0089] Based on the structures corresponding to the connected domains, an image processing task is performed; wherein the image processing task includes at least one of an image segmentation task, a target detection task, a target recognition task, a feature extraction task and an image restoration task.

[0090] In the present application, after determining each connected domain, the area information of the connected domain and the coordinate information in the image can be determined. Optionally, the number of pixels contained in the connected domain can be used as the area information of the connected domain, or the minimum bounding rectangle corresponding to the connected domain can be determined, and the area information of the minimum bounding rectangle can be used as the area information of the connected domain, that is, the number of pixels contained in the minimum bounding rectangle can be used as the area information of the connected domain, and the coordinate information of the connected domain in the image can also be determined. Optionally, the coordinate information of the minimum bounding rectangle corresponding to the connected domain can be used as the coordinate information of the connected domain in the image. The coordinate information of the minimum bounding rectangle includes the coordinate information of the four vertices.

[0091] For each connected domain, the structure corresponding to the connected domain is determined according to the target mark value, area information and coordinate information of the connected domain in the image. The structure contains the target mark value, area information and coordinate information of the connected domain in the image. Then, based on the structure corresponding to each connected domain, an image processing task is performed; wherein the image processing task includes at least one of an image segmentation task, a target detection task, a target recognition task, a feature extraction task and an image restoration task. The connected domain marking method provided in the present application improves the efficiency and accuracy of the connected domain marking, thereby providing a basic condition for improving the accuracy of the image processing task.

[0092] Figure 4 The fourth image connected domain labeling process provided in this application includes the following steps:

[0093] S401: Determine foreground pixels and background pixels in the image; for each row of pixels, determine each group of continuous foreground pixels in the row of pixels, and use each group of continuous foreground pixels as a sub-connected domain;

[0094] S402: for each sub-connected domain, determining a minimum index value of pixels contained in the sub-connected domain; adding a preset value to the minimum index value to obtain a first label value of the sub-connected domain;

[0095] S403: For any first sub-connected domain and any second sub-connected domain in the image, if any first pixel point in the first sub-connected domain is vertically adjacent to any second pixel point in the second sub-connected domain, merge the first sub-connected domain and the second sub-connected domain to obtain respective connected domains;

[0096] S404: for each connected domain, determining each sub-connected domain included in the connected domain, and determining the minimum first label value in each sub-connected domain as the target label value of the connected domain;

[0097] S405: for each connected domain, determining a first number of pixels included in the connected domain; judging whether the first number is within a preset number range, if yes, keeping the target mark value of the connected domain unchanged; if no, updating the target mark value of the connected domain to 0;

[0098] S406: For each connected domain, determine a structure corresponding to the connected domain; wherein the structure includes a target tag value, area information and coordinate information in the image of the connected domain; based on the structures corresponding to each connected domain, perform an image processing task; wherein the image processing task includes at least one of an image segmentation task, a target detection task, a target recognition task, a feature extraction task and an image restoration task.

[0099] In order to facilitate image processing tasks, in this application, based on the structures corresponding to each connected domain, image processing tasks include:

[0100] Determine a first structure array according to the structures corresponding to the connected domains; wherein, for each structure, determine the storage position of the structure in the first structure array according to the target tag value included in the structure;

[0101] Perform image processing tasks according to the first structure array.

[0102] In the present application, a first structure array is determined according to the structures corresponding to each connected domain. The first structure array stores the structures corresponding to each connected domain. Among them, there is an index value corresponding to each position in the structure, for example, the index values ​​of the storage positions are 0, 1, 2, ... respectively. When each structure is saved in the first structure array, the storage position of the structure in the first structure array is determined according to the target mark value included in the structure. Optionally, the target mark value included in the structure is directly used as the storage position of the structure in the first structure array. For example, if the target mark value included in the structure is 10, the structure is stored in the storage position with an index value of 10 in the first structure array. Optionally, if the target mark value included in the structure is obtained by adding a preset value to the index value of the pixel, the preset value can be subtracted from the target mark value included in the structure, and the difference is used as the storage position of the structure in the first structure array. For example, the target mark value included in the structure is 10, and the target mark value is obtained by adding the index value of the pixel to the preset value, and the preset value is 1. Then, it is determined that the index value of the structure stored in the first structure array is 10-1=9, that is, the structure is stored in the storage location of the index value of 9 in the first structure array. Then, the image processing task is performed according to the first structure array.

[0103] In this application, considering that the storage position of the structure in the first structure array is determined according to the target tag value included in the structure, the structures stored in the first structure array obtained in this way are discontinuous, which is not convenient for subsequent image processing tasks. Based on the above considerations, Figure 5 The schematic diagram of the structure array update process provided for this application includes the following steps:

[0104] S501: Determine a flag array corresponding to the first structure array; wherein the flag array includes flags of each storage position of the first structure array, wherein flag 1 represents that the storage position stores a structure, and flag 0 represents that the storage position does not store a structure;

[0105] S502: for each structure in the first structure array, determine the storage position of the structure in the second structure array according to the sum of the flags before the structure in the flag array, and obtain the second structure array;

[0106] S503: Using the second structure array as the updated first structure array, and performing an image processing task according to the updated first structure array.

[0107] In the present application, the flag array corresponding to the first structure array is first determined, and the flag array includes flags of each storage position of the first structure array. Optionally, flag 1 represents that the storage position stores a structure, and flag 0 represents that the storage position does not store a structure. For each structure in the first structure array, the flag and before the structure in the flag array are determined. For example, the position of the storage position index 100 of the first structure array stores a structure, the position of the storage position index 200 stores a structure, the position of the storage position index 300 stores a structure, and the other storage positions do not store structures. Then in the flag array corresponding to the first structure array, the flag corresponding to the position 100 is 1, the flag corresponding to the position 200 is 1, the flag corresponding to the position 300 is 1, and the flags of other positions are all 0.

[0108] In the present application, for each structure in the first structure array, the storage position of the structure in the second structure array is determined according to the sum of the flags before the structure in the flag array. The structure A stored at the storage position index 100 of the first structure array, the sum of the flags before the structure A in the flag array is 0, therefore, the storage position of the structure A in the second structure array is determined to be 0, so the structure A is stored at the storage position 0 in the second structure array. The structure B stored at the storage position index 200 of the first structure array, the sum of the flags before the structure B in the flag array is 1, therefore, the storage position of the structure B in the second structure array is determined to be 1, so the structure B is stored at the storage position 1 in the second structure array. The structure C stored at the storage position index 300 of the first structure array, the sum of the flags before the structure C in the flag array is 2, therefore, the storage position of the structure C in the second structure array is determined to be 2, so the structure C is stored at the storage position 2 in the second structure array. Thus, the second structure array is obtained. Then the second structure array is used as the updated first structure array. The structures stored in the updated first structure array are continuous, so it is convenient to perform image processing tasks according to the updated first structure array.

[0109] The image connected domain labeling process provided by the present application is described in detail below with reference to the accompanying drawings.

[0110] This application proposes some improvement measures to address the problem of imperfect GPU support for existing connected domain labeling algorithms. First, the connected domain labeling algorithm proposed in this application can be accelerated by GPU. Secondly, the robustness of the algorithm is enhanced, and the connected domain can be correctly marked no matter where it appears. Then, a connected domain filtering function is added, and connected domains that are too large or too small can be directly removed to avoid affecting the final result. Finally, the output results are stored continuously for subsequent use. The connected domain labeling method provided in this application includes the following steps:

[0111] Step 1: The input image is processed row by row, and the index value of the first pixel of consecutive non-zero pixels (foreground pixels) in each row relative to the entire image is increased by one (the preset value is 1) and recorded;

[0112] Step 2: When two adjacent rows belong to the same connected domain, merge the labels of these two rows;

[0113] Step 3: Record the mark value of the first pixel of the continuous non-zero pixels as the mark value of the remaining pixels;

[0114] Step 4: Extract connected domain features, such as connected domain tag value, connected domain area, connected domain coordinates, etc., and record them in the structure;

[0115] Step 5: Filter out smaller or larger connected domains to reduce interference;

[0116] Step 6: Make a copy of the final result and copy the discontinuous data into continuous data for subsequent processing.

[0117] Figure 6 A schematic diagram of the tag generation process provided for this application, such as Figure 6 As shown, it is determined whether a whole row has been traversed. If so, the process ends. If not, the subsequent traversal process is performed based on the computing unified device architecture CUDA.

[0118] Each thread in the thread bundle reads a pixel value in turn, which is p1, and uses the ballot_syns instruction to encode whether the pixel is non-zero into a 32-bit mask, which is passed back to each thread and recorded as pixels1;

[0119] According to the mask value pixels1 and the number of the current thread, calculate the position of the current thread in the continuous non-zero pixel segment, recorded as dist1;

[0120] If the current pixel value is non-zero and the current thread is at the starting position of a continuous non-zero pixel segment, the index value of the pixel is incremented by 1 and recorded;

[0121] Only the first thread records the mask value in the shared memory and waits for other thread warps to finish writing;

[0122] Read the mask value of the previous warp record from the shared memory, recorded as pixels2;

[0123] According to the mask value pixels2 and the number of the current thread, calculate the position of the current thread in the continuous non-zero pixel segment of the previous row, recorded as dist2;

[0124] When p1 and p2 are both non-zero values, and one of dist1 and dist2 is 0, merge the mark values ​​of these two rows, that is, use the smaller mark value to cover the larger mark value;

[0125] The thread block is shifted right by 32 pixels; it is determined whether a whole row has been traversed. If so, the process ends; if not, the above process continues.

[0126] In order to eliminate the vertical mark merging operation, this application uses a loop traversal method to process each row of pixels. At the same time, in order to make full use of the characteristics of CUDA, the number of pixels processed at a time is limited to 32, which is the size of a thread bundle. Internal instructions are used to share data between threads to reduce operations on global memory. At the same time, in order to avoid the problem that the connected domain appears in the upper left corner and is marked as 0 and cannot be distinguished from the background, this application adds 1 to the mark value. When two adjacent rows belong to the same connected domain, a merge operation needs to be performed. The merge operation is based on the union-find algorithm. The specific implementation is shown in Algorithm 1. Here, a minus 1 operation needs to be added to eliminate the impact of the previous addition of 1. The entire process of step one is as follows Figure 6 shown.

[0127] Algorithm 1: merge(L, label l , label 2 )

[0128] while label 1 != label 2 and label 1 !=Lllabel 1 ]-1:

[0129] label 1 =L[label 1 ]-1

[0130] end while

[0131] while label 1 != label 2 and label 2 !=L[label 2 ]-1:

[0132] label 2 =L[label 2 ]-1

[0133] end while

[0134] while label 1 != label 2 :

[0135] if label 1 2 :

[0136] swap(label 1 ,label 2 ) ​

[0137] endif

[0138] label 3 =atomicMin(&Llabel 1 ],label 2 +1)-1

[0139] label 1 =label 1 ==label 3 ? label 2 : label 3

[0140] end while

[0141] Since step one only merges the marks within the thread block and the global marks are not completely merged, step two is introduced to merge the global marks. Figure 7 Schematic diagram of the tag merging process provided for this application.

[0142] The thread warp reads the pixel value of the current row;

[0143] The thread warp reads the pixel value of the previous row;

[0144] Use the ballot_syns instruction to encode whether the current row of pixels read by the thread warp is non-zero into a 32-bit mask and pass it back to each thread, recorded as pixels1;

[0145] Use the ballot_syns instruction to encode whether the previous row of pixels read by the thread warp is non-zero into a 32-bit mask and pass it back to each thread, recorded as pixels2;

[0146] When the pixels of the current row obtained by the current thread are non-zero with the pixels of the previous row and at least one of the pixels is the starting pixel, the upper and lower rows are merged; the process ends.

[0147] After step 2, all the labels belonging to the same connected domain have been merged. However, since the previous step only marked the first pixel in order to save the bandwidth of the global memory, step 3 needs to fill the label value into the entire connected domain to which it belongs. The specific process is as follows Figure 8 As shown, the process of finding the root node mark is shown in Algorithm 2, where a minus 1 operation is required. Figure 8 A schematic diagram of the mark filling process provided for this application.

[0148] The thread warp reads the current row pixel value;

[0149] Use the ballot_syns instruction to encode whether the current row of pixels read by the thread warp is non-zero into a 32-bit mask and pass it back to each thread, recorded as pixels;

[0150] According to the mask value pixels and the number of the current thread, calculate the position of the current thread in the continuous non-zero pixel segment, recorded as dist;

[0151] When the pixel value is not empty and dist==0, find the root node mark of the set to which it belongs;

[0152] When the pixel value is not empty, obtain the mark corresponding to the first pixel of the continuous non-zero pixels in the thread bundle, add 1 and record it at the current index; the process ends.

[0153] Algorithm 2: find(L, label)

[0154] while label != L[label]-1:

[0155] label=L[label]-1

[0156] end while

[0157] After completing the connected domain marking, it is necessary to analyze the marked connected domains. The analysis operation is mainly to obtain the description information of each connected domain, such as the mark value, area size and coordinates of the minimum circumscribed matrix of this connected domain, and then save this information into the structure array for use by other image processing task algorithms. The algorithm flow is as follows Fig. 9 As shown, the feature extraction method is shown in Algorithm 3. Fig. 9 Schematic diagram of the connected domain feature extraction process provided in this application.

[0158] Each thread in the thread bundle reads a pixel value in turn, and uses the ballot_syns instruction to encode whether the pixel is non-zero as a 32-bit mask, which is passed back to each thread and recorded as pixels;

[0159] According to the mask value pixels and the number of the current thread, calculate the position of the current thread in the continuous non-zero pixel segment, recorded as dist;

[0160] Determine whether the current pixel value is non-empty and dist=0; if not, the process ends; if so, perform the following steps.

[0161] Get the tag at the current index and find the tag of the root node of the set to which the tag belongs, recorded as label;

[0162] Determine whether the marked value is the same as the current index value; if so, add 1 to the label and record it in the result of the current index; if not, record the size of the marked area and the coordinates of the upper left corner and lower right corner respectively, and the process ends.

[0163] Algorithm 3: feature_extraction(label_info, count, label, index, x, y)

[0164] if label == index:

[0165] label_info[label].label = label + 1

[0166] end if

[0167] atomicAdd(&label_info[label].area, count)

[0168] atomicMin(&label_info[label].rect.ul.x, x)

[0169] atomicMin(&label_info[label].rect.ul.y, y)

[0170] atomicMax(&label_info[label].rect.lr.x, count + x)

[0171] atomicMax(&label_info[label].rect.lr.y, y + 1)

[0172] Since the incoming image is generally not very pure, there will be many connected domains that are too large or too small (such as unimportant small targets or backgrounds) in the marking results, which will interfere with the subsequent algorithms. Therefore, step five mainly realizes the function of connected domain filtering. The flag corresponding to the connected domain that meets the area requirement is set to 1, and the rest are kept to 0. At the same time, the mark is also set to 0, so as to achieve the purpose of clearing the marks of the connected domains that do not meet the requirements and recording the connected domains that meet the requirements. Fig.10 Schematic diagram of the connected domain filtering process provided in this application.

[0173] Read the tag value minus one and record it as label; determine whether label is greater than or equal to 0; if not, the process ends; if so, determine whether the area of ​​the current tag is less than the minimum value or greater than the maximum value; if not, continue to determine whether the label value is equal to the index value; if the area of ​​the current tag is less than the minimum value or greater than the maximum value, set the tag value to 0; and continue to determine whether the label value is equal to the index value; if not, the process ends; if so, set the flag under the current index to 1 and the process ends.

[0174] On the one hand, due to the characteristics of CUDA parallel operation, and on the other hand, due to filtering, the connected domain features finally obtained are discontinuous, and it is not convenient for subsequent algorithms to directly use discontinuous data. Therefore, the purpose of step six is ​​to convert discontinuous data into continuous data by copying, and then pass it to the subsequent algorithm. This step is mainly implemented based on the prefix sum algorithm that does not contain itself. Fig.11 The schematic diagram of the data continuous processing process provided for this application is as follows: Fig.11 As shown, the prefix sum of the flag array is calculated to obtain the offset value of each element in the entire queue; the elements are copied according to the corresponding offset values ​​so that they are stored continuously.

[0175] This application proposes a CUDA-based image connected domain labeling algorithm, which reduces the interference of irrelevant elements in the image on subsequent algorithms by calculating the area size of each connected domain in the feature extraction stage and eliminating connected domains that do not meet the area requirements in the connected domain filtering stage, thereby achieving higher algorithm accuracy. It can effectively label connected domains at any position, has stronger robustness, and can correctly label the connected domain located in the upper left corner of the image, and can distinguish it from the background. In order to improve the operating efficiency of the GPU, the extracted connected domain features are stored in a discrete form, but such a storage form is not conducive to subsequent processing. Therefore, this application adds a discrete data continuity operation, which not only ensures the operating efficiency but also ensures the convenience of subsequent algorithm processing.

[0176] Fig.12 This is a schematic diagram of the structure of the image connected domain marking device provided in this application, and the device includes:

[0177] The first determination module 121 is used to determine foreground pixels and background pixels in the image; for each row of pixels, determine each group of continuous foreground pixels in the row of pixels, and use each group of continuous foreground pixels as a sub-connected domain;

[0178] The second determination module 122 is used for merging, for any first sub-connected domain and any second sub-connected domain in the image, the first sub-connected domain and the second sub-connected domain to obtain respective connected domains if any first pixel point in the first sub-connected domain is vertically adjacent to any second pixel point in the second sub-connected domain; and determining the target mark value of each connected domain.

[0179] The first determination module 121 is further used to determine, for each sub-connected domain, a minimum index value of pixels contained in the sub-connected domain; and add a preset value to the minimum index value to obtain a first label value of the sub-connected domain;

[0180] The second determining module 122 is further configured to determine, for each connected domain, each sub-connected domain included in the connected domain, and determine the minimum first label value in each sub-connected domain as the target label value of the connected domain.

[0181] The second determining module 122 is further configured to determine a background connected domain in the image, wherein the background connected domain includes the background pixels; and determine that a mark value of the background connected domain is 0.

[0182] The second determination module 122 is also used to determine, for each connected domain, a first number of pixel points contained in the connected domain; determine whether the first number is within a preset number range, if so, keep the target mark value of the connected domain unchanged, if not, update the target mark value of the connected domain to 0.

[0183] The second determination module 122 is also used to determine, for each connected domain, a structure corresponding to the connected domain; wherein the structure includes a target mark value, area information and coordinate information in the image of the connected domain; and perform an image processing task based on the structure corresponding to each connected domain; wherein the image processing task includes at least one of an image segmentation task, a target detection task, a target recognition task, a feature extraction task and an image restoration task.

[0184] The second determination module 122 is specifically used to determine a first structure array according to the structures corresponding to the connected domains; wherein, for each structure, the storage position of the structure in the first structure array is determined according to the target tag value included in the structure; and the image processing task is performed according to the first structure array.

[0185] The second determination module 122 is also used to determine the flag array corresponding to the first structure array; wherein the flag array includes flags of each storage position of the first structure array, wherein flag 1 represents that the storage position stores a structure, and flag 0 represents that the storage position does not store a structure; for each structure in the first structure array, the storage position of the structure in the second structure array is determined according to the sum of the previous flags of the structure in the flag array, so as to obtain the second structure array; the second structure array is used as the updated first structure array, and the image processing task is performed according to the updated first structure array.

[0186] The present application also provides an electronic device, such as Fig.13 As shown, it includes: a processor 131, a communication interface 132, a memory 133 and a communication bus 134, wherein the processor 131, the communication interface 132, and the memory 133 communicate with each other through the communication bus 134;

[0187] The memory 133 stores a computer program, and when the program is executed by the processor 131, the processor 131 executes any of the above method steps.

[0188] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0189] The communication interface 132 is used for communication between the above electronic device and other devices.

[0190] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0191] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0192] The present application also provides a computer storage readable storage medium, wherein the computer readable storage medium stores a computer program executable by an electronic device, and when the program runs on the electronic device, the electronic device implements any of the above method steps when executing.

[0193] The present application provides a computer program product, wherein the computer program product comprises an executable program, and when the executable program is executed by a processor, the method described above is implemented.

[0194] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0195] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for marking connected domains in an image, characterized in that: The method comprises: The pixels in the image to be processed are processed in parallel based on the Compute Unified Device Architecture (CUDA); the processing process includes: Determine foreground pixels and background pixels in the image; for each row of pixels, determine each group of continuous foreground pixels in the row of pixels, and use each group of continuous foreground pixels as a sub-connected domain; For any first sub-connected domain and any second sub-connected domain in the image, if any first pixel point in the first sub-connected domain is vertically adjacent to any second pixel point in the second sub-connected domain, merge the first sub-connected domain and the second sub-connected domain to obtain each connected domain; and determine the target label value of each connected domain; Based on the structures corresponding to the connected domains, image processing tasks include: Determine a first structure array according to the structures corresponding to the connected domains; wherein, for each structure, determine the storage position of the structure in the first structure array according to the target tag value included in the structure; Performing an image processing task according to the first structure array; After determining the first structure array, the method further includes: Determine a flag array corresponding to the first structure array; wherein the flag array includes flags of each storage position of the first structure array, wherein flag 1 represents that the storage position stores a structure, and flag 0 represents that the storage position does not store a structure; For each structure in the first structure array, determine the storage position of the structure in the second structure array according to the sum of the flags before the structure in the flag array, and obtain the second structure array; The second structure array is used as the updated first structure array, and an image processing task is performed according to the updated first structure array.

2. The method according to claim 1, characterized in that After taking the groups of continuous foreground pixels as sub-connected domains, the method further comprises: For each sub-connected domain, determine the minimum index value of the pixels contained in the sub-connected domain; add a preset value to the minimum index value to obtain a first label value of the sub-connected domain; Determining the target tag value of each connected domain includes: For each of the connected domains, each of the sub-connected domains included in the connected domain is determined, and the minimum first label value in each of the sub-connected domains is determined as the target label value of the connected domain.

3. The method according to claim 1, characterized in that The method further comprises: A background connected domain in the image is determined, wherein the background connected domain includes the background pixel points; and a label value of the background connected domain is determined to be 0.

4. The method according to claim 3, characterized in that After determining the target tag value of each connected domain, the method further includes: For each connected domain, determine a first number of pixels included in the connected domain; and determine whether the first number is within a preset range of numbers. If so, keep the target mark value of the connected domain unchanged; if not, update the target mark value of the connected domain to 0.

5. The method according to claim 1, characterized in that After determining the target tag value of each connected domain, the method further includes: For each of the connected domains, determine a structure corresponding to the connected domain; wherein the structure includes a target tag value, area information, and coordinate information in the image of the connected domain; Based on the structures corresponding to the connected domains, an image processing task is performed; wherein the image processing task includes at least one of an image segmentation task, a target detection task, a target recognition task, a feature extraction task and an image restoration task.

6. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method according to any one of claims 1 to 5 when executing a program stored in a memory.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer program product, characterized in that The computer program product comprises an executable program, which is executed by a processor to implement the method according to any one of claims 1 to 5.

Images