An Unsupervised Image Segmentation Method Based on Multi-Representation Superpixels

Abstract

This invention belongs to the field of image segmentation technology, specifically relating to an unsupervised image segmentation method based on multi-representation superpixels. To address the problems of poor applicability, insufficient application, and weak stability in existing unsupervised image segmentation methods, this method includes: (1) calculating the superpixel partitioning of the input image; (2) calculating the feature representation of each pixel block in the superpixel representation; (3) calculating the pairwise distance matrix of the superpixel blocks; (4) calculating the local density of each superpixel block; (5) calculating the representativeness of each superpixel block; (6) selecting representative superpixel blocks; (7) performing image pre-segmentation guided by representative superpixel blocks; and (8) using hierarchical clustering to merge similar segmentation regions to obtain the final image segmentation result.

Description

Technical Field

[0001] This invention belongs to the field of image segmentation technology, specifically relating to an unsupervised image segmentation method based on multi-representation superpixels. Background Technology

[0002] Computer vision is one of the hottest research areas in computer science. Image segmentation is a key technology in computer vision, serving as the foundation for many computer vision tasks such as image analysis, image understanding, and object detection. Currently, image segmentation technology is widely used in fields such as medical image analysis, automatic control, and remote sensing image processing. The main purpose of image segmentation is to divide an image into multiple non-overlapping regions, such that each region is composed of similar features, such as color and texture. Image segmentation can be divided into supervised image segmentation and unsupervised image segmentation. Unsupervised image segmentation does not rely on training samples and image labels, thus having a wider range of applications. However, the unsupervised nature also brings challenges in terms of segmentation performance, stability, and robustness.

[0003] Currently, unsupervised image segmentation methods can be categorized into clustering, thresholding, and edge detection. Thresholding uses statistical techniques to set a fixed threshold, then separates the image into non-overlapping regions based on this threshold. Common strategies include learning the threshold based on pixels and local or global information, and frequently used features include image grayscale histogram features or global texture features. Thresholding is only suitable for images with significant differences in grayscale information. Edge detection detects the edges of objects in an image by analyzing local changes in the image's surface. Edge types typically include four categories: step edges, line edges, slope edges, and ridge edges. Edge detection is relatively complex, limiting its application in real-world scenarios. Clustering-based image segmentation methods have received widespread attention since their inception and are currently one of the more effective image segmentation methods. There are two main clustering strategies: hard clustering and soft clustering. Representative clustering methods in image segmentation are the KM algorithm and the FCM algorithm. Both algorithms first initialize representative pixels, then assign other pixels to the regions represented by the representative pixels, update the representative pixels, and iteratively execute these steps until convergence is met. Commonly used convergence conditions include reaching a preset number of iterations or the change in a representative pixel being less than a preset value. These methods are sensitive to noise, and the segmentation results are affected by the initial representative pixels, resulting in poor stability.

[0004] The above analysis shows that unsupervised image segmentation mainly faces the following challenges: thresholding methods can only handle images with large differences in grayscale information, resulting in poor applicability; edge detection methods are relatively complex and have insufficient applicability; clustering methods are easily affected by noise and initial representative pixels, resulting in weak stability.

[0005] In summary, this invention proposes an unsupervised image segmentation method based on multi-representation superpixels. This method uses multi-representation to improve the stability of image segmentation and uses superpixels to enhance the efficiency of image segmentation, thus having high practical value. Summary of the Invention

[0006] To address the above problems, this invention provides an unsupervised image segmentation method based on multi-representation superpixels.

[0007] To achieve the above objectives, the present invention employs the following technical solutions:

[0008] An unsupervised image segmentation method based on multi-representation superpixels includes the following steps:

[0009] Step 1: Calculate the superpixel partitioning of the input image using the SLIC method, i.e., SLIC(I)→S, where each pixel of the input image is a three-dimensional vector represented by the values ​​of the (R, G, B) channels, i.e., p i =(v i R ,v i G ,v i B );

[0010] Step 2: Calculate the feature representation of each superpixel block in the superpixel representation;

[0011] Step 3: Calculate the pairwise distance matrix of the superpixel blocks;

[0012] Step 4: Calculate the local density of each superpixel block;

[0013] Step 5: Calculate the representativeness of each superpixel block;

[0014] Step 6: Select representative superpixel blocks;

[0015] Step 7: Perform image pre-segmentation guided by representative superpixel blocks;

[0016] Step 8: Use hierarchical clustering to merge similar segmented regions to obtain the final image segmentation result.

[0017] Furthermore, in step 2, the feature representation of each superpixel block in the superpixel representation is calculated, and the specific steps are as follows:

[0018] Step 2.1, calculate the set of pixels corresponding to each superpixel block, i.e., {p j |j∈{S j ==i}};

[0019] Step 2.2, the superpixel block is represented by the feature average value of the superpixel point set, i.e. in

[0020] Step 2.3, similarly calculate and obtain and v' i B .

[0021] Furthermore, in step 3, the pairwise distance matrix of the superpixel blocks is calculated, and the specific steps are as follows:

[0022] Treating the hyperblock as the processing object, if the number of hyperblocks is n, then calculate an n×n distance matrix D, and the hyperblock p' i and p' j The distance is calculated using formula (1):

[0023]

[0024] Furthermore, in step 4, the local density of each superpixel block is calculated, and the specific steps are as follows:

[0025] superpixel block p' i The local density is set as the average number of nearest neighbors within its neighborhood radius across the three channels (R, G, B), i.e.:

[0026]

[0027] in, Within the ε radius of the R channel The number of nearest neighbors, Within the ε radius of the G channel The number of nearest neighbors, Within the ε radius of the G channel The number of nearest neighbors, ε, takes the value of:

[0028]

[0029] in, for The floor function is denoted by n, where n represents the number of superpixel blocks. The calculation method is shown in formula (4):

[0030]

[0031] Where |{·}| represents the number of elements in the set {·}, and the result can be calculated similarly. and

[0032] Furthermore, in step 5, the representativeness of each superpixel block is calculated, and the specific steps are as follows:

[0033] Superpixel block p' iA representative calculation is shown in Formula 5):

[0034] r(p' i )=ρ(p' i )×δ(p' i (5)

[0035] in,

[0036] Furthermore, in step 6, representative superpixel blocks are selected, and the specific steps are as follows:

[0037] Sort the representative values ​​r(p') of all superpixel blocks in descending order, and select the first value in the sorted order. The superpixel block is taken as the representative superpixel block and denoted as .

[0038] Furthermore, in step 7, image pre-segmentation is performed using representative superpixel blocks as guidance. The specific steps are as follows:

[0039] Step 7.1: Initialize the segmentation regions. Each representative superpixel block represents a segmentation region. The i-th segmentation region c i Initialize to c i ={z i};

[0040] Step 7.2, assign the superpixel block to the nearest segmentation region, superpixel block p' i The nearest partitioned region is calculated using formula (7):

[0041]

[0042] Assume p' i The nearest partitioned region is c j , will p' i Assigned to c j This is achieved by formula (8):

[0043] c j =p' i ∪c j (8)

[0044] Step 7.3: Based on formulas (7) and (8), all superpixel blocks are assigned to the partitioned regions to obtain the pre-segmentation result.

[0045] Furthermore, in step 8, hierarchical clustering is used to merge similar segmented regions to obtain the final image segmentation result. The specific steps are as follows:

[0046] Taking the image region in the pre-segmentation result as the object, the size is calculated as The similarity matrix D C'Divide the region into c' i and c' j The similarity is calculated using formula (9):

[0047]

[0048] D C' The final segmentation region number k is input into the shortest connection hierarchical clustering to obtain the final image segmentation result, i.e., HC. s (D C' ,k)→C.

[0049] Compared with the prior art, the present invention has the following advantages:

[0050] The method of the present invention has two advantages. First, the method processes superpixel blocks, which can reduce the number of computational objects and improve the color image segmentation rate. Second, unlike KM and FCM methods, the method does not require randomly specifying initial representative points. Instead, it determines multiple representative points by calculating representativeness, thus obtaining stable image segmentation results. Attached Figure Description

[0051] Figure 1 This is a flowchart of the unsupervised image segmentation method based on multi-representation superpixels of the present invention;

[0052] Figure 2 This is a schematic diagram of the framework of the unsupervised image segmentation method based on multi-representation superpixels of the present invention;

[0053] Figure 3 This is a segmentation result diagram of the FCM method, KM method, and the method of the present invention. Detailed Implementation

[0054] Example 1

[0055] like Figure 1 and Figure 2 As shown, an unsupervised image segmentation method based on multi-representation superpixels includes the following steps:

[0056] In this embodiment, the number of segmented regions k is set to 2.

[0057] Step 1: Calculate the superpixel partitioning of the input image using the SLIC method, i.e., SLIC(I)→S, where each pixel of the input image is a three-dimensional vector represented by the values ​​of the (R, G, B) channels. Superpixels are pixel blocks composed of adjacent pixels with similar characteristics. The advantage of superpixels is that they replace a large number of pixels with a small number of pixel blocks, which reduces the complexity of subsequent processing.

[0058] Step 2, calculate the feature representation of each superpixel block in the superpixel representation:

[0059] Step 2.1, calculate the set of pixels corresponding to each superpixel block, i.e., {p j |j∈{S j ==i}};

[0060] Step 2.2, the superpixel block is represented by the feature average value of the superpixel point set, i.e. Among them, the R channel

[0061] Step 2.3, similarly calculate and obtain the G channel. B Channel

[0062] Step 3, calculate the pairwise distance matrix of the superpixel blocks:

[0063] Treating the hyperblock as the processing object, if the number of hyperblocks is n (n is set to 400), then calculate an n×n distance matrix D, and the hyperblock p' i and p' j The distance is calculated using formula (1):

[0064]

[0065] Step 4: Calculate the local density of each superpixel block. The specific steps are as follows:

[0066] superpixel block p' i The local density is set as the average number of nearest neighbors within its neighborhood radius across the three channels (R, G, B), i.e.:

[0067]

[0068] in, Within the ε radius of the R channel The number of nearest neighbors, Within the ε radius of the G channel The number of nearest neighbors, Within the ε radius of the G channel The number of nearest neighbors, ε, takes the value of:

[0069]

[0070] in, for The floor function is denoted by n, where n represents the number of superpixel blocks. The calculation method is shown in formula (4):

[0071]

[0072] Where |{·}| represents the number of elements in the set {·}, and can be calculated similarly:

[0073]

[0074] Step 5: Calculate the representativeness of each superpixel block:

[0075] With superpixel block p' i For example, a representative calculation is shown in formula (5):

[0076] r(p' i )=ρ(p' i )×δ(p' i (5)

[0077] in,

[0078] Step 6, select Representative superpixel blocks:

[0079] Sort the representative values ​​r(p') of all superpixel blocks in descending order, and select the first value in the sorted order. The superpixel block is taken as the representative superpixel block and denoted as .

[0080] Step 7: Perform image pre-segmentation guided by representative superpixel blocks:

[0081] Step 7.1: Initialize the segmentation regions. Each representative superpixel block represents a segmentation region. The i-th segmentation region c i Initialize to c i ={z i};

[0082] Step 7.2, assign the superpixel block to the nearest segmentation region, with superpixel block p' i For example, first find the nearest partitioned region, and calculate using formula (7):

[0083]

[0084] Where k represents the number of segmented regions;

[0085] Assume p' i The nearest partitioned region is c j , will p' i Assigned to c j This is achieved by formula (8):

[0086] c j =p' i ∪c j (8)

[0087] Step 7.3: Based on formulas (7) and (8), all superpixel blocks are assigned to the partitioned regions to obtain the pre-segmentation result.

[0088] Step 8: Merge similar segmented regions using hierarchical clustering to obtain the final image segmentation result:

[0089] Taking the image region in the pre-segmentation result C' as the object, calculate the size as The similarity matrix D C' Divide the region into c' i and c' j The similarity is calculated using formula (9):

[0090]

[0091] D C' The final segmentation region number k is input into the shortest connection hierarchical clustering to obtain the final image segmentation result, i.e., HC. s (D C' ,k)→C.

[0092] For example Figure 3 The following table shows the running times of the input image, the FCM method, the KM method, and the method of this invention:

[0093] FCM KM Method of the present invention Time elapsed (seconds) 1.684 0.556 0.132

[0094] Figure 3 Examples of segmentation results from the FCM method, the KM method, and the method of this invention are shown. There are differences between FCM result example 1 and FCM result example 2, and there are differences between KM result example 1 and KM result example 2. The method of this invention can stably obtain segmentation results as shown in the example of the method of this patent.

Claims

1. An unsupervised image segmentation method based on multi-representation superpixels, characterized in that, Includes the following steps: Step 1: Calculate the superpixel partitioning of the input image using the SLIC method, i.e., SLIC(I) → S, where 'SLIC()' is a function representation of the simple linear iterative clustering algorithm. The input is the image to be segmented, I, and the output is the superpixel partitioning result, S. Each pixel in the input image is a three-dimensional vector represented by RGB channel values, i.e. ; Step 2: Calculate the feature representation of each superpixel block in the superpixel representation; Step 3: Calculate the pairwise distance matrix of the superpixel blocks; The specific steps for calculating the pairwise distance matrix of the superpixel blocks in step 3 are as follows: Treating the hyperblock as the processing object, if the number of hyperblocks is n, then calculate an n×n distance matrix D, and the hyperblock p' i and p' j The distance is calculated using formula (1): Step 2: Calculate the feature representation of each superpixel block in the superpixel representation; Step 3: Calculate the pairwise distance matrix of the superpixel blocks; The specific steps for calculating the pairwise distance matrix of the superpixel blocks in step 3 are as follows: Treating the hyperblock as the processing object, if the number of hyperblocks is n, then calculate an n×n distance matrix D, and the hyperblock p' i and p' j The distance is calculated using formula (1): ； Step 4: Calculate the local density of each superpixel block; Step 5: Calculate the representativeness of each superpixel block; Step 6: Select representative superpixel blocks; Step 7: Perform image pre-segmentation guided by representative superpixel blocks; Step 8: Use hierarchical clustering to merge similar segmented regions to obtain the final image segmentation result; Step 7 involves image pre-segmentation guided by representative superpixel blocks. The specific steps are as follows: Step 7.1: Initialize the segmentation region, where each representative superpixel block represents a segmentation region; where z i Let c represent the i-th representative superpixel block and the i-th segmented region. i Initialize to ; Step 7.2, assign superpixel blocks to the nearest segmentation region, superpixel blocks The nearest partitioned region is calculated using formula (7): ； Where argmax represents 'making the distance dis (p i ',z i The largest j value, dis (p i ',z i ) represents a superpixel block With representative superpixel block z i Pairwise distances; Assumption The nearest partitioned region is c j ,Will Assigned to c j This is achieved by formula (8): (8); Step 7.3: Based on formulas (7) and (8), all superpixel blocks are assigned to the partitioned regions to obtain the pre-segmentation result: ； Step 4 involves calculating the local density of each superpixel block. The specific steps are as follows: superpixel blocks The local density is set to the average number of nearest neighbors within its neighborhood radius across the three RGB channels, i.e.: ； in, Within the ε radius of the R channel The number of nearest neighbors, Within the ε radius of the G channel The number of nearest neighbors, Within the ε radius of channel B The number of nearest neighbors, ε, takes the value of: ； in, for The floor function is denoted by n, where n represents the number of superpixel blocks. The calculation method is shown in formula (4): ； Where |{·}| represents the number of elements in the set {·}, and the result can be calculated similarly. and ; Step 5, which calculates the representativeness of each superpixel block, involves the following steps: The representative calculation of superpixel block p'i is shown in Equation (5): ； in, (6); in, Represents the i-th superpixel block The representative metrics are used to measure the qualification of the superpixel block as a representative superpixel block; Represents the i-th superpixel block Local density; Represents the i-th superpixel block The minimum distance to all superpixel blocks with a local density higher than itself; Represents the j-th superpixel block Local density Greater than the i-th superpixel block Local density D(i,j) represents the i-th superpixel block. With the j-th superpixel block Pairwise distances; In step 8, hierarchical clustering is used to merge similar segmented regions to obtain the final image segmentation result. The specific steps are as follows: Taking the image region in the pre-segmentation result as the object, the size is calculated as The similarity matrix D C Divide the area and The similarity is calculated using formula (9): ； in, Indicates the pre-segmented region The b-th superpixel block in the image. Indicates the pre-segmented region The d-th superpixel block in Index of Chinese super pixel blocks Represents superpixel blocks and The pairwise distance; The final segmentation region number k is input into the shortest connection hierarchical clustering to obtain the final image segmentation result, i.e. ; in, This represents the function representation of the shortest connection hierarchical clustering algorithm, with the similarity matrix as input. Given the target segmentation region number k, the output is the final segmentation result C; 'C' represents the set of final segmentation results, whose elements are the final segmentation region numbers corresponding to each superpixel block.

2. The unsupervised image segmentation method based on multi-representation superpixels according to claim 1, characterized in that, The specific steps for calculating the feature representation of each superpixel block in step 2 are as follows: Step 2.1, calculate the set of pixels corresponding to each superpixel block, i.e., {p j |j∈{S j ==i}}, where S j Sj == i represents the superpixel number corresponding to pixel j, and Sj == i means that the superpixel number of pixel j is equal to i. That is, this set represents all pixels p belonging to the i-th superpixel block. j The set; Step 2.2, the superpixel block is represented by the feature average value of the superpixel point set, i.e. ; in Step 2.3, similarly calculate and obtain and .

3. The unsupervised image segmentation method based on multi-representation superpixels according to claim 1, characterized in that, The steps for selecting representative superpixel blocks in step 6 are as follows: Sort the representative values ​​r(p') of all superpixel blocks in descending order, and select the first value in the sorted order. The superpixel block is taken as the representative superpixel block and denoted as . .

Images