Image superpixel segmentation method and device, storage medium and computer program product

Abstract

The embodiment of the invention provides an image superpixel segmentation method and device, a storage medium and a computer program product, and the method comprises the steps: obtaining N first pixel points corresponding to a first image, dividing the first image based on the N first pixel points, and obtaining K superpixel blocks corresponding to the first image; wherein each superpixel block comprises N / K first pixel points, N is a positive integer, and K is an integer greater than 1; aiming at the K super-pixel blocks, adjusting the K super-pixel blocks based on the connectivity between the pixel points, so that the types of the pixel points contained in each super-pixel block are the same, and obtaining M super-pixel blocks; wherein M is an integer greater than 1; and the boundaries of the M superpixel blocks are updated, so that the distribution precision of boundary pixels is improved, and the segmentation precision of the image is further improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image superpixel segmentation method, device, storage medium and computer program product. Background Art

[0002] Image superpixel segmentation is a regional image segmentation technology that divides an image into several superpixel blocks with similar internal features based on local similarity. In practical applications, image superpixel segmentation technology can reduce the amount of image data while retaining key structural features, thereby reducing the complexity of the image in the subsequent processing process; secondly, image superpixel segmentation technology can maintain and enhance the edge information of the image, and improve the accuracy of subsequent image feature extraction, classification and other tasks. Therefore, image superpixel segmentation technology is usually used as an image preprocessing method, which lays the foundation for subsequent image analysis and processing by providing more efficient and accurate processing units.

[0003] However, current image superpixel segmentation technologies, such as the Simple Linear Iterative Clustering (SLIC) method, require humans to give the expected number of superpixel blocks. If this number is set too large, the superpixel segmentation accuracy will be reduced, while if it is set too small, the computational complexity will increase in the subsequent image processing process. In addition, the SLIC method uses the average value of all pixels in the superpixel block as the center of the superpixel block. When a superpixel block contains pixels of multiple categories, its mean will be offset, which will affect the subsequent pixel allocation, thereby reducing the image segmentation accuracy. Summary of the invention

[0004] The embodiments of the present application provide a method, device, storage medium and computer program product for image superpixel segmentation, which can adaptively adjust the size of superpixel blocks and improve the image segmentation accuracy.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides an image superpixel segmentation method, the method comprising:

[0007] Obtaining N first pixel points corresponding to a first image, and dividing the first image based on the N first pixel points to obtain K super pixel blocks corresponding to the first image; wherein each super pixel block includes N / K first pixel points, N is a positive integer, and K is an integer greater than 1;

[0008] For the K superpixel blocks, based on the connectivity between the pixels, the K superpixel blocks are adjusted so that the categories of the pixels contained in each superpixel block are the same, thereby obtaining M superpixel blocks; wherein M is an integer greater than 1;

[0009] The boundaries of the M superpixel blocks are updated.

[0010] In a second aspect, an embodiment of the present application provides an image superpixel segmentation device, the image superpixel segmentation device comprising: an acquisition unit, a division unit, an adjustment unit, and an update unit; wherein:

[0011] The acquisition unit is used to acquire N first pixel points corresponding to the first image; wherein N is a positive integer;

[0012] The division unit is used to divide the first image based on the N first pixel points to obtain K super pixel blocks corresponding to the first image; wherein each super pixel block includes N / K first pixel points, and K is an integer greater than 1;

[0013] The adjusting unit is used to adjust the K superpixel blocks based on the connectivity between the pixels, so that the categories of the pixels included in each superpixel block are the same, and M superpixel blocks are obtained; wherein M is an integer greater than 1;

[0014] The updating unit is used to update the boundaries of the M super pixel blocks.

[0015] In a third aspect, an embodiment of the present application provides an image superpixel segmentation device, the image superpixel segmentation device comprising: a processor and a memory; wherein:

[0016] The memory is used to store a computer program that can be run on the processor;

[0017] The processor is used to execute the above-mentioned image superpixel segmentation method when running the computer program.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program code stored thereon. When the computer program code is executed by a computer, the image superpixel segmentation method as described above is implemented.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the image superpixel segmentation method as described above.

[0020] An embodiment of the present application provides an image superpixel segmentation method, device, storage medium and computer program product, the method comprising: obtaining N first pixel points corresponding to a first image, and dividing the first image based on the N first pixel points to obtain K superpixel blocks corresponding to the first image; wherein each superpixel block includes N / K first pixel points, N is a positive integer, and K is an integer greater than 1; for the K superpixel blocks, based on the connectivity between the pixel points, the K superpixel blocks are adjusted so that the categories of the pixel points contained in each superpixel block are the same, and M superpixel blocks are obtained; wherein M is an integer greater than 1; and the boundaries of the M superpixel blocks are updated. It can be seen that after obtaining the K superpixel blocks corresponding to the first image, the embodiment of the present application can adaptively adjust the K superpixel blocks based on the connectivity between the pixels, so that the categories of the pixels contained in each superpixel block are the same. That is, the embodiment of the present application avoids the need to manually give the expected number of superpixel blocks, and can adaptively adjust the size of the superpixel blocks based on the connectivity between the pixels, so that the categories of the pixels contained in each superpixel block are the same, thereby improving the allocation accuracy of the boundary pixels when updating the boundaries of the M superpixel blocks, thereby improving the image segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The image superpixel segmentation method proposed in the embodiment of the present application is shown in FIG. Figure 1 ;

[0022] Figure 2 The image superpixel segmentation method proposed in the embodiment of the present application is shown in FIG. Figure 2 ;

[0023] Figure 3 This is a schematic diagram of the image preprocessing results proposed in the embodiment of the present application;

[0024] Figure 4 The structure diagram of the image superpixel segmentation device proposed in the embodiment of the present application is shown in FIG. Figure 1 ;

[0025] Figure 5 The structure diagram of the image superpixel segmentation device proposed in the embodiment of the present application is shown in FIG. Figure 2 . DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It is understood that the specific embodiments described herein are only used to explain the related applications, rather than to limit the applications. It should also be noted that, for ease of description, only the parts related to the related applications are shown in the drawings.

[0027] Image superpixel segmentation is a regional image segmentation technology that divides an image into several superpixel blocks with similar internal features based on local similarity. In practical applications, superpixel segmentation can reduce the amount of image data while retaining key structural features, thereby reducing the complexity of the image in subsequent processing. Secondly, superpixel segmentation can maintain and enhance the edge information of the image, improving the accuracy of subsequent image feature extraction, classification and other tasks. Therefore, image superpixel segmentation is usually used as an image preprocessing method, which lays the foundation for subsequent image analysis and processing by providing more efficient and accurate processing units.

[0028] The current image superpixel segmentation technology mainly includes SLIC and Linear Spectral Clustering (LSC); among them, the SLIC method uniformly initializes the expected number of superpixel blocks and their centers on the image, and then assigns the pixels in the overlapping search area of ​​each superpixel block center to the superpixel block corresponding to the nearest center, then calculates and updates the superpixel block center to make it equal to the average value of all pixels belonging to the superpixel block, and finally the above assignment and update steps will be repeated until the predetermined convergence condition is reached. The LSC method maps the image pixels to a ten-dimensional feature space through a kernel function, and then uniformly samples the expected number of seed pixels on the image as the search center, and its feature vector is used as the initial weighted mean of the corresponding superpixel block. Next, each pixel in the image is assigned to the superpixel block corresponding to the weighted mean closest to its feature vector, and the search center and weighted mean of the superpixel block are updated. Finally, the above assignment and update steps will be repeated until the predetermined convergence condition is reached.

[0029] However, the SLIC method requires artificially given the expected number of superpixel blocks. If this number is set too large, the superpixel segmentation accuracy will decrease, and if it is set too small, the computational complexity will increase in the subsequent image processing process. In addition, SLIC uses the average value of all pixels in the superpixel block as the center of the superpixel block. When a superpixel block contains pixels of multiple categories, its mean will be offset, which will affect the subsequent pixel distribution. Finally, although SLIC considers both the feature information and spatial position information of the pixel, it does not consider the different scales and distributions of the two types of information, but artificially adjusts the weights of the two types of information, which may cause a certain feature to have too large a weight on the subsequent distance calculation, thereby masking the influence of another feature on the distance.

[0030] In order to solve the problem that the current image superpixel segmentation technology causes the image segmentation accuracy to decrease, the embodiments of the present application provide an image superpixel segmentation method, device, storage medium and computer program product, the method comprising: obtaining N first pixel points corresponding to a first image, and dividing the first image based on the N first pixel points to obtain K superpixel blocks corresponding to the first image; wherein each superpixel block includes N / K first pixel points, N is a positive integer, and K is an integer greater than 1; for the K superpixel blocks, based on the connectivity between the pixel points, the K superpixel blocks are adjusted so that the categories of the pixel points contained in each superpixel block are the same, and M superpixel blocks are obtained; wherein M is an integer greater than 1; and the boundaries of the M superpixel blocks are updated. It can be seen that after obtaining the K superpixel blocks corresponding to the first image, the embodiment of the present application can adaptively adjust the K superpixel blocks based on the connectivity between the pixels, so that the categories of the pixels contained in each superpixel block are the same. That is, the embodiment of the present application avoids the need to manually give the expected number of superpixel blocks, and can adaptively adjust the size of the superpixel blocks based on the connectivity between the pixels, so that the categories of the pixels contained in each superpixel block are the same, thereby improving the allocation accuracy of the boundary pixels when updating the boundaries of the M superpixel blocks, thereby improving the image segmentation accuracy.

[0031] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0032] The present application provides an image superpixel segmentation method. Figure 1 The image superpixel segmentation method proposed in the embodiment of the present application is shown in FIG. Figure 1 ,like Figure 1 As shown, the image superpixel segmentation method may include the following steps:

[0033] Step 101, obtain N first pixel points corresponding to a first image, and divide the first image based on the N first pixel points to obtain K super pixel blocks corresponding to the first image; wherein each super pixel block includes N / K first pixel points, N is a positive integer, and K is an integer greater than 1.

[0034] In an embodiment of the present application, the image superpixel segmentation device can obtain N first pixel points corresponding to the first image, and divide the first image based on the N first pixel points to obtain K superpixel blocks corresponding to the first image.

[0035] It should be noted that, in the embodiments of the present application, N is a positive integer, which means that N is an integer greater than 1 or equal to 1, and the present application does not specifically limit the size of N.

[0036] It should be noted that, in the embodiments of the present application, K is an integer greater than 1, which means that K is an integer greater than 2 or equal to 2, and the present application does not specifically limit the size of K.

[0037] Exemplarily, in an embodiment of the present application, the image superpixel segmentation device may obtain N first pixel points corresponding to the first image, assuming that the first pixel point is {x 1 ,x 2 ,…,x N} T , then the first image can be approximately evenly divided into K superpixel blocks of almost equal size {U 1 ,U 2 ,…,U K}; where each superpixel block U i It contains approximately N / K pixels.

[0038]

[0039] Wherein, S represents the number of super pixel blocks obtained by division, N represents the number of first pixel points, and K represents an arbitrary integer set.

[0040] It should be noted that, in the embodiment of the present application, for each first pixel point, considering the spatial position information of each first pixel point in the image, its feature can be expanded to x i =[x i1 ,…,x iV ,y i ,z i ] T ; where [x i1 ,…,x iV ] T is pixel x i V visual features, [y i ,z i ] T Pixel x i The spatial position of the feature in the image coordinate system.

[0041] It should be noted that in an embodiment of the present application, when the image superpixel segmentation device divides the first image into superpixel blocks, it is not necessary to determine the initial clustering center of each superpixel block, but directly divides the first image evenly based on the above-set grid interval, so that each superpixel block is distributed as evenly as possible in space.

[0042] Step 102: For the K superpixel blocks, based on the connectivity between the pixels, adjust the K superpixel blocks so that the categories of the pixels contained in each superpixel block are the same, and obtain M superpixel blocks; wherein M is an integer greater than 1.

[0043] In an embodiment of the present application, after the image superpixel segmentation device obtains N first pixel points corresponding to the first image and divides the first image based on the N first pixel points to obtain K superpixel blocks corresponding to the first image, the K superpixel blocks can be adjusted based on the connectivity between the pixel points so that the pixel points contained in each superpixel block are of the same category, thereby obtaining M superpixel blocks.

[0044] It should be noted that, in an embodiment of the present application, when the image superpixel segmentation device adjusts K superpixel blocks based on the connectivity between pixel points, it can determine, for each superpixel block in the K superpixel blocks, a second pixel point set corresponding to the superpixel block based on the first pixel point included in the superpixel block; wherein the second pixel point set includes some pixel points in the first pixel point, and the second pixel point set is used to characterize the connectivity between the first pixel points; then, based on the second pixel point set, it can be determined whether the categories of the first pixel points included in the superpixel block are the same to obtain a judgment result; and then the K superpixel blocks can be adjusted based on the judgment result.

[0045] It should be noted that, in an embodiment of the present application, the second pixel point set represents a highly connected pixel set corresponding to the superpixel block; wherein the highly connected pixel set represents the connectivity between the first pixel points, that is, in order to ensure that each superpixel block only contains pixels of the same category, it is necessary to ensure that at multiple observation scales, the pixels in the highly connected pixel set are unique. Therefore, the embodiment of the present application can determine whether the categories of the first pixel points contained in the superpixel block are the same based on the second pixel point set to obtain a judgment result; and then the K superpixel blocks can be adaptively adjusted based on the judgment result.

[0046] That is to say, in an embodiment of the present application, the image superpixel segmentation device can measure the consistency of pixels within the superpixel block based on the second pixel point set of the superpixel block, and adaptively update the superpixel block size according to the second pixel point set; so that the superpixel segmentation no longer depends on the artificially given number of superpixel blocks, and at the same time more effectively maintains the consistency of pixels within the superpixel block, ensuring that subsequent image analysis and processing tasks have higher accuracy.

[0047] It should be noted that in an embodiment of the present application, when the image superpixel segmentation device determines the second pixel point set corresponding to the superpixel block based on the first pixel point contained in the superpixel block, it can determine the first similarity matrix corresponding to the superpixel block based on the first pixel point for each superpixel block; wherein the first similarity matrix is ​​used to characterize the similarity between each pixel point in the superpixel block and P adjacent pixel points; wherein P is a positive integer; and then the second pixel point set corresponding to the superpixel block can be determined based on the first similarity matrix.

[0048] It should be noted that in an embodiment of the present application, when the image superpixel segmentation device determines the first similarity matrix corresponding to the superpixel block based on the first pixel point for each superpixel block, it can determine the similarity between each pixel point in the superpixel block and P adjacent pixels based on a preset distance measurement method and a preset function; wherein the preset distance measurement method at least includes the Mahalanobis distance measurement method, and the preset function at least includes the Gaussian kernel function, and then the first similarity matrix can be determined based on the similarity.

[0049] Exemplarily, in an embodiment of the present application, when the image superpixel segmentation device determines the similarity between each pixel point in the superpixel block and P adjacent pixels based on a preset distance measurement method and a preset function, it can be calculated by the following formula (2).

[0050]

[0051] Among them, D(x i ,x j ) represents different pixel points x i and x j The difference between Represents x i The pixel point distance is the set of the first k nearest neighbor pixels. Represents x j The pixel point distance is the set of the k nearest neighboring pixels.

[0052] It should be noted that, in the embodiment of the present application, the embodiment of the present application can represent D(x i ,x j ), can be calculated by the following formula (3).

[0053]

[0054] Among them, H -1 represents the inverse of the covariance matrix.

[0055] It should be noted that, in the embodiment of the present application, the covariance matrix H can be calculated by the following formula (4).

[0056]

[0057] in, Represents the average feature of all N first pixels.

[0058] It should be noted that, in the embodiments of the present application, by introducing the covariance matrix between features, the contribution between features of different scales and distributions can be naturally adjusted, so that visual features and spatial position features are fairly considered in the distance metric.

[0059] It should be noted that, in the embodiments of the present application, the present application adopts the Mahalanobis distance measurement method to represent D(x i ,x j ), it can prevent a certain feature from having too much weight on the distance calculation, thereby masking the influence of another feature on the distance.

[0060] It should be noted that in the embodiment of the present application, after the image superpixel segmentation device determines the similarity between each pixel point in the superpixel block and P adjacent pixels based on a preset distance measurement method and a preset function, it can then determine a first similarity matrix based on the similarity. The first similarity matrix can be obtained by the following formula (5).

[0061]

[0062] Among them, W i Represents the first similarity matrix.

[0063] It should be noted that, in the embodiment of the present application, the super pixel block U i The similarity matrix W of all pixels in the graph represented by the K-Nearest Neighbor (KNN) algorithm i (First similarity matrix), compared with the traditional similarity matrix based on the fully connected graph representation, the embodiment of the present application adopts the KNN graph representation to construct the similarity matrix, and each pixel point only has a similarity greater than 0 with its surrounding k (P) pixels. Therefore, the first similarity matrix represented by the above formula (5) is usually sparse. Such sparsity can effectively reduce the spatial complexity of the similarity matrix of each superpixel block when the number of superpixel blocks K is very small and the total number of pixels N in the image is very large, thereby reducing the time consumption of subsequent operations.

[0064] It should be noted that, in the embodiment of the present application, after determining the first similarity matrix based on the similarity, the image superpixel segmentation device can further determine the second pixel point set corresponding to the superpixel block based on the first similarity matrix.

[0065] It should be noted that, in an embodiment of the present application, when the image superpixel segmentation device determines the second pixel point set corresponding to the superpixel block based on the first similarity matrix, it can generate a second similarity matrix corresponding to the superpixel block based on the first similarity matrix of different powers; then, a first preset operation can be performed on each row element in the second similarity matrix to obtain a first operation value; then, S elements in each row element of the second similarity matrix that are greater than or equal to the corresponding first operation value can be screened out, and a second pixel point set can be generated based on the S elements; wherein S is a positive integer.

[0066] Exemplarily, in an embodiment of the present application, the image superpixel segmentation device can generate a second similarity matrix corresponding to the superpixel block based on first similarity matrices of different powers, and the second similarity matrix can be shown as the following formula (6).

[0067]

[0068] in, represents the second similarity matrix, is a matrix The value at the jjth index describes the pixel x j The T-order connectivity.

[0069] It should be noted that, in an embodiment of the present application, after the image superpixel segmentation device generates a second similarity matrix corresponding to the superpixel block based on a first similarity matrix of different powers, it can perform a first preset operation on each row element in the second similarity matrix to obtain a first operation value. For example, the first preset operation can be to add and sum the absolute values ​​of each row element in the second similarity matrix to obtain the first operation value. The present application does not specifically limit the type of the first preset operation.

[0070] Further, in an embodiment of the present application, when the image superpixel segmentation device screens out S elements in each row of the second similarity matrix that are greater than or equal to the corresponding first operation value, the second similarity matrix The diagonally dominant row in is determined, and the image superpixel segmentation device can filter out S elements in each row that are greater than or equal to the corresponding first operation value through the following formula (7), and then generate a second pixel point set based on the S elements.

[0071]

[0072] in, represents the first operation value, Represents S elements.

[0073] It should be noted that in an embodiment of the present application, after screening out S elements in the second similarity matrix, the image superpixel segmentation device can determine its target pixel point in the corresponding superpixel block through the S elements, and the target pixel point is a highly connected pixel of the superpixel block, and then a second pixel point set can be generated based on the target pixel point; wherein the target pixel point is a part of the first pixel point.

[0074] It should be noted that, in an embodiment of the present application, when the image superpixel segmentation device determines whether the categories of the first pixel points included in the superpixel block are the same based on the second pixel point set, when the power increases from the first set value to the second set value, it determines whether the number of occurrence frequencies of each element in the second pixel point set is equal to the first preset threshold; if the number of occurrence frequencies of each element in the second pixel point set is equal to the first preset threshold, it is determined that the categories of the first pixel points included in the superpixel block are the same; if the number of occurrence frequencies of each element in the second pixel point set is less than or greater than the first preset threshold, it is determined that the categories of the first pixel points included in the superpixel block are different.

[0075] It should be noted that, in the embodiments of the present application, the first set value and the second set value represent the power values ​​of the first similarity matrix, and the present application does not specifically limit the sizes of the first set value and the second set value.

[0076] It should be noted that, in the embodiment of the present application, the first preset threshold may be 1.

[0077] Exemplarily, in the embodiment of the present application, as shown in the following formula (8), when the power is changed from T 1 Increase to T n , determine whether the number of occurrence frequencies of each element in the second pixel point set is equal to 1; if the number of occurrence frequencies of each element in the second pixel point set is equal to 1, it is determined that the categories of the first pixel points included in the superpixel block are the same; if the number of occurrence frequencies of each element in the second pixel point set is less than or greater than 1, it is determined that the categories of the first pixel points included in the superpixel block are different.

[0078]

[0079] Where T represents the power, f(|C (T) |) represents the occurrence frequency of each element in the second pixel point set.

[0080] Furthermore, in an embodiment of the present application, after the image superpixel segmentation device determines whether the categories of the first pixel points included in the superpixel block are the same based on the second pixel point set, it can dynamically divide the size of the superpixel block based on the judgment result, so that the categories of the pixel points included in the divided superpixel blocks are the same, and M superpixel blocks are obtained.

[0081] It should be noted that in an embodiment of the present application, if the image superpixel segmentation device determines that the categories of the first pixel points contained in the superpixel blocks are different, the size of each superpixel block in the K superpixel blocks can be dynamically divided based on the judgment result, so that the categories of the pixel points contained in the divided superpixel blocks are the same, and M superpixel blocks are obtained.

[0082] Exemplarily, in the embodiment of the present application, after the image superpixel segmentation device determines that the categories of the first pixel points included in the superpixel blocks are different, the K superpixel blocks can be further divided. For example, the K superpixel blocks {U 1 ,U 2 ,…,U K} is approximately evenly divided into m 2 A superpixel block of almost the same size but smaller in area So that the divided super pixel block The categories of the pixels included are the same, and M super pixel blocks are obtained.

[0083]

[0084] That is to say, in an embodiment of the present application, after the image superpixel segmentation device divides the first image based on N first pixel points to obtain K superpixel blocks corresponding to the first image, for each superpixel block in the K superpixel blocks, a second pixel point set corresponding to the superpixel block can be determined based on the first pixel points contained in the superpixel block; the second pixel point set is used to characterize the connectivity between the first pixel points; then, based on the second pixel point set, it is determined whether the categories of the first pixel points contained in the superpixel blocks are the same to obtain a judgment result. Assuming that the judgment result characterizes that the categories of the first pixel points contained in the superpixel blocks are different, the K superpixel blocks can be further divided until the categories of the pixel points contained in the divided superpixel blocks are the same, that is, the embodiment of the present application can adaptively adjust the size of the superpixel block based on the second pixel point set, and make each superpixel block contain as many pixels of the same category as possible, so that when the superpixel block boundary is updated, the pixel allocation accuracy of the superpixel block search area can be greatly improved.

[0085] Step 103: Update the boundaries of the M superpixel blocks.

[0086] In an embodiment of the present application, an image superpixel segmentation device adjusts K superpixel blocks based on the connectivity between pixel points so that the categories of pixel points contained in each superpixel block are the same. After obtaining M superpixel blocks, the boundaries of the M superpixel blocks can be updated.

[0087] It should be noted that in the embodiments of the present application, the natural boundaries of the image are often complex and irregular, which may cause some superpixel blocks to span multiple object boundaries; in order to enable the M superpixel blocks to better describe the actual boundaries in the image and retain more image structure information, the embodiments of the present application can reallocate pixels in the overlapping search area around the M superpixel blocks and update the boundaries of each superpixel block.

[0088] Furthermore, in an embodiment of the present application, when the image superpixel segmentation device updates the boundaries of M superpixel blocks, each first pixel point in the L superpixel blocks can be assigned to the corresponding M-1 superpixel blocks to obtain M-1 division results; then, the optimal division result corresponding to each first pixel point in the L superpixel blocks can be determined based on the normalized cut (Ncut) index value corresponding to the M-1 division results; wherein the L superpixel blocks are partial superpixel blocks in the M superpixel blocks.

[0089] It should be noted that, in the embodiment of the present application, the L super pixel blocks may be super pixel blocks in the M super pixel block search area. For example, the search area range size may be 2S. i ×2S i , this application does not specifically limit the size of the search area.

[0090] Exemplarily, in an embodiment of the present application, the image superpixel segmentation device may allocate each first pixel point in L superpixel blocks to corresponding M-1 superpixel blocks to obtain M-1 types of segmentation results. Assume that each pixel point x in the image i , if it is in M ​​superpixel blocks {U 1 ,U 2 ,…,U M}, then it must belong to a super pixel block U among these M super pixel blocks. i , at this time, the pixel point x i Temporarily assigned to the remaining M-1 superpixel blocks U 1 ,…U i-1 ,U i+1 ,…,U M , and then M-1 kinds of division results can be formed, and the M-1 kinds of division results are shown in the following formula (10).

[0091]

[0092] Among them, {U 1 ,x i} means to convert x i Assigned to superpixel block U 1 Middle,U i \xi Indicates that x i Temporarily leave the U i Superpixel block culling, {U M ,x i} means to convert x i Assigned to superpixel block U M middle.

[0093] Furthermore, in an embodiment of the present application, after obtaining M-1 division results, the image superpixel segmentation device can determine the optimal division result corresponding to each first pixel point in the L superpixel blocks based on the Ncut index value corresponding to the M-1 division results.

[0094] It should be noted that, in an embodiment of the present application, when the image superpixel segmentation device determines the optimal division result corresponding to each first pixel point in L superpixel blocks based on the normalized cut Ncut index value corresponding to M-1 division results, it can determine the initial division result corresponding to each first pixel point in the L superpixel blocks; then, the optimal division result corresponding to each first pixel point in the L superpixel blocks can be determined based on the Ncut index value corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result.

[0095] For example, in an embodiment of the present application, the image superpixel segmentation device may first determine the initial segmentation result corresponding to each first pixel point in the L superpixel blocks. Then, the optimal division result corresponding to each first pixel point in the L superpixel blocks can be determined based on the Ncut index value corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result. The Ncut index value corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result can be calculated respectively by the following formula (11), that is, the rationality of all M division results can be measured.

[0096]

[0097] in, Can be expressed w jk Indicates the division result The pixel x in the complement of k and x j The similarity of It's U l In the division results The complement of .

[0098] It should be noted that, in the embodiments of the present application, Vol(U l ) can be expressed by the following formula (12).

[0099]

[0100] Among them, w jk Indicates the division result Middle pixel x j and x k The similarity.

[0101] It should be noted that in an embodiment of the present application, when the image superpixel segmentation device determines the optimal division result corresponding to each first pixel point in L superpixel blocks based on the Ncut index value corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result, it can determine the minimum Ncut index value among the Ncut index values ​​corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result; then, the target pixel block assigned to each first pixel point can be determined based on the division result corresponding to the minimum Ncut index value; and then, the optimal division result corresponding to each first pixel point can be determined based on the target pixel block.

[0102] For example, in the embodiment of the present application, as shown in the above formula (11), the smaller the Ncut index value is, the higher the similarity of the pixels inside each superpixel block in the division result is, and the lower the similarity of the pixels between the superpixel blocks is, the more reasonable the division result is. Therefore, the embodiment of the present application can calculate the total division result by The Ncut index value, where the minimum Ncut index value corresponds to the partition result It means pixel x i Assigned to superpixel block U j (ie the target pixel block) is the most reasonable.

[0103] It should be noted that, in the embodiment of the present application, the image superpixel segmentation device can allocate each first pixel point in the L superpixel blocks to the corresponding M-1 superpixel blocks to obtain M-1 division results, and then the Ncut index value corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result (i.e. Ncut index value), and then the division result corresponding to the minimum Ncut index value can be selected, and then the optimal division result corresponding to each first pixel point can be determined based on the division result corresponding to the minimum Ncut index value.

[0104] That is to say, in an embodiment of the present application, the image superpixel segmentation device can determine the final pixel allocation result by using the normalized cut Ncut index value of different segmentation results formed after allocating pixel points to different superpixel blocks; so that the pixel allocation process takes into account the overall similarity between the pixel to be allocated and the superpixel block, rather than the similarity with a single central pixel, thereby improving the accuracy of pixel allocation.

[0105] To summarize, after the image superpixel segmentation device divides the first image based on N first pixel points to obtain K superpixel blocks corresponding to the first image, for each superpixel block in the K superpixel blocks, a second pixel point set corresponding to the superpixel block can be determined based on the first pixel points included in the superpixel block; the second pixel point set is used to characterize the connectivity between the first pixel points; then, based on the second pixel point set, it is determined whether the categories of the first pixel points included in the superpixel block are the same to obtain a judgment result. Assuming that the judgment result indicates that the categories of the first pixel points included in the superpixel block are different, the K superpixel blocks can be further divided until the divided superpixel blocks The categories of the pixels included are the same, that is, the embodiment of the present application can adaptively adjust the size of the superpixel block based on the second pixel point set, and make each superpixel block contain only pixels of the same category as much as possible, so that when the superpixel block boundary is updated, the pixel allocation accuracy in the superpixel block search area can be greatly improved; and then after each first pixel point in the L superpixel blocks is allocated to a different superpixel block, the normalized cut Ncut index value of the different segmentation results is formed to determine the final pixel allocation result; the pixel allocation process takes into account the overall similarity between the pixel to be allocated and the superpixel block, rather than the similarity with a single central pixel, thereby improving the accuracy of pixel allocation.

[0106] An embodiment of the present application provides an image superpixel segmentation method, the method comprising: obtaining N first pixel points corresponding to a first image, and dividing the first image based on the N first pixel points to obtain K superpixel blocks corresponding to the first image; wherein each superpixel block includes N / K first pixel points, N is a positive integer, and K is an integer greater than 1; for the K superpixel blocks, based on the connectivity between the pixel points, the K superpixel blocks are adjusted so that the categories of the pixel points contained in each superpixel block are the same, and M superpixel blocks are obtained; wherein M is an integer greater than 1; and the boundaries of the M superpixel blocks are updated. It can be seen that after obtaining the K superpixel blocks corresponding to the first image, the embodiment of the present application can adaptively adjust the K superpixel blocks based on the connectivity between the pixels, so that the categories of the pixels contained in each superpixel block are the same. That is, the embodiment of the present application avoids the need to manually give the expected number of superpixel blocks, and can adaptively adjust the size of the superpixel blocks based on the connectivity between the pixels, so that the categories of the pixels contained in each superpixel block are the same, thereby improving the allocation accuracy of the boundary pixels when updating the boundaries of the M superpixel blocks, thereby improving the image segmentation accuracy.

[0107] Based on the above embodiments, another embodiment of the present application provides an image superpixel segmentation method. First, by calculating the number of highly connected pixels (i.e., a second set of pixel points) in the superpixel block, the size of the superpixel block is adaptively adjusted, and each superpixel block contains as many pixels of the same category as possible, thereby avoiding the problem of manually giving the expected number of superpixel blocks. Secondly, in the pixel allocation process, the embodiment of the present application determines the final pixel allocation result by calculating the normalized cut Ncut index value of different segmentation results formed after the pixels are allocated to different superpixel blocks, so that the pixel allocation process considers the overall similarity between the pixel to be allocated and the superpixel block, rather than the similarity with a single central pixel, thereby improving the accuracy of pixel allocation. Finally, the embodiment of the present application introduces the covariance of pixel visual features and spatial position features through the Mahalanobis distance metric, naturally adjusts the contribution between different scales and distribution features, so that visual and spatial features are fairly considered in the distance metric.

[0108] It should be noted that, in the embodiments of the present application, Figure 2 The image superpixel segmentation method proposed in the embodiment of the present application is shown in FIG. Figure 2 ,like Figure 2 As shown, the image superpixel segmentation method may include the following steps: Step 1: construct a high-dimensional feature representation of an image pixel (i.e., a first pixel) and a Mahalanobis distance representation between pixels. For an input image data having N pixels (i.e., the first pixel), Consider each pixel x i The spatial position information in the image, then its features can be expanded to x i =[x i1 ,…,x iV ,y i ,z i ] T . Where [x i1 ,…,x iV ] T is pixel x i V visual features, [y i ,z i ] T is pixel x i Spatial position features in the image coordinate system; Although the above feature representation method takes into account both the visual and spatial information of the pixel points, the visual features and spatial features usually have different scales and distributions. At this time, if the Euclidean distance measurement method commonly used in related technologies is directly adopted, it may cause a certain feature to have too much weight on the distance calculation, thereby masking the influence of another feature on the distance. Therefore, this application uses the Mahalanobis distance measurement method to represent different pixel points x i and x j The difference between D(x i,x j ), which can be calculated by the above formula (3):

[0109] It should be noted that in the embodiments of the present application, by introducing the covariance matrix between features, the embodiments of the present application can naturally adjust the contribution between features of different scales and distributions, so that visual features and spatial position features are fairly considered in the distance metric.

[0110] It should be noted that, in the embodiments of the present application, Figure 2 As shown, the image superpixel segmentation method may further include the following steps: Step 2: Initialize the image superpixel block division. For the above image data X with N pixels, 1 ,x 2 ,…,x N} T , the image superpixel segmentation device is based on The image (i.e., the first image) is approximately uniformly divided into K superpixel blocks {U 1 ,U 2 ,…,U K}, where each superpixel block U i Approximately contains n i ≈N / K pixels. In the initialization stage described in this step, it is not necessary to determine the initial cluster center of each superpixel block. Instead, the image is evenly divided based on the grid interval set above, so that each superpixel block is as evenly distributed as possible in space. Step 3: Construct a similarity matrix (i.e., the first similarity matrix) of the superpixel blocks based on the KNN graph representation. For the K superpixel blocks {U 1 ,U 2 ,…,U K}, each superpixel block U i Contains i ≈N / K pixels, construct this n i KNN graph representation of pixels; specifically, for U i Each pixel x in i , and the first k nearest neighbor pixels arranged from large to small in similarity are recorded as Since the neighborhood relationship is asymmetric, the similarity matrix constructed later will be asymmetric. In the face of this problem, we simply regard the nearest neighbor relationship as mutual rather than unidirectional, that is, the pixel x j is x i The k nearest neighbors, or pixel x i is x j The k nearest neighbors of the pixel point x i and x jA connection relationship will be generated between them; based on the above super pixel block U i All n i The KNN graph constructed by pixels, the pixel x i and x j The connection relationship and similarity w between ij , can be determined by the pixel spacing measurement method and Gaussian kernel function of the above formula (3), that is, the similarity w ij It can be determined by the above formula (2), and then the super pixel block U i All n i The similarity matrix W of pixels based on KNN graph representation i (ie, the first similarity matrix) can be constructed by the above formula (5).

[0111] It should be noted that, in the embodiment of the present application, the super pixel block U i The similarity matrix W of all pixels in the KNN graph i (First similarity matrix), compared with the traditional similarity matrix based on the fully connected graph representation, the embodiment of the present application adopts the KNN graph representation to construct the similarity matrix, and each pixel point only has a similarity greater than 0 with its surrounding k (P) pixels. Therefore, the first similarity matrix represented by the above formula (5) is usually sparse. Such sparsity can effectively reduce the spatial complexity of the similarity matrix of each superpixel block when the number of superpixel blocks K is very small and the total number of pixels N in the image is very large, thereby reducing the time consumption of subsequent operations.

[0112] It should be noted that, in the embodiments of the present application, Figure 2 As shown, the image superpixel segmentation method may also include the following steps: Step 4: Calculate the number of highly connected pixels in the superpixel block. In the feature space of the pixel points, since the category containing more pixel points has a higher density, the pixels in this category will also have a higher connectivity. The connectivity can be directly described by the power matrix of the similarity matrix (i.e., the first similarity matrix), and the similarity matrices under different powers (i.e., the second similarity matrix) describe the connectivity of pixels at different scales. Based on the above conclusions, in order to ensure that each superpixel block contains only pixels of the same category, it is necessary to ensure that pixels with high connectivity are unique at multiple observation scales; specifically, first, the superpixel block U i The connectivity of pixels at different scales can be represented by elements of a similarity matrix (i.e., a second similarity matrix) at different powers. The second similarity matrix can be shown as in the above formula (6).

[0113] It should be noted that, in an embodiment of the present application, after the image superpixel segmentation device generates a second similarity matrix corresponding to the superpixel block based on a first similarity matrix of different powers, it can perform a first preset operation on each row element in the second similarity matrix to obtain a first operation value. For example, the first preset operation can be to add and sum the absolute values ​​of each row element in the second similarity matrix to obtain the first operation value. The present application does not specifically limit the type of the first preset operation.

[0114] Further, in an embodiment of the present application, when the image superpixel segmentation device screens out S elements in each row of the second similarity matrix that are greater than or equal to the corresponding first operation value, the second similarity matrix The diagonally dominant row in is determined, and the image superpixel segmentation device can filter out S elements in each row that are greater than or equal to the corresponding first operation value through the above formula (7), and then generate a second pixel point set based on the S elements.

[0115] It should be noted that, in the embodiment of the present application, after screening out the S elements in the second similarity matrix, the image superpixel segmentation device can determine the target pixel point in the corresponding superpixel block through the S elements, and the target pixel point is a highly connected pixel of the superpixel block, and then a second pixel point set can be generated based on the target pixel point; wherein the target pixel point is a part of the first pixel point. That is, the elements satisfying the above formula (7) Its corresponding super pixel block U i The jth pixel x j , is the super pixel block U i A high connectivity pixel (ie, a target pixel) is selected, and then a second pixel set can be generated based on the target pixel.

[0116] For example, in the embodiments of the present application, (Second similarity matrix) All m that satisfy the above formula (7) T elements, i.e., pixel blocks U i In W i T All m under the corresponding scale T Highly connected pixels can be recorded as (i.e., the second pixel point set). (T) |=m T , indicating C (T) The number of elements in the set, then in the superpixel block U i The corresponding similarity matrix power T is from T 1 Increase to T n In the process (that is, for the pixel block U iThe process of observing the scale from small to large) can get a set of n element values ​​arranged from large to small In this step, the goal is to ensure that the high connectivity pixels of the superpixel block are unique at more observation scales. (T) |) represents a set Chinese element | C (T) |The frequency of occurrence, then the super pixel block U i The number of highly connected pixels m at the maximum observed scale is given by the above formula (8).

[0117] It should be noted that, in an embodiment of the present application, when the image superpixel segmentation device determines whether the categories of the first pixel points included in the superpixel block are the same based on the second pixel point set, when the power increases from the first set value to the second set value, it determines whether the number of occurrence frequencies of each element in the second pixel point set is equal to the first preset threshold; if the number of occurrence frequencies of each element in the second pixel point set is equal to the first preset threshold, it is determined that the categories of the first pixel points included in the superpixel block are the same; if the number of occurrence frequencies of each element in the second pixel point set is less than or greater than the first preset threshold, it is determined that the categories of the first pixel points included in the superpixel block are different.

[0118] It should be noted that, in the embodiments of the present application, the first set value and the second set value represent the power values ​​of the first similarity matrix, and the present application does not specifically limit the sizes of the first set value and the second set value.

[0119] It should be noted that, in the embodiment of the present application, the first preset threshold may be 1.

[0120] Exemplarily, in the embodiment of the present application, as shown in the above formula (8), when the power is changed from T 1 Increase to T n , determine whether the number (m) of occurrence frequencies of each element in the second pixel point set is equal to 1; if the number of occurrence frequencies of each element in the second pixel point set is equal to 1, it is determined that the categories of the first pixel points contained in the superpixel block are the same; if the number of occurrence frequencies of each element in the second pixel point set is less than or greater than 1, it is determined that the categories of the first pixel points contained in the superpixel block are different.

[0121] It should be noted that, in the embodiments of the present application, Figure 2 As shown, the image superpixel segmentation method may further include the following steps: Step 5: determine whether the number m of highly connected pixels of the superpixel block determined by step 4 is 1, if m=1, execute step 6; otherwise, divide the superpixel block based on the result of step 4, and return to step 3, when the superpixel block U i When the number of highly connected pixels m≠1 at the maximum observation scale, it means that Ui contains multiple categories of pixels, which will affect the allocation of pixels in the subsequent superpixel block search area, thereby reducing the image segmentation accuracy. i To contain only pixels of the same category as much as possible, it is necessary to further divide them in a similar way as described in step 2. Set U to the network interval i Approximately evenly divided into m 2 A superpixel block of almost the same size but smaller in area For each superpixel block U ij Return to execute steps 3, 4, and 5, construct a similarity matrix based on the KNN graph, then calculate the number of highly connected pixels and determine whether the number is unique, until the categories of the pixels contained in the divided superpixel blocks are the same, and M superpixel blocks are obtained.

[0122] That is to say, in an embodiment of the present application, after the image superpixel segmentation device divides the first image based on N first pixel points to obtain K superpixel blocks corresponding to the first image, for each superpixel block in the K superpixel blocks, a second pixel point set corresponding to the superpixel block can be determined based on the first pixel points contained in the superpixel block; the second pixel point set is used to characterize the connectivity between the first pixel points; then, based on the second pixel point set, it is determined whether the categories of the first pixel points contained in the superpixel blocks are the same to obtain a judgment result. Assuming that the judgment result characterizes that the categories of the first pixel points contained in the superpixel blocks are different, the K superpixel blocks can be further divided until the categories of the pixel points contained in the divided superpixel blocks are the same, that is, the embodiment of the present application can adaptively adjust the size of the superpixel block based on the second pixel point set, and make each superpixel block contain as many pixels of the same category as possible, so that when the superpixel block boundary is updated, the pixel allocation accuracy of the superpixel block search area can be greatly improved.

[0123] It should be noted that, in the embodiments of the present application, Figure 2 As shown, the image superpixel segmentation method may further include the following steps: step 6: assigning pixels of the superpixel block overlapping search area (i.e., L superpixel blocks), updating the superpixel block boundary, and obtaining the regional image preprocessing result; after steps 3-5, obtaining the adaptive superpixel division result of the image; its adaptability is manifested in that regardless of the initial division of the superpixel block in step 2, the superpixel blocks updated after steps 3-5 mostly have only one high connectivity pixel, that is, most superpixel blocks only contain pixels of the same category, which makes the allocation accuracy of the pixels of the superpixel block search area (i.e., L superpixel blocks) higher when updating the superpixel block boundary; after steps 3-5, the original K superpixel blocks of almost equal size are divided into {U 1 ,U2 ,…,U K}, will be further subdivided into M superpixel blocks of different sizes {U 1 ,U 2 ,…,U M Since these M superpixel blocks are still divided by grids, their boundaries are usually rectangular boundaries. However, the natural boundaries of images are often complex and irregular, which may cause some superpixel blocks to cross multiple object boundaries. 1 ,U 2 ,…,U M To better describe the actual boundaries in the image and retain more image structure information, it is necessary to redistribute the pixels of the overlapping search area (i.e., L superpixel blocks) around the superpixel blocks (i.e., M superpixel blocks) and update the edges of each superpixel block.

[0124] For example, in the embodiment of the present application, for the above M super pixel blocks, the search area size is 2S i ×2S i For each pixel x in the image i , if it is in M ​​superpixel blocks {U 1 ,U 2 ,…,U M}, then it must belong to a super pixel block U among these M super pixel blocks. i At this time, the pixel x i Temporarily assigned to the remaining M-1 superpixel blocks U 1 ,…U i-1 ,U i+1 ,…,U M , forming M-1 new partitioning results Calculate these M-1 new partitioning results and compare them with the original partitioning results The normalized cut Ncut index value is used to measure the rationality of all M partition results. The Ncut index value corresponding to the M-1 partition results and the Ncut index value corresponding to the initial partition result (i.e., the original partition result) can be calculated by the above formula (11), that is, the rationality of all M partition results can be measured.

[0125] For example, in an embodiment of the present application, the image superpixel segmentation device may first determine the initial segmentation result corresponding to each first pixel point in the L superpixel blocks. Then, the optimal division result corresponding to each first pixel point in the L superpixel blocks can be determined based on the Ncut index value corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result. The Ncut index value corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result can be calculated respectively by the above formula (11), that is, the rationality of all M division results can be measured.

[0126] It should be noted that, in the embodiment of the present application, the L super pixel blocks may be super pixel blocks in the M super pixel block search area. For example, the search area range size may be 2S. i ×2S i , this application does not specifically limit the size of the search area.

[0127] It should be noted that in an embodiment of the present application, when the image superpixel segmentation device determines the optimal division result corresponding to each first pixel point in L superpixel blocks based on the Ncut index value corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result, it can determine the minimum Ncut index value among the Ncut index values ​​corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result; then, the target pixel block assigned to each first pixel point can be determined based on the division result corresponding to the minimum Ncut index value; and then, the optimal division result corresponding to each first pixel point can be determined based on the target pixel block.

[0128] For example, in the embodiment of the present application, as shown in the above formula (11), the smaller the Ncut index value is, the higher the similarity of the pixels inside each superpixel block in the division result is, and the lower the similarity of the pixels between the superpixel blocks is, the more reasonable the division result is. Therefore, the embodiment of the present application can calculate the total division result by The Ncut index value, where the minimum Ncut index value corresponds to the partition result It means pixel x i Assigned to superpixel block U j (ie the target pixel block) is the most reasonable.

[0129] It should be noted that, in the embodiment of the present application, the image superpixel segmentation device can allocate each first pixel point in the L superpixel blocks to the corresponding M-1 superpixel blocks to obtain M-1 division results, and then the Ncut index value corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result (i.e. Ncut index value), and then the division result corresponding to the minimum Ncut index value can be selected, and then the optimal division result corresponding to each first pixel point can be determined based on the division result corresponding to the minimum Ncut index value.

[0130] That is to say, in the embodiments of the present application, after the image superpixel segmentation device assigns pixel points to different superpixel blocks, the normalized cut (Ncut) metric values of different segmentation results formed can be used to determine the final assignment result of the pixels; so that the pixel assignment process takes into account the overall similarity between the pixels to be assigned and the superpixel blocks, rather than the similarity with a single central pixel, thereby improving the accuracy of pixel assignment.

[0131] It should be noted that in the embodiments of the present application, through the above step 6, the image superpixel segmentation device reassigns all the pixel points in the search area of the superpixel blocks to update the superpixel block boundaries, making them more conform to the actual boundaries of the image, and obtaining * the final division results of M superpixel blocks, where M * < M, because if the scale difference between two adjacent superpixel blocks is too large, during the pixel assignment process using the Ncut value, the small-scale superpixel blocks may be merged into the large-scale superpixel blocks, resulting in a reduction in the total number of superpixel blocks. Finally, these M * superpixel blocks are the regional-level image preprocessing results.

[0132] It should be noted that in the embodiments of the present application, Figure 3 is a schematic diagram of the image preprocessing result proposed in the embodiments of the present application. As Figure 3 shown, the image superpixel segmentation device can first obtain image data, then perform an initial segmentation of the superpixel blocks, and then complete the superpixel block adaptive update process. Finally, it can perform superpixel block boundary update to finally obtain the image preprocessing result.

[0133] To summarize, after the image superpixel segmentation device divides the first image based on N first pixel points to obtain K superpixel blocks corresponding to the first image, for each superpixel block in the K superpixel blocks, a second pixel point set corresponding to the superpixel block can be determined based on the first pixel points included in the superpixel block; the second pixel point set is used to characterize the connectivity between the first pixel points; then, based on the second pixel point set, it is determined whether the categories of the first pixel points included in the superpixel block are the same to obtain a judgment result. Assuming that the judgment result indicates that the categories of the first pixel points included in the superpixel block are different, the K superpixel blocks can be further divided until the divided superpixel blocks The categories of the pixels included are the same, that is, the embodiment of the present application can adaptively adjust the size of the superpixel block based on the second pixel point set, and make each superpixel block contain only pixels of the same category as much as possible, so that when the superpixel block boundary is updated, the pixel allocation accuracy in the superpixel block search area can be greatly improved; and then after each first pixel point in the L superpixel blocks is allocated to a different superpixel block, the normalized cut Ncut index value of the different segmentation results is formed to determine the final pixel allocation result; the pixel allocation process takes into account the overall similarity between the pixel to be allocated and the superpixel block, rather than the similarity with a single central pixel, thereby improving the accuracy of pixel allocation.

[0134] An embodiment of the present application provides an image superpixel segmentation method, the method comprising: obtaining N first pixel points corresponding to a first image, and dividing the first image based on the N first pixel points to obtain K superpixel blocks corresponding to the first image; wherein each superpixel block includes N / K first pixel points, N is a positive integer, and K is an integer greater than 1; for the K superpixel blocks, based on the connectivity between the pixel points, the K superpixel blocks are adjusted so that the categories of the pixel points contained in each superpixel block are the same, and M superpixel blocks are obtained; wherein M is an integer greater than 1; and the boundaries of the M superpixel blocks are updated. It can be seen that after obtaining the K superpixel blocks corresponding to the first image, the embodiment of the present application can adaptively adjust the K superpixel blocks based on the connectivity between the pixels, so that the categories of the pixels contained in each superpixel block are the same. That is, the embodiment of the present application avoids the need to manually give the expected number of superpixel blocks, and can adaptively adjust the size of the superpixel blocks based on the connectivity between the pixels, so that the categories of the pixels contained in each superpixel block are the same, thereby improving the allocation accuracy of the boundary pixels when updating the boundaries of the M superpixel blocks, thereby improving the image segmentation accuracy.

[0135] Based on the above embodiments, the present application provides an image superpixel segmentation device. Figure 4 Schematic diagram of the structure of the image superpixel segmentation device Figure 1 ,like Figure 4As shown, the device 10 includes: an acquisition unit 11, a division unit 12, an adjustment unit 13 and an update unit 14; wherein,

[0136] The acquisition unit 11 is used to acquire N first pixel points corresponding to the first image; wherein N is a positive integer;

[0137] The division unit 12 is used to divide the first image based on the N first pixel points to obtain K super pixel blocks corresponding to the first image; wherein each super pixel block includes N / K first pixel points, and K is an integer greater than 1;

[0138] The adjusting unit 13 is used to adjust the K super pixel blocks based on the connectivity between the pixels, so that the categories of the pixels included in each super pixel block are the same, and M super pixel blocks are obtained; wherein M is an integer greater than 1;

[0139] The updating unit 14 is used to update the boundaries of the M super pixel blocks.

[0140] In the embodiments of the present application, further, Figure 5 Schematic diagram of the structure of the image superpixel segmentation device Figure 2 ,like Figure 5 As shown, the image superpixel segmentation device 10 proposed in the embodiment of the present application may also include a processor 15 and a memory 16 storing executable instructions of the processor 15. Further, the device 10 may also include a communication interface 17 and a bus 18 for connecting the processor 15, the memory 16 and the communication interface 17.

[0141] In the embodiment of the present application, the processor 15 can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic device used to implement the function of the processor can also be other, and the embodiment of the present application is not specifically limited. The device 10 can also include a memory 16, which can be connected to the processor 15, wherein the memory 16 is used to store executable program code, the program code includes computer operation instructions, and the memory 16 may include a high-speed RAM memory, and may also include a non-volatile memory, for example, at least two disk memories.

[0142] In the embodiment of the present application, the bus 18 is used to connect the communication interface 17, the processor 15 and the memory 16, and the mutual communication between these devices.

[0143] In the embodiment of the present application, the memory 16 is used to store instructions and data.

[0144] Furthermore, in an embodiment of the present application, the above-mentioned processor 15 is used to obtain N first pixel points corresponding to the first image, and divide the first image based on the N first pixel points to obtain K super-pixel blocks corresponding to the first image; wherein each of the super-pixel blocks includes N / K first pixel points, N is a positive integer, and K is an integer greater than 1; for the K super-pixel blocks, based on the connectivity between the pixel points, the K super-pixel blocks are adjusted so that the categories of the pixel points contained in each super-pixel block are the same, and M super-pixel blocks are obtained; wherein M is an integer greater than 1; and the boundaries of the M super-pixel blocks are updated.

[0145] In practical applications, the memory 16 may be a volatile memory, such as a random access memory (RAM); or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 15.

[0146] An embodiment of the present application provides an image superpixel segmentation device, which obtains N first pixel points corresponding to a first image, and divides the first image based on the N first pixel points to obtain K superpixel blocks corresponding to the first image; wherein each superpixel block includes N / K first pixel points, N is a positive integer, and K is an integer greater than 1; for the K superpixel blocks, based on the connectivity between the pixel points, the K superpixel blocks are adjusted so that the categories of the pixel points contained in each superpixel block are the same, and M superpixel blocks are obtained; wherein M is an integer greater than 1; and the boundaries of the M superpixel blocks are updated. It can be seen that after obtaining the K superpixel blocks corresponding to the first image, the embodiment of the present application can adaptively adjust the K superpixel blocks based on the connectivity between the pixels, so that the categories of the pixels contained in each superpixel block are the same. That is, the embodiment of the present application avoids the need to manually give the expected number of superpixel blocks, and can adaptively adjust the size of the superpixel blocks based on the connectivity between the pixels, so that the categories of the pixels contained in each superpixel block are the same, thereby improving the allocation accuracy of the boundary pixels when updating the boundaries of the M superpixel blocks, thereby improving the image segmentation accuracy.

[0147] An embodiment of the present application provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the image superpixel segmentation method as described above.

[0148] Specifically, the program instructions corresponding to the image superpixel segmentation method in this embodiment can be stored on a storage medium such as a CD, a hard disk, or a USB flash drive. When the program instructions corresponding to the image superpixel segmentation method in the storage medium are read or executed by an electronic device, the following steps are included:

[0149] Obtaining N first pixel points corresponding to a first image, and dividing the first image based on the N first pixel points to obtain K super pixel blocks corresponding to the first image; wherein each super pixel block includes N / K first pixel points, N is a positive integer, and K is an integer greater than 1;

[0150] For the K superpixel blocks, based on the connectivity between the pixels, the K superpixel blocks are adjusted so that the categories of the pixels contained in each superpixel block are the same, thereby obtaining M superpixel blocks; wherein M is an integer greater than 1;

[0151] The boundaries of the M superpixel blocks are updated.

[0152] The embodiment of the present application also provides a computer program product, including a computer program, which can be executed by the processor 15 of the image superpixel segmentation device 10 to complete the steps of any of the aforementioned methods.

[0153] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0154] The present application is described with reference to implementation flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which is implemented in the implementation flow diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process described in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0157] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A method for image superpixel segmentation, characterized in that: The method comprises: Obtaining N first pixel points corresponding to a first image, and dividing the first image based on the N first pixel points to obtain K super pixel blocks corresponding to the first image; wherein each super pixel block includes N / K first pixel points, N is a positive integer, and K is an integer greater than 1; For the K superpixel blocks, based on the connectivity between the pixels, the K superpixel blocks are adjusted so that the categories of the pixels contained in each superpixel block are the same, thereby obtaining M superpixel blocks; wherein M is an integer greater than 1; The boundaries of the M superpixel blocks are updated.

2. The method according to claim 1, characterized in that The adjusting the K super pixel blocks based on the connectivity between the pixel points includes: For each superpixel block in the K superpixel blocks, determine a second pixel point set corresponding to the superpixel block based on a first pixel point included in the superpixel block; wherein the second pixel point set includes some pixel points in the first pixel points, and the second pixel point set is used to characterize the connectivity between the first pixel points; Determine whether the categories of the first pixel points included in the super pixel block are the same based on the second pixel point set, and obtain a determination result; The K super pixel blocks are adjusted based on the judgment result.

3. The method according to claim 2, characterized in that The method further comprises: Based on the judgment result, the size of the super pixel block is dynamically divided so that the categories of the pixel points contained in the divided super pixel blocks are the same, thereby obtaining the M super pixel blocks.

4. The method according to claim 1, characterized in that: Updating the boundaries of the M superpixel blocks includes: Assign each first pixel point in the L superpixel blocks to the corresponding M-1 superpixel blocks to obtain M-1 division results; Based on the normalized cut Ncut index value corresponding to the M-1 division results, the optimal division result corresponding to each first pixel point in the L superpixel blocks is determined; wherein the L superpixel blocks are part of the M superpixel blocks.

5. The method according to claim 2, characterized in that: The determining of a second pixel point set corresponding to the super pixel block based on the first pixel point included in the super pixel block includes: For each superpixel block, determine a first similarity matrix corresponding to the superpixel block based on the first pixel point; wherein the first similarity matrix is ​​used to represent the similarity between each pixel point in the superpixel block and P adjacent pixels; wherein P is a positive integer; A second set of pixel points corresponding to the superpixel block is determined based on the first similarity matrix.

6. The method according to claim 5, characterized in that The determining, based on the first pixel point, a first similarity matrix corresponding to the super pixel block includes: Determine the similarity between each pixel in the superpixel block and P adjacent pixels based on a preset distance measurement method and a preset function; wherein the preset distance measurement method at least includes a Mahalanobis distance measurement method, and the preset function at least includes a Gaussian kernel function; The first similarity matrix is ​​determined based on the similarities.

7. The method according to claim 5, characterized in that The determining, based on the first similarity matrix, a second pixel point set corresponding to the super pixel block includes: Generate a second similarity matrix corresponding to the super pixel block based on the first similarity matrix of different powers; Performing a first preset operation on each row element in the second similarity matrix to obtain a first operation value; S elements greater than or equal to the corresponding first operation value in each row element of the second similarity matrix are screened out, and the second pixel point set is generated based on the S elements; wherein S is a positive integer.

8. The method according to claim 7, characterized in that The determining, based on the second pixel point set, whether the categories of the first pixel points included in the super pixel block are the same includes: When the power increases from the first set value to the second set value, determining whether the number of occurrence frequencies of each element in the second pixel point set is equal to a first preset threshold; If the number of occurrence frequencies of each element in the second pixel point set is equal to the first preset threshold, it is determined that the categories of the first pixel points included in the super pixel block are the same.

9. The method according to claim 8, characterized in that The method further comprises: If the number of occurrence frequencies of each element in the second pixel point set is less than or greater than the first preset threshold, it is determined that the categories of the first pixel points included in the super pixel block are different.

10. The method according to claim 4, characterized in that The determining the optimal division result corresponding to each first pixel point in the L super pixel blocks based on the normalized cut Ncut index value corresponding to the M-1 division results includes: Determine an initial division result corresponding to each first pixel point in the L super pixel blocks; The optimal division result corresponding to each first pixel point in the L super pixel blocks is determined based on the Ncut index value corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result.

11. The method according to claim 10, characterized in that The determining the optimal division result corresponding to each first pixel point in the L super pixel blocks based on the Ncut index value corresponding to the M-1 division results and the Ncut index value corresponding to the initial division result includes: Determine a minimum Ncut index value between the Ncut index values ​​corresponding to the M-1 types of division results and the Ncut index value corresponding to the initial division result; Determine the target pixel block assigned to each first pixel point based on the division result corresponding to the minimum Ncut index value; An optimal division result corresponding to each first pixel point is determined based on the target pixel block.

12. An image superpixel segmentation device, characterized in that: The image superpixel segmentation device comprises: an acquisition unit, a division unit, an adjustment unit and an update unit; wherein, The acquisition unit is used to acquire N first pixel points corresponding to the first image; wherein N is a positive integer; The division unit is used to divide the first image based on the N first pixel points to obtain K super pixel blocks corresponding to the first image; wherein each super pixel block includes N / K first pixel points, and K is an integer greater than 1; The adjusting unit is used to adjust the K superpixel blocks based on the connectivity between the pixels, so that the categories of the pixels included in each superpixel block are the same, and M superpixel blocks are obtained; wherein M is an integer greater than 1; The updating unit is used to update the boundaries of the M super pixel blocks.

13. An image superpixel segmentation device, characterized in that: The image superpixel segmentation device comprises: a processor and a memory; wherein, The memory is used to store a computer program that can be run on the processor; The processor is configured to execute the method according to any one of claims 1 to 11 when running the computer program.

14. A computer-readable storage medium, characterized in that: The storage medium stores computer program codes, and when the computer program codes are executed by a computer, the method according to any one of claims 1 to 11 is executed.

15. A computer program product comprising a computer program, characterized in that The computer program implements the method according to any one of claims 1-11 when executed by a processor.

Images