Image segmentation method based on multi-scale superpixel spatial co-occurrence mixture model

Abstract

The present invention discloses an image segmentation method of a multi-scale superpixel spatial symbiotic hybrid model, belonging to the field of image processing. The present invention comprises the steps of: S1, parameter setting: setting the β (coupling coefficient) growth function, the number of segmentation categories, and the combination of superpixels; S2, model construction: constructing a superpixel spatial symbiotic hybrid model of different scales, and superpixels of each scale share a set of Gaussian mixture model parameters on the observed value; S3, multi-scale training: initializing the model parameters using the K-means algorithm, and then alternately training the model with superpixels of multiple scales until the model parameters converge; S4, single pixel segmentation: using the model parameters obtained in S3, determine the category to which each pixel belongs according to the maximum a posteriori probability criterion. The present invention can improve the accuracy of the co-occurrence likelihood function model of neighboring pixel observations in the segmentation of complex texture images and reduce the workload of manual parameter adjustment.

Description

Technical Field

[0001] The present invention relates to an image segmentation method of a multi-scale super-pixel spatial symbiotic hybrid model, belongs to the field of image processing, and can be applied to processing before image semantic recognition. Background Art

[0002] In the traditional image processing process, it is necessary to segment the image into several regions and extract the features of the segmented regions for semantic recognition. The quality of the segmentation results has a crucial impact on the effect of semantic recognition. In image recognition based on deep neural networks, prior pre-segmentation information can improve the accuracy of recognition and reduce the demand for samples for model training. General image segmentation has always been a difficult problem that has not been effectively solved in digital image technology. Image segmentation methods usually segment images based on the clustering of pixels in color space and the correlation in coordinate space. Clustering-based segmentation algorithms often only use the partial features of pixels in color space, and the segmentation results are easily affected by noise, and it is difficult to achieve good results on images with complex textures. The symbiosis in coordinate space is often modeled using a hidden Markov model, but the coupling between the model parameters makes it difficult to solve the model parameters, and the general approximate numerical solution is also difficult to apply in fields with high real-time requirements due to its great complexity.

[0003] Iterated Conditional Mode (ICM) models the distribution characteristics of pixels in color space through Gaussian Mixed Model (GMM), characterizes the symbiosis of pixels in coordinate space through Hidden Markov Model and uses Gibbs Random Field for approximate calculation. Finally, ICM uses the Bayesian formula to combine the prior probability obtained by GMM with the posterior probability obtained by Gibbs Random Field to approximate the target likelihood function in the model parameter solution using the pseudo-likelihood method. This method is simple and fast, but it is difficult to achieve good results for images with rich texture features, and the coupling coefficient β of the Gibbs Random Field needs to be manually determined according to the complexity of the texture. Summary of the invention

[0004] In order to improve the effect of the co-occurrence likelihood function model of neighboring pixel observations on images with rich texture features, the present invention provides an image segmentation method of a multi-scale superpixel spatial co-occurrence hybrid model, which takes into account the spatial co-occurrence information of pixels inside the multi-scale texture when solving the model parameters, and improves the segmentation effect of the texture-dense area while ensuring the resolution of the segmentation result. At the same time, the present invention adopts a method of adaptively and dynamically determining the parameters of the Gibbs random field to reduce the workload of manually determining the model parameters, reduce the workload of manual parameter adjustment and increase the stability of the algorithm. The present invention improves the segmentation effect of the texture-dense area while taking into account the segmentation fineness of the image edge and the sparse texture part.

[0005] The above purpose is achieved through the following technical solutions:

[0006] An image segmentation method based on a multi-scale superpixel spatial symbiotic mixture model, the method comprising the following steps:

[0007] S1. Parameter setting: setting the β (coupling coefficient) growth function, the number of segmentation categories, and the superpixel combination;

[0008] S2, model construction: construct a spatial symbiotic mixture model of superpixels of different scales, and superpixels of each scale share a set of Gaussian mixture model parameters on the observation values;

[0009] S3, multi-scale training: Use the K-means algorithm to initialize the model parameters, and then use superpixels of multiple scales to alternately train the model until the model parameters converge

[0010] S4, single pixel segmentation: using the model parameters obtained in S3, determine the category to which each pixel belongs according to the maximum a posteriori probability criterion.

[0011] Preferably, in S1, the β (coupling coefficient) growth function has examples including but not limited to the following linear function form: f(t) = at + b, where t is the number of iterations, a is the slope of the β function, and b is the initial value of β. Usually b is set to 0.5, and a is an empirical value, which depends on the specific implementation of the algorithm.

[0012] Preferably, in S2, the image is divided into superpixels (size is l×l, l is an odd number), and there is a likelihood function under the superpixel:

[0013]

[0014] in is the neighborhood of the i-th superpixel, U(k) is the number of occurrences of category k in the neighborhood of the superpixel at position i, i = 1, 2...n, n represents the number of superpixels, and k is the number of categories. Using the isotropic multi-level logic (MLL) model as the reference model of the potential function, and only considering the paired subcluster potential function, we have represents the likelihood probability of the visual distribution of the kth image segmentation category weighted by the current superpixel, where represents the qth pixel in the superpixel centered at pixel i, and θk represents the parameters of the visual probability distribution of the kth image segmentation category.

[0015] Beneficial effects:

[0016] The traditional co-occurrence likelihood function model of neighboring pixel observations needs to set the coupling coefficient β of the Gibbs random field according to the complexity of the image texture. Based on the phenomenon that the more complex the image texture is, the more positively correlated it is with the number of algorithm iterations and the β value. The present invention uses a monotonically increasing function with the number of iterations as a single variable to determine the value of β, thereby reducing the workload of manual parameter adjustment and increasing the stability of the algorithm.

[0017] In order to improve the effect of the co-occurrence likelihood function model of neighboring pixel observations on images with rich texture features, the present invention constructs a spatial co-occurrence mixed model of superpixels of different scales. Superpixels of each scale share a set of Gaussian mixture model parameters on the observation values. When solving the model parameters, superpixels of multiple scales are used to update the model parameters in sequence, and finally a single pixel is used to perform the final segmentation of the image. This method takes into account the spatial co-occurrence information of pixels inside multi-scale textures when solving the model parameters, and improves the segmentation effect of densely textured areas while ensuring the resolution of the segmentation results. Ultimately, it can achieve better segmentation effects in densely textured areas, and can also accurately segment images at higher resolutions at regional boundaries. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic flow chart of the method of the present invention;

[0019] Figure 2 is the image to be segmented in the implementation manner;

[0020] Figure 3 It is the intermediate result after parameter initialization;

[0021] Figure 4 It is the segmentation result of the method of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings and specific segmentation examples in the embodiments of the present invention to clearly and completely describe the execution of the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The following will further illustrate the present invention in combination with specific image segmentation examples, but it is not intended to limit the present invention.

[0023] like Figure 1 As shown, an image segmentation method of a multi-scale superpixel spatial symbiotic hybrid model in this embodiment includes:

[0024] S1. Parameter setting: setting the β (coupling coefficient) growth function, the number of segmentation categories, and the superpixel combination;

[0025] S2, model construction: construct a spatial symbiotic mixture model of superpixels of different scales, and superpixels of each scale share a set of Gaussian mixture model parameters on the observation values;

[0026] S3, multi-scale training: Use the K-means algorithm to initialize the model parameters, and then use superpixels of multiple scales to alternately train the model until the model parameters converge

[0027] S4, single pixel segmentation: using the model parameters obtained in S3, determine the category to which each pixel belongs according to the maximum a posteriori probability criterion.

[0028] In S1 of this embodiment:

[0029] β is the coupling coefficient, which indicates the penalty intensity for other pixels in the neighborhood. The parameter β of the potential function is related to the central pixel, but because the amount of direct calculation is too large, a suitable constant is usually used to replace it for approximate calculation.

[0030] β controls the correlation between neighbors. The larger β is, the greater the influence between neighboring pixels is, and the final result is smoother. For an image, it is often necessary to adjust parameters multiple times to determine the appropriate β value. Too much manual participation increases manpower input, and the lack of operator experience increases the instability of the segmentation results. Multiple attempts also increase the amount of calculations caused by repeated program runs. In the execution of the ICM algorithm, the more complex the image texture is, the more iterations are performed, and a larger β is required to smooth the image. There is a positive correlation between the image texture complexity, the number of iterations, and the β value. Therefore, the β value can be set to a dynamic value that increases with the number of iterations. The relationship between the β value and the number of iterations is shown in the following expression: β = f(t). Where f(t) is a monotonically increasing function, and examples of f(t) include but are not limited to the following linear function form: f(t) = at + b, where t is the number of iterations, a is the slope of the β function graph, and b is the initial value of β. Usually b is set to 0.5, and a is an empirical value, depending on the specific implementation of the algorithm.

[0031] The image to be segmented is Figure 2 For example, the image is a photo of a natural scene. Due to the mimicry characteristics of natural animals, the foreground and background of the image are very similar and the texture is complex, which makes segmentation very difficult. Set the number of segmentation categories to 3, the superpixel size to 1×1, 3×3, and 5×5, and the coupling coefficient expressions corresponding to the three superpixel scales are: β 1×1 =t+0.5β 3×3 =0.8t+0.5β 5×5 =0.6t+0.45.

[0032] In S2 of this embodiment:

[0033] The joint category distribution of image pixels is described by the prior probability p(z), Z = {Z1, Z2, ..., Z n} is a latent discrete variable representing the category of pixels, and n is the number of pixels; is a K-dimensional binary vector, K is the number of categories, If xi belongs to the kth category then otherwise

[0034] The spatial symbiosis between pixels is described by the hidden Markov model. In a multi-order neighborhood system, according to the Hammersley-Clifford theorem, the Markov random field is equivalent to the Gibbs random field. And because the computational complexity of p(z) is exponential, the present invention adopts the pseudo-likelihood function To replace p(z), where For z i The neighborhood of p l(z) can effectively decompose p(z).

[0035] The distribution of image pixel observations is described using class conditional probabilities. It is assumed that the visual description of each image segmentation region follows a specific Gaussian distribution Where α represents the combination coefficient of the kth component, and the parameter θ = (α1, ..., α k , μ1, ..., μ k ,∑1,...,∑ k ) is the set of coefficients, means, and covariance matrices of each Gaussian component. Then the visual feature x of each pixel in the area j It can be understood as a sample generated under this probability distribution. If the image is divided into K categories, and the area corresponding to each category corresponds to a Gaussian distribution in the color space, then the distribution of the entire image in the color space corresponds to a Gaussian mixture model.

[0036] When given an image segmentation result, if Then the likelihood function of the observed value of pixel xj can be expressed as f k (x j |θ k )=N(x j , μ k ,∑ k ), θ k =(μ k ,∑ k ). The present invention considers the spatial symbiosis of pixels, that is, the category to which the pixel belongs determines the probability of the pixel appearing in the neighborhood of the position, and there is a formal description: f(x N(i) |z i ), x N(i) For x i neighborhood. Since the distribution of the co-occurrence likelihood function is conditionally independent, the joint distribution of the images is described as

[0037] The generative model of an image is as follows: the category of each pixel is determined by the category distribution within its neighborhood according to Determine, and then use the determined category model component according to f(x N(i) |z i ) generates the observed value of the pixel. The likelihood function of the model is as follows:

[0038] The present invention adopts a decomposition method of a pseudo-likelihood function, and can solve each pixel one by one when solving the implicit image category and the model parameter value.

[0039] First, calculate the category prior probability at pixel i For z at pixel i i There are K possible values, when z i =k, in the Gibbs random field, we have: The present invention uses an isotropic multi-level logic model (MLL) as a reference model of potential function and only considers paired subcluster potential functions. Then:

[0040] When calculating the class conditional probability at pixel i, if Then the likelihood function of the pixel observation value is f(x N(i) |z i )=f k (x N(i) |θ k ). Assuming that the visual observation values ​​of a pixel and other pixels in its neighborhood are independent and identically distributed, we have in Represents x i itself, m is the number of pixels in the neighborhood, and the likelihood function of the model is: There is a log-likelihood function:

[0041] For dense texture areas, the results obtained by using single pixel segmentation have a lot of mis-segmentation phenomena. The present invention uses superpixels to make the pixels with large differences in observed values ​​in dense texture areas be uniformly divided into one category. Effectively improve the performance on dense texture images. The image is divided into superpixels (size is l×l, l is an odd number), and the likelihood function under superpixels is: Then we have the log-likelihood function:

[0042] In S3 of this embodiment:

[0043] The present invention uses ICM to solve the problem, and first uses the k-means algorithm to initialize the model parameters: z i ,θ k , and on this basis, considering the spatial symbiosis of neighboring pixels, a new zi estimate is obtained. For the image to be segmented, the results after parameter initialization are as follows Figure 3 shown.

[0044] For a unit pixel, pass Update θ k :

[0045] For a superpixel, there is z i Valuation: in represents the qth pixel in the superpixel centered at pixel i. Then all pixels in a superpixel have the same category. Parameter θ k It is only related to the category and observation value of the pixel point. The superpixel in the present invention is only used as a classification unit and does not change the observation value of a single pixel. In the superpixel mode, θ k The updating method is the same as single pixel mode.

[0046] During model training, ICM is used to solve Use superpixels of each scale to update the model parameters in turn. For each scale of superpixel, after updating the parameters, Re-obtain a new z i The above process is repeated until the model parameters converge. The formal description of the model training process is shown in the following table:

[0047]

[0048] In S4 of this embodiment:

[0049] The use of superpixel segmentation will cause jagged edges in the segmentation results. The present invention uses hybrid superpixels to solve the model parameters, taking into account the accuracy of superpixels in complex texture areas and the high precision of single-pixel segmentation results. Finally, according to the maximum a posteriori probability criterion at the unit pixel scale, Get the final segmentation result.

[0050] for Figure 2 , the final segmentation result is as follows Figure 4 shown.

[0051] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims

1. An image segmentation method based on a multi-scale superpixel spatial symbiotic mixture model, characterized in that: The method comprises: S1, parameter setting: setting the coupling coefficient β growth function, the number of segmentation categories, and the superpixel combination; S2, model construction: construct a spatial symbiotic mixture model of superpixels of different scales, and superpixels of each scale share a set of Gaussian mixture model parameters on the observation values; S3, multi-scale training: Use the K-means algorithm to initialize the model parameters, and then use superpixels of multiple scales to alternately train the model until the model parameters converge; S4, single pixel segmentation: using the model parameters obtained in S3, determine the category to which each pixel belongs according to the maximum a posteriori probability criterion; In S2, the image is divided into superpixels of size l×l, where l is an odd number, and the likelihood function under the superpixel is: in is the neighborhood of the i-th superpixel, U(k) is the number of occurrences of category k in the neighborhood of the superpixel at position i, i = 1, 2…n, n represents the number of superpixels, k is the number of categories, and the isotropic multi-level logistic model is used as the reference model of the potential function, and only the paired subcluster potential function is considered, then represents the likelihood probability of the visual distribution of the kth image segmentation category weighted by the current superpixel, where represents the qth pixel in the superpixel centered at pixel i, θ k Parameters representing the visual probability distribution of the k-th image segmentation category.

2. The image segmentation method of the multi-scale superpixel spatial symbiotic mixture model according to claim 1 is characterized in that: In S1, the coupling coefficient β that controls the correlation between neighbors is set to a dynamic value that increases with the number of iterations, and β = f(t), where f(t) is a monotonically increasing function with the number of iterations as a variable.

3. The image segmentation method of the multi-scale superpixel spatial symbiotic mixture model according to claim 1 is characterized in that: Examples of monotonically increasing functions f(t) with the number of iterations as a variable include but are not limited to the following linear function form: f(t)=at+b, where t is the number of iterations, a is the slope of the coupling coefficient β function, and b is the initial value of the coupling coefficient β.

4. The image segmentation method of the multi-scale superpixel spatial symbiotic mixture model according to claim 1, characterized in that: In S3, within one round of training, superpixels of different scales are used cyclically to update the parameters of the hybrid model in sequence until the model parameters converge.

Images