Image segmentation method based on multi-view migration and multi-object coarse fuzzy clustering
By employing a multi-view transfer multi-objective coarse fuzzy clustering method, the problems of single viewpoint and noise influence in image segmentation are solved, achieving more accurate and stable image segmentation results.
Patent Information
- Application Number
- CN202311767177.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-20
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-12-20
AI Technical Summary
Existing image segmentation methods are limited by a single-view design when dealing with diverse clustering needs, which makes it impossible to fully explore the potential information between views. They are also susceptible to the influence of initialization centers and noise, resulting in inaccurate and unstable clustering results.
A coarse fuzzy clustering method based on multi-view transfer and multi-objective is adopted. Combining multi-view learning and transfer learning, a multi-view information transfer mechanism is designed. Through superpixel image inpainting strategy and adaptive hybrid cross operator optimization, a complementary coarse fuzzy clustering objective function is constructed to improve the search capability of cluster centers.
It enhances the accuracy and stability of image segmentation results, effectively overcomes the influence of noise, preserves image details, meets diverse practical needs, and improves the quality of segmentation results.
Smart Images

Figure CN117726816B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of digital image processing technology, specifically relating to an image segmentation method that can be used for natural image recognition and computer vision tasks. Background Technology
[0002] In the digital age, images, with their intuitive and information-rich characteristics, have become the preferred medium for humans to perceive the world and acquire information. How to efficiently and accurately extract key information from images using information processing technology has become a hot topic of common concern in many fields, and an important research direction in computer vision. Image segmentation, as one of the core technologies of computer vision, divides complex images into multiple regions with similar characteristics by utilizing features such as color and edges, thereby accurately extracting target information and providing strong data support for subsequent tasks such as target analysis and scene understanding. For complex and diverse application scenarios in reality, scholars have successively proposed a series of image segmentation methods, mainly including threshold-based segmentation methods, clustering-based segmentation methods, region-based segmentation methods, and edge-based segmentation methods. Among them, clustering-based segmentation methods have been widely used and researched due to their advantages of being unsupervised and efficient. Traditional hard clustering algorithms are difficult to handle the uncertain and fuzzy information contained in images; therefore, soft clustering algorithms have emerged. The coarse fuzzy C-means clustering algorithm effectively solves the above problems by combining rough set theory with fuzzy set theory.
[0003] Ji et al. proposed a generalized coarse fuzzy clustering algorithm for brain MRI image segmentation at the 2012 Computer Methods and Programs in Biomedicine conference. This method combines coarse fuzzy clustering with spatial information constraints to overcome the effects of noise and artifacts in brain MRI images. However, this method suffers from two main problems in image segmentation: First, the algorithm is limited to a single-view design, neglecting the diverse clustering needs in practice, thus limiting its generalization and application, and failing to accurately identify cluster structures. Second, the algorithm is based on a single objective function and uses the Lagrange multiplier method to optimize cluster centers, making it susceptible to the influence of initialization centers and noise, thus leading to local optima.
[0004] To address the first issue mentioned above, García et al. proposed a multi-view clustering algorithm based on feature and relational data at Applied Soft Computing in 2021. This algorithm uses different feature sets or dissimilarity relationships to represent views and clusters the data. However, this algorithm fails to fully consider the potential complementary information between views during the clustering process, making it difficult to effectively mine and utilize this information, thus affecting the accuracy of the clustering results.
[0005] To address the second issue mentioned above, Cheng et al. proposed a reference vector-guided multi-objective evolutionary algorithm in the 2016 IEEE Transactions on Evolutionary Computation. This algorithm not only overcomes the limitation of traditional clustering algorithms with a single clustering criterion, but its population-based nature also helps avoid the algorithm from easily getting trapped in local optima. However, because this multi-objective evolutionary algorithm requires the computation of a large number of real objective functions, its performance and efficiency are significantly affected. In response, Chugh et al. proposed using Gaussian processes to construct surrogate models to reduce time costs in the 2018 IEEE Transactions on Evolutionary Computation. However, this algorithm suffers from inefficiency in filling samples due to its inability to adaptively update the filled samples, thus affecting convergence and stability performance. Summary of the Invention
[0006] The purpose of this invention is to address the shortcomings of existing technologies by providing an image segmentation method based on multi-view migration and multi-objective coarse fuzzy clustering. This method fully explores and utilizes the potential information between different perspectives, and optimizes multiple complementary clustering objective functions to meet diverse practical needs, thereby enhancing the search capability for cluster centers and improving the accuracy and stability of image segmentation results.
[0007] To achieve the above objectives, the technology of the present invention includes the following steps:
[0008] (1) Input the image X to be segmented and set the initial parameter values, namely the number of views L, the number of clusters C, and the lower approximation weight w. low The superpixel information term weights β, migration coefficients λ, blurring factor m, population size NI, number of reference vectors H, and the number of evaluations of the maximum objective function FE. max , Proxy model update frequency ω, Maximum proxy model update frequency ω max ;
[0009] (2) Design a superpixel image inpainting strategy, combining the advantages of different superpixel methods to obtain reliable image information with image details.
[0010] 2a) Divide the image X to be segmented to generate superpixel images of non-uniform regions X. (M) ;
[0011] 2b) Divide the image X to be segmented to generate a superpixel image X of a uniform region. (S) ;
[0012] 2c) Superpixel image X of the non-uniform region generated according to steps (2a) and (2b) (M) and superpixel images of uniform regions X (S) The adaptive weighting coefficients are calculated, and the restored superpixel image is obtained by using a linear weighting method.
[0013] (3) The initial reference vector ν and the initial population P are generated using the simplex lattice method and the Latin hypercube sampling method, respectively.
[0014] (4) Combining the uncertainties of multi-view learning, transfer learning, and clustering, the following three objective functions are constructed respectively:
[0015] The coarse fuzzy compactness function F1 based on multi-view migration:
[0016] The fuzzy entropy function F2 with fusion boundary penalty:
[0017] Global fuzzy separation function F3 based on consensus membership:
[0018] Where L represents the number of views, C represents the number of clusters, N represents the number of pixels, λ represents the migration coefficient, J1 represents the coarse fuzzy clustering objective function, J2 represents the multi-view information migration objective function, and τ i μ represents the boundary penalty factor. lij S represents the degree of membership of the j-th pixel in the l-th view to the i-th class. cluster S is used to measure the similarity of pixels within a class. sample Used to measure the degree to which each pixel belongs to each class;
[0019] (5) Calculate the three objective function values for each individual in the initial population P, initialize the number of evaluations FE for these objective functions, i.e. set FE with the population size NI, and initialize the database DB with the initial population.
[0020] (6) Train a Bayesian surrogate model for each individual based on the three objective function values;
[0021] (7) Construct a multi-objective evolutionary framework based on adaptive hybrid crossover operators to update the initial population P:
[0022] 7a) Based on global adaptive coefficients An adaptive hybrid crossover operator strategy is designed to generate a progeny population, which is then merged with the initial population to obtain a candidate population. A Bayesian surrogate model is then used for prediction.
[0023] 7b) Perform environmental selection operations to select NI individuals from the candidate population as the next generation population;
[0024] 7c) Update frequency ω according to the set maximum proxy model. max Perform steps (7a) to (7b) for a total of ω max Next, the updated population is obtained;
[0025] 7d) Based on global adaptive coefficients Select u individuals from the updated population to update the database DB, and calculate the objective function values of u individuals to update the objective function evaluation number FE;
[0026] 7e) Compare the number of objective function evaluations FE with the maximum number of objective function evaluations FE. max To determine whether evolution has ceased:
[0027] If FE<FE max If so, then step (6) is executed for the current population;
[0028] Otherwise, use the database DB as the final population and execute step (8);
[0029] (8) Calculate the coarse fuzzy clustering effectiveness index I of the fused multi-view consensus information of individuals in the final population, select the optimal solution in the final population, and decode it to obtain the cluster center v. i ;
[0030] (9) Using the obtained cluster centers v i Calculate the global membership degree and determine the category of each pixel according to the maximum membership degree principle, and output the segmentation result of image X.
[0031] Compared with the prior art, the present invention has the following advantages:
[0032] First, this invention introduces multi-view learning and transfer learning into the coarse fuzzy clustering algorithm, and designs a coarse fuzzy clustering framework driven by a multi-view information transfer mechanism. This breaks through the limitation of the single view in traditional clustering algorithms and can fully explore and utilize the potential information between multiple views.
[0033] Second, this invention designs a superpixel image restoration strategy. By combining the advantages of different superpixel methods, it not only overcomes the impact of noise on the image, but also effectively solves the problems of image detail loss and negative migration.
[0034] Third, this invention constructs three complementary coarse fuzzy clustering objective functions based on different clustering effectiveness criteria, and optimizes them using a multi-objective evolutionary framework with an adaptive hybrid crossover operator, thereby improving the accuracy and stability of the segmentation results. Simultaneously, a coarse fuzzy clustering effectiveness index is designed, which not only meets diverse practical needs but also obtains representative solutions, improving the quality of the segmentation results. Attached Figure Description
[0035] Figure 1 This is a flowchart illustrating the implementation of the present invention;
[0036] Figure 2 The figure shows the simulation results of image segmentation in the Berkeley database using the present invention and existing methods;
[0037] Figure 3 The figure shows the simulation results of image segmentation of the Weizmann database using the present invention and existing methods. Detailed Implementation
[0038] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.
[0039] It should be noted that the step numbers in the specification and claims of this invention are only for the purpose of clearly describing the embodiments of this invention and facilitating understanding, and their order is not limited.
[0040] Reference Figure 1 The implementation steps of this example include the following:
[0041] Step 1: Input the image X to be segmented and set the initial parameter values.
[0042] Set the number of views L, the number of clusters C, and the lower approximation weight w. low The superpixel information term weights β, migration coefficients λ, blurring factor m, population size NI, number of reference vectors H, and the number of evaluations of the maximum objective function FE. max , Proxy model update frequency ω, Maximum proxy model update frequency ω max .
[0043] This example sets, but is not limited to, L=2, w low =0.9, β=0.8, λ=0.8, m=2, NI=50, H=40, FEmax =300, ω=1, ω max =20.
[0044] Step 2: Divide the image X to be segmented into superpixel images of non-uniform regions. (M) .
[0045] Existing methods for dividing an image into multiple non-uniform regions into superpixel images include the watershed algorithm and multi-scale morphological gradient reconstruction algorithms. This example uses, but is not limited to, the multi-scale morphological gradient reconstruction algorithm, which is implemented as follows:
[0046] (2.1) Perform multi-scale morphological gradient reconstruction operation on the image to be segmented X. It is represented as:
[0047]
[0048] Where g is the mask image of the image X to be segmented. It uses the set structured element radius r to perform a closed reconstruction operation on the image X to be segmented, where r1 is the minimum value of r', r2 is the maximum value of r', and ∨ represents the maximum value of the point;
[0049] (2.2) Set the minimum pixel as the starting point, set the minimum threshold, and make the threshold increase with iteration;
[0050] (2.3) Compare the geodesic distance from the neighboring pixels to the starting point with a set threshold:
[0051] If the geodesic distance from a neighboring pixel to the starting point is less than a set threshold, then these pixels are classified into one category.
[0052] Otherwise, set a dam value for these pixels;
[0053] (2.4) Iteration steps (2.2) to (2.3) continue until the dam value reaches the maximum pixel value, and the final segmentation result is obtained;
[0054] (2.5) Obtain the superpixel image X with non-uniform regions based on the segmentation results. (M) .
[0055] Step 3: Divide the image X to be segmented into superpixel images of uniform regions X. (S) .
[0056] Existing methods for dividing an image into multiple uniform regions of superpixel images include mean-shift algorithms and simple linear iterative clustering algorithms. This example uses, but is not limited to, the simple linear iterative clustering algorithm, which is implemented as follows:
[0057] (3.1) Distribute K seed points evenly on the image X to be segmented with N pixels, and move the seed points to the position with the minimum gradient in the 3×3 neighborhood;
[0058] (3.2) Calculate the distance D' between the pixel and the seed point:
[0059]
[0060] Where, d c It is the color distance, d s It is spatial distance, N c For the maximum color distance, N s The maximum spatial distance;
[0061] (3.3) Assign class labels to pixels within a 2S×2S neighborhood of each seed point according to the minimum distance principle, where the stride between adjacent seeds is... And use the center point of pixels with the same category as the seed point;
[0062] (3.5) Iterate through steps (3.2) to (3.3) until the maximum number of iterations is reached, then merge connected regions to obtain a superpixel image X with uniform regions. (S) .
[0063] Step 4: Calculate adaptive weighting coefficients based on the two superpixel images generated in Step 2 and Step 3, and generate the repaired superpixel image.
[0064] (4.1) Calculate the superpixel image X of the non-uniform region (M) and superpixel images of uniform regions X (S) Difference image X (D) :
[0065] X (D) =|X (M) -X (S) |
[0066] Where |·| represents absolute value;
[0067] (4.2) Calculate the difference image X respectively. (D) The adaptive weighting coefficients α on the three channels: red (R), green (G), and blue (B) R α G α B :
[0068]
[0069]
[0070]
[0071] in, and These represent the R, G, and B components of the difference image, respectively.
[0072] (4.3) According to and The adaptive weighting coefficient α is obtained:
[0073] α={α R ,α G ,α B};
[0074] (4.4) Generate the repaired superpixel image using a linear weighting method
[0075]
[0076] Step 5: Generate the initial reference vector ν.
[0077] Existing methods for generating initial reference vectors include the random direction method and the simplex lattice method. This example uses, but is not limited to, the simplex lattice method, which is implemented as follows:
[0078] (5.1) First, generate a normalized simplex lattice on the unit hyperplane in the first quadrant, and then combine it with... The constraints generate H uniformly distributed points on the hyperplane, where fn is the number of objective functions. Let v be the v-th point, where v∈[1,H];
[0079] (5.2) For the v-th point u v Standardization yields the unit reference vector ν v Then combine them to obtain the initial reference vector ν = {ν1,...,ν} v ,...,ν H}
[0080] Step 6: Generate the initial population P.
[0081] Existing methods for generating the initial population include uniform sampling, random number generator methods, and Latin hypercube sampling. This example uses, but is not limited to, the Latin hypercube sampling method, which is implemented as follows:
[0082] (6.1) Based on the number of categories C of the image to be segmented, generate an NI row (C×D) column matrix in the range [0,1], where D is the dimension;
[0083] (6.2) Calculate the difference between the maximum and minimum values in each column, and multiply the difference by the matrix to obtain the initial population P.
[0084] Step 7: Construct the coarse fuzzy compactness function F1 based on multi-view migration.
[0085] Existing metrics for representing views using dissimilarity measures include Euclidean distance, Mahalanobis distance, Manhattan distance, etc. This example uses, but is not limited to, Euclidean distance and Mahalanobis distance, and its implementation is as follows:
[0086] (7.1) Calculate the Euclidean distance D1(x) between two dissimilarity metrics in the image X to be segmented. j ,v i ) and Mahalanobis distance D2(x j ,v i ):
[0087]
[0088]
[0089] Where, x j Σ is the j-th pixel in the image to be segmented. i Let v be the covariance matrix. i It is the i-th cluster center;
[0090] (7.2) Calculate the restored superpixel image The two dissimilarity metrics in the Euclidean distance and Mahal distance :
[0091]
[0092]
[0093] in, It is a repaired superpixel image. The j-th pixel in;
[0094] (7.3) Based on the obtained and Calculate consensus membership :
[0095]
[0096] Where L represents the number of views, C represents the number of clusters, and m is the fuzzy factor;
[0097] (7.4) Based on the results of steps (7.1) to (7.3), calculate the parameter η. lj :
[0098]
[0099] Where β is the weighting coefficient that controls the superpixel information term, and λ represents the migration coefficient;
[0100] (7.5) Define the membership degree μ of the image to be segmented based on the above parameters. lij Approximate compactness function A and boundary compactness function B:
[0101]
[0102]
[0103]
[0104] Among them, L(C i B(C) represents the sample set of the lower approximate region in the i-th class. i ) represents the set of samples in the boundary region of the i-th class;
[0105] (7.6) Calculate the coarse fuzzy clustering objective function J1 and the multi-view information transfer objective function J2:
[0106]
[0107]
[0108] Among them, w low Indicates the approximate weights, Represents the empty set;
[0109] (7.7) Calculate the coarse fuzzy compactness function F1 based on the two functions J1 and J2:
[0110]
[0111] Where N represents the number of pixels.
[0112] Step 8: Construct the fuzzy entropy function F2 for the fusion boundary penalty.
[0113] (8.1) Calculate the boundary penalty factor τ i :
[0114] τ i =|L(C i )| / |C i |
[0115] Among them, |L(C i | represents the number of samples in the lower approximate region of the i-th class, |C i | represents the number of samples in the i-th class;
[0116] (8.2) The membership degree μ obtained from steps (7.5) and (8.1)lij and boundary penalty factor τ i Calculate the fuzzy entropy function F2 for the fusion boundary penalty:
[0117]
[0118] Step 9: Construct a global fuzzy separation function F3 based on consensus membership.
[0119] (9.1) Based on the consensus membership degree obtained in step (7.3) Design a function S to measure the similarity of pixels within a class. cluster :
[0120]
[0121] (9.2) Based on the consensus membership degree obtained in step (7.3) Designed to measure the membership degree S of each pixel to each class. sample :
[0122]
[0123] (9.3) Combining steps (9.1) and (9.2) to obtain S cluster and S sample Calculate the global fuzzy separation function F3 based on consensus membership:
[0124]
[0125] Step 10: Calculate the three objective function values y = {y1, y2, y3} for individuals in the initial population P, and initialize the evaluation number FE of the objective function and the database DB.
[0126] (10.1) Input the pixel values of individuals in the initial population P and the image to be segmented into the coarse fuzzy compactness function F1 based on multi-view transfer designed above, and calculate the coarse fuzzy compactness function value y1 of multi-view transfer.
[0127] (10.2) Input the pixel values of individuals in the initial population P and the image to be segmented into the fuzzy entropy function F2 of the fusion boundary penalty designed above, and calculate the fuzzy entropy function value y2;
[0128] (10.3) Input the pixel values of individuals in the initial population P and the image to be segmented into the global fuzzy separation function F3 based on consensus membership designed above, and calculate the global fuzzy separation function value y3;
[0129] (10.4) Set the number of evaluations of the objective function FE to NI times, and add individuals from the initial population P to the database DB.
[0130] Step 11: Train the Bayesian surrogate model.
[0131] (11.1) Based on the individuals in the initial population P and their objective function values y, construct the following conditional probability equation:
[0132]
[0133] Where μ and σ 2 These are the mean and variance, respectively.
[0134] (11.2) Take the partial derivatives of μ and σ in the conditional probability equation to obtain the estimated value of the mean. Estimates of variance
[0135] (11.3) Calculate the ratio of NI individuals in the initial population P to the predicted individual p. new The covariance matrix r between individuals, and the covariance matrix Cov between individuals in the initial population P:
[0136] r = (Corr(p) new ,p 1 ),...,Corr(p new ,p NI ))
[0137]
[0138] Among them, Corr(p i ,p j The ) represents the correlation between the i-th individual and the j-th individual;
[0139] (11.4) Based on the estimated value of the mean Estimated value of variance The covariance matrix r between the individual and the predicted individual, and the covariance matrix Cov between individuals, yield the following Bayesian surrogate model:
[0140]
[0141] Where, f(p) j )and Let represent the predicted value and variance of the objective function for the j-th individual, respectively.
[0142] Step 12: Based on global adaptive coefficients An adaptive hybrid crossover operator strategy is designed to generate offspring populations, which are then merged with the initial population to obtain candidate populations. A Bayesian surrogate model is then used for prediction.
[0143] (12.1) Calculate the global adaptive coefficients
[0144]
[0145] Where FE is the number of evaluations of the objective function, FE max It is the number of evaluations of the maximum objective function;
[0146] (12.2) Generate a random number o in the range [0,1], and compare the random number o with the global adaptive coefficient.
[0147] like Perform step (12.3);
[0148] Otherwise, proceed to step (12.4);
[0149] (12.3) Perform simulated binary crossover on all individuals to obtain offspring individuals:
[0150]
[0151] Among them, a 1 and a 2 It is the newly generated offspring individual, b 1 and b 2 It is an individual randomly selected from the initial population, where ρ is the propagation coefficient, which follows a multinomial probability distribution;
[0152] (12.4) Perform differential evolution on each individual to obtain offspring individuals:
[0153] d 1 =b 1 +υ·(b 2 -b 3 )
[0154] Where, d 1 It is a newly generated offspring individual, b 1 b 2 and b 3 These are three individuals randomly selected from the initial population, and υ is the coefficient of variation.
[0155] (12.5) Combine all the new offspring individuals to obtain the offspring population;
[0156] (12.6) Merge the offspring population and the initial population P to obtain a candidate population;
[0157] (12.7) Input the individuals in the candidate population into the three Bayesian surrogate models that predict the objective function values, and obtain the three objective function values of the individuals in the candidate population.
[0158] Step 13: Perform the environment selection operation to select the next generation of population.
[0159] (13.1) Based on the three prediction objective function values of the candidate population, obtain the non-dominated ranking of individuals in the candidate population;
[0160] (13.2) Sort the non-dominant ranking levels in ascending order and select the top NI individuals as the next generation population;
[0161] (13.3) Determine whether the proxy model update frequency ω has reached the maximum proxy model update frequency ω. max :
[0162] If so, reset the agent model update frequency ω to 1, output the updated population P', and execute step 14;
[0163] Otherwise, set the proxy model update frequency ω = ω + 1 and return to step 12.
[0164] Step 14: Based on global adaptive coefficients Select u individuals from the updated population P' to update the database DB.
[0165] (14.1) Global adaptive coefficients Compare with the constant 0.5:
[0166] like Then, population diversity should be given priority, and steps (14.2) through (14.3) should be performed;
[0167] Otherwise, proceed with steps (14.4) through (14.5);
[0168] (14.2) Based on the updated objective function value f' of the population P', calculate the angle between the updated population P' and the reference vector ν.
[0169]
[0170] Wherein, the transformation target vector of the i-th individual in the updated population P' γ ν It is the minimum angle between the reference vectors ν. yes The angle between the vector and the k-th reference vector ν;
[0171] (14.3) Sort the samples in ascending order and select the first u as filler samples, then add them to the database DB.
[0172] (14.4) Combine the updated population P' objective function value f' with the updated population P' objective function value f' to calculate the updated population P' angle penalty distance APD. ik :
[0173]
[0174] in, yes The penalty function;
[0175] (14.5) Regarding APD ik Sort the samples in ascending order and select the first u as filler samples, then add them to the database DB.
[0176] Step 15: Calculate the objective function values of u individuals, update the objective function evaluation times FE, and determine whether the evolution has terminated.
[0177] (15.1) Input u individuals into the three objective functions designed in steps 7-9 respectively. For each individual's objective function value, the number of evaluations of the objective function FE will increase by one.
[0178] (15.2) Determine whether the number of evaluations of the objective function FE has reached the maximum number of evaluations of the objective function FE. max :
[0179] If so, then use the database DB as the final population and proceed to step 16;
[0180] Otherwise, return to step 11.
[0181] Step 16: Select the optimal solution from the final population and calculate the global membership degree to obtain the image segmentation result.
[0182] (16.1) Consensus membership degree obtained from steps (7.3) and (7.5) and membership degree μ lij Calculate the lower approximate migration compactness function A1 and the boundary migration compactness function B1 for individuals in the final population:
[0183]
[0184] (16.2) Based on the two compactness functions A1 and B1, calculate the coarse fuzzy clustering compactness measure E(C) and separability measure S(C):
[0185]
[0186]
[0187] (16.3) Based on the global fuzzy separation function F3 constructed in step 9, calculate the I value of each individual in the final population, and select the individual with the largest I value as the optimal solution:
[0188]
[0189] Where E(1) is a constant;
[0190] (16.4) Obtain the cluster center v from the optimal decoding. i ;
[0191] (16.5) Based on step (7.5) and cluster center v i Obtain the membership degree μ of the image to be segmented under different views. lij Calculate the global membership degree μ′ of the image to be segmented. ij :
[0192]
[0193] (16.6) Based on the principle of maximum membership, each pixel in the image to be segmented is assigned to its maximum global membership degree μ′. ij The final segmentation result of the image is obtained by determining its category.
[0194] The technical effects of the present invention will be further explained below with reference to simulation experiments:
[0195] 1. Simulation conditions:
[0196] The simulation experiment was conducted on a computer with an Intel(R) Core(TM) i7-12700 CPU, 16GB of memory, and MATLAB R2021a software environment.
[0197] 2. Simulation content:
[0198] Simulation 1: The present invention, along with existing methods such as RCM, RFCM, TFCM, FCM-SICM, sRFCM, KRVEA, and BLS-MOEFC, were used to segment image 260058 in the Berkeley database. The results are as follows: Figure 2 As shown, where:
[0199] Figure 2 (a) is the original image of image 260058;
[0200] Figure 2 (b) is the standard segmentation map of image 260058;
[0201] Figure 2 (c) is the segmentation result of the 260058 image using the existing RCM method;
[0202] Figure 2 (d) is the segmentation result of image 260058 using the existing RFCM method;
[0203] Figure 2(e) is the segmentation result of the 260058 image using the existing TFCM method;
[0204] Figure 2 (f) is the segmentation result of the 260058 image using the existing FCM-SICM method;
[0205] Figure 2 (g) is the segmentation result of the 260058 image using the existing sRFCM method;
[0206] Figure 2 (h) shows the segmentation results of image 260058 using the existing KRVEA method;
[0207] Figure 2 (i) is the segmentation result of the 260058 image using the existing BLS-MOEFC method;
[0208] Figure 2 (j) is the segmentation result of image 260058 using the method of the present invention.
[0209] from Figure 2 As can be seen, the segmentation results of this invention, compared with the segmentation results of other comparison algorithms, can more completely divide the desert and sky parts, while maintaining clear contour edges. Its segmentation effect is significantly better than existing RCM, RFCM, TFCM, FCM-SICM, sRFCM, KRVEA and BLS-MOEFC methods.
[0210] Simulation 2: The image numbered 113334665744 in the Weizmann database was segmented using the present invention and existing methods such as RCM, RFCM, TFCM, FCM-SICM, sRFCM, KRVEA, and BLS-MOEFC. The results are as follows: Figure 3 As shown, where:
[0211] Figure 3 (a) is the original image of image 113334665744;
[0212] Figure 3 (b) is the standard segmentation map of image 113334665744;
[0213] Figure 3 (c) is the segmentation result of image 113334665744 using the existing RCM method;
[0214] Figure 3 (d) is the segmentation result of image 113334665744 using the existing RFCM method;
[0215] Figure 3 (e) is the segmentation result of image 113334665744 using the existing TFCM method;
[0216] Figure 3 (f) is the segmentation result of image 113334665744 using the existing FCM-SICM method;
[0217] Figure 3 (g) is the segmentation result of image 113334665744 using the existing sRFCM method;
[0218] Figure 3 (h) is the segmentation result of image 113334665744 using the existing KRVEA method;
[0219] Figure 3 (i) is the segmentation result of image 113334665744 using the existing BLS-MOEFC method;
[0220] Figure 3 (j) is the segmentation result of image 113334665744 using the method of the present invention.
[0221] from Figure 3 It can be seen that the segmentation result of the present invention is most similar to the standard segmentation image, and can accurately delineate the overall outline of the goose from the uneven lake background. Its segmentation effect is significantly better than the existing RCM method, RFCM method, TFCM method, FCM-SICM method, sRFCM method, KRVEA method, and BLS-MOEFC method.
Claims
1. An image segmentation method based on multi-view transfer and multi-object coarse fuzzy clustering, characterized in that, Includes the following steps: (1) Input the image X to be segmented and set the initial parameter values, namely the number of views L, the number of clusters C, and the lower approximation weight w. low The superpixel information term weights β, migration coefficients λ, blurring factor m, population size NI, number of reference vectors H, and the number of evaluations of the maximum objective function FE. max , Proxy model update frequency ω, Maximum proxy model update frequency ω max ; (2) Design a superpixel image inpainting strategy, combining the advantages of different superpixel methods to obtain reliable image information with image details. 2a) Divide the image X to be segmented to generate superpixel images of non-uniform regions X. (M) ; 2b) Divide the image X to be segmented to generate a superpixel image X of a uniform region. (S) ; 2c) Superpixel image X of the non-uniform region generated according to steps (2a) and (2b) (M) and superpixel images of uniform regions X (S) The adaptive weighting coefficients are calculated, and the restored superpixel image is obtained by using a linear weighting method. (3) The initial reference vector ν and the initial population P are generated using the simplex lattice method and the Latin hypercube sampling method, respectively. (4) Combining the uncertainties of multi-view learning, transfer learning, and clustering, the following three objective functions are constructed respectively: The coarse fuzzy compactness function F1 based on multi-view migration: The fuzzy entropy function F2 with fusion boundary penalty: Global fuzzy separation function F3 based on consensus membership: Where L represents the number of views, C represents the number of clusters, N represents the number of pixels, λ represents the migration coefficient, J1 represents the coarse fuzzy clustering objective function, J2 represents the multi-view information migration objective function, and τ i μ represents the boundary penalty factor. lij S represents the degree of membership of the j-th pixel in the l-th view to the i-th class. cluster S is used to measure the similarity of pixels within a class. sample Used to measure the degree to which each pixel belongs to each class; (5) Calculate the three objective function values for each individual in the initial population P, initialize the number of evaluations FE for these objective functions, i.e. set FE with the population size NI, and initialize the database DB with the initial population. (6) Train a Bayesian surrogate model for each individual based on the three objective function values; (7) Construct a multi-objective evolutionary framework based on adaptive hybrid crossover operators to update the initial population P: 7a) Based on global adaptive coefficients An adaptive hybrid crossover operator strategy is designed to generate a progeny population, which is then merged with the initial population to obtain a candidate population. A Bayesian surrogate model is then used for prediction. 7b) Perform environmental selection operations to select NI individuals from the candidate population as the next generation population; 7c) Update frequency ω according to the set maximum proxy model. max Perform steps (7a) to (7b) for a total of ω max Next, the updated population is obtained; 7d) Based on global adaptive coefficients Select u individuals from the updated population to update the database DB, and calculate the objective function values of u individuals to update the objective function evaluation number FE; 7e) Compare the number of objective function evaluations FE with the maximum number of objective function evaluations FE. max To determine whether evolution has ceased: If FE<FE max If so, then step (6) is executed for the current population; Otherwise, use the database DB as the final population and execute step (8); (8) Calculate the coarse fuzzy clustering effectiveness index I of the fused multi-view consensus information of individuals in the final population, select the optimal solution in the final population, and decode it to obtain the cluster center v. i ; (9) Using the obtained cluster centers v i Calculate the global membership degree and determine the category of each pixel according to the maximum membership degree principle, and output the segmentation result of image X.
2. The method according to claim 1, characterized in that, In step 2a), the image to be segmented X is divided into superpixel images of non-uniform regions. (M) It is implemented using a multi-scale morphological gradient reconstruction algorithm, and the steps include the following: (2a1) Perform multi-scale morphological gradient reconstruction on the image to be segmented X. Where g is the mask image of the image X to be segmented. It uses the set structured element radius r to perform a closed reconstruction operation on the image X to be segmented, where r1 is the minimum value of r', r2 is the maximum value of r', and ∨ represents the maximum value of the point; (2a2) Set the minimum pixel as the starting point and set a threshold to make it grow from the minimum value; (2a3) Compare the geodesic distance from the starting point to the set threshold: If the geodesic distance from a neighboring pixel to the starting point is less than a set threshold, then these pixels are classified into one category. Otherwise, set a dam value for these pixels; (2a4) Iteration steps (2a2) - (2a3) continue until the dam value reaches the maximum pixel value, and the final segmentation result is obtained; (2a5) Obtain the superpixel image X with non-uniform regions based on the segmentation results. (M) .
3. The method according to claim 1, characterized in that, In step 2b), the image to be segmented X is divided into superpixel images of uniform regions X. (S) It is implemented using a simple linear iterative clustering algorithm, and the steps include the following: (2b1) Distribute K seed points evenly on the image X to be segmented with N pixels, and move the seed points to the position with the minimum gradient in the 3×3 neighborhood; (2b2) Calculate the distance D' between the pixel and the seed point: Where, d c It is the color distance, d s It is spatial distance, N c For the maximum color distance, N s The maximum spatial distance; (2b3) Assign class labels to pixels within a 2S×2S neighborhood of each seed point according to the minimum distance principle, where the stride of adjacent seeds is... And use the center point of pixels with the same category as the seed point; (2b4) Iteration steps (2b2) – (2b3) continue until the maximum number of iterations is reached, then merge connected regions to obtain a superpixel image X with uniform regions. (S) .
4. The method according to claim 1, characterized in that, In step 2c), based on the superpixel image X of the non-uniform region... (M) and superpixel images of uniform regions X (S) The adaptive weighting coefficients are calculated, and the restored superpixel image is obtained by using a linear weighting method. The implementation is as follows: (2c1) Calculate the superpixel image X of the non-uniform region. (M) and superpixel images of uniform regions X (S) Difference image X (D) : X (D) =|X (M) -X (S) | Where |·| represents absolute value; (2c2) Calculate the adaptive weighting coefficients α for the red (R), green (G), and blue (B) channels respectively. R α G α B : in, and These represent the R, G, and B components of the difference image, respectively. (2c3) According to and The adaptive weighting coefficient α is obtained: α={α R ,α G ,α B}; (2c4) Generate the repaired superpixel image based on the adaptive weighting coefficient α.
5. The method according to claim 1, characterized in that, In step (3), the initial reference vector ν and the initial population P are generated using the simplex lattice method and the Latin hypercube sampling method, respectively, as follows: (3a) Generate the initial reference vector ν using the simplex lattice method: (3a1) Generate a standard simplex lattice on the unit hyperplane in the first quadrant, and then combine it with... The constraints generate H uniformly distributed points on the hyperplane, where fn is the number of objective functions. Let v be the v-th point, where v∈[1,H]; (3a2) for u v Standardization yields the unit reference vector ν v Then combine them to obtain the initial reference vector ν = {ν1,...,ν} v ,...,ν H }; (3b) Initial population P using Latin hypercubic sampling: (3b1) Generate an NI row (C×D) column matrix in the range [0,1] based on the number of categories C of the image to be segmented, where D is the dimension; (3b2) Calculate the difference between the maximum and minimum values in each column, and multiply the difference by the matrix to obtain the initial population P.
6. The method according to claim 1, characterized in that, In step (4), three objective functions F1, F2, and F3 are constructed and implemented as follows: (4a) Calculate consensus membership Where L represents the number of views, C represents the number of clusters, and m is the fuzzy factor. v represents the j-th pixel in the repaired superpixel image. i It is the cluster center of the i-th class; (4b) Obtain the consensus membership degree according to step (4a). Calculate parameter η lj : Where β is the weighting coefficient controlling the superpixel information term, λ represents the transfer coefficient, and x j This represents the j-th pixel in the image to be segmented; (4c) Define the membership degree μ of the image to be segmented. lij Approximate compactness function A and boundary compactness function B: Among them, L(C i B(C) represents the sample set of the lower approximate region in the i-th class. i ) represents the set of samples in the boundary region of the i-th class; (4d) Calculate the coarse fuzzy clustering objective function J1 and the multi-view information transfer objective function J2: Among them, w low Indicates the approximate weights, Represents the empty set; (4e) Based on step (4d), calculate the coarse fuzzy compactness function F1 based on multi-view migration using the two functions J1 and J2: Where N represents the number of pixels; (4f) Calculate the boundary penalty factor τ i : t i =|L(C i )| / |C i | Among them, |L(C i | represents the number of samples in the lower approximate region of the i-th class, |C i | represents the number of samples in the i-th class; (4g) The membership degree μ obtained from steps (4c) and (4f) lij and boundary penalty factor τ i Calculate the fuzzy entropy function F2 for the fusion boundary penalty: (4h) Combine the consensus membership degree obtained in step (4a) Design a function S to measure the similarity of pixels within a class. cluster and S, used to measure the membership degree of each pixel to each class. sample : (4i) Based on the results of step (4h), calculate the global fuzzy separation function F3 based on consensus membership:
7. The method according to claim 1, characterized in that, In step (6), a Bayesian surrogate model is trained based on the three objective function values for each individual, as follows: (6a) Construct the following conditional probability equation using the individuals in the initial population P and their objective function values y: Where μ and σ 2 These are the mean and variance, respectively. (6b) Take the partial derivatives of μ and σ in the conditional probability equation to obtain the estimated value of the mean. Estimates of variance (6c) Based on the estimated value of the mean Estimated value of variance The covariance matrix r between the individual and the predicted individual, and the covariance matrix Cov between individuals, yield the following Bayesian surrogate model: Where, f(p) j )and Let represent the predicted value and variance of the objective function for the j-th individual, respectively.
8. The method according to claim 1, characterized in that, Step 7a) based on global adaptive coefficients Design an adaptive hybrid crossover operator strategy to generate the offspring population, as follows: (7a1) Calculate the global adaptive coefficients Where FE is the number of evaluations of the objective function, FE max It is the number of evaluations of the maximum objective function; (7a2) Generate a random number o in the range [0,1], and compare the random number o with the global adaptive coefficient. like Perform step (7a3); Otherwise, proceed to step (7a4); (7a3) Perform simulated binary crossover on all individuals to obtain offspring individuals: Among them, a 1 and a 2 It is the newly generated offspring individual, b 1 and b 2 It is an individual randomly selected from the initial population, where ρ is the propagation coefficient, which follows a multinomial probability distribution; (7a4) Perform differential evolution on each individual to obtain offspring individuals: d 1 =b 1 +υ·(b 2 -b 3 ) Where, d 1 It is a newly generated offspring individual, b 1 b 2 and b 3 These are three individuals randomly selected from the initial population, and υ is the coefficient of variation. (7a5) Combine all the new offspring individuals to obtain the offspring population.
9. The method according to claim 1, characterized in that, Step 7d) Based on global adaptive coefficients Selecting u individuals from the updated population to update the database DB involves adjusting the global adaptive coefficients. Compare with the constant 0.5: like Prioritize population diversity and select u filler samples with the smallest angles to add to the database DB; Otherwise, select u filler samples with the minimum angular penalty distance (APD) and add them to the database DB.
10. The method according to claim 1, characterized in that, In step 7d), the number of evaluations of the objective function FE for calculating the objective function value of u individuals is to input u filled sample individuals into the three objective functions designed in step (4). For each individual's objective function value calculated, the number of evaluations of the objective function FE will increase by one.
11. The method according to claim 1, characterized in that, In step (8), the coarse fuzzy clustering effectiveness index I of the fused multi-view consensus information of individuals in the final population is calculated, and the optimal solution is selected in the final population as follows: (8a) Based on the membership degree μ of the image to be segmented under different views lij and consensus membership The lower approximate migration compactness function A1 and the boundary migration compactness function B1 are defined as follows: Where C represents the number of clusters, β is the weighting coefficient controlling the superpixel information term, and v i Let x be the i-th cluster center. j and Let L(C) represent the j-th pixel in the image to be segmented and the restored superpixel image, respectively. i B(C) represents the sample set of the lower approximate region in the i-th class. i ) represents the set of samples in the boundary region of the i-th class; (8b) Calculate the coarse fuzzy clustering compactness measure E(C) and separability measure S(C) based on the two compactness functions A1 and B1: Where L represents the number of views, w low Indicates the approximate weights, Represents the empty set; (8c) Based on the global fuzzy separation function F3 constructed in step (4) based on consensus membership, calculate the I value of each individual in the final population, and select the individual with the largest I value as the optimal solution: Where E(1) is a constant.
12. The method according to claim 1, characterized in that, In step (9), the obtained cluster centers v are used i Calculate the global membership degree and determine the category of each pixel according to the maximum membership degree principle, as follows: (9a) Based on the membership degree μ of the image to be segmented under different views lij Calculate the global membership degree μ′ of the image to be segmented. ij : Where L represents the number of views; (9b) The category of each pixel in the image to be segmented is determined by combining the global membership degree and the maximum membership degree principle, as shown in the following formula: Where C represents the number of clusters, Y j The label represents the i-th pixel.
Citation Information
Patent Citations
Brain image segmentation method based on evolutionary multi-view migration rough fuzzy clustering
CN116612278A
Systems and methods for image / video recoloring, color standardization, and multimedia analytics
US20160314567A1
