Method for segmenting rgb-d images based on weighted rough membership clustering of complex distances

Abstract

The application discloses a kind of based on composite distance's weighted rough membership clustering RGB-D image segmentation method.For extracting the image feature of RGB-D image, the image feature includes color feature, point cloud feature and normal feature;Initial class center is selected based on density method;Based on adaptive weighting and composite distance measurement method, clustering algorithm is optimized, and weighted rough membership clustering algorithm is obtained;Adaptive weighting includes by local feature weighting method to each feature dimension adaptive weighting;Composite distance measurement method includes by the method of fusing Euclidean distance and cosine distance to calculate the similarity between sample and clustering center;Based on weighted rough membership clustering algorithm, RGB-D image is clustered and region merging is carried out, and segmentation image is generated.The application can better adapt to the characteristics of different types of features by introducing feature weighting and improving similarity measurement method, and ultimately realize higher quality segmentation effect.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and particularly relates to a weighted coarse membership clustering RGB-D image segmentation method based on composite distance. Background Technology

[0002] In 2010, Microsoft launched the Kinect camera, which can capture depth information in a scene. Incorporating depth information into RGB images has enhanced computer vision's ability to process images. Depth information is unaffected by factors such as lighting and shadows. Combining RGB images with depth images to generate RGB-D images allows for better handling of uncertain regions in color images. Currently, RGB-D image segmentation is widely used in autonomous driving, robot navigation, smart homes, and agricultural automation. Existing RGB-D image segmentation methods mainly include deep learning-based methods and methods based on traditional segmentation techniques. Deep learning-based RGB-D image segmentation methods offer high segmentation accuracy, but their training process requires a large amount of labeled sample data and places high demands on hardware performance. In contrast, traditional image segmentation methods are widely used due to their convenience and speed. Traditional image segmentation algorithms mainly include threshold-based methods, edge detection-based methods, and clustering-based methods. Clustering-based RGB-D image segmentation algorithms utilize only the information within the image itself, have low algorithmic complexity, and can achieve satisfactory segmentation results even in the absence of labeled data or prior knowledge.

[0003] Existing clustering-based RGB-D image segmentation algorithms directly segment images without considering the multimodal features of RGB-D images. The direct segmentation process fails to account for the differences between different modal features, making it difficult to effectively fuse information from various features. Furthermore, existing methods use Euclidean distance to calculate the similarity between pixels; however, since features of different modalities may have different attributes and distributions, Euclidean distance cannot accurately measure their similarity. To address these issues and improve the segmentation accuracy of RGB-D images, this invention proposes a weighted composite distance-rough membership C-means clustering (WCD-RMCM) RGB-D image segmentation method. A composite distance for RGB-D multimodal features is constructed, and an improved weighted coarse membership clustering algorithm is proposed. Summary of the Invention

[0004] This invention proposes a weighted coarse membership clustering RGB-D image segmentation method based on composite distance to solve the problems existing in the prior art.

[0005] To achieve the above objectives, this invention provides a weighted coarse membership clustering RGB-D image segmentation method based on composite distance, comprising the following steps:

[0006] Extract image features from an RGB-D image, including color features, point cloud features, and normal features;

[0007] Density-based methods select class centers by using high-density regions as initial class centers.

[0008] The clustering algorithm is optimized based on adaptive weighting and composite distance metric methods to obtain a weighted coarse membership clustering algorithm;

[0009] Adaptive weighting involves adaptively weighting each feature dimension using a local feature weighting method;

[0010] Composite distance metrics include methods that combine Euclidean distance and cosine distance to calculate the similarity between samples and cluster centers;

[0011] The weighted coarse membership clustering algorithm is used to cluster RGB-D images, and the clustering results are merged to generate segmented images.

[0012] Preferably, when selecting the initial class center, the high-density point is selected as the initial class center by calculating the neighborhood density of each point.

[0013] Preferably, in adaptive weighting, different weights are assigned to color features, point cloud features, and normal features according to the importance of each feature in clustering, and the weights are dynamically adjusted according to the intra-cluster distance during the iteration process.

[0014] Preferably, in the composite distance metric method, Euclidean distance is used to calculate the similarity of color features and point cloud features, cosine distance is used to calculate the similarity of normal features, and the different distance metrics are normalized by a linear normalization method.

[0015] Preferably, the expression for the weighted coarse membership clustering algorithm is:

[0016]

[0017] In the formula, n, M, and C represent the number of data samples, the spatial dimension of the features, and the number of cluster centers, respectively, and W cm W represents the feature weight of the m-th dimension in the c-th class, where q is the exponential parameter used to adjust the weights W. cm The intensity of the effect on the feature, For sample x i To cluster center x cBased on the rough membership degree of the lattice neighborhood binary relation R, d f Represents sample point x i To cluster center b c The composite distance, x im Let b represent the m-th dimension feature of the i-th sample. cm Let m represent the m-th feature of the c-th cluster center.

[0018] Preferably, this method updates the coarse membership degree by using the Lagrangian function and constraints, comprehensively considering the local structure and global characteristics of the data.

[0019] Preferably, the region merging includes: treating each labeled region as a node, merging nodes that meet the conditions by checking the candidate eligibility, merging eligibility and merging consistency of the nodes, until there are no more candidate nodes that can be merged.

[0020] This invention also provides a weighted coarse membership clustering RGB-D image segmentation system based on composite distance, comprising:

[0021] The feature extraction module is used to extract image features from RGB-D images, including color features, point cloud features, and normal features.

[0022] An initialization module is used to improve the traditional random initial class center selection method by selecting points in high-density regions as initial class centers.

[0023] The optimization module is used to optimize the clustering algorithm based on adaptive weighting and composite distance metric methods to obtain a weighted coarse membership clustering algorithm.

[0024] Among them, adaptive weighting includes adaptive weighting of each feature dimension through local feature weighting;

[0025] Among them, the composite distance metric method includes a composite distance metric method that combines Euclidean distance and cosine distance to calculate the similarity between samples and cluster centers;

[0026] The image segmentation module is used to cluster RGB-D images based on a weighted coarse membership clustering algorithm, and to merge the clustering results to generate segmented images.

[0027] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0028] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method.

[0029] Compared with the prior art, the present invention has the following advantages and technical effects:

[0030] This invention discloses a weighted coarse membership clustering method for RGB-D image segmentation based on composite distance, comprising: extracting image features from the RGB-D image, including color features, point cloud features, and normal features; initial cluster center selection based on density; optimizing the clustering algorithm based on adaptive weighting and composite distance metric to obtain a weighted coarse membership clustering algorithm; adaptive weighting including adaptive weighting of each feature dimension through local feature weighting; composite distance metric including calculating the similarity between samples and cluster centers through a composite distance metric that fuses Euclidean distance and cosine distance; clustering the RGB-D image based on the weighted coarse membership clustering algorithm, and merging the clustering results into regions to generate a segmented image. This invention employs an adaptive weighting method, assigning different weights to different features according to their importance and contribution. This strategy aims to help the algorithm fully explore the potential relationships between features, enabling different features to complement each other, adapt to the characteristics of different features, and achieve effective information fusion. To address the differences in representation and data distribution characteristics among various features, this invention designs a differentiated similarity measurement method: for RGB and point cloud features, Euclidean distance is used to measure their spatial distribution similarity; for normal features, cosine distance is selected to evaluate geometric structural similarity based on their directional characteristics. This method effectively preserves the inherent characteristics of the features and enhances the generalization ability for complex tasks. In summary, by introducing feature weighting and improving the similarity measurement method, this invention effectively captures the inherent characteristics of different modalities of data, enhances the adaptability to complex data, and achieves higher-quality segmentation accuracy. Attached Figure Description

[0031] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0032] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram illustrating the impact of feature weighting on image clustering and segmentation quality according to an embodiment of the present invention.

[0034] Figure 3 This is a schematic diagram illustrating the impact of composite distance on image clustering and segmentation quality according to an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram illustrating the time consumption of the segmentation results of each method in the embodiments of the present invention;

[0036] Figure 5 The original images of 01 and 08 from the NYUD2 RGB-D dataset;

[0037] Figure 6 The comparison method 1 (hard C-means clustering algorithm) of this invention is shown as the segmentation result of NYUD2 RGB-D images 01 and 08.

[0038] Figure 7 The comparison method 2 (fast robust fuzzy C-means clustering algorithm) of this invention is shown as the segmentation result of NYUD2 RGB-D images 01 and 08.

[0039] Figure 8 The comparison method 3 (multilayer clustering algorithm) of this invention is shown as the segmentation result of NYUD2 RGB-D images 01 and 08.

[0040] Figure 9 The comparison method 4 (joint color-space-orientation clustering and region merging algorithm) of this invention is shown as the segmentation result of NYUD2 RGB-D images 01 and 08.

[0041] Figure 10 The comparison method 5 (chaotic kbest gravity search algorithm + fuzzy c-means clustering algorithm) of this invention is shown as the segmentation result of NYUD2 RGB-D images 01 and 08.

[0042] Figure 11 The comparison method 6 (random Henry gas solubility optimization algorithm + fuzzy c-means clustering algorithm) of the present invention is shown as the segmentation result of NYUD2 RGB-D images 01 and 08.

[0043] Figure 12 The image shows the segmentation results of the method of the present invention on NYUD2 RGB-D images 01 and 08 in an embodiment of the present invention.

[0044] Figure 13 These are the original images 41 and 43 from the SUN RGB-D dataset.

[0045] Figure 14 The comparison method 1 (hard C-means clustering algorithm) of this invention is shown as the segmentation result on SUN RGB-D images 41 and 43.

[0046] Figure 15 The comparison method 2 (fast robust fuzzy C-means clustering algorithm) of this invention is shown as the segmentation result on SUN RGB-D images 41 and 43.

[0047] Figure 16The comparison method 3 (multilayer clustering algorithm) of this invention is shown as the segmentation result on SUN RGB-D images 41 and 43.

[0048] Figure 17 The comparison method 4 (joint color-space-orientation clustering and region merging algorithm) of this invention is shown as the segmentation result on SUN RGB-D images 41 and 43.

[0049] Figure 18 The comparison method 5 (chaotic kbest gravity search algorithm + fuzzy c-means clustering algorithm) of this invention is shown as the segmentation result on SUN RGB-D images 41 and 43.

[0050] Figure 19 The comparison method 6 (random Henry gas solubility optimization algorithm + fuzzy C-means clustering algorithm) of this invention is shown as the segmentation result on SUN RGB-D images 41 and 43.

[0051] Figure 20 The image shows the segmentation results of the method of the present invention on SUN RGB-D images 41 and 43, according to an embodiment of the present invention. Detailed Implementation

[0052] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0053] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0054] RGB-D images play a crucial role in autonomous driving technology, providing vehicles with precise 3D environmental perception capabilities by fusing color information and depth data. These images not only enable autonomous driving systems to effectively detect and avoid obstacles, but also help vehicles identify and track lane lines, detect free space, and recognize traffic signs and signals. Furthermore, RGB-D images have significant applications in vehicle localization and mapping, human-machine interaction, augmented reality displays, path planning, and vehicle control. They allow vehicles to understand their surroundings more comprehensively and accurately, thereby significantly improving the safety and reliability of autonomous driving and providing strong support for the development of autonomous driving technology.

[0055] Example 1

[0056] like Figure 1As shown, this embodiment provides a weighted coarse membership clustering RGB-D image segmentation method based on composite distance, including the following steps:

[0057] Extract image features from RGB-D images, including color features, point cloud features, and normal features;

[0058] Density-based class center selection uses points in high-density regions as initial class centers.

[0059] The clustering algorithm is optimized based on adaptive weighting and composite distance metric methods to obtain a weighted coarse membership clustering algorithm;

[0060] Adaptive weighting involves adaptively weighting each feature dimension using a local feature weighting method;

[0061] Composite distance metrics include methods that combine Euclidean distance and cosine distance to calculate the similarity between samples and cluster centers;

[0062] The weighted coarse membership clustering algorithm is used to cluster RGB-D images, and the clustering results are merged to generate segmented images.

[0063] The specific technical solution consists of the following steps:

[0064] (1) Initialization phase;

[0065] For multi-object image segmentation tasks, a reasonable initial class center strategy can significantly improve segmentation accuracy and accelerate algorithm convergence. By using high-density points as initial class centers, the inherent convergence instability of random initialization methods can be effectively overcome, thereby improving image segmentation accuracy. First, K similar samples are grouped into a dense point, called the "density". Its calculation formula is as follows:

[0066]

[0067] Where, d f (x i ,x j ) represents the sample point x i Its neighboring sample points x j The composite distance between them. First, by calculating the sample point x i The local density of a sample point is obtained by summing the distances of its K nearest neighbors. Then, the sample densities are sorted, and a density threshold ρ is set. t Points with a density greater than the threshold are selected as high-density points. Finally, C points are randomly selected from these high-density points as initial cluster centers.

[0068] b in_c ={xi |ρ i >ρ t} (2)

[0069] Where b in_c This indicates the initialization of cluster centers.

[0070] (2) Construction of feature-weighted clustering criterion function;

[0071] In multimodal data analysis, different modal features often exhibit significant differences in importance, with some key features playing a more significant discriminative role in clustering results. However, current image segmentation algorithms based on traditional FCM generally fail to adequately consider the differences in the importance of multimodal features, thus failing to fully reflect the dominant role of key features in the optimal clustering process. To address this issue, a local feature weighting scheme is designed to improve clustering accuracy, adaptively weighting each feature dimension. This allows different features to be adjusted according to their actual importance and contribution when calculating similarity. The adaptive weights can be dynamically adjusted through prior analysis of the dataset or feedback from the training set, thereby optimizing the entire clustering process. Considering all the above, the improved feature-weighted clustering criterion function is expressed as:

[0072]

[0073] In formula (3), n, M, and C represent the number of data samples, the spatial dimension of the features, and the number of cluster centers, respectively, and W cm W represents the feature weight of the m-th dimension in the c-th class, where q is the exponential parameter used to adjust the weights W. cm The intensity of the effect on the feature, For x i To x c Based on the rough membership degree of the lattice neighborhood binary relation R, d f Represents sample point x i To cluster center b c The composite distance, x im Let b represent the m-th dimension feature of the i-th sample. cm Let m represent the m-th feature of the c-th cluster center.

[0074] (3) Calculation of composite distance;

[0075] Compared to traditional RGB images, RGB-D images, in addition to color features In addition, it also includes depth features. To deeply mine the feature information of depth images, this invention performs feature extraction on the depth information to obtain the point cloud features of the image. and normal features Among them, the first two dimensions of point cloud features Represents the coordinate position of a pixel in a two-dimensional plane space. Corresponding to its depth value; normal features This describes the orientation information of the pixel in three-dimensional space. By concatenating the above geometric features with the original RGB color features, a nine-dimensional feature vector space is finally constructed. Therefore, in the proposed representation method, each pixel of the RGB-D image can be represented by this nine-dimensional feature vector.

[0076] Traditional clustering methods typically use Euclidean distance to calculate the similarity between sample points. However, for feature analysis of RGB-D images, a single metric is insufficient to characterize the essential differences between multidimensional features. Therefore, this invention proposes a composite distance metric to assess the differences between different feature spaces. Since normals exist as vectors, cosine distance can more effectively characterize their spatial distribution. During clustering, the similarity between samples and class centers is measured using composite distance. For RGB features and point cloud features, Euclidean distance is used to calculate their similarity, while for normal features, cosine distance is used to measure their similarity.

[0077] Composite distance d f The calculation method is as follows:

[0078] d f =d r '+d e (4)

[0079] in

[0080]

[0081] In the formula x im Let b represent the m-th dimension feature of the i-th sample. cm Let d represent the m-th feature of the c-th cluster center. r d represents the Euclidean distance from a sample point to the cluster center. e This represents the cosine distance from a sample point to the cluster center. To eliminate the dimensional differences between different distance metrics, a linear normalization method is used to normalize the distance values. Specifically, for each distance d... r and d e The normalized value d r 'and d e The calculation formula is:

[0082]

[0083] Where d rmax and d rmin Let d be the maximum and minimum values ​​of the Euclidean distance. emax and d eminThese are the maximum and minimum values ​​in the cosine distance.

[0084] (4) Clustering optimization;

[0085] Optimization of rough membership degree U;

[0086] The clustering criterion function in equation (3) has the following constraints:

[0087]

[0088] Based on the constraints, the Lagrange function can be obtained:

[0089]

[0090] Find the relationship between F and u. ci Taking the partial derivative and setting it to 0, we get:

[0091]

[0092] but

[0093]

[0094] Because it satisfies Given the constraints, we can obtain

[0095]

[0096] but

[0097]

[0098] As λ→0, if If the value is the minimum value within the range 1 ≤ l ≤ C, then the above expression approaches 1; otherwise, it is 0. Finally, we obtain u. ci The update rules are as follows:

[0099]

[0100] but

[0101]

[0102] Among them, R it Represents pixel x i With pixel x t The binary relation between them is defined by equation (17), R it =1 indicates pixel x t Located at pixel x i In the neighborhood of I×I, the converse is not satisfied.

[0103]

[0104] Optimization of cluster weight W;

[0105] Fixed cluster centers C and u ci For F, ask about W cm Taking the partial derivative and setting it to 0, we get:

[0106]

[0107] At the same time, we can obtain:

[0108]

[0109] By constraints We can obtain:

[0110]

[0111] but

[0112]

[0113] Will Substituting into formula (19), we get:

[0114]

[0115] The principle of weighting multimodal data in clustering is to assign larger weights to feature variables with smaller sums of intra-cluster distances and smaller weights to feature variables with larger sums of intra-cluster distances. This principle is called the "cluster weighting principle," which can prevent the generation of unbalanced clusters. When the parameter q = 0, formula (23) is equal to the unweighted coarse membership clustering algorithm. When 0 < q < 1, the larger the sum of intra-cluster distances, the larger the weight, which contradicts the "cluster weighting principle." Therefore, the parameter q should take values ​​in the range of q < 0 and q > 1.

[0116] Optimization of cluster center b;

[0117] Fixed feature weights W cm and u ci Find the relationship between F and b. cm Taking the partial derivative and setting it to 0, we get:

[0118]

[0119] achievable

[0120]

[0121] Table 1 shows the pseudocode of the proposed composite distance-weighted coarse membership clustering RGB-D image segmentation algorithm.

[0122] Table 1

[0123]

[0124] The calculation process of the present invention will be described in detail below with reference to Table 1 and the following embodiments, but the present invention is not limited to these embodiments.

[0125] First, feature extraction is performed on the depth image, extracting point cloud features and normal features. The point cloud represents the position information of image pixels in planar space, and the normal features represent the orientation of pixels in three-dimensional space. The point cloud and normal features are then combined with RGB color features to form a 9-dimensional combined feature set.

[0126] (1) Initialization phase;

[0127] Rows 1 to 3 of Table 1 describe the initialization phase. Row 3 introduces a density-based initialization method, which prioritizes high-density regions as cluster centers by analyzing the distribution of data points. First, similar samples are grouped into dense points, and their density is reflected by calculating the local density of the sample points. For RGB-D multimodal data, Euclidean distance is used for color and point cloud features, and cosine distance is used for normal features. Then, the sample densities are sorted, and a density threshold is set, selecting points with densities greater than the threshold as high-density points. Finally, C points are randomly selected from these high-density points as initial cluster centers.

[0128] (2) Iteration phase;

[0129] Rows 4 to 10 of Table 1 represent the iterative phase, which uses the criterion function. We calculate the partial derivatives with respect to each parameter to derive the analytical optimization formulas for each parameter. Specifically, d in line 5... f By combining Euclidean and cosine distances, the potential information of each feature dimension is fully explored, and modal features are effectively integrated, improving the accuracy and comprehensiveness of data analysis. Rows 6-7 show the coarse membership degree. The update process of coarse membership is as follows. The update of coarse membership is based on the neighborhood relationship between pixels and surrounding pixels, the distance between pixels and cluster centers, and the weight distribution of features in each dimension. Under the multi-factor fusion optimization framework, it can simultaneously capture the local spatial structure and global statistical characteristics of the data, thereby accurately reflecting the degree of membership of each pixel to the cluster center. Line 8 updates the weights of feature variables. In this method, features with smaller intra-cluster distances are assigned larger weights, while features with larger intra-cluster distances are assigned smaller weights. Based on this principle, it can ensure that more attention is paid to more closely related features, avoid serious imbalances between clusters during the clustering process, and improve the accuracy and rationality of the clustering results. Line 9 represents the update of cluster centers, which is based on the current data point classification and coarse membership matrix. For each cluster C, the new cluster center b cmThe data points in the coarse membership matrix are weighted and averaged, with the weights being the coarse membership degree between each data point and the cluster center. If some sample points in a cluster have a high membership degree to that cluster, it indicates that these sample points are spatially distant from the cluster center. During the cluster center update operation, the cluster center will be adjusted towards the location of these sample points.

[0130] (3) Regional merging strategy;

[0131] Row 13 of Table 1 illustrates the region merging method. During the region merging process, each labeled region is treated as a node. By checking the candidate eligibility, merging eligibility, and merging consistency of each node one by one, it is determined which nodes can be merged. After the merge operation, the information of the merged nodes and their edges is updated promptly to reflect the new merged structure. This iterative process continues until there are no more candidate nodes to merge in the system, thus ensuring the comprehensiveness and effectiveness of the region merging. In this way, more accurate region segmentation results can be obtained.

[0132] In RGB-D image segmentation based on weighted coarse membership clustering, the region merging step optimizes the clustering results through rigorous qualification screening and consistency verification. First, candidate regions are identified through neighborhood search; these regions are spatially adjacent and have feature similarity higher than a preset threshold. Then, the merging qualifications of these candidate region pairs are checked, including feature similarity, area ratio, and boundary consistency, to ensure that the merged regions maintain consistency in features and structure. Finally, merging consistency is verified to further evaluate whether the features and boundaries of the merged regions remain consistent. This process iterates until no more candidate regions can be merged, thereby generating a more accurate and robust segmented image.

[0133] To verify the beneficial effects of the present invention, the inventors conducted experiments on images in the NYUD2 and SUN RGB-D datasets using the weighted coarse membership degree clustering method based on composite distance as described in the embodiments of the present invention. The various experimental conditions are as follows.

[0134] 1. Experimental conditions;

[0135] The experimental test environment was a HP computer with Windows 11 (64) operating system, configured with an AMD Ryzen 77735H processor 3.20GHz and 16GB of memory, and the experiment was conducted on the MATLAB 2022a platform.

[0136] 2. Test image;

[0137] NYUD2 dataset, SUN RGB-D dataset.

[0138] 3. Experiment content;

[0139] (1) The method of this invention (hereinafter referred to as the method of this invention) was evaluated against Hard C-means Clustering Algorithm (HCM), Fast Robust Fuzzy C-means Clustering Algorithm (FRFCM), Multilayer Clustering Algorithm (MLC), Joint Color-Space-Oriented Clustering and Region Merging Algorithm (JCSD-RM), Chaotic Kbest Gravity Search Algorithm (cKGSA-FC), and Stochastic Henry's Law Gas Solubility Optimization Algorithm (RHGSO-FC) under the same number of clusters. The visual clustering effect is... Figures 5-20 Tables 2 and 4 show the evaluation metrics analysis results of randomly sampled images on the NYUD2 and SUN RGB-D datasets, respectively, while Table 3 summarizes the average performance of the proposed method on the NYUD2 dataset. Regarding experimental parameter settings, the following configurations were used for different datasets: In the NYUD2 dataset, the number of clusters was set to 13, the neighborhood size to 5×5, and the weight parameter q=2 to balance the distance metric in the feature space; in the SUN RGB-D dataset, the number of clusters was set to 7, using the same 5×5 neighborhood size and weight parameter q=2 to ensure consistent comparison of the algorithm in different scenarios. Furthermore, to ensure the stability and convergence efficiency of the algorithm, a uniform convergence threshold ε=0.0001 and a maximum iteration count iter were set. max =50. Evaluation metrics include Boundary Displacement Error (BDE), Probability Rand Index (PRI), Pixel Accuracy, and Normalized Mutual Information (NMI). For evaluation metrics, the lower the BDE value and the higher the PRI, ACC, and NMI values, the better the segmentation effect of the method.

[0140] Table 2

[0141]

[0142]

[0143] Table 3

[0144]

[0145] Experimental results show that this method exhibits good segmentation performance on the NYUD2 indoor scene dataset. Figure 5-12As shown, compared with existing methods, the method of this invention exhibits significant improvements in the following aspects: First, it demonstrates higher boundary accuracy in wall segmentation tasks, effectively reducing the missegmentation rate in shadow areas; second, it maintains good segmentation accuracy for small objects in the scene. These advantages fully demonstrate that this method has stronger robustness and higher segmentation accuracy when handling complex indoor scenes.

[0146] As shown in Tables 2 and 3, the segmentation performance of the method of this invention on NYUD2 is significantly better than that of existing methods. Table 3 data shows that the method of this invention achieves optimal values ​​in BDE, PRI, and ACC, indicating that it has higher segmentation accuracy and more accurate boundary recognition capabilities. Experimental results show that this method can effectively reduce the missegmentation rate and missed segmentation rate, exhibiting excellent stability and accuracy when processing complex scenes and images with rich details. Although the JCSD-RM method performs best in the NMI index, indicating that it has good performance in similarity evaluation, the difference between the NMI value of this method and the JCSD-RM method is small, and it performs better in BDE, PRI, and ACC, demonstrating superior overall performance and ultimately achieving the best segmentation effect.

[0147] Table 4

[0148]

[0149] Figures 13-20 This paper presents a visual comparison of the image segmentation results of different algorithms on the SUN RGB-D dataset. It can be seen that the method of this invention exhibits the best visual segmentation performance, accurately identifying stools and planar objects while maintaining clear boundary divisions and rich detail features. Simultaneously, this method significantly reduces segmentation noise, effectively suppressing missegmentation and redundant regions, further verifying its robustness and accuracy in complex scenes. Through precise boundary detection and region division, the method of this invention performs excellently in handling complex backgrounds and details, avoiding common artifacts and unnecessary region divisions, resulting in clearer and more accurate segmentation of each region in the final image. Table 4 shows the comparison results of all compared algorithms on the above images in terms of PRI, ACC, and NMI. The method of this invention achieves the best performance in all three metrics, fully verifying its superiority in the multi-dimensional evaluation system. Specifically, the method of this invention can effectively distinguish different objects, comprehensively and accurately extract key information from the image, and effectively improve the overall segmentation accuracy. Although it does not achieve the best BDE metric, based on the comprehensive evaluation of visual segmentation effect and overall metrics, it can be seen that the method of this invention performs better than other compared algorithms.

[0150] Figure 2This paper demonstrates the impact of feature weighting strategies on the clustering and segmentation quality of randomly sampled images from the NYUD2 dataset. This embodiment uses the NMI metric for quantitative evaluation. Figure 2 As can be seen, the NMI value of the weighted algorithm is generally higher than that of the unweighted algorithm. Specifically, for the NYUD2_367 image, the NMI value of the weighted algorithm is close to 0.7, a significant improvement compared to the unweighted algorithm's NMI value of 0.63; for the NYUD2_39 image, the NMI of the weighted algorithm is approximately 0.55, also higher than the unweighted algorithm's NMI value of 0.48. In the segmentation results of the remaining images, the NMI value of the weighted algorithm shows a certain improvement compared to the unweighted algorithm. Figure 3 This diagram illustrates the impact of the proposed composite distance metric on the image clustering and segmentation quality of the NYUD2 dataset. Experimental results show that in multiple test images (such as NYUD2_396 and NYUD2_420), the clustering performance using the composite distance metric is significantly better than that using traditional Euclidean distance. Specifically, for... Figure 3 The comparison results show that the highest NMI value using the composite distance metric method is 0.68, which is higher than the highest NMI value of 0.63 for Euclidean distance. For eight randomly selected images from the NYUD2 dataset, the NMI values ​​using the composite distance metric method are all higher than those using the traditional Euclidean distance method. Experimental data demonstrate that the composite distance metric method can more effectively measure the similarity between RGB-D multimodal data, thereby improving the clustering effect. In summary, the method of this invention combines the advantages of feature weighting and composite distance, effectively improving clustering performance.

[0151] Figure 4 The CPU time consumption of different algorithms in image segmentation tasks was compared. The cKGSA-FC and RHGSO-FC methods employ a SAS framework segmentation technique based on five-subgraph aggregation. This approach not only has a complex algorithm design but also requires adjustment of multiple parameters, leading to a significant increase in computational complexity. Experimental data show that compared to the cKGSA-FC and RHGSO-FC methods, the method of this invention has a significant advantage in computational efficiency. Compared to the JCSD-RM method, its computation time is slightly lower, thanks to its lower algorithmic complexity. Although the HCM, FRFCM, and MLC methods have shorter computation times, their segmentation performance is inferior. Therefore, the method of this invention can achieve good segmentation results while ensuring high computational efficiency.

[0152] This embodiment also provides a weighted coarse membership clustering RGB-D image segmentation system based on composite distance, including:

[0153] An initialization module is used to improve the traditional random initial class center selection method by selecting points in high-density regions as initial class centers.

[0154] The feature extraction module is used to extract image features from RGB-D images, including color features, point cloud features, and normal features.

[0155] The optimization module is used to optimize the clustering algorithm based on adaptive weighting and composite distance metric methods to obtain a weighted coarse membership clustering algorithm.

[0156] Among them, adaptive weighting includes adaptive weighting of each feature dimension through local feature weighting;

[0157] Among them, the composite distance metric method includes a composite distance metric method that combines Euclidean distance and cosine distance to calculate the similarity between samples and cluster centers;

[0158] The image segmentation module is used to cluster RGB-D images based on a weighted coarse membership clustering algorithm, and to merge the clustering results to generate segmented images.

[0159] This embodiment also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0160] This embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method.

[0161] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A weighted coarse membership clustering method for RGB-D image segmentation based on composite distance, comprising the following steps: Extract image features from an RGB-D image, including color features, point cloud features, and normal features; Density-based methods select class centers by using high-density regions as initial class centers. The clustering algorithm is optimized based on adaptive weighting and composite distance metric methods to obtain a weighted coarse membership clustering algorithm; Adaptive weighting involves adaptively weighting each feature dimension using a local feature weighting method; Composite distance metrics include methods that combine Euclidean distance and cosine distance to calculate the similarity between samples and cluster centers; The weighted coarse membership clustering algorithm is used to cluster RGB-D images, and the clustering results are merged to generate segmented images.

2. The method according to claim 1, characterized in that, When selecting the initial class center, the high-density points are selected as the initial class centers by calculating the neighborhood density of each point.

3. The method according to claim 1, characterized in that, In adaptive weighting, different weights are assigned to color features, point cloud features, and normal features according to the importance of each feature in clustering, and the weights are dynamically adjusted according to the intra-cluster distance during the iteration process.

4. The method according to claim 1, characterized in that, In the composite distance metric method, Euclidean distance is used to calculate the similarity of color features and point cloud features, cosine distance is used to calculate the similarity of normal features, and linear normalization is used to normalize the different distance metrics.

5. The method according to claim 1, characterized in that, The criterion function expression for the weighted coarse membership clustering algorithm is as follows: In the formula, n, M, and C represent the number of data samples, the spatial dimension of the features, and the number of cluster centers, respectively, and W cm W represents the feature weight of the m-th dimension in the c-th class, where q is the exponential parameter used to adjust the weights W. cm The intensity of the effect on the feature, For sample x i To cluster center b c Based on the rough membership degree of the lattice neighborhood binary relation R, d f Represents sample point x im To cluster center b cm The composite distance, x im Let b represent the m-th dimension feature of the i-th sample. cm Let m represent the m-th feature of the c-th cluster center.

6. The method according to claim 1, characterized in that, This method updates the coarse membership degree by using the Lagrangian function and constraints, taking into account both the local structure and global characteristics of the data.

7. The method according to claim 1, characterized in that, The region merging process includes treating each labeled region as a node, checking the candidate eligibility and merging eligibility of nodes, and verifying merging consistency, merging nodes that meet the conditions until there are no more candidate nodes to be merged.

8. A weighted coarse membership clustering RGB-D image segmentation system based on composite distance, characterized in that, include: An initialization module is used to improve the traditional random initial class center selection method by selecting points in high-density regions as initial class centers. The feature extraction module is used to extract image features from RGB-D images, including color features, point cloud features, and normal features. The optimization module is used to optimize the clustering algorithm based on adaptive weighting and composite distance metric methods to obtain a weighted coarse membership clustering algorithm. Among them, adaptive weighting includes adaptive weighting of each feature dimension through local feature weighting; Among them, the composite distance metric method includes calculating the similarity between samples and cluster centers by fusing Euclidean distance and cosine distance; The image segmentation module is used to cluster RGB-D images based on a weighted coarse membership clustering algorithm, and to merge the clustering results to generate segmented images.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-7.

Images