A 3D image data sampling method based on unsupervised clustering

Abstract

The application provides a 3D image data sampling method based on unsupervised clustering, comprising the following steps: S1, collecting 3D image data to be sampled, and performing certain pretreatment on the collected data to remove impurity information; S2, after data cleaning, manually extracting the shallow features of target objects in each slice; S3, performing unsupervised clustering according to the shallow feature set of each 2D image; and S4, finally selecting 1-3 2D images with the largest object outlines from each cluster. The application converges and classifies the feature set of 2D images of 3D image data through unsupervised clustering, which can perfectly utilize the correlation between image features, thereby providing strong support for subsequent data sampling, obtaining complete and obviously specific 2D data, and being applicable to actual business.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a 3D image data sampling method based on unsupervised clustering. Background Technology

[0002] 3D image data sampling refers to the process of filtering a 3D image dataset input into a deep learning model so that the selected 2D images contain most of the feature information of the 3D data. 3D image data sampling can be widely used in 3D analysis, security monitoring, image classification, regression prediction, and other fields.

[0003] Traditional 3D image sampling primarily relies on selecting the 2D image with the largest target contour as the representative input for 3D image data. For example, the paper "Joint multi-task learning for survival prediction of gastriccancer patients using CT images" asks a specialist to select the slice with the largest tumor target area from each patient. In contrast, the papers "Act-based deep learning radiomics nomogram for predicting the response to neoadjuvant chemotherapy in patients with locally advanced gastric cancer: A multicenter cohort study" and "A deep learning risk prediction model for overall survival in patients with gastric cancer: A multicenter study, Radiotherapy and Oncology" select three available slices and stack them into a three-channel image as the model input, with each two-dimensional image representing the patient's entire data space.

[0004] However, while the above methods can capture the main feature information of the entire 3D image data space, they also result in the omission of a large amount of feature information, leading to incomplete feature information input into the deep learning model and affecting the prediction performance of the final model. Therefore, a more effective 3D image data sampling method is needed to obtain more specific and complete 2D image data. Summary of the Invention

[0005] The purpose of this invention is to provide a sampling method for 3D image data, which aims to solve the problem of acquiring 2D images from 3D image data. This method combines two key steps: shallow feature extraction and unsupervised clustering, to improve the accuracy and completeness of the sampled data.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A 3D image data sampling method based on unsupervised clustering includes the following steps:

[0008] S1: Collect the 3D image data to be sampled, and preprocess the collected 3D image data to remove impurities.

[0009] S2: After data cleaning, the shallow features of the target object in each 2D image are manually extracted and divided into three categories: morphological features, statistical features, and model features.

[0010] S3: Based on the shallow feature set of each extracted 2D image, perform unsupervised clustering to assign each 2D image of the 3D image data to different clusters according to the correlation of its shallow features.

[0011] S4: Finally, select 1-3 2D images with the largest object outlines from each cluster.

[0012] In S1, the 3D image data is data obtained from the actual environment, and the preprocessing operation is a filtering and clearing operation of the 3D image data, including clearing 2D images that do not contain the outline of the target object and 3D image data with missing outlines of the target object.

[0013] In S2, the morphological features describe the shape, surface area, and other information of the lesion; the statistical features describe one-dimensional statistical features and high-dimensional statistical features; and the model-based features describe wavelet transform, Fourier transform, and fractal dimension, etc.

[0014] In S3, the silhouette coefficient method is used to iteratively calculate the average silhouette coefficient S for dividing different numbers of clusters:

[0015]

[0016]

[0017] Among them, y i Let s(y) be the feature of the i-th 2D image in the 3D image data, 1≤i≤n. i ) is the sample y i The profile coefficient, a(y i ) is the sample y iThe average distance to other samples in the same cluster, b(y) i ) is the sample y i The minimum average distance between samples and other clusters. Based on this, the number of clusters k is determined. Then, k cluster centers {C1, C2, C3, ..., C} are initialized. k}, 1 <k≤n。

[0018] Then, by calculating the Euclidean distance dis(y) from each sample to each cluster center... i C j ):

[0019]

[0020] In the formula, C j Let y represent the j-th cluster center, 1≤j≤k; it C represents the t-th feature of the i-th 2D image, 1≤t≤m; jt Let represent the t-th feature of the j-th cluster center. By sequentially comparing the distance from each sample to each cluster center, the target region slice is assigned to the cluster of the nearest cluster center, resulting in k clusters {H1, H2, H3, ..., H...}. k}

[0021] Finally, based on the different clusters, update the cluster center C of each cluster. t The calculation formula is as follows:

[0022]

[0023] Repeat the operation until the cluster centers no longer change or the change is very small (the distance between the newly calculated centroid and the original centroid is less than a certain set threshold), and obtain the final clustering result.

[0024] Compared with the prior art, the present invention has the following advantages and beneficial effects: The present invention aggregates and classifies the feature set of 2D images in 3D image data through unsupervised clustering. This method can perfectly utilize the correlation between image features, thereby providing strong support for subsequent data sampling, obtaining complete and highly specific 2D data that can be applied to actual business. Attached Figure Description

[0025] Figure 1 A flowchart of a 3D image data sampling method based on unsupervised clustering provided in this embodiment of the invention.

[0026] Figures 2(a) to 2(d) The four image data samples provided for the implementation of this invention require cleaning. As can be seen from the figure, the target object outlines in the 3D image data are of different sizes and their positions are not fixed.

[0027] Figure 3 A visual flowchart of the 3D image data sampling method provided for the implementation of this invention. Detailed Implementation

[0028] The implementation process of the present invention will now be described in detail with reference to the accompanying drawings and technical solutions.

[0029] like Figure 1 As shown, this invention provides a 3D image data sampling method based on unsupervised clustering, which specifically includes:

[0030] Step 1: Collect the 3D image data to be sampled, and perform certain preprocessing on the collected data to remove impurities, such as... Figures 2(a) to 2(d) As shown, the target object outlines in the 3D image data vary in size and position, so it is necessary to remove 2D images that do not contain the target object outlines.

[0031] The 3D image data is data obtained from the actual environment. The preprocessing operation is a filtering and clearing operation of the 3D image data, including clearing 2D images that do not contain the outline of the target object and some 3D image data with missing outlines of the target object.

[0032] The beneficial effects of adopting the above-mentioned further scheme are that, on the one hand, removing unclear data can reduce interference with data sampling, and on the other hand, it can make the obtained feature set achieve better clustering results.

[0033] Step 2: After data cleaning, the shallow features of the target object within each slice are manually extracted, such as... Figure 3 As shown in the feature extraction stage, shallow features are divided into three categories: morphological features, statistical features, and model features. Morphological features describe information such as the shape and surface area of ​​lesions, including mass area, boundary roughness, area ratio, and roundness. Statistical features describe one-dimensional statistical features (such as histogram features) and high-dimensional statistical features (such as texture features). Model-based features describe wavelet transform, Fourier transform, and fractal dimension.

[0034] The specific feature extraction methods are as follows:

[0035] i. Morphological characteristics

[0036] The gray-level co-occurrence matrix (GLCM) is used to describe image morphological features. It not only contains gray-level information but also reflects spatial information such as gray-level variations with respect to direction, adjacent spacing, and amplitude. The elements in the GLCM represent the frequency of occurrence of a pair of pixels with a specific distance and direction. Typical directions are 0°, 45°, 90°, and 135°, with a distance d = 1 (for θ = 0°, 90°). (For θ = 45°, 135°). For the elements h of a K×K gray-level co-occurrence matrix H(d,θ) i,j Let d represent all pixels at a distance of d, K be the pixel size of the co-occurrence matrix, and i,j represent the total number of pixel pairs in the i-th row and j-th column of the image at a horizontal angle of θ.

[0037] Commonly used features for describing textures are extracted based on the gray-level co-occurrence matrix, mainly including energy, inertia, inverse difference moment, spur, correlation, mean of sum, mean of difference, entropy of sum, and entropy of difference.

[0038] Energy: Reflects the uniformity of an image. Higher energy indicates a more uniform image, and vice versa. The energy feature is sensitive to the presence or absence of grayscale variations within the image, but not to the magnitude of those variations. The expression for energy is:

[0039] Inertia: Describes the uniformity of an image. The smaller the inertia, the more uniform the image, and vice versa. The expression for inertia is:

[0040] Inverse contrast moment: assigning low weight to high contrast, defined as follows:

[0041] Entropy: Reflects the coarseness of an image Degree, which can be used to distinguish the texture roughness of an image, is defined as follows:

[0042] Correlation: Represents image homogeneity, measuring the similarity of gray levels in the row or column direction. It reflects the similarity of local gray levels; the larger the value, the more similar the images, and vice versa. The definition is as follows: Where μ x ,μ y ,σ x ,σ y This represents the mean and standard deviation of the marginal distribution.

[0043] The mean of the sum:

[0044] Entropy of sum:

[0045] ii. Statistical characteristics

[0046] The gray-level histogram of an image is a function of gray levels, and is expressed by the following formula.

[0047]

[0048] Where i represents the gray level, L represents the number of gray levels, and n iLet H(i) represent the number of pixels with gray level i, and N represent the total number of pixels. Then H(i) represents the frequency of occurrence of pixels with gray level i. Histogram features can reflect the amplitude and frequency characteristics of pixel values ​​within the region of interest, providing a global description of the image, but lacking spatial features of lesions.

[0049] Histogram features require a series of statistics to reflect them, which may include mean, variance, third moment, fourth moment, entropy, consistency, standard deviation, etc.

[0050] Mean: Reflects the average gray value of the image.

[0051] Variance: Reflects the roughness of an image.

[0052] Third moment: Reflects the asymmetry of the gray-level histogram. The greater the slope, the more asymmetrical the gray-level histogram, and vice versa.

[0053] Fourth-order moment: also known as kurtosis, reflects the relative flatness of the gray-level histogram.

[0054] Entropy: Reflects the uniformity of gray-level distribution in an image.

[0055] Consistency: Used to describe the smoothness of an image. The stronger the consistency, the smoother the image; conversely, the weaker the consistency, the coarser the image.

[0056] Standard deviation: reflects the roughness of an image.

[0057] iii Model Features

[0058] Because the Gabor transform closely resembles the visual field perception of simple cells in the human visual cortex and possesses multi-directional selectivity, Gabor wavelets were employed as the model extraction method. Two-dimensional Gabor filters exhibit not only good frequency selectivity but also good direction selectivity, enabling multi-scale image analysis. This study extracted different texture directions and frequency scale information of the image's Region of Interest (ROI). The two-dimensional Gabor filter function is expressed as:

[0059]

[0060]

[0061] Where, k j σ represents the center frequency of the filter, σ represents the scale factor in scale space, x represents the image coordinates of the positioning tool, and φ represents the image coordinates of the positioning tool. u This reflects the directional selectivity of the filter. In natural images, Used to compensate for energy spectrum decay, Used to constrain the Gaussian envelope function of a plane wave. This represents a complex-valued plane wave, whose real part is a cosine plane wave. The imaginary part is a sinusoidal plane wave.

[0062] Ultimately, we can obtain the feature set Y = [y1, y2, ..., y] of all 2D images in the 3D image data. n ], y n Let be the set of features extracted from the target region of the nth 2D image, where each feature set has m-dimensional features.

[0063] Step 3: Then according to Figure 3 As shown in the clustering analysis stage, unsupervised clustering is performed on the shallow feature set of each extracted 2D image, thereby assigning each 2D image of the 3D image data to different clusters based on the correlation of its features.

[0064] The average silhouette coefficient S for dividing different numbers of clusters is calculated iteratively using the silhouette coefficient method:

[0065]

[0066]

[0067] Among them, y i Let s(y) be the feature of the i-th 2D image in the 3D image data, 1≤i≤n. i ) is the sample y i The profile coefficient, a(y i ) is the sample y i The average distance to other samples in the same cluster, b(y) i ) is the sample y i The minimum average distance between samples and other clusters. Based on this, the number of clusters k is determined. Then, k cluster centers {C1, C2, C3, ..., C} are initialized. k}, 1 <k≤n。

[0068] Then, the Euclidean distance from each sample to each cluster center is calculated.

[0069]

[0070] In the above formula, C j Let y represent the j-th cluster center, 1≤j≤k. it Let C represent the t-th feature of the i-th 2D image, 1≤t≤m. jtLet represent the t-th feature of the j-th cluster center. By sequentially comparing the distance from each sample to each cluster center, the target region slice is assigned to the cluster of the nearest cluster center, resulting in k clusters {H1, H2, H3, ..., H...}. k}

[0071] Finally, based on the different clusters, the cluster centers of each cluster are updated, and the calculation formula is as follows:

[0072]

[0073] Repeat the above steps until the cluster centers no longer change or change very little (the distance between the newly calculated centroid and the original centroid is less than a certain set threshold), and obtain the final clustering result.

[0074] Step 4: Finally, select 1-3 of the largest 2D images from each cluster. This ensures that the images contain more feature information while reducing excessive redundant information.

Claims

1. A 3D image data sampling method based on unsupervised clustering, characterized in that, The steps include the following: S1: Collect the 3D image data to be sampled, and preprocess the collected 3D image data to remove impurities. S2: After data cleaning, the shallow features of the target object in each 2D image are manually extracted and divided into three categories: morphological features, statistical features, and model features. S3: Based on the shallow feature set of each extracted 2D image, perform unsupervised clustering to divide each 2D image of the 3D image data into different clusters according to the correlation of its shallow features; S4: Finally, select 1-3 2D images with the largest object outlines from each cluster; In S1, the 3D image data is data obtained from the actual environment, and the preprocessing operation is a filtering and clearing operation of the 3D image data, including clearing 2D images that do not contain the outline of the target object and 3D image data with missing outline of the target object. In S2, the morphological features describe the shape and surface area information of the lesion; the statistical features describe one-dimensional statistical features and high-dimensional statistical features; and the model-based features describe wavelet transform, Fourier transform, and fractal dimension. In S3, the silhouette coefficient method is used to iteratively calculate the average silhouette coefficient S for dividing different numbers of clusters: in, The features of the i-th 2D image in the 3D image data. , For the sample The profile coefficient, For the sample The average distance to other samples in the same cluster. For the sample The minimum average distance between samples and other clusters is used to determine the number of clusters k. Then, k cluster centers are initialized. , ; Then, the Euclidean distance from each sample to each cluster center is calculated. : In the formula, Let j represent the j-th cluster center. ; This represents the t-th feature of the i-th 2D image. ; The j-th cluster center represents the t-th feature; the distance from each sample to each cluster center is compared sequentially, and the target region slice is assigned to the cluster with the nearest cluster center, resulting in k clusters. ; Finally, based on the different clusters, update the cluster centers of each cluster. The calculation formula is as follows: Repeat the operation until the cluster centers no longer change or change very little, that is, the distance between the newly calculated centroid and the original centroid is less than a certain set threshold, and the final clustering result is obtained.

Images