Heterogeneous feature clustering method based on privacy image

Abstract

The invention relates to a heterogeneous feature clustering method based on a privacy image, which is characterized in that a diffusion generation process is converted from a pixel domain to a feature domain through a characteristic noise and moment matching technology, and a to-be-trained diffusion generator model taking a conditional denoising UNet architecture as a core is constructed; training by combining the sample-level consistency constraint and the distribution-level alignment constraint, and constructing feature pairs with mutually aligned semantics by using a pseudo image generated by the trained diffusion generator model; then training a variational auto-encoder model, realizing cross-domain feature mapping through double-level alignment loss, and finally forming a unified domain feature data set and performing clustering; the method has the advantages that cross-domain feature unification is completed on the premise of passive domain data and no paired samples, original image leakage is effectively avoided, a high-quality unified feature set can be directly output, and a safe and efficient solution is provided for cross-domain visual analysis in a privacy limited scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing and data clustering, and particularly relates to a heterogeneous feature clustering method based on privacy images, which is suitable for cross-domain feature clustering analysis under the condition that original data is unknown. BACKGROUND

[0002] In sensitive application scenarios such as medical image analysis, financial risk control and cross-platform user portrait, data privacy and security are increasingly becoming the primary consideration. This leads to a general demand: under the strict privacy protection premise that the original data (such as image pixels) itself is invisible and cannot be concentrated, joint analysis and clustering of heterogeneous features from different sources (domains) are needed. These heterogeneous features are usually generated by different pre-trained feature extraction networks, such as EfficientNet network, ResNet network, etc., and their feature space dimensions and statistical distributions are significantly different.

[0003] Existing technologies in handling privacy image cross-domain analysis usually rely on direct access to original image data or require features to have highly consistent distribution assumptions. For example, traditional feature alignment and transfer learning methods often need to perform data augmentation, style conversion or adversarial training at the pixel level, which not only increases the computational complexity, but also brings the risk of privacy leakage. In addition, most existing methods rely on paired cross-domain samples or shared latent space assumptions, which are difficult to implement effectively in actual scenarios where original data is lacking and paired samples cannot be obtained. In other words, under the strict restrictions that original images are invisible, features are heterogeneous and there is no unsupervised alignment signal, how to achieve high-quality semantic alignment and unified representation of cross-domain features is still a key problem that existing technologies have not properly solved. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a heterogeneous feature clustering method based on privacy images, so as to realize high-precision alignment and joint clustering of cross-domain heterogeneous features under the constraint that original images are invisible.

[0005] The technical solution adopted by the present application to solve the above technical problem is: a heterogeneous feature clustering method based on privacy images, comprising the following steps: Step 1-1, obtain two privacy image sample sets respectively belonging to two different domains, and clip the sample size in the two privacy image sample sets to X×Y respectively to obtain a first sample set belonging to a first target domain and a second sample set belonging to a second target domain, perform feature extraction on the image samples in the first sample set through a first feature extraction network to obtain first target domain features and form a first target domain feature dataset, perform feature extraction on the image samples in the second sample set through a second feature extraction network to obtain second target domain features and form a second target domain feature dataset, and the first feature extraction network and the second feature extraction network adopt a parallel and independent architecture; Step 1-2, obtain the mean and covariance of the first target domain feature dataset, and re-parameterize and color a random Gaussian noise with the same dimension as the first target domain feature to obtain a first feature noise; obtain the mean and covariance of the second target domain feature dataset, and re-parameterize and color a random Gaussian noise with the same dimension as the second target domain feature to obtain a second feature noise; Step 2-1, construct a trained diffusion generator model with a conditional denoising UNet architecture as the core, set the diffusion time step of the trained diffusion generator model and the noise intensity of each time step, combine the first feature noise with each first target domain feature at each time step according to the noise intensity of the time step to form a first noisy target feature of each first target domain feature at each time step, and combine the second feature noise with each second target domain feature at each time step according to the noise intensity of the time step to form a second noisy target feature of each second target domain feature at each time step; Step 2-2, input the first noisy target feature and the second noisy target feature as conditions to guide the trained diffusion generator model at each time step, and the trained diffusion generator model performs denoising on a random Gaussian noise with a size of X×Y to obtain a denoising image at each time step, and performs feature extraction on the denoising image through the first feature extraction network to obtain a first predicted feature, and performs feature extraction on the denoising image through the second feature extraction network to obtain a second predicted feature at each time step; Step 2-3, set the total loss function of the trained diffusion generator model by using the same-domain sample level consistency constraint and the distribution level alignment constraint, so that the first predicted feature is close to the first noisy target feature corresponding to the same time step, and the second predicted feature is close to the second noisy target feature corresponding to the same time step, and finally train to obtain a trained diffusion generator model; input the random Gaussian noise with a size of X×Y into the trained diffusion generator model as a condition to guide the trained diffusion generator model, and output a denoising image, and define the denoising image output at this time as a pseudo image. Step 3-1, inputting the pseudo image into a first feature extraction network to obtain a first synthetic image domain feature, inputting the pseudo image into a second feature extraction network to obtain a second synthetic image domain feature, and the first synthetic image domain feature and the second synthetic image domain feature constituting a feature pair with mutual semantic alignment; Step 3-2, constructing a variational autoencoder model to be trained, taking the first synthetic image domain feature as an encoding input, taking the second synthetic image domain feature of the same feature pair as a decoding target, adopting a KL regular loss and combining a sample-level consistency constraint and a distribution-level alignment constraint to construct a total loss function of the variational autoencoder model to be trained, training the variational autoencoder model to be trained, and obtaining a trained variational autoencoder model; Step 3-3, mapping all first target domain features to the same feature space as the second target domain feature dataset through the trained variational autoencoder model, and then forming a unified space feature dataset together with the second target domain feature dataset; Step 4, clustering all data in the unified domain feature dataset to obtain a clustering center and a clustering label, and completing the clustering process.

[0006] Compared with the prior art, the advantages of the present application are that the diffusion generator model is converted from the pixel domain to the feature domain through the featureized noise and moment matching technology, and is trained in combination with the sample-level consistency constraint and the distribution-level alignment constraint; the generated pseudo image is used to construct a semantic alignment feature pair, an improved variational autoencoder is trained, cross-domain feature mapping is realized by replacing the target domain prior and the double-level alignment loss, and finally a unified domain feature dataset is formed and clustering is performed.

[0007] Under the premise of no source domain data and no paired samples, the present application realizes feature unification of cross-domain private images, effectively avoiding the risk of leakage of original images; by using the featureized noise and moment matching technology, the generation process of the diffusion generator model is converted from the pixel domain to the feature domain, which significantly improves the training efficiency; the training strategy combining the cross-domain sample-level consistency constraint and the distribution-level alignment constraint ensures that the generated pseudo image can not only retain key semantic information but also conform to the statistical characteristics of the target domain, thereby improving the performance of the downstream cross-domain recognition task; the variational autoencoder model enhances the robustness to domain shift and noise by introducing the KL regular loss and combining the sample-level consistency constraint and the distribution-level alignment constraint, and simplifies the complex double-space mapping to unified feature space learning, effectively improving the clustering quality. The entire process does not require any original image or additional annotation, and can directly output a unified space feature dataset with semantic labels, providing an efficient and secure solution for cross-domain visual analysis in privacy-restricted scenarios.

[0008] Preferably, in the step 1-2, the first characterized noise is denoted as v A , , μ A denotes the mean of the first target domain feature dataset, ∑ A denotes the covariance of the first target domain feature dataset, denotes a random Gaussian noise with the same dimension as the first target domain feature; the second characterized noise is denoted as v B , , μ B denotes the mean of the second target domain feature dataset, ∑ B denotes the covariance of the second target domain feature dataset, denotes a random Gaussian noise with the same dimension as the second target domain feature. By using rigorous mathematical modeling and statistical transformation, the standard random Gaussian noise is re-parameterized and colored according to the statistical characteristics such as the mean and covariance of the first target domain feature dataset and the second target domain feature dataset, to obtain the first characterized noise and the second characterized noise which are statistically close to the distributions of the two domains respectively.

[0009] Preferably, in the step 2-1, the first target domain feature in the first target domain feature dataset is denoted as a i the first noisy target feature at the time step t is denoted as , wherein, , N A denotes the total number of the first target domain features in the first target domain feature dataset, r t is a noise intensity scheduling function which monotonically decreases with the time step, 0≤ r t ≤1, 1≤ t ≤ T , T denotes the total number of the time steps set in the diffusion time steps; the second target domain feature in the second target domain feature dataset is denoted as j b j the second noisy target feature at the time step t is denoted as , , 1≤ j ≤ N B ,​​N B denotes the total number of the second target domain features.

[0010] By further introducing a noise intensity scheduling function monotonically decreasing with time steps, the first and second noisy target features are constructed in the feature space, which evolve from high noise to low noise continuously. The design theoretically guarantees the stability and controllability of the generation process, and provides a clear structure and explicit statistical properties of the supervision signal for subsequent diffusion generator model training, which is conducive to the alignment and unification of cross-domain features.

[0011] Preferably, the specific process of training the diffusion generator model in step 2-3 is as follows: Step 2-3-1, setting the total loss function of the diffusion generator model to be trained , wherein, ω t denotes the loss term for different time steps, weighted summation of the time step weight coefficient, , denotes the intra-domain sample level consistency constraint, denotes the mean square error between the pseudo image at time step and the first predicted feature and the first noisy target feature , , denotes the square of the L2 norm of the vector; denotes the mean square error between the pseudo image at time step t and the second predicted feature and the second noisy target feature , denotes the distribution level alignment loss function calculated by the multi-bandwidth RBF kernel in the reproducing kernel Hilbert space, , denotes the set of first predicted features in the same batch at time step t , denotes the set of second predicted features in the same batch at time step t , MMD 2 denotes the maximum mean difference operation; is a weight hyperparameter representing the importance of in , ​​Step 2-3-2, set the maximum number of training times of the diffusion generator model to be trained to 500 times, and perform back propagation and joint optimization on the parameters of the diffusion generator model to be trained according to the total loss function of the diffusion generator model to be trained through the Adam optimizer, and stop training when the maximum number of training times is reached, to obtain the trained diffusion generator model.

[0012] By constructing a composite loss function that combines the same domain sample level consistency constraint and distribution level alignment constraint, the collaborative optimization of the diffusion generator model at multiple time steps is realized. This method simultaneously constrains the sample-by-sample matching accuracy between the generated predicted features and the noisy target features in the same domain at each time step, as well as the overall distribution alignment of cross-domain features in the reproducing kernel Hilbert space, thereby effectively promoting the unification of the dual-domain feature distribution while preserving individual semantic information. Through the joint training mechanism of time step weighting and Adam optimizer, the stability and efficiency of model convergence are ensured, providing reliable training guarantee for generating semantically consistent and distribution controllable cross-domain pseudo images.

[0013] Preferably, in step 3-2, the total loss function of the variational autoencoder model to be trained is , , represents the KL regular loss function, represents the average value of the KL divergence between the posterior distribution obtained by the encoder in the variational autoencoder model to be trained for each first synthetic image domain feature in the current batch and the prior distribution of the second synthetic image domain, and the KL regular loss makes the latent space converge to the target domain distribution during training, reducing cross-domain mismatch; , α , β , are preset weight hyperparameters, α = 0.8, β = 0.2, , represents the sample level consistency loss function, , wherein, is the i-th second synthetic image domain feature in the dataset composed of second synthetic image domain features, m 1≤ m ≤ S , S is the total number of semantically aligned feature pairs, is the output of the variational autoencoder model to be trained corresponding to the input first synthetic image domain feature , is the length of the vector calculation symbol; Distribution-level alignment loss function using multi-bandwidth RBF kernel to calculate maximum mean discrepancy in reproducing kernel Hilbert space, , MMD 2 represents the maximum mean discrepancy operation, B1 represents the set of second synthetic image domain features, and B2 represents the feature dataset output by the decoder in the variational autoencoder model to be trained. By fusing the sample-level consistency constraint and the distribution-level maximum mean discrepancy alignment, a double-level alignment mechanism is constructed, which not only ensures the individual semantic alignment of the generated features and the target features, but also realizes the overall distribution space matching.

[0014] Preferably, the specific process of step 4 is as follows: Step 4-1, using the k-means clustering method with k-means++ initialization strategy to construct a clustering model, inputting the samples in the unified spatial feature dataset into the clustering model as input samples of the clustering model; Step 4-2, within the preset range of candidate cluster numbers, running the k-means clustering with k-means++ initialization strategy once for each candidate cluster number, calculating the corresponding average silhouette coefficient, and selecting the optimal clustering cluster number K according to the maximum value of the average silhouette coefficient; Step 4-3, initializing K clustering centers on the unified spatial feature dataset using the k-means++ initialization strategy, taking the sum of squared Euclidean distances from input samples to their corresponding clustering centers as the objective function, and iteratively updating the clustering center and sample cluster assignment until the objective function converges to obtain the final clustering model; the sample cluster labels and corresponding clustering centers output by the final clustering model are taken as the clustering results of the samples in the input unified spatial feature dataset, and the clustering process is completed.

[0015] Compared with other clustering methods (such as hierarchical clustering, DBSCAN, Gaussian mixture model, etc.), the k-means clustering method has the advantages of high computational efficiency, simple implementation, strong adaptability to large-scale data sets, etc., and is especially suitable for the clustering task of the unified spatial feature dataset in this scenario. Although existing clustering methods such as hierarchical clustering, spectral clustering or density-based clustering method can also realize clustering, k-means with k-means++ initialization strategy can effectively avoid the problem of sensitive initial center selection, improve the clustering stability and effect, and thus is more suitable for the demand of efficient and automatic division of feature distribution in this method. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a step flowchart of the present application; Figure 2 is a method principle diagram of the present application; Figure 3For the embodiment, the unified spatial feature data set processed by the method of the embodiment is input into the t-SNE dimension reduction module, and the high-dimensional features are mapped to a two-dimensional plane to visualize the result graph. DETAILED DESCRIPTION

[0017] A heterogeneous feature clustering method based on privacy images, as shown in Figure 1 , Figure 2 includes the following steps: Step 1-1, obtaining two privacy image sample sets respectively belonging to two different domains, and cutting the sample size in the two privacy image sample sets to XxY, respectively obtaining a first sample set belonging to a first target domain and a second sample set belonging to a second target domain, performing feature extraction on the image samples in the first sample set through a first feature extraction network to obtain first target domain features and form a first target domain feature data set, performing feature extraction on the image samples in the second sample set through a second feature extraction network to obtain second target domain features and form a second target domain feature data set, the first feature extraction network and the second feature extraction network adopt a parallel and independent architecture, that is, the two networks are independent of each other, do not form a cascade relationship, and do not share normalization or convolution layer weights, and in this way, the heterogeneity of the extracted features is ensured through the difference between the two structures.

[0018] In the embodiment, the first feature extraction network includes an EfficientNet-B4 feature backbone pre-trained on an ImageNet data set, an average pooling layer, and a first flattening layer. The features of the input image samples in the first sample set are cut and pooled, and then flattened to form a vector representation. As a preferred solution, the first feature extraction network freezes the parameters in the present heterogeneous feature clustering method and only performs feature extraction without participating in gradient update.

[0019] The second feature extraction network includes a ResNet-18 backbone, an adaptive average pooling layer, and a second flattening layer. It uses a ResNet-18 model pre-trained on an ImageNet data set as the backbone, removes the last two layers to the classification head, and only retains the convolution feature layer, then connects the adaptive average pooling and flattening to obtain a vector representation; as a preferred solution, the second feature extraction network can fine-tune or freeze its convolution backbone parameters as needed.

[0020] Step 1-2, obtaining the mean and covariance of the first target domain feature data set, and reparameterizing and coloring a random Gaussian noise with the same dimension as the first target domain feature to obtain a first feature noise; obtaining the mean and covariance of the second target domain feature data set, and reparameterizing and coloring a random Gaussian noise with the same dimension as the second target domain feature to obtain a second feature noise; specifically, the first feature noise is denoted as v A , μ A denotes the mean of the first target domain, ∑ A denotes the covariance of the first target domain, denotes a random Gaussian noise with the same dimension as the first target domain feature; let the second featureized noise be denoted as v B μ B denotes the mean of the second target domain, ∑ B denotes the covariance of the second target domain, denotes a random Gaussian noise with the same dimension as the second target domain feature.

[0021] Step 2-1, construct a trained diffusion generator model with a conditional denoising UNet architecture as the core, which is an existing structure, set the diffusion time step of the trained diffusion generator model and the noise intensity of each time step, combine the first featureized noise with each first target domain feature at each time step according to the noise intensity of the time step to form the first noisy target feature of each first target domain feature at each time step; combine the second featureized noise with each second target domain feature at each time step according to the noise intensity of the time step to form the second noisy target feature of each second target domain feature at each time step.

[0022] In step 2-1, the first target domain feature data set is denoted as a i The first noisy target feature at time step t is denoted as , wherein, , N A denotes the total number of first target domain features in the first target domain feature data set, r t is a noise intensity scheduling function that monotonically decreases with time step, 0≤ r t ≤1, 1≤ t ≤ T , T denotes the total number of time steps set in the diffusion time step; The second target domain feature data set is denoted as j b j The second noisy target feature at time step t is denoted as , , 1≤​​​​​j ≤ N B , N B denotes the total number of the second target domain features.

[0023] Step 2-2, the first noisy target feature and the second noisy target feature are guided as conditions according to time steps to input a trained diffusion generator model, the trained diffusion generator model denoises a random Gaussian noise of a rectangular shape with a size of XxY to obtain a denoised image in each time step, and extracts features of the denoised image through a first feature extraction network to obtain a first predicted feature, and extracts features of the denoised image through a second feature extraction network to obtain a second predicted feature in each time step. Thus, at different time steps, the features extracted by the first feature extraction network and the second feature extraction network from the denoised image gradually approach the respective target states with the noise intensity, thereby realizing cross-domain structure alignment in a unified semantic scale and providing a stable alignment benchmark for subsequent mapping and clustering.

[0024] Step 2-3, the total loss function of the trained diffusion generator model is set by using the same domain sample level consistency constraint and the distribution level alignment constraint, so that the first predicted feature approaches the first noisy target feature corresponding to the same time step, and the second predicted feature approaches the second noisy target feature corresponding to the same time step, and finally a trained diffusion generator model is trained.

[0025] The first noisy target feature and the second noisy target feature are guided as conditions for the trained diffusion generator model, a random Gaussian noise of a rectangular shape with a size of XxY is input into the trained diffusion generator model and a denoised image is output, and the denoised image output at this time is defined as a pseudo image.

[0026] The specific process of step 2-3 is as follows: Step 2-3-1, setting the total loss function of the trained diffusion generator model , wherein, ω t denotes the time step weight coefficient for the weighted sum of the loss terms of different time steps, , , denotes the same domain sample level consistency constraint, denotes the mean square error between the pseudo image at time step and the first predicted feature and the first noisy target feature , , denotes the square of the L2 norm of the vector; Indicates the pseudo-image at time step t In the second predicted feature Features of the second noisy target The mean square error between them ; This represents the distribution-level alignment loss function used to compute the maximum mean difference in the reproducing kernel Hilbert space using a multi-bandwidth RBF kernel. , Indicates the time step as t The set of the first predicted features in the same batch at that time. Indicates the time step as t The set of second predicted features in the same batch at that time. MMD 2 This represents the operation of finding the maximum mean difference. It means exist The weight hyperparameters representing the importance of the elements. .

[0027] Step 2-3-2: Set the maximum number of training iterations for the diffusion generator model to be trained to 500. Based on the total loss function of the diffusion generator model to be trained, use the Adam optimizer to perform backpropagation and joint optimization on the parameters of the diffusion generator model to be trained. Stop training when the maximum number of training iterations is reached to obtain the trained diffusion generator model.

[0028] During the above training process, the time step weight coefficients were adjusted. With weight hyperparameters A segmented annealing scheduling method is employed. Specifically, in the early stages of training, a higher weight is assigned to the consistency constraint at the same domain level to ensure that the diffusion generator model to be trained can learn basic noise modeling capabilities. As training progresses, the weight of the distribution-level alignment loss is gradually increased to enhance the cross-domain alignment effect. To improve the stability and generalization ability of training, a gradient pruning strategy is also adopted to prevent gradient explosion, as well as a parameter exponential moving average strategy, which obtains smoother and more stable model parameters by weighted averaging of the historical values ​​of the parameters.

[0029] A diffusion generator with conditional denoising UNet as its backbone is constructed. The inverse denoising process is supervised by feature consistency loss and distribution-level alignment loss, rather than relying on traditional noise prediction loss, thereby achieving alignment of heterogeneous features when they are not visible in the original image.

[0030] Step 3-1, inputting the pseudo image into the first feature extraction network to obtain a first synthetic image domain feature, inputting the pseudo image into the second feature extraction network to obtain a second synthetic image domain feature, and the first synthetic image domain feature and the second synthetic image domain feature constituting a feature pair with mutual semantic alignment.

[0031] Step 3-2, constructing a variational autoencoder model to be trained, taking the first synthetic image domain feature as an encoding input, taking the second synthetic image domain feature of the same feature pair as a decoding target, adopting a KL regularization loss, and jointly constructing a total loss function of the variational autoencoder model to be trained with a sample-level consistency constraint and a distribution-level alignment constraint, training the variational autoencoder model to be trained, and obtaining a trained variational autoencoder model; wherein the total loss function of the variational autoencoder model to be trained is , , represents a KL regularization loss function, represents an average value of the KL divergence between the posterior distribution obtained by each first synthetic image domain feature in the current batch through the encoder in the variational autoencoder model to be trained and the prior distribution of the second synthetic image domain, and the KL regularization loss function makes the latent space converge to the target domain distribution during the training process, reducing the cross-domain mismatch.

[0032] , α , β , are preset weight hyperparameters, α = 0.8, β = 0.2, , represents a sample-level consistency loss function, , wherein is the i-th second synthetic image domain feature in the data set composed of the second synthetic image domain features, m 1≤ m ≤ S , S is the total number of the feature pairs with mutual semantic alignment, is the output of the variational autoencoder model to be trained when the first synthetic image domain feature is input, is the length of the vector.

[0033] represents a distribution-level alignment loss function calculated by using a multi-bandwidth RBF kernel in a reproducing kernel Hilbert space, , MMD 2 represents a maximum mean difference operation, B1 represents a set of second synthetic image domain features, and B2 represents a feature data set output by the decoder in the variational autoencoder model to be trained.

[0034] Step 3-3, mapping all the first target domain features into the same feature space as the second target domain feature dataset by the trained variational autoencoder model, and then forming a unified space feature dataset together with the second target domain feature dataset.

[0035] Step 4, clustering all the data in the unified domain feature dataset to obtain clustering centers and clustering labels, and completing the clustering process, the specific process is as follows: Step 4-1, using the k-means clustering method with k-means++ initialization strategy to construct a clustering model, and inputting the samples in the unified space feature dataset into the clustering model as input samples of the clustering model.

[0036] Step 4-2, within the preset range of candidate cluster numbers, running the k-means clustering with k-means++ initialization strategy once for each candidate cluster number, calculating the corresponding average silhouette coefficient, and selecting the optimal clustering cluster number K according to the maximum value of the average silhouette coefficient.

[0037] Step 4-3, initializing K clustering centers on the unified space feature dataset using the k-means++ initialization strategy, taking the sum of squared Euclidean distances from input samples to their corresponding clustering centers as the objective function, and iteratively updating the clustering centers and sample cluster assignments until the objective function converges to obtain the final clustering model; outputting the sample cluster labels and corresponding clustering centers of the final clustering model as the clustering results of the samples in the input unified space feature dataset, and completing the clustering process.

[0038] In one specific example of the embodiment, to verify the effectiveness of the heterogeneous feature clustering method based on privacy images in the unified feature space alignment and clustering, qualitative and quantitative evaluations are respectively performed, where the heterogeneous feature clustering method based on privacy images is referred to as the method of the embodiment.

[0039] First, without the participation of real labels, the unified space features obtained by the embodiment are analyzed visually. Specifically, the unified space feature dataset processed by the method of the embodiment is input into the t-SNE dimension reduction module, the high-dimensional features are mapped to a two-dimensional plane, and visualized as shown in Figure 3 From Figure 3 It can be observed that the samples in the same cluster present a relatively compact cluster distribution in the feature space, and there is a clear separation boundary between different clusters, indicating that the method of the embodiment can obtain a clustering result with clear structure and good class separability in the unified feature space under the condition that the original image is invisible.

[0040] Further, to evaluate the clustering performance of the present application from a quantitative perspective, an evaluation scene with real labels is constructed on the public data set. A subset of images of 5 categories in the ImageNet data set is selected as the test data set, two routes of heterogeneous features are extracted, and are input into the trained diffusion generator model and the trained variational autoencoder model in the present embodiment in turn to obtain unified spatial feature representation; the clustering process of the present embodiment is used in the unified space to obtain unsupervised clustering results; then, the Hungarian algorithm is used to one-to-one match the clustering clusters and the real category labels, and the clustering accuracy is calculated based on the matching results. The experimental results show that under the above settings, the clustering accuracy obtained by the method of the present embodiment can reach 95.4%; under the same data and clustering cluster number conditions, if the traditional k-means clustering is directly used on the original heterogeneous features and the same evaluation method is used, due to the different conditions such as the dimension of the heterogeneous features, the clustering accuracy is only 48.3%. The comparison results show that the method of the present embodiment significantly improves the separability and clustering performance of the heterogeneous features in the unified feature space through the joint design of the diffusion alignment guided by the featureization noise, the mapping of the variational autoencoder model, and the unified space clustering.

[0041] The above only describes the preferred embodiments of the present application, and it should be noted that those skilled in the art can make several improvements and refinements without departing from the technical principles of the present application, and these improvements and refinements should also be considered within the protection scope of the present application.

Claims

1. A heterogeneous feature clustering method based on privacy images, characterized in that... Includes the following steps: Step 1: Obtain two sets of privacy image samples belonging to the first target domain and the second target domain respectively. Extract features through the first feature extraction network and the second feature extraction network, which are independent of each other, to obtain the first target domain feature dataset belonging to the first target domain and the second target domain feature dataset belonging to the second target domain. Generate the first feature noise based on the first target domain feature dataset and the second feature noise based on the second target domain feature dataset. Step 2: Construct a diffusion generator model with conditional denoising UNet as the core, and generate the first noisy target feature based on the first characteristic noise at each time step; At each time step, a second noisy target feature is generated based on the second characteristic noise; the first noisy target feature and the second noisy target feature are used as conditions to guide the diffusion generator model for training, resulting in a trained diffusion generator model and generating a pseudo image. Step 3: Input the pseudo-image into the first feature extraction network to obtain the first synthetic image domain features, input the pseudo-image into the second feature extraction network to obtain the second synthetic image domain features, construct a variational autoencoder model, train it with the first synthetic image domain features as the encoding input and the second synthetic image domain features as the decoding target, and obtain the trained variational autoencoder model. Map the first target domain features to the feature space where the second target domain features are located to form a unified spatial feature dataset. Step 4: Cluster the unified spatial feature dataset to obtain cluster centers and cluster labels.

2. The heterogeneous feature clustering method based on privacy images according to claim 1, characterized in that... The specific process of step 1 is as follows: Step 1-1: Obtain two sets of privacy image samples belonging to two different domains, and crop the sample size of both sets to X×Y to obtain a first sample set belonging to the first target domain and a second sample set belonging to the second target domain. Extract features from the image samples in the first sample set using a first feature extraction network to obtain features of the first target domain and form a first target domain feature dataset. Extract features from the image samples in the second sample set using a second feature extraction network to obtain features of the second target domain and form a second target domain feature dataset. The first and second feature extraction networks adopt a parallel and independent architecture. Steps 1-2: Obtain the mean and covariance of the first target domain feature dataset, and reparameterize and color a random Gaussian noise with the same dimension as the first target domain feature to obtain the first feature noise; Obtain the mean and covariance of the second target domain feature dataset, and reparameterize and color a random Gaussian noise with the same dimension as the second target domain feature to obtain the second feature noise.

3. The heterogeneous feature clustering method based on privacy images according to claim 2, characterized in that... In steps 1-2, the first characteristic noise is denoted as... v A , , μ A This represents the mean of the feature dataset of the first target domain. Σ A This represents the covariance of the feature dataset of the first target domain. Let the second characteristic noise be a random Gaussian noise with the same dimension as the feature of the first target domain; let the second characteristic noise be denoted as... v B , , μ B This represents the mean of the feature dataset of the second target domain. Σ B This represents the covariance of the feature dataset in the second target domain. This represents a random Gaussian noise with the same dimension as the feature of the second target domain.

4. The heterogeneous feature clustering method based on privacy images according to claim 3, characterized in that... The specific process of step 2 is as follows: Step 2-1: Construct a diffusion generator model to be trained based on the conditional denoising UNet architecture. Set the diffusion time step and the noise intensity of each time step. At each time step, combine the first feature noise with each first target domain feature according to the noise intensity of that time step to form the first noisy target feature of each first target domain feature at that time step. At each time step, combine the second feature noise with each second target domain feature according to the noise intensity of that time step to form the second noisy target feature of each second target domain feature at that time step. Step 2-2: According to the time step, the first noisy target feature and the second noisy target feature are used as conditions to guide the synchronous input of the diffusion generator model to be trained. In each time step, the diffusion generator model to be trained denoises a random Gaussian noise of a rectangular shape with size X×Y to obtain a denoised image, and extracts features from the denoised image through the first feature extraction network to obtain the first predicted feature. In each time step, it extracts features from the denoised image through the second feature extraction network to obtain the second predicted feature. Steps 2-3: Set the total loss function of the diffusion generator model to be trained by using the same-domain sample-level consistency constraint and the distribution-level alignment constraint, so that the first predicted feature is close to the first noisy target feature corresponding to the same time step, and the second predicted feature is close to the second noisy target feature corresponding to the same time step, and finally train the diffusion generator model. Steps 2-4: Use the first noisy target features and the second noisy target features as conditions to guide the trained diffusion generator model. Input random Gaussian noise of size X×Y into the trained diffusion generator model and output a denoised image. Define the denoised image output at this time as a pseudo image.

5. A heterogeneous feature clustering method based on privacy images according to claim 4, characterized in that... In step 2-1, the first target domain feature dataset is selected from the first target domain feature dataset. Features of the first target domain a i At time step t The first noisy target feature is denoted as , ,in, , N A This represents the total number of features in the first target domain feature dataset. r t It is a noise intensity scheduling function that monotonically decreases with time step, 0≤ r t ≤1, 1≤ t ≤ T , T This indicates the total number of time steps set in the diffusion time step; The second target domain feature dataset is the first j Features of the second target domain b j At time step t The second noisy target feature is denoted as , ,1≤ j ≤ N B , N B This represents the total number of features in the second target domain.

6. The heterogeneous feature clustering method based on privacy images according to claim 5, characterized in that... The specific process of training the diffusion generator model to be trained in steps 2-3 to obtain the trained diffusion generator model is as follows: Step 2-3-1: Set the total loss function for the diffusion generator model to be trained. , ,in, ω t Represents the loss term used for different time steps. The weighting coefficients for the time steps in the weighted summation. , This indicates a consistency constraint at the same domain sample level. Indicates the pseudo-image at time step When in the first predicted feature Features of the first noisy target The mean square error between them , The square of the L2 norm of a vector; Indicates the pseudo-image at time step t In the second predictive feature Features of the second noisy target The mean square error between them ; This represents the distribution-level alignment loss function used to compute the maximum mean difference in the reproducing kernel Hilbert space using a multi-bandwidth RBF kernel. , Indicates the time step as t The set of the first predicted features in the same batch at that time. Indicates the time step as t The set of second predicted features in the same batch at that time. MMD 2 This represents the operation of finding the maximum mean difference. It means exist The weight hyperparameters representing the importance of the elements. ; Step 2-3-2: Set the maximum number of training iterations for the diffusion generator model to be trained to 500. Based on the total loss function of the diffusion generator model to be trained, use the Adam optimizer to perform backpropagation and joint optimization on the parameters of the diffusion generator model to be trained. Stop training when the maximum number of training iterations is reached to obtain the trained diffusion generator model.

7. A heterogeneous feature clustering method based on privacy images according to claim 6, characterized in that... The specific process of step 3 is as follows: Step 3-1: Input the pseudo image into the first feature extraction network to obtain the first synthetic image domain feature, and input the pseudo image into the second feature extraction network to obtain the second synthetic image domain feature. The first synthetic image domain feature and the second synthetic image domain feature constitute a semantically aligned feature pair. Step 3-2: Construct the variational autoencoder model to be trained. Use the first synthetic image domain features as the encoding input and the second synthetic image domain features of the same feature pair as the decoding target. Use KL regularization loss and combine sample-level consistency constraints and distribution-level alignment constraints to construct the total loss function of the variational autoencoder model to be trained. Train the variational autoencoder model to be trained to obtain the trained variational autoencoder model. Step 3-3: Map all features of the first target domain to the same feature space as the feature dataset of the second target domain through the trained variational autoencoder model, and then form a unified spatial feature dataset together with the feature dataset of the second target domain.

8. The heterogeneous feature clustering method based on privacy images according to claim 7, characterized in that... In step 3-2, the total loss function of the variational autoencoder model to be trained is: , , Let KL be the KL regularized loss function, and let KL be the average KL divergence between the posterior distribution of each feature of the first synthetic image domain obtained by the encoder in the variational autoencoder model to be trained and the prior distribution of the second synthetic image domain in the current batch. , α , β , These are the preset weight hyperparameters. α =0.8, β =0.2, , This represents the sample-level consistency loss function. ,in, The first in the dataset consisting of the second synthetic image domain features m A second synthesized image domain feature, 1≤ m ≤ S , S This represents the total number of semantically aligned feature pairs. Input first synthetic image domain features The output of the variational autoencoder model to be trained at that time. The sign for calculating the length of a vector; This represents the distribution-level alignment loss function used to compute the maximum mean difference in the reproducing kernel Hilbert space using a multi-bandwidth RBF kernel. , MMD 2 B1 represents the maximum mean difference operation, B2 represents the set of features in the second synthetic image domain, and B3 represents the feature dataset output by the decoder in the variational autoencoder model to be trained.

9. A heterogeneous feature clustering method based on privacy images according to claim 1, characterized in that... The specific process of step 4 is as follows: Step 4-1: Construct a clustering model using the k-means clustering method with k-means++ initialization strategy, and input the samples in the unified spatial feature dataset as input samples into the clustering model. Step 4-2: Within the preset range of candidate clusters, perform k-means clustering with k-means++ initialization strategy once for each candidate cluster, calculate the corresponding average silhouette coefficient, and select the optimal number of clusters K based on the maximum value of the average silhouette coefficient. Step 4-3: Initialize K cluster centers on the unified spatial feature dataset using the k-means++ initialization strategy. Use the sum of squared Euclidean distances from the input sample to its respective cluster center as the objective function, and iteratively update the cluster centers and sample cluster assignments until the objective function converges to obtain the final clustering model. Use the sample cluster labels and corresponding cluster centers output by the final clustering model as the clustering results of the samples in the input unified spatial feature dataset to complete the clustering process.

Images