A privacy image based heterogeneous feature clustering method

Abstract

The application relates to a heterogeneous feature clustering method based on a privacy image, characterized in that a diffusion generation process is converted from a pixel domain to a feature domain through a featureized noise and matrix matching technology, a to-be-trained diffusion generator model with a conditional denoising UNet architecture as a core is constructed, and training is conducted in combination with a sample-level consistency constraint and a distribution-level alignment constraint, then a feature pair with mutually aligned semantics is constructed by using a pseudo image generated by the trained diffusion generator model; subsequently, a variational autoencoder model is trained, cross-domain feature mapping is realized through a double-layer alignment loss, and finally a unified domain feature dataset is formed and clustering is conducted; the method has the advantages that cross-domain feature unification is completed under the premise that no source domain data and no paired samples are available, original image leakage is effectively avoided, a high-quality unified feature set can be directly output, and a safe and efficient solution is provided for cross-domain visual analysis under a privacy limited scene.

Description

Technical Field

[0001] This invention relates to the field of image processing and data clustering technology, and in particular to a heterogeneous feature clustering method based on privacy images, which is applicable to cross-domain feature clustering analysis when the original data is unknown. Background Technology

[0002] In sensitive application scenarios such as medical image analysis, financial risk control, and cross-platform user profiling, data privacy and security are increasingly becoming top priorities. This leads to a common need: to jointly analyze and cluster heterogeneous features from different sources (domains) under strict privacy protection, where the original data (such as image pixels) is invisible and cannot be centralized. These heterogeneous features are typically generated by different, pre-trained feature extraction networks, such as EfficientNet and ResNet, and their feature space dimensions and statistical distributions differ significantly.

[0003] Existing technologies for cross-domain analysis of privacy-preserving images typically rely on direct access to the original image data or assumptions about highly consistent feature distributions. For example, traditional feature alignment and transfer learning methods often require pixel-level data augmentation, style transfer, or adversarial training, which not only increases computational complexity but also introduces the risk of privacy breaches. Furthermore, most existing methods depend on paired cross-domain samples or shared latent space assumptions, making them difficult to implement effectively in real-world scenarios where original data is scarce and paired samples are unavailable. In other words, achieving high-quality semantic alignment and unified representation of cross-domain features remains a critical problem that current technologies have yet to adequately address, given the constraints of invisible original images, heterogeneous features, and unsupervised alignment signals. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a heterogeneous feature clustering method based on privacy images, so as to achieve high-precision alignment and joint clustering of cross-domain heterogeneous features under the constraint that the original image is not visible.

[0005] The technical solution adopted by this invention to solve the above-mentioned technical problems is as follows: a heterogeneous feature clustering method based on privacy images, comprising the following steps:

[0006] Step 1-1: Obtain two sets of privacy image samples belonging to two different domains, and crop the sample size of both sets to X×Y to obtain a first sample set belonging to the first target domain and a second sample set belonging to the second target domain. Extract features from the image samples in the first sample set using a first feature extraction network to obtain features of the first target domain and form a first target domain feature dataset. Extract features from the image samples in the second sample set using a second feature extraction network to obtain features of the second target domain and form a second target domain feature dataset. The first and second feature extraction networks adopt a parallel and independent architecture.

[0007] Steps 1-2: Obtain the mean and covariance of the first target domain feature dataset, and reparameterize and color a random Gaussian noise with the same dimension as the first target domain feature to obtain the first feature noise; Obtain the mean and covariance of the second target domain feature dataset, and reparameterize and color a random Gaussian noise with the same dimension as the second target domain feature to obtain the second feature noise.

[0008] Step 2-1: Construct a diffusion generator model to be trained based on the conditional denoising UNet architecture. Set the diffusion time step and the noise intensity of each time step. At each time step, combine the first feature noise with each first target domain feature according to the noise intensity of that time step to form the first noisy target feature of each first target domain feature at that time step. At each time step, combine the second feature noise with each second target domain feature according to the noise intensity of that time step to form the second noisy target feature of each second target domain feature at that time step.

[0009] Step 2-2: According to the time step, the first noisy target feature and the second noisy target feature are used as conditions to guide the synchronous input of the diffusion generator model to be trained. In each time step, the diffusion generator model to be trained denoises a random Gaussian noise of a rectangular shape with size X×Y to obtain a denoised image, and extracts features from the denoised image through the first feature extraction network to obtain the first predicted feature. In each time step, it extracts features from the denoised image through the second feature extraction network to obtain the second predicted feature.

[0010] Steps 2-3: Set the total loss function of the diffusion generator model to be trained by using the same-domain sample-level consistency constraint and the distribution-level alignment constraint, so that the first predicted feature is close to the first noisy target feature corresponding to the same time step, and the second predicted feature is close to the second noisy target feature corresponding to the same time step, and finally train the diffusion generator model.

[0011] The first and second noisy target features are used as conditions to guide the trained diffusion generator model. Random Gaussian noise of size X×Y in the shape of a rectangle is input into the trained diffusion generator model and the denoised image is output. The denoised image output at this time is defined as a pseudo image.

[0012] Step 3-1: Input the pseudo image into the first feature extraction network to obtain the first synthetic image domain feature, and input the pseudo image into the second feature extraction network to obtain the second synthetic image domain feature. The first synthetic image domain feature and the second synthetic image domain feature constitute a semantically aligned feature pair.

[0013] Step 3-2: Construct the variational autoencoder model to be trained. Use the first synthetic image domain features as the encoding input and the second synthetic image domain features of the same feature pair as the decoding target. Use KL regularization loss and combine sample-level consistency constraints and distribution-level alignment constraints to construct the total loss function of the variational autoencoder model to be trained. Train the variational autoencoder model to be trained to obtain the trained variational autoencoder model.

[0014] Step 3-3: Map all features of the first target domain to the same feature space as the feature dataset of the second target domain through the trained variational autoencoder model, and then form a unified spatial feature dataset together with the feature dataset of the second target domain.

[0015] Step 4: Cluster all data in the unified domain feature dataset to obtain cluster centers and cluster labels, thus completing the clustering process.

[0016] Compared with existing technologies, the advantages of this invention are that it transforms the diffusion generation process from the pixel domain to the feature domain by using feature-based noise and moment matching techniques, constructs a diffusion generator model with conditional denoising UNet as the core, and trains it by combining sample-level consistency constraints and distribution-level alignment constraints; it constructs semantically aligned feature pairs using generated pseudo-images, trains an improved variational autoencoder, and achieves cross-domain feature mapping by replacing the target domain prior and using two-level alignment loss, ultimately forming a unified domain feature dataset and performing clustering.

[0017] This scheme achieves feature unification for cross-domain privacy-preserving images under the premise of passive domain data and no paired samples, effectively avoiding the risk of leakage of original images. By employing feature-based noise and moment matching techniques, the generation process of the diffusion generator model is transformed from the pixel domain to the feature domain, significantly improving training efficiency. The training strategy combining intra-domain sample-level consistency constraints and distribution-level alignment constraints ensures that the generated pseudo-images retain key semantic information while conforming to the statistical characteristics of the target domain, thereby improving the performance of downstream cross-domain recognition tasks. The variational autoencoder model enhances robustness to domain shifts and noise by introducing KL regularization loss and combining sample-level consistency constraints and distribution-level alignment constraints, and simplifies the complex dual-space mapping into a unified feature space learning, effectively improving clustering quality. The entire process requires no original images or additional annotations and can directly output a unified spatial feature dataset with semantic labels, providing an efficient and secure solution for cross-domain visual analysis in privacy-constrained scenarios.

[0018] Preferably, in steps 1-2, the first characteristic noise is denoted as... v A , , μ A This represents the mean of the feature dataset of the first target domain. Σ A This represents the covariance of the feature dataset of the first target domain. Let the second characteristic noise be a random Gaussian noise with the same dimension as the feature of the first target domain; let the second characteristic noise be denoted as... v B , , μ B This represents the mean of the feature dataset of the second target domain. Σ B This represents the covariance of the feature dataset in the second target domain. This represents a random Gaussian noise with the same dimension as the features of the second target domain. By using rigorous mathematical modeling and statistical transformations, and based on the statistical properties such as the mean and covariance of the first and second target domain feature datasets, the standard random Gaussian noise is reparameterized and colored to obtain the first and second characteristic noises that are statistically close to the distributions of the two domains, respectively.

[0019] Preferably, in step 2-1, the first target domain feature dataset is selected from the first target domain feature dataset. Features of the first target domain a i At time step t The first noisy target feature is denoted as , ,in, ,N A This represents the total number of features in the first target domain feature dataset. r t It is a noise intensity scheduling function that monotonically decreases with time step, 0≤ r t ≤1, 1≤ t ≤ T , T This indicates the total number of time steps set in the diffusion time step;

[0020] The second target domain feature dataset is the first j Features of the second target domain b j At time step t The second noisy target feature is denoted as , ,1≤ j ≤ N B , N B This represents the total number of features in the second target domain.

[0021] By further introducing a noise intensity scheduling function that monotonically decreases with time step, a first noisy target feature and a second noisy target feature that continuously evolve from high noise to low noise are constructed in the feature space. This design theoretically guarantees the stability and controllability of the generation process, and provides a clearly structured and statistically defined supervision signal for the subsequent training of the diffusion generator model, thereby facilitating the alignment and unification of cross-domain features.

[0022] Preferably, the specific process of training the diffusion generator model to be trained in steps 2-3 to obtain the trained diffusion generator model is as follows:

[0023] Step 2-3-1: Set the total loss function for the diffusion generator model to be trained. ,

[0024] ,in, ω t Represents the loss term used for different time steps. The weighting coefficients for the time steps in the weighted summation. , This indicates a consistency constraint at the same domain sample level. Indicates the pseudo-image at time step When in the first predicted feature Features of the first noisy target The mean square error between them , The square of the L2 norm of a vector;

[0025] Indicates the pseudo-image at time step t In the second predictive feature Features of the second noisy target The mean square error between them ;

[0026] This represents the distribution-level alignment loss function used to compute the maximum mean difference in the reproducing kernel Hilbert space using a multi-bandwidth RBF kernel. , Indicates time step as t The set of the first predicted features in the same batch at that time. Indicates the time step as t The set of second predicted features in the same batch at that time. MMD 2 This represents the operation of finding the maximum mean difference. It means exist The weight hyperparameters representing the importance of the elements. ;

[0027] Step 2-3-2: Set the maximum number of training iterations for the diffusion generator model to be trained to 500. Based on the total loss function of the diffusion generator model to be trained, use the Adam optimizer to perform backpropagation and joint optimization on the parameters of the diffusion generator model to be trained. Stop training when the maximum number of training iterations is reached to obtain the trained diffusion generator model.

[0028] By constructing a composite loss function that integrates intra-domain sample-level consistency constraints and distribution-level alignment constraints, the diffusion generator model achieves collaborative optimization across multiple time steps. This method simultaneously constrains the sample-by-sample matching accuracy between the generated predicted features and noisy target features in the same domain, as well as the overall distribution alignment of cross-domain features in the regenerating kernel Hilbert space, at each time step. This effectively promotes the unification of the dual-domain feature distribution while preserving individual semantic information. The joint training mechanism of time-step weighting and the Adam optimizer ensures the stability and efficiency of model convergence, providing a reliable training guarantee for generating semantically consistent and controllable cross-domain pseudo-images.

[0029] Preferably, in step 3-2, the total loss function of the variational autoencoder model to be trained is: , , Let KL regularization loss function represent the average KL divergence between the posterior distribution of each feature of the first synthetic image domain obtained by the encoder in the variational autoencoder model to be trained in the current batch and the prior distribution of the second synthetic image domain. KL regularization loss makes the latent space move closer to the target domain distribution during training, reducing cross-domain mismatch.

[0030] , α , β , These are the preset weight hyperparameters. α =0.8, β =0.2, , This represents the sample-level consistency loss function. ,in, The first in the dataset consisting of the second synthetic image domain features m A second synthesized image domain feature, 1≤ m ≤ S , S This represents the total number of semantically aligned feature pairs. Input first synthetic image domain features The output of the variational autoencoder model to be trained at that time. The sign for calculating the length of a vector;

[0031] This represents the distribution-level alignment loss function used to compute the maximum mean difference in the reproducing kernel Hilbert space using a multi-bandwidth RBF kernel. , MMD 2 The maximum mean difference operation is represented by B1, which represents the set of features in the second synthesized image domain, and B2 represents the feature dataset output by the decoder in the variational autoencoder model to be trained. By fusing sample-level consistency constraints and distribution-level maximum mean difference alignment, a two-level alignment mechanism is constructed, which not only ensures the individual semantic alignment between generated features and target features, but also achieves spatial matching of the overall distribution.

[0032] Preferably, the specific process of step 4 is as follows:

[0033] Step 4-1: Construct a clustering model using the k-means clustering method with k-means++ initialization strategy, and input the samples in the unified spatial feature dataset as input samples into the clustering model.

[0034] Step 4-2: Within the preset range of candidate clusters, perform k-means clustering with k-means++ initialization strategy once for each candidate cluster, calculate the corresponding average silhouette coefficient, and select the optimal number of clusters K based on the maximum value of the average silhouette coefficient.

[0035] Step 4-3: Initialize K cluster centers on the unified spatial feature dataset using the k-means++ initialization strategy. Use the sum of squared Euclidean distances from the input sample to its respective cluster center as the objective function, and iteratively update the cluster centers and sample cluster assignments until the objective function converges to obtain the final clustering model. Use the sample cluster labels and corresponding cluster centers output by the final clustering model as the clustering results of the samples in the input unified spatial feature dataset to complete the clustering process.

[0036] Compared to other clustering methods (such as hierarchical clustering, DBSCAN, Gaussian mixture models, etc.), k-means clustering offers advantages such as high computational efficiency, simple implementation, and strong adaptability to large-scale datasets, making it particularly suitable for clustering tasks with uniform spatial feature datasets in this scenario. Although existing clustering methods such as hierarchical clustering, spectral clustering, or density-based clustering can also achieve clustering, the k-means clustering method combined with the k-means++ initialization strategy effectively avoids the sensitivity issue of initial center selection, improving clustering stability and performance, thus making it more suitable for the requirement of efficient and automatic partitioning of feature distributions in this method. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the steps of the present invention;

[0038] Figure 2 This is a schematic diagram illustrating the principle of the method of the present invention;

[0039] Figure 3 In this embodiment, the unified spatial feature dataset processed by the method of this embodiment is input into the t-SNE dimensionality reduction module, and the high-dimensional features are mapped to a two-dimensional plane to visualize the result. Detailed Implementation

[0040] A heterogeneous feature clustering method based on privacy images, such as Figure 1 , Figure 2 As shown, it includes the following steps:

[0041] Step 1-1: Obtain two sets of privacy image samples belonging to two different domains, and crop the sample size of both sets to X×Y to obtain a first sample set belonging to the first target domain and a second sample set belonging to the second target domain. Extract features from the image samples in the first sample set using a first feature extraction network to obtain features for the first target domain and form a first target domain feature dataset. Extract features from the image samples in the second sample set using a second feature extraction network to obtain features for the second target domain and form a second target domain feature dataset. The first and second feature extraction networks adopt a parallel and independent architecture, that is, the two networks are independent of each other, do not form a cascade relationship, and do not share normalization or convolutional layer weights. Here, the heterogeneity of the extracted features is ensured by the difference between the two structures.

[0042] In this embodiment, the first feature extraction network comprises three parts: an EfficientNet-B4 feature backbone pre-trained on the ImageNet dataset, an average pooling layer, and a first flattening layer. It extracts features and pooling mean values ​​from image samples in the input first sample set, and then flattens them to form vector representations. Preferably, the first feature extraction network freezes its parameters in this heterogeneous feature clustering method, performing only feature extraction and not participating in gradient updates.

[0043] The second feature extraction network consists of three parts: a ResNet-18 backbone, an adaptive average pooling layer, and a second flattening layer. It uses a ResNet-18 model pre-trained on the ImageNet dataset as its backbone, removes the last two layers to the classification head, retains only the convolutional feature layers, and then connects to adaptive average pooling and flattening to obtain a vector representation. As a preferred option, the second feature extraction network can fine-tune or freeze its convolutional backbone parameters as needed.

[0044] Steps 1-2: Obtain the mean and covariance of the first target domain feature dataset; reparameterize and color a random Gaussian noise with the same dimension as the first target domain features to obtain the first feature noise; obtain the mean and covariance of the second target domain feature dataset; reparameterize and color a random Gaussian noise with the same dimension as the second target domain features to obtain the second feature noise; specifically, denoted as the first feature noise... v A , , μ A This represents the mean of the first target domain. Σ A This represents the covariance of the first target domain. Let the second characteristic noise be a random Gaussian noise with the same dimension as the feature of the first target domain; let the second characteristic noise be denoted as... v B , , μ B This represents the mean of the second target domain. Σ B This represents the covariance of the second objective domain. This represents a random Gaussian noise with the same dimension as the feature of the second target domain.

[0045] Step 2-1: Construct a diffusion generator model to be trained based on the conditional denoising UNet architecture. This model is an existing structure. Set the diffusion time step and the noise intensity of each time step for the diffusion generator model to be trained. At each time step, combine the first feature noise with each first target domain feature according to the noise intensity of that time step to form the first noisy target feature of each first target domain feature at that time step. At each time step, combine the second feature noise with each second target domain feature according to the noise intensity of that time step to form the second noisy target feature of each second target domain feature at that time step.

[0046] In step 2-1, the first target domain feature dataset is selected from the first target domain feature dataset. Features of the first target domain a i At time step t The first noisy target feature is denoted as , ,in, , N A This represents the total number of features in the first target domain feature dataset. r t It is a noise intensity scheduling function that monotonically decreases with time step, 0≤ r t ≤1, 1≤ t ≤ T , T This indicates the total number of time steps set in the diffusion time step;

[0047] The second target domain feature dataset is the first j Features of the second target domain b j At time step t The second noisy target feature is denoted as , ,1≤ j ≤ N B , N B This represents the total number of features in the second target domain.

[0048] Step 2-2: Following the time steps, the first and second noisy target features are used as conditional inputs to the diffusion generator model to be trained. At each time step, the diffusion generator model denoises a rectangular shape with dimensions X×Y containing random Gaussian noise to obtain a denoised image. The first feature extraction network then extracts features from the denoised image to obtain the first predicted feature. At each time step, the second feature extraction network extracts features from the denoised image to obtain the second predicted feature. Thus, at different time steps, the features extracted by the first and second feature extraction networks gradually approach their respective target states as the noise intensity increases, achieving cross-domain structural alignment under a unified semantic scale and providing a stable alignment benchmark for subsequent mapping and clustering.

[0049] Steps 2-3: Set the total loss function of the diffusion generator model to be trained by using the same-domain sample-level consistency constraint and the distribution-level alignment constraint, so that the first predicted feature is close to the first noisy target feature corresponding to the same time step, and the second predicted feature is close to the second noisy target feature corresponding to the same time step, and finally train the diffusion generator model.

[0050] The first and second noisy target features are used as conditions to guide the trained diffusion generator model. Random Gaussian noise of size X×Y in the shape of a rectangle is input into the trained diffusion generator model and the denoised image is output. The denoised image output at this time is defined as a pseudo image.

[0051] The specific process of steps 2-3 is as follows:

[0052] Step 2-3-1: Set the total loss function for the diffusion generator model to be trained. ,

[0053] ,in, ω t Represents the loss term used for different time steps. The weighting coefficients for the time steps in the weighted summation. , This indicates a consistency constraint at the same domain sample level. Indicates the pseudo-image at time step When in the first predicted feature Features of the first noisy target The mean square error between them , The square of the L2 norm of a vector;

[0054] Indicates the pseudo-image at time step t In the second predictive feature Features of the second noisy target The mean square error between them ;

[0055] This represents the distribution-level alignment loss function used to compute the maximum mean difference in the reproducing kernel Hilbert space using a multi-bandwidth RBF kernel. , Indicates the time step as t The set of the first predicted features in the same batch at that time. Indicates the time step as t The set of second predicted features in the same batch at that time. MMD 2 This represents the operation of finding the maximum mean difference. It means exist The weight hyperparameters representing the importance of the elements. .

[0056] Step 2-3-2: Set the maximum number of training iterations for the diffusion generator model to be trained to 500. Based on the total loss function of the diffusion generator model to be trained, use the Adam optimizer to perform backpropagation and joint optimization on the parameters of the diffusion generator model to be trained. Stop training when the maximum number of training iterations is reached to obtain the trained diffusion generator model.

[0057] During the above training process, the time step weight coefficients were adjusted. With weight hyperparameters A segmented annealing scheduling method is employed. Specifically, in the early stages of training, a higher weight is assigned to the consistency constraint at the same domain level to ensure that the diffusion generator model to be trained can learn basic noise modeling capabilities. As training progresses, the weight of the distribution-level alignment loss is gradually increased to enhance the cross-domain alignment effect. To improve the stability and generalization ability of training, a gradient pruning strategy is also adopted to prevent gradient explosion, as well as a parameter exponential moving average strategy, which obtains smoother and more stable model parameters by weighted averaging of the historical values ​​of the parameters.

[0058] A diffusion generator with conditional denoising UNet as its backbone is constructed. The inverse denoising process is supervised by feature consistency loss and distribution-level alignment loss, rather than relying on traditional noise prediction loss, thereby achieving alignment of heterogeneous features when they are not visible in the original image.

[0059] Step 3-1: Input the pseudo-image into the first feature extraction network to obtain the first synthetic image domain feature, and input the pseudo-image into the second feature extraction network to obtain the second synthetic image domain feature. The first synthetic image domain feature and the second synthetic image domain feature constitute a semantically aligned feature pair.

[0060] Step 3-2: Construct the variational autoencoder model to be trained. Use the features of the first synthesized image domain as the encoding input and the features of the second synthesized image domain with the same feature pair as the decoding target. Employ KL regularization loss and combine sample-level consistency constraints with distribution-level alignment constraints to construct the total loss function of the variational autoencoder model to be trained. Train the variational autoencoder model to obtain the trained variational autoencoder model. The total loss function of the variational autoencoder model to be trained is: , , Let KL be the KL regularized loss function, which represents the average KL divergence between the posterior distribution of each feature of the first synthetic image domain obtained by the encoder in the variational autoencoder model to be trained in the current batch and the prior distribution of the second synthetic image domain. The KL regularized loss function makes the latent space move closer to the target domain distribution during training, reducing cross-domain mismatch.

[0061] , α , β , These are the preset weight hyperparameters. α =0.8, β =0.2, , This represents the sample-level consistency loss function. ,in, The first in the dataset consisting of the second synthetic image domain features m A second synthesized image domain feature, 1≤ m ≤ S , S This represents the total number of semantically aligned feature pairs. Input first synthetic image domain features The output of the variational autoencoder model to be trained at that time. The symbol for calculating the length of a vector.

[0062] This represents the distribution-level alignment loss function used to compute the maximum mean difference in the reproducing kernel Hilbert space using a multi-bandwidth RBF kernel. , MMD 2 B1 represents the maximum mean difference operation, B2 represents the set of features in the second synthetic image domain, and B3 represents the feature dataset output by the decoder in the variational autoencoder model to be trained.

[0063] Step 3-3: Map all features of the first target domain to the same feature space as the feature dataset of the second target domain through the trained variational autoencoder model, and then form a unified spatial feature dataset together with the feature dataset of the second target domain.

[0064] Step 4: Cluster all data in the unified domain feature dataset to obtain cluster centers and cluster labels, completing the clustering process. The specific process is as follows:

[0065] Step 4-1: Construct a clustering model using the k-means clustering method with k-means++ initialization strategy, and input the samples in the unified spatial feature dataset as input samples into the clustering model.

[0066] Step 4-2: Within the preset range of candidate cluster numbers, perform k-means clustering with the k-means++ initialization strategy once for each candidate cluster number, calculate the corresponding average silhouette coefficient, and select the optimal number of clusters K based on the maximum value of the average silhouette coefficient.

[0067] Step 4-3: Initialize K cluster centers on the unified spatial feature dataset using the k-means++ initialization strategy. Use the sum of squared Euclidean distances from the input sample to its respective cluster center as the objective function, and iteratively update the cluster centers and sample cluster assignments until the objective function converges to obtain the final clustering model. Use the sample cluster labels and corresponding cluster centers output by the final clustering model as the clustering results of the samples in the input unified spatial feature dataset to complete the clustering process.

[0068] In a specific example of this embodiment, in order to verify the effectiveness of the heterogeneous feature clustering method based on privacy images in terms of unified feature space alignment and clustering, the evaluation is carried out from both qualitative and quantitative perspectives. The heterogeneous feature clustering method based on privacy images is referred to as the method of this embodiment.

[0069] First, without the participation of real labels, the unified spatial features obtained in this embodiment are visualized and analyzed. Specifically, the unified spatial feature dataset processed by the method of this embodiment is input into the t-SNE dimensionality reduction module to map the high-dimensional features to a two-dimensional plane, and as shown... Figure 3 Visualization is shown below. From Figure 3 It can be observed that samples within the same cluster exhibit a relatively compact clustered distribution in the feature space, and there are obvious interval boundaries between different clusters. This indicates that the method of this embodiment can obtain clustering results with clear structure and good inter-class separability in a unified feature space even when the original image is not visible.

[0070] Furthermore, to quantitatively evaluate the clustering performance of this invention, an evaluation scenario with real labels was constructed on a public dataset. A subset of images from five categories in the ImageNet dataset was selected as the test dataset. Two heterogeneous features were extracted and sequentially input into the trained diffusion generator model and the trained variational autoencoder model in this embodiment to obtain a unified spatial feature representation. Unsupervised clustering results were obtained in the unified space using the clustering process of this embodiment. Subsequently, the Hungarian algorithm was used to match the clusters with the real category labels one by one, and the clustering accuracy was calculated based on the matching results. Experimental results show that, under the above settings, the clustering accuracy obtained by the method in this embodiment can reach 95.4%. Under the same data and number of clusters, if traditional k-means clustering is directly applied to the original heterogeneous features and the same evaluation method is used, the clustering accuracy is only 48.3% due to the different dimensions and other conditions of the heterogeneous features. The comparison results show that the method in this embodiment significantly improves the separability and clustering performance of heterogeneous features in a unified feature space through the joint design of characteristic noise-guided diffusion alignment, variational autoencoder model mapping, and unified space clustering.

[0071] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A heterogeneous feature clustering method based on privacy images, characterized in that... Includes the following steps: Step 1: Obtain two sets of privacy image samples belonging to the first target domain and the second target domain respectively. Extract features through the first feature extraction network and the second feature extraction network, which are independent of each other, to obtain the first target domain feature dataset belonging to the first target domain and the second target domain feature dataset belonging to the second target domain. Generate the first feature noise based on the first target domain feature dataset and the second feature noise based on the second target domain feature dataset. Step 2: Construct a diffusion generator model with conditional denoising UNet as the core, and generate the first noisy target feature based on the first characteristic noise at each time step; At each time step, a second noisy target feature is generated based on the second characteristic noise; the first noisy target feature and the second noisy target feature are used as conditions to guide the diffusion generator model for training, resulting in a trained diffusion generator model and generating a pseudo image. Step 3: Input the pseudo-image into the first feature extraction network to obtain the first synthetic image domain features, input the pseudo-image into the second feature extraction network to obtain the second synthetic image domain features, construct a variational autoencoder model, train it with the first synthetic image domain features as the encoding input and the second synthetic image domain features as the decoding target, and obtain the trained variational autoencoder model. Map the first target domain features to the feature space where the second target domain features are located to form a unified spatial feature dataset. Step 4: Cluster the unified spatial feature dataset to obtain cluster centers and cluster labels. The specific process is as follows: Step 4-1: Construct a clustering model using the k-means clustering method with k-means++ initialization strategy, and input the samples in the unified spatial feature dataset as input samples into the clustering model. Step 4-2: Within the preset range of candidate clusters, perform k-means clustering with k-means++ initialization strategy once for each candidate cluster, calculate the corresponding average silhouette coefficient, and select the optimal number of clusters K based on the maximum value of the average silhouette coefficient. Step 4-3: Initialize K cluster centers on the unified spatial feature dataset using the k-means++ initialization strategy. Use the sum of squared Euclidean distances from the input sample to its respective cluster center as the objective function, and iteratively update the cluster centers and sample cluster assignments until the objective function converges to obtain the final clustering model. Use the sample cluster labels and corresponding cluster centers output by the final clustering model as the clustering results of the samples in the input unified spatial feature dataset to complete the clustering process.

2. The heterogeneous feature clustering method based on privacy images according to claim 1, characterized in that... The specific process of step 1 is as follows: Step 1-1: Obtain two sets of privacy image samples belonging to two different domains, and crop the sample size of both sets to X×Y to obtain a first sample set belonging to the first target domain and a second sample set belonging to the second target domain. Extract features from the image samples in the first sample set using a first feature extraction network to obtain features of the first target domain and form a first target domain feature dataset. Extract features from the image samples in the second sample set using a second feature extraction network to obtain features of the second target domain and form a second target domain feature dataset. The first and second feature extraction networks adopt a parallel and independent architecture. Steps 1-2: Obtain the mean and covariance of the first target domain feature dataset, and reparameterize and color a random Gaussian noise with the same dimension as the first target domain feature to obtain the first feature noise; Obtain the mean and covariance of the second target domain feature dataset, and reparameterize and color a random Gaussian noise with the same dimension as the second target domain feature to obtain the second feature noise.

3. The heterogeneous feature clustering method based on privacy images according to claim 2, characterized in that... In steps 1-2, the first characteristic noise is denoted as... v A , , μ A This represents the mean of the feature dataset of the first target domain. Σ A This represents the covariance of the feature dataset of the first target domain. Let the second characteristic noise be a random Gaussian noise with the same dimension as the feature of the first target domain; let the second characteristic noise be denoted as... v B , , μ B This represents the mean of the feature dataset of the second target domain. Σ B This represents the covariance of the feature dataset in the second target domain. This represents a random Gaussian noise with the same dimension as the feature of the second target domain.

4. The heterogeneous feature clustering method based on privacy images according to claim 3, characterized in that... The specific process of step 2 is as follows: Step 2-1: Construct a diffusion generator model to be trained based on the conditional denoising UNet architecture. Set the diffusion time step and the noise intensity of each time step. At each time step, combine the first feature noise with each first target domain feature according to the noise intensity of that time step to form the first noisy target feature of each first target domain feature at that time step. At each time step, combine the second feature noise with each second target domain feature according to the noise intensity of that time step to form the second noisy target feature of each second target domain feature at that time step. Step 2-2: According to the time step, the first noisy target feature and the second noisy target feature are used as conditions to guide the synchronous input of the diffusion generator model to be trained. In each time step, the diffusion generator model to be trained denoises a random Gaussian noise of a rectangular shape with size X×Y to obtain a denoised image, and extracts features from the denoised image through the first feature extraction network to obtain the first predicted feature. In each time step, it extracts features from the denoised image through the second feature extraction network to obtain the second predicted feature. Steps 2-3: Set the total loss function of the diffusion generator model to be trained by using the same-domain sample-level consistency constraint and the distribution-level alignment constraint, so that the first predicted feature is close to the first noisy target feature corresponding to the same time step, and the second predicted feature is close to the second noisy target feature corresponding to the same time step, and finally train the diffusion generator model. Steps 2-4: Use the first noisy target features and the second noisy target features as conditions to guide the trained diffusion generator model. Input random Gaussian noise of size X×Y into the trained diffusion generator model and output a denoised image. Define the denoised image output at this time as a pseudo image.

5. A heterogeneous feature clustering method based on privacy images according to claim 4, characterized in that... In step 2-1, the first target domain feature dataset is selected from the first target domain feature dataset. Features of the first target domain a i At time step t The first noisy target feature is denoted as , ,in, , N A This represents the total number of features in the first target domain feature dataset. r t It is a noise intensity scheduling function that monotonically decreases with time step, 0≤ r t ≤1, 1≤ t ≤ T , T This indicates the total number of time steps set in the diffusion time step; The second target domain feature dataset is the first j Features of the second target domain b j At time step t The second noisy target feature is denoted as , ,1≤ j ≤ N B , N B This represents the total number of features in the second target domain.

6. The heterogeneous feature clustering method based on privacy images according to claim 5, characterized in that... The specific process of training the diffusion generator model to be trained in steps 2-3 to obtain the trained diffusion generator model is as follows: Step 2-3-1: Set the total loss function for the diffusion generator model to be trained. , ,in, ω t Represents the loss term used for different time steps. The weighting coefficients for the time steps in the weighted summation. , This indicates a consistency constraint at the same domain sample level. Indicates the pseudo-image at time step When in the first predicted feature Features of the first noisy target The mean square error between them , The square of the L2 norm of a vector; Indicates the pseudo-image at time step t In the second predictive feature Features of the second noisy target The mean square error between them ; This represents the distribution-level alignment loss function used to compute the maximum mean difference in the reproducing kernel Hilbert space using a multi-bandwidth RBF kernel. , Indicates the time step as t The set of the first predicted features in the same batch at that time. Indicates the time step as t The set of second predicted features in the same batch at that time. MMD 2 This represents the operation of finding the maximum mean difference. It means exist The weight hyperparameters representing the importance of the elements. ; Step 2-3-2: Set the maximum number of training iterations for the diffusion generator model to be trained to 500. Based on the total loss function of the diffusion generator model to be trained, use the Adam optimizer to perform backpropagation and joint optimization on the parameters of the diffusion generator model to be trained. Stop training when the maximum number of training iterations is reached to obtain the trained diffusion generator model.

7. A heterogeneous feature clustering method based on privacy images according to claim 6, characterized in that... The specific process of step 3 is as follows: Step 3-1: Input the pseudo image into the first feature extraction network to obtain the first synthetic image domain feature, and input the pseudo image into the second feature extraction network to obtain the second synthetic image domain feature. The first synthetic image domain feature and the second synthetic image domain feature constitute a semantically aligned feature pair. Step 3-2: Construct the variational autoencoder model to be trained. Use the first synthetic image domain features as the encoding input and the second synthetic image domain features of the same feature pair as the decoding target. Use KL regularization loss and combine sample-level consistency constraints and distribution-level alignment constraints to construct the total loss function of the variational autoencoder model to be trained. Train the variational autoencoder model to be trained to obtain the trained variational autoencoder model. Step 3-3: Map all features of the first target domain to the same feature space as the feature dataset of the second target domain through the trained variational autoencoder model, and then form a unified spatial feature dataset together with the feature dataset of the second target domain.

8. The heterogeneous feature clustering method based on privacy images according to claim 7, characterized in that... In step 3-2, the total loss function of the variational autoencoder model to be trained is: , , Let KL be the KL regularized loss function, and let KL be the average KL divergence between the posterior distribution of each feature of the first synthetic image domain obtained by the encoder in the variational autoencoder model to be trained and the prior distribution of the second synthetic image domain in the current batch. , α , β , These are the preset weight hyperparameters. α =0.8, β =0.2, , This represents the sample-level consistency loss function. ,in, The first in the dataset consisting of the second synthetic image domain features m A second synthesized image domain feature, 1≤ m ≤ S , S This represents the total number of semantically aligned feature pairs. Input first synthetic image domain features The output of the variational autoencoder model to be trained at that time. The sign for calculating the length of a vector; This represents the distribution-level alignment loss function used to compute the maximum mean difference in the reproducing kernel Hilbert space using a multi-bandwidth RBF kernel. , MMD 2 B1 represents the maximum mean difference operation, B2 represents the set of features in the second synthetic image domain, and B3 represents the feature dataset output by the decoder in the variational autoencoder model to be trained.

Images