Multi-view clustering method for commodity images

Abstract

The invention provides a commodity image-oriented multi-view clustering method, and belongs to the technical field of image processing, and the method comprises the steps: obtaining a multi-view commodity image; a multi-layer connection attention auto-encoder model is introduced to learn vector representation of a multi-view commodity image; on the basis of the learned multi-layer connection attention auto-encoder model, exploring complementation and consistency information of the multi-view commodity images, and measuring a feature consistency structure of the multi-view commodity images; based on the semantic information of the feature consistency structure, obtaining a similarity matrix through a multi-layer attention perceptron; and clustering the views by using the similarity matrix. According to the method, the problem of attention loss of an existing multi-view commodity image clustering method is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a multi-view clustering method for commodity images. Background Art

[0002] Multi-view clustering is an important machine learning method that enhances clustering effects by combining information from multiple views. Traditional single-view clustering methods usually rely on a single data source or feature view, while multi-view clustering can more comprehensively capture the intrinsic structure of the data by integrating data from different views, thereby improving the accuracy and robustness of clustering. In the fields of computer vision and pattern recognition, the diversity and complexity of image data make effective image clustering challenging. Especially in the fashion industry, the diversity of product images and rapidly changing trends increase the complexity of this task. Each product image may contain multiple views, such as color, texture, shape, and structure, and each view carries different useful information. If clustering is performed based on a single view, it is often impossible to fully utilize all the information of the image, resulting in unsatisfactory clustering effects.

[0003] With the popularity of e-commerce platforms, the types and quantity of fashion goods have increased dramatically. Effective image clustering methods can help platforms better manage goods, optimize recommendation systems, and provide users with personalized shopping experiences. For example, by clustering similar goods, recommendations based on visual similarity can be achieved, which can improve users' purchase conversion rates. In addition, the fashion industry is highly dependent on changes in trends. Through clustering analysis of a large number of product images, popular trends and patterns can be identified, thereby helping designers and brands to adjust product design and marketing strategies in a timely manner. Multi-view clustering methods can capture popular elements in different views and provide more accurate trend analysis. Inventory management in the fashion industry also faces huge challenges. Goods of different styles and colors need to be accurately classified and managed to reduce inventory backlogs and out-of-stock phenomena. Through multi-view clustering methods, fine classification of inventory goods can be achieved and the efficiency of inventory management can be improved. Multi-view clustering methods are of great value in these application scenarios. However, existing multi-view clustering methods still have some problems, such as high computational complexity, sensitivity to noisy data, and insufficient information fusion between views. Therefore, proposing a new multi-view clustering method that can more efficiently integrate multi-view information and improve the accuracy and robustness of clustering is an important topic in commodity image clustering research. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a multi-view clustering method for commodity images, which solves the problem of lack of attention in the existing multi-view commodity image clustering methods.

[0005] In order to achieve the above objectives, the technical solution adopted by the present invention is: a multi-view clustering method for commodity images, comprising the following steps:

[0006] S1, obtaining multi-view product images;

[0007] S2, introduce a multi-layer connected attention autoencoder model to learn the vector representation of multi-view product images;

[0008] S3. Based on the learned multi-layer connected attention autoencoder model, explore the complementary and consistent information of multi-view product images and measure the feature consistency structure of multi-view product images;

[0009] S4, based on the semantic information of the feature consistency structure, the similarity matrix is ​​obtained through a multi-layer attention perceptron;

[0010] S5. Cluster the views using the similarity matrix.

[0011] Beneficial effects of the present invention: The input of the present invention is product image data, which contains multiple views and is product images of different styles and angles (such as fashion product images). The output is the clustering results of the product images, which are obtained by finding complementary and consistent information in multiple views and the similarity matrix obtained by the multi-layer attention perceptron. These clustering results have high accuracy and stability and can effectively process product image data (such as fashion product images). The present invention makes full use of the self-expression characteristics of existing features in the data, has good robustness for the processing of multi-view features, and solves the shortcomings of traditional methods in dealing with such problems. Therefore, the present invention has good accuracy and robustness in processing the clustering problem of multi-view product image data.

[0012] Furthermore, the step S2 is specifically as follows:

[0013] For the data sample of the vth view of the i-th sample It is converted into an embedded feature representation through an attention encoder The attention mechanism and semantic connection block are introduced in the attention encoder to extract consistent feature representations between different views to learn the vector representation of multi-view product images.

[0014] The beneficial effect of the above further solution is that by introducing the attention mechanism and semantic connection block in the attention encoder, it is possible to more accurately capture and extract consistent feature representations between different views. This method not only effectively enhances the model's ability to understand multi-view data, but also improves the accuracy and expressiveness of feature representation.

[0015] Furthermore, the embedded feature representation The expression is as follows:

[0016]

[0017] in, and They represent the encoder function and weight parameters of view v respectively, represents the data sample of the vth view of the i-th sample;

[0018] The expression of the output of the semantic connection block is as follows:

[0019] E i' (x) = F(E i'-1 (x),W i )+W i E i'-1 (x)

[0020] Among them, E i' (x) represents the output of the semantic connection block at the i'th layer, F represents the calculation function of the semantic connection block, and E i'-1 (x) represents the output of the semantic connection block at layer i'-1, W i Represents the computational weight of the semantically connected block.

[0021] The beneficial effect of the above further solution is that by accurately defining and calculating the embedded feature representation and the output of the semantic connection block, the present invention can better capture and express the complex semantic relationship between multi-view product images. This not only improves the richness and expressiveness of the feature representation, but also ensures the consistency and complementarity of information between views. This solution significantly enhances the robustness of the model, making subsequent clustering operations more accurate and stable, thereby further improving the overall performance of multi-view product image clustering.

[0022] Furthermore, the expression of the reconstruction loss function of the multi-layer connected attention autoencoder model in step S3 is as follows:

[0023]

[0024] Among them, L pre represents the reconstruction loss function, i represents the i-th sample, v represents the v-th view, M represents the number of views, N represents the number of samples, represents the data sample of the vth view of the i-th sample, Representation Embedding The reconstruction result obtained by reconstructing the original data sample through the decoder, represents the decoder function, represents the decoder weights.

[0025] The beneficial effect of the above further solution is that by defining a reconstruction loss function, the present invention can effectively measure the performance of the multi-layer connected attention autoencoder model in reconstructing multi-view commodity image data. Specifically, the reconstruction loss function guides the training process of the model by comparing the difference between the original data sample and its reconstruction result, ensuring that the embedded representation can accurately retain the key information of the original data.

[0026] Furthermore, the step S4 is specifically as follows:

[0027] Based on the semantic information of the feature consistency structure and the feature representation Z of multiple views, a multi-layer attention perceptron and softmax function are stacked on the feature representation Z to obtain the cluster distribution matrix H, where Represent the embedded feature representation;

[0028] Calculate the attention mechanism channel matrices Q and K on the cluster distribution matrix H to obtain the score matrix S;

[0029] Consider the score matrix S as a similarity matrix.

[0030] The beneficial effect of the above further scheme is that by stacking multiple layers of attention perceptrons and softmax functions on the feature representation, the present invention can further enhance the model's ability to capture features of multi-view product images. Specifically, this design can not only extract richer semantic information in the feature consistency structure, but also normalize the feature representation through the softmax function to generate a clustering distribution matrix with higher discrimination. In the process of calculating the attention mechanism channel matrices Q and K, the present invention utilizes the correlation between the feature representations to generate a score matrix S, which can effectively reflect the similarity relationship between product images. The generation process of this similarity matrix can not only improve the accuracy of the clustering results, but also enhance the robustness and stability of the model when processing multi-view product images.

[0031] Furthermore, the loss function for clustering views in step S5 is expressed as follows:

[0032] L total =L pre +L fine

[0033]

[0034] Among them, L total represents the loss function for clustering views, L pre represents the reconstruction loss function, L finerepresents the fine-tuning clustering consistency loss function, m represents the mth view of the sample data, M represents the number of views of the sample data, n represents the nth view, Indicates H m and H n The label contrast loss between m represents the cluster representation of the m-th view, H n represents the cluster representation of the nth view that is different from the mth one, j represents the jth sample compared during calculation, K represents the number of clusters, represents the regular term, v represents the vth view compared during calculation, Represents cluster samples and The cosine similarity of L represents the temperature coefficient, represents the cluster representation of the m-th view of the i-th sample, represents the cluster representation of the nth view of the jth sample, <,> represents the dot product calculation, i represents the i-th sample, Represents regular calculation.

[0035] The beneficial effect of the above further solution is that by introducing a comprehensive loss function for clustering views, the present invention can balance the key factors in the multi-view product image clustering process and ensure the accuracy and robustness of the clustering results. Specifically, the parameter L pre It ensures the effective reconstruction of feature representation, so that the key information of the original data can be retained; the parameter L fine The label comparison loss is introduced in the clustering process to strengthen the consistency between different views. By comparing the clustering representations of similar samples, the aggregation of clustering results is enhanced. The features are normalized through regularization terms to prevent overfitting of the model and further improve the generalization ability of the model. This design of multiple loss functions enables the present invention to not only accurately capture the multi-view features in commodity image data, but also effectively process the differences between samples in the clustering process, thereby obtaining more stable and accurate clustering results. This design is particularly suitable for complex multi-view data scenarios, greatly improving the applicability and practicality of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0037] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0038] Example

[0039] Before describing the present invention, the meanings of the following parameters are explained:

[0040]

[0041]

[0042] like Figure 1 As shown, the present invention provides a multi-view clustering method for commodity images, and its implementation method is as follows:

[0043] S1, obtaining multi-view product images;

[0044] In this embodiment, multi-view product images (such as fashion product images, which are taken as an example below) are prepared. Among them, there are N fashion product image samples, X (v) represents the vth view of a multi-view fashion product image, d v represents the feature dimension in the vth view, each view X (v) , each with N features. Multi-view clustering (MVC) aims to classify N fashion product image samples into c clusters by integrating all views, where c is the clustering analysis result to be obtained.

[0045] S2. Introduce a multi-layer connected attention autoencoder model to learn the vector representation of multi-view product images. Specifically:

[0046] For the data sample of the vth view of the i-th sample It is converted into an embedded feature representation through an attention encoder The attention mechanism and semantic connection block are introduced in the attention encoder to extract consistent feature representations between different views to learn the vector representation of multi-view product images.

[0047] In this embodiment, a multi-layer connected attention autoencoder model is introduced to learn the vector representation of multi-view fashion product images. The goal of the present invention is to optimize parameter initialization. For a specific data sample of the vth view of the i-th sample It is converted into an embedded feature representation through an attention encoder

[0048]

[0049] in, and They represent the encoder function and related weight parameters of view v respectively, represents the data sample of the vth view of the ith sample.

[0050] In this process, the present invention introduces an attention mechanism and a semantic connection module in the encoder to extract highly consistent feature representations between different views. This strategy aims to capture and strengthen the consistency information between views, thereby retaining more semantic information about the relationship between the original data samples during the encoding process. In this way, the embedded representation It not only captures the features within a single view, but also reflects the common semantic features between different views.

[0051] In this embodiment, the semantic connection block adds direct connections so that the input can be directly transmitted to the output, thereby retaining the original semantic information and the consistent semantic information in multiple views. In the encoder structure, it can be described as:

[0052] E i (x) = F(E i-1 (x),W i )+W i E i-1 (x) (2)

[0053] Among them, F represents the computation function of the semantic connection block, E i'-1 (x) represents the output of the semantic connection block at layer i'-1, W i represents the computational weight of the semantic connection block, E i' (x) indicates that the output of the semantic connection block of the i'th layer comes from the output of the semantic connection block of the previous layer and the calculation of this layer. Each semantic connection block retains the most original semantic information, thereby ensuring the stability of the consistent features.

[0054] S3. Based on the learned multi-layer connected attention autoencoder model, explore the complementary and consistent information of multi-view product images and measure the feature consistency structure of multi-view product images;

[0055] In this embodiment, the present invention explores the complementary and consistent information of multi-view fashion product images based on the multi-layer connected attention autoencoder model and measures the feature consistency structure of multi-view fashion product images. The original data samples are reconstructed through the decoder module, and the reconstruction result is expressed as

[0056]

[0057] In the pre-training stage, the overall goal of the present invention is to minimize the reconstruction loss across all views, which is obtained by the following formula:

[0058]

[0059] Among them, L pre represents the reconstruction loss function, i represents the i-th sample, v represents the v-th view, M represents the number of views, N represents the number of samples, represents the data sample of the vth view of the i-th sample, Representation Embedding The reconstruction result obtained by reconstructing the original data sample through the decoder, represents the decoder function, represents the decoder weight, and Both are view v specific.

[0060] In this embodiment, the introduction of the semantic connection module and the attention mechanism significantly enhances the ability of the encoder, which can not only extract features within the view, but also capture and utilize the consistency information between different views, thereby improving the performance of multi-view learning tasks.

[0061] S4. Based on the semantic information of the feature consistency structure, the similarity matrix is ​​obtained through a multi-layer attention perceptron, which is specifically:

[0062] Based on the semantic information of the feature consistency structure and the feature representation Z of multiple views, a multi-layer attention perceptron and softmax function are stacked on the feature representation Z to obtain the cluster distribution matrix H, where Represent the embedded feature representation;

[0063] Calculate the attention mechanism channel matrices Q and K on the cluster distribution matrix H to obtain the score matrix S;

[0064] Consider the score matrix S as a similarity matrix.

[0065] In this embodiment, a similarity matrix is ​​obtained through a multi-layer attention perceptron based on feature consistency structural semantic information. (in From the vth view), the present invention stacks a multi-layer perceptron (MLP) and a softmax function on Z to obtain a cluster distribution matrix H. The present invention calculates the attention mechanism channel matrices Q and K on the cluster distribution matrix H to obtain a score matrix S, which is regarded as a similarity matrix S that captures the intrinsic similarities between different cluster distributions.

[0066] S5. Cluster the views using the similarity matrix.

[0067] In this embodiment, a Laplacian matrix is ​​constructed from the similarity matrix S, and spectral clustering is performed to obtain a clustering result C.

[0068] Specifically, a set of features obtained by formula (1) (v represents the vth view), the present invention notes that these features represent weakly consistent multi-view feature representations, called weakly consistent representations. In order to enhance these representations, the goal of the present invention is to obtain strongly consistent cluster labels, called strongly consistent representations. To this end, A three-layer linear MLP is applied on it, expressed as W H represents the weight parameter of MLP, F represents the calculation function of MLP, represents the embedded feature representation, v represents the vth view, and V represents a total of V views, i.e., the number of views, which will produce a set of cluster representations The request matrix Q and key matrix K are calculated independently through the attention mechanism and expressed as W Q and W K Respectively represent the weight parameters of Q and K in the attention mechanism, i represents the i-th sample, and N represents the number of samples. The similarity matrix S is calculated as follows: The clustering result C is obtained by using spectral clustering.

[0069] In the weak consistency space, the reconstruction goal given by formula (1) ensures that The representation ability of is preserved, thus alleviating the model collapse problem. In the strong consistency space, contrastive learning is used to ensure that H and C move closer to learning universal semantic consistency.

[0070] In practice, due to the influence of view-specific information, some views may still have inconsistent cluster labels. To ensure robustness, this paper aims to achieve cluster consistency, that is, the same cluster labels on all views should represent the same semantic clusters. To achieve this goal, contrastive learning is used. For the mth view, similar cluster labels Form a (MK-1) tag pair, that is in It is constructed into (M-1) positive label pairs, and the remaining M(K-1) label pairs are regarded as negative label pairs. represents the cluster representation of the m-th view of the j-th sample, which is also the sample label being compared. represents the cluster representation of the nth view of the jth sample, representing the sample label for comparison, represents the cluster representation of the nth view of the kth sample, n represents the nth view, k represents the kth sample, M represents the number of views, and K represents the total number of clusters. The present invention uses the cosine distance to measure the similarity between two feature vectors: L total =L pre +L fine

[0071]

[0072] Further define H m and H n The label comparison loss between them is as follows:

[0073]

[0074] The final fine-tuned clustering consistency loss is defined as:

[0075]

[0076] is a regular term. The total loss function consists of two parts: pre-training loss and contrastive learning loss:

[0077] L total =L pre +L fine (8)

[0078] Among them, L total represents the loss function for clustering views, L pre represents the reconstruction loss function, L fine represents the fine-tuning clustering consistency loss function, m represents the mth view of the sample data, M represents the number of views of the sample data, n represents the nth view, Indicates H m and H n The label contrast loss between m represents the cluster representation of the m-th view, H n represents the cluster representation of the nth view that is different from the mth one, j represents the jth sample compared during calculation, K represents the number of clusters, represents the regular term, v represents the vth view compared during calculation, Represents cluster samples and The cosine similarity of L represents the temperature coefficient, represents the cluster representation of the m-th view of the i-th sample, represents the cluster representation of the nth view of the jth sample, <,> represents the dot product calculation, i represents the i-th sample, Represents regular calculation.

[0079] In this embodiment, the similarity matrix may be constructed in other ways, such as multiplying label matrices.

[0080] In multi-view learning, it is crucial to achieve feature consistency between different views. Traditional autoencoders mainly focus on representing the features of a single view and cannot effectively include consistent features between different views. The present invention adds a semantic connection module and an attention module to the multi-view feature extraction process to achieve feature consistency, solving the problem that feature extraction in the prior art is too simple and lacks consistency information. In addition, the present invention uses the score matrix and spectral clustering in the attention mechanism to utilize labels, maintain view information consistency, and solve the problem of large influence of private views in the prior art.

Claims

1. A multi-view clustering method for commodity images, characterized in that: The following steps are involved: S1, obtaining multi-view product images; S2, introduce a multi-layer connected attention autoencoder model to learn the vector representation of multi-view product images; S3. Based on the learned multi-layer connected attention autoencoder model, explore the complementary and consistent information of multi-view product images and measure the feature consistency structure of multi-view product images; S4, based on the semantic information of the feature consistency structure, the similarity matrix is ​​obtained through a multi-layer attention perceptron; S5. Cluster the views using the similarity matrix.

2. The multi-view clustering method for commodity images according to claim 1, characterized in that: The step S2 is specifically as follows: For the data sample of the vth view of the i-th sample It is converted into an embedded feature representation through an attention encoder The attention mechanism and semantic connection block are introduced in the attention encoder to extract consistent feature representations between different views to learn the vector representation of multi-view product images.

3. The multi-view clustering method for commodity images according to claim 2, characterized in that: The embedded feature representation The expression is as follows: in, and They represent the encoder function and weight parameters of view v respectively, represents the data sample of the vth view of the i-th sample; The expression of the output of the semantic connection block is as follows: E i' (x)=F(E i'-1 (x),W i )+W i E i'-1 (x) Among them, E i' (x) represents the output of the semantic connection block at the i'th layer, F represents the calculation function of the semantic connection block, and E i'-1 (x) represents the output of the semantic connection block at layer i'-1, W i Represents the computational weight of the semantically connected block.

4. The multi-view clustering method for commodity images according to claim 1, characterized in that: The expression of the reconstruction loss function of the multi-layer connected attention autoencoder model in step S3 is as follows: Among them, L pre represents the reconstruction loss function, i represents the i-th sample, v represents the v-th view, M represents the number of views, N represents the number of samples, represents the data sample of the vth view of the i-th sample, Representation Embedding The reconstruction result obtained by reconstructing the original data sample through the decoder, represents the decoder function, represents the decoder weights.

5. The multi-view clustering method for commodity images according to claim 1, characterized in that: The step S4 is specifically as follows: Based on the semantic information of the feature consistency structure and the feature representation Z of multiple views, a multi-layer attention perceptron and softmax function are stacked on the feature representation Z to obtain the cluster distribution matrix H, where Represent the embedded feature representation; Calculate the attention mechanism channel matrices Q and K on the cluster distribution matrix H to obtain the score matrix S; Consider the score matrix S as a similarity matrix.

6. The multi-view clustering method for commodity images according to claim 1, characterized in that: The expression of the loss function for clustering views in step S5 is as follows: L total =L pre +L fine Among them, L total represents the loss function for clustering views, L pre represents the reconstruction loss function, L fine represents the fine-tuning clustering consistency loss function, m represents the mth view of the sample data, M represents the number of views of the sample data, n represents the nth view, Indicates H m and H n The label contrast loss between m represents the cluster representation of the m-th view, H n represents the cluster representation of the nth view that is different from the mth one, j represents the jth sample compared during calculation, K represents the number of clusters, represents the regular term, v represents the vth view compared during calculation, Represents cluster samples and The cosine similarity of L represents the temperature coefficient, represents the cluster representation of the m-th view of the i-th sample, represents the cluster representation of the nth view of the jth sample, <,> represents the dot product calculation, i represents the i-th sample, Represents regular calculation.

Images