A clustering method for incomplete multi-view scene image data based on integration of local and global anchor graphs

Abstract

A method for clustering incomplete multi-view scene image data based on the integration of local and global anchor graphs belongs to the field of scene image data clustering processing in multimedia information processing. First, the data of each view is randomly sampled to obtain anchor sets and non-anchor sets, and then the edge weight matrix is ​​solved by a sparse subspace model to construct a basic local incomplete anchor graph. Then, multiple basic local incomplete anchor graphs are weighted averaged and fused to obtain a local incomplete anchor graph of each view, and then weighted averaged and fused according to a sample missing indicator matrix to obtain a basic global complete anchor graph. Finally, the basic global complete anchor graph is weighted averaged to obtain a basic global complete anchor graph, and cluster analysis is performed using singular value decomposition and K-means clustering to obtain the final clustering result. Compared with other methods, the present invention has higher accuracy and more robust performance.

Description

Technical Field

[0001] The invention belongs to the field of scene image data processing in multimedia information processing, and in particular relates to an incomplete multi-view scene image data clustering method based on the integration of local and global anchor graphs. Background Art

[0002] In the field of multimedia information processing, scene image data is a widely available data. Multi-view analysis of scene image data can mine potential category information such as the characteristics of different scene angles and the similarities and differences between scenes. Among the many multi-view data learning techniques, multi-view clustering is one of the most popular techniques. It can use information from multiple perspectives to perform unsupervised clustering of data and can be applied to a variety of practical applications. However, in the process of collecting multi-view scene image data, information loss or incomplete data sources may occur, resulting in inconsistent availability of each sample at different perspectives, thus causing the problem of incomplete multi-view scene data. Because existing multi-view clustering methods can only handle the clustering problem of complete multi-view scene data, effective novel technical methods are needed to solve the clustering problem of incomplete multi-view scene image data. Summary of the invention

[0003] In order to solve the above problems, the present invention provides an incomplete multi-view scene image data clustering method based on local and global anchor graph integration, characterized in that the method comprises the steps of:

[0004] Extract existence samples of each view from an incomplete multi-view scene image dataset.

[0005] The existence samples of each perspective are randomly sampled to obtain the anchor set, non-anchor set and non-anchor index set, and a basic local incomplete anchor graph is constructed based on the anchor set and non-anchor set.

[0006] The edge weight matrix of the basic local incomplete anchor graph is obtained by solving the sparse subspace model.

[0007] The first two steps are repeated multiple times to obtain multiple basic local incomplete anchor maps, which are then weighted averaged and fused according to the non-anchor index set to obtain a local incomplete anchor map for each perspective.

[0008] According to the sample missing indicator matrix, the local incomplete anchor maps of all perspectives are weighted averaged and fused to obtain the basic global complete anchor map.

[0009] Repeat the first four steps multiple times to obtain multiple basic global complete anchor graphs, and perform weighted averaging to obtain a global complete anchor graph.

[0010] The edge weight matrix of the obtained global complete anchor graph is subjected to singular value decomposition, and the first c components of the left singular vector are selected for K-means clustering (c is the number of clusters) to obtain the final clustering result. Finally, the clustering accuracy of the final clustering result is calculated.

[0011] Further, the existence samples of each view extracted from the incomplete multi-view scene image dataset are defined as:

[0012]

[0013] in, Represents the sample feature matrix of all viewpoints, is the existence feature matrix of view v, n v is the number of samples of view v, d v is the characteristic dimension of view v, and m is the number of viewpoints.

[0014] Furthermore, the non-anchor set and the basic local incomplete anchor graph are defined as follows:

[0015]

[0016] Among them, v represents a perspective, j represents the subscript of the basic local incomplete anchor graph, represents the anchor set, represents the non-anchor set, represents all samples, Represents the basic local incomplete anchor graph.

[0017] Furthermore, the edge weight matrix of the basic local complete anchor graph is expressed as:

[0018]

[0019] in, represents the edge weight matrix of the jth basic local incomplete anchor graph, represents the corresponding non-anchor set feature matrix, Represents the feature matrix of the corresponding anchor set; the constraint condition requires The elements of are greater than or equal to 0, and The sum of the elements in any row of is 1.

[0020] Furthermore, the expression formula of the non-anchor index set and weighted average fusion is:

[0021]

[0022] Among them, v represents a certain perspective, i represents the subscript of a certain sample, represents the non-anchor index set of sample i, Z (v) represents the edge weight matrix of the local incomplete anchor graph, represents the jth basic local incomplete anchor graph, represents all edge weights of sample i in the jth basic local incomplete anchor graph, Representation Set The number of elements of .

[0023] Furthermore, the expression formula of the weighted average fusion is:

[0024]

[0025] Among them, i represents the subscript of the basic global complete anchor map, v represents a certain perspective, and k represents the subscript of a certain sample. represents the set of viewpoints of sample k, M represents the sample missing indicator matrix, M[k,v] represents the element in the kth row and vth column of matrix M, that is, whether the viewpoint v of sample k exists. represents the edge weight matrix of the i-th basic complete anchor graph, represents all edge weights of sample k in the i-th basic complete anchor graph, Representing a collection The number of elements of .

[0026] Furthermore, the weighted average expression formula is:

[0027]

[0028] in, represents the i-th basic global complete anchor graph, Z * represents the globally complete anchor graph.

[0029] Furthermore, according to the final clustering result, the clustering accuracy is calculated.

[0030] The present invention provides an incomplete multi-view scene image data clustering method based on the integration of local and global anchor graphs, which has the following advantages:

[0031] (1) The method adopts a multi-view clustering framework, which fully utilizes the view consistency and difference of the incomplete multi-view scene image dataset and effectively mines the potential cluster structure.

[0032] (2) The method adopts random anchor point selection technology to obtain anchor sets and non-anchor sets, and constructs a basic local incomplete anchor map based on them, which not only reduces the computational complexity and memory consumption, but also effectively alleviates the impact of incomplete multi-view scene image data.

[0033] (3) The method adopts local integration and global integration techniques to establish the similarity relationship between samples by constructing a complete and unified global anchor graph, which alleviates the problem of non-aligned anchor points and improves the robustness of clustering. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 It is a flow chart of a clustering method for incomplete multi-view scene image data based on the integration of local and global anchor graphs provided by the present invention;

[0036] Figure 2 This is a schematic diagram of part of the Scene-15 scene image dataset;

[0037] Figure 3 It is the clustering accuracy of the present invention compared with the existing methods on the Scene-15 dataset with five incomplete sample proportions. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0039] Exemplary Methods

[0040] like Figure 1 The present invention provides a method for clustering an incomplete multi-view scene image dataset based on the integration of local and global anchor graphs, and the method steps are as follows:

[0041] Step S110: First, extract the existing samples of each view from the incomplete multi-view scene image dataset. The existing samples are composed of a set of feature matrices and a mask matrix M∈{0,1} n×m Indicates that n v is the number of samples of view v, d v is the feature dimension of the view v, and m is the number of views. Specifically, if there is a j-th view of the i-th sample, then M[i,j]=1.

[0042] Step S120: obtain anchor sets and non-anchor sets by randomly sampling existing samples and construct a basic local incomplete anchor graph, which is expressed as follows:

[0043]

[0044] in, From all existing samples The anchor set of view v randomly sampled from , is the corresponding non-anchor set. It is the basic local incomplete anchor graph between the anchor set and the non-anchor set.

[0045] Step S130: Obtain the edge weight matrix of the local incomplete anchor graph by solving the sparse subspace model, and its expression formula is as follows:

[0046]

[0047] in, represents the edge weight matrix of the jth basic local incomplete anchor graph, represents the corresponding non-anchor set feature matrix, represents the corresponding anchor set feature matrix, and its constraints require The elements of are greater than or equal to 0, and The sum of the elements in any row of is 1.

[0048] Step S140: Repeat steps S120-S130 multiple times to obtain t 2 A basic local incomplete anchor graph and a corresponding non-anchor index set. According to the non-anchor index set, t 2 The weighted average fusion of the basic local incomplete anchor graphs is performed to obtain the local incomplete anchor graph, and its expression formula is as follows.

[0049]

[0050] in, represents the non-anchor index set of sample i under view v, Z (v) represents the edge weight matrix of the local incomplete anchor graph, represents the jth basic local incomplete anchor graph, represents all edge weights of sample i in the jth basic local incomplete anchor graph, Representing a collection The number of elements of .

[0051] Step S150: According to the sample missing indicator matrix, the local incomplete anchor maps of all perspectives are weighted averaged and fused to obtain a basic global complete anchor map, which is expressed as follows.

[0052]

[0053] Among them, i represents the subscript of the basic global complete anchor map, v represents a certain perspective, and k represents the subscript of a certain sample. represents the set of viewpoints of sample k, M represents the sample missing indicator matrix, M[k,v] represents the element in the kth row and vth column of matrix M, that is, whether the viewpoint v of sample k exists. represents the edge weight matrix of the i-th basic complete anchor graph, represents all edge weights of sample k in the i-th basic complete anchor graph, Representing a collection The number of elements of .

[0054] Step S160: Repeat steps S120-S150 multiple times to obtain t 1 The basic global complete anchor graphs are weighted averaged to obtain the global complete anchor graph, which is expressed as follows.

[0055]

[0056] in, represents the i-th basic global complete anchor graph, Z * represents the globally complete anchor graph.

[0057] Step S170: Edge weight matrix Z of the global complete anchor graph * Perform singular value decomposition and select the first c components of the left singular vector for K-means clustering (c is the number of clusters). After obtaining the final clustering result, calculate its clustering accuracy.

[0058] Through this implementation, first, the data of each perspective is randomly sampled to obtain an anchor set and a non-anchor set, and then the edge weight matrix of the basic local incomplete anchor graph is solved by the sparse subspace model to obtain the basic local incomplete anchor graph of each perspective. Secondly, multiple basic local incomplete anchor graphs are weighted averaged and fused to obtain a local incomplete anchor graph of each perspective, and then the local incomplete anchor graphs of all perspectives are weighted averaged and fused according to the sample missing indicator matrix to obtain a basic global complete anchor graph. Finally, multiple basic global complete anchor graphs are weighted averaged to obtain a global complete anchor graph, and then singular value decomposition and K-means clustering are used to perform cluster analysis to obtain the final clustering result, and its clustering accuracy is calculated.

[0059] To further illustrate, assuming that an incomplete multi-view scene image dataset is clustered according to this embodiment, a clustering result with a higher clustering accuracy than most methods will be obtained.

[0060] Detailed description of the invention Results

[0061] This implementation uses the publicly available Scene-15 scene image dataset and simulates an incomplete Scene-15 scene image dataset according to different sample missing ratios. The details of the dataset are described as follows:

[0062] The Scene-15 scene image dataset is a grayscale image dataset with 15 indoor and outdoor scene categories. The dataset has the following two perspectives:

[0063] View 1 is the PHOG view. The characteristic of this view is that it calculates features at different scales when the image size is fixed.

[0064] View 2 is the GIST view. The characteristic of this view is that it uses Gabor filtering to perform convolution at m scales and n directions to obtain mn features to describe the image.

[0065] In order to verify the superiority of this implementation (OURS), this implementation is compared with several existing methods for clustering Scene-15 scene image data, including ANIMC, BMVC, DAIMC, GFSC and other methods. The clustering accuracy (ACC) of these methods for five incomplete sample ratios on the above public datasets will be compared. The specific data comparison is shown in Table 1.

[0066] Table 1 Clustering accuracy of Scene-15 dataset (%)

[0067]

[0068]

[0069] By comparing the data in the above table, it can be clearly seen that OURS achieves the best performance and significantly improves the clustering accuracy of the incomplete multi-view scene image dataset. The quantitative results fully demonstrate the superiority of OURS, because OURS can better capture the perspective consistency information and perspective difference information in the incomplete multi-view scene image dataset. OURS constructs local anchor maps and global anchor maps by randomly sampling anchor sets, thereby improving the robustness to scene image data. A large number of experiments have shown that this method is superior to existing methods. Regarding the setting of the parameters of this implementation, in all experiments, the number of anchor points n a Set to 5c, c is the number of clusters, and the number of basic global complete anchor graphs is t 1 Set to 10, the number of basic local complete anchor graphs t 2 Set to 20.

[0070] This embodiment proposes a clustering method for incomplete multi-view scene image data sets based on the integration of local and global anchor graphs, which is used to perform cluster analysis on incomplete scene image data commonly seen in life. Anchor sets and non-anchor sets are selected by random sampling of data, and the data of each view is randomly sampled and the sparse subspace model is solved. Multiple basic local incomplete anchor graphs are fused to obtain a global complete anchor graph, and then singular value decomposition and K-means clustering are used to perform cluster analysis to obtain the final clustering result, and its accuracy is calculated. The experimental results on five incomplete sample ratios of the public dataset Scene-15 show that this embodiment has a higher clustering accuracy and better superiority than other methods.

[0071] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.

Claims

1. A clustering method for incomplete multi-view scene image data based on integration of local and global anchor graphs, characterized in that: The method comprises the steps of: S110. Extracting existing samples of each perspective from an incomplete multi-view scene image dataset; S120. randomly sampling the existing samples of each perspective to obtain an anchor set, a non-anchor set, and a non-anchor index set, and constructing a basic local incomplete anchor graph based on the anchor set and the non-anchor set; S130. Obtaining the edge weight matrix of the basic local incomplete anchor graph by solving the sparse subspace model; S140. Repeat steps S120 to S130 multiple times to obtain multiple basic local incomplete anchor maps, and fuse them by weighted average according to the non-anchor index set to obtain a local incomplete anchor map for each perspective; S150. According to the sample missing indicator matrix, the local incomplete anchor maps of all perspectives are weighted averaged and fused to obtain a basic global complete anchor map; S160. Repeat steps S120 to S150 multiple times to obtain multiple basic global complete anchor graphs, and perform weighted averaging to obtain a global complete anchor graph; S170. Perform singular value decomposition on the edge weight matrix of the obtained global complete anchor graph, and select the first c components of the left singular vector for K-means clustering, where c is the number of clusters, to obtain the final clustering result, and finally, calculate the clustering accuracy of the final clustering result; In S120, an anchor set and a non-anchor set are obtained by randomly sampling existing samples of each perspective, and a basic local incomplete anchor map is constructed between the anchor set and the non-anchor set. The anchor set, the non-anchor set, and the basic local incomplete anchor map are defined as follows: Among them, v represents a certain perspective, j represents the subscript of the basic local incomplete anchor map, represents the anchor set, represents the non-anchor set, Indicates all existing samples, Represents the basic local incomplete anchor graph; In S130, the edge weight matrix of the basic local incomplete anchor graph is obtained by solving the sparse subspace model, and its expression formula is as follows: in, represents the edge weight matrix of the jth basic local incomplete anchor graph, represents the corresponding non-anchor set feature matrix, represents the corresponding anchor set feature matrix, and the constraint condition requires The elements of are greater than or equal to 0, and The sum of the elements in any row of is 1; In the S140, steps S120 to S130 are repeated multiple times to obtain t2 basic local incomplete anchor graphs and corresponding non-anchor index sets, and weighted average fusion is performed on the t2 basic local incomplete anchor graphs according to the non-anchor index sets to obtain a local incomplete anchor graph. The definitions of the non-anchor index set and the weighted average fusion are as follows: Among them, v represents a certain perspective, l represents a sample in the local incomplete anchor map, represents the non-anchor index set of sample l, Z (v) represents the edge weight matrix of the local incomplete anchor graph, represents the jth basic local incomplete anchor graph, represents all edge weights of sample l in the jth basic local incomplete anchor graph, Representing a collection The number of elements, n v is the number of samples of view v; In S150, according to the sample missing indicator matrix, the local incomplete anchor maps of all perspectives are weighted average fused to obtain the basic global complete anchor map. The formula of weighted average fusion is as follows: Among them, i represents the subscript of the basic global complete anchor map, v represents a certain perspective, and k represents a sample in the global complete anchor map. represents the set of viewpoints of sample k, M represents the sample missing indicator matrix, M[k,v] represents the element in the kth row and vth column of matrix M, that is, whether the viewpoint v of sample k exists. represents the edge weight matrix of the i-th basic complete anchor graph, represents all edge weights of sample k in the i-th basic complete anchor graph, Representing a collection The number of elements of ; In S160, steps S120 to S150 are repeated multiple times to obtain t1 basic global complete anchor graphs, and these basic global complete anchor graphs are weighted averaged to obtain a global complete anchor graph. The weighted average formula is as follows: in, represents the i-th basic global complete anchor graph, Z * represents the globally complete anchor graph.

2. The incomplete multi-view scene image data clustering method based on local and global anchor graph integration according to claim 1, characterized in that: Extract the existence samples of each view from the incomplete multi-view Scene image dataset: in, Represents the sample feature matrix of all viewpoints, is the existence feature matrix of view v, n v is the number of samples of view v, d v is the characteristic dimension of view v, and m is the number of viewpoints.

Images