A subspace clustering method for multi-view incomplete images

Abstract

The application provides a multi-view incomplete image subspace clustering method, which comprises the following steps: preparing data in the form of a three-dimensional tensor, unifying the sizes of multiple three-dimensional tensors, and slicing the three-dimensional tensors in the same way to generate data of multiple views, wherein the data of one view is composed of a complete part and an incomplete part; obtaining a self-expression matrix of each view; fusing the self-expression matrices of the views to obtain a total self-expression matrix; filling the incomplete part; and making the self-expression matrices of the views have a low-rank characteristic and approximate a block diagonal structure. The method can effectively restore the missing features of multi-view incomplete three-dimensional tensor image samples and the subspace structure of incomplete multi-view data, and finally realizes unsupervised and effective clustering of the multi-view incomplete image data set.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a subspace clustering method for multi-view incomplete images. Background Technology

[0002] Data described from multiple perspectives is common in fields such as computer vision. Compared to single-view features, multi-view features can usually provide more comprehensive information for exploring the underlying structure of data.

[0003] In practical applications, data may be affected by the loss of certain samples or features from certain perspectives. For example, in disease diagnosis, each test a patient undergoes can be seen as adding a new perspective, but due to some uncontrollable factors, some patients may not undergo all types of tests.

[0004] In multi-view data problems, the incompleteness of the data makes it extremely difficult to explore information from incomplete multi-view data. Furthermore, since different views may contain varying numbers of samples, the amount of information they represent in clustering tasks is unbalanced. Therefore, directly utilizing traditional multi-view clustering methods struggles to capture the clustering structure of incomplete multi-view data. Multi-view clustering, which aims to partition data into their proper clusters in an unsupervised manner, has received extensive research attention in recent years.

[0005] In the medical field, while a wealth of medical information is available for data science research, many incomplete datasets lack the values ​​required by numerous machine learning algorithms. For example, the diversity of information sources used for data collection in modern intensive care units (ICUs) makes the resulting databases particularly prone to data loss. There are two common strategies for managing missing data: estimating and imputing values, and deleting values. The former can lead to bias or loss of information, while the latter can result in bias and loss of statistical power, causing clustering results to be misled by redundant information. Summary of the Invention

[0006] Therefore, it is necessary to provide a subspace clustering method for medical data that can fill in missing data, integrate information from multiple perspectives, and improve clustering results.

[0007] This invention provides a subspace clustering method for multi-view incomplete images, comprising the following steps:

[0008] Step S10: Prepare data in the form of three-dimensional tensors, unify the size of multiple three-dimensional tensors, and slice them in the same way to generate data from multiple perspectives, where the data from one perspective consists of complete and incomplete parts.

[0009] Step S20: Obtain the self-expression matrix from each perspective;

[0010] Step S30: Merge the self-expression matrices from each perspective to obtain the overall self-expression matrix;

[0011] Step S40: Fill in the incomplete parts;

[0012] Step S50, to make the self-expression matrix of each viewpoint have low-rank properties to approximate the block diagonal structure.

[0013] Preferably, in step S10, for handwritten digit images, the data matrices from multiple sources are unified in terms of sample size and feature number, and stacked along the third dimension to form a three-dimensional tensor; for medical images, multiple scans of the images generate multiple three-dimensional tensors.

[0014] Preferably, in step S10, the multiple perspectives generated by the slicing include a first perspective, a second perspective, and a third perspective. D features are extracted from each slice to obtain data. Where d v and n v These represent the number of features and the sample size for each perspective.

[0015] Preferably, the first perspective is a transverse perspective, the second perspective is a coronal perspective, and the third perspective is a sagittal perspective.

[0016] Preferably, the data for the v-th viewpoint (v = 1, 2, 3) is represented as follows: in and All are permutation matrices, containing only 0 and 1 elements, with 1 elements in... and The two numbers represent the complete sample index and the incomplete sample index, respectively.

[0017] Preferably, in step S20, the formula for calculating the self-expression matrix of each viewpoint is:

[0018]

[0019] In the objective function, E (v) =X (v) -X (v) Z (v) Z (v) E represents the self-expression matrix from each perspective. (v) To address the gap between self-expression from various perspectives and real data, ||·|| F Let F be the norm of the matrix. ||·|| * λ is the nuclear norm of the tensor, and λ1 is the balance coefficient.

[0020] Preferably, in step S30, the formula for calculating the total self-expression matrix is:

[0021]

[0022] in, The overall self-expression matrix is ​​obtained by fusing the self-expression matrices from various perspectives, and is initialized. And through iteration Update.

[0023] Preferably, in step S40, the incomplete portion The formula for calculation is:

[0024]

[0025] Where Tr(·) represents the trace of the matrix, L nh It is based on A hyperLaplace matrix for constructing a non-uniform hypergraph, f(L) nh ) is L nh The function, here f(L) nh ) = X (v) L nh X (v)′ .

[0026] Preferably, in step S50, the self-expression matrix Z of each viewpoint is... (v) The formula for approximating the block diagonal structure with low-rank properties is as follows:

[0027]

[0028] in, and For tensors The final formula for the product of tensor singular value decomposition is:

[0029]

[0030] Wherein, λ1, λ2, and λ3 are balance coefficients.

[0031] The present invention provides a subspace clustering method for multi-view incomplete images, comprising the following steps: preparing data in the form of three-dimensional tensors; for handwritten digit images, the sample size and feature number of multi-source data matrices can be unified and stacked along the third dimension to form a three-dimensional tensor; or for medical images, multiple scans of the image can generate multiple three-dimensional tensors; unifying the size of multiple three-dimensional tensors and slicing them in the same way to generate data from multiple perspectives, wherein the data of one perspective consists of complete and incomplete parts; obtaining the self-expression matrix of each perspective; fusing the self-expression matrices of each perspective to obtain the total self-expression matrix; filling in the incomplete parts; and making the self-expression matrix of each perspective have low-rank characteristics to approximate a block diagonal structure. By preparing data, obtaining the self-expression matrix of each viewpoint, fusing them to obtain the overall self-expression matrix, filling in incomplete data, and exploring the low-rank structure of the data, this method considers the high-order correlation between samples and viewpoints. Using mathematical tools such as tensor singular value decomposition, matrix operations, hypergraphs, and norms, it can effectively recover the missing features of incomplete 3D tensor image samples from multiple views and the subspace structure of incomplete multi-view data. Ultimately, it can achieve unsupervised and effective clustering of incomplete multi-view image datasets. Attached Figure Description

[0032] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0033] Figure 1 A flowchart of a subspace clustering method for multi-view incomplete images provided in one embodiment;

[0034] Figure 2 A schematic diagram of the data preparation stage for a subspace clustering method for multi-view incomplete images provided in one embodiment;

[0035] Figure 3 A schematic diagram illustrating the acquisition of the self-expression matrix of each viewpoint in a subspace clustering method for multi-view incomplete images provided in one embodiment;

[0036] Figure 4 A schematic diagram illustrating the fusion of subspace clustering methods for multi-view incomplete images provided in one embodiment to obtain the overall self-expression matrix;

[0037] Figure 5 A schematic diagram illustrating the filling of incomplete data in a subspace clustering method for multi-view incomplete images provided in one embodiment; and

[0038] Figure 6This is a schematic diagram of the low-rank structure of exploratory data for a subspace clustering method for multi-view incomplete images provided in one embodiment. Detailed Implementation

[0039] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] As used herein, the terms “component,” “module,” and “system,” etc., are intended to refer to a computer-related entity that can be hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, executable code, an executing thread, a program, and / or a computer. For illustration, an application running on a server and the server itself can both be components. One or more components may reside in a process and / or an executing thread, and components may be located within a single computer and / or distributed across two or more computers.

[0041] As used herein, the term "inference" or "reasoning" generally refers to the process of inferring or reasoning about the state of a system, environment, and / or user from a set of observations captured via events and / or data. For example, inference can be used to identify a specific context or action, or a probability distribution that can generate states. Inference can be probabilistic, i.e., calculating the probability distribution of the state of interest based on considerations of data and events. Inference can also refer to techniques used to synthesize higher-level events from a set of events and / or data. Such inference results in the construction of new events or actions from a set of observed events and / or stored event data, regardless of whether the events are related in adjacent time periods, or whether the events and data come from one or several event and data sources.

[0042] like Figures 1-6 As shown, this invention provides a subspace clustering method for multi-view incomplete images, comprising the following steps:

[0043] Step S10: Prepare data in the form of a three-dimensional tensor. For handwritten digit images, the data matrix from multiple sources can be unified in terms of sample size and feature number, and stacked along the third dimension to form a three-dimensional tensor. Or for medical images, multiple three-dimensional tensors can be generated by scanning the image multiple times. The multiple three-dimensional tensors can be unified in size and sliced ​​in the same way to generate data from multiple perspectives. The data from one perspective consists of complete and incomplete parts.

[0044] Step S20: Obtain the self-expression matrix from each perspective;

[0045] Step S30: Merge the self-expression matrices from each perspective to obtain the overall self-expression matrix;

[0046] Step S40: Fill in the incomplete parts;

[0047] Step S50, to make the self-expression matrix of each viewpoint have low-rank properties to approximate the block diagonal structure.

[0048] Specifically, in step S10, a handwritten digit image is typically a two-dimensional matrix. As a subsequent step in the implementation method, the sample size and feature count of the multi-source data matrices are unified, and they are stacked along the third dimension to form a three-dimensional tensor. The number of "sources" represents the number of viewpoints. A medical image is typically a three-dimensional tensor; N scans will generate N three-dimensional tensors, where N is the sample size. As a subsequent step in the implementation method, all three-dimensional tensors need to be of uniform size and sliced ​​in the same way to generate multi-view data. This method obtains slice angles commonly used in medicine: transverse (as the first viewpoint), coronal (as the second viewpoint), and sagittal (as the third viewpoint). Then, D features are extracted from each slice, but in the simplest case, each slice image can be directly reconstructed into a column vector.

[0049] Thus, the data was obtained. Where d v and n v Let be the number of features and the sample size for each perspective, respectively. Therefore, for incomplete data from three perspectives, it is obvious that... or like Figure 2 As shown, two incomplete samples are presented, with missing data in viewpoint 1, viewpoint 2, and viewpoint 3, respectively.

[0050] Furthermore, the data for the v-th view (v = 1, 2, 3) consists of two parts: complete (without missing features) and incomplete (without missing features). In another form, it can be expressed as in and All are permutation matrices, containing only 0 and 1 elements, with 1 elements in... and The indices are for complete and incomplete samples, respectively. It's important to note that missing data has not yet been repaired at this point. The existence of this is equivalent to the initial "creation from nothing", and here we take the average of the features of all samples.

[0051] In step S20, the self-expression matrix of each perspective is obtained, which is the Z in the overall process. (v) The subproblem is: The optimization problem to be solved here is:

[0052]

[0053] In the objective function, E (v) =X (v) -X (v) Z (v) Z (v) E represents the self-expression matrix from each perspective. (v) This represents the gap between the self-expression from each perspective and the actual data, ||·|| F Let Frobenius norm (F-norm) be the matrix. This is equivalent to stacking multiple two-dimensional tensors (matrices) along the third dimension to form a three-dimensional tensor, and vice versa. ||·|| * λ is the nuclear norm of the tensor, and λ1 is the balance coefficient.

[0054] In the constraints, diag(Z) (v) The condition ) = 0 requires that the elements on the diagonal are not 0, that is, to avoid singular solutions; Require matrix Z (v) All elements in Z are non-negative, meaning that negative correlations between two samples are not allowed; (v) =Z (v)′ Require matrix Z (v) symmetry.

[0055] In step S30, the self-expression matrices from each perspective are fused to obtain the overall self-expression matrix, which is the matrix in the overall process. The subproblem is: The optimization problem to be solved here is:

[0056]

[0057] in, The overall self-expression matrix is ​​obtained by fusing the self-expression matrices from various perspectives, and is initialized. And through iteration Update.

[0058] In step S40, the incomplete parts of the data are filled in, that is... This is X in the overall process. (v) The subproblem is: The optimization problem to be solved here is:

[0059]

[0060] Where Tr(·) represents the trace of the matrix, L nh It is based on A hyperLaplace matrix for constructing a non-uniform hypergraph, f(L) nh ) is L nh The function, here f(L) nh ) = X (v) L nh X (v)′ .

[0061] In step S50, Z is made (v) It possesses low-rank properties to approximate a block diagonal structure, which is part of the overall process. and The subproblem is: The optimization problem to be solved here is:

[0062]

[0063] This optimization problem is similar to solving the self-expression matrix mentioned earlier, but its application is different. and In fact, it is a tensor The product of tensor singular value decomposition is a solution to restrict the rank of tensors, thereby saving computational resources while taking into account the block diagonal structure of the self-expression matrix.

[0064] Therefore, the overall optimization problem is as follows:

[0065]

[0066] λ1, λ2, and λ3 are balancing coefficients that can be set manually. In some image datasets, they can be enumerated in {0.01, 0.1, 0.5, 1, 2, 5, 10} to select the best combination.

[0067] The present invention provides a subspace clustering method for multi-view incomplete images, comprising the following steps: preparing data in the form of three-dimensional tensors; for handwritten digit images, the sample size and feature number of multi-source data matrices can be unified and stacked along the third dimension to form a three-dimensional tensor; or for medical images, multiple scans of the image can generate multiple three-dimensional tensors; unifying the size of multiple three-dimensional tensors and slicing them in the same way to generate data from multiple perspectives, wherein the data of one perspective consists of complete and incomplete parts; obtaining the self-expression matrix of each perspective; fusing the self-expression matrices of each perspective to obtain the total self-expression matrix; filling in the incomplete parts; and making the self-expression matrix of each perspective have low-rank characteristics to approximate a block diagonal structure. By preparing data, obtaining the self-expression matrix of each viewpoint, fusing them to obtain the overall self-expression matrix, filling in incomplete data, and exploring the low-rank structure of the data, this method considers the high-order correlation between samples and viewpoints. Using mathematical tools such as tensor singular value decomposition, matrix operations, hypergraphs, and norms, it can effectively recover the missing features of incomplete 3D tensor image samples from multiple views and the subspace structure of incomplete multi-view data. Ultimately, it can achieve unsupervised and effective clustering of incomplete multi-view image datasets.

[0068] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A subspace clustering method for multi-view incomplete images, characterized in that, Includes the following steps: Step S10: Prepare data in the form of three-dimensional tensors, unify the size of multiple three-dimensional tensors, and slice them in the same way to generate data from multiple perspectives, where the data from one perspective consists of complete and incomplete parts. Step S20: Obtain the self-expression matrix from each perspective. Step S30: Fuse the self-expression matrices from each perspective to obtain the total self-expression matrix; Step S40: Fill in the incomplete portion; Step S50, to make the self-expression matrix of each viewpoint have low-rank characteristics and approximate a block diagonal structure, In step S10, the multiple perspectives generated by the slice include a first perspective, a second perspective, and a third perspective. D features of each slice are extracted to obtain data. Where d v and n v These represent the number of features and the sample size for each perspective. The data from the v-th viewpoint (v = 1, 2, 3) is represented as follows: in and All are permutation matrices, containing only 0 and 1 elements, with 1 elements in... and The two indices are the complete sample index and the incomplete sample index, respectively. In step S20, the formula for calculating the self-expression matrix of each viewpoint is: In the objective function, E (v) =X (v) -X (v) Z (v) Z (v) E represents the self-expression matrix from each perspective. (v) To address the gap between self-expression from various perspectives and real data, ||·|| F Let F be the norm of the matrix. ||·|| * Let λ be the nuclear norm of the tensor, and λ1 be the balance coefficient. In step S30, the formula for calculating the total self-expression matrix is: in, The overall self-expression matrix is ​​obtained by fusing the self-expression matrices from various perspectives, and is initialized. And through iteration Update.

2. The subspace clustering method for multi-view incomplete images according to claim 1, characterized in that, In step S10, for handwritten digit images, the data matrices from multiple sources are unified in terms of sample size and feature number, and stacked along the third dimension to form a three-dimensional tensor; for medical images, multiple scans of the images generate multiple three-dimensional tensors.

3. The subspace clustering method for multi-view incomplete images according to claim 1, characterized in that, The first perspective is a transverse perspective, the second perspective is a coronal perspective, and the third perspective is a sagittal perspective.

4. The subspace clustering method for multi-view incomplete images according to claim 1, characterized in that, In step S40, the incomplete portion The formula for calculation is: Where Tr(·) represents the trace of the matrix, L nh It is based on A hyperLaplace matrix for constructing a non-uniform hypergraph, f(L) nh ) is L nh The function, here f(L) nh ) = X (v) L nh X (v)' .

5. The subspace clustering method for multi-view incomplete images according to claim 4, characterized in that, In step S50, the self-expression matrix Z of each viewpoint is... (v) The formula for approximating the block diagonal structure with low-rank properties is as follows: in, and For tensors The final formula for the product of tensor singular value decomposition is: Wherein, λ1, λ2, and λ3 are balance coefficients.

Images