Incomplete multi-view contrast clustering method and device

Abstract

The present invention discloses an incomplete multi-perspective comparative clustering method and device, which belongs to the field of multimedia technology. The method includes: extracting multi-perspective data features and establishing a reconstruction loss function; constructing a feature fusion model to obtain a consistent representation across perspectives; constructing a feature prediction model to predict the consistent representation through the multi-perspective features and establish a prediction loss function; constructing a linear projection model to establish a comparative loss function for learning low-dimensional feature space and clustering space; establishing an objective function for incomplete multi-perspective clustering based on the reconstruction loss function, prediction loss function and comparative loss function, and training a model; inferring the consistent representation of incomplete multi-perspective data across perspectives based on the feature prediction model, obtaining cluster labels, and determining clustering results. The consistency information recovery strategy provided by this method infers the consistent representation across perspectives based on single-perspective features, thereby realizing multi-perspective clustering in the case of missing perspectives.

Description

Technical Field

[0001] The present invention relates to the field of multimedia technology, and in particular to an incomplete multi-view contrast clustering method and device. Background Art

[0002] Cluster analysis is a key technology in unsupervised learning. It explores the inherent structural relationships within data to partition data instances into clusters with similar characteristics or behaviors, while ensuring that clusters are distinct from one another. Cluster analysis can help uncover patterns, regularities, and structures in data, providing a foundation and guidance for applications such as data analysis, anomaly detection, and recommendation systems.

[0003] With the rapid development of network communications and multimedia technologies, data from multiple sources, in multiple forms, from multiple perspectives, or even multiple modalities has become widespread. This data, encompassing multiple perspectives, is called multi-perspective data. Each perspective corresponds to a different set of data features or representations, providing more comprehensive information from different angles. Multi-perspective clustering methods overcome the limitations of a single perspective by leveraging consistent and complementary information across perspectives, achieving more accurate clustering. In recent years, multi-perspective clustering, as an important unsupervised learning method and data mining technique, has garnered widespread attention in numerous fields.

[0004] In multi-view cluster analysis, learning discriminative, cross-view consistent representations is crucial for achieving superior performance. Contrastive learning, a key paradigm in self-supervised learning, has demonstrated promising performance in representation learning. The goal of contrastive learning is to maximize the similarity between positive samples while minimizing the similarity between negative samples, enabling the model to learn more discriminative representations. This representation, during learning, preserves structural features across different viewpoints as much as possible, enabling the learned features to better describe the multifaceted nature of the data. Contrastive learning has achieved remarkable success in cluster analysis. This further highlights the powerful ability of contrastive learning in discovering data features and patterns, as well as its potential value in multi-view data clustering.

[0005] Previous research on multi-view clustering methods is mostly based on a common assumption that all perspectives of each data instance in multi-view data are complete. However, due to the complexity of the data acquisition and transmission process, multi-view data collected in the real world often has missing perspectives. The missing perspective will cause serious information loss, seriously destroy the natural alignment characteristics between different perspectives of multi-view data, and aggravate the information imbalance problem between different perspectives, which in turn leads to a significant decline in the performance of traditional multi-view clustering methods. Therefore, in recent years, researchers have designed a series of incomplete multi-view clustering methods for incomplete multi-view data by taking advantage of the inherent correlation between known partially complete data and perspectives, which to a certain extent overcome the problems caused by missing perspectives.

[0006] Although existing incomplete multi-view clustering methods have greatly improved the performance and representation of multi-view clustering methods in the absence of view, these methods still have some defects:

[0007] 1) Some methods adopt a direct connection strategy to obtain consistent representation across views, ignoring the structural features between different views, which may lead to potential information loss;

[0008] 2) Some methods only utilize partial alignment information in incomplete multi-view data to learn consensus representations, while ignoring the information that may be contained in the missing data;

[0009] 3) Some methods use the strategy of missing view recovery to solve the incomplete multi-view problem. The performance of clustering methods is highly dependent on the quality of the recovered data. Summary of the Invention

[0010] In order to at least solve one of the technical problems existing in the prior art to a certain extent, the object of the present invention is to provide an incomplete multi-view comparative clustering method and device based on consistency inference.

[0011] The technical solution adopted in the present invention is:

[0012] An incomplete multi-view comparative clustering method includes the following steps:

[0013] Perform feature extraction on the multi-view dataset, obtain multi-view features, and establish a reconstruction loss function for the first feature subspace learning;

[0014] Constructing a feature fusion model to obtain a consistent representation of the multi-view data sample across views based on the multi-view features and the feature fusion model;

[0015] Constructing a feature prediction model, predicting the cross-view consistency representation of the multi-view data sample based on the multi-view features and the feature prediction model, and establishing a prediction loss function for first feature subspace learning;

[0016] Constructing a linear projection model, obtaining multi-view low-dimensional features and cross-view low-dimensional consistent representations, as well as multi-view cluster assignments and cross-view consistent cluster assignments based on the linear projection model, the multi-view features, and the consistent representation, and establishing a contrast loss function;

[0017] Establishing an objective function for incomplete multi-view clustering based on the reconstruction loss function, the prediction loss function, and the contrast loss function, and training a model based on the objective function and the complete partial data in the incomplete multi-view data;

[0018] Based on the feature prediction model, a consistent representation of the incomplete multi-view data across perspectives is inferred; based on the consistent representation of the incomplete multi-view data across perspectives and a cluster space linear projection model, cluster labels are obtained and clustering results are determined.

[0019] Furthermore, the prediction loss function of the first feature subspace learning is expressed as:

[0020]

[0021] Where V is the number of views of the multi-view data, Z represents the consistency of the multi-view data across views, and Z v is the feature matrix of the vth perspective of the multi-view data, P v represents the feature prediction model corresponding to the v-th view, and ‖·‖2 represents the Euclidean norm.

[0022] Furthermore, the construction of the linear projection model includes:

[0023] Constructing a low-dimensional space linear projection model, and obtaining multi-view low-dimensional features and cross-view low-dimensional consistent representations based on the low-dimensional space linear projection model, the multi-view features, and the cross-view consistent representations of the multi-view data samples;

[0024] A cluster space linear projection model is constructed, and based on the cluster space linear projection model, the multi-view features and the cross-view consistency representation of the multi-view data samples, multi-view cluster allocation and cross-view consistency cluster allocation are obtained.

[0025] Furthermore, establishing the contrast loss function includes:

[0026] Based on the multi-view low-dimensional features and the cross-view low-dimensional consistency representation, a contrast loss function for learning the second feature subspace is established;

[0027] Based on multi-view cluster assignment and cross-view consistent cluster assignment, a contrastive loss function for learning the third feature subspace is established.

[0028] Furthermore, the contrast loss function of the second feature subspace learning includes:

[0029] Contrastive loss between low-dimensional features from different perspectives in the low-dimensional feature space; and,

[0030] A contrastive loss between multi-view low-dimensional features and low-dimensional consistent representations across views in a low-dimensional feature space.

[0031] Furthermore, the contrast loss function of the third feature subspace learning includes:

[0032] Contrastive loss between cluster features at different views in the cluster space; and,

[0033] Contrastive loss between multi-view clustering features and cross-view consistent clustering features in clustering space.

[0034] Furthermore, the objective function is expressed as:

[0035]

[0036] Where, is the reconstruction loss function within the viewing angle in the first feature subspace, is the consistency representation prediction loss function in the first feature subspace, It is the cumulative function of the contrast loss of the second feature subspace learning and the contrast loss of the third feature subspace learning, and λ1 and λ2 are parameters used to balance the losses.

[0037] Furthermore, the inferring of a consistent representation of incomplete multi-view data across views includes:

[0038] For the complete sample data in the incomplete multi-view dataset, feature extraction is performed through the feature extraction model to obtain the corresponding multi-view features, and then the multi-view features are fused through the feature fusion model to obtain a consistent representation;

[0039] For incomplete sample data in an incomplete multi-view dataset, feature extraction is performed through a feature extraction model to obtain partial view features, which are then input into the corresponding consistent representation prediction model to infer consistent representations across views.

[0040] The consistent representation across views of the complete partial data and the consistent representation across views inferred from the incomplete partial data are combined to obtain a consistent representation of the incomplete multi-view dataset as a whole.

[0041] Furthermore, the method of obtaining cluster labels and determining clustering results based on the cross-view consistency representation of incomplete multi-view data and the cluster space linear projection model includes:

[0042] The consistent representation of incomplete multi-view data across perspectives is input into the cluster space linear projection model to obtain the consistent clustering assignment matrix of the incomplete multi-view data set. Each row of the matrix represents the probability distribution of the corresponding instance in each cluster. Through this probability distribution, each sample is divided into the corresponding cluster to obtain the clustering result.

[0043] The second technical solution adopted by the present invention is:

[0044] An incomplete multi-view comparative clustering system, comprising:

[0045] The feature extraction module is used to obtain multi-view features of multi-view data samples and obtain consistent representations across views;

[0046] a consistency inference module for predicting the obtained consistent representation based on the multi-view features; in the case of missing views, it is used to infer the consistent representation across views using partially complete view features;

[0047] The contrastive learning module is used to perform contrastive learning of low-dimensional features and cluster features in the low-dimensional feature space and cluster space respectively, aligning the low-dimensional feature representation of the same sample data under different perspectives with the feature representation of the same cluster;

[0048] The clustering module is used to obtain the cluster probability distribution of multi-view sample data through cross-view consistency representation and determine the clustering results.

[0049] The third technical solution adopted by the present invention is:

[0050] An incomplete multi-view contrast clustering device, comprising:

[0051] at least one processor;

[0052] at least one memory for storing at least one program;

[0053] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0054] The fourth technical solution adopted by the present invention is:

[0055] A computer-readable storage medium stores a program executable by a processor, wherein the program executable by the processor is used to perform the method described above when executed by the processor.

[0056] The present invention has the following advantages and effects compared to the prior art:

[0057] (1) The incomplete multi-view contrastive clustering method disclosed in the present invention performs representation learning based on a contrastive learning method and integrates multi-view features using a contrastive fusion strategy. This method can preserve the structural features of multi-view data and effectively capture the consistency information between multi-view data.

[0058] (2) The incomplete multi-view comparative clustering method disclosed in this invention proposes a new consistency information recovery strategy, which predicts the cross-view consistency representation based on the single-view feature, solves the information loss problem caused by the missing view, and avoids the adverse effects of low-quality completion.

[0059] (3) The present invention discloses an incomplete multi-view comparative clustering method, which integrates the two processes of consistent representation learning and clustering of multi-view data into a unified framework, avoiding the local optimal solution that may be caused by multi-stage tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0061] Figure 1 1 is a flow chart of an incomplete multi-view comparative clustering method based on consistency inference in an embodiment of the present invention;

[0062] Figure 2 3 is a structural diagram of an incomplete multi-view comparative clustering system based on consistency inference in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention. The step numbers in the following embodiments are provided for ease of explanation only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0064] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on the present invention.

[0065] In the description of the present invention, the meaning of "several" is one or more, the meaning of "many" is two or more, and "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of the first and the second, it is only for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, "and / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship.

[0066] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0067] like Figure 1 As shown, this embodiment provides an incomplete multi-view comparative clustering method based on consistency inference, including the following steps:

[0068] S1. Perform feature extraction on the multi-view dataset, obtain multi-view features, and establish a reconstruction loss function for the first feature subspace learning.

[0069] Specifically, the complete part of the multi-view dataset Input them into the feature extraction model corresponding to multiple views respectively to obtain multi-view features:

[0070]

[0071] Among them, the complete part of the multi-view original features Including M data samples from V perspectives, Indicates that the dimension of the i-th sample at the v-th viewing angle is d v The original characteristics, E v represents the feature extraction encoder corresponding to the v-th view, It represents the feature representation of the i-th sample at the v-th perspective.

[0072] It is worth mentioning that multi-view data can include various types of information such as images, text, audio, etc. In image data, multi-view data can include various visual information such as shape, texture, color, etc.

[0073] In some embodiments, the method of this embodiment uses an autoencoder as a feature extraction model. To explore the potential structure and high-level features in the original data features and avoid model collapse, this method trains the autoencoder using a reconstruction loss to learn the data features. The reconstruction loss is defined as follows:

[0074]

[0075] Among them, D v represents the feature extraction decoder corresponding to the v-th view, and ‖·‖2 represents the Euclidean norm.

[0076] S2. Construct a feature fusion model, and obtain a consistent representation of the multi-view data sample across perspectives based on the multi-view features and the feature fusion model.

[0077] Specifically, the concatenation vector of multi-view features Input into the feature fusion model to obtain a consistent representation across perspectives:

[0078]

[0079] Among them, cat(·) is the connection function, which connects the feature vectors of multiple perspectives into one vector as the input of the feature fusion network, and f(·) is the feature fusion network responsible for connecting the connected vectors Mapped into a unified consistent representation Z.

[0080] S3. Construct a feature prediction model, predict the consistency representation based on the multi-view features and the feature prediction model, and establish a prediction loss function for first feature subspace learning.

[0081] Specifically, the multi-view features Input them into the feature prediction model corresponding to multiple views respectively to obtain the predicted value of the consistent representation:

[0082]

[0083] in, is the consistent representation obtained by predicting the features of the vth view of multi-view data, P v Denotes the feature prediction model corresponding to the vth view. The feature prediction model is trained by optimizing the following prediction loss to achieve inference of consistent representations in the presence of missing views:

[0084]

[0085] S4. Construct a linear projection model, and based on the linear projection model, the multi-perspective features and the consistency representation, obtain multi-perspective low-dimensional features and cross-perspective low-dimensional consistency representation, as well as multi-perspective clustering assignment and cross-perspective consistency clustering assignment, and establish a contrast loss function.

[0086] Specifically, a low-dimensional space linear projection model is constructed, and the multi-view features and the consistency representation are input into the low-dimensional space linear projection to obtain multi-view low-dimensional features and cross-view low-dimensional consistency representation:

[0087]

[0088] H=g I (H)#(7)

[0089] Among them, g I (·) is a linear projection into the low-dimensional space, which is responsible for projecting the eigenvectors in the first eigensubspace into the low-dimensional space. is the low-dimensional feature of the i-th sample at the v-th perspective, and H is the low-dimensional consistency representation across perspectives.

[0090] Specifically, a cluster space linear projection model is constructed, and the multi-view features and the consistency representation are input into the cluster space linear projection to obtain multi-view cluster allocation and cross-view consistency cluster allocation:

[0091]

[0092] C=g C (Z)#(9)

[0093] Among them, g C (·) is the linear projection of the cluster space, which is responsible for projecting the eigenvectors in the first eigensubspace into the cluster space. is the probability distribution of the i-th sample in each cluster under the v-th perspective, and C is the consistent clustering probability distribution across perspectives.

[0094] In order to better capture the consistency information between different perspectives of multi-perspective data while better preserving the structural features of multi-perspective data, this method further adjusts the parameters of the feature extraction model, the feature fusion model and the linear projection through contrast loss.

[0095] Specifically, for the multi-view low-dimensional feature set Any low-dimensional feature in (NV-1) sample pairs can be constructed in, The (V-1) positive sample pairs for contrastive learning represent the characteristics of the same data sample from different perspectives. The remaining V(M-1) sample pairs are negative sample pairs for contrastive learning, representing the characteristics of different data samples. The goal of contrastive learning is to maximize the similarity between positive samples while minimizing the similarity between negative samples. This method uses the cosine similarity as shown below to measure the similarity between sample pairs:

[0096]

[0097] On this basis, the method of this embodiment uses the following contrast loss to optimize the similarity between sample pair features:

[0098]

[0099] in, The low-dimensional feature H representing the viewpoint u u and the low-dimensional features H of the view w w The contrast loss between I Represents the temperature coefficient. The contrast loss is calculated for all viewpoints of the multi-view data, and the overall contrast loss of different viewpoint features in the low-dimensional feature space is obtained:

[0100]

[0101] In addition, the contrast loss between the multi-view low-dimensional features and the low-dimensional consistent representation across views is defined as follows:

[0102]

[0103] in, Low-dimensional feature H representing the view v v The contrast loss between the low-dimensional features of the view and the low-dimensional consistency representation H. The overall contrast loss between the low-dimensional features of the view and the low-dimensional consistency representation across views is:

[0104]

[0105] Then the overall contrast loss in the low-dimensional feature space is:

[0106]

[0107] In the cluster space In, C v The i-th column It can be regarded as the feature representation of the i-th cluster. The following contrast loss is defined to optimize the similarity between cluster features:

[0108]

[0109] in, represents the cluster assignment C of view u u and the cluster assignment C of the view w w The contrast loss between C Represents the temperature coefficient. The contrast loss is calculated for all viewpoints of the multi-view data, and the overall contrast loss of the cluster features under different viewpoints in the cluster space is obtained:

[0110]

[0111] in, is the cluster assignment probability The entropy of is used to avoid the situation where most instances are assigned to the same cluster.

[0112] Similarly, the contrast loss of cluster features under multiple views and the consistent cluster features across views and the overall contrast loss can be defined as:

[0113]

[0114]

[0115] Then the overall contrast loss in the cluster space is:

[0116]

[0117] Finally, the overall contrast loss consists of two parts: the contrast loss in the low-dimensional feature space and the contrast loss in the cluster feature space:

[0118]

[0119] S5. Establish an objective function for incomplete multi-view clustering based on the reconstruction loss function, the prediction loss function, and the contrast loss function, and train a model based on the objective function and the complete partial data in the incomplete multi-view data.

[0120] Specifically, the intra-view reconstruction loss in the first feature subspace, the consistency representation prediction loss, and the contrast loss in the low-dimensional feature space and clustering space are put into a unified objective function:

[0121]

[0122] Wherein, λ1 and λ2 are parameters used to balance various losses. By minimizing the objective function, the feature extraction model, the feature fusion model, the feature prediction model, and the linear projection are jointly trained, and the model is optimized using the gradient descent method.

[0123] S6. Based on the feature prediction model, infer the consistent representation of the incomplete multi-view data across perspectives, obtain cluster labels based on the consistent representation and the cluster space linear projection model, and determine the clustering results.

[0124] Specifically, for the complete partial sample data in the incomplete multi-view data set, feature extraction is performed through the feature extraction model to obtain corresponding multi-view features; and then the obtained multi-view features are fused through the feature fusion model to obtain a consistent representation.

[0125] Specifically, for some sample data with missing viewpoints in the incomplete multi-view dataset, feature extraction is performed using the feature extraction model to obtain partial viewpoint features {Z v}, where v represents the complete perspective in the sample with missing perspective; then Z v The data is input into the corresponding consistent representation prediction model to infer the consistent representation across perspectives of sample data with missing perspectives.

[0126] The cross-view consistency representation of the complete partial data and the cross-view consistency representation inferred from the incomplete partial data are combined to obtain a consistent representation of the incomplete multi-view dataset as a whole. This consistent representation is input into the linear projection of the cluster space to obtain a consistent cluster assignment matrix for the incomplete multi-view dataset. Each row of the matrix represents the probability distribution of the corresponding instance in each cluster. Using this probability distribution, each example is assigned to the corresponding cluster, resulting in a clustering result.

[0127] like Figure 2 As shown, this embodiment also provides an incomplete multi-view contrast clustering system based on consistency inference, which can implement the above steps. The system includes:

[0128] The feature extraction module is used to obtain multi-view features of multi-view data samples and obtain consistent representations across viewpoints.

[0129] The consistency inference module is used to predict the obtained consistent representation using the obtained multi-view features. In the case of missing views, it is used to infer the cross-view consistent representation using partially complete view features.

[0130] The contrastive learning module is used to perform contrastive learning of low-dimensional features and cluster features in the low-dimensional feature space and cluster space respectively, and align the low-dimensional feature representation of the same sample data and the feature representation of the same cluster under different perspectives.

[0131] The clustering module is used to obtain the cluster probability distribution of multi-view sample data through cross-view consistency representation and determine the clustering results.

[0132] Specifically, for the cross-view consistency clustering assignment C, the semantic label y corresponding to the i-th sample is determined by the following formula: i :

[0133] y i =argmax(C i )#(twenty three)

[0134] The incomplete multi-view comparative clustering system based on consistency inference provided in the above embodiment can implement the above incomplete multi-view comparative clustering method based on consistency inference and has corresponding beneficial effects, which will not be described in detail here.

[0135] This embodiment further provides an incomplete multi-view contrast clustering device, comprising:

[0136] at least one processor;

[0137] at least one memory for storing at least one program;

[0138] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 1 The method shown.

[0139] An incomplete multi-perspective comparative clustering device of this embodiment can execute an incomplete multi-perspective comparative clustering method provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0140] The present application also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 1 The method shown.

[0141] This embodiment also provides a storage medium that stores instructions or programs that can execute an incomplete multi-perspective comparative clustering method provided by an embodiment of the method of the present invention. When the instructions or program are run, any combination of implementation steps of the method embodiment can be executed, and the corresponding functions and beneficial effects of the method can be obtained.

[0142] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0143] Furthermore, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise indicated, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It will also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the ordinary skill of an engineer. Therefore, a person skilled in the art using ordinary skill will be able to implement the present invention set forth in the claims without undue experimentation. It will also be understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0144] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0145] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0146] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0147] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0148] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0149] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0150] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. An incomplete multi-view contrast clustering method, characterized in that: The following steps are involved: Performing feature extraction on a multi-view data set to obtain multi-view features and establish a reconstruction loss function for first feature subspace learning; the multi-view data includes images, text, and audio; Constructing a feature fusion model to obtain a consistent representation of the multi-view data sample across views based on the multi-view features and the feature fusion model; Constructing a feature prediction model, predicting the cross-view consistency representation of the multi-view data sample based on the multi-view features and the feature prediction model, and establishing a prediction loss function for first feature subspace learning; Constructing a linear projection model, obtaining multi-view low-dimensional features and cross-view low-dimensional consistent representations, as well as multi-view cluster assignments and cross-view consistent cluster assignments based on the linear projection model, the multi-view features, and the consistent representation, and establishing a contrast loss function; Establishing an objective function for incomplete multi-view clustering based on the reconstruction loss function, the prediction loss function, and the contrast loss function, and training a model based on the objective function and the complete partial data in the incomplete multi-view data; Based on the feature prediction model, inferring the consistent representation of the incomplete multi-view data across views, obtaining cluster labels based on the consistent representation of the incomplete multi-view data across views and the cluster space linear projection model, and determining the clustering results; The linear projection model is constructed, comprising: Constructing a low-dimensional space linear projection model, and obtaining multi-view low-dimensional features and cross-view low-dimensional consistent representations based on the low-dimensional space linear projection model, the multi-view features, and the cross-view consistent representations of the multi-view data samples; Constructing a cluster space linear projection model, and obtaining multi-view cluster assignments and cross-view consistent cluster assignments based on the cluster space linear projection model, the multi-view features, and the cross-view consistency representation of the multi-view data samples; The inferring of a consistent representation of incomplete multi-view data across views includes: For the complete sample data in the incomplete multi-view dataset, feature extraction is performed through the feature extraction model to obtain the corresponding multi-view features, and then the multi-view features obtained are fused through the feature fusion model to obtain a consistent representation; For incomplete sample data in an incomplete multi-view dataset, feature extraction is performed through a feature extraction model to obtain partial view features, which are then input into the corresponding consistent representation prediction model to infer consistent representations across views. The consistent representation across views of the complete partial data and the consistent representation across views inferred from the incomplete partial data are combined to obtain a consistent representation of the incomplete multi-view dataset as a whole.

2. The incomplete multi-view contrast clustering method according to claim 1, characterized in that: The expression of the prediction loss function of the first feature subspace learning is: Where, is the number of perspectives for multi-view data, Represents the consistent representation of multi-view data across views, For multi-view data The feature matrix of each perspective, Indicates the corresponding Feature prediction model from different perspectives, represents the Euclidean norm.

3. The incomplete multi-view contrast clustering method according to claim 1, characterized in that: The establishing of the contrast loss function includes: Based on the multi-view low-dimensional features and the cross-view low-dimensional consistency representation, a contrast loss function for learning the second feature subspace is established; Based on multi-view cluster assignment and cross-view consistent cluster assignment, a contrastive loss function for learning the third feature subspace is established.

4. The incomplete multi-view contrast clustering method according to claim 3, characterized in that: The contrast loss function of the second feature subspace learning includes: Contrastive loss between low-dimensional features from different perspectives in the low-dimensional feature space; and, A contrastive loss between multi-view low-dimensional features and low-dimensional consistent representations across views in a low-dimensional feature space.

5. The incomplete multi-view contrast clustering method according to claim 3, characterized in that: The contrast loss function of the third feature subspace learning includes: Contrastive loss between cluster features at different views in the cluster space; and, Contrastive loss between multi-view clustering features and cross-view consistent clustering features in clustering space.

6. The incomplete multi-view contrast clustering method according to claim 3, characterized in that: The expression of the objective function is: Where, is the reconstruction loss function, is the prediction loss function, It is the cumulative function of the contrast loss of the second feature subspace learning and the contrast loss of the third feature subspace learning, and is a parameter used to balance various losses.

7. The incomplete multi-view contrast clustering method according to claim 1, characterized in that: The method of obtaining cluster labels and determining cluster results based on the cross-view consistency representation of incomplete multi-view data and the cluster space linear projection model includes: The consistent representation of incomplete multi-view data across perspectives is input into the cluster space linear projection model to obtain the consistent clustering assignment matrix of the incomplete multi-view data set. Each row of the matrix represents the probability distribution of the corresponding instance in each cluster. Through this probability distribution, each sample is divided into the corresponding cluster to obtain the clustering result.

8. An incomplete multi-view contrast clustering device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

Images