Clustering method, device, equipment and storage medium for multiple view data

Abstract

The embodiments of the present application provide a clustering method, device, equipment and storage medium for multiple view data, and relate to the field of data processing technology. The method includes: obtaining view data corresponding to multiple views respectively, for each view, projecting the corresponding view data into a low-dimensional space based on the projection matrix of the view to obtain a subspace representation matrix and an error matrix, calculating the local total information according to the subspace representation matrix and the weight matrix, constructing an optimization target item according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix and the local total information, solving the optimization objective function to obtain the representation optimization matrix corresponding to each view, and clustering data according to all the representation optimization matrices to obtain at least one clustering result. Subspace representation is performed synchronously in the process of dimensionality reduction, and in the process of optimizing the objective function, the projection matrix and the subspace representation matrix are mutually promoted and co-evolved, so as to improve the accuracy of the subspace representation and the clustering quality.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly to a clustering method, apparatus, device, and storage medium for multi-view data. Background Art

[0002] In the Customer to Factory (C2F) manufacturing mode, it is often necessary to use the multi-view clustering (MVC) method to integrate data from multiple different views, and then obtain multi-modal data to meet the processing requirements of the flexible manufacturing system.

[0003] In the related art, in order to reduce the interference of noise and redundant information in high-dimensional data on the clustering result, the MVC method usually adopts the strategy of first reducing the dimension of the data and then learning the corresponding subspace representation. Although this method can reduce redundant information by means of dimensionality reduction, it will lose fine-grained structural information, resulting in a decline in the final clustering performance. In addition, there is also a method of directly performing representation learning on the original data, and then using low-rank constraints to reduce redundancy and retain global structural information. However, this method ignores the local information contained in the data and cannot fully capture the complex correlations between different data, resulting in the clustering result deviating from the actual data distribution. Summary of the Invention

[0004] The main purpose of the embodiments of this application is to propose a clustering method, apparatus, device, and storage medium for multi-view data, and improve the accuracy of the clustering result of multi-view data.

[0005] To achieve the above object, the first aspect of the embodiments of this application proposes a clustering method for multi-view data, including:

[0006] Obtain the view data corresponding to multiple views respectively;

[0007] For each of the views, based on the projection matrix of the view, project the corresponding view data into a low-dimensional space to obtain a subspace representation matrix and an error matrix, and obtain a weight matrix according to the product of the transpose matrix of the view data and the view data, and calculate local structural information according to the subspace representation matrix and the weight matrix;

[0008] Accumulate the local structural information of each of the views to obtain local total information. For each of the views, construct an optimization objective term according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, and the local total information, and obtain the constraint condition of the optimization objective term according to the view data, the projection matrix, the subspace representation matrix, and the error matrix;

[0009] An optimization objective function is obtained based on the constraint conditions and the corresponding optimization objective terms, and the optimization objective function is solved to obtain a representation optimization matrix corresponding to each view;

[0010] Data clustering is performed based on all the representation optimization matrices to obtain at least one clustering result.

[0011] In some embodiments, constructing the optimization objective terms based on the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, and the local total information includes:

[0012] Obtain a reference subspace representation matrix corresponding to each subspace representation matrix, where the view of the reference subspace representation matrix is different from that of the subspace representation matrix;

[0013] Calculate a diversity matrix based on the reference subspace representation matrix corresponding to the subspace representation matrix, and accumulate all the diversity matrices to obtain the diversity constraint information;

[0014] Construct optimization objective terms based on the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, the local total information, and the diversity constraint information.

[0015] In some embodiments, obtaining the constraint conditions of the optimization objective terms based on the view data, the projection matrix, the subspace representation matrix, and the error matrix includes:

[0016] Use the product of the transposed matrix of the projection matrix and the view data as the first data item. After multiplying the first data item by the subspace representation matrix and adding the error matrix, obtain the second data item, and set the first data item to be equal to the second data item to obtain the first constraint condition;

[0017] Set the subspace representation matrix to be not less than zero to obtain the second constraint condition;

[0018] Use the product of the transposed matrix of the view data and the projection matrix as the third data item, and set the product of the first data item and the third data item to be the identity matrix to obtain the third constraint condition;

[0019] Obtain the constraint conditions based on the first constraint condition, the second constraint condition, and the third constraint condition.

[0020] In some embodiments, solving the optimization objective function to obtain a representation optimization matrix corresponding to each view includes:

[0021] Using the auxiliary matrix corresponding to each of the views to replace the subspace representation matrix to obtain an updated nuclear norm, and adding a fourth constraint condition that sets the corresponding auxiliary matrix to be equal to the subspace representation matrix in the constraint conditions;

[0022] Adjust the optimization objective function according to the updated nuclear norm and the updated constraint conditions to obtain an updated optimization objective function, add the constraint conditions to the updated optimization objective function to obtain an augmented Lagrangian function, and solve the augmented Lagrangian function to obtain a representation optimization matrix corresponding to each of the views.

[0023] In some embodiments, adding the constraint conditions to the updated optimization objective function to obtain an augmented Lagrangian function includes:

[0024] Obtain the diagonal matrix of the weight matrix, obtain the difference matrix between the diagonal matrix and the weight matrix, obtain the first matrix trace according to the diagonal matrix, the difference matrix and the weight matrix, and obtain the local structure information according to the first matrix trace;

[0025] Obtain the second matrix trace according to the subspace representation matrix and the corresponding reference subspace representation matrix, and obtain the diversity matrix according to the second matrix trace;

[0026] For each of the views, obtain the augmented Lagrangian function corresponding to the updated optimization objective function at least according to the updated nuclear norm, the preset norm, the first matrix trace, the diversity matrix, the subspace representation matrix, the error matrix, the projection matrix, the auxiliary matrix, the first Lagrange multiplier and the second Lagrange multiplier, and the constraint condition of the augmented Lagrangian function is the third constraint condition.

[0027] In some embodiments, solving the augmented Lagrangian function to obtain a representation optimization matrix corresponding to each of the views includes:

[0028] For each of the views, obtain the update functions corresponding to the solution parameters according to the augmented Lagrangian function, and alternately update the solution parameters according to the update functions, and the solution parameters at least include the projection matrix, the subspace representation matrix, the auxiliary matrix, the error matrix, the first Lagrange multiplier and the second Lagrange multiplier;

[0029] Execute at least one update process until a preset convergence condition is satisfied, and use the finally updated subspace representation matrix as the representation optimization matrix.

[0030] In some embodiments, when the solution parameter is the projection matrix, obtaining the update functions corresponding to the solution parameters according to the augmented Lagrangian function, and alternately updating the solution parameters according to the update functions includes:

[0031] Obtaining the projection data item related to the projection matrix from the augmented Lagrangian function;

[0032] Obtaining the update function of the projection matrix according to the projection data item, and calculating the gradient of the update function with respect to the projection matrix to obtain the gradient function;

[0033] Performing manifold optimization to solve according to the update function and the gradient function to obtain the updated projection matrix.

[0034] In some embodiments, when the solution parameter is the subspace representation matrix, obtaining the update functions corresponding to the solution parameters according to the augmented Lagrangian function, and alternately updating the solution parameters according to the update functions includes:

[0035] Obtaining at least one representation matrix data item related to the subspace representation matrix from the augmented Lagrangian function;

[0036] Obtaining the first partial derivative of the representation matrix data item with respect to the subspace representation matrix, accumulating the first partial derivatives to obtain an accumulated data item, and setting the accumulated data item equal to zero to obtain the update function corresponding to the subspace representation matrix;

[0037] Performing matrix solution on the update function to obtain the updated subspace representation matrix.

[0038] In some embodiments, when the solution parameter is the auxiliary matrix, obtaining the update functions corresponding to the solution parameters according to the augmented Lagrangian function, and alternately updating the solution parameters according to the update functions includes:

[0039] Obtaining the auxiliary data item related to the auxiliary matrix from the augmented Lagrangian function;

[0040] Obtaining the second partial derivative of the auxiliary data item with respect to the auxiliary matrix, and setting the second partial derivative equal to zero to obtain the update function corresponding to the auxiliary matrix;

[0041] Solving the update function using the singular value threshold to obtain the updated auxiliary matrix.

[0042] In some embodiments, when the solution parameter is the error matrix, obtaining the update functions corresponding to the solution parameters respectively according to the augmented Lagrangian function, and alternately updating the solution parameters according to the update functions includes:

[0043] Obtaining the error data item related to the error matrix from the augmented Lagrangian function;

[0044] Obtaining the third partial derivative of the error data item with respect to the error matrix, setting the third partial derivative equal to zero, and combining with the shrinkage operator to obtain the update function corresponding to the error matrix;

[0045] Solving the update function to obtain the updated error matrix.

[0046] In some embodiments, when the solution parameter is the first Lagrange multiplier or the second Lagrange multiplier, obtaining the update functions corresponding to the solution parameters respectively according to the augmented Lagrangian function, and alternately updating the solution parameters according to the update functions includes:

[0047] Obtaining the first multiplier data item related to the first Lagrange multiplier or the second multiplier data item related to the second Lagrange multiplier from the augmented Lagrangian function;

[0048] Obtaining the fourth partial derivative of the first multiplier data item with respect to the first Lagrange multiplier, or the fifth partial derivative of the second multiplier data item with respect to the second Lagrange multiplier;

[0049] Setting the fourth partial derivative equal to zero to obtain the update function corresponding to the first Lagrange multiplier, or setting the fifth partial derivative equal to zero to obtain the update function corresponding to the second Lagrange multiplier;

[0050] Solving the update function to obtain the updated first Lagrange multiplier or the second Lagrange multiplier.

[0051] In some embodiments, clustering the data according to all the representation optimization matrices to obtain at least one clustering result includes:

[0052] Calculating the average representation matrix corresponding to all the representation optimization matrices;

[0053] Performing spectral clustering on the average representation matrix to obtain at least one of the clustering results.

[0054] To achieve the above object, a second aspect of the embodiments of the present application provides a clustering device for multi-view data, including:

[0055] Data acquisition module: configured to acquire view data corresponding to multiple views respectively;

[0056] Projection calculation module: for each of the views, based on the projection matrix of the view, project the corresponding view data into a low-dimensional space to obtain a subspace representation matrix and an error matrix, and obtain a weight matrix according to the product of the transpose matrix of the view data and the view data, and calculate local structure information according to the subspace representation matrix and the weight matrix;

[0057] Optimization objective construction module: configured to accumulate each local structure information to obtain local total information, for each of the views, construct an optimization objective term according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, and the local total information, and obtain the constraint conditions of the optimization objective term according to the view data, the projection matrix, the subspace representation matrix, and the error matrix;

[0058] Optimization solution module: configured to obtain an optimization objective function according to the constraint conditions and the corresponding optimization objective terms, and solve the optimization objective function to obtain a representation optimization matrix corresponding to each of the views;

[0059] Clustering module: configured to perform data clustering according to all the representation optimization matrices to obtain at least one clustering result.

[0060] To achieve the above object, a third aspect of the embodiments of the present application proposes an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect above is implemented.

[0061] To achieve the above object, a fourth aspect of the embodiments of the present application proposes a storage medium, the storage medium is a storage medium, the storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0062] The clustering method, device, equipment, and storage medium for multiple view data proposed in the embodiments of this application obtain the view data corresponding to multiple views respectively. For each view, based on the projection matrix of the view, project the corresponding view data into a low-dimensional space to obtain a subspace representation matrix and an error matrix, and obtain a weight matrix according to the product of the transpose matrix of the view data and the view data. Calculate the local structure information according to the subspace representation matrix and the weight matrix. Next, accumulate each local structure information to obtain the local total information. For each view, construct an optimization objective term according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, and the local total information, and obtain the constraint conditions of the optimization objective term according to the view data, the projection matrix, the subspace representation matrix, and the error matrix. Obtain the optimization objective function according to the constraint conditions and the corresponding optimization objective term, solve the optimization objective function to obtain the representation optimization matrix corresponding to each view, and finally perform data clustering according to all the representation optimization matrices to obtain at least one clustering result. The embodiments of this application perform a dimensionality reduction operation on the view data with the help of the projection matrix, and perform subspace representation synchronously during the dimensionality reduction process. In view of the fact that different projection methods will cause the data to present different spatial representations in the low-dimensional space, the embodiments of this application combine the two, and during the optimization of the objective function, realize the mutual promotion and co-evolution of the projection matrix and the subspace representation matrix, so as to find the best projection that most matches the original feature space for each view data, maximize the retention of key information related to tasks such as clustering in each view, prevent excessive information loss, ensure that the representation of the data in the low-dimensional space can accurately reflect its essential characteristics in the original high-dimensional space, thereby improving the accuracy of the subspace representation and ultimately improving the clustering quality. In addition, adding local total information during the optimization process aims to retain the data structure of the data itself, so that the clustering performance of high-dimensional data can be effectively enhanced. Description of the Drawings

[0063] Figure 1 is a flowchart of the clustering method for multiple view data provided by the embodiments of this application.

[0064] Figure 2 is a flowchart of constructing an optimization objective term according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, and the local total information provided by the embodiments of this application.

[0065] Figure 3 is a flowchart of obtaining the constraint conditions of the optimization objective term according to the view data, the projection matrix, the subspace representation matrix, and the error matrix provided by the embodiments of this application.

[0066] Figure 4 is a flowchart of solving the optimization objective function to obtain the representation optimization matrix corresponding to each view provided by the embodiments of this application.

[0067] Figure 5 It is a flowchart of adding constraint conditions to an optimization objective update function to obtain an augmented Lagrangian function provided by an embodiment of the present application.

[0068] Figure 6 It is a flowchart of solving the augmented Lagrangian function to obtain a representation optimization matrix corresponding to each view provided by an embodiment of the present application.

[0069] Figure 7 It is a flowchart of obtaining update functions corresponding to solution parameters respectively according to the augmented Lagrangian function and alternately updating the solution parameters according to the update functions provided by an embodiment of the present application.

[0070] Figure 8 It is another flowchart of obtaining update functions corresponding to solution parameters respectively according to the augmented Lagrangian function and alternately updating the solution parameters according to the update functions provided by an embodiment of the present application.

[0071] Figure 9 It is another flowchart of obtaining update functions corresponding to solution parameters respectively according to the augmented Lagrangian function and alternately updating the solution parameters according to the update functions provided by an embodiment of the present application.

[0072] Figure 10 It is another flowchart of obtaining update functions corresponding to solution parameters respectively according to the augmented Lagrangian function and alternately updating the solution parameters according to the update functions provided by an embodiment of the present application.

[0073] Figure 11 It is another flowchart of obtaining update functions corresponding to solution parameters respectively according to the augmented Lagrangian function and alternately updating the solution parameters according to the update functions provided by an embodiment of the present application.

[0074] Figure 12 It is a schematic diagram of the overall process of a clustering method for multi-view data provided by an embodiment of the present application.

[0075] Figure 13 It is a block diagram of the structure of a clustering device for multi-view data provided by another embodiment of the present application.

[0076] Figure 14 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0077] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0078] It should be noted that although the functional modules are divided in the device schematic diagram and the logical sequence is shown in the flowchart, in some cases, the steps shown or described may be executed in a different module division from that in the device or a different order from that in the flowchart.

[0079] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0080] Industrial data has characteristics such as diverse sources, high dimensionality, heterogeneity, and noise. Reasonable integration and classification of it can provide strong support for the intelligent optimization of the manufacturing process.

[0081] In the Customer to Factory (C2F) manufacturing mode, there are often data from multiple different views. Given that each view can provide specific information, it is usually necessary to adopt a multi-view clustering (MVC) method to integrate data from multiple different views and effectively utilize the complementary and comprehensive information from multiple views to improve the clustering performance. In this way, the similarity of multi-modal data in the production process is analyzed in real time to achieve intelligent scheduling and decision-making of production plans to meet the processing requirements of flexible manufacturing systems.

[0082] In the related art, to reduce the interference of noise and redundant information in high-dimensional data on the clustering result, the MVC method usually adopts the strategy of first reducing the dimension of the data and then learning the corresponding subspace representation. Although this method can reduce redundant information through the dimension reduction operation, it will lose fine-grained structural information, resulting in a decline in the final clustering performance. In addition, there is also a method of directly carrying out the representation learning process based on the original data and then using low-rank constraints to reduce redundancy and retain global structural information. However, this method ignores the local information contained in the data and cannot fully capture the complex correlations between different data, thus leading to the clustering result deviating from the actual data distribution.

[0083] Based on this, the embodiments of the present application provide a clustering method, apparatus, device, and storage medium for multiple view data. A dimensionality reduction operation is performed on the view data by means of a projection matrix, and subspace representation is synchronously carried out during the dimensionality reduction process. Since different projection methods will cause the data to exhibit different spatial representations in the low-dimensional space, the embodiments of the present application combine the two. During the optimization of the objective function, the projection matrix and the subspace representation matrix promote each other and co-evolve, so as to find the best projection that most matches the original feature space for each view data, maximize the retention of key information related to tasks such as clustering in each view, prevent excessive information loss, ensure that the representation of the data in the low-dimensional space can accurately reflect its essential features in the original high-dimensional space, thereby improving the accuracy of the subspace representation and ultimately enhancing the clustering quality. In addition, local total information is added during the optimization process to retain the data structure of the data itself, so that the clustering performance of high-dimensional data can be effectively enhanced.

[0084] The embodiments of the present application provide a clustering method, apparatus, device, and storage medium for multiple view data, which will be specifically described through the following embodiments. First, the clustering method for multiple view data in the embodiments of the present application will be described.

[0085] The clustering method for multiple view data provided by the embodiments of the present application relates to the technical field of data processing. The clustering method for multiple view data provided by the embodiments of the present application can be applied to a terminal, or to a server side, or can be a computer program running on a terminal or a server side. For example, the computer program can be a native program or a software module in an operating system; it can be a local (Native) application (Application, APP), that is, a program that needs to be installed in the operating system to run, such as a client that supports clustering of multiple view data, that is, a program that only needs to be downloaded to a browser environment to run; it can also be a small program that can be embedded in any APP. In short, the above computer program can be any form of application program, module, or plug-in. Among them, the terminal communicates with the server through a network. The clustering method for multiple view data can be executed by the terminal or the server, or jointly executed by the terminal and the server.

[0086] In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart watch, etc. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms; it can also be a service node in a blockchain system, and the service nodes in this blockchain system form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). The terminal and the server can be connected through communication connection methods such as Bluetooth, Universal Serial Bus (USB), or network, and this embodiment does not limit this here.

[0087] This application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This application can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0088] The clustering method of multiple view data in the embodiments of this application is described below.

[0089] Figure 1 is an optional flowchart of the clustering method of multiple view data provided by the embodiments of this application. Figure 1 The method in can include but is not limited to steps 110 to 150. At the same time, it can be understood that this embodiment does not specifically limit the order of steps 110 to 150 in Figure 1 and the order of steps can be adjusted according to actual needs, or some steps can be reduced or added.

[0090] Step 110: Obtain the view data corresponding to multiple views respectively.

[0091] In one embodiment, each view data corresponds to the data obtained in its respective view, and a matrix structure is used to store the data elements under the corresponding view. The rows and columns of the matrix can be used to represent information in different dimensions. The view data here includes various types, covering sensor data of different devices, product image features, inventory data, production parameters, etc. Among them, multiple different view data usually describe the same instance using different characteristics. For example, in the field of industrial manufacturing, the sensor data of different devices can constitute the corresponding view data. Similarly, for product image features, if the data corresponding to multiple feature dimensions (such as color features, texture features, etc.) of the extracted images is organized, corresponding view data can also be formed.

[0092] Step 120: For each view, based on the projection matrix of the view, project the corresponding view data into a low-dimensional space to obtain a subspace representation matrix and an error matrix, and obtain a weight matrix according to the product of the transpose matrix of the view data and the view data. Calculate the local structure information based on the subspace representation matrix and the weight matrix.

[0093] In one embodiment, assume the number of views is , and at this time, project the view data corresponding to each view. The following takes the v-th view as an example for illustration, where, .

[0094] First, obtain the low-dimensional projection matrix corresponding to the v-th view. Next, use the projection matrix to project the view data corresponding to the v-th view into a low-dimensional space to achieve the purpose of dimensionality reduction. Represent the projection process in the form of a dictionary, then the dictionary at this time is: , and then use the dictionary to learn the subspace representation matrix , so as to automatically reduce the noise and redundancy in the view data.

[0095] At this time, the relationship between the projection matrix, the view data, and the subspace learning matrix can be expressed as:

[0096]

[0097] Among them, represents the error matrix.

[0098] In one embodiment, in view of the fact that when facing a large number of view data, the spatial relationship between different measurement data points plays a very important role in the overall analysis, a weight matrix is introduced to characterize the spatial compactness of the measurement data points in the view data. The weight matrix can quantitatively reflect the degree of closeness of the spatial distribution of each measurement data point. For example, when analyzing the infrared view data at different positions on the surface of a three-dimensional object, the degree of closeness between the temperature values of each measurement data point is not completely the same. The temperature change trends of measurement data points that are closer are often more similar. Correspondingly, their spatial compactness is stronger. Specifically, in this embodiment, the weight matrix is obtained by multiplying the transpose matrix of the view data by the view data.

[0099] Therefore, the weight matrix is expressed as:

[0100]

[0101] In one embodiment, since the local structure in the view data is retained in the subspace representation matrix, the local structure information can be calculated using the subspace representation matrix and the weight matrix. The specific process can be described as follows:

[0102] First, represent the index of each element value in the subspace representation matrix as {i, j}, whose function is similar to coordinates. With the help of the index combination, the specific element value in the matrix can be accurately located. At the same time, it is stipulated that i is set as the first index and j is set as the second index to achieve an ordered identification.

[0103] Next, obtain the first index vector corresponding to the first index i in the subspace representation matrix , and the second index vector corresponding to the second index j. When the subspace representation matrix is a two-dimensional matrix, the first index vector can be regarded as a set of vector values along the row direction of the matrix, while the second index vector can be regarded as a set of vector values along the column direction.

[0104] Subsequently, considering that if a pair of measurement data points are close in the view data space, then they should also be close in the subspace representation matrix, that is , so in the embodiment of this application, the vector difference between the first index vector and the second index vector is calculated, and further the vector difference norm value corresponding to this vector difference is obtained. This vector difference norm value is used to measure the degree of difference between these two vectors.

[0105] After that, obtain the weight coefficients of the first index i and the second index j in the weight matrix and after accumulating all the weight coefficients, obtain the total coefficient , where the weight matrix is calculated through the original feature space and is used to directly describe the local structural relationship between samples. The total coefficient is used as a fixed value after calculation. Then multiply the vector difference norm value by the total coefficient to obtain the local structural information characterizing the local structural relationship between the elements in the subspace representation matrix . Since the objective function is minimized later, when the total coefficient is larger, the corresponding vector difference is smaller, which means that the measured data points and are closer, and the possibility of belonging to the same class is higher.

[0106] Step 130: Accumulate each local structural information to obtain the local total information. For each view, construct an optimization objective term according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, and the local total information, and obtain the constraint conditions of the optimization objective term according to the view data, the projection matrix, the subspace representation matrix, and the error matrix.

[0107] In one embodiment, in the multi-view data clustering process, an optimization objective function is constructed to solve the optimal parameters. Given that the subspace representation matrix can show the distribution of data in different subspaces from multiple dimensions, thereby reflecting the internal structural characteristics of the data; the error matrix is mainly used to measure the deviation degree between the actual data and the expected data or the model calculation result; the local structural information reflects the local structural related information of each view. Therefore, the embodiment of the present application constructs an optimization objective term at least based on the subspace representation matrix, the error matrix, and the local total information, and then generates an optimization objective function in combination with the optimization objective term.

[0108] In one embodiment, during the optimization process, for the subspace representation matrix, it is necessary to impose a low-rank constraint on the subspace representation matrix of each view to capture the global structural information of the data of each view. Therefore, in this embodiment, the nuclear norm of the subspace representation matrix is used to implement the low-rank constraint, where the nuclear norm is defined as the sum of the singular values of the matrix. During the optimization process, minimizing the nuclear norm can achieve the low-rank constraint.

[0109] Among them, the nuclear norm of the subspace representation matrix is expressed as:

[0110]

[0111] In one embodiment, during the optimization process, it is also necessary to eliminate noise based on the error matrix, so it is necessary to obtain the error matrix The preset norm of is expressed as:

[0112]

[0113] where represents the calculation method of the preset norm. Using this method makes the columns of the error matrix sparse, and makes the norm of some columns approach 0, which can better handle the noise existing in the column vectors. And represents an optimization parameter, and its value range is , this parameter is a discrete value, and it takes values at intervals of , for example { }.

[0114] In one embodiment, each view has its own unique local structure information. Only by integrating these local structure information can the overall data characteristics be presented more comprehensively. For example, when analyzing product images taken from multiple different angles, there is local structure information about the product structure, texture, etc. in each view. After accumulating the local structure information from each view, the local total information covering the local feature situations of all views can be obtained. The embodiments of the present application use the local total information to grasp the overall data characteristics.

[0115] At this time, the local total information is expressed as:

[0116]

[0117] In one embodiment, during the optimization process, the local total information is used as the smoothing regularization term in the subsequent optimization objective function. At this time, a tuning parameter can be introduced to adjust the participation degree of the local total information during the optimization solution process. Therefore, the update of the local total information is expressed as:

[0118]

[0119] where the tuning parameter has the same value range as the optimization parameter .

[0120] According to the above process, the optimization objective term corresponding to view v is expressed as:

[0121]

[0122] In one embodiment, considering that the view data of each view has its unique information, it is desired that the similarity matrix of each view can retain as much information of this view as possible. Therefore, a sparsity constraint between views is also used to increase in the optimization objective term, making the subspace representation matrices as sparse as possible. Refer to Figure 2 , Figure 2 FIG. Figure 2 is a flowchart of constructing an optimization objective term according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, and the local total information provided by an embodiment of the present application, which specifically includes the following steps:

[0123] Step 210: Obtain the reference subspace representation matrix corresponding to each subspace representation matrix.

[0124] In one embodiment, the view of the reference subspace representation matrix is different from that of the subspace representation matrix. That is to say, for the subspace representation matrix corresponding to each view, a cross-view strategy is adopted to select a subspace representation matrix of another view as the reference subspace representation matrix of this view. For example, suppose there are three images taken from different perspectives as three views, corresponding to three subspace representation matrices A, B, and C respectively. Then for the subspace representation matrix corresponding to A, the subspace representation matrix corresponding to B may be selected as its reference subspace representation matrix, and so on.

[0125] In one embodiment, for the subspace representation matrix , its reference subspace representation matrix can be , where v and h are different.

[0126] The purpose of doing this is that by introducing the subspace representation matrix under other views as a reference, the data structure characteristics of the current view can be compared and analyzed from different angles. Since different views often focus on different data features, with the help of this difference, the correlation between each subspace representation matrix can be more comprehensively understood, providing a richer and more diverse reference basis for subsequent clustering operations, which helps to improve the quality of the entire multi-view data processing.

[0127] Step 220: Calculate the diversity matrix according to the reference subspace representation matrix corresponding to the subspace representation matrix, and accumulate all the diversity matrices to obtain the diversity constraint information.

[0128] In one embodiment, calculate the Hadamard product of the subspace representation matrix and the corresponding reference subspace representation matrix, and then calculate the zero norm of the Hadamard product to obtain the diversity matrix. Next, accumulate all the diversity matrices to obtain the diversity constraint information.

[0129] Therefore, the diversity matrix is expressed as:

[0130]

[0131] Among them, represents the Hadamard product operation, represents the 0-norm.

[0132] The diversity constraint information is expressed as:

[0133]

[0134] Step 230: Construct an optimization objective term according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, the local total information, and the diversity constraint information.

[0135] In one embodiment, the diversity constraint information obtained according to the above process can update the optimization objective term, and the updated optimization objective term is expressed as:

[0136]

[0137] Next, describe the constraint conditions corresponding to the optimization objective term. Refer to Figure 3 , Figure 3 is a flowchart of the constraint conditions for obtaining the optimization objective term according to the view data, the projection matrix, the subspace representation matrix, and the error matrix provided by the embodiment of the present application, specifically including the following steps:

[0138] Step 310: Use the product of the transpose matrix of the projection matrix and the view data as the first data item. After multiplying the first data item by the subspace representation matrix and adding the error matrix, obtain the second data item. Set the first data item to be equal to the second data item to obtain the first constraint condition.

[0139] In one embodiment, the first constraint condition is expressed as:

[0140]

[0141] Among them, represents the transpose matrix of the projection matrix, represents the first data item, represents the second data item.

[0142] Step 320: Set the subspace representation matrix to be not less than zero to obtain the second constraint condition.

[0143] In one embodiment, the second constraint condition is expressed as:

[0144]

[0145] Step 330: Use the product of the transpose matrix of the view data and the projection matrix as the third data item. Set the product of the first data item and the third data item to be the identity matrix to obtain the third constraint condition.

[0146] In one embodiment, the third constraint condition is expressed as:

[0147]

[0148] where represents the third data item, represents an identity matrix with dimension d, and this dimension is set according to the actual situation.

[0149] Step 340: Obtain the constraint conditions according to the first constraint condition, the second constraint condition, and the third constraint condition.

[0150] In one embodiment, the constraint condition is expressed as:

[0151]

[0152] Step 140: Obtain the optimization objective function according to the constraint conditions and the corresponding optimization objective items, and solve the optimization objective function to obtain the representation optimization matrix corresponding to each view.

[0153] In one embodiment, on the premise of satisfying the constraint conditions, minimizing all the optimization objective items can obtain the optimization objective function. Therefore, the optimization objective function is expressed as:

[0154]

[0155] It can be seen that the purpose of the optimization objective function is to adjust data such as the subspace representation matrix, the error matrix, and the weight matrix within the range of the constraint conditions, so that the optimization objective items corresponding to each view are minimized. In the process of optimizing the objective function in the embodiments of the present application, the redundancy of data and the influence brought by noise can be effectively reduced, and the data dimensionality reduction and the learning of the coefficient matrix can be integrated into an organic whole as the basic framework. By using a variable low-dimensional projection matrix, the best projection that most fits the original feature space is explored for each view, thereby promoting the learning process of the subspace representation. Subsequently, the learned subspace representation matrix will in turn serve as feedback information to guide the learning of the projection matrix. In the iterative process, the projection matrix and the subspace representation promote each other and co-evolve, achieving a good situation where they promote each other and co-evolve, and then finding the best projection that most matches the original feature space for each view data, maximizing the retention of key information related to tasks such as clustering in each view, avoiding excessive information loss, ensuring that the representation of data in the low-dimensional space can accurately reflect its essential features in the original high-dimensional space, thereby improving the accuracy of the subspace representation and ultimately improving the clustering quality.

[0156] Next, describe how to solve the above optimization objective function.

[0157] In one embodiment, referring to Figure 4 , Figure 4 is a flowchart for obtaining the representation optimization matrix corresponding to each view by solving the optimization objective function provided in the embodiments of the present application, specifically including the following steps:

[0158] Step 410: Use the auxiliary matrix corresponding to each view to replace the subspace representation matrix to obtain an updated nuclear norm, and add a fourth constraint condition that sets the corresponding auxiliary matrix equal to the subspace representation matrix in the constraint conditions.

[0159] In one embodiment, considering that the nuclear norm is included in the optimization objective function will make the optimization objective function non-smooth, resulting in the optimization problem in the embodiments of the present application becoming non-convex and difficult to directly solve. Usually, the gradient-based optimization methods used in the optimization process cannot be directly applied because the nuclear norm is non-differentiable at some points and the gradient of the optimization objective function containing the nuclear norm cannot be directly calculated. Therefore, in the embodiments of the present application, for each subspace representation matrix an auxiliary matrix

[0160] is set for each to perform variable separation, so as to realize the solution of the alternating iteration process.

[0161]

[0162] The fourth constraint condition is expressed as:

[0163]

[0164] Step 420: Adjust the optimization objective function according to the updated nuclear norm and the updated constraint conditions to obtain an optimized objective update function, add the constraint conditions to the optimized objective update function to obtain an augmented Lagrangian function, and solve the augmented Lagrangian function to obtain the representation optimization matrix corresponding to each view.

[0165] In one embodiment, the optimized objective update function is expressed as:

[0166]

[0167] The updated constraint conditions are expressed as:

[0168]

[0169] In one embodiment, next, the constraint conditions need to be added to the optimized objective update function to obtain an augmented Lagrangian function. Referring to Figure 5 , Figure 5This is a flowchart of adding constraints to the optimization target update function to obtain an augmented Lagrangian function provided by an embodiment of the present application, which specifically includes the following steps:

[0170] Step 510: Obtain the diagonal matrix of the weight matrix, obtain the difference matrix between the diagonal matrix and the weight matrix, obtain the first matrix trace according to the diagonal matrix, the difference matrix and the weight matrix, and obtain the local structure information according to the first matrix trace.

[0171] In one embodiment, in order to facilitate calculation during iteration, local structural information needs to be reconstructed.

[0172] First, get the diagonal matrix of the weight matrix , while the diagonal matrix The diagonal elements of Expressed as:

[0173]

[0174] Get the diagonal matrix and weight matrix The difference matrix of , expressed as:

[0175]

[0176] Then according to the diagonal matrix 、Difference Matrix and weight matrix Get the first matrix trace, expressed as:

[0177]

[0178] Next, the first matrix trace is used to replace the local structure information, and the adjustment parameters are introduced, so the local structure information is expressed as:

[0179]

[0180] Step 520: Obtain a second matrix trace according to the subspace representation matrix and the corresponding reference subspace representation matrix, and obtain a diversity matrix according to the second matrix trace.

[0181] In one embodiment, in order to facilitate iterative calculation, it is necessary to rewrite the diversity matrix. The specific operation is as follows: first obtain the product of the subspace representation matrix and the corresponding reference subspace representation matrix, then calculate the trace of the product result, and then obtain the second matrix trace. Therefore, the second matrix trace can be expressed as:

[0182]

[0183] Next, the second matrix trace is used to replace the diversity matrix, so the diversity matrix is ​​expressed as:​​​

[0184]

[0185] Step 530: For each view, obtain the augmented Lagrangian function corresponding to the optimization objective update function based on at least the updated nuclear norm, preset norm, first matrix trace, diversity matrix, subspace representation matrix, error matrix, projection matrix, auxiliary matrix, first Lagrange multiplier, and second Lagrange multiplier.

[0186] In one embodiment, the augmented Lagrangian function is generally used to solve optimization problems with constraints by adding the constraint conditions to the objective function and introducing Lagrange multipliers to handle the constraints. Therefore, the constraint conditions are applied to the optimization objective update function to obtain the augmented Lagrangian function.

[0187] Among them, the augmented Lagrangian function at least includes the projection matrix , subspace representation matrix , auxiliary matrix , error matrix , first Lagrange multiplier and second Lagrange multiplier , expressed as . Among them, is the penalty parameter.

[0188] Specifically, taking the third constraint condition as the constraint condition of the augmented Lagrangian function, the obtained augmented Lagrangian function is expressed as:

[0189]

[0190]

[0191] Among them, is an intermediate parameter, represents calculating the Frobenius norm of the correlation matrix.

[0192] In one embodiment, after obtaining the augmented Lagrangian function, solving calculations need to be performed. Referring to Figure 6 , Figure 6 is the flowchart for solving the augmented Lagrangian function to obtain the representation optimization matrix corresponding to each view provided by the embodiment of the present application, specifically including the following steps:

[0193] Step 610: For each view, obtain the update functions corresponding to the solution parameters according to the augmented Lagrangian function, and alternately update the solution parameters according to the update functions.

[0194] In one embodiment, the solution parameters in the augmented Lagrangian function at least include the projection matrix , Subspace representation matrix , Auxiliary matrix , Error matrix , First Lagrange multiplier and second Lagrange multiplier . The specific update process refers to solving each solution parameter. During the solution process, each solution parameter is updated in sequence according to the order of the solution parameters. When updating a certain solution parameter, other solution parameters are kept fixed.

[0195] In one embodiment, when the solution parameter is the projection matrix, refer to Figure 7 , Figure 7 is a flowchart provided by an embodiment of the present application for obtaining update functions corresponding to solution parameters according to the augmented Lagrangian function and alternately updating the solution parameters according to the update functions, specifically including the following steps:

[0196] Step 710: Obtain the projection data item related to the projection matrix from the augmented Lagrangian function.

[0197] In one embodiment, the projection data item in the augmented Lagrangian function related to the projection matrix is:

[0198]

[0199]

[0200] Step 720: Obtain the update function of the projection matrix according to the projection data item, and calculate the gradient of the update function with respect to the projection matrix to obtain the gradient function.

[0201] In one embodiment, assume , , based on the constraint term, expand the projection data item as:

[0202]

[0203] Furthermore, the update function is expressed as:

[0204]

[0205] The gradient function obtained by calculating the gradient of the update function with respect to the projection matrix is expressed as:

[0206]

[0207] Step 730: Perform manifold optimization solution according to the update function and the gradient function to obtain the updated projection matrix.

[0208] In one embodiment, manifold optimization is a method for optimizing on a non-linear constraint space (manifold), and its goal is to find the minimum value of the objective function while satisfying specific constraint conditions. Therefore, in the embodiments of the present application, the intermediate update function is used as the objective function of manifold optimization and is optimized in combination with the gradient function.

[0209] During the manifold optimization process, first, under the condition of satisfying the constraint conditions, an appropriate initial value of the projection matrix is selected. . Then an iterative process is carried out. In each iteration, the gradient function is used to update the value of the projection matrix, and the update formula is:

[0210]

[0211] where, represents the step size of the k-th iteration, The operation is an operation that maps a vector in the tangent space back to the manifold, and the specific operation process depends on the specific structure of the manifold selected in practice. For example, The operation can be the matrix exponential mapping, etc. The step size here can be determined using a line search method so that the objective function can decrease sufficiently in each iteration.

[0212] In addition, during the iterative process, it is judged whether to end the iteration according to the convergence condition. If the convergence condition is satisfied, the iteration is stopped, and the current value is the approximate optimal solution of the updated projection matrix. Among them, the convergence condition can be that the norm of the gradient is less than a preset threshold , and this convergence condition is expressed as: . It can also be that the change amount of the objective function value is less than a certain threshold , and this convergence condition is expressed as: . The threshold here can be adjusted according to actual needs.

[0213] In one embodiment, when the parameter to be solved is the subspace representation matrix, referring to Figure 8 , Figure 8 is another flowchart provided by the embodiments of the present application for obtaining the update functions corresponding to the parameters to be solved according to the augmented Lagrangian function and alternately updating the parameters to be solved according to the update functions, which specifically includes the following steps:

[0214] Step 810: Obtain at least one representation matrix data item related to the subspace representation matrix from the augmented Lagrangian function.

[0215] In one embodiment, there are three representation matrix data items related to the subspace representation matrix in the augmented Lagrangian function, which are respectively:

[0216]

[0217]

[0218]

[0219] Step 820: Obtain the first partial derivative of the matrix data item with respect to the subspace representation matrix, accumulate the first partial derivatives to obtain an accumulated data item, and set the accumulated data item equal to zero to obtain the update function corresponding to the subspace representation matrix.

[0220] In one embodiment, the partial derivatives of the three obtained matrix data items with respect to the subspace representation matrix are calculated, and this partial derivative is called the first partial derivative.

[0221] Among them, for the first matrix data item, according to the matrix derivative formula , the first partial derivative obtained by calculation is expressed as: .

[0222] For the second matrix data item, according to the matrix derivative formula , the first partial derivative obtained by calculation is expressed as: .

[0223] The calculation of the partial derivative of the third matrix data item is relatively complex, and the specific process is as follows:

[0224] Based on the previous A and B, add the assumption , and the first partial derivative is expressed as: .

[0225] Next, add the above three first partial derivatives to obtain an accumulated data item, and set the accumulated data item equal to zero to obtain the update function corresponding to the subspace representation matrix as follows:

[0226]

[0227] Step 830: Solve the matrix for the update function to obtain the updated subspace representation matrix.

[0228] In one embodiment, the above update function is solved by the Bartels-Stewart algorithm, which solves the matrix equation through eigenvalue decomposition and intermediate matrix construction.

[0229] The specific process of the solution is described as follows:

[0230] First, organize the update function into the standard matrix equation form, where A, B, and C are composed of , , , Matrices obtained through corresponding operations on equal matrices. Next, eigenvalue decomposition is performed on matrices A and B obtained from the above process. Among them, and , and respectively represent diagonal matrices containing the eigenvalues of A or B, and are the corresponding eigenvector matrices.

[0231] Then construct the intermediate matrix such that: . Substitute into the standard matrix equation to obtain: . Simplify it to obtain: .

[0232] Denote , then the equation becomes: . Since and are diagonal matrices, each element in can be solved through , where and are respectively and 's diagonal elements, and is the diagonal element of D. Finally, through obtain the solution of to obtain the updated subspace representation matrix.

[0233] In one embodiment, when the solution parameter is an auxiliary matrix, refer to Figure 9 , Figure 9 is another flow chart provided by the embodiment of the present application for obtaining the update functions corresponding to the solution parameters according to the augmented Lagrangian function and alternately updating the solution parameters according to the update functions, specifically including the following steps:

[0234] Step 910: Obtain the auxiliary data items related to the auxiliary matrix from the augmented Lagrangian function.

[0235] In one embodiment, the auxiliary data items related to the auxiliary matrix in the augmented Lagrangian function are expressed as:

[0236]

[0237] Step 920: Obtain the second partial derivative of the auxiliary data item with respect to the auxiliary matrix, and set the second partial derivative equal to zero to obtain the update function corresponding to the auxiliary matrix.

[0238] In one embodiment, the partial derivative of the obtained auxiliary data item with respect to the auxiliary matrix is calculated, and this partial derivative is referred to as the second partial derivative. At the same time, since the nuclear norm is not differentiable at some points, the subgradient property of the nuclear norm is combined in the process of calculating the second partial derivative.

[0239] Among them, the subgradient of the nuclear norm is as follows:

[0240]

[0241] Among them, represents the operator of the singular value threshold SVT, and the optimization parameter represents the threshold.

[0242] Let the second partial derivative be zero, and we get:

[0243]

[0244] Among them, can be , next, let , and after arranging the above formula, the update function is expressed as:

[0245]

[0246] Step 930: Solve the update function using the singular value threshold to obtain the updated auxiliary matrix.

[0247] In one embodiment, using the singular value threshold operation, the matrix is singular value decomposed, the singular values less than the threshold are set to zero, and the singular values greater than the threshold are subtracted by the threshold for low-rank approximation to obtain the updated auxiliary matrix.

[0248] In one embodiment, when the solution parameter is the error matrix, referring to Figure 10 , Figure 10 is another flowchart provided by the embodiment of the present application for obtaining the update functions corresponding to the solution parameters according to the augmented Lagrangian function and alternately updating the solution parameters according to the update functions, which specifically includes the following steps:

[0249] Step 1010: Obtain the error data item related to the error matrix from the augmented Lagrangian function.

[0250] In one embodiment, the error data item related to the error matrix in the augmented Lagrangian function is as follows:

[0251]

[0252] Step 1020: Obtain the third partial derivative of the error data item with respect to the error matrix, set the third partial derivative to zero, and combine it with the shrinkage operator to obtain the update function corresponding to the error matrix.

[0253] In one embodiment, take the partial derivative of the obtained error data item with respect to the error matrix and call this partial derivative the third partial derivative, where the third partial derivative is expressed as:

[0254]

[0255] Set the third partial derivative to zero to obtain:

[0256]

[0257] where represents an optimization parameter whose value range is , and this parameter is a discrete value that takes values at intervals of within the value range.

[0258] Next, let , rearrange the above formula to obtain the update function expressed as:

[0259]

[0260] Step 1030: Solve the update function to obtain the updated error matrix.

[0261] In one embodiment, solve it based on the solution method of the shrinkage operator.

[0262] First, perform singular value decomposition on the matrix to obtain the corresponding diagonal matrix . Then calculate , and the elements on its diagonal are , where is the element on the diagonal of the diagonal matrix . Then use to obtain the updated error matrix.

[0263] In one embodiment, when the solution parameter is the first Lagrange multiplier or the second Lagrange multiplier, refer to Figure 11 , Figure 11 which is another flowchart provided by the embodiments of the present application for obtaining the update functions corresponding to the solution parameters according to the augmented Lagrangian function and alternately updating the solution parameters according to the update functions, specifically including the following steps:

[0264] Step 1110: Obtain a first multiplier data item related to a first Lagrange multiplier or a second multiplier data item related to a second Lagrange multiplier from an augmented Lagrangian function.

[0265] In one embodiment, the first multiplier data item related to the first Lagrange multiplier in the augmented Lagrangian function is expressed as:

[0266]

[0267] The second multiplier data item related to the second Lagrange multiplier is expressed as:

[0268]

[0269] Step 1120: Obtain a fourth partial derivative of the first multiplier data item with respect to the first Lagrange multiplier, or a fifth partial derivative of the second multiplier data item with respect to the second Lagrange multiplier.

[0270] In one embodiment, take the partial derivative of the obtained first multiplier data item with respect to the first Lagrange multiplier and call this partial derivative the fourth partial derivative, where the fourth partial derivative is expressed as:

[0271]

[0272] In one embodiment, take the partial derivative of the obtained second multiplier data item with respect to the second Lagrange multiplier and call this partial derivative the fifth partial derivative, where the fifth partial derivative is expressed as:

[0273]

[0274] Step 1130: Set the fourth partial derivative equal to zero to obtain an update function corresponding to the first Lagrange multiplier, or set the fifth partial derivative equal to zero to obtain an update function corresponding to the second Lagrange multiplier.

[0275] In one embodiment, when setting the fourth partial derivative equal to zero, the update function corresponding to the first Lagrange multiplier is expressed as:

[0276]

[0277]

[0278] where the penalty parameter has a value range related to , these two constants defined according to actual requirements.

[0279] Next, set the fifth partial derivative equal to zero, and the update function corresponding to the second Lagrange multiplier is expressed as:

[0280]

[0281] Step 1140: Solve the update function to obtain the updated first Lagrange multiplier or the second Lagrange multiplier.

[0282] In one embodiment, for the update function corresponding to the first Lagrange multiplier, according to the updated projection matrix calculated previously , subspace representation matrix , auxiliary matrix , error matrix and the updated value of the first Lagrange multiplier . For the update function corresponding to the second Lagrange multiplier, calculate the updated value of the second Lagrange multiplier , auxiliary matrix according to the updated subspace representation matrix calculated previously .

[0283] Through the above process, the projection matrix , subspace representation matrix , auxiliary matrix , error matrix , first Lagrange multiplier and second Lagrange multiplier are updated in sequence.

[0284] Step 620: Perform at least one update process until the preset convergence condition is satisfied, and use the last updated subspace representation matrix as the representation optimization matrix.

[0285] In one embodiment, the above update operation is performed once in each iteration process. After each update, it is necessary to determine whether the preset convergence condition is satisfied. The convergence condition here can be set as the change amount between the two consecutive update values is less than the preset judgment value, or other convergence conditions defined according to actual requirements. If the preset convergence condition is satisfied, then select the subspace representation matrix obtained from the last update as the representation optimization matrix. It should be clear that the calculation process of the representation optimization matrix belongs to a comprehensive optimization process. In this process, the optimal solutions are obtained for both the projection matrix and the error matrix, etc.

[0286] Step 150: Perform data clustering based on all the representation optimization matrices to obtain at least one clustering result.

[0287] In one embodiment, for each view, the representation optimization matrix is obtained by constructing an optimization objective function considering various factors such as the subspace representation matrix, the error matrix, and the local total information, and then performing a series of solving operations under the satisfaction of the constraint conditions. Therefore, as the optimized data feature representation form under each view, the representation optimization matrix can more accurately reflect the data structure and internal correlation of that view.

[0288] After the representation optimization matrix corresponding to each view is obtained in the embodiment of the present application, the next step is to calculate the average representation matrix corresponding to all the representation optimization matrices. With the help of the average representation matrix, integration and averaging operations are performed on the optimized data features of each view, and then a data matrix that can comprehensively reflect the commonalities and average states of all views is obtained, so as to grasp the overall characteristics of the data of multiple views from an overall perspective. Finally, spectral clustering operation is performed on the average representation matrix. Spectral clustering is a clustering method based on graph theory and matrix eigenvalue decomposition. It analyzes the similarity graph structure constructed by the average representation matrix, and uses relevant information such as the eigenvalues and eigenvectors of the matrix to divide the data into different categories, so as to obtain at least one clustering result.

[0289] As can be seen from the above overall process, in the optimization process of the embodiment of the present application, low-rank constraints are used to capture the global structure, and at the same time, a smoothing regularization term is used to retain the local structure of the data, so as to effectively retain the global and local geometric structures of the data of multiple views. Given the complementary information between different views, diversity constraint information is used to ensure the information consistency between multiple views, so that the representation optimization matrix of each view can retain more information unique to that view. In addition, the embodiment of the present application integrates the reduction of redundant information and the learning of the representation matrix into the same optimization framework, realizes automatically reducing the influence of redundant and noise information while learning the representation matrix, and achieves efficient information interaction between multiple views. By means of this integration method, information is allowed to flow, avoiding the loss of structural information, and effectively improving the overall effect of multi-view clustering. It reduces the influence of problems such as the disconnection between redundant dimensionality reduction and subspace representation learning, the difficulty of eliminating noise information, and insufficient information transmission between views in the multi-view clustering method in the related art on the clustering result.

[0290] In one embodiment, referring to Figure 12 , Figure 12 is the overall process schematic diagram of the clustering method for multiple view data provided by the embodiment of the present application. Figure 12It schematically shows the view data corresponding to multiple views, and then uses the projection matrix of the view to project the corresponding view data into a low-dimensional space to obtain a subspace representation matrix. Next, local total information and diversity constraint information are introduced to participate in the optimization process. In the overall optimization process, at least the projection matrix of the optimal solution is obtained and the subspace representation matrix . Then, the subspace representation matrix obtained from the last update is selected as the representation optimization matrix, and data clustering is performed on the representation optimization matrix to obtain at least one clustering result.

[0291] The technical solution provided by the embodiments of the present application obtains the view data corresponding to multiple views respectively. For each view, based on the projection matrix of the view, the corresponding view data is projected into a low-dimensional space to obtain a subspace representation matrix and an error matrix, and a weight matrix is obtained according to the product of the transposed matrix of the view data and the view data. The local structure information is calculated according to the subspace representation matrix and the weight matrix. Next, each local structure information is accumulated to obtain local total information. For each view, an optimization objective term is constructed according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, and the local total information, and the constraint conditions of the optimization objective term are obtained according to the view data, the projection matrix, the subspace representation matrix, and the error matrix. The optimization objective function is obtained according to the constraint conditions and the corresponding optimization objective term, and the representation optimization matrix corresponding to each view is obtained by solving the optimization objective function. Finally, data clustering is performed according to all the representation optimization matrices to obtain at least one clustering result. The embodiments of the present application perform a dimensionality reduction operation on the view data by means of a projection matrix, and perform subspace representation synchronously during the dimensionality reduction process. In view of the fact that different projection methods will cause the data to show different spatial representations in the low-dimensional space, the embodiments of the present application combine the two, and during the process of optimizing the objective function, realize the joint dynamic adjustment of the projection matrix and the subspace representation matrix, realize the mutual promotion and co-evolution of the optimization results, and then find the best projection that most matches the original feature space for each view data, maximize the retention of key information related to tasks such as clustering in each view, prevent excessive information loss, ensure that the representation of the data in the low-dimensional space can accurately reflect its essential characteristics in the original high-dimensional space, thereby improving the accuracy of the subspace representation and finally improving the clustering quality. In addition, local total information is added during the optimization process to retain the data structure of the data itself, so that the clustering performance of high-dimensional data can be effectively enhanced.

[0292] The embodiments of the present application also provide a clustering device for multiple view data, which can implement the above-mentioned clustering method for multiple view data. Refer to Figure 13 , the device includes:

[0293] Data acquisition module 1310: It is used to acquire view data corresponding to multiple views respectively.

[0294] Projection calculation module 1320: For each view, based on the projection matrix of the view, project the corresponding view data into a low-dimensional space to obtain a subspace representation matrix and an error matrix, and obtain a weight matrix according to the product of the transpose matrix of the view data and the view data, and calculate the local structure information according to the subspace representation matrix and the weight matrix.

[0295] Optimization objective construction module 1330: It is used to accumulate each local structure information to obtain the local total information. For each view, construct an optimization objective term according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, and the local total information, and obtain the constraint conditions of the optimization objective term according to the view data, the projection matrix, the subspace representation matrix, and the error matrix.

[0296] Optimization solution module 1340: It is used to obtain an optimization objective function according to the constraint conditions and the corresponding optimization objective term, and solve the optimization objective function to obtain a representation optimization matrix corresponding to each view.

[0297] Clustering module 1350: It is used to perform data clustering according to all the representation optimization matrices to obtain at least one clustering result.

[0298] The specific implementation manner of the clustering device for multiple view data in this embodiment is basically the same as the specific implementation manner of the above-mentioned clustering method for multiple view data, and will not be elaborated here.

[0299] An embodiment of the present application further provides an electronic device, including:

[0300] At least one memory;

[0301] At least one processor;

[0302] At least one program;

[0303] The program is stored in the memory, and the processor executes the at least one program to implement the above-mentioned clustering method for multiple view data of the present application. This electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (Personal Digital Assistant, PDA), an in-vehicle computer, etc.

[0304] Please refer to Figure 14 , Figure 14 which schematically shows the hardware structure of an electronic device in another embodiment. The electronic device includes:

[0305] The processor 1401 can be implemented in the form of a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application; the memory 1402 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 1402 can store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1402 and are called by the processor 1401 to execute the clustering method for multiple view data in the embodiments of the present application; the input / output interface 1403 is used to implement information input and output; the communication interface 1404 is used to implement communication interaction between this device and other devices, and can implement communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.); and the bus 1405 is used to transmit information between various components of the device (such as the processor 1401, the memory 1402, the input / output interface 1403, and the communication interface 1404); among them, the processor 1401, the memory 1402, the input / output interface 1403, and the communication interface 1404 are communicatively connected to each other inside the device through the bus 1405.

[0306] The embodiments of the present application also provide a storage medium. The storage medium is a storage medium that stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned clustering method for multiple view data.

[0307] As a non-transitory storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include high-speed random access memory, and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory optionally includes a memory remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0308] The clustering method, device, equipment, and storage medium for multiple view data proposed in the embodiments of this application obtain the view data corresponding to multiple views respectively. For each view, based on the projection matrix of the view, project the corresponding view data into a low-dimensional space to obtain a subspace representation matrix and an error matrix, and obtain a weight matrix according to the product of the transpose matrix of the view data and the view data. Calculate the local structure information according to the subspace representation matrix and the weight matrix. Next, accumulate each local structure information to obtain the local total information. For each view, construct an optimization objective term according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, and the local total information, and obtain the constraint conditions of the optimization objective term according to the view data, the projection matrix, the subspace representation matrix, and the error matrix. Obtain the optimization objective function according to the constraint conditions and the corresponding optimization objective term, solve the optimization objective function to obtain the representation optimization matrix corresponding to each view, and finally perform data clustering according to all the representation optimization matrices to obtain at least one clustering result.

[0309] The embodiments of this application perform dimensionality reduction operations on view data with the help of a projection matrix and synchronously perform subspace representation during the dimensionality reduction process. Given that different projection methods will cause data to exhibit different spatial representations in the low-dimensional space, the embodiments of this application combine the two. During the process of optimizing the objective function, the projection matrix and the subspace representation matrix promote and co-evolve with each other, thereby finding the best projection that most matches the original feature space for each view data, maximizing the retention of key information related to tasks such as clustering in each view, preventing excessive information loss, ensuring that the representation of data in the low-dimensional space can accurately reflect its essential features in the original high-dimensional space, thereby improving the accuracy of subspace representation and ultimately enhancing the clustering quality. In addition, local total information is added during the optimization process to retain the data structure of the data itself, so that the clustering performance of high-dimensional data can be effectively enhanced.

[0310] The embodiments described in the embodiments of this application are to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are equally applicable to similar technical problems.

[0311] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application. There may be more or fewer steps than those shown in the figures, or some steps may be combined, or different steps may be involved. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, and their appropriate combinations.

[0312] As used in the specification of the present application and the above-mentioned drawings, the terms "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order different from those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that comprises a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0313] It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Here, A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (item) of the following" or its similar expressions refer to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0314] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the above-mentioned division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical or other forms.

[0315] The units described above as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, the functional units in each embodiment of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0316] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store programs.

[0317] The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings. However, this does not limit the scope of the rights of the embodiments of the present application. Any modifications, equivalent replacements, and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall fall within the scope of the rights of the embodiments of the present application.

Claims

1. A clustering method for multiple view data, characterized in that: include: Get the view data corresponding to multiple views respectively; For each of the views, based on a projection matrix of the view, the corresponding view data is projected into a low-dimensional space to obtain a subspace representation matrix and an error matrix, a weight matrix is ​​obtained according to a product of a transposed matrix of the view data and the view data, and local structure information is calculated according to the subspace representation matrix and the weight matrix; Accumulating each of the local structural information to obtain local total information, for each of the views, constructing an optimization target item according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix and the local total information, and obtaining a constraint condition of the optimization target item according to the view data, the projection matrix, the subspace representation matrix and the error matrix; Obtaining an optimization objective function according to the constraint conditions and the corresponding optimization objective items, and solving the optimization objective function to obtain a representation optimization matrix corresponding to each of the views; Performing data clustering according to all the representation optimization matrices to obtain at least one clustering result; The obtaining of the constraint condition of the optimization target item according to the view data, the projection matrix, the subspace representation matrix and the error matrix comprises: Taking the product of the transposed matrix of the projection matrix and the view data as a first data item, multiplying the first data item by the subspace representation matrix, and then adding the error matrix to obtain a second data item, setting the first data item equal to the second data item, and obtaining a first constraint condition; The subspace representation matrix is ​​set to be not less than zero, thereby obtaining a second constraint condition; Taking the product of the transposed matrix of the view data and the projection matrix as a third data item, setting the product of the first data item and the third data item as a unit matrix, and obtaining a third constraint condition; The constraint condition is obtained according to the first constraint condition, the second constraint condition and the third constraint condition.

2. A clustering method for multiple view data according to claim 1, characterized in that: The constructing the optimization target item according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix and the local total information includes: Obtaining a reference subspace representation matrix corresponding to each of the subspace representation matrices, wherein a view of the reference subspace representation matrix is ​​different from that of the subspace representation matrix; Calculating a diversity matrix according to the reference subspace representation matrix corresponding to the subspace representation matrix, and accumulating all the diversity matrices to obtain diversity constraint information; An optimization target item is constructed according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix, the local total information and the diversity constraint information.

3. The clustering method of multiple view data according to claim 1, characterized in that: The step of solving the optimization objective function to obtain a representation optimization matrix corresponding to each of the views includes: The subspace representation matrix is ​​replaced by the auxiliary matrix corresponding to each of the views to obtain an updated nuclear norm, and a fourth constraint condition is added to the constraint condition, which sets the corresponding auxiliary matrix equal to the subspace representation matrix; The optimization objective function is adjusted according to the updated nuclear norm and the updated constraint condition to obtain an optimization objective update function, the optimization condition is added to the optimization objective update function to obtain an augmented Lagrangian function, and the augmented Lagrangian function is solved to obtain a representation optimization matrix corresponding to each of the views.

4. The clustering method of multiple view data according to claim 3, characterized in that: The step of adding the optimization condition to the optimization target update function to obtain an augmented Lagrangian function includes: Obtaining a diagonal matrix of the weight matrix, obtaining a difference matrix between the diagonal matrix and the weight matrix, obtaining a first matrix trace according to the diagonal matrix, the difference matrix and the weight matrix, and obtaining the local structure information according to the first matrix trace; Obtain a second matrix trace according to the subspace representation matrix and the corresponding reference subspace representation matrix, and obtain a diversity matrix according to the second matrix trace; For each of the views, the augmented Lagrangian function corresponding to the optimization objective update function is obtained at least based on the update kernel function, the preset norm, the first matrix trace, the diversity matrix, the subspace representation matrix, the error matrix, the projection matrix, the auxiliary matrix, the first Lagrangian multiplier and the second Lagrangian multiplier, and the constraint condition of the augmented Lagrangian function is the third constraint condition.

5. The clustering method of multiple view data according to claim 4, characterized in that: The step of solving the augmented Lagrangian function to obtain a representation optimization matrix corresponding to each of the views includes: For each of the views, obtaining update functions corresponding to solution parameters respectively according to the augmented Lagrangian function, and alternately updating the solution parameters according to the update functions, wherein the solution parameters at least include the projection matrix, the subspace representation matrix, the auxiliary matrix, the error matrix, the first Lagrangian multiplier, and the second Lagrangian multiplier; At least one updating process is performed until a preset convergence condition is met, and the subspace representation matrix updated last is used as the representation optimization matrix.

6. The method for clustering multiple view data according to claim 5, characterized in that: When the solution parameter is the projection matrix, obtaining update functions corresponding to the solution parameters respectively according to the augmented Lagrangian function, and alternately updating the solution parameters according to the update function, comprises: Acquire projection data items related to the projection matrix from the augmented Lagrangian function; Obtaining the update function of the projection matrix according to the projection data item, and calculating the gradient of the update function with respect to the projection matrix to obtain a gradient function; A manifold optimization solution is performed according to the update function and the gradient function to obtain an updated projection matrix.

7. The method for clustering multiple view data according to claim 5, characterized in that: When the solution parameter is the subspace representation matrix, obtaining update functions corresponding to the solution parameters respectively according to the augmented Lagrangian function, and alternately updating the solution parameters according to the update function, comprises: Acquire at least one representation matrix data item related to the subspace representation matrix from the augmented Lagrangian function; Obtaining a first partial derivative of the representation matrix data item with respect to the subspace representation matrix, accumulating the first partial derivative to obtain an accumulated data item, setting the accumulated data item equal to zero, and obtaining the update function corresponding to the subspace representation matrix; A matrix solution is performed on the update function to obtain an updated subspace representation matrix.

8. The method for clustering multiple view data according to claim 5, characterized in that: When the solution parameter is the auxiliary matrix, obtaining update functions corresponding to the solution parameters respectively according to the augmented Lagrangian function, and alternately updating the solution parameters according to the update function, comprises: Acquire auxiliary data items related to the auxiliary matrix from the augmented Lagrangian function; Obtaining a second partial derivative of the auxiliary data item with respect to the auxiliary matrix, setting the second partial derivative equal to zero, and obtaining the update function corresponding to the auxiliary matrix; The update function is solved using a singular value threshold to obtain the updated auxiliary matrix.

9. The method for clustering multiple view data according to claim 5, characterized in that: When the solution parameter is the error matrix, obtaining update functions corresponding to the solution parameters respectively according to the augmented Lagrangian function, and alternately updating the solution parameters according to the update function, comprises: Obtaining error data items related to the error matrix from the augmented Lagrangian function; Obtaining the third partial derivative of the error data item with respect to the error matrix, setting the third partial derivative equal to zero, and combining with a shrinkage operator to obtain the update function corresponding to the error matrix; The update function is solved to obtain the updated error matrix.

10. The method for clustering multiple view data according to claim 5, characterized in that: When the solution parameter is the first Lagrangian multiplier or the second Lagrangian multiplier, acquiring update functions corresponding to the solution parameters according to the augmented Lagrangian function, and alternately updating the solution parameters according to the update functions, comprises: Acquire a first multiplier data item related to the first Lagrangian multiplier or a second multiplier data item related to the second Lagrangian multiplier from the augmented Lagrangian function; Obtaining a fourth partial derivative of the first multiplier data item with respect to the first Lagrangian multiplier, or a fifth partial derivative of the second multiplier data item with respect to the second Lagrangian multiplier; Setting the fourth partial derivative equal to zero to obtain the update function corresponding to the first Lagrangian multiplier, or setting the fifth partial derivative equal to zero to obtain the update function corresponding to the second Lagrangian multiplier; The update function is solved to obtain the updated first Lagrangian multiplier or the second Lagrangian multiplier.

11. The method for clustering multiple view data according to any one of claims 1 to 10, characterized in that: The step of clustering data according to all the representation optimization matrices to obtain at least one clustering result includes: Calculate the average representation matrix corresponding to all the representation optimization matrices; Perform spectral clustering on the average representation matrix to obtain at least one clustering result.

12. A device for clustering multiple view data, characterized in that: include: Data acquisition module: used to acquire view data corresponding to multiple views; A projection calculation module: configured to project the corresponding view data to a low-dimensional space based on the projection matrix of the view for each view, obtain a subspace representation matrix and an error matrix, obtain a weight matrix according to the product of the transposed matrix of the view data and the view data, and obtain local structure information according to the subspace representation matrix and the weight matrix; An optimization target construction module: used for accumulating each of the local structural information to obtain local total information, for each of the views, constructing an optimization target item according to the nuclear norm of the subspace representation matrix, the preset norm of the error matrix and the local total information, and obtaining the constraint conditions of the optimization target item according to the view data, the projection matrix, the subspace representation matrix and the error matrix; Optimization solution module: used for obtaining an optimization objective function according to the constraint conditions and the corresponding optimization objective items, and solving the optimization objective function to obtain a representation optimization matrix corresponding to each of the views; Clustering module: used for clustering data according to all the representation optimization matrices to obtain at least one clustering result; The obtaining of the constraint condition of the optimization target item according to the view data, the projection matrix, the subspace representation matrix and the error matrix comprises: Taking the product of the transposed matrix of the projection matrix and the view data as a first data item, multiplying the first data item by the subspace representation matrix, and then adding the error matrix to obtain a second data item, setting the first data item equal to the second data item, and obtaining a first constraint condition; The subspace representation matrix is ​​set to be not less than zero, thereby obtaining a second constraint condition; Taking the product of the transposed matrix of the view data and the projection matrix as a third data item, setting the product of the first data item and the third data item as a unit matrix, and obtaining a third constraint condition; The constraint condition is obtained according to the first constraint condition, the second constraint condition and the third constraint condition.

13. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the clustering method for multiple view data according to any one of claims 1 to 11 when executing the computer program.

14. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for clustering a plurality of view data according to any one of claims 1 to 11 is implemented.

Images