Multi-modal clustering method and device based on Tucker and processing equipment

Abstract

The invention provides a Tucker-based multi-modal clustering method, a Tucker-based multi-modal clustering device and processing equipment, which are used for providing a series of specific matching schemes in a manner of optimizing related algorithms by specifically focusing on Tucker decomposition when multiple clustering processing is carried out based on tensor decomposition. Therefore, a more efficient and accurate clustering effect can be achieved for large-scale multi-source heterogeneous data, and the multi-modal application requirement of a high-quality social information network is met.

Description

Technical Field

[0001] The present application relates to the field of data analysis, and specifically to a Tucker-based multimodal clustering method, apparatus, and processing equipment. Background Art

[0002] With the explosive growth of social media platforms and the increasing connectivity of society, social information networks have become a rich source of data for data analysis. Large-scale heterogeneous data are generated from multiple sources, such as social media, news media, and social activities, which contain a large amount of valuable information. It is necessary to perform correlation analysis on these multi-source heterogeneous data.

[0003] Clustering is an important data analysis method, and various clustering methods have been proposed. However, traditional clustering methods focus on processing single-modal data and are not suitable for processing multi-source heterogeneous data. They fail to extract the knowledge and values ​​hidden in multidimensional space. Therefore, multimodal clustering technology has been introduced to address these challenges.

[0004] Current multimodal clustering methods mainly include multi-view clustering, alternating clustering, and subspace clustering. Multi-view clustering aims to effectively perform multi-view data by considering the diversity and complementarity of different views; alternating clustering aims to find all different groupings of data, making each grouping high-quality and different from the others; subspace clustering aims to find clusters in different subspaces by removing irrelevant and redundant dimensions in high-dimensional datasets.

[0005] However, most existing multimodal clustering methods are tailored for specific applications, which poses challenges in meeting the diverse needs of complex social information networks. In addition, large-scale high-dimensional data also poses unique challenges to multimodal clustering.

[0006] As a new data representation tool, tensors can not only integrate multimodal and heterogeneous data in social information networks but also uncover hidden knowledge and value through correlation analysis. Based on this, a tensor-based multi-clustering (TMC) method is proposed. This method constructs a weighted tensor based on a selected feature space and further calculates a similarity matrix by measuring the Selectable Weighted Tensor Distance (SWTD). Therefore, the TMC method can meet the needs of different scenarios and achieve good clustering results.

[0007] However, TMC methods often encounter the following two problems in big data environments. On the one hand, due to the curse of dimensionality, as the size of the tensor increases, the time and space consumption increase exponentially, and the time complexity of TMC methods increases exponentially. On the other hand, the original tensor objects formed by fusing multi-source data contain a large amount of redundant and noisy information, which will greatly affect the accuracy of the clustering results.

[0008] Tensor decomposition methods, such as CANDECOMP / PARAFAC (CP), Tucker, and tensor training (TT) decomposition, can effectively compress data by truncating unnecessary information, thereby reducing noise and alleviating the curse of dimensionality. Consequently, tensor decomposition methods have been applied in many fields and achieved outstanding results. Based on the principles of tensor decomposition methods, the TT-based multi-clustering (TTMC) method and the tensor decomposition-based multi-clustering (TDMC) method have been introduced.

[0009] The TTMC method aims to improve the efficiency or quality of multimodal clustering by decomposing high-order tensors into low-order core tensors (usually third-order) and operating these core tensors in parallel for clustering. However, the TTMC method faces a dilemma: an excessively large intermediate TT core tensor will have an adverse effect on the computational speed, while an excessively small TT core scale will reduce the clustering accuracy. In contrast, the Tucker decomposition, which is more widely used in multimodal clustering, is a noteworthy alternative. The TDMC method uses Tucker decomposition, a widely used tensor compression method, to reduce the noise of multi-source heterogeneous data when solving the similarity matrix by measuring SWTD. However, the Tucker decomposition only reduces noise by truncating small noise features (small rank), and then reconstructs the decomposition result into a tensor in the TDMC method. This clustering method only slightly improves the accuracy of clustering, but cannot alleviate the dimensionality curse because it cannot utilize Tucker-based tensor operations for clustering.

[0010] Although TMC methods can achieve higher-precision clustering of multi-source heterogeneous data compared to traditional methods and provide diverse services for social information networks by selecting appropriate feature spaces, challenges remain. TDMC and TTMC methods can improve clustering accuracy and efficiency to a certain extent by utilizing tensor decomposition to compress data size and reduce noise, but they can only address some of these challenges individually. In complex social information networks, clustering large-scale heterogeneous data with higher accuracy and efficiency to meet diverse application requirements remains extremely challenging. Furthermore, the efficiency of existing multimodal clustering methods is limited by the time complexity of computing SWTD, which primarily calculates the metric coefficient for every two elements of the clustered objects. Summary of the Invention

[0011] The present application provides a Tucker-based multimodal clustering method, apparatus, and processing equipment for performing multiple clustering processing based on tensor decomposition, specifically focusing on Tucker decomposition, and providing a series of specific supporting solutions in order to optimize the algorithms involved. In this way, a more efficient and accurate clustering effect can be achieved for large-scale multi-source heterogeneous data, meeting the high-quality multimodal application requirements of social information networks.

[0012] In a first aspect, the present application provides a Tucker-based multimodal clustering method, the method comprising:

[0013] Identify different social information networks to be analyzed for correlation;

[0014] Acquire multi-source heterogeneous data from different social information networks;

[0015] Tucker-based multi-clustering processing is carried out on multi-source heterogeneous data to obtain corresponding multimodal clustering results. In the processing process of Tucker-based multi-clustering processing, for the weight tensor learning objective, the Tucker-based multi-linear attribute weight ranking learning method and the Tucker-based multi-relational attribute weight ranking method are specifically adopted.

[0016] In a second aspect, the present application provides a Tucker-based multimodal clustering device, comprising:

[0017] a determination unit, configured to determine different social information networks to be subjected to correlation analysis;

[0018] An acquisition unit, used to acquire multi-source heterogeneous data from different social information networks;

[0019] The clustering unit is used to perform Tucker-based multi-clustering processing on multi-source heterogeneous data to obtain corresponding multimodal clustering results. In the process of Tucker-based multi-clustering processing, for the weight tensor learning objective, the Tucker-based multi-linear attribute weight ranking learning method and the Tucker-based multi-relational attribute weight ranking method are specifically adopted.

[0020] In a third aspect, the present application provides a processing device comprising a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the method provided in the first aspect of the present application or any possible implementation of the first aspect of the present application is executed.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the method provided in the first aspect of the present application or any possible implementation of the first aspect of the present application.

[0022] From the above content, it can be concluded that this application has the following beneficial effects:

[0023] Under the goal of correlation analysis of social information networks, this application focuses on Tucker decomposition when performing multimodal clustering processing based on tensor decomposition, and provides a series of specific supporting solutions in order to optimize the algorithms involved. In this way, more efficient and accurate clustering effects can be achieved for large-scale multi-source heterogeneous data, meeting the high-quality multimodal application needs of social information networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 It is a logical diagram of the TMC method;

[0026] Figure 2 A logical diagram of Tucker decomposition of a third-order tensor;

[0027] Figure 3 A schematic diagram of an application framework for Tucker-based multimodal clustering services;

[0028] Figure 4 A flowchart of the Tucker-based multimodal clustering method;

[0029] Figure 5 A logical diagram for dimensional alignment of Tucker-form associated tensors;

[0030] Figure 6 A logical diagram for summing fiber elements in Tucker form;

[0031] Figure 7 It is a logical diagram of non-zero fiber normalization;

[0032] Figure 8 It is a logical diagram of zero-fiber normalization;

[0033] Figure 9 A logical diagram for Tucker-based SWTD calculation;

[0034] Figure 10 This is an example diagram of pseudo code;

[0035] Figure 11 A schematic diagram of the structure of a Tucker-based multimodal clustering device;

[0036] Figure 12A structural diagram of the processing equipment. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0038] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can be changed in the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0039] The division of modules in this application is a logical division. In actual application, there may be other division methods. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in this application. Moreover, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed into multiple circuit modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this application.

[0040] Before introducing the Tucker-based multimodal clustering method provided by this application, the background content involved in this application is first introduced.

[0041] The Tucker-based multimodal clustering method, device and computer-readable storage medium provided in this application can be applied to processing equipment for multimodal clustering processing based on tensor decomposition, specifically focusing on Tucker decomposition to optimize the algorithms involved, and providing a series of specific supporting solutions. In this way, more efficient and accurate clustering effects can be achieved for large-scale multi-source heterogeneous data, meeting the high-quality multimodal application needs of social information networks.

[0042] The Tucker-based multimodal clustering method mentioned in this application can be implemented by a Tucker-based multimodal clustering device, or by a server, physical host, or user equipment (UE) or other processing devices that integrate the Tucker-based multimodal clustering device. The Tucker-based multimodal clustering device can be implemented in hardware or software, and the UE can be a terminal device such as a smartphone, tablet computer, laptop computer, desktop computer, or personal digital assistant (PDA). The processing device can be set up in a device cluster.

[0043] In practical applications, considering that the present application solution is mainly a clustering process carried out on the basis of ready-made multi-source heterogeneous data from social information networks, the processing equipment that executes the Tucker-based multimodal clustering method of the present application or is equipped with the corresponding application service of the Tucker-based multimodal clustering method of the present application only needs to meet the required data processing capability requirements, and the specific equipment type and equipment deployment form are relatively flexible.

[0044] If it involves real-time collection and processing of multi-source heterogeneous data or further application of clustering results, corresponding adaptability adjustments need to be made based on actual needs.

[0045] For example, the present application solution can be applied to the operator of a social information network, that is, the processing device can be the operator's backend device, so that in one or more social information networks operated by itself, a series of multi-source heterogeneous data involved in the network can be clustered to perform corresponding correlation analysis, providing convenient and accurate data support for network operation work.

[0046] Next, a brief introduction is given to the basic concepts that may be involved in the specific solution description of this application.

[0047] Specifically, this involves the basic knowledge of tensor operations in TMC, SWTD, and Tucker forms. Here, we first briefly explain some of the symbols involved in the subsequent content through the following Table 1:

[0048] Table 1-Brief description of symbols

[0049]

[0050] Next, the relevant background concepts involved in this application solution are briefly described.

[0051] The TMC method mainly includes two main processes, Figure 1 A logical diagram of the TMC method is shown. The TMC method measures the importance of attribution combinations through the multilinear attribute weight ranking (MLAW) learning algorithm, and secondly calculates the SWTD with the selected feature combination to produce multimodal clustering results according to different requirements.

[0052] SWTD is an improvement on tensor distance (TD). By introducing feature selection coefficients and weight factors into the original tensor distance, it can more flexibly select attribute combinations according to context and improve the clustering quality of the final result.

[0053] SWTD (ie d SWTD ) is calculated as follows:

[0054]

[0055] Among them, w l and w m is the vectorized weight tensor at the l and m indices T w The element, g lm is the metric coefficient, g is the metric coefficient matrix based on element distance, g lm It can reflect the intrinsic relationship between different coordinates of high-order data objects, and the measurement coefficient g lm Defined as:

[0056]

[0057] Where σ is the regularization parameter, ||p l -p m ||2 is X (i1i2…i N )(corresponding to x l )and X (i′1i′2…i′ N )(corresponding to x m ) is defined as follows:

[0058]

[0059] Among them, v j (j=1,2,…,N) is the feature space selection vector v∈{0,1} N, 1 indicates that the corresponding feature space is selected, and 0 indicates that the corresponding feature space is not selected. Then the measured distance between the two tensor objects can be obtained.

[0060] The tensor operations in the Tucker form involved in this application include Hadamard product and tensor addition. Among them, Tucker decomposition is a widely used tensor decomposition method, which can be regarded as a high-order extension of principal component analysis (PCA). This decomposition obtains a core tensor and N factor matrices. Figure 2 A logical diagram of Tucker decomposition of a third-order tensor is shown. Tucker decomposition can be implemented by high-order orthogonal iteration (HOOI), HOSVD method, etc. The relevant formulas are defined as follows:

[0061] For a tensor Its Tucker form is as follows:

[0062]

[0063] in, G (X) Known as X The core tensor of X Much smaller, because R n <<I n , Known as X The factor matrix (singular value matrix) of .

[0064] Tensor Hadamard product in Tucker form:

[0065] For two tensors and The Tucker form is as above, The Tucker form is as follows:

[0066]

[0067] Then, X and Y Their Hadamard product in Tucker format is defined as:

[0068]

[0069] in, G (Z) and It can be calculated by the following formula:

[0070]

[0071] in, is the Kronecker product operation of two tensors, Represents a Kronecer product operation on two tensors along rank 1.

[0072] Tensor addition in Tucker form:

[0073] For two tensors and Their Tucker forms are as above, and their addition is as follows:

[0074]

[0075] in, G (Z) and It can be calculated by the following formula:

[0076]

[0077] in, It is a direct sum operation of two tensors. Represents a direct sum operation on two tensors along rank 1.

[0078] Next, we will introduce the Tucker-based multimodal clustering method provided by this application.

[0079] exist Figure 3 Based on the schematic diagram of an application framework of a Tucker-based multimodal clustering service, reference Figure 4 A flow chart of a Tucker-based multimodal clustering method is shown. The Tucker-based multimodal clustering method provided in this application may specifically include the following steps S401 to S403:

[0080] Step S401, determining different social information networks to be subjected to correlation analysis;

[0081] It can be understood that the multimodal clustering processing or correlation analysis to be performed in this application is aimed at large-scale multi-source heterogeneous data from social information networks. Therefore, when executing the solution of this application, the first thing to do is to determine the data source, that is, to determine the different social information networks to be subjected to correlation analysis.

[0082] Among them, the number of social information networks involved in different social information networks is at least two, which corresponds to the data characteristics of multi-source heterogeneous data itself.

[0083] In specific operations, the determination of different social information networks can be performed manually or autonomously according to corresponding autonomous determination strategies / rules, and can be specifically processed in the form of a multimodal clustering task.

[0084] As an exemplary embodiment herein, the different social information networks of the present application may specifically relate to three types of social information networks: social space, virtual space, and physical space.

[0085] It is understandable that in actual situations, there are a large number of social information networks. In order to promote the better implementation of the present application scheme and also to achieve a better clustering effect for multi-source heterogeneous data, the social information networks involved in this application can specifically involve three types of networks: social space, virtual space and physical space. When implementing the scheme, it is preferred to consider accessing these three types of networks at the same time.

[0086] Step S402, acquiring multi-source heterogeneous data from different social information networks;

[0087] After determining the different social information networks currently being targeted, multi-source heterogeneous data that can be used for multimodal clustering processing can be obtained from these data sources.

[0088] Among them, for these social information networks, when implementing the solution of this application, it is usually pre-agreed on the acquisition of multi-source heterogeneous data for corresponding multimodal clustering processing, or the acquired multi-source heterogeneous data itself is open source, or the operators of these social information networks themselves are the owners of the equipment implementing the solution of this application.

[0089] The multi-source heterogeneous data obtained here are usually generated by the behavior of different types of users (including individual users and institutional users) in social information networks. For example, for institutional users, it may involve social media, news media and social activities.

[0090] Step S403, perform Tucker-based multimodal clustering processing on multi-source heterogeneous data to obtain corresponding multimodal clustering results. In the process of Tucker-based multimodal clustering processing, for the weight tensor learning objective, specifically adopts Tucker-based multilinear attribute weight sorting learning method and Tucker-based multi-relational attribute weight sorting method.

[0091] After obtaining the data input for multimodal clustering processing, that is, multi-source heterogeneous data, it is obvious that specific multimodal clustering processing can be carried out.

[0092] Specifically, the multimodal clustering processing carried out in this application involves the application of tensors, specifically Tucker-based multimodal clustering processing. In the prior art, Tucker-based multimodal clustering processing has corresponding clustering processing solutions based on the existing concept of tensors (with limitations in processing efficiency and processing accuracy). Therefore, this application focuses on the improvements and optimizations of this application in the processing process of Tucker-based multimodal clustering processing.

[0093] Specifically, in the process of Tucker-based multimodal clustering processing, this application introduces Tucker's multilinear attribute weight ranking learning method (TuMLAW) and Tucker-based multi-relational attribute weight ranking method (TuMRAW) to effectively calculate weight tensors with lower noise, assisting in promoting more efficient and accurate multimodal clustering effects.

[0094] Both involve the calculation of association tensors, transformation tensors, and attribute combination weight tensors in Tucker format. The main difference lies in the calculation of weight vectors.

[0095] Specifically, as an exemplary embodiment, the Tucker-based multi-linear attribute weight ranking learning method (TuMLAW) may include the following processing content:

[0096] 1) In terms of associated tensor calculations in Tucker form, all tensor objects in Tucker form can be added together through Tucker-based tensor addition, and the Tucker Rounding operation can be used to reduce the dimension;

[0097] When calculating associated tensors, the present application can add all tensor objects in Tucker form through Tucker-based tensor addition, which is proved to be equivalent to adding all original tensor objects in the previous basic knowledge introduction of tensor addition in Tucker form. Considering that continuous tensor addition operations based on Tucker will cause the dimension of the Tucker kernel to increase, the Tucker Rounding operation can also be used to reduce the dimension, which can effectively alleviate the dimensionality growth while maintaining accuracy and efficiency.

[0098] 2) In terms of the calculation of the transfer tensor in Tucker form, we first ensure that each order of the association tensor has the same dimension by expanding the order of the lower dimension, then sum each fiber of the equal-dimensional association tensor along each order, normalize the fibers of the association tensor in Tucker form, and finally obtain N transfer tensors in Tucker form. Where N is the order of the tensor object;

[0099] Specifically, when calculating the transfer tensor, the adjoint matrix corresponding to the smaller dimension of the associated tensor in the Tucker form can be padded with zero fiber operations according to the maximum dimension of the tensor object, such as Figure 5 A logical diagram showing the alignment of the dimensions of the Tucker form associated tensors is shown, and then according to Figure 6 The following is a logical diagram of the Tucker form of summing fiber elements. The sum of each fiber along each order of the obtained association tensor is calculated, and the reciprocal of the value whose sum is not 0 is assigned to the corresponding fiber of a non-zero fiber auxiliary tensor. If the fiber sum is 0, the reciprocal of the corresponding original dimension is assigned to the corresponding fiber of a zero-fiber auxiliary tensor. Secondly, the two auxiliary tensors obtained at each order are decomposed into Tucker form, and then the association tensor and the decomposed non-zero fiber auxiliary tensor are subjected to the Hadamard product operation, as shown in the following example: Figure 7 A logical diagram of non-zero fiber normalization is shown, and the result is then added to the decomposed 0-fiber auxiliary tensor, as shown in Figure 8 A logical diagram of zero-fiber normalization is shown. Finally, a transfer tensor in Tucker form is obtained along each order.

[0100] 3) In terms of Tucker form attribute combination weight tensor calculation, for each transfer tensor Perform the following operations to calculate the corresponding weight vector:

[0101] w l =α tr (l) ×1w l …× l-1 w l × l+1 w l …× N w l +(1-α)u,

[0102] Among them, α is a prime number adjustment factor, 0<α<1, u is a random transition probability vector,

[0103] The weight tensor is obtained by performing an outer product operation on the weight vector calculation result. T w , The corresponding operations are as follows:

[0104]

[0105] For the weight tensor T w Perform Tucker decomposition and get

[0106] In addition, as an exemplary embodiment, the Tucker-based multi-relationship attribute weight ranking method (TuMRAW) may include the following processing contents:

[0107] 1) Compared with the Tucker-based multi-linear attribute weight ranking learning method, the Tucker-based multi-relational attribute weight ranking method does not require alignment of the corresponding association tensors;

[0108] 2) In terms of Tucker form attribute combination weight tensor calculation, the weight vector is calculated using the following formula:

[0109] w l =α tr (l) ×1w l …× l-1 w l × l+1 w l …× N w l +(1-α)u,

[0110] The two stages here are the main differences from the above Tucker-based multi-linear attribute weight ranking learning method.

[0111] 3) Corresponding to the outer product operation and Tucker decomposition involved in the Tucker-based multilinear attribute weight ranking learning method, continue vector processing (i.e., also perform the 3) processing involved in the above Tucker-based multilinear attribute weight ranking learning method).

[0112] It can be seen that the above two embodiments, based on specific quantitative formulas, respectively provide specific implementation solutions for the Tucker-based multi-linear attribute weight ranking learning method and the Tucker-based multi-relational attribute weight ranking method specially designed for this application, which has better practical significance. The following embodiments also have this feature.

[0113] In addition, the present application solution can also be based on Tucker's SWTD, and further remove the noise and redundancy of multi-source heterogeneous data through Tucker decomposition to obtain a high-precision clustering effect.

[0114] Specifically, as an exemplary embodiment, the Tucker-based multimodal clustering process may further include calculating a similarity matrix based on Tucker's optional weighted tensor distance, and correspondingly includes the following processing content:

[0115] 1) The obtained metric coefficient matrix Converted into a tensor G of order 2N, Then perform Tucker decomposition on the tensor G to obtain the metric coefficient tensor in Tucker form. For two tensors in Tucker form and Perform subtraction (which can be done by to achieve this), and then the Hadamard product in Tucker form is obtained

[0116] 2) If Figure 9 A logical diagram of Tucker-based SWTD calculation is shown. and Perform two tensor product operations to obtain an optionally weighted tensor distance in Tucker form. The corresponding definition is:

[0117]

[0118] In this way, we get the SWTD in Tucker format Additionally, the accuracy of the Tucker decomposition can be tuned to achieve better clustering results while ensuring that the tensor distances obtained from tensor objects in both Tucker and non-Tucker formats are approximately close to an acceptable range.

[0119] For the solution setting of the embodiment here, you can also combine Figure 10 An example diagram of the pseudo code shown below is provided for a more vivid understanding.

[0120] In addition, according to the time complexity analysis, the computational cost of the metric coefficient matrix G determines the efficiency of the entire multimodal clustering. Therefore, this application also considers that the clustering efficiency can be further improved by optimizing the computational efficiency of SWTD and configuring an improved multimodal clustering method based on optimized SWTD (OSWTD's ITuMC method).

[0121] Specifically, according to the SWTD calculation formula mentioned above, the following inferences can be obtained:

[0122] When the j-order of the feature space combination vector is 0, that is, v=[1,1,…,1,0,1,…,1], v j = 0, l and l′ represent the element indices of the vectorized tensor objects a and b, respectively, and are mapped to the following coordinate relationships:

[0123]

[0124] Then g lm =g l′m , which means that when there is zero in the feature space vector, the g that needs to be calculated islm The amount will decrease, and then the formula of SWTD can be changed as follows:

[0125]

[0126] Therefore, when v≠[1,1,…,1], the time complexity of calculating SWTD is much smaller, which greatly improves the efficiency of multimodal clustering.

[0127] Correspondingly, as an exemplary embodiment, the present application calculates the similarity matrix based on Tucker's optional weighted tensor distance, and may also include the following processing content:

[0128] When the j-th order of the feature space combination vector is 0, the weighted tensor distance can be defined as:

[0129]

[0130] Among them, g lm is the metric coefficient.

[0131] After completing efficient and high-precision multimodal clustering processing and realizing correlation analysis for the current multi-source heterogeneous data, it will be possible to provide high-quality data support for the multimodal application needs of social information networks.

[0132] In this regard, the present application solution may also involve subsequent related social information network multimodal application services.

[0133] Specifically, as an exemplary embodiment, the method of the present application may further include:

[0134] Based on the multimodal clustering results, corresponding business services are carried out, among which the scope of business services specifically involves community recommendations, opinion analysis, group analysis and behavior prediction.

[0135] It is understandable that both the operational work required for decision-making by the operators of social information networks and the external analysis of the internal situation of social information networks may involve business services corresponding to different service needs, and these can be carried out through the corresponding clustering features or correlation analysis results mined through data mining, that is, multimodal clustering results.

[0136] Among the specific business services that can be involved, there may be services such as community recommendation, opinion analysis, group analysis and behavior prediction. Specific services can be added, reduced or replaced in actual applications, and adaptive adjustments can be made according to actual needs.

[0137] Finally, in general, for the above solution content, under the goal of correlation analysis of social information networks, this application focuses on Tucker decomposition when performing multimodal clustering processing based on tensor decomposition, and provides a series of specific supporting solutions in order to optimize the algorithms involved. In this way, more efficient and accurate clustering effects can be achieved for large-scale multi-source heterogeneous data, meeting the high-quality multimodal application needs of social information networks.

[0138] The above is an introduction to the Tucker-based multimodal clustering method provided in this application. In order to facilitate better implementation of the Tucker-based multimodal clustering method provided in this application, this application also provides a Tucker-based multimodal clustering device from the perspective of functional modules.

[0139] See Figure 11 , Figure 11 Schematic diagram of a Tucker-based multimodal clustering device of the present application. In the present application, the Tucker-based multimodal clustering device 1100 may specifically include the following structure:

[0140] A determining unit 1101 is configured to determine different social information networks to be subjected to correlation analysis;

[0141] An acquisition unit 1102 is configured to acquire multi-source heterogeneous data from different social information networks;

[0142] Clustering unit 1103 is used to perform Tucker-based multi-clustering processing on multi-source heterogeneous data to obtain corresponding multimodal clustering results. In the process of Tucker-based multi-clustering processing, for the weight tensor learning objective, a Tucker-based multi-linear attribute weight ranking learning method and a Tucker-based multi-relational attribute weight ranking method are specifically adopted.

[0143] In an exemplary embodiment, the Tucker-based multi-linear attribute weight ranking learning method includes the following processing content:

[0144] 1) In terms of associated tensor calculations in Tucker form, all tensor objects in Tucker form are added together through Tucker-based tensor addition, and Tucker Rounding operation is used to reduce the dimension;

[0145] 2) In terms of the calculation of the transfer tensor in Tucker form, we first ensure that each order of the association tensor has the same dimension by expanding the order of the lower dimension, then sum each fiber of the equal-dimensional association tensor along each order, normalize the fibers of the association tensor in Tucker form, and finally obtain N transfer tensors in Tucker form. Where N is the order of the tensor object;

[0146] 3) In terms of Tucker form attribute combination weight tensor calculation, for each transfer tensor Perform the following operations to calculate the corresponding weight vector:

[0147] w l =α tr (l) ×1w l …× l-1 w l × l+1 w l …× N w l +(1-α)u,

[0148] Among them, α is a prime number adjustment factor, 0<α<1, u is a random transition probability vector,

[0149] The weight tensor is obtained by performing an outer product operation on the weight vector calculation result. T w , The corresponding operations are as follows:

[0150]

[0151] For the weight tensor T w Perform Tucker decomposition and get

[0152] In another exemplary embodiment, the Tucker-based multi-relationship attribute weight ranking method includes the following processing content:

[0153] 1) Compared with the Tucker-based multi-linear attribute weight ranking learning method, the Tucker-based multi-relational attribute weight ranking method does not require alignment of the corresponding association tensors;

[0154] 2) In terms of Tucker form attribute combination weight tensor calculation, the weight vector is calculated using the following formula:

[0155] w l =α tr (l) ×1w l …× l-1 w l × l+1 w l …× N w l +(1-α)u,

[0156] 3) Continue vector processing corresponding to the outer product operation and Tucker decomposition involved in the Tucker-based multilinear attribute weight ranking learning method.

[0157] In another exemplary embodiment, the Tucker-based multi-clustering process further includes calculating a similarity matrix based on Tucker's optional weighted tensor distance, and correspondingly includes the following processing content:

[0158] 1) The obtained metric coefficient matrix Converted into a tensor G of order 2N, Then perform Tucker decomposition on the tensor G to obtain the metric coefficient tensor in Tucker form. For two tensors in Tucker form and Perform the subtraction operation and then obtain the Hadamard product in Tucker form

[0159] 2) According to the corresponding order and Perform two tensor product operations to obtain an optionally weighted tensor distance in Tucker form. The corresponding definition is:

[0160]

[0161] In yet another exemplary embodiment, the similarity matrix is ​​calculated based on Tucker's optional weighted tensor distance, further comprising the following processing:

[0162] When the j-th order of the feature space combination vector is 0, the weighted tensor distance can be defined as:

[0163]

[0164] Among them, g lm is the metric coefficient.

[0165] In another exemplary embodiment, the different social information networks specifically relate to three types of social information networks: social space, virtual space, and physical space.

[0166] In yet another exemplary embodiment, the apparatus further comprises:

[0167] The developing unit 1104 is used to develop appropriate business services based on the multimodal clustering results, wherein the scope of the business services specifically involves community recommendation, opinion analysis, group analysis and behavior prediction.

[0168] This application also provides a processing device from the perspective of hardware structure, see Figure 12, Figure 12 The present invention shows a schematic diagram of a processing device. Specifically, the present invention may include a processor 1201, a memory 1202, and an input / output device 1203. The processor 1201 is configured to execute a computer program stored in the memory 1202. Figure 1 Each step of the Tucker-based multimodal clustering method in the corresponding embodiment; or, when the processor 1201 is used to execute the computer program stored in the memory 1202, Figure 11 The memory 1202 is used to store the functions of each unit in the corresponding embodiment. Figure 1 The computer program required for the Tucker-based multimodal clustering method in the corresponding embodiment.

[0169] For example, the computer program may be divided into one or more modules / units, one or more of which are stored in the memory 1202 and executed by the processor 1201 to complete the present application. One or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in a computer device.

[0170] The processing device may include, but is not limited to, a processor 1201, a memory 1202, and an input / output device 1203. Those skilled in the art will appreciate that the illustrations are merely examples of processing devices and do not limit the processing device. The processing device may include more or fewer components than shown, or a combination of certain components, or different components. For example, the processing device may also include a network access device, a bus, etc., and the processor 1201, the memory 1202, the input / output device 1203, etc. may be connected via a bus.

[0171] The processor 1201 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor. The processor is the control center of the processing device and connects various parts of the entire device using various interfaces and lines.

[0172] Memory 1202 can be used to store computer programs and / or modules. Processor 1201 implements various functions of the computer device by running or executing the computer programs and / or modules stored in memory 1202 and accessing data stored in memory 1202. Memory 1202 may primarily include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function, and the data storage area may store data generated based on the use of the processing device. Furthermore, memory may include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0173] When the processor 1201 is used to execute the computer program stored in the memory 1202, it can specifically implement the following functions:

[0174] Identify different social information networks to be analyzed for correlation;

[0175] Acquire multi-source heterogeneous data from different social information networks;

[0176] Tucker-based multi-clustering processing is carried out on multi-source heterogeneous data to obtain corresponding multimodal clustering results. In the processing process of Tucker-based multi-clustering processing, for the weight tensor learning objective, the Tucker-based multi-linear attribute weight ranking learning method and the Tucker-based multi-relational attribute weight ranking method are specifically adopted.

[0177] Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working process of the Tucker-based multimodal clustering device, processing equipment and corresponding units described above can refer to the following: Figure 1 The description of the Tucker-based multimodal clustering method in the corresponding embodiment will not be repeated here.

[0178] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0179] To this end, the present application provides a computer-readable storage medium, which stores a plurality of instructions, which can be loaded by a processor to execute the present application as follows: Figure 1The steps of the Tucker-based multimodal clustering method in the corresponding embodiment, the specific operations can be referred to as follows Figure 1 The description of the Tucker-based multimodal clustering method in the corresponding embodiment will not be repeated here.

[0180] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0181] Due to the instructions stored in the computer readable storage medium, the present application can be executed as follows: Figure 1 The steps of the multimodal clustering method based on Tucker in the corresponding embodiment can thus be implemented as follows: Figure 1 The beneficial effects that can be achieved by the Tucker-based multimodal clustering method in the corresponding embodiment are detailed in the previous description and will not be repeated here.

[0182] The above is a detailed introduction to the Tucker-based multimodal clustering method, device, processing equipment and computer-readable storage medium provided by this application. Specific examples are used in this article to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the core idea of ​​this application; at the same time, for technical personnel in this field, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on this application.

Claims

1. A Tucker-based multimodal clustering method, characterized in that: The method comprises: Identify different social information networks to be analyzed for correlation; Acquiring multi-source heterogeneous data from the different social information networks; A Tucker-based multi-clustering process is performed on the multi-source heterogeneous data to obtain corresponding multimodal clustering results. During the processing of the Tucker-based multi-clustering process, a Tucker-based multi-linear attribute weight sorting learning method and a Tucker-based multi-relational attribute weight sorting method are specifically adopted for the weight tensor learning objective.

2. The method according to claim 1, characterized in that The Tucker-based multi-linear attribute weight ranking learning method includes the following processing contents: 1) In terms of associated tensor computation in Tucker form, all tensor objects in the Tucker form are added together by Tucker-based tensor addition, and Tucker Rounding operation is used to reduce the dimension; 2) In terms of the calculation of the transfer tensor in the Tucker form, first, by expanding the order of the lower dimension to ensure that each order of the association tensor has the same dimension, then summing along each fiber of the equal-dimensional association tensor of each order, normalizing the fibers of the association tensor in the Tucker form, and finally obtaining N transfer tensors in the Tucker form Where N is the order of the tensor object; 3) In terms of the Tucker form attribute combination weight tensor calculation, for each transfer tensor Perform the following operations to calculate the corresponding weight vector: w l =a tr (l) ×1w l …× l-1 w l × l+1 w l …× N w l +(1-α)u, Among them, α is a prime number adjustment factor, 0<α<1, u is a random transition probability vector, The weight tensor is obtained by performing an outer product operation on the weight vector calculation result. T w , The corresponding operations are as follows: For the weight tensor T w Perform Tucker decomposition and get 3. The method according to claim 2, characterized in that The Tucker-based multi-relationship attribute weight ranking method includes the following processing contents: 1) Compared with the Tucker-based multi-linear attribute weight ranking learning method, the association tensors corresponding to the Tucker-based multi-relational attribute weight ranking method do not need to be aligned; 2) In terms of the Tucker form attribute combination weight tensor calculation, the weight vector is calculated using the following formula: w l =a tr (l) ×1w l …× l-1 w l × l+1 w l …× N w l +(1-α)u, 3) Continue vector processing corresponding to the outer product operation and the Tucker decomposition involved in the Tucker-based multilinear attribute weight ranking learning method.

4. The method according to claim 1, wherein The Tucker-based multi-clustering process also includes calculating a similarity matrix based on Tucker's optional weighted tensor distance, which includes the following processing content: 1) The obtained metric coefficient matrix Converted into a tensor G of order 2N, Then for the tensor G Perform Tucker decomposition to obtain the metric coefficient tensor in Tucker form. For the two tensors in Tucker form and Perform the subtraction operation and then obtain the Hadamard product in the Tucker form 2) According to the corresponding order and Perform two tensor product operations to obtain the optionally weighted tensor distance in Tucker form. The corresponding definition is:

5. The method according to claim 4, characterized in that The similarity matrix is ​​calculated based on the Tucker's optionally weighted tensor distance, and also includes the following processing content: When the j-th order of the feature space combination vector is 0, the weighted tensor distance can be defined as: Among them, g lm is the metric coefficient.

6. The method according to claim 1, characterized in that The different social information networks specifically relate to three types of social information networks: social space, virtual space and physical space.

7. The method according to claim 1, characterized in that The method further comprises: Based on the multimodal clustering results, corresponding business services are carried out, wherein the scope of the business services specifically involves community recommendation, opinion analysis, group analysis and behavior prediction.

8. A multimodal clustering device based on Tucker, characterized in that: The device comprises: a determination unit, configured to determine different social information networks to be subjected to correlation analysis; an acquisition unit, configured to acquire multi-source heterogeneous data from the different social information networks; A clustering unit is used to perform Tucker-based multi-clustering processing on the multi-source heterogeneous data to obtain corresponding multimodal clustering results. During the processing of the Tucker-based multi-clustering processing, for the weight tensor learning objective, a Tucker-based multi-linear attribute weight ranking learning method and a Tucker-based multi-relational attribute weight ranking method are specifically adopted.

9. A processing device, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the method according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method according to any one of claims 1 to 7.

Images