A dual supergraph regularized dual deep coding-decoding conceptual decomposition method

Abstract

The embodiment of the application provides a double hypergraph regularization double deep coding-decoding concept decomposition method, and belongs to the technical field of data processing, and specifically comprises: inputting text data; constructing a document-word feature matrix according to the text data; constructing a word-side hypergraph to calculate a word-side hypergraph Laplace matrix; constructing a document-side hypergraph to calculate a document-side hypergraph Laplace matrix; defining and initializing parameters of a double hypergraph regularization double deep coding-decoding concept decomposition model; constructing an objective function, and deducing a non-negative multiplication update formula corresponding to a concept representation matrix and each layer weight matrix; alternately updating the concept representation matrix and each layer weight matrix to obtain a converged concept representation matrix; constructing a text low-dimensional representation matrix according to the converged concept representation matrix; and based on the text low-dimensional representation matrix, adopting a clustering algorithm to perform theme clustering on the text and outputting a result. Through the scheme of the application, the modeling accuracy, stability and interpretability are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a double hypergraph regularization double deep coding-decoding concept decomposition method. BACKGROUND

[0002] At present, the existing deep non-negative matrix decomposition method based on graph or hypergraph regularization mainly faces numerical data such as images, and usually only constructs a graph structure in a single perspective, for example, only a hypergraph is constructed at the sample level, and the high-order semantic relationship at the term level is underutilized. At the same time, most methods only use a single deep decomposition structure, lacking consistent constraints similar to the coding-decoding closed loop. For high-dimensional sparse text data, complex high-order co-occurrence relationships exist among documents, terms, and documents-terms, and a single hypergraph or a single deep decomposition is difficult to fully depict these structural information, and the low-dimensional representation of the document still has room for improvement in terms of theme aggregation and discrimination.

[0003] Therefore, there is an urgent need for a double hypergraph regularization double deep coding-decoding concept decomposition method that can significantly improve the accuracy and stability of text clustering and theme modeling, while enhancing the interpretability of the obtained theme representation. SUMMARY

[0004] Therefore, the embodiments of the present application provide a double hypergraph regularization double deep coding-decoding concept decomposition method, which at least partially solves some of the problems in the prior art.

[0005] The embodiments of the present application provide a double hypergraph regularization double deep coding-decoding concept decomposition method, comprising:

[0006] Step 1, input the text data to be clustered;

[0007] Step 2, construct a document-term feature matrix according to the text data, wherein the columns of the document-term feature matrix correspond to the documents, and the rows correspond to the terms;

[0008] Step 3, construct a term-side hypergraph based on the similarity between terms, and calculate the corresponding term-side hypergraph Laplacian matrix accordingly;

[0009] Step 4, construct a document-side hypergraph based on the similarity between documents, and calculate the corresponding document-side hypergraph Laplacian matrix accordingly;

[0010] Step 5, define and initialize the parameters of the double hypergraph regularization double deep coding-decoding concept decomposition model, including: setting the number of concepts, defining a non-negative concept representation matrix, constructing a double deep coding-decoding network composed of multiple linear mappings, setting the weight matrix of each layer, and non-negatively initializing the concept representation matrix and the weight matrix of each layer;

[0011] Step 6, construct the objective function of the double hypergraph regularized double deep encoding-decoding concept decomposition model, and derive the non-negative multiplication update formula corresponding to the concept representation matrix and the weight matrix of each layer;

[0012] Step 7, based on the update formula, alternately and iteratively update the concept representation matrix and the weight matrix of each layer until the objective function converges or reaches a preset maximum number of iterations, and obtain the converged concept representation matrix;

[0013] Step 8, according to the converged concept representation matrix, construct a text low-dimensional representation matrix;

[0014] Step 9, based on the text low-dimensional representation matrix, use a clustering algorithm to perform topic clustering on the text and output the result.

[0015] According to a specific implementation manner of an embodiment of the present application, the step 3 specifically comprises:

[0016] The term is taken as a vertex, a hyperedge is constructed based on the similarity between terms, an association matrix A is defined , a hyperedge weight matrix W is defined , a vertex degree matrix D is defined , and a hyperedge degree matrix D is defined , and a term-side hypergraph Laplacian matrix is calculated according to a first formula, wherein the first formula is:

[0017] .

[0018] Wherein, represents transposition.

[0019] According to a specific implementation manner of an embodiment of the present application, the step 4 specifically comprises:

[0020] The document is taken as a vertex, a hyperedge is constructed based on the similarity between documents, an association matrix A is defined , a hyperedge weight matrix W is defined , a vertex degree matrix D is defined , and a hyperedge degree matrix D is defined , and a document-side hypergraph Laplacian matrix is calculated according to a second formula, wherein the second formula is:

[0021] .

[0022] According to a specific implementation manner of an embodiment of the present application, the double deep encoding-decoding network is composed of L linear mappings, and the overall mapping matrix is defined as the product of the weight matrices of each layer, that is, .

[0023] According to a specific implementation manner of an embodiment of the present application, the expression of the objective function is:

[0024] ;

[0025] in, The overall mapping matrix is ​​composed of the weight matrices of each layer. The feature matrix of the text data to be clustered. For concept numbers, For the vertex at The representation matrix in the 3D concept space, where each column corresponds one-to-one with a vertex. For the second depth in a dual-depth encoder-decoder network The weight matrix of the layer, , For the number of network layers, These are the regularization parameters for document-side encoding consistency and word-side encoding consistency, respectively. For the regularization parameter of the closed-loop consistency term, These are the weight parameters for the hypergraph structure regularization terms on the word side and document side, respectively.

[0026] According to a specific implementation of the present invention, the step of deriving the non-negative multiplication update formulas corresponding to the conceptual representation matrix and the weight matrices of each layer includes:

[0027] Construct a Lagrange function based on the objective function, and take partial derivatives with respect to the concept representation matrix and the global mapping matrix respectively;

[0028] The gradient is decomposed into a combination of positive and negative terms;

[0029] Based on the Karush-Kuhn-Tucker conditions, the multiplicative update formulas for the concept representation matrix and the weight matrices of each layer are derived, wherein the multiplicative update formula for the concept representation matrix is:

[0030] ;

[0031] in, This indicates element-wise multiplication. and They are respectively or Lagrange functions with respect to The positive and negative terms of the gradient are split;

[0032] The multiplication update formula for the weight matrices of each layer is as follows:

[0033] ;

[0034] in, This indicates the range from level 1 to level 2. Substitution operation of the layer weight matrix, Indicates the first layer to the first Substitution operation of the layer weight matrix.

[0035] According to a specific implementation of an embodiment of the present invention, step 7 specifically includes:

[0036] Step 7.1: Given the text data and the dual hypergraph structure, determine the nonnegative concept representation matrix. and the weight matrix of each layer Perform initialization, and set the initial value of the iteration count, as well as the pre-defined maximum number of iterations and the convergence condition of the objective function;

[0037] Step 7.2, in the first iteration, while maintaining the weight matrices of each layer... Without changing the concept representation matrix, the multiplication update formula is used to update the concept representation matrix. Perform an update to obtain the updated concept representation matrix;

[0038] Step 7.3: With the updated concept representation matrix fixed, the multiplication update formula for each layer's weight matrix is ​​used to update the weight matrix of each layer sequentially. The mapping is updated, and a new overall mapping matrix is ​​obtained accordingly. ;

[0039] Step 7.4, after completing the concept representation matrix for the current round. and the weight matrix of each layer After the update, calculate the objective function at this time. The value of the objective function is compared with the value of the objective function in the previous iteration. The iteration terminates when the difference between the objective functions of two adjacent iterations meets the pre-set convergence condition, or when the number of iterations reaches the pre-set maximum number of iterations, and the converged concept representation matrix is ​​obtained. and the weight matrix of each layer Otherwise, continue to the next iteration and repeat steps 7.2 and 7.3.

[0040] According to a specific implementation of an embodiment of the present invention, step 8 specifically includes:

[0041] The convergent conceptual representation matrix The column vectors serve as low-dimensional feature representations of the corresponding text in the concept space, forming a low-dimensional text representation matrix. ,in, Low-dimensional representation matrix of text The rows correspond to the text, and the columns correspond to the conceptual dimensions.

[0042] According to a specific implementation of an embodiment of the present invention, step 9 specifically includes:

[0043] Step 9.1, generate the low-dimensional representation matrix of the text. The Row vector , represents the first text in the concept space low-dimensional feature representation;

[0044] Step 9.2, set the cluster number of text clustering to The low-dimensional feature vector of all texts As input, use Kmeans clustering algorithm to cluster and divide it, and cluster the text by minimizing the clustering function:

[0045] ;

[0046] Wherein, The cluster indication matrix of the text satisfies:

[0047] ;

[0048] Step 9.3, according to the cluster indication matrix or clustering label obtained by solving, the texts with the same clustering label are divided into the same topic category, and the clustering result of the text data is obtained.

[0049] The double hypergraph regularization double deep encoding-decoding concept decomposition scheme in the embodiment of the application, comprising: step 1, input the text data to be clustered; step 2, construct a document-word feature matrix according to the text data, wherein the columns of the document-word feature matrix correspond to the documents, and the rows correspond to the word items; step 3, construct a word-side hypergraph based on the similarity between the word items, and calculate the corresponding word-side hypergraph Laplace matrix accordingly; step 4, construct a document-side hypergraph based on the similarity between the documents, and calculate the corresponding document-side hypergraph Laplace matrix accordingly; step 5, define and initialize the parameters of the double hypergraph regularization double deep encoding-decoding concept decomposition model, including: setting the number of concepts, defining a non-negative concept representation matrix, constructing a double deep encoding-decoding network composed of multiple linear mappings, setting the weight matrix of each layer, and non-negative initializing the concept representation matrix and the weight matrix of each layer; step 6, construct the objective function of the double hypergraph regularization double deep encoding-decoding concept decomposition model, and derive the non-negative multiplication update formula corresponding to the concept representation matrix and the weight matrix of each layer; step 7, based on the update formula, alternately update the concept representation matrix and the weight matrix of each layer until the objective function converges or reaches a preset maximum iteration number, and obtain the converged concept representation matrix; step 8, construct a text low-dimensional representation matrix according to the converged concept representation matrix; step 9, based on the text low-dimensional representation matrix, use a clustering algorithm to perform topic clustering on the text and output the result.

[0050] The embodiment of the application has the following beneficial effects:

[0051] Firstly, the present application introduces a double-depth coding-decoding structure on the basis of the existing non-negative matrix factorization and concept decomposition model, extracts features of text data layer by layer through multi-layer linear mapping, and adds coding consistency constraints and closed-loop consistency constraints in the objective function, so that the concept representation can not only reconstruct the original document-word feature matrix well, but also maintain a stable and consistent structure in the coding space; therefore, compared with the traditional shallow non-negative matrix factorization or one-sided coding model, the present application can obtain a more discriminative and robust low-dimensional text concept representation.

[0052] Secondly, the present application constructs a word-side hypergraph and a document-side hypergraph, models the high-order relationships between word items and between documents in the text data, and respectively introduces word-side and document-side double hypergraph regularization terms for constraining the concept representation to maintain the manifold structure information of the original data in two perspectives; compared with the existing method using only standard graph regularization or single hypergraph constraint, the present application can fully utilize the multi-order geometric structure information of the text data in the word item space and the document space, thereby obtaining a data representation more consistent with the real semantic structure.

[0053] Thirdly, the present application organically combines the double-depth coding-decoding concept decomposition with the word-side and document-side double hypergraph regularization, realizes the joint optimization of reconstruction error, coding consistency, closed-loop consistency and double-perspective structure preservation in a unified framework, can more fully mine the potential topic structure and manifold information of the text data, significantly improves the accuracy and stability of text clustering and topic modeling, and enhances the interpretability of the obtained topic representation. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0055] Figure 1 A flowchart of a double-hypergraph regularization double-depth coding-decoding concept decomposition method provided by the embodiments of the present application. DETAILED DESCRIPTION

[0056] The embodiments of the present application will be described in detail below with reference to the drawings.

[0057] Following are the detailed description of the embodiments of the present application with specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the disclosure. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. The present application can also be implemented or applied by other different specific embodiments, and various modifications or changes can be made based on different views and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0058] It should be noted that the various aspects of the embodiments described below are within the scope of the appended claims. It should be apparent that the aspects described herein can be embodied in a wide variety of forms and that any specific structure and / or function described herein is merely illustrative. Based on the teachings herein one skilled in the art should appreciate that an aspect described herein can be implemented independently of any other aspects and that an aspect can be implemented both as any number of software running on a device and / or as an apparatus manufactured for a particular purpose. For example, an aspect can be implemented as a software routine running on a general purpose computer or be implemented as a hardware device, such as an application- specific integrated circuit ("ASIC") or a field-programmable gate array ("FPGA"). An aspect can be any combination of software and / or hardware.

[0059] It should also be noted that the figures provided in the following embodiments are only to illustrate the basic concept of the present application in a schematic manner, and only the components related to the present application are shown in the figures, not drawn according to the number, shape and size of the components when actually implemented, the actual implementation of each component can be a random change in shape, number and proportion, and the layout of the components can also be more complex.

[0060] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, one skilled in the art will understand that the described aspects can be practiced without these specific details.

[0061] With the rapid development of the Internet, social media and various online platforms, a large amount of text data is continuously generated in applications such as news recommendation, academic retrieval, public opinion analysis and intelligent question answering. How to automatically mine the hidden topic structure from a large amount of unlabeled text data has become an important research direction in the field of text mining and natural language processing. Topic modeling can provide an effective low-dimensional representation for downstream tasks such as text clustering, document retrieval and visualization analysis by representing documents as a combination of a number of latent topics.

[0062] The existing topic modeling methods mainly include a probability graph model based method and a matrix decomposition based method. A typical probability graph model such as a latent Dirichlet allocation can describe the relationship among a document, a topic and a word in a probabilistic form, but the model structure is complex, the inference cost is high, and the efficiency is low when processing a large-scale corpus. In comparison, a non-negative matrix factorization can obtain a low-dimensional topic representation of a document while maintaining interpretability by decomposing a document-word matrix into a non-negative base matrix and a representation matrix, has advantages of simple model and efficient optimization, and is therefore widely used for text clustering and topic modeling.

[0063] A traditional NMF model usually only utilizes the information of the document-word matrix itself, and ignores the geometric structure and similarity relationship among documents and among word items. For this purpose, existing work introduces a graph regularization term in NMF, can maintain the local manifold structure of samples in a low-dimensional representation space by constructing a document similarity graph and adding a Laplace regularization, and thus improves the clustering and topic modeling performance. However, most of the graph regularization NMF methods are based on a first-order adjacency relationship, only utilize simple document-document similarity, and are difficult to describe high-order semantic associations commonly existing in text data, such as co-occurrence of multiple word items in the same document and multiple documents containing a group of keywords.

[0064] The deep non-negative matrix factorization method based on graph or hypergraph regularization mainly faces image and other numerical data, usually only constructs a graph structure in a single perspective, for example, only constructs a hypergraph at the sample level, and is insufficient in utilizing high-order semantic relationships at the word item level. Meanwhile, most of the methods only adopt a single-sided deep decomposition structure, and lack consistency constraints similar to an encoding-decoding closed loop. For high-dimensional sparse text data, complex high-order co-occurrence relationships commonly exist among documents, among word items and between documents and word items, and it is difficult for a single hypergraph or a single-sided deep decomposition to fully describe these structural information, and the low-dimensional representation of the document still has room for improvement in terms of topic aggregation and discriminability.

[0065] Embodiments of the present application provide a double hypergraph regularization double deep encoding-decoding conceptual decomposition method, which can be applied to a text clustering process in a data processing scene.

[0066] Referring to Figure 1 A flowchart of a double hypergraph regularization double deep encoding-decoding conceptual decomposition method provided by embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the method mainly includes the following steps: Figure 1

[0067] Step 1, input text data to be clustered;

[0068] Step 2, construct a document-word feature matrix according to the text data, wherein the columns of the document-word feature matrix correspond to documents, and the rows correspond to word items; ​

[0069] In implementation, a document-word feature matrix can be constructed according to the text content and / or semantic features , wherein the columns of the matrix correspond to texts (documents), and the rows correspond to word items, and the element represents the weight of the th word in the th text.

[0070] Step 3, constructing a word-side hypergraph based on the similarity between word items, and calculating a corresponding word-side hypergraph Laplacian matrix;

[0071] In implementation, a word-side hypergraph can be constructed based on the similarity of word items in a feature space or a co-occurrence space, to obtain an associated matrix, a degree matrix, and calculate a corresponding word-side hypergraph Laplacian matrix .

[0072] Specifically, the vertex set of the word-side hypergraph can be denoted as , the hyperedge set can be denoted as , the associated matrix is defined, when the word item belongs to the hyperedge , let , otherwise let it be 0, and assign a weight to each hyperedge . The vertex degree and the hyperedge degree are defined as follows:

[0073] ;

[0074] Accordingly, the vertex degree matrix , the hyperedge degree matrix , and the hyperedge weight matrix are constructed, and the word-side hypergraph Laplacian matrix is defined as:

[0075] .

[0076] Step 4, constructing a document-side hypergraph based on the similarity between documents, and calculating a corresponding document-side hypergraph Laplacian matrix;

[0077] In implementation, a document-side hypergraph can be constructed based on the similarity between texts, to obtain an associated matrix, a degree matrix, and calculate a corresponding document-side hypergraph Laplacian matrix .

[0078] Specifically, the vertex set of the document-side hypergraph can be denoted as , the hyperedge set can be denoted as , the associated matrix , the hyperedge weight matrix , and the vertex degree matrix and superedge degree matrix , the definition form of document-side vertex degree, superedge degree and Laplacian matrix is the same as that of word-side hypergraph in step 3), only replace the word item vertex with the document vertex, and the following is obtained:

[0079] .

[0080] Step 5, define and initialize the parameters of the double hypergraph regularized double deep coding-decoding concept decomposition model, including: setting the number of concepts, defining the non-negative concept representation matrix, constructing the double deep coding-decoding network composed of multiple linear mappings, setting the weight matrix of each layer, and non-negative initializing the concept representation matrix and the weight matrix of each layer;

[0081] In specific implementation, define and initialize the parameters in the double deep coding-decoding concept decomposition model, specifically including:

[0082] (5.1) Set the number of concepts , define the non-negative concept representation matrix , which is used to represent the low-dimensional representation of each vertex in the concept space;

[0083] (5.2) Construct a double deep coding-decoding network composed of multiple linear mappings, set the weight matrix of each layer, and let the overall mapping matrix ;

[0084] (5.3) Non-negative random initialization or pre-training based initialization is performed on the concept representation matrix and the weight matrix of each layer, so as to satisfy the non-negative constraint; (5.4) Set the iteration control parameters, specifically including: the maximum number of iterations and the convergence threshold , the regularization parameter in the double deep coding-decoding concept decomposition model, and the double hypergraph regularization parameter , which is used to balance the weights of reconstruction error, coding consistency constraint, closed-loop consistency constraint and two kinds of hypergraph structure constraints in the objective function.

[0085] Step 6, construct the objective function of the double hypergraph regularized double deep coding-decoding concept decomposition model, and derive the non-negative multiplication update formula corresponding to the concept representation matrix and the weight matrix of each layer;

[0086] In specific implementation, the steps for obtaining the double hypergraph regularized double deep coding-decoding concept decomposition model and its update formula are as follows:

[0087] (6.1) Define the objective equation of the double hypergraph regularized double deep coding-decoding concept decomposition model, the expression is:

[0088] ;

[0089] wherein, is an overall mapping matrix composed of each layer weight matrix, is a feature matrix of text data to be clustered, is a concept number, is a representation matrix of vertices in dimensional concept space, and the matrix row corresponds to a vertex, is a weight matrix of the th layer in the dual deep encoding-decoding network, , is a network layer number, are regularization parameters of document-side and word-side encoding consistency, respectively, is a regularization parameter of closed-loop consistency, are weight parameters of word-side and document-side hypergraph structure regularization terms, respectively.

[0090] (6.2) Introduce Lagrange multipliers and , and according to the objective equation of the dual hypergraph regularized dual deep encoding-decoding concept decomposition model, the Lagrange function is obtained, and its expression is:

[0091] ;

[0092] wherein: , is a Lagrange multiplier matrix for constraining , and is a Lagrange multiplier matrix for constraining .

[0093] Define an intermediate quantity:

[0094] ;

[0095] The original objective function (plus the Lagrange multiplier) can be written as:

[0096] ;

[0097] (6.3) Partially differentiate and , respectively, and use the Karush-Kuhn-Tucker condition to obtain the update formula, and the specific process is as follows:

[0098] Fix all (equivalent to fixing and ), and partially differentiate Taking derivative, we have

[0099] ;

[0100] To get non-negative multiplication update, we write as , where and are non-negative matrices, and split the gradient by "positive term / negative term", and take a reasonable division to get

[0101] ;

[0102] ;

[0103] Then decompose the hypergraph Laplacian as , and we have , which can be easily incorporated into and . Using KKT condition , we get the update:

[0104] ;

[0105] Fixing , we write the objective equation as , and taking derivative of the objective equation with respect to we can get an overall gradient:

[0106] ;

[0107] where is the gradient of the closed loop consistent term with respect to .

[0108] Similarly, we can write it as , and construct a multiplication form of update:

[0109] ;

[0110] Subsequently, using the chain rule, we pass to each layer and define for the th layer:

[0111] ;

[0112] Then , and using the chain rule, we can get:

[0113] ;

[0114] And write this gradient as:

[0115] ;

[0116] KKT conditions with non-negative constraints , we get the multiplicative update structure of each layer:

[0117] .

[0118] Step 7, based on the update formula, alternately iteratively update the concept representation matrix and the weight matrix of each layer until the objective function converges or reaches the preset maximum number of iterations, and obtain the converged concept representation matrix;

[0119] In specific implementation, the specific process of iterative optimization using the update formula can be as follows:

[0120] (7.1) Given the text data and the double hypergraph structure, initialize the non-negative concept representation matrix and the weight matrix of each layer . And set the initial value of the iteration count, and the maximum number of iterations and the objective function convergence condition preset in advance;

[0121] (7.2) In the first iteration, keep the weight matrix of each layer unchanged, and update the non-negative concept representation matrix once using the multiplicative update formula of given in step (6.3), to obtain the updated ;

[0122] (7.3) In the case of keeping the updated unchanged, update each layer weight matrix using the multiplicative update formula of the weight matrix of each layer given in step (6.3), and obtain the new overall mapping matrix ;

[0123] (7.4) After completing the update of and each layer in the current round, calculate the value of the objective function at this time, and compare it with the value of the objective function in the last iteration; when the difference between the objective functions of adjacent iterations meets the preset convergence condition, or the number of iterations reaches the maximum number of iterations preset in advance, terminate the iteration, and obtain the converged concept representation matrix and the weight matrix of each layer , otherwise continue to the next iteration, repeat steps (7.2) and (7.3).

[0124] Step 8, constructing a text low-dimensional representation matrix according to the converged concept representation matrix;

[0125] In specific implementation, after the iteration process is terminated, the converged concept representation matrix is obtained , and the column vector of the concept representation matrix is regarded as a low-dimensional feature representation of the corresponding text in the concept (topic) space , and a text low-dimensional representation matrix is constructed , wherein the rows correspond to texts, and the columns correspond to concept (topic) dimensions.

[0126] Step 9, based on the text low-dimensional representation matrix, a clustering algorithm is used to perform topic clustering on the text and output the result.

[0127] In specific implementation, the specific process of performing kmeans clustering based on the final text low-dimensional representation matrix and outputting the result is as follows:

[0128] (9.1) Let be the final text low-dimensional representation matrix obtained in step (8), and the i-th row of the matrix is denoted as vector , representing the low-dimensional feature representation of the i-th text in the concept space;

[0129] (9.2) Set the number of clusters of text clustering to , and use the Kmeans clustering algorithm to perform clustering division on the low-dimensional feature vectors of all texts as input, and perform clustering on the texts by minimizing the objective function shown in the following formula:

[0130] ;

[0131] , wherein is a cluster indication matrix of the texts, and satisfies:

[0132] ;

[0133] (9.3) According to the obtained cluster indication matrix or clustering label, the texts with the same clustering label are divided into the same topic category, and the clustering result of the text data is obtained.

[0134] The double-hypergraph regularization double-deep encoding-decoding concept decomposition method provided in the embodiment can introduce word-side and document-side high-order structure information at the same time, and finally obtain a more robust, semantically consistent and discriminative text low-dimensional representation by using Kmeans clustering, so as to improve the performance of topic modeling and text clustering.

[0135] ​​The method of the present application will be further described below in combination with a specific embodiment, step 1) inputting text data of the text to be clustered:

[0136] In this embodiment, the Classic3 text data set is selected as the text data to be clustered, and there are 3891 English documents. After preprocessing the original documents such as word segmentation and stop word removal, 4303 words in the term set are taken as feature dimensions to obtain the preprocessed text data for constructing the document-term feature matrix.

[0137] Step 2) constructing a document-term feature matrix:

[0138] The weight of each term in each document is calculated according to the term frequency-inverse document frequency (TF-IDF) of the preprocessed text data, and a document-term feature matrix with a dimension of is obtained .

[0139] Step 3) constructing a word-side hypergraph (first hypergraph):

[0140] The 4303 words in the term set are taken as the vertices of the first hypergraph, and the 10 nearest neighbor vertices with the highest cosine similarity in the feature space are selected for each vertex to form a hyperedge. The expressions of the word-side vertex degree and the hyperedge degree are as follows:

[0141] ;

[0142] The Laplacian matrix of the word-side hypergraph is calculated as follows:

[0143] ;

[0144] Step 4) constructing a document-side hypergraph (second hypergraph):

[0145] The 3891 documents in the document data set are taken as the vertices of the second hypergraph, and the 10 vertices with the most similar structure are selected for each vertex to form a hyperedge. The document-side vertex degree, hyperedge degree and word-side hypergraph are the same, only the term vertex is replaced by the document vertex. The Laplacian matrix of the document-side hypergraph is calculated as follows:

[0146] ;

[0147] Step 5) defining and initializing the parameters of the double-depth encoding-decoding concept decomposition model:

[0148] In this embodiment, the number of concepts is taken as 3; the concept representation matrix and the weight matrix of each layer are taken as the interval initialized by non-negative initialization of uniform random numbers on [0, 1]; the two encoding consistency and closed loop consistency regularization parameters are set as The double hypergraph structure regularization term weight is set as ; in the iterative optimization process, the maximum iteration number is set as 200, and the convergence threshold is set as less than .

[0149] Step 6) Construct the objective function of the double hypergraph regularization double deep encoding-decoding concept decomposition model, and give the parameter update formula, the implementation steps are as follows:

[0150] 6(a) Construct the objective function of the double hypergraph regularization double deep encoding-decoding concept decomposition model:

[0151] ;

[0152] wherein, is the overall mapping matrix composed of weight matrices of each layer, is the feature matrix of the text data to be clustered, is the number of concepts, is the representation matrix of the vertex in the dimensional concept space, and the matrix row is one-to-one corresponding to the vertex, is the weight matrix of the layer in the double deep encoding-decoding network, , is the number of network layers, are the regularization parameters of the document side encoding consistency and the word side encoding consistency respectively, is the regularization parameter of the closed loop consistency, are the weight parameters of the word side and the document side hypergraph structure regularization terms respectively.

[0153] 6(b) According to the objective function of the double hypergraph regularization double deep encoding-decoding concept decomposition model, the Lagrange function is obtained, and its expression is as follows:

[0154] ;

[0155] wherein: is the Lagrange multiplier matrix for constraining is the Lagrange multiplier matrix for constraining .

[0156] 6(c) Use the Lagrange function ​The partial derivatives of the concept representation matrix and the weight matrix of each layer are calculated respectively, and the gradient is divided into positive and negative parts by combining the Karush-Kuhn-Tucker condition, and the multiplication update formula satisfying the non-negative constraint is derived;

[0157] The update formula of the concept representation matrix is:

[0158] ;

[0159] Wherein:

[0160] ;

[0161] ;

[0162] The update formula of the weight matrix of each layer is:

[0163] ;

[0164] Wherein:

[0165] , , ;

[0166] , , ;

[0167] ;

[0168] ;

[0169] Step 7) update the concept representation matrix and the weight matrix of each layer using the update formula when the iteration count :

[0170] (7a) update the concept representation matrix using the concept representation matrix update formula to obtain the updated concept representation matrix;

[0171] (7b) update the weight matrix of each layer using the weight matrix update formula to obtain the updated weight matrix of each layer, and obtain a new overall mapping matrix;

[0172] (7c) after completing a round of update, calculate the objective function value of the current iteration, compare it with the objective function value of the last iteration, when the difference between the objective function values of the adjacent two iterations is less than the preset threshold , or the iteration count reaches the maximum iteration number , terminate the iteration, otherwise let , repeat the above update process until the iteration termination condition is met.

[0173] Step 8) Constructing the text low-dimensional representation matrix: after the iteration process is terminated, the converged concept representation matrix is obtained , and the low-dimensional feature representation of its first column vector in the dimensional concept (topic) space is constructed to obtain the text low-dimensional representation matrix , wherein the rows correspond to the texts and the columns correspond to the concept dimensions.

[0174] Step 9) Text clustering and result output: selecting the number of clustering clusters The row vectors of the text low-dimensional representation matrix obtained in step 8 are taken as input features, and the K-means clustering algorithm is used to cluster all the texts, so that the texts with the same clustering label are divided into the same topic cluster, to obtain the topic clustering result of the Classic3 text dataset and output the result.

[0175] It should be understood that the parts of the present application can be realized by hardware, software, firmware or a combination thereof.

[0176] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A dual-hypergraph regularized dual-depth coding-decoding concept decomposition method, characterized in that, include: Step 1: Input the text data to be clustered; Step 2: Construct a document-word feature matrix based on the text data, where the columns of the document-word feature matrix correspond to documents and the rows correspond to words; Step 3: Construct a word-side hypergraph based on the similarity between terms, and calculate the corresponding Laplacian matrix of the word-side hypergraph accordingly; Step 4: Construct a document-side hypergraph based on the similarity between documents, and calculate the corresponding document-side hypergraph Laplacian matrix accordingly; Step 5: Define and initialize the parameters of the dual hypergraph regularized dual deep coding-decoding concept decomposition model, including: setting the number of concepts, defining the non-negative concept representation matrix, constructing a dual deep coding-decoding network composed of multiple layers of linear mapping, setting the weight matrix of each layer, and performing non-negative initialization on the concept representation matrix and the weight matrix of each layer. Step 6: Construct the objective function of the dual-hypergraph regularized dual-depth encoder-decoder concept decomposition model, and derive the non-negative multiplicative update formulas corresponding to the concept representation matrix and the weight matrices of each layer. The expression for the objective function is: in, The overall mapping matrix is ​​composed of the weight matrices of each layer. The feature matrix of the text data to be clustered. For concept numbers, For the vertex at The representation matrix in the 3D concept space, where each column corresponds one-to-one with a vertex. For the second depth in a dual-depth encoder-decoder network The weight matrix of the layer, , For the number of network layers, These are the regularization parameters for document-side encoding consistency and word-side encoding consistency, respectively. For closed-loop consistency, the regularization parameter is... These are the weight parameters for the hypergraph structure regularization terms on the word side and document side, respectively. Step 7: Based on the update formula, iteratively update the concept representation matrix and the weight matrix of each layer until the objective function converges or reaches the preset maximum number of iterations, and obtain the converged concept representation matrix. Step 7 specifically includes: Step 7.1: Given the text data and the dual hypergraph structure, determine the nonnegative concept representation matrix. and the weight matrix of each layer Perform initialization, and set the initial value of the iteration count, as well as the pre-defined maximum number of iterations and the convergence condition of the objective function; Step 7.2, in the first iteration, while maintaining the weight matrices of each layer... Without changing the concept representation matrix, the multiplication update formula is used to update the concept representation matrix. Perform an update to obtain the updated concept representation matrix; Step 7.3: With the updated concept representation matrix fixed, the multiplication update formula for each layer's weight matrix is ​​used to update the weight matrix of each layer sequentially. The mapping is updated, and a new overall mapping matrix is ​​obtained accordingly. ; Step 7.4, after completing the concept representation matrix for the current round. and the weight matrix of each layer After the update, calculate the objective function at this time. The value of the objective function is compared with the value of the objective function in the previous iteration. The iteration terminates when the difference between the objective functions of two adjacent iterations meets the pre-set convergence condition, or when the number of iterations reaches the pre-set maximum number of iterations, and the converged concept representation matrix is ​​obtained. and the weight matrix of each layer Otherwise, continue to the next iteration, repeating steps 7.2 and 7.3; Step 8: Construct a low-dimensional text representation matrix based on the converged concept representation matrix; Step 9: Based on the low-dimensional representation matrix of the text, a clustering algorithm is used to perform topic clustering on the text and output the results.

2. The method according to claim 1, characterized in that, Step 3 specifically includes: Using terms as vertices, construct hyperedges based on the similarity between terms, and define an association matrix. Hyperedge weight matrix Vertex degree matrix and hypermarginality matrix The Laplacian matrix of the word-side hypergraph is calculated according to the first formula, where the first formula is: in, This indicates transpose.

3. The method according to claim 2, characterized in that, Step 4 specifically includes: Using documents as vertices, construct hyperedges based on the similarity between documents, and define the association matrix. Hyperedge weight matrix Vertex degree matrix and hypermarginal matrix And calculate the document-side hypergraph Laplacian matrix according to the second formula, where the second formula is: 。 4. The method according to claim 3, characterized in that, The dual-depth encoder-decoder network consists of L layers of linear mappings, and the overall mapping matrix is ​​defined as the product of the weight matrices of each layer, i.e. .

5. The method according to claim 4, characterized in that, The steps for deriving the conceptual representation matrix and the non-negative multiplicative update formulas corresponding to each layer's weight matrix include: Construct a Lagrange function based on the objective function, and take partial derivatives with respect to the concept representation matrix and the global mapping matrix respectively; The gradient is decomposed into a combination of positive and negative terms; Based on the Karush-Kuhn-Tucker conditions, the multiplicative update formulas for the concept representation matrix and the weight matrices of each layer are derived, wherein the multiplicative update formula for the concept representation matrix is: in, This indicates element-wise multiplication. and They are respectively or Lagrange functions with respect to The positive and negative terms of the gradient are split; The multiplication update formula for the weight matrices of each layer is as follows: in, This indicates the range from level 1 to level 2. Substitution operation of the layer weight matrix, Indicates the first layer to the first Substitution operation of the layer weight matrix.

6. The method according to claim 5, characterized in that, Step 8 specifically includes: The convergent conceptual representation matrix The column vectors serve as low-dimensional feature representations of the corresponding text in the concept space, forming a low-dimensional text representation matrix. ,in, Low-dimensional representation matrix of text The rows correspond to the text, and the columns correspond to the conceptual dimensions.

7. The method according to claim 6, characterized in that, Step 9 specifically includes: Step 9.1, generate the low-dimensional representation matrix of the text. The Run-length as a vector , indicating the first Low-dimensional feature representation of a text in the concept space; Step 9.2, set the number of clusters for text clustering to... All low-dimensional feature vectors of the text As input, the K-means clustering algorithm is used to cluster the text, and the text is clustered by minimizing the clustering function: in, Let be the cluster indicator matrix of the text, satisfying: ； Step 9.3: Based on the cluster indicator matrix or cluster labels obtained by solving, the texts with the same cluster labels are divided into the same topic category to obtain the clustering results of the text data.

Images