A matrix completion method
Update the matrix parameters through adaptive loss function terms and gradient descent method to capture the self-similarity structure inside the matrix, solve the problems of low flexibility and narrow application scope of the existing matrix completion algorithm, and achieve high-precision matrix completion Effect.
Patent Information
- Application Number
- CN202210425712.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-21
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2042-04-21
AI Technical Summary
The existing matrix completion algorithm is not flexible and has a narrow scope of application, making it difficult to effectively utilize multiple types of prior information, resulting in poor completion effect, especially when the prior information is inconsistent with the regular term.
Adaptive loss function terms are used to iteratively update the matrix W[l] and Wc through gradient descent method. The regular term tradeoff coefficients λ and μ are used to capture the self-similarity structure inside the matrix, and adapt to different types of prior information for matrix complementation. Complete.
When the prior information type is unknown, the similarity structure inside the matrix can be adaptively captured, the accuracy of matrix completion can be improved, and it is suitable for high-rank and low-rank matrices, which significantly improves the recovery effect of matrix completion.
Smart Images
Figure CN114783513B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a matrix completion method, and in particular to a matrix completion method applied to technical fields such as commodity recommendation systems, image restoration and drug-protein relationship prediction. Background Art
[0002] In many practical applications, such as product recommendation systems, image restoration, and drug-protein relationship prediction, matrices are a common form of expression. In the actual sampling, transmission, and storage process, due to measurement limitations or human interference, the data obtained is incomplete and the missing data needs to be completed. This type of problem is abstracted as the matrix completion problem.
[0003] Mathematically, the matrix completion problem is an ill-posed inverse problem. Suppose the matrix to be restored is X * ∈R m×n , the observed matrix X * The size of the elements in some positions in the matrix. Let the position index set of the observed elements be Ω, then the matrix completion problem can be expressed as: Given the position index set Ω, solve the matrix X so that the matrix X is as close to the matrix X as possible. * The matrix completion problem is ill-posed when the prior is unknown. In the prior art, the matrix X is completed * Different types of prior information need to be used when , One type of prior information is low-rank prior information, that is, the known matrix X * is a low-rank matrix. Another type of prior information is similarity prior information. For example, when the matrix X * When it is image type data, X * Each element in may be similar to its surrounding elements; in particular, when the rows of the matrix represent users, the columns represent items, and the elements represent the user's ratings on the items, users with similar ratings on observed items are more likely to give similar ratings to an item. For these two types of prior information, matrix completion is usually achieved by adding specific regularization terms. When the regularization terms and the priors of the matrix itself are inconsistent, the recovery effect will be poor.
[0004] In order to solve the problem of inconsistency between the added regularization term and the matrix itself, a deep matrix decomposition model is proposed. See the literature [1] Sanjeev Arora, Nadav Cohen, Wei Hu, & Yuping Luo (2019). Implicit Regularization in Deep Matrix Factorization Neural Information Processing Systems.
[0005] This model solution can be expressed as follows:
[0006]
[0007] Among them, X * ∈R m×n , l=0,1,...,L-1,let n 0 =m,n L =n, when l≠0, Ln l Can be any value, usually n l =max{m,n}. In this model, It will implicitly converge to a low-rank matrix X * , and since there is no need to [l] The dimension of the matrix X is restricted. * This model can obtain good completion results when the matrix X is a low-rank matrix, that is, when the low-rank prior information is known. * When it is not a low-rank matrix, this model will not impose prior information on the recovered matrix X * , we can still get good completion results. However, when the prior information is of other types, such as similarity prior information, when the above model is used for completion, the objective function cannot reflect the use of such prior information, so the completion effect is often poor. Summary of the invention
[0008] The technical problem to be solved by the present invention is: to provide a matrix completion method in view of the low flexibility and narrow application scope of the existing algorithms. The present invention can adaptively capture the self-similar structure inside the matrix and obtain a high precision completion matrix for various types of prior information.
[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is:
[0010] A matrix completion method, assuming the matrix to be completed is X * , it is known that for the matrix X * The position index set of the observed elements is Ω, which is characterized by using the following formula to solve the matrix W [l] , l = 0, 1, ..., L-1:
[0011]
[0012] Among them, λ, μ are the trade-off coefficients of the regularization term, which are determined according to the actual situation and used to weigh and The relative size between. 1 m represents a column vector of length m, W r ∈R m×m ,; Wc ∈R n×n , the operator exp(·) acts on each element of the matrix. Then the completed matrix is
[0013] Furthermore, we use gradient descent to solve the matrix W [l] , that is, iteratively update the parameters W in the model through the gradient descent method [l] , W r , W c ,Right now:
[0014]
[0015] In the above formula, W [l] (t) is the W at the tth iteration [l] The value of W r (t) is the W at the tth iteration r The value of W c (t) is the W at the tth iteration c The value of is the t-th iteration loss function The value of is the loss function To W [l] The gradient value of is the loss function To W r The gradient value of is the loss function To W c The gradient value of , η is the learning rate.
[0016] remember is the matrix completed in the tth iteration, and the iteration stopping criterion is: ||X(t * +1)-Xt*<ε. That is, according to this algorithm, the completed matrix obtained is Xt*.
[0017] The beneficial effects of the present invention are:
[0018] In view of the situation where the type of prior information is unknown in actual application scenarios, an adaptive loss function term is proposed Whether it is a high-rank matrix or a low-rank matrix, the similarity structure inside the matrix can be adaptively captured, and this similarity structure can be used as a priori for matrix completion. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 Schematic diagram of basic elements of an embodiment of the present invention.
[0020] Figure 2 Graph showing the experimental results of randomly losing the true grayscale image in an embodiment of the present invention.
[0021] Figure 3 A graph showing the experimental results of a true grayscale image with text type loss in an embodiment of the present invention.
[0022] Figure 4 Figure 1. Experimental results of real grayscale image with block loss in an embodiment of the present invention.
[0023] Figure 5 A simulation recommendation matrix diagram in an embodiment of the present invention.
[0024] Figure 6 Drug protein interaction relationship matrix diagram in the embodiments of the present invention. DETAILED DESCRIPTION
[0025] like Figure 1 As shown, including (a), (b), (c) three figures, among which: X * is a complete matrix, Ω is a position index set, that is, a set of row and column indices of all observed elements, for example, element 1 is observed at (1, 1), element 3 is observed at (1, 3), etc. The goal of matrix completion is to find X as close to the matrix X as possible. * , as shown in (c).
[0026] The present invention is further described below through specific examples, and the reconstruction results are given.
[0027] The first embodiment is an experiment of reconstructing a randomly lost real grayscale image, and the row and column values of the matrix corresponding to the image are both 240. Figure 2 The leftmost image is the real image X *; The second image from the left is the image after random loss. Each pixel in the image is lost with the same probability. The figure shows the result of randomly losing 50% of the pixels, and the corresponding position indicator set Ω is obtained; the third to seventh images from the left are the images using the algorithm KNN (see reference [2] KNN: Wang She-hu. "An Advanced K-Nearest Neighbors Matrix Completion Method Based on K-Value Self-Adaption and Local Search" Electronic Information Warfare Technology (2014): n.pag.), SVD (see reference [3] Trevor Hastie, Rahul Mazumder, Jason D. Lee, & Reza Bosagh Zadeh (2015). Matrix completion and low-rank SVD via fast alternating least squares Journal of Machine Learning Research, 16, 3367-3402.), PNMC (see reference [4] PNMC: Mingming Yang, & Songhua Xu (2020). A novel patch-based nonlinear matrix completion algorithm for image analysis through convolutional neural network Neurocomputing, 389, 5682.), DMF (see reference [5] DMF: Sanjeev Arora, Nadav Cohen, Wei Hu, & Yuping Luo (2019). Implicit Regularization in Deep Matrix Factorization Neural Information Processing Systems.), RDMF (the method in this paper) is used to complete the matrix. The rightmost image is the result obtained using the algorithm proposed in this paper.
[0028] The second embodiment is an experiment of reconstructing a text-type lost real grayscale image, and the number of rows and columns of the matrix corresponding to the image is 240. Figure 3 The left one shows the real image X *, the second from the left is the image after the text type is lost, that is, compared with the original image, the extra black part is the missing element, and the corresponding position index set Ω is obtained. The third to seventh from the left are the results of matrix completion using algorithms KNN, SVD, PNMC, DMF, and RDMF, and the first from the right is the result obtained using the algorithm proposed by the present invention.
[0029] The third embodiment is an experiment of reconstructing a real grayscale image with block loss, and the number of rows and columns of the matrix corresponding to the image is 240. Figure 4 The left one shows the real image X * , the second from the left is the image after block loss, and the corresponding position index set Ω is obtained. The third to seventh from the left are the results of matrix completion using algorithms KNN, SVD, PNMC, DMF, and RDMF, and the last column is the result obtained using the algorithm proposed by the present invention.
[0030] The fourth embodiment is to reconstruct a random missing simulated recommendation matrix data, such as Figure 5 As shown in the figure, the number of rows in the matrix is 150, the number of columns is 200, and the matrix elements are from {0, 1, 2, 3, 4, 5}. Different rows in the matrix represent different users, and different columns represent different items. The value of each element in the matrix is the score given by the user in its row to the item in its column. The figure shows the complete scoring data. However, in reality, it is impossible for every user to score every item. Now assume that every user scores an item with a probability of 20% to simulate the reality, then we can get the corresponding Ω. The results are shown in Table 1.
[0031] The fifth embodiment is to reconstruct a drug protein interaction relationship matrix, such as Figure 6 As shown in the figure, different rows of the matrix represent different drugs, different columns represent different proteins, and the element values in the matrix are the binding effects of the drugs in the row on the corresponding proteins. When the element value is 1, it means there is an effect, and when the element value is 0, it means the effect is not significant. This matrix is usually constructed through experiments, but the experimental cost is usually high. It is very practical to predict untested drug-protein pairs based on partial experiments. This problem can be abstracted as a matrix completion problem. We randomly lose 20% of the matrix element values, representing untested drug-protein pairs.
[0032] All the above embodiments calculate the matrix recovery error numerically. The specific numerical results are shown in Table 1. The numerical values in the table represent the recovered matrix X and the true value of the matrix X. * The normalized mean absolute error (NMAE) between them is defined as:
[0033]
[0034] where max X* , min X * X * The maximum and minimum values of an element, |Ω| represents the number of elements in Ω. The algorithms compared in the table are KNN[2], SVD[3], PNMC[4], and DMF[5].
[0035] The above embodiments act on three different data types, namely images, recommendation data, and drug proteins, and three different loss modes, namely random, text, and block loss. These situations are all a priori unknown. The method of the present application achieves better recovery effect than existing methods by adaptively capturing the a priori in different data types and loss modes.
[0036] Table 1
[0037]
Claims
1. A matrix completion method is applied to reconstruct grayscale images. Let the matrix corresponding to the image be the matrix to be completed X * , it is known that for the matrix X * The position index set of the observed elements is Ω, which is characterized by: Solve the matrix W using the following formula [l] , l = 0, 1, ..., L-1: Among them, λ and μ are the trade-off coefficients of the regularization term, which are determined according to the actual situation; 1 m represents a column vector of length m, W r ∈R m×m , W c ∈R n×n , the operator exp(·) acts on each element of the matrix; the completed matrix is obtained Among them, the matrix W is solved using gradient descent [l] , that is, iteratively update the parameters W in the model through the gradient descent method [l] , W r , W c ,Right now: In the above formula, W [l] (t) is the W at the tth iteration [l] The value of W r (t) is the W at the tth iteration r The value of W c (t) is the W at the tth iteration c The value of is the t-th iteration loss function The value of is the loss function To W [l] The gradient value of is the loss function To W r The gradient value of is the loss function To W c The gradient value of , η is the learning rate.
2. A matrix completion method is used to reconstruct the drug-protein interaction relationship matrix. Suppose the matrix to be completed is X * , it is known that for the matrix X * The position index set of the observed elements is Ω, which is characterized by: Solve the matrix W using the following formula [l] , l = 0, 1, ..., L-1: Among them, λ and μ are the trade-off coefficients of the regularization term, which are determined according to the actual situation; 1 m represents a column vector of length m, W r ∈R m×m , W c ∈R n×n , the operator exp(·) acts on each element of the matrix; the completed matrix is obtained Among them, the matrix W is solved using gradient descent [l] , that is, iteratively update the parameters W in the model through the gradient descent method [l] , W r , W c ,Right now: In the above formula, W [l] (t) is the W at the tth iteration [l] The value of W r (t) is the W at the tth iteration r The value of W c (t) is the W at the tth iteration c The value of is the t-th iteration loss function The value of is the loss function To W [l] The gradient value of is the loss function To W r The gradient value of is the loss function To W c The gradient value of , η is the learning rate.