Depth robust non-negative matrix factorization method based on alternate factorization
Through a deep robust non-negative matrix decomposition method based on alternating decomposition, combined with deep learning and non-negative matrix decomposition, the problems of insufficient deep feature mining and noise impact in SAR image feature extraction in the prior art are solved, achieving more accurate and Stable feature extraction significantly improves the recognition rate.
Patent Information
- Application Number
- CN202411263881.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-10
- Publication Date
- 2024-11-26
AI Technical Summary
The existing feature extraction methods are difficult to effectively mine deep features when processing SAR images, and are susceptible to noise and outliers, resulting in the extracted features being inaccurate and stable enough.
Using a deep robust non-negative matrix decomposition method based on alternating decomposition (l2,1-DANMF), a multi-layer feature extraction framework is constructed through deep learning, combining the alternating decomposition of the basis matrix W and the coefficient matrix H, and using the l2,1 norm constraint and Karush-Kohn-Tucker conditions, deduce the update rules for fast convergence, and gradually extract the complex information of the image.
This method can more comprehensively capture the structure and texture characteristics of the image, improve the recognition rate and stability, and combine alternating decomposition and deep learning to obtain more representative and discriminant feature descriptions, which significantly improves the accuracy of image recognition .
Smart Images

Figure CN119027682A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image recognition, and in particular relates to a deep robust non-negative matrix decomposition method based on alternating decomposition. Background Art
[0002] With the continuous development of Synthetic Aperture Radar (SAR) systems, remote sensing data is becoming increasingly abundant, making the accurate interpretation of SAR images more and more critical. In the field of image recognition, feature extraction is a key step, the purpose of which is to extract the most representative and discriminative features from complex and high-dimensional raw data, which helps to overcome the "dimensionality curse" problem faced by classification and its training process and improve subsequent recognition performance.
[0003] As a classic feature extraction method, Nonnegative Matrix Factorization (NMF) adopts a specific multiplication update strategy and introduces positive definite constraints to effectively reveal the intrinsic structure of the data. This constraint makes the decomposition results of NMF more practical and physically meaningful, and helps to improve the numerical stability and computational efficiency of the algorithm. However, the NMF algorithm is easily disturbed by noise and outliers when processing data, which may cause the algorithm to introduce erroneous features, making the extracted features inaccurate or showing instability between different data sets. This study proposes a robust nonnegative matrix factorization algorithm (RNMF), which innovatively adopts l 2,1 Compared with the traditional l2 norm constraint, the norm constraint cost function effectively reduces the adverse effects of outliers and noise on the square error calculation and improves the robustness and reliability of the algorithm.
[0004] However, NFM is a single-layer structure that focuses mainly on the shallow characteristics of the data. For rich and complex data sets, it may not be able to fully explore the direction and deep representation of the data. In contrast, deep learning methods can automatically extract multi-level representations of data by building a hierarchical model architecture, capture more complex patterns and features, and significantly improve the learning ability of the model. In this context, Deep Nonnegative Matrix Factorization (DNMF) came into being. DNMF combines the advantages of deep learning and NMF, and gradually extracts the deep features of the data through a multi-layer decomposition process. It not only retains the non-negativity and compactness of NMF, but also has the powerful representation ability of deep learning, and can more effectively process complex SAR image data.
[0005] The existing DNMF algorithm framework mainly adopts two decomposition strategies: one is to deeply explore and extract the core essential features of the image, such as texture, shape, etc.; the other focuses on the allocation of feature weights, by optimizing the combination of features to more accurately map the image content and structure. Although both methods have shown good results in different application scenarios, they only focus on the decomposition of a single matrix and fail to fully utilize the potential relationship and complementarity between matrices. In order to better mine the intrinsic structure and characteristics of the data, it is necessary to propose a more efficient deep framework, which is an important challenge in current research. Summary of the invention
[0006] The purpose of this paper is to solve the shortcomings of traditional feature extraction methods in efficiently and accurately mining deep features of SAR targets. A new deep NMF feature extraction model framework is constructed using the deep learning concept, and a deep robust non-negative matrix decomposition method based on alternating decomposition is provided. 2,1 -Constrained Deep Alternating Nonnegative MatrixFactorization, l 2,1 -DANMF).
[0007] The present invention adopts the following technical solution:
[0008] A deep robust non-negative matrix factorization method based on alternating decomposition comprises the following steps:
[0009] Step 1. Perform data processing on the existing image samples and represent them as a data matrix V. Assume that the data matrix V∈R m×n , that is, the number of samples is n, and each column represents an image sample containing m features. If the size of an image is p×p, that is, m=p 2 . Normalize V, convert the data from [0,255] to [0,1], and randomly shuffle it.
[0010] Step 2. Train the existing data matrix V and use the alternating robust non-negative matrix factorization to initialize the base matrix W, coefficient matrix H, and diagonal element matrix D for each layer. Specifically, assume that the deep framework structure has l layers, and the first layer decomposes the data matrix V. After that, when the current layer u is an odd number, the base matrix W is decomposed; when the current layer u is an even number, the coefficient matrix H is decomposed. Let the i-th layer decomposition matrix be The i-th layer decomposition matrix v i The number of rows is M i , the i-th layer decomposition matrix v i The number of columns is N i , each layer of dimension reduction is r i , the loss function of this layer is:
[0011]
[0012] Where W i represents the basis matrix of the i-th layer, H i represents the coefficient matrix of the i-th layer; according to the Karush-Kohn-Tucker (KKT) condition, we get W i and H i The update iteration rule is:
[0013]
[0014] where a=1,…,M i ; b=1,…,N i ; t=1,…,r i a is the parameter for the number of rows; b is the parameter for the number of columns, and t is the dimension r of the reduced dimension i Parameters.
[0015] The above update iteration rules are executed alternately, that is, when H i After the complete update, W i Updates.
[0016] Step 3. Based on the initial decomposition, a multi-layer deep decomposition model is constructed. The decomposition process is as follows:
[0017] V=W1H1
[0018] H1=W2H2W2=W3H3
[0019] …
[0020] W l-2 =W l-1 H l-1 H l-1 =W l H l
[0021] Assume that the deep frame is l layers, and the current layer is k layers. If k = 1, decompose the original data matrix V to obtain V = W1H1. After that, if k is an even number, decompose the coefficient matrix H, that is, H k =W k+1 H k+1 , H k is the decomposition matrix of the k-th layer; if k is an odd number, the decomposition basis matrix W is W k =W k+1 H k+1 , W k is the decomposition matrix of the k-th layer.
[0022] 1. l is an odd number
[0023] Assume l=2k-1.
[0024] (1) Loss Function
[0025]
[0026]
[0027] (2) Update rule of W
[0028] In order to better understand and derive the theory, the following parameters are introduced: P W Indicates the product of all W on the left side in the loss function, Q W Denotes the product of all W on the right side of W in the loss function, D W is obtained by taking the partial derivative of the loss function, and A H It is the product of all H in the loss function.
[0029]
[0030] in, P corresponding to the i-th W from left to right in the loss function W ; W 2y-1 Corresponding to the yth W in the loss function;
[0031]
[0032] It is the Q corresponding to the i-th W from left to right in the loss function W ;
[0033]
[0034] It is the D corresponding to the i-th W from left to right in the loss function W ; W 2i-1 represents the i-th W in the loss function, 2i-1 is the subscript, W 2i-1 Represents W of the 2i-1th layer; where i = 1, 2,…, k.
[0035] A H =H 2k-1 H 2k-2 …H4H2
[0036] Calculate F with respect to W i Partial derivative of:
[0037]
[0038] Choose the step size:
[0039]
[0040] Thus we get W 2i-1 The update rule is:
[0041]
[0042] (3) Update rule of H
[0043] In order to better understand and derive the theory, the following parameters are introduced:
[0044] P H Represents the product of all H on the left side of H in the loss function;
[0045] Q H Represents the product of all H on the right side of H in the loss function;
[0046] D H Taking the partial derivative of the loss function, we get:
[0047] and A w It is the product of all W in the loss function.
[0048]
[0049] is the P corresponding to the jth H from right to left in the loss function H ;
[0050]
[0051] Q corresponding to the jth H from right to left is the loss function H ;
[0052]
[0053] It is the D corresponding to the jth H from left to right in the loss function H ;
[0054] W 2i-1 represents the i-th W in the loss function, 2i-1 is the subscript, W 2i-1 W represents the 2i-1th layer;
[0055] A W =W1W3…W 2k-1
[0056] Where j = 1, 2,…k.
[0057] Calculate F with respect to H j Partial derivative of:
[0058]
[0059] Choose the step size:
[0060]
[0061] This gives the update rule for H:
[0062]
[0063] 2. l is an even number
[0064] Assume l=2k.
[0065] (1) Loss Function
[0066]
[0067] (2) Update rule of W
[0068] In order to better understand and derive the theory, the following parameters are introduced: P W Represents the product of all W on the left side of W in the loss function, Q W Represents the product of all W on the right side of W in the loss function, D W is the diagonal matrix obtained by taking the partial derivative of the loss function, and A H It is the product of all H in the loss function.
[0069]
[0070] in:
[0071] P corresponding to the i-th W from left to right in the loss function W ;
[0072] W 2y-1 Corresponding to the yth W in the loss function;
[0073]
[0074] It is the Q corresponding to the i-th W from left to right in the loss function W ;
[0075]
[0076] It is the D corresponding to the i-th W from left to right in the loss function W ; W 2i-1 represents the i-th W in the loss function, 2i-1 is the subscript, W 2i-1 Represents W of the 2i-1th layer; where i = 1, 2,…, k.
[0077] AH =H 2k …H4H2
[0078] Where i=1,2,…,k+1.
[0079] Calculate F with respect to W i Partial derivative of:
[0080]
[0081] Choose the step size:
[0082]
[0083] Thus we get W i The update rule is:
[0084]
[0085] (3) Update rule of H
[0086] In order to better understand and derive the theory, the following parameters are introduced:
[0087] P H Represents the product of all H on the left side of H in the loss function;
[0088] Q H Represents the product of all H on the right side of H in the loss function;
[0089] D H Taking the partial derivative of the loss function, we get:
[0090] and A W It is the product of all W in the loss function.
[0091]
[0092] is the P corresponding to the jth H from right to left in the loss function H ;
[0093]
[0094] Q corresponding to the jth H from right to left is the loss function H ;
[0095]
[0096] It is the D corresponding to the jth W from left to right in the loss function H ; where i = 1, 2,…, k.
[0097] AW =W1W3…W 2k-1 W 2k
[0098] Where j = 1, 2,…, k.
[0099] Calculate F with respect to H j Partial derivative of:
[0100]
[0101] Choose the step size:
[0102]
[0103] Thus we get H j The update rule is:
[0104]
[0105] Step 4. After the basis matrix W is updated, the training samples and test samples are further processed in the feature space.
[0106] 1. Perform classifier training and reproject all training samples:
[0107]
[0108] Among them, H train ∈R r×n is the training sample matrix V train ∈R m×n In the feature space A W Projection on . Using the feature representation H of the training set data train And the corresponding labels are used to train a support vector machine (SVM) classifier.
[0109] 2. Project the test sample:
[0110]
[0111] Among them, H test ∈R r×q is the test sample matrix V test ∈R p×q In the feature space A W The projection on the test set data is performed using the trained SVM classifier. test Perform classification to obtain predicted labels. Compare the predicted labels of the SVM classifier with the actual labels of the test set and calculate the recognition accuracy.
[0112] Beneficial effects of the present invention:
[0113] Based on the traditional robust non-negative matrix factorization, this paper proposes a deep robust non-negative matrix factorization based on alternating decomposition ( 2,1 -DANMF) method, which adopts a deep feature extraction framework of alternating decomposition. The algorithm combines deep learning and non-negative matrix factorization, and has both the multi-level feature extraction ability of deep learning and the interpretability of NMF. In the feature extraction process, the algorithm comprehensively considers the characteristics of the basis matrix W and the coefficient matrix H, and gradually extracts the complex information of the image through alternating decomposition to form a multi-level feature representation. This can not only capture the structure and texture characteristics of the image more comprehensively, but also optimize the features by adjusting the weights to obtain a more representative and discriminative feature description. On the basis of satisfying the KKT conditions, l 2,1 -DANMF algorithm uses the approximate solution obtained based on gradient descent to derive an update rule algorithm with fast convergence.
[0114] Compared with the existing feature extraction methods, the present invention makes full use of the potential relationship and complementary properties between the basis matrix W and the coefficient matrix H, better mines the intrinsic structure and characteristics of the data, and improves the recognition rate and stability. BRIEF DESCRIPTION OF THE DRAWINGS
[0115] Figure 1 Image display of MSTAR target slice reading for 2S1 self-propelled artillery.
[0116] Figure 2 Image display of MSTAR target slice reading for BMP2 infantry fighting vehicle.
[0117] Figure 3 Image display of MSTAR target slice reading for BRDM reconnaissance vehicle.
[0118] Figure 4 Image display of MSTAR target slice reading for BTR60 armored vehicle.
[0119] Figure 5 Image display of MSTAR target slice reading for BTR70 armored vehicle.
[0120] Figure 6 Image display of slice reading for D7 bulldozer MSTAR target.
[0121] Figure 7 Image display of MSTAR target slice reading for T62 tank.
[0122] Figure 8 Image display of MSTAR target slice reading for T72 tank.
[0123] Fig. 9 Image display of MSTAR target slice reading for ZIL131 truck.
[0124] Fig.10 Image display for the ZSU_23 anti-aircraft gun MSTAR target slice reading.
[0125] Fig.11 The recognition rate statistics of five methods for SAR ten types of target recognition tasks with different data scales.
[0126] Fig.12 The recognition rate statistics of the three algorithms in the ablation experiment for different data scale recognition tasks.
[0127] Fig.13 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION
[0128] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0129] like Fig.13 As shown, a deep robust non-negative matrix factorization method based on alternating decomposition includes the following steps:
[0130] Step 1. Perform data processing on the existing image samples and represent them as a data matrix V. Assume that the data matrix V∈R m×n , that is, the number of samples is n, and each column represents an image sample containing m features. If the size of an image is p×p, that is, m=p 2 . Normalize V, convert the data from [0,255] to [0,1], and randomly shuffle it.
[0131] Step 2. Train the existing data matrix V and use the alternating robust non-negative matrix factorization to initialize the base matrix W, coefficient matrix H, and diagonal element matrix D for each layer. Specifically, assume that the deep framework structure has l layers, and the first layer decomposes the data matrix V. After that, when the current layer u is an odd number, the base matrix W is decomposed; when the current layer u is an even number, the coefficient matrix H is decomposed. Let the i-th layer decomposition matrix be The i-th layer decomposition matrix v i The number of rows is M i , the i-th layer decomposition matrix v i The number of columns is N i , each layer of dimension reduction is r i , the loss function of this layer is:
[0132]
[0133] Where W i represents the basis matrix of the i-th layer, H i represents the coefficient matrix of the i-th layer; according to the Karush-Kohn-Tucker (KKT) condition, we get W i and H i The update iteration rule is:
[0134]
[0135] where a=1,…,M i ; b=1,…,N i ; t=1,…,r i a is the parameter for the number of rows; b is the parameter for the number of columns, and t is the dimension r of the reduced dimension i Parameters.
[0136] The above update iteration rules are executed alternately, that is, when H i After the complete update, W i Updates.
[0137] Step 3. Based on the initial decomposition, a multi-layer deep decomposition model is constructed. The decomposition process is as follows:
[0138] V=W1H1
[0139] H1=W2H2 W2=W3H3
[0140] …
[0141] W l-2 =W l-1 H l-1 H l-1 =W l H l
[0142] Assume that the deep frame is l layers, and the current layer is k layers. If k = 1, decompose the original data matrix V to obtain V = W1H1. After that, if k is an even number, decompose the coefficient matrix H, that is, H k =W k+1 H k+1 , H k is the decomposition matrix of the k-th layer; if k is an odd number, the decomposition basis matrix W is W k =W k+1 H k+1 , W k is the decomposition matrix of the k-th layer.
[0143] 1. l is an odd number
[0144] Assume l=2k-1.
[0145] (1) Loss Function
[0146]
[0147] (2) Update rule of W
[0148] In order to better understand and derive the theory, the following parameters are introduced: P W Indicates the product of all W on the left side in the loss function, Q W Denotes the product of all W on the right side of W in the loss function, D W is the diagonal matrix obtained by taking the partial derivative of the loss function, and A H It is the product of all H in the loss function.
[0149]
[0150] in, P corresponding to the i-th W from left to right in the loss function W ; W 2y-1 Corresponding to the yth W in the loss function;
[0151]
[0152] is the Q corresponding to the i-th W from left to right in the loss function W ;
[0153]
[0154] It is the D corresponding to the i-th W from left to right in the loss function W ; W 2i-1 represents the i-th W in the loss function, 2i-1 is the subscript, W 2i-1 Represents W of the 2i-1th layer; where i = 1, 2,…, k.
[0155] A H =H 2k-1 H 2k-2 …H4H2
[0156] Calculate F with respect to W i Partial derivative of:
[0157]
[0158] Choose the step size:
[0159]
[0160] Thus we get W 2i-1 The update rule is:
[0161]
[0162] (3) Update rule of H
[0163] In order to better understand and derive the theory, the following parameters are introduced:
[0164] P H Represents the product of all H on the left side of H in the loss function;
[0165] Q H Represents the product of all H on the right side of H in the loss function;
[0166] D H Taking the partial derivative of the loss function, we get:
[0167] and A w It is the product of all W in the loss function.
[0168]
[0169] is the P corresponding to the jth H from right to left in the loss function H ;
[0170]
[0171] Q corresponding to the jth H from right to left is the loss function H ;
[0172]
[0173] It is the D corresponding to the jth H from left to right in the loss function H ; where j = 1, 2,…k.
[0174] A W =W1W3…W 2k-1
[0175] Calculate F with respect to H j Partial derivative of:
[0176]
[0177] Choose the step size:
[0178]
[0179] This gives the update rule for H:
[0180]
[0181] 2. l is an even number
[0182] Assume l=2k.
[0183] (1) Loss Function
[0184]
[0185] (2) Update rule of W
[0186] In order to better understand and derive the theory, the following parameters are introduced: P W Indicates the product of all W on the left side in the loss function, Q W Denotes the product of all W on the right side of W in the loss function, D W is the diagonal matrix obtained by taking the partial derivative of the loss function, and A H It is the product of all H in the loss function.
[0187]
[0188] in, P corresponding to the i-th W from left to right in the loss function W ; W 2y-1 Corresponding to the yth W in the loss function;
[0189]
[0190] It is the Q corresponding to the i-th W from left to right in the loss function W ;
[0191]
[0192] It is the D corresponding to the i-th W from left to right in the loss function W ; W 2i-1 represents the i-th W in the loss function, 2i-1 is the subscript, W 2i-1 Represents W of the 2i-1th layer; where i = 1, 2,…, k+1.
[0193] A H =H 2k …H4H2
[0194] Calculate F with respect to W i Partial derivative of:
[0195]
[0196] Choose the step size:
[0197]
[0198] Thus we get W i The update rule is:
[0199]
[0200] (3) Update rule of H
[0201] In order to better understand and derive the theory, the following parameters are introduced:
[0202] P H Represents the product of all H on the left side of H in the loss function;
[0203] Q H Represents the product of all H on the right side of H in the loss function;
[0204] D H Taking the partial derivative of the loss function, we get:
[0205] and A W It is the product of all W in the loss function.
[0206]
[0207] is the P corresponding to the jth H from right to left in the loss function H ;
[0208]
[0209] Q corresponding to the jth H from right to left is the loss function H ;
[0210]
[0211] It is the D corresponding to the i-th W from left to right in the loss function W ;
[0212] Where j = 1, 2,…, k.
[0213] A W =W1W3…W 2k-1 W 2k
[0214] Calculate F with respect to H j Partial derivative of:
[0215]
[0216] Choose the step size:
[0217]
[0218] Thus we get H j The update rule is:
[0219]
[0220] Step 4. After the basis matrix W is updated, the training samples and test samples are further processed in the feature space.
[0221] 1. Perform classifier training and reproject all training samples:
[0222]
[0223] Among them, H train ∈R r×n is the training sample matrix V train ∈R m×n In the feature space A W Projection on . Using the feature representation H of the training set data train And the corresponding labels are used to train a support vector machine (SVM) classifier.
[0224] 2. Project the test sample:
[0225]
[0226] Among them, H test ∈R r×q is the test sample matrix V test ∈R p×q In the feature space A W The projection on the test set data is performed using the trained SVM classifier. test Perform classification to obtain predicted labels. Compare the predicted labels of the SVM classifier with the actual labels of the test set and calculate the recognition accuracy.
[0227] like Figure 1-Figure 12 As shown, the present invention is further illustrated below by simulating the application of the actual algorithm by taking the MSTAR ten-category target image recognition task as an example.
[0228] Table 1 Categories and numbers of the ten types of MSTAR targets
[0229]
[0230] The experiment is based on the MSTAR dataset, and all slices are 64×64 pixel data. The training set of this experiment uses SAR target samples with a pitch angle of 17 degrees, and the test set uses SAR target samples with a pitch angle of 15 degrees. For an example of target slice reading images, see the attached Figure 1-Figure 10 .
[0231] The total number of training samples in the experiment is 2747, and the total number of test samples is 3203, including outliers of ten types of target samples. In the comparative experiment, the robust non-negative matrix factorization (RNMF), deep non-negative matrix factorization (DNMF), deep robust non-negative matrix factorization (DRNMF), l-based 2,1 The deep incremental non-negative matrix factorization of the norm (l 2,1 -DANMF) recognition accuracy of five methods under different sample data scales. In order to ensure the consistency and credibility of the experimental results, each method has been rigorously tested in 5 independent experiments, and the average value is finally used as the evaluation standard.
[0232] like Fig.11 As shown in the figure, the recognition accuracy curve of species methods under different data scales. In the trend presented in the figure, l 2,1 -DANMF's accuracy steadily improves with the increase of data scale, and its performance in feature extraction surpasses the other four methods. Its recognition accuracy is always higher than that of the other four feature extraction methods, and finally stabilizes at a high level of 94.5614%, with good generalization ability.
[0233] The present invention designs two comparison algorithms based on the depth framework and the selection of settings and update rules, namely, DARNMF, an algorithm based on the depth framework of alternating decomposition but using a multiplicative update rule, and DRNMF, an algorithm based on the depth framework of decomposition H and using a multiplicative update rule. Fig.12 As shown in Figure 1, the recognition accuracy curves of the three methods under different data scales show that the present invention maintains the best performance in the entire data range, showing the significant advantages of combining the alternating depth framework setting with the additive update rule. 2,1 -The accuracy of DANMF reaches 94.5614%, which is 0.327% higher than DARNMF and 0.7056% higher than DRNMF. The experimental results strongly support the rationality and superiority of the key decision of the design of the present invention, that is, the deep framework structure of alternating decomposition combined with additive update rule.
[0234] The experimental results confirmed that 2,1 -The superiority of DANMF in the field of feature extraction. This invention captures the structure and texture characteristics of the image more comprehensively by alternating decomposition and abstraction layer by layer, and obtains more representative and discriminative features by optimizing weight distribution. At the same time, it inherits the advantages of robust non-negative matrix decomposition, effectively reduces the negative impact of noise and outliers on the recognition results, and significantly improves the effect of image recognition.
[0235] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A deep robust non-negative matrix factorization method based on alternating decomposition, characterized in that: The steps include: Step 1. Perform data processing on the existing image samples, expressed as a data matrix V. Assume that the data matrix V∈R m×n , that is, the number of samples is n, each column represents an image sample containing m features. If the size of an image is p×p, that is, m=p 2 , normalize V, convert the data from [0, 255] to [0, 1], and randomly shuffle it; Step 2. Train the existing data matrix V and use alternating robust non-negative matrix factorization to initialize it, and get the basis matrix W, coefficient matrix H and diagonal element matrix D of each layer. Assume that the deep framework structure has l layers. The first layer decomposes the data matrix V. After that, when the current layer u is an odd number, the basis matrix W is decomposed; when the current layer u is an even number, the coefficient matrix H is decomposed; let the i-th layer decomposition matrix be The i-th layer decomposition matrix v i The number of rows is M i , the i-th layer decomposition matrix v i The number of columns is N i , each layer of dimension reduction is r i , the loss function of this layer is: According to KKT conditions, W i represents the basis matrix of the i-th layer, H i Represents the coefficient matrix of the i-th layer, and gets W i and H i The update iteration rule is: Where a=1,…,M i ; b = 1, ..., N i ; t = 1, ..., r i ; a represents the parameter of the number of rows; b represents the parameter of the number of columns, and t is the dimension r of the dimensionality reduction i Parameters; The above update iteration rules are executed alternately, that is, when H i After the complete update, W i Updates; Step 3. Based on the initial decomposition, a multi-layer deep decomposition model is constructed. The decomposition process is as follows: V=W1H1 H1=W2H2W2=W3H3 … W l-2 =W l-1 H l-1 H l-1 =W l H l Assume that the deep framework is l layers, and the current layer is k layers. If k = 1, the original data matrix V is decomposed to obtain V = W1H1. After that, if k is an even number, the coefficient matrix H is decomposed, that is, H k =W k+1 H k+1 , H k is the decomposition matrix of the k-th layer; if k is an odd number, the decomposition basis matrix W is W k =W k+1 H k+1 , W k is the decomposition matrix of the k-th layer; S301.l is an odd number Let l = 2k-1; (1) Loss Function (2) Update rule of W Introduce the following parameters: P W It represents the product of all W on the left side of W in the loss function, Q W Denotes the product of all W on the right side of W in the loss function, D W is the diagonal matrix obtained by taking the partial derivative of the loss function, and A H The product of all H in the loss function; in, P corresponding to the i-th W from left to right in the loss function W ; W 2y-1 Corresponding to the yth W in the loss function; is the Q corresponding to the i-th W from left to right in the loss function W ; It is the D corresponding to the i-th W from left to right in the loss function W ; W 2i-1 represents the i-th W in the loss function, 2i-1 is the subscript, W 2i-1 represents W of the 2i-1th layer; where i = 1, 2, ..., k; the contents in brackets represent the boundary conditions of the corresponding formula, and the matrix D is a diagonal matrix; A H =H 2k-1 H 2k-2 …H4H2 Calculate F with respect to W i Partial derivative of: Choose the step size: Thus we get W 2i-1 The update rule is: (3) Update rule of H Introduce the following parameters: P H Represents the product of all H on the left side of H in the loss function; Q H Represents the product of all H on the right side of H in the loss function; D H Taking the partial derivative of the loss function, we get: and A w It is the product of all W in the loss function; is the P corresponding to the jth H from right to left in the loss function H ; Q corresponding to the jth H from right to left is the loss function H ; It is the D corresponding to the jth H from left to right in the loss function H ; where j = 1, 2, ... k; A W =W1W3…W 2k-1 Calculate F with respect to H j Partial derivative of: Choose the step size: This gives the update rule for H: S302.l is an even number Let l = 2k; (1) Loss Function F=||V-W1W2…W 2k-1 W 2k H 2k …H4H2|| 2,1 (2) Update rule of W Introduce the following parameters: P W Represents the product of all W on the left side of W in the loss function; Q W Represents the product of all W on the right side of W in the loss function; D W It is the diagonal matrix obtained by taking the partial derivative of the loss function; and A H It is the product of all H in the loss function; P corresponding to the i-th W from left to right in the loss function W ; W 2y-1 Corresponding to the yth W in the loss function; is the Q corresponding to the i-th W from left to right in the loss function W ; It is the D corresponding to the i-th W from left to right in the loss function W ; where i = 1, 2, ..., k + 1; A H =H 2k …H4H2 Calculate F with respect to W i Partial derivative of: Choose the step size: Thus we get W i The update rule is: (3) Update rule of H Introduce the following parameters: P H Represents the product of all H on the left side of H in the loss function; Q H Represents the product of all H on the right side of H in the loss function; D H Taking the partial derivative of the loss function, we get: and A W It is the product of all W in the loss function; is the P corresponding to the jth H from right to left in the loss function H ; Q corresponding to the jth H from right to left is the loss function H ; It is the D corresponding to the jth W from left to right in the loss function H ; where j = 1, 2, ..., k; A W =W1W3…W 2k-1 W 2k Calculate F with respect to H j Partial derivative of: Choose the step size: Thus we get H j The update rule is: Step 4. After the basis matrix W is updated, the training samples and test samples are further processed in the feature space; S401. Perform classifier training and reproject all training samples: Among them, H train ∈R r×n is the training sample matrix V train ∈R m×n In the feature space A W Projection on the training set data using the feature representation H train And the corresponding labels train the SVM classifier; S402. Project the test sample: Among them, H test ∈R r×q is the test sample matrix V test ∈R p×q In the feature space A W The projection on the test set data is used to represent the feature H of the trained SVM classifier. test Perform classification to obtain predicted labels, compare the predicted labels of the SVM classifier with the actual labels of the test set, and calculate the recognition accuracy.
