Depth robust non-negative matrix factorization method based on hybrid matrix factorization

Through the deep robust non-negative matrix decomposition method based on hybrid matrix decomposition, the feature is extracted layer by layer and fine-tuned by activation function and robust non-negative matrix decomposition method, the problem of identifying accuracy fluctuations in the deep non-negative matrix decomposition method is solved, and the problem of realizing accuracy fluctuations is realized. The stable improvement of recognition rate and the robustness of feature extraction are improved.

CN119027681APending Publication Date: 2024-11-26CHENGDU UNIVERSITY OF TECHNOLOGY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411263780.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-10
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing deep non-negative matrix decomposition method will show a decrease in recognition accuracy when the number of training layers increases, making it difficult to achieve a monotonic increase in recognition accuracy with the increase of layers.

Method used

Using a deep robust non-negative matrix decomposition method based on hybrid matrix decomposition, by standardizing the original data and tanh function activation, combining robust non-negative matrix decomposition, features are extracted layer by layer, and fine-tuning and noise reduction are performed on the last layer. , to stabilize feature extraction and improve recognition accuracy.

Benefits of technology

The recognition rate is stable when the number of training layers increases, the robustness and generalization ability of feature extraction are improved, the judgment information of high-dimensional image data is retained, and the recognition rate of traditional deep non-negative matrix decomposition methods is solved. The problem of increasing the number of training layers and decreasing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027681A_ABST
    Figure CN119027681A_ABST
Patent Text Reader

Abstract

The invention relates to the field of image recognition, in particular to a depth robust non-negative matrix factorization method based on hybrid matrix factorization. Different from a traditional feature extraction model training method, feature extraction is completed by adopting a learning method of decomposing a hybrid matrix W and adding a nonlinear activation function, and the method is applied to image recognition. As a depth robust non-negative matrix factorization method, the target recognition accuracy of the W-based DRNMF algorithm is monotonically increased along with the increase of the number of samples, the problem that the recognition rate of traditional depth non-negative matrix factorization is decreased along with the increase of the number of layers is successfully solved, and the final recognition accuracy reaches 94.74%. The result shows that the algorithm has high robustness and high generalization ability, and the model stability is maintained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a deep robust non-negative matrix decomposition method based on hybrid matrix decomposition. The present invention solves the problem that the recognition accuracy of the deep non-negative matrix decomposition method decreases with the increase of the number of layers. Background Art

[0002] Automatic target recognition is a process of automatically detecting and identifying targets in images. This technology is widely used in various fields, including security monitoring, autonomous driving, medical image analysis, industrial production, etc. However, directly applying raw data to target recognition algorithms may face problems such as low model recognition performance and waste of computing resources. Therefore, feature extraction has become a key step in the automatic target recognition task.

[0003] In today's automatic target recognition field, image data generally exhibits a non-negative characteristic, that is, the pixel value range is usually limited to between 0 and 255. This characteristic makes most existing feature extraction algorithms face the problem of low interpretability. Non-negative matrix factorization (NMF), as a matrix decomposition method specifically for processing non-negative data, has shown significant advantages in such applications. By decomposing the non-negative image data matrix into the product of two non-negative matrices, NMF can not only maintain the non-negative nature of the data, but also effectively reveal its potential structure and characteristics. However, although NMF has significant advantages in processing non-negative image data, single-layer NMF is often difficult to fully capture all the features of the data due to its limited expression ability when processing complex image data, which becomes a key factor limiting its application effect.

[0004] In order to overcome the limitations of single-layer NMF in processing complex image data, researchers proposed the concept of deep non-negative matrix factorization (Deep NMF, DNMF). Deep NMF extends NMF to a multi-layer structure by introducing the idea of ​​deep learning, thereby improving the model's ability to extract features from complex data. However, most of the existing deep NMF methods are based on the decomposition of the encoding matrix H matrix. This decomposition method recombines the existing features in the mixed matrix, while the potential and deep features in the original data are not fully mined. In addition, experimental results show that as the number of layers increases, the recognition accuracy of DNMF will show a characteristic of decreasing volatility, which is contrary to the perception that model performance increases with the number of layers. Therefore, how to achieve a monotonically increasing trend in recognition accuracy as the number of training layers increases has become an urgent problem to be solved. Summary of the invention

[0005] In view of the above-mentioned problems or shortcomings, in order to solve the problem that the recognition accuracy of traditional feature extraction methods fluctuates to varying degrees when the number of training layers continues to increase, the present invention changes the strategy of traditional deep non-negative matrix decomposition method based on encoding matrix decomposition, and uses activation function to perform nonlinear mapping on the original data, and proposes a deep robust non-negative matrix factorization method based on hybrid matrix decomposition (W-Based Deep Robust NonnegativeMatrix Factorization, W-Based DRNMF).

[0006] The present invention adopts the following technical solution:

[0007] A deep robust non-negative matrix decomposition method based on hybrid matrix decomposition, comprising:

[0008] Step 1. First, standardize the existing image sample V, and then activate the sample through the tanh function to obtain the sample matrix Then, the activated samples are subjected to robust non-negative matrix decomposition to obtain the mixed matrix after the first layer of activation and the encoding matrix H1 of the first layer; in RNMF, for an activated sample matrix Each column represents a training sample with m pixels, totaling n training samples, which are decomposed into the mixed matrix after the first layer activation The encoding matrix H1∈R of the first layer r×n , where r is the number of features, W1 is the first layer mixing matrix, m is the number of pixels per image, and n is the number of samples, that is,

[0009]

[0010] Where W≥0, H≥0,

[0011] The objective function of RNMF is:

[0012]

[0013] From the KKT condition, we can know that the sufficient condition for the objective function to achieve the minimum value is:

[0014]

[0015] From this, the update formula of RNMF can be deduced as:

[0016]

[0017] Where i = 1,…,m; j = 1,…,n; D1 is a diagonal matrix, and its expression is:

[0018] D1=diag((D1) 11 ,(D1) 22 ,…,(D1) nn )

[0019]

[0020] Iterate to the termination condition to obtain the mixing matrix after the first layer is activated and the encoding matrix H1 of the first layer;

[0021] Step 2. Activate the mixed matrix of the first layer Continue to perform robust non-negative matrix decomposition to obtain the mixed matrix after the second layer is activated and the encoding matrix H2 of the second layer, and then repeatedly perform robust non-negative matrix decomposition on the activated mixing matrix of the latest layer until the preset number of layers is reached;

[0022] When the mixing matrix after the kth layer is activated When performing robust non-negative matrix factorization, the formula is:

[0023]

[0024] in and H k+1 The update formulas are:

[0025]

[0026] Where i = 1,…,m; j = 1,…,n; D k+1 is a diagonal matrix, and its expression is:

[0027] D k+1 =diag((D k+1 ) 11 ,(D k+1 ) 22 ,…,(D k+1 ) nn )

[0028]

[0029] Iterate to the termination condition to obtain the mixing matrix after the activation of the kth layer The corresponding activated mixing matrix and the encoding matrix H k+1 ;

[0030] Combining the above formulas, we can deduce that the loss function of the deep robust non-negative matrix decomposition method based on hybrid matrix decomposition is:

[0031]

[0032] Where t is the number of layers in the model, is the mixing matrix after the last layer is activated;

[0033] To reduce the mixing matrix after the activation of the kth layer The error in is calculated by fine-tuning strategy;

[0034] To reduce the last layer of mixing matrix The error in is calculated by fine-tuning strategy;

[0035] Step 3. Mixed matrix after the last layer activation After the update is completed, the training sample space is projected to the feature space;

[0036]

[0037] Among them, H train is the training sample matrix after activation Mixing matrix after activation in layer n projection;

[0038] Then, after the samples to be identified are standardized and activated, the projection of the sample space to be identified to the feature space is performed:

[0039]

[0040] Among them, H test is the sample matrix to be identified after activation Mixing matrix after activation in layer n projection;

[0041] Step 4. After feature extraction, classification training is performed to train the features H of the training samples. train For training, the feature H of the recognition sample is treated test Perform classification and identification.

[0042] Furthermore, the fine-tuning strategy is as follows: after completing the update of the kth layer in step 2, H needs to be calculated. k ,H k-1 ,…,H1, that is:

[0043] H t =H k H k-1 …H1

[0044] Then the k-th layer activated mixing matrix and the encoding matrix H of the tth layer t The robust non-negative matrix factorization algorithm was introduced and iterated 150 times to reduce the overall error caused by the increase in the number of layers.

[0045] Furthermore, the mixing matrix after the last layer activation is The denoising strategy is: Step 2 solves the mixing matrix after the last layer is activated After that, the mixed matrix after the last layer activation needs to be The robust non-negative matrix factorization algorithm is used alone for 150 iterations to reduce the mixing matrix after the last layer is activated. The impact of noise on model performance.

[0046] Beneficial effects of the present invention:

[0047] 1. Based on the traditional deep robust non-negative matrix decomposition method, the present invention proposes a learning method of hybrid matrix decomposition and adding nonlinear activation function. In the feature extraction task, the algorithm can gradually extract the potential and more essential features in the original data. At the same time, the use of activation functions can complete the nonlinear mapping of the original data, thereby further improving the model performance. As an effective feature extraction method, W-Based DRNMF not only provides high robustness and high generalization feature extraction capabilities while ensuring that high-dimensional image data can obtain as much decision information as possible, but also solves the problem that the recognition accuracy of the traditional deep non-negative matrix decomposition method decreases with the increase of the number of layers.

[0048] In summary, compared with the existing feature extraction methods, the present invention can achieve direct learning of training sample features by decomposing the mixing matrix, and compared with other NMF derivative algorithms, the present invention can show higher recognition accuracy on a new test set; it can ensure the stability of the feature extraction model while retaining effective decision information as much as possible after the image information is extracted through features; more importantly, the present invention has the characteristic of steadily improving the recognition rate when the number of training layers increases, and successfully solves the problem that the recognition rate of the existing deep non-negative matrix decomposition algorithm decreases with the increase of the number of training layers. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flow chart of the feature extraction method of the present invention.

[0050] Figure 2 Image display of BMP2 armored vehicle read for MSTAR target slice.

[0051] Figure 3 Display image of BTR70 armored vehicle read for MSTAR target slice.

[0052] Figure 4 Display image of T72 tank read for MSTAR target slice.

[0053] Figure 5Display image of 2S1 tank read for MSTAR target slice.

[0054] Figure 6 Reading BRDM tank imagery for MSTAR target slices.

[0055] Figure 7 Display image of BTR60 tank read for MSTAR target slice.

[0056] Figure 8 Display of D7 tank image reading for MSTAR target slice.

[0057] Fig. 9 Display image of T62 tank read for MSTAR target slice.

[0058] Fig.10 Reading ZIL tank imagery for MSTAR target slices.

[0059] Fig.11 Reading ZUS tank imagery for MSTAR target slices.

[0060] Fig.12 Recognition rate statistics of four methods for MSTAR SAR ten-category target recognition learning task.

[0061] Fig.13 This is a graph showing the accuracy change trend of W-Based DRNMF as the number of layers increases. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] like Figure 1 As shown, a deep robust non-negative matrix decomposition method based on hybrid matrix decomposition includes:

[0064] Step 1. First, standardize the existing image sample V, and then activate the sample through the tanh function to obtain the sample matrix Then, the activated samples are subjected to robust non-negative matrix decomposition to obtain the mixed matrix after the first layer of activation and the encoding matrix H1 of the first layer; in RNMF, for an activated sample matrix Each column represents a training sample with m pixels, totaling n training samples, which are decomposed into the mixed matrix after the first layer activation The encoding matrix H1∈R of the first layer r×n , where r is the number of features, W1 is the first layer mixing matrix, m is the number of image pixels, and n is the number of samples, that is,

[0065]

[0066] Where W≥0, H≥0,

[0067] The objective function of RNMF is:

[0068]

[0069] From the KKT condition, we can know that the sufficient condition for the objective function to achieve the minimum value is:

[0070]

[0071]

[0072] From this, the update formula of RNMF can be deduced as:

[0073]

[0074] Where i = 1,…,m; j = 1,…,n; D1 is a diagonal matrix, and its expression is:

[0075] D1=diag((D1) 11 ,(D1) 22 ,…,(D1) nn )

[0076]

[0077] Iterate to the termination condition to obtain the mixing matrix after the first layer is activated and the encoding matrix H1 of the first layer;

[0078] Step 2. Activate the mixed matrix of the first layer Continue to perform robust non-negative matrix decomposition to obtain the mixed matrix after the second layer is activated and the encoding matrix H2 of the second layer, and then repeatedly perform robust non-negative matrix decomposition on the activated mixing matrix of the latest layer until the preset number of layers is reached;

[0079] When the mixing matrix after the kth layer is activated When performing robust non-negative matrix factorization, the formula is:

[0080]

[0081] in and H k+1 The update formulas are:

[0082]

[0083] Where i = 1,…,m; j = 1,…,n; D k+1 is a diagonal matrix, and its expression is:

[0084] D k+1 =diag((D k+1 ) 11 ,(D k+1 ) 22 ,…,(D k+1 ) nn )

[0085]

[0086] Iterate to the termination condition to obtain the mixing matrix after the kth layer mixing The corresponding activated mixing matrix and the encoding matrix H k+1 ;

[0087] Combining the above formulas, we can deduce that the loss function of the deep robust non-negative matrix decomposition method based on hybrid matrix decomposition is:

[0088]

[0089] Where t is the number of layers in the model, is the mixing matrix after the last layer of activation,

[0090] H n is the encoding matrix of the last layer;

[0091] To reduce the mixing matrix after the activation of the kth layer The error in is calculated by fine-tuning strategy;

[0092] To reduce the last layer of mixing matrix The error in is calculated by fine-tuning strategy;

[0093] Step 3. Mixed matrix after the last layer activation After the update is completed, the training sample space is projected to the feature space;

[0094]

[0095] Among them, H train is the training sample matrix after activation Mixing matrix after activation in layer n projection;

[0096] Then, after the samples to be identified are standardized and activated, the projection of the sample space to be identified to the feature space is performed:

[0097]

[0098] Among them, H test is the sample matrix to be identified after activation Mixing matrix after activation in layer n projection;

[0099] Step 4. After feature extraction, classification training is performed to train the features H of the training samples. train For training, the feature H of the recognition sample is treated test Perform classification and identification.

[0100] Furthermore, the fine-tuning strategy is as follows: after completing the update of the kth layer in step 2, H needs to be calculated. k ,H k-1 ,…,H1, that is:

[0101] H t =H k H k-1 …H1

[0102] Then the k-th layer activated mixing matrix and the t-th layer encoding matrix H t The robust non-negative matrix factorization algorithm is introduced and iterated a small number of times to reduce the overall error caused by the increase in the number of layers.

[0103] Furthermore, the last layer of the mixing matrix The denoising strategy is: Step 2 solves the mixing matrix after the last layer of activation After that, the mixed matrix after the last layer activation needs to be Substitute the robust non-negative matrix factorization algorithm into it for a few iterations to reduce the mixing matrix after the last layer is activated. The impact of noise on model performance.

[0104] Table 1: Distribution of sample image data of ten types of MSTAR targets

[0105]

[0106] The samples used in the experiment are ten types of target slices from MSTAR SAR. Table 1 shows the distribution of the ten types of targets. Figure 2-Figure 11 .

[0107] The present invention aims to make DRNMF have high generalization ability and deep learning ability based on mixed matrix decomposition. Therefore, the experiment first activates the sample through a nonlinear function to enhance the nonlinearity of the model so that it has higher generalization ability. Secondly, after completing the first robust non-negative matrix decomposition, different from the indirect optimization algorithm in the existing research, which further extracts the characteristics and structure of the data by decomposing the coding matrix H to achieve deeper data analysis and processing, the present invention uses a method based on decomposing the mixed matrix W with better interpretability of the model to directly optimize the algorithm, and adds noise to the training sample and the test sample to verify its robustness. Under the same other control conditions, the training sample reflects the effect of this feature extraction through the recognition accuracy of the test sample.

[0108] The experiment sets the number of training samples without incremental learning to 2747, and the number of initial training samples with incremental learning to 200. The number of new training samples is 200 per batch (less than 200 samples are also added to the training as a group of incremental samples), and they are obtained in 13 batches. The recognition accuracy of the four methods, deep non-negative matrix factorization (DNMF), deep incremental non-negative matrix factorization (DINMF), deep incremental robust non-negative matrix factorization (DIRNMF), and deep robust non-negative matrix factorization based on hybrid matrix factorization (W-Based DRNME), is statistically analyzed after each sample is obtained. The recognition accuracy curves of the four methods as the number of samples increases are shown in the figure below. Fig.12 As shown in the figure, it can be seen that the recognition accuracy of W-Based DRNME is finally higher than that of DNMF, DINMF, and DIRNMF, and the final recognition accuracy reaches 94.73618%. In addition, in the training process of extracting features based on hybrid matrix decomposition, the learning effect of W-Based DRNME always increases with the increase of training layers and maintains a stable growth, while the learning effects of DNMF, DINMF, and DIRNMF fluctuate to varying degrees with the increase of training layers.

[0109] In addition, during each training process, the accuracy trend of W-Based DRNME with the increase of layers is shown in Fig.13 The target recognition accuracy of the W-Based DRNME algorithm always shows an upward trend with the increase of the number of samples, especially when the number of samples increases from 200 to 800, the growth rate of the accuracy slows down. Finally, the accuracy of this method reaches 94.73618%, indicating that it is stable and reliable in the feature extraction task in image recognition.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep robust non-negative matrix factorization method based on hybrid matrix decomposition, characterized in that ,include: Step 1. First, standardize the existing image sample V, and then activate the sample through the tanh function to obtain the sample matrix Then, the activated samples are subjected to robust non-negative matrix decomposition to obtain the mixed matrix after the first layer of activation and the encoding matrix H1 of the first layer; in RNMF, for an activated sample matrix Each column represents a training sample with m pixels, totaling n training samples, which are decomposed into the mixed matrix after the first layer activation The encoding matrix H1∈R of the first layer r×n , where r is the number of features, W1 is the first layer mixing matrix, m is the number of pixels per image, and n is the number of samples, that is, Where W≥0, H≥0, The objective function of RNMF is: According to the KKT condition, the sufficient condition for the objective function to achieve the minimum value is: The update formula of RNMF is derived as follows: Where i = 1,…,m; j = 1,…,n; D1 is a diagonal matrix, and its expression is: D1=diag((D1) 11 ,(D1) 22 ,…,(D1) nn ) Iterate to the termination condition to obtain the mixing matrix after the first layer is activated and the encoding matrix H1 of the first layer; Step 2. Activate the mixed matrix of the first layer Continue to perform robust non-negative matrix decomposition to obtain the mixed matrix after the second layer is activated and the encoding matrix H2 of the second layer, and then repeat the robust non-negative matrix decomposition operation on the latest layer of activated mixing matrix until the preset number of layers is reached; When the mixing matrix is ​​activated for the kth layer When performing robust non-negative matrix factorization, the formula is: in and H k+1 The update formulas are: Where i = 1,…,m; j = 1,…,n; D k+1 is a diagonal matrix, and its expression is: D k+1 =diag((D k+1 ) 11 ,(D k+1 ) 22 ,…,(D k+1 ) nn ) Iterate to the termination condition to obtain the mixed matrix after the activation of the kth layer The corresponding activated mixing matrix and the encoding matrix H k+1 ; Combining the above formulas, the loss function of the deep robust non-negative matrix decomposition method based on hybrid matrix decomposition is derived as follows: Where t is the number of layers in the model, is the mixing matrix after the last layer is activated; To reduce the k-th layer mixing matrix The error in is calculated by fine-tuning strategy; To reduce the last layer of mixing matrix The noise in the image is adjusted by fine-tuning strategy; Step 3. Mixed matrix after the last layer activation After the update is completed, the training sample space is projected to the feature space; Among them, H train is the training sample matrix after activation Mixing matrix after activation in layer n projection; Then, after the samples to be identified are standardized and activated, the projection of the sample space to be identified to the feature space is performed: Among them, H test is the sample matrix to be identified after activation Mixing matrix after activation in layer n projection; Step 4. After feature extraction, classification training is performed to train the features H of the training samples. train For training, the feature H of the recognition sample is test Perform classification and identification.

2. The method according to claim 1, characterized in that Its fine-tuning strategy is: after completing the update of the kth layer in step 2, H needs to be calculated k ,H k-1 ,…,H1, that is: H t =H k H k-1 …H1 Then the k-th layer activated mixing matrix and the encoding matrix H of the tth layer t The robust non-negative matrix factorization algorithm was introduced and iterated 150 times to reduce the overall error caused by the increase in the number of layers.

3. The method according to claim 1, characterized in that The last layer of mixing matrix The fine-tuning strategy is: Step 2 solves the mixing matrix after the last layer of activation After that, the mixed matrix after the last layer activation needs to be The robust non-negative matrix factorization algorithm is used alone for 150 iterations to reduce the mixing matrix after the last layer is activated. The impact of noise on model performance.