A probability tensor CPD algorithm based on dynamic and static Lagrange multipliers and orthogonality factors

Abstract

This invention discloses a probability tensor CPD algorithm based on dynamic and static Lagrange multipliers and orthogonal factors, comprising the following steps: (1) constructing a complex-valued tensor model; (2) preprocessing the complex-valued tensor model using the FF-SVD algorithm based on static Lagrange multipliers; (3) further updating the FF-SVD algorithm parameters based on dynamic Lagrange multipliers; (4) transforming the complex-valued tensor model into a probability model; and (5) solving the probability model using the μ-VB algorithm. This invention introduces the influence of Lagrange multipliers from both static and dynamic dimensions on existing variational Bayesian algorithms, automatically determining the tensor rank, reducing running time, and improving the accuracy of the algorithm. Furthermore, this invention is applied to wireless communication technology and linear image coding technology.

Description

Technical Field

[0001] This invention relates to the fields of wireless communication and linear image coding technology, specifically to a probability tensor CPD algorithm based on dynamic and static Lagrange multipliers and orthogonal factors. Background Technology

[0002] Tensor decomposition has become one of the most important tools in machine learning theoretical research and various real-world applications. Among all tensor decompositions, CPD is the most fundamental form. It not only provides representations of many real-world multidimensional data, but it also allows for the recovery of a unique factor matrix down to negligible scaling and permutation ambiguities. This uniqueness helps in discovering interpretable knowledge from tensor data, giving tensor CPD an advantage over other learning methods (deep models) in various data analysis tasks, including wireless communication, linear image coding, social group mining, drug discovery, biomedical data analysis, and functional magnetic resonance imaging. CPD is defined as a linear combination of a set of rank-1 tensors. To find the factor matrix in CPD, Carroll JD et al. proposed alternating least squares, fixing other factor matrices in each iteration and optimizing one factor matrix at a time. However, many applications lead to orthogonal structures in tensor models. In 2012, The algorithm initially proposed by M et al. was formulated within an orthogonal-constrained least squares framework. Implementing these algorithms requires knowledge of tensor rank. Unfortunately, extracting tensor rank from tensor data is considered a nondeterministic polynomial-time hard (NP-hard) problem. The main approach using existing tensor quantum space learning algorithms is to run multiple parallel algorithms, assuming different ranks, and selecting the model with the smallest rank best suited to the data. This trial-and-error approach is widely accepted in tensor research, but inevitably leads to a significant computational burden. Summary of the Invention

[0003] Purpose of the invention: The purpose of this invention is to provide a probability tensor CPD algorithm based on dynamic and static Lagrange multipliers and orthogonal factors, further improving the updating of hyperparameters in the posterior distribution. It delves into the impact of introducing Lagrange multipliers on existing variational Bayesian algorithms from both static and dynamic dimensions. This invention automatically determines the tensor rank, reduces running time, and improves the accuracy of the algorithm. At the same time, it applies the algorithm to wireless communication technology and linear image coding technology.

[0004] Technical solution: The probability tensor CPD algorithm based on dynamic and static Lagrange multipliers and orthogonality factors described in this invention includes the following steps:

[0005] (1) Construct a complex-valued tensor model;

[0006] (2) The FF-SVD algorithm based on static Lagrange multipliers is used to preprocess the complex-valued tensor model;

[0007] (3) Further update the FF-SVD algorithm parameters based on dynamic Lagrange multipliers;

[0008] (4) Transform the complex-valued tensor model into a probabilistic model;

[0009] (5) Solve the probability model using the μ-VB algorithm.

[0010] Furthermore, the formula for step (1) is as follows:

[0011]

[0012] in, This represents additive white Gaussian noise. And each component is independently and identically distributed; S (N) y represents a pure signal tensor, and y represents a signal tensor containing noise.

[0013] Furthermore, step (2) is as follows:

[0014] First, let:

[0015] Here, the expansion operator is defined. Represents an Nth-order tensor Expanding along the t-th mode, specifically defined as Indicates except the first One position, the rest are all 0.

[0016] Define the estimated signal matrix as: The residual matrix is

[0017] Then, constrained estimation is performed on the residual signal matrix:

[0018]

[0019] The Lagrange function is

[0020]

[0021] The solution to the constrained optimization problem satisfies the KKT conditions, as shown in the following formula:

[0022]

[0023] Based on the properties of block matrices:

[0024]

[0025]

[0026] And the following relationships:

[0027]

[0028] get:

[0029]

[0030] The final result is:

[0031]

[0032] The signal-to-noise ratio is:

[0033]

[0034] Furthermore, step (3) is as follows: the signal-to-noise ratio model of the singular value sequence is shown in the following equation:

[0035]

[0036] Where, σ i The singular values ​​after decomposition are: μ = aβ + ε, or μ = e b×β +c.

[0037] Furthermore, step (4) is as follows:

[0038] First, let the CPD tensor model be:

[0039]

[0040]

[0041] Let the probability model be:

[0042]

[0043] Among them, there are O factor matrices Mapping to the Stiefel manifold, i.e., satisfying the condition 1≤n≤O:

[0044] When 1≤n≤O, the factor matrix It can be regarded as a uniform distribution in Above, that is:

[0045]

[0046] When O+1≤n≤N, it follows a Gaussian distribution, that is:

[0047]

[0048] Secondly, set Θ to The set; the unknown factor matrix is ​​obtained from the probability model. The marginal probability density function is:

[0049]

[0050] in,

[0051]

[0052] Next, the variational distribution of the probability density function Q(Θ) is calculated using KL divergence. Represented as:

[0053]

[0054] Let the variational probability density take the form of complete decomposition: Q(Θ)=∏ t Q(Θ t ),Θ t ∈Θ; Perform constraint operations on the first O factor matrices, and assume the variational density is in the form of the Dirac function.

[0055] The probability density function Q(Θ) of the variational distribution t It can be parsed to obtain:

[0056]

[0057] and

[0058]

[0059] Furthermore, step (5) includes the following steps:

[0060] (51) Calculate the factor matrix of the orthogonal constraints; the details are as follows:

[0061]

[0062]

[0063] The derived variational density function shows that the orthogonality factor follows Matrix distribution. Using property 1, we obtain a closed-form solution to the problem.

[0064] Property 1: Assumption Matrix obey Matrix distribution, where parameters If Z = UΣV HIt is obtained through SVD of matrix Z The only mode is UV H ;

[0065] Based on property 1, the factor matrix containing orthogonal structure is estimated to obtain... Among them U (t) and V (t) It is for Z (t) The left and right orthogonal matrices obtained by performing SVD;

[0066] (52) Calculate the unconstrained factor matrix as follows:

[0067]

[0068] get:

[0069]

[0070] (53) For θ l The regularization terms are as follows:

[0071]

[0072] (54) The specific calculation of noise accuracy is as follows:

[0073]

[0074] in and

[0075]

[0076] The device of the present invention includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements a probability tensor CPD algorithm based on dynamic and static Lagrange multipliers and orthogonal factors as described in any one of the claims.

[0077] The present invention discloses a storage medium storing a computer program, characterized in that the computer program is designed to implement, at runtime, any of the probability tensor CPD algorithms based on dynamic and static Lagrange multipliers and orthogonal factors as described in the present invention.

[0078] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: it further improves the updating of hyperparameters in the posterior distribution, introduces the influence of Lagrange multipliers from both static and dynamic dimensions on the existing variational Bayes algorithm, automatically determines the tensor rank, reduces running time and improves the accuracy of the algorithm, and applies the present invention to wireless communication technology and linear image coding technology. Attached Figure Description

[0079] Figure 1 This is a flowchart of the present invention.

[0080] Figure 2 This is a performance convergence diagram based on static Lagrange multipliers of the present invention;

[0081] Figure 3 The present invention provides MSE based on different Lagrange multiplier algorithms;

[0082] Figure 4 The running time of the present invention is based on different Lagrange multiplier algorithms;

[0083] Figure 5 This invention relates to applications in wireless communication.

[0084] Figure 6 This invention relates to the application of linear image coding. Detailed Implementation

[0085] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0086] like Figure 1 As shown, this embodiment of the invention provides a probability tensor CPD algorithm based on dynamic and static Lagrange multipliers and orthogonal factors, including the following steps:

[0087] (1) Construct the complex-valued tensor model; the formula is as follows:

[0088]

[0089] in, This represents additive white Gaussian noise. And each component is independently and identically distributed; S (N) y represents a pure signal tensor, and y represents a signal tensor containing noise.

[0090] (2) The complex-valued tensor model is preprocessed using the FF-SVD algorithm based on static Lagrange multipliers; the details are as follows:

[0091] First, let:

[0092] Here, the expansion operator is defined. Represents an Nth-order tensor Expanding along the t-th mode, specifically defined as Indicates except the first One position, the rest are all 0.

[0093] Define the estimated signal matrix as: The residual matrix is

[0094] Then, constrained estimation is performed on the residual signal matrix:

[0095]

[0096] The Lagrange function is

[0097]

[0098] The solution to the constrained optimization problem satisfies the KKT conditions, as shown in the following formula:

[0099]

[0100] Based on the properties of block matrices:

[0101]

[0102]

[0103] And the following relationships:

[0104]

[0105] get:

[0106]

[0107] The final result is:

[0108]

[0109] The signal-to-noise ratio is:

[0110]

[0111] (3) Further update the FF-SVD algorithm parameters based on dynamic Lagrange multipliers; specifically as follows: The signal-to-noise ratio model of singular value sequences is shown in the following equation:

[0112]

[0113] Where, σ i The singular values ​​after decomposition are: μ = aβ + ε, or μ = e b×β +c.

[0114] (4) Transform the complex-valued tensor model into a probabilistic model; specifically as follows: First, let the CPD tensor model be:

[0115]

[0116]

[0117] Let the probability model be:

[0118]

[0119] Among them, there are O factor matrices Mapping to the Stiefel manifold, i.e., satisfying the condition 1≤n≤O:

[0120] When 1≤n≤O, the factor matrix It can be regarded as a uniform distribution in Above, that is:

[0121]

[0122] When O+1≤n≤N, it follows a Gaussian distribution, that is:

[0123]

[0124] Secondly, set Θ to The set; the unknown factor matrix is ​​obtained from the probability model. The marginal probability density function is:

[0125]

[0126] in,

[0127]

[0128] Next, the variational distribution of the probability density function Q(Θ) is calculated using KL divergence. Represented as:

[0129]

[0130] Suppose the variational probability density is in the form of complete decomposition. The first O factor matrices are constrained, and the variational density is assumed to be in the form of the Dirac function.

[0131] probability density function of variational distribution It can be parsed to obtain:

[0132]

[0133] and

[0134]

[0135] (5) Solve the probabilistic model using the μ-VB algorithm. This includes the following steps:

[0136] (51) Calculate the factor matrix of the orthogonal constraints; the details are as follows:

[0137]

[0138]

[0139] The derived variational density function shows that the orthogonality factor follows Matrix distribution. Using Property 1 below, a closed-form solution to the problem can be obtained.

[0140] Property 1: Assumption Matrix obey Matrix distribution, where parameters If Z = UΣV H It is obtained through the SVD of matrix Z, then The only mode is UV H .

[0141] Based on the principle of Property 1, factor matrices containing orthogonal structures can be estimated to obtain... Among them U (t) and V (t) It is for Z (t) The left and right orthogonal matrices obtained by performing SVD.

[0142] (52) Calculate the unconstrained factor matrix as follows:

[0143]

[0144] get:

[0145]

[0146] (53) For θ l The regularization terms are as follows:

[0147]

[0148] (54) The specific calculation of noise accuracy is as follows:

[0149]

[0150] in and

[0151]

[0152] The theoretical framework of this invention is shown in Table 1.

[0153] Table 1

[0154]

[0155] Figure 2The final visualization of the μ-VB algorithm is shown, where (1)-(3) are Hinton plots of the factor matrix, with 25 components effectively pruned and the tensor rank correctly estimated to be 5. The significant differences between the Hinton plots of the factor matrix under orthogonal structural constraints and without constraints are shown. (1)-(2) are without orthogonal constraints, with each column appearing very compact; (3) is with orthogonal constraints, with each column having a larger spacing. Figure 2 (4) represents the posterior distribution mean of the regularization term, i.e., the tensor rank is automatically determined; Figure 2 (5) represents the minimum mean squared error (MSE) value in each iteration, indicating that the μ-VB algorithm has a relatively fast convergence speed and high accuracy; Figure 2 (6) Represents the posterior distribution of noise accuracy. From Figure 2 It can be seen that after introducing the Lagrange multiplier in this invention, the μ-VB algorithm can estimate the size of the tensor rank with 100% accuracy, while ensuring high precision.

[0156] Figure 3 The variation of MSE values ​​based on different Lagrange multiplier algorithms is shown. Figure 4 The study demonstrates the runtime variations of algorithms based on different Lagrange multipliers. For static Lagrange multipliers, different values ​​of μ have varying effects on the μ-VB algorithm. Smaller μ values ​​do not significantly improve the performance of the μ-VB algorithm; larger μ values ​​result in better performance improvements. For dynamic Lagrange multipliers, the linear model demonstrates their success, but at high signal-to-noise ratios, the exponential model's growth rate is too rapid, exceeding the optimal model for Lagrange multipliers and noise accuracy. The signal-to-noise ratio model based on singular value sequences shows that dynamic Lagrange multipliers significantly improve algorithm performance.

[0157] In wireless communication applications, Figure 5 The algorithm presented in this invention demonstrates a lower bit error rate and faster, more stable performance after 5000 independent experiments. In a DS-CDMA system, the signal t transmitted by the r-th user in the i-th symbol period... r (i) Multiply by the spreading sequence [s] 1r ,s 2r ,…,s Jr ], where s jr This refers to the j-th symbol of the spreading code used. Received data:

[0158]

[0159] Convert to tensor form

[0160]

[0161] Figure 5 This invention demonstrates that it achieves a lower bit error rate and faster convergence speed in DS-CDMA wireless communication systems.

[0162] In applications of linear image coding, Figure 6 The algorithm presented in this invention has a lower classification error rate and faster running time, as demonstrated in 50 independent experiments.

[0163] The database is the Yale Face Database. It forms the tensor model.

[0164]

[0165] Furthermore, it indicates

[0166]

[0167] During the experiment, two people were randomly selected. Twelve images were used for the training set (six images for each person), and the remaining ten images were used for the test set. Using an SVM classifier, 50 independent experiments were conducted. Figure 6 This invention demonstrates that it achieves faster processing speed and lower classification error rate in linear image coding.

[0168] This invention also provides a device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements a probability tensor CPD algorithm based on dynamic and static Lagrange multipliers and orthogonal factors as described in any one of the embodiments.

[0169] This invention also provides a storage medium storing a computer program, characterized in that the computer program is designed to implement, at runtime, any of the probability tensor CPD algorithms based on dynamic and static Lagrange multipliers and orthogonal factors described in the present invention.

Claims

1. A probabilistic tensor CPD algorithm based on dynamic and static Lagrange multipliers and orthogonal factors, characterized in that, The steps include: (1) constructing a complex-valued tensor model; the formula is as follows: wherein, denotes an additive white Gaussian noise, and each component is independently identically distributed; denotes a pure signal tensor, denotes a signal tensor containing noise; (2) pre-processing the complex-valued tensor model based on a static Lagrange multiplier FF-SVD algorithm; the specifics are as follows: First, let's set: , Here, the expansion operator is defined. ,express order tensor Along the first Modal expansion, specifically defined as , Indicates except the first One position, the rest are all 0; The estimated signal matrix is defined as: ; and the residual matrix is ; Then, the residual signal matrix is estimated with constraints: The Lagrange function is Wherein, the solution of the constrained optimization problem satisfies the KKT condition, and the formula is as follows: According to the properties of the block matrix: And the following relationship: Get: Finally, we get: Wherein, the signal-to-noise ratio is: ； (3) further updating the FF-SVD algorithm parameters based on a dynamic Lagrange multiplier; the specifics are as follows: the signal-to-noise ratio model of the singular value sequence is as follows: wherein denotes the singular values after decomposition: or ; (4) converting the complex-valued tensor model into a probability model; the specifics are as follows: First, set the CPD tensor model as: ； Set the probability model as: wherein a factor matrix is mapped onto a Stiefel manifold, i.e. under the condition ;​ When the factor matrix can be seen as uniformly distributed on the SM i.e.: When is Gaussian, i.e.: Second, set For , , The marginal probability density function of the unknown factor matrix ​ Wherein, Again, the probability density function is calculated using the KL divergence as the variational distribution is expressed as: Let the variational probability density take the form of a complete factorization ; the constraint operation on the first ; The probability density function of the variational distribution It can be analytically obtained that: And ； (5) solving the probability model using a mu-VB algorithm.

2. The probability tensor CPD algorithm based on dynamic and static Lagrange multipliers and orthogonal factors according to claim 1, characterized in that, The step (5) includes the following steps: (51) calculating the orthogonal constraint factor matrix; the specifics are as follows: Property 1: Hypothesis matrix Subject to Matrix distribution, where parameters ; if is obtained by SVD of matrix The unique mode of ;​ Based on property 1, the factor matrix containing orthogonal structure is estimated to obtain wherein and are the left and right orthogonal matrices obtained by performing SVD on ​ (52) calculating the unconstrained factor matrix, the specifics are as follows: Get: (53) to The regularization term is specified as follows: (54) calculating the noise precision, the specifics are as follows: wherein and 。 3. An apparatus comprising a memory, a processor, and a program stored on the memory and executable on the processor, wherein, The processor executes the program to realize the probability tensor CPD algorithm based on dynamic and static Lagrange multipliers and orthogonal factors according to any one of claims 1-2.

4. A storage medium storing a computer program, characterized by The computer program is designed to realize the probability tensor CPD algorithm based on dynamic and static Lagrange multipliers and orthogonal factors according to any one of claims 1-2 when running.

Images