L0 factor analysis method based on Newton interior point algorithm

Abstract

The invention discloses an L0 factor analysis method based on a Newton interior point algorithm, and the method comprises the steps: generating a random sample based on a first factor model with noise; obtaining a covariance matrix of the random sample; according to the covariance matrix, combining KL divergence and L0 norm to construct a first optimization problem with positive semidefinite constraint; converting the first optimization problem into an unconstrained second optimization problem according to a logarithmic barrier function; vectorizing the second optimization problem to obtain a third optimization problem with unconstrained vector variables; and through a Newton method, according to the third optimization problem, obtaining estimation of the number of load factors and a heterogeneous noise covariance matrix. The method can significantly improve the efficiency of data analysis and optimization, and can be widely applied to the technical field of data optimization.

Description

Technical Field

[0001] This invention relates to the field of data optimization technology, and in particular to an L0 factor analysis method based on Newton's interior point algorithm. Background Technology

[0002] In today's era of booming digital economy and rapidly evolving artificial intelligence technology, the era of big data has fully arrived. Massive amounts of data are constantly being generated in fields such as the internet, the Internet of Things, and biomedicine, with data dimensions exploding. The "curse of dimensionality" brought about by high-dimensional data poses enormous challenges to data processing and analysis. Factor analysis, as an important tool in statistics and machine learning, has been widely used for data dimensionality reduction and feature extraction since its inception. In market research, it can summarize numerous consumer evaluation indicators into a few core consumer preference factors; in gene expression data analysis, it can help screen and diagnose disease biomarkers by uncovering potential gene regulatory factors. By introducing potential "common factors" and "specific factors," factor analysis successfully represents observed variables as linear combinations of factors, thereby revealing the intrinsic structure between variables.

[0003] With the continuous deepening and expansion of research in data science and statistics, under certain conditions, the estimation problems of the number of loading factors and the heterogeneous noise covariance matrix in factor analysis can be considered as additive decomposition problems of low-rank and sparse matrices of high-dimensional sample covariance matrices. In terms of sparsity measurement, the L0 norm, as the most natural sparsity index, can accurately express the number of non-zero elements in a variable, providing an ideal sparsity constraint tool for factor analysis. To address the problems arising from the non-convex discreteness and positive semidefinite constraints of the L0 norm during the optimization process, some studies have designed Block Coordinate Descent (BCD) and Alternating Direction Multiplier Method (ADMM) for solving these problems. The BCD algorithm employs a block-based processing strategy, optimizing low-rank and sparse variables separately, and gradually approximating the optimal solution through iterative updates of the variables in blocks. The ADMM algorithm introduces auxiliary variables to transform the complex semidefinite constraint problem, first centrally processing the semidefinite constraints, and then performing minimization operations on each optimization variable individually. However, both of these traditional algorithms have revealed significant drawbacks in practical applications, namely slow convergence speeds. In particular, the ADMM algorithm, due to the complexity of the solution space of non-convex optimization problems under semidefinite constraints in real-world scenarios, often requires a large number of iterations, consuming a lot of computational resources and time costs to achieve an acceptable convergence effect. This greatly limits its application in scenarios with high real-time requirements or large-scale computing scenarios. Summary of the Invention

[0004] In view of this, the main objective of this invention is to provide an L0 factor analysis method based on Newton's interior point algorithm, in order to solve at least one of the problems in the prior art. This invention can improve the efficiency of data analysis and optimization.

[0005] To achieve the above objectives, one aspect of this invention provides an L0 factor analysis method based on Newton's interior-point algorithm, the method comprising:

[0006] Random samples are generated based on a noisy first factorial model.

[0007] Obtain the covariance matrix of the random sample;

[0008] Based on the covariance matrix, combined with KL divergence and L0 norm, a first optimization problem with positive semidefinite constraints is constructed.

[0009] Based on the logarithmic barrier function, the first optimization problem is transformed into an unconstrained second optimization problem;

[0010] Vectorizing the second optimization problem yields a third optimization problem with unconstrained vector variables;

[0011] Using Newton's method, estimates of the number of loading factors and the heterogeneous noise covariance matrix are obtained based on the third optimization problem. In some embodiments, the formula used to generate random samples based on the noisy first factor model includes:

[0012] y i =a+Γu i +w i i = 1, 2, ..., N

[0013] In the formula, y i Γ represents the observation vector; a represents the mean vector; Γ represents the full-rank factor loading matrix; u i Indicates hidden factor; w i represents the heterogeneous noise term; N represents the number of samples.

[0014] In some embodiments, the formula used to obtain the covariance matrix of the random sample includes:

[0015]

[0016] In the formula, The covariance matrix of the random sample is represented by N; N represents the sample size, i = 1, 2, ..., N; y i Represents the observation vector; This indicates the transpose operation.

[0017] In some embodiments, constructing a first optimization problem with positive semidefinite constraints based on the covariance matrix, combined with KL divergence and L0 norm, includes the following steps:

[0018] Obtain the nuclear norm of the first symmetric matrix and the L0 norm of the second symmetric matrix;

[0019] Based on the nuclear norm, L0 norm, and regularization parameters, a second factor model is constructed.

[0020] Based on the covariance matrix and KL divergence, construct the constraints for the second factor model;

[0021] The KL divergence constraint in the constraints is embedded into the second factor model to obtain the first optimization problem.

[0022] In some embodiments, the formula used to construct the first optimization problem with positive semidefinite constraints based on the covariance matrix, combined with the KL divergence and L0 norm, includes:

[0023]

[0024] stL,S≥0,L+S>0

[0025] In the formula, L represents the first symmetric matrix; S represents the second symmetric matrix; tr(·) represents the trace of the matrix; μ represents the penalty parameter of the KL divergence term; denoted as the covariance matrix of the random sample; det(·) denotes the determinant of the matrix; λ denotes the regularization parameter; and ||·|0 denotes the L0 norm.

[0026] In some embodiments, transforming the first optimization problem into an unconstrained second optimization problem according to the logarithmic barrier function includes the following steps:

[0027] The logarithmic barrier function and its corresponding logarithmic barrier parameters are added to the first optimization problem to obtain the second optimization problem.

[0028] In some embodiments, the formula used to transform the first optimization problem into an unconstrained second optimization problem according to the logarithmic barrier function includes:

[0029]

[0030] In the formula,

[0031]

[0032] Where L represents the first symmetric matrix; S represents the second symmetric matrix; λ represents the regularization parameter; ||·||0 represents the L0 norm; tr(·) represents the trace of the matrix; μ represents the penalty parameter for the KL divergence term; represents the covariance matrix of the random sample; -log det(·) represents the logarithmic barrier function; v represents the logarithmic barrier parameter.

[0033] In some embodiments, vectorizing the second optimization problem to obtain a third optimization problem with unconstrained vector variables includes the following steps:

[0034] Construct an orthonormal basis for the set of the first symmetric matrix and the second symmetric matrix;

[0035] The first symmetric matrix is ​​decomposed into a first linear combination representation of the orthonormal basis;

[0036] The second symmetric matrix is ​​decomposed into a second linear combination representation of the orthonormal basis;

[0037] Based on the first linear combination representation and the second linear combination representation, the second optimization problem is vectorized to obtain the third optimization problem.

[0038] In some embodiments, obtaining the estimate of the number of loading factors and the heterogeneous noise covariance matrix using Newton's method based on the third optimization problem includes the following steps:

[0039] Determine whether the logarithmic obstacle parameter is greater than a preset threshold. If the logarithmic obstacle parameter is greater than the preset threshold, obtain the stationary point of the third optimization problem with respect to the preset parameter.

[0040] Based on the stationary points, construct nonlinear stationary point equations;

[0041] The stationary point equation is solved using Newton's method to reduce the value of the logarithmic barrier parameter, and the process returns to the step of determining whether the logarithmic barrier parameter is greater than a preset threshold.

[0042] In some embodiments, the step of obtaining the estimate of the number of loading factors and the heterogeneous noise covariance matrix by means of Newton's method according to the third optimization problem further includes the following steps:

[0043] Determine whether the logarithmic barrier parameter is greater than a preset threshold. When the logarithmic barrier parameter is less than or equal to the preset threshold, output the estimated number of loading factors and the heterogeneous noise covariance matrix.

[0044] To achieve the above objectives, another aspect of this invention proposes an L0 factor analysis device based on Newton's interior-point algorithm, the device comprising:

[0045] The first module is used to generate random samples based on a noisy first factor model;

[0046] The second module is used to obtain the covariance matrix of the random sample;

[0047] The third module is used to construct a first optimization problem with positive semidefinite constraints based on the covariance matrix, combined with KL divergence and L0 norm.

[0048] The fourth module is used to transform the first optimization problem into an unconstrained second optimization problem based on the logarithmic barrier function.

[0049] The fifth module is used to vectorize the second optimization problem to obtain the third optimization problem with unconstrained vector variables;

[0050] The sixth module is used to obtain estimates of the number of loading factors and the heterogeneous noise covariance matrix based on the third optimization problem using Newton's method.

[0051] To achieve the above objectives, another aspect of the present invention provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described above.

[0052] To achieve the above objectives, another aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0053] To achieve the above objectives, another aspect of the present invention provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions to cause the computer device to perform the aforementioned method.

[0054] The embodiments of the present invention include at least the following beneficial effects: The present invention provides an L0 factor analysis method based on Newton's interior-point algorithm. This scheme generates random samples based on a noisy first factor model; obtains the covariance matrix of the random samples; constructs a first optimization problem with positive semidefinite constraints based on the covariance matrix, combined with KL divergence and L0 norm; transforms the first optimization problem into an unconstrained second optimization problem based on the logarithmic barrier function; vectorizes the second optimization problem to obtain a third optimization problem with unconstrained vector variables; and obtains estimates of the number of loading factors and the heterogeneous noise covariance matrix based on the third optimization problem using Newton's method, which can significantly improve the efficiency of data analysis optimization. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart of the L0 factor analysis method based on Newton's interior point algorithm provided in an embodiment of the present invention;

[0057] Figure 2(a) is a schematic diagram comparing the convergence rates of the interior point method, ADMM, and BCD when the attenuation ratio is θ = 0.5 according to the embodiment of the present invention.

[0058] Figure 2(b) is a schematic diagram comparing the convergence rates of the interior point method, ADMM, and BCD when the attenuation ratio is θ = 0.8 according to the embodiment of the present invention.

[0059] Figure 3 This is a flowchart of the L0 factor analysis process from modeling to solving using Newton's interior point algorithm, provided in an embodiment of the present invention.

[0060] Figure 4 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0062] It should be noted that although functional modules are divided in the system diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first / S100" and "second / S200" in the specification, claims, and the foregoing drawings may be used herein to describe various concepts, but unless specifically stated otherwise, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another. For example, first information may also be referred to as second information without departing from the scope of the embodiments of the invention, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" or "when" as used herein may be interpreted as "when," "in response to a determination," or "in the event of a determination."

[0063] The terms “at least one,” “multiple,” “each,” “any,” etc., used in this invention, “at least one” includes one, two, or more than two; “multiple” includes two or more than two; “each” refers to each of the corresponding multiple; and “any” refers to any one of the multiple.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0065] like Figure 1 As shown, this embodiment of the invention provides an L0 factor analysis method based on Newton's interior point algorithm, which may include, but is not limited to, steps S100 to S600:

[0066] Step S100: Generate random samples based on the noisy first factor model;

[0067] Step S200: Obtain the covariance matrix of the random sample;

[0068] Step S300: Based on the covariance matrix, and combining the KL divergence and L0 norm, construct the first optimization problem with positive semidefinite constraints;

[0069] Step S400: Based on the logarithmic barrier function, the first optimization problem is transformed into an unconstrained second optimization problem;

[0070] Step S500: Vectorize the second optimization problem to obtain a third optimization problem with unconstrained vector variables;

[0071] Step S600: Using Newton's method, based on the third optimization problem, the estimated number of loading factors and the heterogeneous noise covariance matrix are obtained.

[0072] In steps S100 to S600 of some embodiments, the properties of the L0 norm proximal operator are deeply integrated with Newton's method, giving full play to the advantage of Newton's method in using the first and second derivatives of the objective function to determine the iteration direction and step size, thus overcoming the slow convergence of traditional algorithms. This invention breaks through the limitations of traditional convex optimization theory and has good applicability to complex non-convex optimization problems such as the estimation of the number of loading factors and the covariance matrix of heterogeneous noise in factor analysis.

[0073] In step S100 of some embodiments, the noisy first factor model is as follows:

[0074] y i =a+Γu i +w ii = 1, 2, ..., N

[0075] In the formula, Represents the observation vector; Represents the mean vector; Denotes a factor loading matrix with full column rank; random vector u i ~N(0,I r ) represents the hidden factor, I r w represents an r-dimensional identity matrix; i ~N(0,S * ) represents the heterogeneous noise term, which is unrelated to the hidden factor; N represents the number of samples. A series of samples can be generated using the noisy first factorial model.

[0076] In step S200 of some embodiments, based on the noisy first factor model, N independent and identically distributed samples y are considered. i The problem then becomes estimating the factor loading matrix Γ and the heterogeneous noise covariance matrix S. * This is equivalent to estimating the rank matrix r. and heterogeneous noise variance matrix S * In factor analysis, assuming the mean vector a = 0, we can obtain y i The covariance matrix is:

[0077] ∑=L * +S *

[0078] When rank r << n, and the heterogeneous noise covariance matrix S * When sparse, matrix L * Having a lower rank, it can be deduced that the covariance matrix ∑ is of the form of "low-rank plus sparse", thus affecting the factor loading matrix Γ and the heterogeneous noise covariance matrix S. * The estimation can be viewed as an additive decomposition problem of low-rank and sparse matrices. In practical analysis, the covariance matrix of random sample data is calculated as follows:

[0079]

[0080] In the formula, The covariance matrix of the random sample is represented by N; N represents the sample size, i = 1, 2, ..., N; y i Represents the observation vector; This indicates the transpose operation.

[0081] In some embodiments, step S300 may include, but is not limited to, steps S310 to S340:

[0082] Step S310: Obtain the nuclear norm of the first symmetric matrix and the L0 norm of the second symmetric matrix;

[0083] Step S320: Construct a second factor model based on the nuclear norm, L0 norm, and regularization parameter;

[0084] Step S330: Construct the constraints of the second factor model based on the covariance matrix and KL divergence.

[0085] Step S340: Embed the KL divergence constraint in the constraints into the second factor model to obtain the first optimization problem.

[0086] In steps S310 to S320 of some embodiments, it is assumed that the estimation of the sample covariance matrix is... Since the matrix is ​​positive definite, we obtain the nuclear norm of the first symmetric positive semi-definite matrix L, which is equal to its trace tr(L). This nuclear norm can be used to constrain the rank of the first symmetric matrix L. We obtain the zero norm ‖S‖0 of the second symmetric matrix S, which represents the number of non-zero elements in the second symmetric matrix S. Based on the nuclear norm and L0 norm, combined with the regularization parameter, we can construct the following second factor model:

[0087]

[0088] In step S330 of some embodiments, based on the covariance matrix of the random sample data Obtain the observation vector y i The covariance matrix ∑ and the covariance matrix of random sample data The KL divergence between these two positive definite matrices measures the true sample covariance matrix ∑ and its estimated value. The "distance" between them. Then the following constraints apply to the second factor model:

[0089] stL,S≥0,L+S>0

[0090]

[0091] In the constraints,

[0092]

[0093] In the formula, δ represents the parameter used to constrain the estimation effect of the covariance matrix; n represents the dimension of the sample vector.

[0094] In step S340 of some embodiments, the covariance matrix Σ term of the observation vector in the KL divergence constraint of the second factor model is replaced with the equality constraint Σ=L+S, and then the KL divergence constraint is relaxed into the second factor model. Finally, the factor analysis is modeled as a regular optimization problem with a positive semidefinite constraint, namely the first optimization problem:

[0095]

[0096] stL,S≥0,L+S>0

[0097] In the formula, L represents the first symmetric matrix; S represents the second symmetric matrix; tr(·) represents the trace of the matrix; μ represents the penalty parameter of the KL divergence term; denoted as the covariance matrix of the random sample; det(·) denotes the determinant of the matrix; λ denotes the regularization parameter; and ||·|0 denotes the L0 norm.

[0098] In step S400 of some embodiments, a logarithmic barrier function and its corresponding logarithmic barrier parameter are introduced into the first optimization problem, thereby transforming the original L0 regularized optimization problem with semidefinite constraints into a series of unconstrained L0 regularized optimization problems, resulting in the second optimization problem. For example, by adding the logarithmic barrier function -log det(·) and its corresponding logarithmic barrier parameter τ to the first optimization problem, since the logarithmic barrier function causes the optimization problem to fail when it touches the semidefinite constraint boundary, the introduction of this function ensures that the optimization variables L and S always remain within the feasible region, avoiding touching the semidefinite constraint boundary, thus guaranteeing the effectiveness of the optimization problem. The following unconstrained optimization problem, i.e., the second optimization problem, is then obtained:

[0099]

[0100] In the formula,

[0101]

[0102] Where L represents the first symmetric matrix; S represents the second symmetric matrix; λ represents the regularization parameter; ||·||0 represents the L0 norm; tr(·) represents the trace of the matrix; μ represents the penalty parameter for the KL divergence term; represents the covariance matrix of the random sample; -log det(·) represents the logarithmic barrier function; τ represents the logarithmic barrier parameter.

[0103] In some embodiments, as the iterative variables gradually approach the feasible region boundary of the unconstrained optimization problem during the optimization process, the value of the logarithmic barrier function becomes increasingly larger. However, as the logarithmic barrier parameter τ approaches 0, the influence of the logarithmic barrier function gradually weakens, and the augmented objective function f... τ (L,S)+λ‖S‖0 approaches the first optimization problem, and the corresponding optimization solution approximates the feasible solution of the first optimization problem. During the process of the logarithmic barrier parameter τ approaching 0, a decreasing sequence of values ​​τ1>τ2>…>τ is generated. jDifferent logarithmic barrier parameters correspond to different constrained optimization problems. Therefore, solving the first optimization problem is transformed into solving a series of unconstrained L0 regularized optimization problems (the second optimization problem).

[0104] In some embodiments, step S500 may include, but is not limited to, steps S510 to S540:

[0105] Step S510: Construct an orthonormal basis for the set of the first symmetric matrix and the second symmetric matrix;

[0106] Step S520: Decompose the first symmetric matrix into a first linear combination representation of the orthonormal basis;

[0107] Step S530: Decompose the second symmetric matrix into a second linear combination representation of the orthonormal basis;

[0108] Step S540: Based on the first linear combination representation and the second linear combination representation, the second optimization problem is vectorized to obtain the third optimization problem.

[0109] In step S510 of some embodiments, based on the symmetry properties of the first symmetric matrix L and the second symmetric matrix S, a set of orthonormal bases {E1, E2, ..., E...} of the symmetric matrix set is first constructed. m},in Decomposing a symmetric matrix into a linear combination of orthonormal bases, we have matrix variables. make Then we can obtain the vector equivalent form of the second optimization problem based on matrix variables:

[0110]

[0111] In the formula,

[0112]

[0113] Among them, E i Represents each basis vector; E j Represent each basis vector; Let L be a first symmetric matrix in the orthonormal basis {E1, E2, ..., E...} m The coordinate vector under}; l i Describes the basis vector E corresponding to the first symmetric matrix L. i The coordinate components; This indicates that the second symmetric matrix S lies in the orthonormal basis {E1, E2, ..., E...} m The coordinate vector under}; s j The basis vector E corresponding to the second symmetric matrix S is represented. jThe coordinate components.

[0114] In some embodiments, step S600 may include, but is not limited to, steps S610 to S630:

[0115] Step S610: Determine whether the logarithmic obstacle parameter is greater than a preset threshold. When the logarithmic obstacle parameter is greater than the preset threshold, obtain the stationary point of the third optimization problem with respect to the preset parameter.

[0116] Step S620: Construct a nonlinear stationary point equation based on the stationary point;

[0117] Step S630: Solve the stationary point equation using Newton's method to reduce the value of the logarithmic barrier parameter, and return to the step of determining whether the logarithmic barrier parameter is greater than a preset threshold.

[0118] In steps S610 to S620 of some embodiments, it is determined whether the logarithmic barrier parameter is greater than a preset threshold ∈. When the logarithmic barrier parameter is greater than the preset threshold, i.e., the logarithmic barrier parameter is not sufficiently small, the third optimization problem corresponding to the logarithmic barrier parameter is solved using Newton's method. For example, firstly, using the concept of proximal operators of the L0 norm, the γ-stationary point of the third optimization problem is defined:

[0119] For iteration point (l) * ,s * If there exists a γ>0 satisfying

[0120]

[0121] in, and They represent functions h respectively τ About l * and s * The gradient of l is called (l) * ,s * () represents the γ-stationary point in the third optimization problem. Using... Closed-form solution:

[0122]

[0123] γ-stationary points can also be represented as:

[0124]

[0125] Among them, supp(s * ) represents vector s *The support set. For the third optimization problem, it has been theoretically proven that a local optimum can be obtained by finding a γ-stationary point. To make the expression of the γ-stationary point more favorable for second-order algorithm design, the following index set is defined for any iteration point (l, s):

[0126]

[0127] Then, based on this index set, the following stationary point equations are constructed:

[0128]

[0129] It is easy to deduce that the points that satisfy the above stationary point equations are all γ-stationary points of the third optimization problem, and also local minima.

[0130] In step S630 of some embodiments, a Newton algorithm is designed to solve the nonlinear stationary point equation, reduce the value of the logarithmic barrier parameter by a certain attenuation ratio, and return to the step of determining whether the logarithmic barrier parameter is greater than a preset threshold, until the logarithmic barrier parameter is attenuated to a sufficiently small value, and output estimates of the number of loading factors and the heterogeneous noise covariance matrix. For example, the specific steps of Newton's method are as follows:

[0131] 1) Initialize parameters and variables;

[0132] 2) For the current iteration point (l) k ,s k ), calculate the corresponding index set T k =T γ (l k ,s k ;λ);

[0133] 3) If Proceed to step 4); otherwise proceed directly to step 6.

[0134] 4) By solving the system of linear equations Calculate the direction of Newton's descent d k Expand the system of linear equations:

[0135]

[0136] Firstly, it is easy to conclude... Substituting it back into the above system of equations, we can calculate...

[0137]

[0138] Update Newton's direction

[0139] 5) Determine the update step size (l) using the modified backtracking line search. k+1 ,sk+1 )=(l k (α),s k (α)), where

[0140]

[0141] Let k = k + 1, then proceed to step 2);

[0142] 6) Output (l) k ,s k ).

[0143] After the logarithmic barrier parameter of the outer loop decays to a sufficiently small threshold, the optimal solution (L) is output. * ,S * ).

[0144] In some embodiments, step S600 may also include, but is not limited to, step S640:

[0145] Step S640: Determine whether the logarithmic barrier parameter is greater than a preset threshold. When the logarithmic barrier parameter is less than or equal to the preset threshold, output the estimated number of loading factors and the heterogeneous noise covariance matrix.

[0146] In step S640 of some embodiments, it is determined whether the logarithmic barrier parameter is greater than a preset threshold. When the logarithmic barrier parameter is less than or equal to the preset threshold, that is, when the logarithmic barrier parameter decays to a sufficiently small value, the number r of loading factors in the output factor model is determined. * Estimation of the covariance matrix S of heterogeneous noise * The method for estimating the number of loading factors is as follows:

[0147]

[0148] Where λ1>λ2>…>λ n For matrix L * eigenvalues, i max The ratio λ of adjacent eigenvalues i+1 / λ i The minimum index subscript corresponding to <0.05.

[0149] In some embodiments, the convergence rate of Newton's interior-point method is compared with that of other existing algorithms. The convergence rates of Newton's interior-point method, BCD, and ADMM algorithms are compared under different decay ratios of the logarithmic barrier parameter. Specifically, when the decay ratios of the logarithmic barrier parameter are θ = 0.5 and θ = 0.8, the number of outer loop iterations of the interior-point method is 18 (as shown in Figure 2(a)) and 58 (as shown in Figure 2(b)), respectively. The horizontal axis of the red curve for the interior-point method records the total number of iterations of the Newton method within the inner loop. Despite this, the superiority of the interior-point method in terms of convergence rate is still evident.

[0150] In some embodiments, using a synthetic dataset as an example, the L0 factor analysis method based on Newton's interior-point algorithm of the present invention will be described, with reference to... Figure 3 This includes the following steps:

[0151] Step 1: Based on the following noisy factor model:

[0152] y i =a+Γu i +w i i = 1, 2, ..., N

[0153] Assuming the mean vector a = 0, N = 1200 samples are randomly generated based on a noisy factor model with a sample dimension n = 40 and a rank r = 5 for the loading matrix. The signal-to-noise ratio of these samples is ||ΓΓ|. T || F / ||S * || F =1. Calculate the covariance matrix corresponding to the sample data using the following formula:

[0154]

[0155] Step 2: Combine KL divergence and the L0 norm of the matrix to construct the following optimization model:

[0156]

[0157] stL,S≥0,L+S>0

[0158] Among them Replace the generated covariance matrix with the covariance matrix corresponding to N=1200 samples, and then solve the minimization problem with optimization variables L and S.

[0159] Step 3: For the L0 regularized optimization problem with semidefinite constraints, add the logarithmic barrier function -logdet(·) and the corresponding barrier parameter τ to the original objective function. Since the -logdet(·) function will cause the optimization problem to fail when it touches the semidefinite constraint boundary, the introduction of this function forces the optimization variables L and S to always be inside the feasible region. In this way, the original optimization problem with semidefinite constraints is transformed into the following unconstrained optimization problem.

[0160]

[0161] in,

[0162]

[0163] As the optimization variables gradually approach the feasible region boundary during the iteration process, among which The logarithmic barrier term becomes increasingly larger. The initial value of the barrier parameter τ is 0.5, and then the value of parameter τ is decreased proportionally to θ as the iteration progresses. When τ→0, the augmented objective function f... τ (L,S)+λ‖S‖0 approaches the original objective function, and the corresponding optimized solution approximates the feasible solution of the original problem. During this process, a decreasing sequence of values ​​τ1>τ2>…>τ is generated. j Each value of the logarithmic barrier parameter corresponds to an unconstrained optimization problem. Therefore, solving the original problem is transformed into solving a series of unconstrained L0 regularized optimization problems. Here, we take the barrier parameter value τ as an example and consider the following minimization problem.

[0164]

[0165] Step 4: Vectorize the unconstrained L0 regularized optimization problem of the above matrix variables. Since the optimization variables L and S are symmetric matrices, construct an orthonormal basis {E1, E2, ..., E...} of a 40-dimensional symmetric matrix set. m}, where m = 820. Therefore, the matrix variables Write down the coordinates of these two matrix variables in the orthonormal basis. Then we obtain the vector equivalent form of the matrix variable optimization problem above:

[0166]

[0167] in

[0168] Step 5: Solve the unconstrained L0 regularized optimization problem for the above vector variables using Newton's method. For any iteration point (l, s), define the following index set:

[0169]

[0170] Then, based on this index set, the following stationary point equations are constructed:

[0171]

[0172] Next, the above nonlinear stationary point equations are solved using Newton's algorithm.

[0173] 1) Initialize parameters and variables; let (l0,s0) are the coordinates of (L0,S0) under an orthonormal basis. The initial iteration variable is set to (l0,s0), and the proximal operator step size parameter γ = 10. -4 The algorithm's convergence threshold parameter ∈1=10 -4 The values ​​of tuning parameters λ and μ are selected through cross-validation, with a maximum number of iterations k. max =1000, initial iteration count k=0.

[0174] 2) For the current iteration point (l) k ,s k ), calculate the corresponding index set T k =T γ (l k ,s k ;λ);

[0175] 3) If And k <k max If the above steps are not valid, proceed to step 4; otherwise, proceed directly to step 6.

[0176] 4) By solving the system of linear equations Calculate the direction of Newton's descent d k Expanding this system of linear equations, we can obtain... Substituting this back into the expanded system of equations, we can calculate...

[0177]

[0178] Update Newton's direction

[0179] 5) Update step size (l) k+1 ,s k+1 )=(l k (α k ),s k (α k )),in v k It is the smallest non-negative integer that satisfies the following conditions

[0180] Update variables according to the following rules:

[0181]

[0182] Let k = k + 1, then proceed to step 2);

[0183] 6) Output (l) k ,s k ).

[0184] The logarithmic barrier parameter τ in the outer loop is less than 10. -6 Then, output the optimal solution (L) * ,S * ).

[0185] Step 6: Estimate the number of loading factors r using the following estimation method. * :

[0186]

[0187] Where λ1>λ2>…>λn For matrix L * eigenvalues, i max The ratio λ of adjacent eigenvalues i+1 / λ i The minimum index subscript corresponding to <0.05.

[0188] In some embodiments, taking economic data analysis as an example, this is applicable to scenarios such as industry risk assessment, and can be referenced. Figure 3 The L0 factor analysis method based on Newton's interior point method includes the following steps:

[0189] Step 1: Apply the L0 factor analysis method based on Newton's interior point method proposed in this invention to a real economic dataset. Optionally, this dataset includes beta coefficient, debt-to-equity ratio, effective tax rate, unleveraged beta coefficient, modified unleveraged beta coefficient, high and low volatility risk, and standard deviation of operating income. This dataset reflects the financial characteristics of various industries under a specific economic cycle, with a corresponding sample size N = 92 and sample variable dimensions n = 9. Calculate the covariance matrix corresponding to the sample data according to the following formula:

[0190]

[0191] Step 2: Construct an optimization model by combining KL divergence and the L0 norm of the matrix, where... Replace the covariance matrix with the generated N=92 samples, and then solve the minimization problem with optimization variables L and S.

[0192] Step 3: For the L0 regular optimization problem with semidefinite constraints, introduce a logarithmic barrier function to ensure that the optimization variables L and S are within the feasible region, thus transforming the original optimization problem with semidefinite constraints into an unconstrained optimization problem.

[0193] Step 4: Vectorize the unconstrained L0 regularized optimization problem of the above matrix variables. First, construct an orthonormal basis {E1, E2, ..., E...} for a 9-dimensional symmetric matrix set. m}, where m = 45. Therefore, the matrix variables Write down the coordinates of these two matrix variables in the orthonormal basis. Then we obtain the vector equivalent form of the matrix variable optimization problem above.

[0194] Step 5: Solve the unconstrained L0 regularized optimization problem for the vector variables using Newton's method. For any iteration point (l, s), define an index set, and then construct the stationary point equation based on this index set. Next, solve the nonlinear stationary point equation using Newton's algorithm:

[0195] 1) Parameter and variable initialization: Let (l0,s0) are the coordinates of (L0,S0) under an orthonormal basis. The initial iteration variable is set to (l0,s0), and the proximal operator step size parameter γ = 10. -4 The algorithm's convergence threshold parameter ∈1=10 -4 The values ​​of tuning parameters λ and μ are selected through cross-validation, with a maximum number of iterations k. max =300, initial iteration number k=0.

[0196] 2) For the current iteration point (l) k ,s k ), calculate the corresponding index set T k =T γ (l k ,s k ;λ);

[0197] 3) If And k <k max If the above steps are not valid, proceed to step 4; otherwise, proceed directly to step 6.

[0198] 4) By solving the system of linear equations Calculate the direction of Newton's descent d k Expanding this system of linear equations, we can obtain... Substituting it back into the system of equations, we can calculate...

[0199]

[0200] Update Newton's direction

[0201] 5) Update step size (l) k+1 ,s k+1 )=(l k (α k ),s k (α k )),in v k It is the smallest non-negative integer that satisfies the following conditions

[0202] Update variables according to the following rules

[0203]

[0204] Let k = k + 1, then proceed to step 2);

[0205] 6) Output (l) k ,s k ).

[0206] The logarithmic barrier parameter τ in the outer loop is less than 10. -6 Then, output the optimal solution (L)* ,S * ).

[0207] Step 6: Estimate the number of loading factors r using the following estimation method. * :

[0208]

[0209] Where λ1>λ2>…>λ n For matrix L * eigenvalues, i max The ratio λ of adjacent eigenvalues i+1 / λ i The minimum index subscript corresponding to <0.05.

[0210] Finally, this embodiment estimates a rank of 3 for the aforementioned economics dataset. This result can be interpreted as indicating that portfolio returns are primarily driven by three common factors: market trends, fiscal policy, and industry characteristics. By constructing a simulated portfolio using factor loading matrices, the impact of specific factors can be tracked, optimizing the risk and return structure. This estimation provides a low-dimensional, efficient factor structure for the portfolio, facilitating systematic risk diversification and return attribution by adjusting factor exposure levels.

[0211] In addition to its application in economic dataset analysis, the embodiments of this invention can also be used for data analysis and optimization in fields such as system identification, signal processing, and machine learning.

[0212] This invention also provides an L0 factor analysis device based on Newton's interior-point algorithm, which can implement the above-mentioned L0 factor analysis method based on Newton's interior-point algorithm. The device includes:

[0213] The first module is used to generate random samples based on a noisy first factor model;

[0214] The second module is used to obtain the covariance matrix of the random sample;

[0215] The third module is used to construct a first optimization problem with positive semidefinite constraints based on the covariance matrix, combined with KL divergence and L0 norm.

[0216] The fourth module is used to transform the first optimization problem into an unconstrained second optimization problem based on the logarithmic barrier function.

[0217] The fifth module is used to vectorize the second optimization problem to obtain the third optimization problem with unconstrained vector variables;

[0218] The sixth module is used to obtain estimates of the number of loading factors and the heterogeneous noise covariance matrix based on the third optimization problem using Newton's method.

[0219] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0220] This invention also provides an electronic device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including a tablet computer, an in-vehicle computer, or similar device.

[0221] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0222] refer to Figure 4 , Figure 4 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0223] The processor 701 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0224] The memory 702 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 702 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 702 and is called and executed by the processor 701.

[0225] The input / output interface 703 is used to implement information input and output;

[0226] The communication interface 704 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0227] Bus 705 transmits information between various components of the device (e.g., processor 701, memory 702, input / output interface 703, and communication interface 704);

[0228] The processor 701, memory 702, input / output interface 703, and communication interface 704 are connected to each other within the device via bus 705.

[0229] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0230] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0231] This invention also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions to cause the computer device to perform the aforementioned method.

[0232] In summary, the L0 factor analysis method based on Newton's interior-point algorithm according to embodiments of the present invention has the following advantages:

[0233] 1. This invention abandons the complex traditional method of introducing auxiliary variables to handle positive semidefinite constraints, avoiding the increased computational complexity and algorithmic complexity caused by the introduction of auxiliary variables. It achieves efficient and concise handling of positive semidefinite constraints, significantly reducing computational resource consumption and redundant operations in the solution process. Specifically, this invention adds a logarithmic barrier function -logdet(·) to the objective function of the optimization problem. Utilizing the property that the logarithmic barrier function tends to infinity at the feasible region boundary, it transforms the original non-convex optimization problem with semidefinite constraints into a series of unconstrained non-convex optimization problems. In particular, the unconstrained optimization problems are L0 regularized optimization problems, which are difficult to directly apply using traditional efficient solution algorithms based on convex optimization theory.

[0234] 2. The embodiments of the present invention deeply integrate the properties of the L0 norm proximal operator with Newton's method, giving full play to the advantage of Newton's method in using the first and second derivatives of the objective function to determine the iteration direction and step size, and overcoming the defect of slow convergence of traditional algorithms.

[0235] 3. The embodiments of the present invention break through the limitations of traditional convex optimization theory and have good applicability to complex non-convex optimization problems such as the estimation of the number of loading factors and the covariance matrix of heterogeneous noise in factor analysis.

[0236] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0237] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0238] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0239] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0240] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0241] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0242] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0243] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0244] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for L0 factor analysis based on Newton's interior-point algorithm, characterized in that, Includes the following steps: Random samples are generated based on a noisy first factorial model. Obtain the covariance matrix of the random sample; Based on the covariance matrix, combined with KL divergence and L0 norm, a first optimization problem with positive semidefinite constraints is constructed. Based on the logarithmic barrier function, the first optimization problem is transformed into an unconstrained second optimization problem; Vectorizing the second optimization problem yields a third optimization problem with unconstrained vector variables; Using Newton's method, based on the third optimization problem, estimates of the number of loading factors and the heterogeneous noise covariance matrix are obtained.

2. The method according to claim 1, characterized in that, The formula used to generate random samples based on the noisy first factor model includes: y i =a+Γu i +w i ,i=1,2,…,N In the formula, y i Γ represents the observation vector; a represents the mean vector; Γ represents the full-rank factor loading matrix; u i Indicates hidden factor; w i represents the heterogeneous noise term; N represents the number of samples.

3. The method according to claim 1, characterized in that, The formula used to obtain the covariance matrix of the random sample includes: In the formula, The covariance matrix of the random sample is represented by N; N represents the sample size, i = 1, 2, ..., N; y i Represents the observation vector; (·) T This indicates the transpose operation.

4. The method according to claim 1, characterized in that, The step of constructing a first optimization problem with positive semidefinite constraints based on the covariance matrix, combined with the KL divergence and L0 norm, includes the following steps: Obtain the nuclear norm of the first symmetric matrix and the L0 norm of the second symmetric matrix; Based on the nuclear norm, L0 norm, and regularization parameters, a second factor model is constructed. Based on the covariance matrix and KL divergence, construct the constraints for the second factor model; The KL divergence constraint in the constraints is embedded into the second factor model to obtain the first optimization problem.

5. The method according to claim 1, characterized in that, The first optimization problem with positive semidefinite constraints is constructed based on the covariance matrix, combined with KL divergence and L0 norm, and the formulas used include: stL,S≥0,L+S>0 In the formula, L represents the first symmetric matrix; S represents the second symmetric matrix; tr(·) represents the trace of the matrix; μ represents the penalty parameter of the KL divergence term; denoted as the covariance matrix of the random sample; det(·) denotes the determinant of the matrix; λ denotes the regularization parameter; and ||·|0 denotes the L0 norm.

6. The method according to claim 1, characterized in that, The process of transforming the first optimization problem into an unconstrained second optimization problem based on the logarithmic barrier function includes the following steps: The logarithmic barrier function and its corresponding logarithmic barrier parameters are added to the first optimization problem to obtain the second optimization problem.

7. The method according to claim 1, characterized in that, The formula used to transform the first optimization problem into an unconstrained second optimization problem based on the logarithmic barrier function includes: In the formula, Where L represents the first symmetric matrix; S represents the second symmetric matrix; λ represents the regularization parameter; ||·||0 represents the L0 norm; tr(·) represents the trace of the matrix; μ represents the penalty parameter for the KL divergence term; represents the covariance matrix of the random sample; -logdet(·) represents the logarithmic barrier function; τ represents the logarithmic barrier parameter.

8. The method according to claim 1, characterized in that, The process of vectorizing the second optimization problem to obtain a third optimization problem with unconstrained vector variables includes the following steps: Construct an orthonormal basis for the set of the first symmetric matrix and the second symmetric matrix; The first symmetric matrix is ​​decomposed into a first linear combination representation of the orthonormal basis; The second symmetric matrix is ​​decomposed into a second linear combination representation of the orthonormal basis; Based on the first linear combination representation and the second linear combination representation, the second optimization problem is vectorized to obtain the third optimization problem.

9. The method according to claim 6, characterized in that, The process of obtaining estimates of the number of loading factors and the heterogeneous noise covariance matrix using Newton's method, based on the third optimization problem, includes the following steps: Determine whether the logarithmic obstacle parameter is greater than a preset threshold. If the logarithmic obstacle parameter is greater than the preset threshold, obtain the stationary point of the third optimization problem with respect to the preset parameter. Based on the stationary points, construct nonlinear stationary point equations; The stationary point equation is solved using Newton's method to reduce the value of the logarithmic barrier parameter, and the process returns to the step of determining whether the logarithmic barrier parameter is greater than a preset threshold.

10. The method according to claim 6, characterized in that, The process of obtaining estimates of the number of loading factors and the heterogeneous noise covariance matrix using Newton's method based on the third optimization problem further includes the following steps: Determine whether the logarithmic barrier parameter is greater than a preset threshold. When the logarithmic barrier parameter is less than or equal to the preset threshold, output the estimated number of loading factors and the heterogeneous noise covariance matrix.

Images