Deep learning-based ill-conditioned matrix SVD (Singular Value Decomposition) preprocessing method, equipment and medium

Abstract

The invention discloses an ill-conditioned matrix SVD decomposition preprocessing method and device based on deep learning, and a medium, and belongs to the technical field of numerical algebra and deep learning. The method comprises the following steps: constructing an iterative deep neural network learning framework, extracting matrix features by using a convolutional layer, and constructing an orthogonal matrix through a House holder reflection decomposition method; training the network by using a mixed loss function, forcing the network to output an approximate diagonal matrix by punishing off-diagonal elements, and performing multiple rounds of iterative optimization by taking a learning result as the input of a new round of training; a precondition is constructed based on a matrix obtained through training, an original ill-conditioned linear equation set is preprocessed, and a preconditioned equation set is solved through an iterative algorithm. The method can be adapted to an ill-conditioned general matrix without a special structure, effectively solves the problems of insufficient approximation precision and poor generalization ability of a traditional method, and remarkably improves the numerical stability, convergence efficiency and calculation precision of high-dimensional ill-conditioned matrix solution.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of numerical algebra and deep learning, in particular to a singular value decomposition (SVD) preconditioning method based on deep learning, a device and a medium, which are suitable for high-dimensional data fitting and singular matrix preconditioning, etc. The preconditioner is constructed by a deep neural network to improve the numerical stability of the singular matrix and adapt to the iterative algorithm for general matrix types to achieve high-precision and high-efficiency solution. BACKGROUND

[0002] The statements in this section merely provide background information related to the present disclosure and can not constitute the prior art.

[0003] In the field of numerical calculation, efficient solution of a singular matrix (condition number ≥ ) is a core technical requirement, especially in the scenarios of large-scale linear equation solving, matrix equation approximation, and numerical optimization of data fitting. However, due to the large condition number, the solution of the linear equation system or the matrix eigenvalue problem is easily distorted by small perturbations. The current methods for solving singular linear equations mainly fall into three categories. The first is the direct elimination method, which is typically represented by the Gauss elimination method. This method transforms the linear equation system into an upper triangular equation system by gradually eliminating unknowns, and then obtains the analytical solution through back substitution. The principle is intuitive and the logic is clear, but when facing high-dimensional singular matrices, the computational complexity reaches , and the floating-point error is amplified in the singular matrix, resulting in a large relative error of the solution, which cannot meet the accuracy requirements. The second is the stationary iterative method, including Jacobi iteration, Gauss-Seidel iteration, etc. This method is based on matrix splitting to construct an iterative format, which is simple to implement but for high-singular matrices, the matrix spectral radius is likely to be greater than 1, leading to iteration divergence or slow convergence speed. The third is the non-stationary iterative method, represented by the generalized minimal residual method (GMRES), the bi-conjugate gradient method (BiCG), the conjugate gradient method (CGS), and the bi-conjugate gradient stabilized method (Bi-CGSTAB). The core of this method is to construct an iterative sequence based on the Krylov subspace, and to approximate the true solution by gradually expanding the subspace.

[0004] To address the above issues, preconditioning techniques are needed to improve the matrix condition to enhance the solution efficiency. The existing preconditioning methods mainly include Jacobi preconditioner, ILU (0) preconditioner, and SSOR preconditioner. However, the existing methods still have certain problems and defects: Problem 1: The existing preconditioner has poor adaptability and cannot cover singular general matrices Defect performance: traditional preconditioners such as Jacobi, ILU (0), SSOR depend on the specific structure of the matrix (such as diagonal dominance, sparsity), and the preconditioning effect of the general matrix without special structure is significantly reduced, and even cannot improve the matrix state.

[0005] Causes: traditional preconditioner design is based on the explicit structure characteristics of the matrix, without considering the randomness and diversity of the eigenvalue distribution of the ill-conditioned matrix, and lacks the ability to adaptively learn the characteristics of the matrix. The eigenvalue distribution of the general type of ill-conditioned matrix has no fixed rule, and the manually designed universal preconditioner needs to consider multiple matrix characteristics, making it difficult to balance adaptability and computational efficiency.

[0006] Problem 2: The approximation accuracy of the preconditioner is insufficient, and the numerical stability and solving efficiency are difficult to balance Defect performance: the preconditioner constructed by the existing method is difficult to accurately approximate the optimal preconditioner, resulting in slow residual convergence during iterative solving, and the computational complexity is still high (close to O (n³)) in high-dimensional scenarios.

[0007] Causes: the mathematical properties of matrix orthogonal transformation and diagonalization are not fully utilized, and there is a lack of effective loss function to guide the optimization of the preconditioner, which cannot balance the "approximation accuracy" and "computational cost". Traditional methods cannot iteratively optimize the preconditioner through data-driven methods, rely on empirical parameter adjustment, and are difficult to adapt to matrices with different degrees of ill-conditioning (condition number difference). SUMMARY

[0008] The purpose of the present application is to solve the technical problems of poor adaptability, insufficient approximation accuracy of the preconditioner, and difficulty in balancing numerical stability and computational efficiency when existing preconditioning techniques (such as Jacobi, ILU, etc.) are used to process general ill-conditioned matrices without special structure, resulting in slow convergence or even divergence during iterative solving, etc. The present application provides a deep learning-based singular value decomposition (SVD) preconditioning method for ill-conditioned matrices, equipment, and media. The method is based on the mathematical property of the SVD of the matrix that is easy to invert, and uses an iterative deep neural network learning framework combined with Householder reflection decomposition technology to adaptively construct a numerically stable orthogonal matrix. A hybrid loss function that penalizes non-diagonal elements is used to guide the network to perform multiple rounds of iterative training to accurately approximate a diagonal matrix, and then an efficient preconditioner that can significantly reduce the condition number of the coefficient matrix is constructed, achieving high-precision and high-efficiency solving of high-dimensional ill-conditioned linear equations.

[0009] The technical solutions of the present application are as follows: The deep learning-based SVD decomposition preconditioning method for ill-conditioned matrices comprises: The iterative deep neural network learning framework is constructed, the deep neural network is based on the core idea of SVD decomposition, matrix features are extracted by using a convolution layer, and orthogonal matrices U and V are constructed by using a Householder reflection decomposition method; The deep neural network is trained by using a hybrid loss function, the network output is forced to approximate a diagonal matrix Sigma by penalizing non-diagonal elements, the learning result of the deep neural network is taken as an input of a new round of training for multiple rounds of iteration, and the orthogonal matrices U and V and the diagonal matrix Sigma corresponding to the target ill-conditioned matrix are obtained; Based on the mathematical property that SVD is easy to invert, the orthogonal matrices U and V and the diagonal matrix Sigma obtained by training are used to construct a preconditioner M=VΣ -1 U ; The preconditioner M is used for preprocessing the original linear equation set Ax=b to obtain a preconditioned equation set MAx=Mb, and an iterative algorithm is used to solve the preconditioned equation set.

[0010] Further, the deep neural network comprises an input layer, a hidden layer structure and an output layer connected in sequence. The input layer is configured to receive a tensor converted from a matrix to be processed. The hidden layer structure comprises a plurality of cascaded convolution blocks, each of which comprises a two-dimensional convolution layer, a normalization layer and a nonlinear activation layer; and the plurality of cascaded convolution blocks are configured as follows: the convolution kernel size and the number of channels are increased layer by layer in the front convolution blocks to extract the global structural features of the matrix, and the convolution kernel size and the number of channels are reduced layer by layer in the rear convolution blocks to aggregate the features and compress the feature dimension. The output layer is configured to output a feature vector required for constructing an orthogonal matrix.

[0011] Further, the orthogonal matrices U and V are constructed by using the Householder reflection decomposition method, which specifically comprises: The feature vector output by the output layer is subjected to L2 normalization processing to obtain a series of normalized feature vectors y; A unit matrix I is introduced, and the formula H=I-2yy The Householder reflection matrix H is constructed; The orthogonal matrices U and V are constructed by using the Householder reflection matrix H.

[0012] Further, the deep neural network is trained by using a hybrid loss function, which specifically comprises: The product of the transpose of the orthogonal matrix U, the matrix to be processed A and the orthogonal matrix V is calculated to obtain a transformation matrix U AV; constructing the mixed loss function, the mixed loss function comprising a penalty term for non-diagonal elements of the transformation matrix U a penalty term for non-diagonal elements of the AV; by minimizing the mixed loss function, reducing the numerical value of the non-diagonal elements, so that the transformation matrix approximates the diagonal matrix Σ.

[0013] Further, before training the deep neural network using the mixed loss function, further comprising: a step of constructing a batch training set: generating a random matrix conforming to a normal distribution as a general matrix; using SVD decomposition to construct a matrix with singular values distributed in a preset interval as a general matrix of an ill-conditioned matrix, the condition number of the ill-conditioned matrix satisfying a preset ill-conditioned characteristic requirement; converting the general matrix and the ill-conditioned general matrix into batch tensor data as training input of the deep neural network.

[0014] Further, based on the mathematical property of SVD that is easy to invert, the orthogonal matrix U, V and diagonal matrix Σ obtained by training are used to construct a preconditioner M=VΣ -1 U Further comprising:using low-rank approximation or block construction to lightweight the preconditioner M to reduce the storage and calculation cost of the preconditioner M.

[0015] Further, the iterative algorithm comprises at least one of the following: Jacobi iteration method, Gauss-Seidel iteration method, successive over-relaxation iteration method (SOR), generalized minimum residual method (GMRES), bi-conjugate gradient method (BiCG) or bi-conjugate gradient stabilized method (BiCGSTAB).

[0016] Further, the solving of the preconditioned equation set using the iterative algorithm specifically comprises: presetting a maximum number of iteration steps and a target accuracy threshold; During the solving process using the iterative algorithm, the residual norm of the current solution or the change amount of the solution is calculated; if the residual norm or the change amount of the solution is less than the target accuracy threshold, or the number of iterations reaches the maximum number of iteration steps, the iteration is stopped and the solving result is output.

[0017] The application further provides an electronic device, comprising: at least one processor; and a memory connected in communication with the at least one processor; The memory stores instructions executable by the at least one processor, and the at least one processor executes the instructions stored in the memory, so that the at least one processor executes the method as described above.

[0018] The application further provides a computer readable storage medium for storing instructions, when the instructions are executed, the method as described above is realized.

[0019] Compared with the prior art, the application has the following beneficial effects: 1. The adaptability is significantly improved: the general matrix with no special structure can be processed, covering multiple scenes such as high-dimensional data fitting and matrix equation approximation, and breaking through the structure dependence limitation of the traditional preconditioner.

[0020] 2. The solving accuracy is improved: the mixed loss function is used to guide the network to approximate the optimal diagonalization effect, and the solving residual of the equation set after preconditioning is reduced by 1-2 orders of magnitude compared with the traditional method, and the absolute error of the solution is controlled in a smaller range.

[0021] 3. The solving efficiency is improved: the preconditioner reduces the convergence step number of the iterative algorithm by 30%-60%, combined with the parallel training characteristics of the neural network, the overall computational complexity of the high-dimensional matrix (n≥1000) solving is reduced to O (n²) level, far exceeding the traditional O(n³) method.

[0022] 4. The numerical stability is enhanced: the orthogonal matrix constructed by the Householder reflection has natural numerical stability, effectively inhibiting the amplification of floating-point number errors in the ill-conditioned matrix, and the matrix condition number can be reduced to below 10³, and the iterative process has no risk of divergence.

[0023] 5. Strong compatibility: can directly adapt to multiple mainstream iterative algorithms such as GMRES, BiCGSTAB, and SOR, without modifying the algorithm itself, and is convenient for engineering application. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments described in the embodiments of the application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0025] Figure 1 It is a full process of SVD decomposition preprocessing and solving of an ill-conditioned matrix based on deep learning. Figure 2 It is a convolutional neural network (CNN) structure. Figure 3Line graphs showing the convergence of errors before and after the application of preconditioners for different types of matrices in six iterative algorithms; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0026] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0027] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0028] Example 1 The deep learning-based preprocessing method for ill-conditioned matrix SVD decomposition revolves around the core process of "iterative deep learning to construct orthogonal matrices → high-precision approximation via SVD decomposition → preconditioner generation → efficient solution of ill-conditioned matrix equations." Through multiple rounds of iterative optimization, it overcomes the diagonalization bottleneck of single-round training, ultimately achieving high-precision and high-efficiency solutions for ill-conditioned general matrices. The overall process consists of five core steps: data preparation, neural network and multi-round iterative training, SVD preconditioner generation, preprocessing iterative solution, and result verification.

[0029] The essential core structures include: SVD preconditioner neural network design, hybrid loss function, iterative learning framework, preconditioner construction logic, and iterative algorithm adaptation module; optional structures involve: neural network backbone network type, loss function weight configuration, number of iterative training rounds, preconditioning method, iterative algorithm selection, parallel optimization strategy, regularization term addition, etc., which can be flexibly adjusted according to the actual scenario.

[0030] Please see Figure 1This document fully presents the complete technical chain of this invention, encompassing "ill-conditioned matrix data support - deep learning modeling - preconditioner construction - iterative solution - effect verification." It clearly demonstrates the logical connection between the five core stages: "data preparation → neural network and multi-round iterative training → preconditioner generation → iterative solution → result verification." It intuitively reflects the transformation process from raw matrix data to high-precision solution results, serving as the core execution path diagram for the engineering implementation of this solution. Specifically, it is as follows: I. Data Preparation Core objective: To construct a batch training set consisting of "general matrices (normally distributed random matrices) + ill-conditioned general matrices (SVD construction, condition numbers satisfying ill-conditioned properties)," and simultaneously provide right-hand side data for verifying the solution of the system of equations. This provides diverse matrix feature data sources for subsequent neural network training and lays the data foundation for "comparison with real solutions" in the result verification stage, serving as the underlying support for the entire scheme. It simulates real-world application scenarios (such as high-dimensional data fitting and matrix equation approximation) to ensure that the neural network can learn the features of different types of matrices and adapt to the subsequent preconditioner generation requirements.

[0031] General matrix: Generates a random matrix that conforms to a normal distribution using np.random.randn(n,n) to ensure data universality.

[0032] Ill-conditioned general matrices: Using the SVD decomposition method, through "orthogonal matrices" Singular value matrix "Construction" (Singularity in) Uniform distribution ensures condition number (satisfying pathological characteristics).

[0033] Batch construction of training data: The single matrix generation logic is expanded in batches to output corresponding tensor data, which can meet the batch training requirements of neural networks.

[0034] The optional structure involves three aspects: First, the matrix dimension, which can be flexibly adjusted according to the scale of the actual application scenario. Matrixes of different dimensions can adapt to the neural network structure parameters to meet the solution requirements of different scenarios. Second, the eigenvalue range, which has flexible adjustment space for the eigenvalue distribution interval. It can be adapted to the degree of ill-conditioning of the matrix. For different scenarios such as weakly ill-conditioning and strongly ill-conditioning, the eigenvalue distribution range can be adjusted to make the preconditioner training more in line with the matrix characteristics and ensure the stability of the solution. Finally, the number of samples, which can be flexibly increased or decreased according to data resources. In the case of small datasets, the training time can be shortened while ensuring the basic training accuracy. In the case of large datasets, the number of samples can be increased to further improve the adaptability of the preconditioner.

[0035] II. Construction of Deep Neural Networks This method extracts local and global features of a matrix using 2D convolutional layers, constructs a numerically stable orthogonal matrix using the Householder reflection decomposition method, and employs a hybrid loss function (penalizing off-diagonal elements) to guide diagonal matrix approximation. Combined with the Adam optimizer, it achieves multi-round iterative training, overcoming the diagonalization accuracy bottleneck of traditional single-round training and improving the accuracy of SVD decomposition. This is a core technical step in solving the problem of insufficient accuracy in traditional preconditioner approximation. Details are as follows: Core objective: To achieve adaptive construction of orthogonal matrices U / V based on deep neural networks, and to improve the accuracy of SVD decomposition through multiple rounds of iterative optimization.

[0036] Feature extraction layer: A fixed 2D convolutional structure is used to extract local and global features of the matrix by sliding the convolution kernel, providing feature support for the generation of orthogonal matrices.

[0037] Householder reflection matrix construction: Utilizing the learnable identity matrix and the normalized features of the network output, using formula I - 2yy Generate orthogonal matrices to ensure numerical stability.

[0038] Iterative learning framework: The neural network encapsulates multi-round iterative logic. By using the output of the previous round of training as the input of the next round, it achieves multi-round optimization, thereby realizing the cumulative update of the orthogonal matrix and continuous diagonalization approximation.

[0039] Loss function: A hybrid loss function is used, which forces the network output to approximate the diagonal matrix Σ by penalizing off-diagonal elements.

[0040] Optimizer configuration: The base optimizer is Adam, and the learning rate is... Adjust the core parameters.

[0041] There are five main possible structural options: First, the number of convolutional layers, which can be flexibly adjusted according to feature extraction requirements to adapt to ill-conditioned matrix scenarios with varying complexity; second, the learning rate, which needs to be optimized based on the convergence speed of the loss function. A learning rate that is too large is prone to oscillation, while a learning rate that is too small will result in slow convergence; third, the optimizer, which can be replaced with HN_Adam, suitable for large data scenarios. By dynamically adjusting the parameter update step size through adaptive norm, it can effectively improve the model's generalization performance and prevent overfitting; fourth, the activation function, which can be flexibly replaced according to the characteristics of the matrix elements to adapt to matrix scenarios with different element distributions and ensure the effectiveness of neural network feature transformation; and fifth, the number of iterative learning epochs, which is an empirical value based on experimental verification. The core of this parameter is to be flexibly adjusted based on the matrix complexity, the degree of ill-conditioning (the size of the condition number), and the difficulty of feature extraction.

[0042] In this embodiment, the deep neural network is as follows: Figure 2As shown, the architecture of the core feature extraction module in the deep neural network structure—the convolutional neural network (CNN)—is demonstrated. This network is responsible for extracting local and global features from the input high-dimensional matrix, providing key parameters for the subsequent construction of the Householder reflection matrix and the approximation of singular value decomposition (SVD), and directly serving the technical goals of "low-dimensional approximation of SVD" and "generation of preconditioning subparameters".

[0043] The specific functions of each module are explained below: Input layer: Receives batch processing of n×n matrices (n can be extended to more than 1000), converts them into 4-dimensional tensors (shape [B, 1, n, n], where B is the batch size and 1 is the number of input channels), adapts to the input format of CNN, and supports parallel processing of high-dimensional matrices.

[0044] The CNN feature extraction module consists of 6 convolutional layers, progressively extracting hierarchical features from the matrix, specifically including: First convolutional layer: 1→16 channels, kernel size 3×3, padding=1 (keeping the size unchanged), captures the nearest neighbor correlation features of matrix elements through local receptive fields; Batch normalization (BatchNorm2d(16)): standardizes the output features of this layer, accelerates training convergence and alleviates gradient vanishing; SiLU activation function: Introduces nonlinear transformation (f(x)=x·sigmoid(x)) to enhance feature mapping capability and avoid feature expression bottleneck caused by linear transformation; The second convolutional layer: 16→32 channels, 5×5 kernel, padding=2, to expand the receptive field to capture medium-range element association features, followed by BatchNorm2d(32) and SiLU; The third convolutional layer: 32→64 channels, 7×7 convolutional kernel, padding=3, further expanding the receptive field to extract global matrix structure features, followed by BatchNorm2d(64) and SiLU; Convolutional layers 4-6: channels 64→32, 32→16, 16→2, with kernel size decreasing in reverse (7×7→5×5→3×3). Feature aggregation is achieved through channel compression, and the final output is a 2-channel feature vector (shape [B, 2, n,n]), followed by corresponding batch normalization and SiLU activation.

[0045] Output layer: Outputs 2-channel n×n feature vectors, which, after L2 normalization, are used to construct the Householder reflection matrix (using formula I - 2yy). , where y is the normalized eigenvector element), provides the basic parameters for the iterative update of the U and V matrices, and realizes high-precision approximation of SVD decomposition.

[0046] III. Preconditioner generation: Based on the orthogonal matrices U and V and the diagonal matrix Σ output by the neural network, M = VΣ is calculated according to the formula. -1 U Constructing preconditioners and simultaneously reducing storage and computation costs through lightweight modifications such as low-rank approximation or block construction effectively improves the numerical properties of the original ill-conditioned matrix and controls the matrix condition number within a reasonable range. This is a crucial transformation step connecting deep learning models and iterative solutions; specifically as follows: Core objective: To construct efficient preconditioners based on the SVD decomposition results output by neural networks, thereby reducing the condition number of the original matrix.

[0047] Preconditioner construction logic: based on the SVD decomposition principle A=UΣV Utilizing the mathematical property that SVD is easy to invert, a preconditioner M=VΣ is constructed using the orthogonal matrices U and V and the diagonal matrix Σ predicted after neural network training. -1 U The core idea is to use the fact that the inverse of an orthogonal matrix is ​​equal to its transpose, and Σ is a diagonal matrix. The inverse matrix is ​​easy to calculate, that is, by taking the reciprocals of the diagonal elements, which reduces the construction complexity of M.

[0048] Lightweight preconditioner strategy: Lightweight modification is carried out through low-rank approximation or block preconditioner construction to reduce the storage and computation costs of preconditioners.

[0049] The optional structures include two aspects: First, the preconditioner construction method, which combines traditional preconditioning methods or hybrid strategies, with the core objective of ensuring a significant reduction in the condition number; Second, the selection of a lightweight preconditioner strategy, which can include methods such as sparsity pruning, with the core objective of reducing the storage and computation costs of the preconditioner while ensuring the reduction in the condition number, thus adapting to high-dimensional matrices or resource-constrained scenarios.

[0050] IV. Iterative Solution The original system of equations Ax=b is transformed into a preconditioned system of equations (e.g., MAx=Mb) using a preconditioning method. This is combined with different types of stopping criteria, such as the change in solution and the residual norm, along with a maximum iteration step limit. This ensures solution accuracy while avoiding invalid iterations, achieving a balance between "high accuracy" and "high efficiency." This is the core execution step for solving ill-conditioned matrix equation systems in this invention; specifically as follows: Core objective: To combine preconditioners with iterative algorithms to achieve efficient solutions to ill-conditioned matrix equation systems.

[0051] Iterative algorithm integration: By combining the preconditioner M with the iterative algorithm, the original equation system Ax = b is preconditioned, which significantly reduces the condition number and improves the iteration convergence speed.

[0052] Stopping Criteria: A dual stopping condition is adopted, and a variety of stopping criteria (including the change in solution, residual norm, etc.) and the maximum number of iterations are used to balance the accuracy and efficiency of the solution.

[0053] The optional structures mainly have three directions: First, the maximum number of iterations, which can be adjusted according to the theoretical convergence steps corresponding to the matrix dimension; second, the algorithm halting criterion, which can be replaced with various methods such as error as needed, and can be adapted to convergence judgment scenarios under different data scales; third, the preconditioning sub-application method, which can select right preconditioning or symmetric preconditioning according to actual needs to adapt to different application scenarios.

[0054] V. Result Verification Using the "error norm (i.e., the L2 norm of the difference between the iterative solution and the exact solution)" as the core quantitative indicator, this study compares the convergence effects of different iterative algorithms before and after using preconditioners. Furthermore, by solving the same matrix multiple times, the fluctuations of key indicators are analyzed. The effectiveness of the preconditioners and the robustness of the method are verified from both "accuracy" and "stability" dimensions, providing a quantitative basis for the technical advantages of the proposed solution. Specifically: Core objective: To quantitatively verify the effectiveness of preconditioners and compare the convergence performance of different solution iterative methods.

[0055] Core metric calculation: The error norm is selected as the standard for quantifying effectiveness. For different iterative methods, the line graphs of the error norm changing with the number of iterations before and after using the preconditioner are compared. This intuitively and accurately measures the difference between the current solution and the true solution, and objectively evaluates the effect of the preconditioner. Compared with other judgment methods, it has better numerical stability and reliability.

[0056] Stability verification: Solve the same matrix multiple times, analyze and compare key indicators to verify the robustness of the method.

[0057] The optional structure covers two aspects: first, the number of repeated verifications, which can be adjusted according to different scenario requirements, with the core being to verify the stability of the results through multiple solutions; second, additional accuracy indicators, which can be used to further refine the convergence stability analysis and enhance the persuasiveness of the verification by adding auxiliary means such as "residual norm decay rate" and "relative error fluctuation range".

[0058] In this embodiment, it should also be noted that, Figure 3 Line graphs showing the convergence of errors before and after the application of preconditioners for different types of matrices in six iterative algorithms; Figure 3Focusing on two scenarios—"general matrices" and "ill-conditioned matrices of general types"—the system presents six iterative algorithms: Jacobi iteration, Gauss-Seidel iteration, SOR, BiCG, BiCGSTAB, and GMRES. It compares the error norm with the number of iterations under two states: "without using the deep neural network to learn and construct the preconditioner" and "with the preconditioner applied." The core verification demonstrates the adaptability and convergence optimization effect of the deep neural network preconditioner of this invention on different types of matrices and different iterative algorithms.

[0059] The comparison dimensions are: horizontally, the number of iteration steps (reflecting the progress of the solution process), and vertically, the error norm (the L2 norm of the iterative solution and the exact solution; the smaller the value, the higher the solution accuracy); within the same subgraph, different colors are used to distinguish between the two types of curves: "without preconditioner" and "with preconditioner".

[0060] Grouping logic: Subgraphs are divided according to "matrix type + iterative algorithm", which includes 2 types of matrices × 6 algorithms = 12 subgraphs, clearly showing the convergence differences of different combinations; all curves are based on the same initial conditions, target accuracy threshold and maximum number of iterations to ensure the fairness of the comparison.

[0061] Results Explanation: Stable iterative algorithms (Jacobi, Gauss-Seidel, SOR): For both general and ill-conditioned general matrices, without the application of preconditioners, the error norm exhibits numerical divergence, failing to achieve the target accuracy. This confirms the convergence limitations of traditional iterative algorithms for certain matrix types. After applying the preconditioner constructed by the deep neural network, the error norms of both types of matrices decrease rapidly with the number of iterations, eventually converging stably to the target accuracy range without any risk of divergence, thus verifying the key improvement effect of the preconditioner on convergence.

[0062] Non-stationary iterative algorithms (BiCG, BiCGSTAB, GMRES): For general matrices and ill-conditioned general matrices, without the application of preconditioners, although the error norm can gradually decrease and eventually reach the target accuracy, the number of convergence steps is relatively large, resulting in low convergence efficiency. After applying the preconditioner, the rate of decrease of the error norm corresponding to the two types of matrices is significantly accelerated, and the number of iterations required to reach the target accuracy is greatly reduced, verifying the optimization value of the preconditioner for convergence speed.

[0063] In summary, the deep learning-based ill-conditioned matrix SVD decomposition preprocessing method proposed in this embodiment innovatively combines the Householder reflection decomposition method with deep learning. Through adaptive construction via neural network, it has both numerical stability and trainability compared to traditional orthogonalization methods, and its construction efficiency is higher. Simultaneously, a hybrid loss function is designed to achieve matrix diagonalization approximation by penalizing off-diagonal elements, thus solving the problem of insufficient accuracy in traditional preconditioner approximation.

[0064] Furthermore, an innovative iterative deep neural network learning framework was designed, which uses the learning results of the previous round of the neural network as the input for the next round of training. Through multiple rounds of iteration, it is gradually optimized, breaking through the diagonal approximation bottleneck of single-round training.

[0065] Furthermore, based on the preconditioner construction logic of SVD decomposition, the results of deep learning output are transformed into efficient preconditioners, realizing a deep integration of "data-driven learning" and "numerical computation optimization".

[0066] Based on the same technical concept, embodiments of the present invention also provide an electronic device that can implement the deep learning-based ill-conditioned matrix SVD decomposition preprocessing method provided in the above embodiments of the present invention. In one embodiment, the electronic device can be a server, a terminal device, or other electronic devices. Figure 4 As shown, the electronic device may include: At least one processor and a memory connected to the at least one processor. In this embodiment of the invention, the specific connection medium between the processor and the memory is not limited. Figure 4 The example used is the connection between the processor and memory via a bus. The bus... Figure 4 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. Buses can be divided into address buses, data buses, control buses, etc., but for ease of representation, [the specific bus type is not shown here]. Figure 4 The processor is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, a processor can also be called a controller; there are no restrictions on the name.

[0067] In this embodiment of the invention, the memory stores instructions executable by at least one processor. By executing the instructions stored in the memory, the at least one processor can perform the deep learning-based ill-conditioned matrix SVD decomposition preprocessing method discussed above. The processor can implement... Figure 4 The functions of each module in the device shown.

[0068] The processor is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory and calling data stored in memory, it can monitor the device's various functions and process data, thereby enabling overall monitoring of the device.

[0069] In an alternative design, the processor may include one or more processing units. The processor may integrate an application processor and a modem processor, wherein the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may also not be integrated into the processor. In some embodiments, the processor and memory may be implemented on the same chip; in some embodiments, they may also be implemented separately on separate chips.

[0070] The processor can be a general-purpose processor, such as a CPU, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the deep learning-based ill-conditioned matrix SVD decomposition preprocessing method disclosed in the embodiments of this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0071] Memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia cards, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), and electrically erasable programmable read-only memory (EPROM). Only memory (EEPROM), magnetic storage, magnetic disks, optical disks, etc. A memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory in embodiments of this invention can also be a circuit or any other device capable of performing storage functions for storing program instructions and / or data.

[0072] By designing and programming the processor, the code corresponding to the deep learning-based ill-conditioned matrix SVD decomposition preprocessing method described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute the steps of the methods described in the foregoing embodiments during runtime. How to design and program the processor is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0073] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the deep learning-based ill-conditioned matrix SVD decomposition preprocessing method described above.

[0074] In some alternative embodiments, the present invention also provides that various aspects of the deep learning-based ill-conditioned matrix SVD decomposition preprocessing method can also be implemented as a program product comprising program code that, when the program product is run on a device, causes the control device to perform the steps in the deep learning-based ill-conditioned matrix SVD decomposition preprocessing method described above according to various exemplary embodiments of the present invention.

[0075] It should be noted that although several units or sub-units of the apparatus have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the invention, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units. Furthermore, although the operation of the method of the invention is described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0076] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can be implemented in one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs) containing computer-usable program code. The form of a computer program product implemented on ROM, optical memory, etc.

[0077] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a server, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] Program code for performing the operations of this invention can be written using any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0079] In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0080] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0081] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0082] In addition, in some embodiments, a computer program product is proposed that, when executed by a processor, implements the above-described deep learning-based ill-conditioned matrix SVD decomposition preprocessing method.

[0083] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

[0084] This background section is provided to generally present the context of the invention. The work of the currently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither expressly nor impliedly acknowledged as prior art to the invention.

Claims

1. A method for ill-conditioned matrix SVD decomposition preprocessing based on deep learning, characterized in that, The application relates to a method for constructing a preconditioner for a linear equation system. The method comprises the following steps: An iterative deep neural network learning framework is constructed, the deep neural network is based on the core idea of SVD decomposition, matrix features are extracted by using a convolution layer, and an orthogonal matrix U and V are constructed by using a Householder reflection decomposition method; Based on the mathematical property that SVD is easy to inverse, by using the SVD inverse formula, the orthogonal matrix U, V and the diagonal matrix Σ obtained by training are used to realize the construction of the preconditioner M=VΣ -1 U ; A hybrid loss function is used to train the deep neural network, the network output is forced to approximate a diagonal matrix Sigma by punishing non-diagonal elements, and the learning result of the deep neural network is taken as the input of a new round of training for multiple rounds of iteration to obtain the orthogonal matrix U, V and the diagonal matrix Sigma corresponding to the target ill-conditioned matrix; 2. The deep learning based ill-conditioned matrix SVD decomposition preprocessing method of claim 1, wherein, The original linear equation system Ax=b is preprocessed by using the preconditioner M to obtain a preconditioned equation system MAx=Mb, and an iterative algorithm is used to solve the preconditioned equation system. The deep neural network comprises an input layer, a hidden layer structure and an output layer which are connected in sequence. The input layer is used for receiving a tensor converted from a to-be-processed matrix; The hidden layer structure is composed of multiple cascaded convolution blocks, each convolution block comprises a two-dimensional convolution layer, a normalization layer and a nonlinear activation layer; the multiple cascaded convolution blocks are configured as follows: the convolution kernel size and the channel number are increased layer by layer in the front convolution blocks to extract the global structure features of the matrix, and the convolution kernel size and the channel number are reduced layer by layer in the rear convolution blocks to aggregate the features and compress the feature dimension; 3.The deep learning based ill-conditioned matrix SVD decomposition preprocessing method of claim 2, wherein, The output layer is used for outputting a feature vector required for constructing an orthogonal matrix. The feature vector output by the output layer is subjected to L2 normalization processing to obtain a series of normalized feature vectors y; Introducing the identity matrix I, the formula H = I - 2yy Constructing the Householder reflection matrix H; The Householder reflection matrix H is used to construct the orthogonal matrix U and V.

4. The deep learning based ill-conditioned matrix SVD decomposition preprocessing method of claim 3, wherein, The deep neural network is trained by using the hybrid loss function, and the transformation matrix is forced to approximate the diagonal matrix Sigma by minimizing the hybrid loss function and reducing the numerical value of the non-diagonal elements. computing the product of the transpose of the orthogonal matrix U, the matrix A to be processed and the orthogonal matrix V, obtaining a transformation matrix U AV; The mixed loss function is constructed, which includes a penalty term for the off-diagonal elements of the transformation matrix U a penalty term for off-diagonal elements of the transformation matrix U in AV Before the deep neural network is trained by using the hybrid loss function, the following steps are further included:

5. The deep learning based ill-conditioned matrix SVD decomposition preprocessing method according to claim 4, characterized in that, A step of constructing a batch training set is included: A random matrix conforming to a normal distribution is generated as a general matrix; An SVD decomposition is used to construct a matrix with singular values distributed in a preset interval as an ill-conditioned general matrix, and the condition number of the ill-conditioned matrix meets the preset ill-conditioned characteristic requirement; The general matrix and the ill-conditioned general matrix are converted into batch tensor data as the training input of the deep neural network. The preconditioner M is lightweighted by using a low-rank approximation or a block construction method to reduce the storage and calculation cost of the preconditioner M.

6. The deep learning based ill-conditioned matrix SVD decomposition preprocessing method according to claim 5, characterized in that, The SVD-based mathematical property of easy inversion is used to obtain the orthogonal matrix U, V and diagonal matrix Σ by training, and the SVD inversion formula is used to realize the construction of preconditioner M=VΣ -1 U , and further comprising: The iterative algorithm comprises at least one of the following: a Jacobi iteration method, a Gauss-Seidel iteration method, a successive over-relaxation iteration method (SOR), a generalized minimum residual method (GMRES), a bi-conjugate gradient method (BiCG) or a bi-conjugate gradient stabilized method (BiCGSTAB).

7. The deep learning based ill-conditioned matrix SVD decomposition preprocessing method according to claim 6, characterized in that, The preconditioned equation system is solved by using the iterative algorithm, and the method comprises the following steps:

8. The deep learning based ill-conditioned matrix SVD decomposition preprocessing method according to claim 7, characterized in that, A maximum iteration step number and a target precision threshold value are preset. ​ In the solving process by using the iterative algorithm, a residual norm of a current solution or a variation of the solution is calculated; if the residual norm or the variation of the solution is less than the target precision threshold value or the number of iterations reaches the maximum number of iteration steps, the iteration is stopped and a solving result is output.

9. An electronic device, comprising: Comprise: at least one processor; and a memory connected in communication with the at least one processor; Wherein, the memory stores instructions executable by the at least one processor, and the at least one processor, by executing the instructions stored in the memory, causes the at least one processor to perform the method of any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium is used to store instructions, when the instructions are executed, the method of any one of claims 1-8 is realized.

Images