A method and device for predicting offshore climate based on fast matrix decomposition method
Through the MPI+OpenMP task model and improved two-dimensional data layout, distributed low-rank Cholesky decomposition is optimized, solving the problem of low efficiency of large-scale matrix decomposition in distributed environments, and achieving efficient offshore climate prediction and resource utilization.
Patent Information
- Application Number
- CN202111617373.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-27
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-12-27
AI Technical Summary
In a distributed environment, it is difficult for the existing technology to effectively parallelize large-scale low-rank matrix decomposition algorithms, resulting in low computational efficiency and resource utilization of offshore climate prediction, and developers need to learn APIs of multiple tools, which increases learning costs.
Using the MPI+OpenMP task model, the distributed low-rank Cholesky decomposition is realized by generating task dependency graphs and non-blocking communications, and an improved two-dimensional data layout and efficient decomposition and compression strategy are proposed to optimize the calculation and communication process.
It improves the operation efficiency of large-scale decomposition algorithms, reduces communication overhead and memory usage, realizes efficient offshore climate prediction, and simplifies the programming process, which is suitable for distributed environments.
Smart Images

Figure CN114298406B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method and device for forecasting offshore climate based on a fast matrix decomposition method. The method runs in a distributed system, relies on MPI and OpenMP task models, and can be used for the rapid forecasting of physical data such as temperature, precipitation, wind speed, air pressure, humidity, etc. in offshore geostatistical applications. Background Art
[0002] Climate and environmental prediction applications are one of the main physical applications running on supercomputers in various countries today. Such applications are indispensable for scientists to conduct geospatial statistics-related research. The climate of offshore areas has a significant impact on marine fishery resources. Limited by the location of the collection equipment, it is usually necessary to predict the remaining unevenly distributed locations based on existing observation data and models to obtain complete climate physical data. Before prediction, scientists usually measure a large amount of physical data (physical data can be temperature, precipitation, humidity, wind speed, air pressure, etc.) in a specified geographical area regularly or irregularly, and then use Gaussian spatial random field models to model these data. The data model is finally determined by estimating the model parameters based on the Gaussian log-likelihood function based on a large covariance matrix. With the help of the observed values that have been obtained, the model can be used to predict physical data at other geographical points in the area. The main steps of this type of prediction problem can be summarized as a large symmetric positive definite linear system solution process, which can be further decomposed into matrix Cholesky pre-decomposition and solution based on forward and backward propagation. Since the calculation amount of the solution process is too large, the main running time of the prediction program will be concentrated in this process, so it is very necessary to optimize the solution process of the prediction program.
[0003] One optimization method is to use a higher-performance solution algorithm. The parallel model based on the OpenMP task dependency graph on a shared memory architecture has been proven to be very suitable for decomposition algorithms with irregular characteristics. This model allows the program to dynamically generate a task dependency graph at runtime and dynamically schedule all available threads to execute on the decomposed tasks without violating the dependencies between tasks. Compared with the traditional parallel model, this model eliminates the explicit synchronization barriers in the algorithm and maximizes the use of dynamic scheduling strategies to improve performance and resource utilization. However, it is very difficult to design a decomposition algorithm based on the task dependency graph in a distributed environment, because the MPI standard itself does not provide any functions related to the task model, and there is no interoperability between the task models provided by solutions such as OpenMP, which makes the algorithm difficult to port. Some existing task model solutions in distributed environments are tools implemented by different teams that bypass the MPI standard and the OpenMP standard. For other developers, they need to learn the API of each tool from scratch, which greatly increases the learning cost of parallel decomposition algorithms.
[0004] Another optimization method is to use low-rank matrix decomposition. The covariance coefficient matrix in the prediction program is usually a dense matrix overall, but has sparse characteristics locally. A low-rank approximation algorithm can be used to low-rank some computing cores with excessive computational intensity during the decomposition process, thereby effectively improving the efficiency of the algorithm at the cost of reducing some accuracy.
[0005] Therefore, it is necessary to redesign the large-scale low-rank matrix decomposition and solution process in the prediction program using the MPI+OpenMP task model under the latest standard, and try to analyze and optimize the operating efficiency, memory usage and other aspects of the low-rank decomposition under the new hybrid architecture. Summary of the invention
[0006] The purpose of the present invention is to propose a near-shore climate prediction method based on a fast matrix decomposition method in response to the deficiencies of the prior art. The present invention proposes a stable and easily reusable parallelization strategy under a new MPI+X paradigm (i.e., MPI+OpenMP task model), thereby realizing a near-shore climate rapid prediction method that relies on low-rank Cholesky decomposition under the new paradigm.
[0007] The technical solution steps adopted by the present invention to solve the technical problem are as follows:
[0008] A method for offshore climate prediction based on fast matrix decomposition is to use the existing Gaussian random field model based on Matern kernel function and the collected offshore climate physical data to predict the physical value of any spatial point in the area; use OpenMP to generate a task dependency graph and schedule the decomposed tasks in the dependency order; use MPI to control the matrix block, and then perform non-blocking communication between different processes. The method is implemented using the following steps:
[0009] Step (1): Each process divides the data and calculates the distance between any positions;
[0010] 1-1 Obtain the coordinate data matrix L1 of the position to be predicted, the coordinate data matrix L2 of the observed position and the corresponding physical value observation vector Z2;
[0011] The physical value observation vector Z2 includes temperature, precipitation, wind speed, air pressure, humidity, etc.;
[0012] The coordinate data matrix includes the longitude and latitude values of each coordinate point;
[0013] 1-2 configure multiple MPI processes based on MPI communication. Each process combines the improved two-dimensional data layout and divides the data distribution in step 1-1 according to the number of processes. Each process will be divided into several matrix blocks.
[0014] 1-3 Based on L1 and L2, combined with formula (1), each process calculates the distance matrix D1 between any two observation positions and the distance matrix D2 between all observation positions and the positions to be predicted;
[0015] Step (2): using the existing Gaussian random field model based on the Matern kernel function, each process obtains the correlation of observation values between any positions;
[0016] Step (3): Each process combines two sets of covariance matrices ∑1 and ∑2 to solve the physical value prediction vector Z1; specifically:
[0017] 3-1 According to the Kriging method, the calculation formula for obtaining the physical value prediction vector Z1 is:
[0018]
[0019] 3-2 Take Z1*∑2 -1 =X, then establish a symmetric positive definite linear system ∑1*X=Z2 for the above formula (4) to solve X; specifically:
[0020] 1) Distributed Low-Rank Cholesky Decomposition
[0021] 2) Forward and backward propagation based on Cholesky decomposition results
[0022] Step (4): The main process uses the complete solution vector to calculate the physical value prediction vector Z1=∑2*X according to standard matrix multiplication, which is the offshore climate prediction value.
[0023] Another object of the present invention is to provide a near-shore climate prediction device, comprising:
[0024] A data acquisition module is used to acquire the coordinate data matrix L1 of the position to be predicted, the coordinate data matrix L2 of the observed position, and the corresponding physical value observation vector Z2;
[0025] A parallel data determination module, used for determining the number of threads of OpenMP and the number of processes of MPI according to the improved two-dimensional data layout;
[0026] The first calculation module is used for each MPI process to calculate the distance matrix D1 between any two observation positions and the distance matrix D2 between all observation positions and the positions to be predicted;
[0027] In the second calculation module, each process obtains the correlation of observation values between any positions using the existing Gaussian random field model based on the Matern kernel function;
[0028] The third computing module includes a symmetric positive definite linear system building module, a distributed low-rank Cholesky decomposition module, and a forward and reverse propagation module; the symmetric positive definite linear system building module is used to build a symmetric positive definite linear system equation based on a physical value prediction vector; the distributed low-rank Cholesky decomposition module is used to use OpenMP multi-threading to perform decomposition tasks, compression tasks, triangle solution tasks, tail update tasks, and communication tasks to obtain Cholesky decomposition results; the forward and reverse propagation module is used to use OpenMP multi-threading to perform triangle solution tasks, vector update tasks, and communication tasks to achieve the solution of the symmetric positive definite linear system;
[0029] The fourth calculation module is used to obtain the physical value prediction vector by using the solution of the symmetric positive definite linear system according to the standard matrix multiplication.
[0030] The present invention has the following beneficial effects:
[0031] 1. The present invention proposes a new MPI+X paradigm (i.e., MPI+OpenMP task model), which can be run in a distributed environment and can also be programmed using the native OpenMP task model. Multi-threaded execution communication can effectively improve the operating efficiency of large-scale decomposition algorithms.
[0032] 2. The present invention proposes a new two-dimensional data layout in a distributed environment to balance the load on each process in the distributed decomposition algorithm; this layout can distribute tasks with high computational complexity to different processes, thereby reducing the communication overhead of the decomposition algorithm and improving efficiency.
[0033] 3. The present invention applies the new paradigm to the distributed low-rank Cholesky method in the offshore climate prediction method, and further proposes an efficient decomposition and compression strategy to overlap calculation and communication; this strategy has very high operating efficiency and has advantages in memory usage.
[0034] 4. The present invention applies a new paradigm to the solution process based on forward and backward propagation in the offshore climate prediction method, so that the process can also use the OpenMP task model in a distributed environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 The figure is a flow chart of the method of the present invention.
[0036] Figure 2 (a) is the improved distributed two-dimensional data layout in the present invention, Figure 2 (b) in the figure is the traditional distributed two-dimensional data layout.
[0037] Figure 3 The following is a diagram of the working principle when threads switch between communication tasks.
[0038] Figure 4 Graph of the low-rank update process in the update task.
[0039] Figure 5 It is a task execution area diagram in the decomposed task. DETAILED DESCRIPTION
[0040] The specific implementation scheme of the present invention is further described in detail below in conjunction with the accompanying drawings.
[0041] A method for forecasting offshore climate based on fast matrix decomposition, such as Figure 1 Specifically:
[0042] Step (1): Each process divides the data and calculates the distance between any positions; specifically:
[0043] 1-1 Obtain the coordinate data matrix L1 of the position to be predicted, the coordinate data matrix L2 of the observed position and the corresponding physical value observation vector Z2;
[0044] The physical value observation vector Z2 includes temperature, precipitation, wind speed, air pressure, humidity, etc.;
[0045] The coordinate data matrix includes the longitude and latitude values of each coordinate point;
[0046] 1-2 configure multiple MPI processes based on MPI communication. Each process combines the improved two-dimensional data layout and divides the data distribution in step 1-1 according to the number of processes. Each process will be divided into several matrix blocks.
[0047] The improved two-dimensional data layout is specifically that there are multiple single-cycle areas in a two-dimensional cycle layout, wherein the side length of each single-cycle area is vp, and each column in the single-cycle area contains vp matrix blocks with different process numbers; the process numbers on the diagonal of the entire two-dimensional cycle layout are replaced with the serial numbers on the corresponding columns of the single-cycle area where they are located; if the single-cycle area where the process number on the diagonal is located is in the i-th row, the replaced serial number is the i-th row process number of the column where it is located;
[0048] like Figure 2 As shown in (a)-(b) in the figure, the difference between the two is the order of processes on the diagonal. Taking the layout of 9 processes as an example, the traditional layout is only roughly divided, and only processes 0, 4, and 8 hold the data blocks on the diagonal. The improved layout further divides the data blocks on the diagonal, such as Figure 2 This division enables all processes to have diagonal data blocks, sharing the tasks of diagonal matrix blocks with large computational complexity. In addition, this change also retains the balance of data division in the traditional layout to the greatest extent.
[0049] 1-3 Based on L1 and L2, combined with formula (1), each process calculates the distance matrix D1 between any two observation positions and the distance matrix D2 between all observation positions and the positions to be predicted;
[0050] The distance matrix is calculated as follows:
[0051]
[0052] The E function is the standard Euclidean distance calculation function, (x1, y1) and (x2, y2) are the longitude and latitude coordinates of two arbitrary positions;
[0053] Step (2): Using the existing Gaussian random field model based on the Matern kernel function, each process obtains the correlation of observation values between any positions; specifically:
[0054] The Matern kernel function in the Gaussian random field model based on the Matern kernel function is as follows:
[0055]
[0056] Where θ1 represents the variance, θ2 represents the spatial range, θ3 represents the smoothness, γ represents the standard gamma function, and K represents the Bezier curve function modified according to the smoothness θ3;
[0057] The covariance matrix calculation method based on the Matern kernel function is as follows:
[0058]
[0059] The diagonal position elements in the Σ matrix are θ1, and the off-diagonal position elements are the calculation results of the Matern kernel function between any positions;
[0060] For the distance matrices D1 and D2 obtained in step (1), two sets of covariance matrices ∑1 and ∑2 are obtained respectively according to formulas (2)-(3);
[0061] Step (3): Each process combines two sets of covariance matrices ∑1 and ∑2 to solve the physical value prediction vector Z1; specifically:
[0062] 3-1 According to the Kriging method, the calculation formula for obtaining the physical value prediction vector Z1 is:
[0063]
[0064] 3-2 Take Z1*∑2 -1 =X, then establish a symmetric positive definite linear system ∑1*X=Z2 for the above formula (4) to solve X; specifically:
[0065] 3) Distributed low-rank Cholesky decomposition; The pseudo code of the main steps of the low-rank Cholesky algorithm in the present invention is as follows:
[0066]
[0067]
[0068] S11: Initialize MPI and specify the MPI startup parameter as MPI_THREAD_MULTIPLE, so that OpenMP multithreading is enabled in each process for independent non-blocking communication; the multithreading includes a master thread and multiple slave threads; the master thread is responsible for creating all tasks and generating a task dependency graph; while the master thread is working on tasks, the remaining idle slave threads will automatically obtain executable tasks from the OpenMP task queue according to the task dependency graph for binding and execution; the OpenMP task queue includes decomposition tasks, compression tasks, triangle solution tasks, tail update tasks, and communication tasks;
[0069] The graph nodes in the task dependency graph are the tasks in the OpenMP task queue, and the edges between two graph nodes are the front-back dependency relationship between the two tasks;
[0070] S12: A process selects a thread to perform the decomposition task, and specifically uses the standard Cholesky decomposition on the diagonal block; Figure 5 It is a task execution area diagram in the decomposed task.
[0071] S13: After the current slave thread completes the decomposition task, the threads of all processes synchronize the communication task, and then release the dependency of the communication task; then the process after executing the decomposition task selects any slave thread to execute the communication task, and sends the Cholesky decomposition result to the corresponding process; the synchronization task is completed by a separate dependent variable, and the dependency of the variable is output in this task, and the variable is received as a task dependency in all other communication tasks. This synchronization method can ensure that the communication tasks in the first communication area and the second communication area in the above pseudo code will not overlap between areas; in the absence of global dependency, the synchronization barrier can avoid deadlock caused by incorrect global dependency;
[0072] Figure 3 The following is a working diagram of the thread switching in the communication task. In the communication task, an explicit thread scheduling point is set for the non-blocking communication task; when the thread encounters a scheduling point after executing a send or receive operation, it will pause the execution of the current task and switch to other executable tasks in the OpenMP task queue; when the current thread is schedulable again, it will return to the scheduling point to continue executing the original task; before scheduling, the communication status is tested through MPI, and the thread will poll the communication status and determine whether it needs to be scheduled or continue to complete the task; the design method of this communication task solves the efficiency problem caused by thread blocking in the MPI+OpenMP task model; the working principle of thread switching is as follows Figure 3 As shown. Figure 3 For example, at time t1, the thread enters a communication task and starts executing. At time t2, since the result of MPI_Test is false, the communication task is not completed, so it switches to other tasks (which can be computing tasks or other communication tasks). At time t3, non-blocking communication is completed, and the communication task can continue to execute, but the same switching point may not exist in other tasks, so it is necessary to wait for the thread to complete that task at time t4 and enter the scheduling state again. At time t4, when the thread schedules and executes the original communication task again, it finds that it can jump out of the while loop, so it continues to execute to the end of the task, and completely ends the task at time t5.
[0073] The pseudo code of the sending type communication task in the present invention is as follows (the receiving type communication task is similar to the pseudo code):
[0074]
[0075]
[0076] S14: instead of using the pre-compression processing in the existing method, after each cycle of step S12, each process selects a slave thread to perform compression tasks and triangular solution tasks in sequence; the compression task specifically uses the adaptive cross approximation algorithm to perform low-rank compression on the matrix block to obtain two compression result matrices X and Y; the triangular solution task specifically performs a low-rank version of the triangular solution on the compressed matrix block, and the solution result process is as follows:
[0077] AR=X*(Y*AC -1 )=X*Y' formula (5)
[0078] AR is the result after each triangular solution, which can be obtained by the compressed results X and Y, and the decomposition result inverse matrix AC received in the communication. -1 Obtained; The low-rank triangle solution algorithm used in this step will not completely calculate the final result, but continue to retain the low-rank result representation, such as Y' combined with the last two terms of formula (5); Although this method will increase the task execution time of the second communication area (compared with full-rank communication, two low-rank matrices need to be transmitted in full-rank size), it is a necessary prerequisite for step 5 to perform more efficient low-rank updates;
[0079] S15: After the current slave thread completes the triangle solution task, the threads of all processes synchronize the communication task and then release the dependency of the communication task; finally, the process that has executed the low-rank triangle solution sends the two low-rank matrices and the corresponding matrix ranks to the corresponding process; the implementation method of this communication task is the same as step S13;
[0080] S16: Each process selects a slave thread to perform a tail update task on all tail matrix blocks. Figure 4 Graph of the low-rank update process in the update task.
[0081] The specific solution process of the tail update task is as follows:
[0082] S16-1: Construct tail update formula:
[0083] BR=B-X1*((Y'1*X2)*Y'2) Formula (6)
[0084] Wherein, BR is the result after each tail update, which can be obtained by the matrix block B where the process is located, and in step S15, the low-rank result representations (X1, Y'1) and (X2, Y'2) of the two matrices are obtained through communication;
[0085] S16-2: First, calculate the intermediate result W1=Y'1*X2 of formula (6), and then solve the next intermediate result W2=W1*Y'2 based on W1; since the low-rank matrix has a smaller rank, these two steps only require a lower amount of calculation; finally, calculate the complete full-rank matrix W3=X1*W2, and use this full-rank matrix for the final full-rank matrix update BR=B-W3; Figure 4 The process of low-rank update is shown, where the smaller edge lengths of the intermediate calculation results reveal why this low-rank update is more efficient.
[0086] S17: After the update, the current loop ends and returns to step S12 for the next loop; the next loop will only repeat the operation on the tail matrix update area in the previous loop; until entering the last loop and decomposing the last matrix block; in each loop, the division of the task execution area is as follows Figure 5 As shown in Figure 1. Since Cholesky decomposition is generally performed on symmetric matrices, only the lower triangular part of the complete matrix needs to be stored. When an upper triangular matrix block is needed, it can be obtained from the transposed result of the corresponding lower triangular matrix block.
[0087] 4) Forward and reverse propagation based on Cholesky decomposition results; The pseudo code of the forward and reverse propagation solution steps based on the Cholesky decomposition results in the present invention is as follows:
[0088]
[0089] S21: Obtain the above distributed low-rank Cholesky decomposition result; the MPI initialization method and the OpenMP working method are the same as the above decomposition process; wherein the OpenMP task queue includes triangle solution tasks, vector update tasks, and communication tasks;
[0090] S22: A process selects a slave thread to perform a triangulation task on a diagonal block; after the current slave thread completes the triangulation task, the threads of all processes synchronize the communication task and then release the dependency of the communication task; finally, the process that has performed the triangulation solution sends the solution result to the corresponding process; the implementation method of the communication task is the same as step S13;
[0091] S23: Each process selects a thread to perform a vector update task on the matrix block; the update can be completed using standard full-rank matrix multiplication;
[0092] S24: After the update, end this loop and return to step S22 to perform the next loop; the next loop will only repeat the operation on the tail matrix area in the previous loop; until entering the last loop and solving the last part of the solution vector; after all are completed, the main process starts to collect the solution vectors on other processes and combine them into a complete solution vector; these data are distributed in the improved two-dimensional data layout, the process where the diagonal data block is located;
[0093] Step (4): The main process uses the complete solution vector to calculate the physical value prediction vector Z1=Σ2*X according to standard matrix multiplication, which is the offshore climate prediction value; the prediction value vector can be merged with the observation value vector into a new complete data vector and applied to other studies.
Claims
1. A method for forecasting offshore climate based on fast matrix decomposition, characterized in that The following steps are involved: Step (1): Each process divides the data and calculates the distance between any positions; specifically: 1-1 Obtain the coordinate data matrix L1 of the position to be predicted, the coordinate data matrix L2 of the observed position and the corresponding observed physical value vector Z2; 1-2 configure multiple MPI processes based on MPI communication. Each process combines the improved two-dimensional data layout and divides the data distribution in step 1-1 according to the number of processes. Each process will be divided into several matrix blocks. 1-3 Based on L1 and L2, combined with formula (1), each process calculates the distance matrix D1 between any two observation positions and the distance matrix D2 between all observation positions and the positions to be predicted; The distance matrix is calculated as follows: The E function is the standard Euclidean distance calculation function, (x1, y1) and (x2, y2) are the longitude and latitude coordinates of two arbitrary positions; Step (2): Using the Gaussian random field model based on the Matern kernel function, each process obtains the correlation of observations between any positions; specifically: The Matern kernel function in the Gaussian random field model based on the Matern kernel function is as follows: Where θ1 represents the variance, θ2 represents the spatial range, θ3 represents the smoothness, γ represents the standard gamma function, and K represents the Bezier curve function modified according to the smoothness θ3; The covariance matrix calculation method based on the Matern kernel function is as follows: The diagonal position elements in the Σ matrix are θ1, and the off-diagonal position elements are the calculation results of the Matern kernel function between any positions; For the distance matrices D1 and D2 obtained in step (1), two sets of covariance matrices ∑1 and ∑2 are obtained respectively according to formulas (2)-(3); Step (3): Each process combines two sets of covariance matrices ∑1 and ∑2 to solve the predicted physical value vector Z1; specifically: 3-1 According to the Kriging method, the calculation formula for the predicted physical value vector Z1 is obtained: 3-2 Take Z1*∑2 -1 =X, then establish a symmetric positive definite linear system Σ1*X=Z2 for the above formula (4) and solve for X; specifically: 1) Distributed Low-Rank Cholesky Decomposition S11: Initialize MPI and specify the MPI startup parameter as MPI_THREAD_MULTIPLE, so that OpenMP multithreading is enabled in each process for independent non-blocking communication; the multithreading includes a master thread and multiple slave threads; the master thread is responsible for creating all tasks and generating a task dependency graph; while the master thread is working on tasks, the remaining idle slave threads will automatically obtain executable tasks from the OpenMP task queue according to the task dependency graph for binding and execution; the OpenMP task queue includes decomposition tasks, compression tasks, triangle solution tasks, tail update tasks, and communication tasks; The graph nodes in the task dependency graph are the tasks in the OpenMP task queue, and the edges between two graph nodes are the front-back dependency relationship between the two tasks; S12: A process selects a thread to perform the decomposition task, and specifically uses the standard Cholesky decomposition on the diagonal block; S13: After the current slave thread completes the decomposition task, the threads of all processes synchronize the communication task and then release the dependency of the communication task; then the process that executes the decomposition task selects any slave thread to execute the communication task and sends the Cholesky decomposition result to the corresponding process; In the communication task, an explicit thread scheduling point is set for the non-blocking communication task; when the thread encounters a scheduling point after executing a send or receive operation, the execution of the current task will be suspended and switched to other executable tasks in the OpenMP task queue; when the current thread is schedulable again, it will return to the scheduling point and continue to execute the original task; before scheduling, the communication status is tested through MPI, and the thread will poll the communication status and determine whether it needs to be scheduled or continue to complete the task; S14: After each cycle of step S12, each process selects a slave thread to perform compression tasks and triangle solving tasks in sequence; S15: After the current slave thread completes the triangle solution task, the threads of all processes synchronize the communication tasks and release the dependency of the communication tasks; then the process that has executed the low-rank triangle solution sends the two low-rank matrices and the corresponding matrix ranks to the corresponding process; S16: Each process selects a slave thread to perform a tail update task on all tail matrix blocks; S17: Return to step S12 until entering the last loop and completing the decomposition task for the last matrix block; 2) Forward and backward propagation based on Cholesky decomposition results S21: Obtain the distributed low-rank Cholesky decomposition result of the above step 1); initialize MPI, and specify the MPI startup parameter as MPI_THREAD_MULTIPLE, so that OpenMP multi-threading is enabled in each process for independent non-blocking communication; the above multi-threading includes a master thread and multiple slave threads; the master thread is responsible for creating all tasks and generating a task dependency graph; while the main thread is working on the task, the remaining idle slave threads will automatically obtain executable tasks from the OpenMP task queue according to the task dependency graph for binding and execution; the OpenMP task queue includes triangle solution tasks, vector update tasks, and communication tasks; S22: A process selects a slave thread to perform a triangulation task on a diagonal block; after the current slave thread completes the triangulation task, the threads of all processes synchronize the communication task and release the dependency of the communication task; then the process that has performed the triangulation solution sends the solution result to the corresponding process; the implementation method of the communication task is the same as step S13; S23: Each process selects a thread to perform a vector update task on the matrix block; S24: Return to step S22 until entering the last loop and solving the last diagonal block solution vector; 3) The main process collects the solution vectors from other processes and combines them into a complete solution vector; Step (4): The main process uses the complete solution vector to calculate the predicted physical value vector Z1 = Σ2*X according to standard matrix multiplication, which is the offshore climate prediction value.
2. A method for predicting offshore climate based on fast matrix decomposition as claimed in claim 1, characterized in that The observed physical value vector Z2 in step 1-1 includes air temperature, precipitation, wind speed, air pressure, and humidity.
3. A method for predicting offshore climate based on fast matrix decomposition as claimed in claim 1, characterized in that The coordinate data matrix in step 1-1 includes the longitude and latitude values of each coordinate point.
4. A method for predicting offshore climate based on fast matrix decomposition as claimed in claim 1, characterized in that The improved two-dimensional data layout described in step 1-2 is specifically that there are multiple single-loop regions in a two-dimensional loop layout, wherein the side length of each single-loop region is vp, and each column in the single-loop region contains matrix blocks with vp different process numbers; the process numbers on the diagonal of the entire two-dimensional loop layout are replaced in sequence with the serial numbers on the corresponding columns of the single-loop region where they are located; if the single-loop region where the process number on the diagonal is located is in the i-th row, then the replaced serial number is the i-th row process number of the column where it is located.
5. A method for predicting offshore climate based on fast matrix decomposition as claimed in claim 4, characterized in that The compression task in step S14 specifically uses the adaptive cross approximation algorithm to perform low-rank compression on the matrix block to obtain two compression result matrices X and Y; the triangular solution task specifically performs a low-rank version of the triangular solution on the compressed matrix block to obtain the solution result process as follows: AR=X*(Y*AC -1 )=X*Y' formula(5) AR is the result after triangle solution, AC -1 Represents the inverse matrix of the decomposition result received in the communication; for the convenience of calculation, Y*AC -1 Denoted as Y'.
6. A method for predicting offshore climate based on fast matrix decomposition as claimed in claim 5, characterized in that The specific solution process of the tail update task in step S16 is as follows: S16-1: Construct tail update formula: BR=B-X1*((Y'1*X2)*Y'2) Formula (6) Wherein, BR is the result after each tail update, which can be obtained by the matrix block B where the process is located, and in step S15, the low-rank result representations (X1, Y'1) and (X2, Y'2) of the two matrices are obtained through communication; S16-2: First calculate the intermediate result W1=Y'1*X2 of formula (6), and then solve the next intermediate result W2=W1*Y'2 based on W1; since the low-rank matrix has a smaller rank, these two steps only require a lower amount of calculation; finally calculate the complete full-rank matrix W3=X1*W2, and use this full-rank matrix for the final full-rank matrix update BR=B-W3.
7. A device for predicting offshore climate, characterized in that include: A data acquisition module is used to acquire the coordinate data matrix L1 of the position to be predicted, the coordinate data matrix L2 of the observed position, and the corresponding physical value observation vector Z2; A parallel data determination module, used for determining the number of threads of OpenMP and the number of processes of MPI according to the improved two-dimensional data layout; The first calculation module is used for each MPI process to calculate the distance matrix D1 between any two observation positions and the distance matrix D2 between all observation positions and the positions to be predicted; wherein the distance matrix calculation method is as follows: The E function is the standard Euclidean distance calculation function, (x1, y1) and (x2, y2) are the longitude and latitude coordinates of two arbitrary positions; The second calculation module uses the existing Gaussian random field model based on the Matern kernel function, and each process obtains the correlation of observation values between any positions; specifically: The Matern kernel function in the Gaussian random field model based on the Matern kernel function is as follows: Where θ1 represents the variance, θ2 represents the spatial range, θ3 represents the smoothness, γ represents the standard gamma function, and K represents the Bezier curve function modified according to the smoothness θ3; The covariance matrix calculation method based on the Matern kernel function is as follows: The diagonal position elements in the Σ matrix are θ1, and the off-diagonal position elements are the calculation results of the Matern kernel function between any positions; For the distance matrices D1 and D2 obtained, two sets of covariance matrices Σ1 and Σ2 are obtained respectively according to formulas (8)-(9); The third computing module includes a symmetric positive definite linear system building module, a distributed low-rank Cholesky decomposition module, and a forward and reverse propagation module; Symmetric positive definite linear system building module, used to build symmetric positive definite linear system equations based on physical value prediction vectors; Specifically: According to the Kriging method, the calculation formula for the predicted physical value vector Z1 is obtained: Take Z1*∑2 -1 =X, then establish a symmetric positive definite linear system ∑1*X=Z2 for the above formula (10); The distributed low-rank Cholesky decomposition module is used to use OpenMP multi-threading to perform decomposition tasks, compression tasks, triangle solving tasks, tail update tasks, and communication tasks to obtain Cholesky decomposition results; specifically: S11: Initialize MPI and specify the MPI startup parameter as MPI_THREAD_MULTIPLE, so that OpenMP multithreading is enabled in each process for independent non-blocking communication; the multithreading includes a master thread and multiple slave threads; the master thread is responsible for creating all tasks and generating a task dependency graph; while the master thread is working on tasks, the remaining idle slave threads will automatically obtain executable tasks from the OpenMP task queue according to the task dependency graph for binding and execution; the OpenMP task queue includes decomposition tasks, compression tasks, triangle solution tasks, tail update tasks, and communication tasks; The graph nodes in the task dependency graph are the tasks in the OpenMP task queue, and the edges between two graph nodes are the front-back dependency relationship between the two tasks; S12: A process selects a thread to perform the decomposition task, and specifically uses the standard Cholesky decomposition on the diagonal block; S13: After the current slave thread completes the decomposition task, the threads of all processes synchronize the communication task and then release the dependency of the communication task; then the process that executes the decomposition task selects any slave thread to execute the communication task and sends the Cholesky decomposition result to the corresponding process; In the communication task, an explicit thread scheduling point is set for the non-blocking communication task; when the thread encounters a scheduling point after executing a send or receive operation, the execution of the current task will be suspended and switched to other executable tasks in the OpenMP task queue; when the current thread is schedulable again, it will return to the scheduling point and continue to execute the original task; before scheduling, the communication status is tested through MPI, and the thread will poll the communication status and determine whether it needs to be scheduled or continue to complete the task; S14: After each cycle of step S12, each process selects a slave thread to perform compression tasks and triangle solving tasks in sequence; S15: After the current slave thread completes the triangle solution task, the threads of all processes synchronize the communication tasks and release the dependency of the communication tasks; then the process that has executed the low-rank triangle solution sends the two low-rank matrices and the corresponding matrix ranks to the corresponding process; S16: Each process selects a slave thread to perform a tail update task on all tail matrix blocks; S17: Return to step S12 until entering the last loop and completing the decomposition task for the last matrix block; The forward and reverse propagation modules are used to use OpenMP multithreading to perform triangle solving tasks, vector updating tasks, and communication tasks to solve positive linear systems. Specifically: S21: Obtain the distributed low-rank Cholesky decomposition result; initialize MPI and specify the MPI startup parameter as MPI_THREAD_MULTIPLE, so that OpenMP multi-threading is enabled in each process for independent non-blocking communication; the above multi-threading includes a master thread and multiple slave threads; the master thread is responsible for creating all tasks and generating a task dependency graph; while the main thread is working on the task, the remaining idle slave threads will automatically obtain executable tasks from the OpenMP task queue according to the task dependency graph for binding and execution; the OpenMP task queue includes triangle solution tasks, vector update tasks, and communication tasks; S22: A process selects a slave thread to perform a triangulation task on a diagonal block; after the current slave thread completes the triangulation task, the threads of all processes synchronize the communication task and release the dependency of the communication task; then the process that has performed the triangulation solution sends the solution result to the corresponding process; the implementation method of the communication task is the same as step S13; S23: Each process selects a thread to perform a vector update task on the matrix block; S24: Return to step S22 until entering the last loop and solving the last diagonal block solution vector; The fourth calculation module is used to obtain the physical value prediction vector by using the solution of the positive definite linear system according to the standard matrix multiplication.
