Iterative algorithm-oriented multi-precision guess execution method and system

Abstract

The invention discloses an iterative algorithm-oriented multi-precision guess execution method and system, and the method comprises the steps: giving a to-be-optimized iterative algorithm model, setting a plurality of threads and a task queue on a computing device, and enabling the plurality of threads to share the task queue; constructing an iterative task executed at low precision in the task queue, and extracting the task by a thread to execute an iterative algorithm for solving; according to a creation mode and a creation condition, expanding and creating new iteration branch tasks with different precisions in a tree structure at different time points of task execution, storing the iteration branch tasks in a task queue, and extracting and running the iteration branch tasks by an idle thread; and when the iteration branch task in charge of any thread reaches the iteration convergence condition, outputting an optimization solution result of the iteration algorithm model, and stopping running other threads. According to the method, the iterative algorithm model system data can be solved more accurately, and the operation cost and the operation time of the iterative algorithm are reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of data analysis and processing, and in particular relates to a multi-precision guessing execution method and system for iterative algorithms. Background Art

[0002] In scientific and engineering computing applications such as computational electromagnetics and computational fluid dynamics, many practical problems need to be solved by numerical simulation, and sparse linear systems, i.e. iterative algorithm models, such as the solution of Ax=b, is one of the key steps in the numerical simulation process. Among them, A is a sparse matrix with most elements being 0, and x and b are dense vectors. The solution methods of sparse linear systems are divided into direct solution and iterative solution. The direct solution method will lead to significant computational and storage overheads for solving sparse linear systems, which is not suitable for solving large-scale problems. The iterative solution methods based on Krylov subspace, such as the conjugate gradient method (CG) and the generalized minimum residual method (GMRES), continuously approach the exact solution in an iterative manner. Starting from an initial solution vector, the coefficient matrix and the right-hand side vector of the sparse linear equations are repeatedly used to calculate the new solution vector, and then the new solution vector is used as the initial value of the next iteration until the preset convergence condition is met. In each iteration, the current solution is corrected according to the approximate solution of the previous step, and the true solution is gradually approached. Existing work mainly focuses on how to use CPU or GPU to accelerate and optimize the main computing kernel SpMV of iterative algorithms, or use distributed platforms to solve large-scale sparse linear equations. With the continuous development of artificial intelligence technology and the emergence of low-precision floating-point formats, iterative algorithm acceleration optimization based on mixed precision has become one of the current research hotspots.

[0003] Existing mixed-precision iterative model optimization mainly uses numerical calculations of different floating-point precisions to reduce the computational cost and memory access overhead of computing devices, thereby reducing the running time of iterative algorithms. For example, low-precision floating-point representation is used to perform preconditioning calculations to obtain the initial solution, and high-precision floating-point representation is used to perform the main body of the iteration to obtain an approximate solution that meets the error requirements; or low-precision is used for inner iterations in iterative algorithms containing multiple layers of iterations. These technologies have improved the computational efficiency of iterative algorithms on computing devices to a certain extent. However, since scientific and engineering computing applications are more sensitive to computational precision, existing mixed-precision computing methods may cause the iterative solution process of some sparse linear equations to not converge, and whether it converges is difficult to predict in advance. In addition, due to the inevitable conversion overhead between different precisions and the increase in the number of iterations caused by low-precision calculations, the performance benefits of mixed-precision iterative algorithm optimization are not obvious, and sometimes even negative optimization may occur. Therefore, how to design a mixed-precision iterative method through the specific CPU or GPU of the computing device, which can fully utilize the advantages of low-precision computing, reduce the computing time of the computing device, and not affect the iterative convergence, that is, to ensure the accuracy and effectiveness of the model calculation results, is a very challenging task. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a multi-precision guessing execution method and system for iterative algorithms to solve the problems existing in the prior art.

[0005] To achieve the above object, the present invention provides a multi-precision guessing execution method for iterative algorithms, comprising:

[0006] An iterative algorithm model is obtained, and a plurality of threads and a task queue are set on a computing device, wherein the plurality of threads share the task queue;

[0007] Constructing a guessing iteration task in the task queue, and extracting the guessing iteration task through a thread to solve the iterative algorithm model; wherein the guessing iteration task is an initial construction and uses the lowest precision guessing task;

[0008] According to the creation mode and creation conditions, at different time points of the execution of the guessed execution task, iterative branch tasks with different precisions are expanded and created in a tree structure, and all iterative branch tasks are stored in a task queue, and each iterative branch task is extracted and run through an idle thread;

[0009] When the iterative branch task executed by any thread reaches the convergence condition, the optimization solution result of the iterative algorithm model is output.

[0010] Optionally, the computing device includes a CPU and a GPU device.

[0011] Optionally, a guaranteed task is initially constructed in the task queue, and the guaranteed task is extracted and run through a thread to iteratively optimize and solve the iterative algorithm model with the highest accuracy to obtain an optimized solution result of the guaranteed iterative algorithm model.

[0012] A guaranteed task is constructed in the task queue, wherein the guaranteed task is an iterative task using full high precision. The guaranteed task is extracted through a thread to perform iterative algorithm solution to ensure that when the extended iterative branch task does not converge, a high-precision iterative solution is obtained.

[0013] Optionally, the creation conditions include: when the number of iterations of the same precision reaches a set threshold and the residual change trend changes, the current iterative branch task does not use the highest floating point precision and the total number of the current iterative branch tasks does not reach an upper limit.

[0014] Optionally, the creation mode process includes: fixed step size guessing execution and dynamic step size guessing execution,

[0015] The fixed-step guessing execution includes: when the guessing execution task or the current iterative branch task meets the preset step length, according to the time node that meets the preset step length, separating and creating the next level iterative branch task with higher accuracy than the current iterative branch task;

[0016] Dynamic step size guessing execution includes, when guessing the execution task or the current iterative branch task, if the residual of the iterative solution process of the iterative algorithm model cannot maintain a stable downward trend during the running process, then separating and creating the next level iterative branch task with higher accuracy than the current iterative branch task at the time node where the stable downward trend cannot be maintained.

[0017] Optionally, the process of creating iterative branch tasks with different precisions includes:

[0018] During the execution of the guessing execution task, it is judged in turn whether the creation mode and the creation requirements are met. When the judgment result is satisfied, a first-level iterative branch task with higher precision is separated and created based on the time node that meets the judgment result, wherein the iterative branch task includes several levels of iterative branch tasks;

[0019] During the execution of the first-level iterative branch task, it is determined in turn whether the creation mode and creation requirements are met. When the judgment result is satisfied, a second-level iterative branch task with higher precision is separated and created based on the time node that meets the judgment result;

[0020] Based on the second-level iterative branch tasks, the process of creating modes and creation requirements and separating and creating the next-level iterative branch tasks is repeated until the total number of guessed execution tasks, running iterative branch tasks and iterative branch tasks in the task queue reaches the maximum number of iterative branch tasks, then separation and creation of new iterative branch tasks are stopped.

[0021] Optionally, when the total number of guessed execution tasks and iterative branch tasks reaches the maximum number of iterative branch tasks, iterative branch tasks that are running and have no obvious reduction in residual errors and have not reached the highest accuracy are switched to higher accuracy for running.

[0022] Optionally, the architecture of the computing device adopts a CPU+GPU heterogeneous computing architecture, and the guess execution tasks and iterative branch tasks are run through a GPU device or several threads of a CPU device, or the guess execution tasks and iterative branch tasks are run through several CPU devices combined with several threads of several GPU devices;

[0023] When a guessing execution task and an iterative branch task are run through several threads of a GPU device or a CPU device, when a new iterative branch task is created, the solution vector corresponding to the time node when the previous iterative branch task is created is backed up, and the backed-up solution vector is provided to the new iterative branch task. The new iterative branch task iteratively optimizes and solves the iterative algorithm model based on the backed-up solution vector during runtime;

[0024] When guessing execution tasks and iterative branch tasks are run through several CPU devices combined with several threads of several GPU devices, when a new iterative branch task is created, the solution vector corresponding to the time node when the previous level iterative branch task is created is backed up. If the new iterative branch task is running on the thread of another computing device, the current computing device will transmit the backed-up solution vector to the corresponding computing device through data communication, and the new iterative branch task will iteratively optimize and solve the iterative algorithm model based on the transmitted solution vector during operation.

[0025] In order to better achieve the above technical objectives, the present invention also provides a multi-precision guessing execution system for iterative algorithms, which is used to execute the above method.

[0026] Compared with the prior art, the present invention has the following advantages and technical effects:

[0027] The multi-precision guessing execution provided by the present invention breaks through the serial dependency bottleneck of traditional iterative algorithms. By maximizing the use of low-precision calculations, it is possible to fully utilize computing resources to achieve rapid problem solving, thereby reducing the running time of iterative algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0029] Figure 1 An example diagram of multi-precision guessing execution according to an embodiment of the present invention;

[0030] Figure 2 A schematic diagram of a dynamic step size guessing execution process according to an embodiment of the present invention;

[0031] Figure 3 A thumbnail diagram of the sparse matrix H2O according to an embodiment of the present invention;

[0032] Figure 4 This is a pseudo code example diagram of the GMRES iterative algorithm according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0035] Inspired by the serial program optimization technology based on speculative execution, the present invention proposes a speculative execution method for iterative algorithm models based on mixed precision computing. The present invention mainly involves the iterative solution of large-scale sparse linear equations.

[0036] The main idea is to use lower precision to execute at the beginning of the iteration, generate several branches with higher precision after several iterations, and the new iterative branch task continues to iterate with higher precision based on the intermediate solution obtained with low precision. The iterative branch task is a task for iterative calculation of a sparse linear equation system (which is described as an iterative algorithm model in the present invention), and its task execution process involves different iterative processes, the solution vector corresponding to the iteration, the residual and the time node of each iteration completion. After running for a period of time, the newly generated iterative branch task can also generate a higher precision iterative sub-branch task (the next level of iterative branch task). Since low precision has higher memory access efficiency and lower computing intensity, the speed of low-precision iterative tasks is faster than that of high-precision iterative branch tasks, but at the same time, due to the low computing accuracy, under given iterative convergence conditions, the value solved by the low-precision iterative branch task may converge, and residual turbulence may also occur in the middle of the iteration, resulting in the inability of the iteration to converge. Therefore, all low-precision iterative branch tasks and high-precision iterations generated based on low-precision iterations are guessed iterative branches, and performance benefits will be obtained when the intermediate solution or final solution obtained by the low-precision iteration is useful. At any time, all iterative branch tasks run concurrently on the same computing device or multiple computing devices until the solution obtained by one of the iterative branch tasks meets the residual requirement of convergence, or the number of iterations in the solution process of all iterative branch tasks reaches the upper limit. In the process of multiple branch evolution, when the value solved by a branch task converges, the execution of other branch tasks can be terminated in advance. Multi-precision guessing execution breaks through the serial dependency bottleneck of traditional iterative algorithms. By maximizing the use of low-precision calculations, it can make full use of computing resources to achieve rapid problem solving.

[0037] For a practical application problem, such as circuit simulation, electromagnetic simulation, etc., the iterative algorithm model to be solved, that is, the sparse linear system, is encoded and recorded, and the iterative algorithm model to be analyzed is programmed as a code file or a processing program file that can be processed and analyzed by a computer. At the same time, the sparse matrix and dense vector related to the iterative algorithm model are stored as a text file or a specific sparse matrix compression format, such as CSR (Compressed Sparse Row), CSC (Compressed Sparse Column) or COO (Coordinate). The multi-precision guessing execution method provided by the present invention is used to perform relevant analysis and processing on the above external technical data to reduce the computational overhead of the computer in solving the iterative algorithm model, thereby reducing the solution time and improving the solution efficiency.

[0038] Figure 1 An example based on multi-precision guessing execution is shown. First, the precision in the multi-precision guessing execution method refers to floating point formats with different precisions, including BF16, FP16, FP32, and FP64.l Indicates FP16 or BF16, with P m and P h Represents FP32 and FP64 respectively. The lowest precision P is used at the beginning of the algorithm iteration. l Start speculative execution, denoted as GMRES(P l ), due to the high memory access efficiency and low computational intensity when calculating with low precision, multiple iterations can be completed at a faster speed. However, due to the low precision of floating point data representation, it is not guaranteed whether the iterations will converge if they are always carried out with low precision. Therefore, after completing several iterations with low precision and obtaining the guessed solution, a high precision P is created. m The iterative branch task is executed based on the guess solution generated by the previous low-precision iteration. This iterative branch task (first-level iterative branch task) is recorded as GMRES (P l P m ). Since the intermediate solution formed by low-precision iteration is not necessarily a valid solution, the newly created GMRES (P l P m ) and the original low-precision iterative branch task GMRES(P l ) are all guessing execution branch tasks. If the solution obtained based on low-precision iteration is effective, the characteristics of low-precision and efficient solution can be fully utilized to improve the problem solving efficiency. Furthermore, although the iterative branch task GMRES (P l P m ) uses higher precision than GMRES(P l ) is more likely to converge, but this does not mean that GMRES(P l P m ) will definitely converge, so use precision P m After several iterations, a higher precision P is created. h Iterative branch task (second level iterative branch task) GMRES (P l P m P h ). By analogy, selecting different precision iterations at different iteration steps will form multiple iterative branch tasks that are executed with guesses, forming a concurrent iteration situation. When any iterative branch task reaches the convergence condition, the execution of the remaining branch tasks can be terminated immediately. If the converged branch task is not a high-precision branch task, a higher performance acceleration ratio will be obtained.

[0039] Taking the typical CPU+GPU heterogeneous computing architecture as an example, multiple branch tasks of mixed-precision computing can run on a single GPU or on multiple GPUs. When there is only one GPU in the system, multiple iterative branch tasks reuse the same computing device in time-sharing. In this case, certain performance gains can also be achieved, because a multi-core CPU can run multiple threads, each thread is responsible for an iterative branch task. At the same time, multiple iterative branch tasks can improve the utilization of GPU devices and give full play to the computing power advantages of GPU hardware. The mixed-precision iterations of multiple iterative branch tasks use the same GPU for calculation, which can fully tap the performance advantages of the GPU and improve the utilization of the GPU.

[0040] It is undeniable that running multiple mixed-precision guessing iterative branch tasks on the same GPU will still have an impact on performance. Different iterative branch tasks will inevitably interfere with each other in memory access and calculation during operation. Therefore, multiple GPU devices can further improve the efficiency of guessing execution. When there are multiple GPU devices, a single guessing iterative branch task can be run on the same GPU device, or multiple guessing iterative branch tasks can be run at the same time to give full play to the computing efficiency of multiple GPUs. The International Supercomputer Ranking Top 500 shows that a single node in the current mainstream supercomputers is configured with multiple GPU devices, so it is feasible to use these devices to implement guessing execution on multiple GPUs.

[0041] When speculative execution starts, multiple threads are first created on the CPU. The number of threads corresponds to the maximum number of iterative branches of speculative execution. Multiple threads share an iterative branch task queue. When each thread is idle, it will obtain an iterative branch task from the iterative branch task queue for execution. Initially, there is only one speculative execution task (basic iterative branch task) and one guarantee task in the task queue, namely GMRES (P l ) and GMRES(P h GMRES(P h ) is the iteration branch that runs with the highest accuracy from the beginning. When all guessed iteration branch tasks fail, GMRES(P h ) will give the final solution result, so the execution of this branch should be started from the beginning. When a thread gets the guess iterative branch task GMRES(P l) is executed, new iteration branch tasks of different precisions will be generated at different time points and put into the iteration branch task queue. The idle thread will obtain these tasks and run them immediately. Since new iteration branch tasks will also generate new iteration branch tasks during the running process, the number of tasks in the queue will change dynamically. Considering that too many iteration branch tasks will affect the overall computing efficiency, it is necessary to set a maximum upper limit for the number of guessed iteration branches. When the total number of iterative branches being run plus the iterative branches in the queue reaches the upper limit, the creation of new iteration branches will be stopped. According to general experience, the total number of iterative branches should not exceed 1.5 to 2 times the total number of GPU devices.

[0042] Creating a new guess branch requires the following creation conditions:

[0043] 1. The number of iterations of the same precision reaches a certain threshold, and the residual changes slowly or becomes larger. Among them, whether the residual changes slowly can be obtained by recording the residuals of the most recent iterations. Specifically, assuming that the current iteration is the i-th iteration, given r i-t ,r i-(t-2) ,r i-(t-3) ,…,r i-1 is the residual of the most recent t iterations. The following indicators can be used to determine whether the residual change is slowing down:

[0044] (1) The number of times the residual decreases in the most recent t iterations: desc:

[0045]

[0046] Among them, r k represents the residual value after the kth iteration, t represents the number of iterations, and f(x) represents whether the output function is decreased.

[0047] (2) The residual decrease rate descR of the most recent t iterations

[0048]

[0049] (3) The relative standard deviation (rsd) of the residuals of the most recent t iterations. In the following formula, avg is the average residual of the most recent t iterations:

[0050]

[0051] For example, when t is 100, desc=50 means that the residuals of 50 iterations in the last 100 iterations are increasing or remaining unchanged. In this case, a new iteration branch task with higher precision can be created to improve the residuals. In actual use, different thresholds can be customized for the above three indicators.

[0052] 2. The current iteration branch task iteration process does not use the highest floating point precision, and higher precision is available;

[0053] 3. The total number of iterative branch tasks in the current system has not reached the upper limit;

[0054] When all the above conditions are met, the current iteration branch task can generate a new iteration branch and put it into the iteration task queue. It should be noted that when generating a new iteration branch, the current solution vector needs to be backed up for the new iteration branch task to use. The backup can be located on the GPU device running the current iteration branch task. If the new iteration branch task is also run on the same device later, there is no need to transfer data between devices; if the new iteration branch task is run on different devices, data communication needs to be carried out through the connection channel between devices (such as NVLink connection between GPUs). After the new iteration branch task obtains the intermediate solution vector, it can continue to execute with higher precision. Assuming that there are multiple precisions to choose from above the current iteration precision, the new iteration branch task selects one of the precisions to start execution. When the total number of iteration branch tasks in the system approaches the upper limit of the total number of tasks, the new iteration branch task should choose a higher precision to run to avoid generating too many iteration branch tasks. When only the first two of the above three conditions are met, that is, the total number of iteration branch tasks in the current system has reached the upper limit, and the residual of the current iteration branch task has no obvious downward trend, the current iteration branch can be switched to a higher floating point precision to continue running to avoid wasting computing resources.

[0055] Similar to the speculative execution technology for parallelizing serial programs, in the speculative iteration based on mixed precision, the time point selection for generating iterative branch tasks is very critical. Generating new iterative branch tasks too early will result in lower performance benefits, and generating new iterative branch tasks too late may lead to more iterations or even non-convergence of iterations. The present invention proposes two modes of creating iterative branches: fixed-step guessing execution and dynamic-step guessing execution.

[0056] It should be noted that in the creation process of fixed-step guess execution, only two creation conditions are selected for creation based on whether the number of iterations of the same precision reaches a certain threshold and whether there is a higher precision, and there is no need to judge the residual. In the creation process of dynamic step guess execution, all the creation conditions based on which a new guess branch is created need to be met before it can be created. Specifically:

[0057] In the fixed-step guessing execution, assuming that the preset maximum number of iterations is I, a maximum of k precision guesses are made, and when the guessing step interval is S = I / k, that is, each iteration branch automatically generates a higher precision iteration branch after running S iteration steps, and can generate (k×(k+1)) / 2+k+1 iteration branches in total. Taking k=3 as an example, no matter how the total number of iteration steps is, there are a total of 10 combinations of different precisions, as shown in Table 1 below.

[0058] Table 1

[0059]

[0060] In the dynamic step size guessing execution, new iterative branch tasks are no longer generated based on the fixed step size. Instead, the change in the residual of the past iteration is monitored to determine whether to generate a new iterative branch task. If the residual maintains a stable downward trend, the iteration should continue using the current precision, otherwise a new iterative branch task with higher precision should be generated. For example, if the precision of the current iterative branch task is FP16, there are two options, FP32 and FP64, which means that only 2 new iterative branch tasks will be generated during the entire iteration process; if the precision of the current iterative branch task is FP32, there is only one option, FP64, which means that only one new iterative branch task will be generated during the entire run. In addition to the FP64 branch task that runs with the highest precision from the beginning, there will be up to 5 guessed iterative branch tasks at the same time during the dynamic step size guessing execution process, such as Figure 2 shown.

[0061] When any of the multiple iterative branch tasks reaches the convergence condition, the running of other iterative branch tasks can be terminated. The simplest way is to set a global flag on the CPU, and all threads will check this flag at each iteration step to determine whether to exit execution. At the same time, the branch task that ends early also needs to clear the iterative task queue as soon as possible to avoid new iterative branch tasks from starting to run. The iterative branch that reaches the convergence condition will copy the corresponding solution vector back to the CPU as the final solution.

[0062] In order to save computing resources to the maximum extent, you can also monitor the residual changes of all branches in the system synchronously during operation, terminate other undesirable guess branches in time, or cancel tasks in the task queue to reduce the computing power consumption caused by the simultaneous execution of multiple guess branches. Specifically, there are two situations:

[0063] 1. If the current iteration branch reaches the convergence condition, other iteration branches can be canceled unconditionally;

[0064] 2. The current iteration branch is not the iteration branch corresponding to the highest precision. It is currently running at the highest precision and has reached the upper limit of the number of iterations. You can stop running.

[0065] Take the sparse matrix H2O from the famous sparse matrix dataset SuiteSparse Matrix Collection as an example. This matrix comes from the field of quantum chemistry, with a row and column size of about 67K and a non-zero number of about 2M. Figure 3 The thumbnail of the matrix is ​​shown in Figure 1. The non-white pixels in the figure represent the non-zero elements of the sparse matrix. Taking the GMRES iterative algorithm as an example, Figure 4 The pseudo code is shown. The GMRES iterative algorithm is used as a computer executable code file or a processing program file, the H2O matrix is ​​used as the sparse matrix A in the algorithm, the vector x0 is initialized to all 0s, and the value of the vector b is usually related to the actual problem. A set of values ​​can be given in experimental tests. Because the iterative algorithm is relatively sensitive to floating point precision, the feasibility of using low-precision calculations for the iterative algorithm as a whole is low. Therefore, the above multi-precision guessing execution is mainly used in the sparse matrix-vector multiplication (SpMV), which is the main bottleneck operator of the iterative algorithm ( Figure 4 ). According to the test, when the error threshold of iterative convergence is 1E-6, the iterative branch of FP16-FP32 achieves the fastest convergence, and its performance is 1.27 times that of iterative solution using the default high precision (FP64).

[0066] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A multi-precision guessing execution method for iterative algorithms, characterized in that: include: An iterative algorithm model is obtained, and a plurality of threads and a task queue are set on a computing device, wherein the plurality of threads share the task queue; Constructing a guessing iteration task in the task queue, and extracting the guessing iteration task through a thread to solve the iterative algorithm model; wherein the guessing iteration task is an initial construction and uses the lowest precision guessing task; According to the creation mode and creation conditions, at different time points of the execution of the guessed execution task, iterative branch tasks with different precisions are expanded and created in a tree structure, and all iterative branch tasks are stored in a task queue, and each iterative branch task is extracted and run through an idle thread; When the iterative branch task executed by any thread reaches the convergence condition, the optimization solution result of the iterative algorithm model is output.

2. The method according to claim 1, characterized in that in, The computing device includes a CPU and a GPU device.

3. The method according to claim 1, characterized in that A guaranteed task is constructed in the task queue, wherein the guaranteed task is an iterative task using full high precision. The guaranteed task is extracted through a thread to perform iterative algorithm solution to ensure that when the extended iterative branch task does not converge, a high-precision iterative solution is obtained.

4. The method according to claim 1, characterized in that: The creation conditions include: when the number of iterations with the same precision reaches a set threshold and the residual change trend changes, the current iterative branch task does not use the highest floating point precision and the total number of the current iterative branch tasks does not reach the upper limit.

5. The method according to claim 1, characterized in that The creation mode process includes: fixed step size guessing execution and dynamic step size guessing execution, The fixed-step guessing execution includes: when the guessing execution task or the current iterative branch task meets the preset step length, according to the time node that meets the preset step length, separating and creating the next level iterative branch task with higher accuracy than the current iterative branch task; Dynamic step size guessing execution includes, when guessing the execution task or the current iterative branch task, if the residual of the iterative solution process of the iterative algorithm model cannot maintain a stable downward trend during the running process, then separating and creating the next level iterative branch task with higher accuracy than the current iterative branch task at the time node where the stable downward trend cannot be maintained.

6. The method according to claim 1, characterized in that The process of creating iterative branch tasks of different precisions includes: During the execution of the guessing execution task, it is judged in turn whether the creation mode and the creation requirements are met. When the judgment result is satisfied, a first-level iterative branch task with higher precision is separated and created based on the time node that meets the judgment result, wherein the iterative branch task includes several levels of iterative branch tasks; During the execution of the first-level iterative branch task, it is determined in turn whether the creation mode and creation requirements are met. When the judgment result is satisfied, a second-level iterative branch task with higher precision is separated and created based on the time node that meets the judgment result; Based on the second-level iterative branch tasks, the process of creating mode and creation requirement judgment and separating and creating the next-level iterative branch tasks is repeated until the total number of guessed execution tasks, running iterative branch tasks and iterative branch tasks in the task queue reaches the maximum number of iterative branch tasks, then separation and creation of new iterative branch tasks are stopped.

7. The method according to claim 6, characterized in that When the total number of guessed execution tasks and iterative branch tasks reaches the maximum number of iterative branch tasks, the iterative branch tasks that are running and have no obvious reduction in residuals and have not reached the highest accuracy are switched to higher accuracy for operation.

8. The method according to claim 2, characterized in that: The architecture of the computing device adopts a CPU+GPU heterogeneous computing architecture, and runs the guess execution tasks and iterative branch tasks through a GPU device or several threads of a CPU device, or runs the guess execution tasks and iterative branch tasks through several CPU devices combined with several threads of several GPU devices; When a guessing execution task and an iterative branch task are run through several threads of a GPU device or a CPU device, when a new iterative branch task is created, the solution vector corresponding to the time node when the previous iterative branch task is created is backed up, and the backed-up solution vector is provided to the new iterative branch task. The new iterative branch task iteratively optimizes and solves the iterative algorithm model based on the backed-up solution vector during runtime; When guessing execution tasks and iterative branch tasks are run through several CPU devices combined with several threads of several GPU devices, when a new iterative branch task is created, the solution vector corresponding to the time node when the previous level iterative branch task is created is backed up. If the new iterative branch task is running on the thread of another computing device, the current computing device will transmit the backed-up solution vector to the corresponding computing device through data communication, and the new iterative branch task will iteratively optimize and solve the iterative algorithm model based on the transmitted solution vector during operation.

9. A multi-precision guessing execution system for iterative algorithms, characterized in that: Used to execute the method according to any one of claims 1 to 8.

Images