A CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition

Abstract

The present invention discloses a CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition, including the following: starting and initializing a CFD solver; using an asynchronous multi-stream method; the asynchronous multi-stream method is divided into: a method of decomposing according to the data correlation of calculation variables and a method of reconstructing the process according to the data correlation of calculation process. By designing an asynchronous multi-stream CFD performance optimization technology for multi-node multi-card heterogeneous parallel MPI-X based on the data dependency of the calculation process, and an asynchronous multi-stream CFD performance optimization technology for multi-node multi-card heterogeneous parallel MPI-X based on the decomposition of the data correlation of calculation variables, the asynchronous multi-stream CFD performance optimization technology for multi-node multi-card heterogeneous parallel MPI-X is achieved by adjusting the calculation call process without changing the underlying calculation core functions and data structures, so as to achieve the performance improvement of asynchronous multi-stream on large-scale CFD.

Description

Technical Field

[0001] The present invention relates to the field of high-performance CFD computing technology, and in particular to a CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition. Background Art

[0002] Heterogeneous parallel computing, especially high-performance parallel platforms based on central processing units (CPUs), are increasingly widely used in the field of computational fluid dynamics (CFD). As the scale of CFD problems continues to increase, the storage and computing capabilities of a single machine and a single card have reached their limits and are difficult to apply to large-scale computing tasks, which has prompted computing devices to rapidly develop towards multi-machine and multi-card platforms. In actual applications, the frequent communication overhead between the CPU and GPU and the virtual unit data exchange overhead between CPUs have led to the dilemma of poor scalability of heterogeneous parallel CFD computing models for single machines and single cards on multi-machine and multi-card platforms.

[0003] In response to the additional data transmission problems caused by multi-machine and multi-card platforms, researchers have tried to use a GPU-to-GPU direct communication strategy to avoid data communication between the CPU and GPU, but the effect is poor in actual CFD applications. Therefore, most current studies still use a CPU-based communication strategy, taking advantage of the GPU's ability to perform calculations and data transmission at the same time, and using asynchronous multi-stream technology to achieve overlap between calculations and data transmission, thereby improving the resource utilization of the device and enhancing the overall performance of the application. However, asynchronous multi-stream technology is mainly based on regional decomposition and focuses on relatively simple structured grid CFD solvers. For non-structured CFD solvers, due to the complex topological relationship, the asynchronous multi-stream technology based on regional decomposition has a large amount of changes to the data structure and mapping relationship, making it difficult to achieve effective heterogeneous multi-machine and multi-card performance optimization.

[0004] At present, the existing asynchronous multi-stream CFD implementation scheme mainly involves further decomposing the calculation area, and alternately initiating solution instructions through multiple calculation areas, so as to achieve the overlap of calculation and data transmission. Specifically, for the heterogeneous multi-stream of the multi-machine multi-card MPI-X model (MPI (Message Passing Interface) message passing interface), where X refers to the GPU programming model, it is first necessary to divide the overall calculation solution area so that the number of regional divisions matches the number of MPI processes, and establish a mapping relationship for the virtual unit data. Then, a finer level of grid division is performed on the divided area, and a mapping relationship is established for the virtual unit data at this level. This process requires corresponding adjustments to the underlying data structure and the calculation core function, which is relatively complex to implement, and the communication overhead of the final calculation area is difficult to effectively overlap with the calculation process. This implementation scheme is usually applied to the field of structured grids. For unstructured networks, since their own mapping relationships are already quite complex, further division and establishment of mapping relationships are difficult to implement on unstructured grids. At present, there is no specific implementation of heterogeneous parallel asynchronous multi-stream of multi-machine multi-card MPI-X for unstructured CFD. Summary of the invention

[0005] The purpose of the present invention is to provide a CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition to address the above-mentioned problems, hoping to improve the problems of complex existing asynchronous multi-stream CFD implementation schemes, limited calculation coverage, large differences in implementation between structured grids and unstructured grids, and the lack of multi-machine and multi-card MPI-X heterogeneous parallel asynchronous multi-stream technology in the field of unstructured CFD.

[0006] The technical solution adopted by the present invention is as follows:

[0007] A CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition, the method comprising the following: starting a CFD solver and initializing the CFD solver; using an asynchronous multi-stream method; wherein the asynchronous multi-stream method comprises: a method of decomposing according to the data correlation of calculation variables and a method of reconstructing a process according to the data correlation of calculation processes; the method of decomposing according to the data correlation of calculation variables is an asynchronous multi-stream optimization strategy for a node Gaussian gradient module in a CFD solver, and realizes concurrent execution of gradient calculation by analyzing the data dependency between a gradient calculation module for calculating five variables and a symmetry plane correction calculation module for a velocity component; the method of reconstructing a process according to the data correlation of calculation processes distributes multiple calculation flows according to the data correlation between each calculation module, so that multiple calculation flows can be processed concurrently, thereby completing the mutual covering of GPU calculation and CPU memory access and MPI communication.

[0008] The asynchronous multi-stream solution constructed based on data dependencies in this method mainly includes two parts: an asynchronous multi-stream method that decomposes according to the data correlation of calculation variables and an asynchronous multi-stream method that reconstructs the process according to the data correlation of calculation processes. Through this method, the overall performance of the CFD solver can be significantly improved.

[0009] Further: the method of decomposing by calculation variable data correlation includes the following: step S100: identifying six calculation modules in the node Gaussian gradient module, the modules including the gradient calculation modules of five variables and the symmetry plane correction calculation module of the velocity, performing data dependency analysis on the six calculation modules, wherein the gradient calculation of the five variables is concurrent; step S200: allocating five calculation streams for alternating asynchronous submission, for parallel execution of GPU calculation, CPU memory access and MPI communication.

[0010] It should be noted that the gradient calculations of the five variables are concurrent, so they can initiate calculations at the same time without affecting the calculation results.

[0011] Further: the five calculation streams are: density calculation stream Stream rho, velocity component u calculation stream Stream u, velocity component v calculation stream Stream v, velocity component w calculation stream Stream w, pressure calculation stream Stream p; the step S200 includes the following: step S201: use the default calculation stream Default Stream to enter the gradient calculation module; step S202: start the symmetry surface correction calculation of the three velocity components, submit the calculation stream Stream u, when Stream u is in the communication between GPU and CPU and between CPU, submit the Stream rho calculation stream, and start the density rho gradient calculation on the GPU; step S203: after submitting the Stream rho calculation stream, when Stream rho encounters the memory access and MPI communication module, submit the Stream p calculation stream, and start the pressure gradient calculation on the GPU; step S204: after submitting the Stream p calculation stream, when Stream p encounters the GPU memory access and MPI communication module, submit the Stream u calculation stream again, at this time, the GPU memory access and MPI communication have been completed on Stream u, and the symmetry surface correction calculation of the three velocity components is continued; step S205: after submitting the Stream u calculation stream, when Stream When u reaches the GPU memory access and CPU-to-CPU MPI communication module again, the Streamrho computing flow is started again to keep the GPU computing core busy at all times; similarly, the Stream v computing flow and the Stream w computing flow are also submitted alternately in sequence.

[0012] It should be noted that the five computing flows are submitted alternately and asynchronously, which can achieve mutual masking of GPU computing, GPU memory access, and MPI communication, thereby maximizing the use of GPU computing and memory access resources.

[0013] Further: The method for reconstructing the process according to the data correlation of the calculation process is initiated according to the calculation process, and is divided into a laminar flow solution module and a turbulent flow solution module, including the following steps: Step T100: Construct a laminar flow solution module, which is constructed according to the data correlation dependency of the laminar flow module calculation process; wherein, the laminar flow solver iteratively updates and solves the flow field variable q[5], including a right-hand side residual calculation module and a linear equation group iterative update module, and the linear equation group iteration will update the flow field variable q[5]. For multi-node MPI calculations, the virtual unit flow field variable q[5] needs to be updated before the laminar flow solver starts, and the remaining modules of the laminar flow solver, until the right-hand side residual calculation module is completed, the flow field variable q[5] remains unchanged; the gradient variables dqdx[5], dqdy[5], dqdz[5] are directly obtained from the flow field variable q[5], and the limiter variable Limit[ 5] is obtained by combining the flow field variable q[5] and the gradient variables dqdx[5], dqdy[5], and dqdz[5], wherein dqdx[5] is the component of the gradient variable in the x direction, dqdy[5] is the component of the gradient variable in the y direction, and dqdz[5] is the component of the gradient variable in the z direction. The calculation of flux Flux[5] depends on the flow field variable q[5], the gradient variables dqdx[5], dqdy[5], dqdz[5], and the limiter variable Limit[5]. Step T200: construct a turbulence solution module, wherein the turbulence solver is divided into a right-hand side residual calculation module and a linear equation group iterative update module, and the solution variable is only a single turbulence variable. Before the turbulence solution module starts, the turbulence variable needs to be updated by a virtual unit, and the remaining modules remain unchanged until the turbulence right-hand side residual calculation module ends.

[0014] Further: the step T100 includes: step T101: when the laminar flow solver initially updates the virtual unit flow field variable q[5], an additional initial module calculation flow is allocated, the initial module contains the calculation data that needs to be initialized in the subsequent calculation, and covers the CPU and GPU data transmission and the MPI communication between the CPU and the CPU when the virtual unit flow field variable q[5] is updated; secondly, based on the node Gaussian gradient asynchronous multi-flow masking node Node value transfer constructed by the method of decomposing the calculation variable data correlation, part of the limiter module is migrated to the virtual unit gradient value update module after the gradient calculation is completed, and the two calculation flows correspond to part of the limiter module and the virtual unit gradient value update module respectively. The two calculation flows can be executed concurrently to cover the CPU and GPU data transmission and the MPI communication between the CPU and the CPU when the virtual unit gradient value is updated; step T102: when the calculation process is carried out To the remaining calculation part of the limiter variable Limit[5]. There is no CPU-GPU data transmission and inter-MPI communication until the final limiter variable Limit[5] virtual unit communication. At this time, if the laminar flow calculation involves fluid viscosity, the temperature gradient calculation is performed in parallel with multiple flows here, and the calculation process of the limiter variable Limit[5] is used to cover the virtual unit node Node value, CPU-GPU data transmission and inter-MPI communication in the temperature gradient calculation; Step T103: After the calculation of the limiter variable Limit[5] is completed, the Limit[5] virtual unit communication begins. At this time, the time stepping module is used to cover the communication process on it. The calculation data of the time stepping module only depends on the flow field variable q[5]. The Limit[5] virtual unit communication flow and the time stepping module calculation flow can be executed concurrently; Step T104: Perform the flux Flux[5] calculation.

[0015] It should be noted that the limiter calculation mainly includes the maximum and minimum values ​​of the flow field variables and some limiter calculation processes that are not related to the gradient value. The two calculation flows can be executed concurrently to cover the CPU and GPU data transmission when the virtual unit gradient value is updated and the MPI communication between CPUs.

[0016] Further: the step T103 includes: firstly calculating the contribution of the fluid inviscidity to the flux Flux[5], and the calculation process does not involve communication; if the calculation involves the fluid viscosity, the virtual unit communication after the temperature gradient calculation is completed is placed here and executed concurrently with the calculation module of the contribution of the fluid inviscidity to the flux Flux[5], covering the temperature gradient virtual unit communication; if the calculation involves the fluid viscosity, then continue to calculate the contribution of the fluid viscosity to the flux Flux[5], at this time all the information related to the temperature has been obtained, and the calculation will no longer involve virtual unit communication.

[0017] Further: the asynchronous multi-flow construction method of the turbulence solution module of step T200 includes the following steps: step T201: the initial turbulence variable virtual unit of the turbulence module is updated, and the turbulent kinetic energy generation calculation flow is executed synchronously at this time. The calculation process only depends on the data of the laminar flow solution module and can be executed concurrently with the initial turbulence variable virtual unit update, thereby concealing the communication process therein; step T202: the turbulence gradient calculation module requires the virtual unit node Node value and the turbulence variable value communication and the virtual unit gradient value update after the gradient calculation is completed, and the turbulent viscosity flux term and the source term are migrated to the turbulence gradient calculation module. The two calculation flows are executed synchronously, thereby concealing the communication; step T203: after the gradient calculation is completed, it proceeds to the turbulent inviscid flux part. This calculation module has no virtual unit communication, and the single flow completes the calculation.

[0018] Further: the gradient calculation module of the five variables includes three velocity components u, v, w, pressure p and density rho, and the gradient calculation of the five variables is concurrent, the symmetry surface correction of the three velocity components needs to be completed in advance, and initiating the calculation at the same time does not affect the calculation result.

[0019] Furthermore: the turbulence gradient calculation in step T202 has no effect on the calculation of the turbulence viscosity flux term and the source term, and the turbulence viscosity flux term and the source term are migrated to the turbulence gradient calculation module, so that the two calculation flows are executed synchronously.

[0020] Further: the two computing flows are divided into computing flow 0 and computing flow 1.

[0021] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0022] The present invention adopts a CFD multi-GPU asynchronous multi-flow optimization method based on data dependency decomposition, a method of decomposing according to the data correlation of calculation variables and a method of reconstructing according to the data correlation of calculation processes, which can cover all CPU and GPU data transmission and MPI communication modules on laminar and turbulent solvers, thereby achieving the purpose of maximizing the utilization of equipment resources and improving the overall performance.

[0023] The present invention realizes CFD asynchronous multi-stream technology based on the MPI-X model in a multi-node and multi-card heterogeneous environment, which can effectively cover the CPU and GPU data transmission and the MPI communication overhead between CPUs in the CFD calculation process, thereby significantly improving the overall performance of CFD applications.

[0024] The present invention achieves the performance improvement of asynchronous multi-stream on large-scale CFD by only adjusting the calculation calling process without changing the underlying calculation core function kernel and data structure of the multi-node multi-card heterogeneous parallel MPI-X. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart of the method of the present invention;

[0026] Figure 2 It is a heterogeneous parallel multi-machine multi-card MPI-X calculation flow chart of the present invention;

[0027] Figure 3 It is a multivariable node Gaussian method gradient calculation flow chart of the present invention;

[0028] Figure 4 It is the multivariable node Gaussian method gradient calculation data correlation of the present invention;

[0029] Figure 5 It is a multivariable node Gaussian gradient asynchronous multi-flow call graph of the present invention;

[0030] Figure 6 It is a data correlation process reconstruction asynchronous multi-stream laminar flow solution calculation flow chart of the present invention;

[0031] Figure 7 It is a data correlation process reconstruction calculation flow chart of asynchronous multi-flow turbulence solution of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0033] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0034] Example 1

[0035] like Figure 1 As shown, this embodiment proposes a CFD multi-GPU asynchronous multi-flow optimization method based on data dependency decomposition. Starting from the data dependency, without changing the computing core function and the underlying data structure, the asynchronous multi-flow optimization of CFD heterogeneous parallel computing on a multi-machine multi-card platform is realized. This technology can be used for unstructured / structured grids, compressible / incompressible CFD, laminar / turbulent flow solutions. Compared with the complex regional decomposition asynchronous multi-flow implementation, the implementation process of this technology is simple. After applying this technology, the performance of heterogeneous parallel CFD on a multi-machine multi-card platform can be greatly improved.

[0036] The asynchronous multi-stream solution constructed based on data dependency in this method mainly includes two parts: asynchronous multi-stream decomposition according to the data dependency of computing variables and process reconstruction method according to the data dependency of computing process.

[0037] The method of decomposing by calculation variable data correlation is an asynchronous multi-flow optimization strategy for the node Gaussian gradient module in the CFD solver, which realizes the concurrent execution of gradient calculation by analyzing the data dependency between the gradient calculation module for calculating five variables and the calculation module for correcting the symmetry surface of a velocity component;

[0038] The method for reconstructing the process according to the data correlation of the calculation process allocates multiple calculation flows according to the data correlation between each calculation module, so that multiple calculation flows can be executed concurrently, thereby realizing the mutual concealment of GPU calculation and CPU memory access and MPI communication. The asynchronous multi-flow technology is applied to the laminar flow solution module and the turbulent flow solution module.

[0039] By improving the existing method through this method, the overall performance of the CFD solver can be effectively improved.

[0040] Example 2

[0041] like Figure 2 As shown, Figure 2 This is a flow chart of heterogeneous parallel multi-machine multi-card MPI-X calculation. The figure provides a CFD calculation and communication model involving two MPI processes. This model is suitable for structural / unstructured CFD solutions. Among them, q[5] represents the original variables of the flow field, including three velocity components (u, v, w), pressure p and density rho, a total of five original variables. The process completes the gradient calculation of the five original variables (u, v, w, p, rho) in sequence and updates the latest calculated values ​​of the virtual unit between processes. The main core calculation functions include node Gaussian gradient, limiter, inviscid / viscous flux, turbulence module, time step term, etc.

[0042] For the node Gaussian method, when calculating the sum of node value weights, it is also necessary to complete the update of the virtual unit node (Node) value between processes.

[0043] Multiple MPI communications can enhance the robustness of the code and improve the overall convergence. If the calculation flow of a single process is obtained and the MPI communication module is removed, a single-machine single-card CFD solution version can be obtained.

[0044] In the prior art, it is assumed that the stand-alone CFD version has completed the preliminary heterogeneous parallel transformation, and the addition of the MPI communication module can obtain a heterogeneous parallel multi-machine multi-card version. This embodiment improves the performance of CFD calculations for heterogeneous parallel multi-machine multi-card MPI-X (X represents a heterogeneous programming model, such as CUDA, HIP, SYCL, OpenCL, and OpenACC, etc.).

[0045] This embodiment can improve the performance of large-scale multi-node multi-card heterogeneous parallel MPI-X (X represents a heterogeneous programming model, including CUDA, HIP, SYCL, OpenCL, OpenACC, etc.) CFD calculations.

[0046] Example 3

[0047] like Figure 3 As shown in the figure, the traditional node Gaussian gradient calculation process is often executed sequentially, that is, the gradient values ​​of each variable are calculated in turn, and the gradient values ​​of the virtual unit are updated between processes. However, this sequential execution method limits the parallel utilization of computing resources, resulting in low computing efficiency. In order to overcome this problem, an asynchronous multi-stream method based on the data correlation of computing variables is proposed. This method aims to divide the computing tasks into multiple streams that can be executed concurrently by analyzing the data correlation between computing variables, thereby maximizing the utilization of computing resources and improving computing efficiency.

[0048] First, perform data correlation analysis on each variable in the node Gaussian gradient calculation; determine which variables' gradient calculations can be executed concurrently (i.e., there is no direct data dependency between them) and which need to be executed sequentially (i.e., there is a data dependency between them).

[0049] Based on the results of data correlation analysis, the computational task is divided into multiple computational flows. In this example, five computational flows can be divided for five different variables (three velocity components u, v, w, pressure p, and density rho).

[0050] Each computational flow is responsible for calculating the gradient value of a variable. The computational flows are submitted in an interleaved manner according to the GPU memory access and MPI communication conditions, and necessary inter-process data exchange (such as updating the gradient value of the virtual unit) is performed during the computation. In this process, the GPU computing core is ensured to be continuously busy, thus covering up the communication overhead.

[0051] These computation flows are executed in an asynchronous multi-stream manner. Specifically, multiple computation flows are started at the same time and executed in parallel on different computing resources (such as different CPU cores and GPU cores).

[0052] During the calculation process, when a calculation flow encounters data that needs to be waited for (such as waiting for the results of other calculation flows), the calculation flow can be paused and other prepared calculation flows can be executed instead.

[0053] In this way, it can be ensured that computing resources are always busy, thereby improving overall computing efficiency.

[0054] In node Gaussian gradient calculation, it is usually necessary to calculate the weighted sum of node values ​​and update the node values ​​of virtual units between processes. This is to ensure that the information of neighboring nodes can be taken into account when calculating the gradient.

[0055] In the asynchronous multi-stream execution framework, this step also needs to be processed in parallel, that is, when calculating the gradient value of each variable, the node value of the virtual unit is updated at the same time.

[0056] For the three velocity components on the symmetry plane, symmetry plane correction calculation needs to be performed. This step also involves updating the node values ​​of virtual units between processes. In asynchronous multi-stream execution, the symmetry plane correction calculation can be processed as an independent calculation flow, or it can be combined with the gradient calculation flow of a velocity component (depending on the results of data correlation analysis).

[0057] Figure 3 The original heterogeneous parallel node Gaussian computation process is presented, which completes the gradient computation of five variables in sequence and updates the gradient value of the virtual unit between processes in each computation step. This sequential execution method limits the parallel utilization of computing resources. Specifically, the data communication between MPI processes and the data communication between CPU and GPU cannot be covered with GPU computing resources, resulting in idle computing resources or communication modules. In contrast, the asynchronous multi-stream method decomposed by the data correlation of computing variables overcomes this limitation by executing multiple computing flows in parallel, thereby significantly improving computing efficiency.

[0058] In summary, the "asynchronous multi-flow decomposition by computational variable data correlation" method is an optimization technology for the node Gaussian gradient module in the laminar solver. It divides the computational flow by analyzing the data correlation between computational variables, and uses asynchronous multi-flow execution to improve the utilization of computing resources and overall computing efficiency.

[0059] Example 4

[0060] like Figure 4 As shown, Figure 4 Data dependency graph for multivariate nodal Gaussian gradient computation.

[0061] Data dependency analysis is performed on the six calculation modules in Example 3 (gradient calculation of five variables and symmetry plane correction calculation of velocity). The gradient calculation of five variables (including three velocity components u, v, w, pressure p and density rho) is concurrent, that is, initiating the calculation at the same time has no effect on the result, but the symmetry plane correction of the three velocity components must be completed before the velocity variable gradient calculation begins.

[0062] In order to solve the additional data transmission problems caused by the multi-machine and multi-card platform, by taking advantage of the concurrent nature of GPU computing and data transmission, an asynchronous multi-stream strategy is used to achieve mutual masking of computing and data transmission, improve the resource utilization of the device, and thus improve the overall performance of the code. This embodiment starts with data dependencies and implements asynchronous multi-stream optimization of CFD heterogeneous parallel computing on a multi-machine and multi-card platform without changing the computing core functions and underlying data structures of the prior art. This technology can be used for unstructured / structured grids, compressible / incompressible CFD, and laminar / turbulent solutions. Compared with the existing asynchronous multi-stream implementation of complex domain decomposition, this method has a simple implementation process. After applying this technology, the performance of heterogeneous parallel CFD on a multi-machine and multi-card platform can be greatly improved.

[0063] Example 5

[0064] like Figure 5 As shown, Figure 5 For the asynchronous multi-stream call graph of the multivariable node gradient method, the scheme of the method of decomposing according to the correlation of the calculation variable data is as follows: first, five calculation streams are allocated: Stream rho (density calculation stream), Stream u (velocity component u calculation stream), Stream v (velocity component v calculation stream), Stream w (velocity component w calculation stream), Stream p (pressure p calculation stream). Before the node Gaussian gradient calculation starts, the default calculation stream Default Stream is used to enter the gradient calculation module and randomly start the multi-stream calculation call.

[0065] in, Figure 5 The medium blue progress bar indicates that the GPU computing core is busy, and the pink progress bar indicates that the GPU memory access unit and the MPI inter-process communication module are working.

[0066] First, submit the calculation stream Stream u and start the symmetry surface correction calculation of the three velocity components on the calculation stream Stream u. When Stream u reaches the communication between GPU and CPU and between CPUs, submit the Stream rho calculation stream and start the density rho gradient calculation on the GPU.

[0067] Similarly, when Stream rho encounters the memory access and MPI communication module, Stream p is submitted to start the pressure p gradient calculation on the GPU. When Stream p encounters the GPU memory access and MPI communication module, Stream u is submitted again. At this time, Stream u has completed GPU memory access and MPI communication, and the three velocity component symmetry surface correction calculations are continued. When Stream u reaches the GPU memory access and CPU-to-CPU MPI communication module again, Stream rho is started again to ensure that the GPU computing core is always busy. Similarly, Stream v and Stream w are submitted alternately in the same mode. The above five streams are submitted alternately and asynchronously to complete the mutual covering of GPU computing, GPU memory access and MPI communication, thereby maximizing the use of GPU computing and memory access resources.

[0068] Example 6

[0069] like Figure 6 As shown, Figure 6 Solve flow graphs for asynchronous multi-stream laminar flows.

[0070] For the node Gaussian gradient module in the laminar flow solver, an asynchronous multi-flow optimization method based on the decomposition of the calculation variable data correlation is adopted. The original process involves completing the gradient calculation of five variables in sequence, and updating the gradient value of the virtual unit between processes during the calculation process. In addition, the node Gaussian method also needs to update the node value of the gradient value of the virtual unit when the weight is summed, and for the three velocity components on the symmetry plane, a symmetry plane correction is required, which also involves the update of the node value of the virtual unit. By performing data dependency analysis on these six calculation modules (gradient calculation of five variables and symmetry plane correction calculation of one velocity component), it is found that the gradient calculation of the five variables can be executed concurrently, that is, there is no data dependency between them, so the calculation can be initiated at the same time without affecting the final result. However, the symmetry plane correction calculation of the three velocity components needs to be completed before the corresponding velocity variable gradient calculation starts to ensure the consistency and correctness of the data.

[0071] The asynchronous multi-stream technology reconstructs the process according to the data correlation of the computing process. It allocates multiple computing flows according to the data correlation between each computing module, so that multiple computing flows can be executed concurrently, thereby realizing the mutual coverage of GPU computing, GPU memory access and MPI communication. This asynchronous multi-stream technology is applied to the laminar flow solution module and the turbulent flow solution module.

[0072] Among them, the construction of the asynchronous multi-flow laminar flow solver module first starts from the data correlation solution dependency of the module calculation process. The laminar flow solver mainly performs iterative update and solution on the flow field variable q[5], including the right-hand side residual calculation module and the linear equation group iterative update module. The linear equation group iteration will update the flow field variable q[5]. For multi-node MPI calculation, the flow field variable q[5] of the virtual unit needs to be updated before the laminar flow solver starts, and the remaining modules of the laminar flow solver will not change the flow field variable q[5] until the right-hand side residual calculation module ends. The gradient variables dqdx[5], dqdy[5], dqdz[5] are directly obtained from the flow field variable q[5]. The limiter variable Limit[5] is obtained by combining the flow field variable q[5] and the gradient variables dqdx[5], dqdy[5], dqdz[5]. The calculation of flux Flux[5] depends on the flow field variable q[5], the gradient variables dqdx[5], dqdy[5], dqdz[5] and the limiter variable Limit[5]. If the calculation takes fluid viscosity into consideration, the temperature variable as well as the contribution of temperature gradient and temperature to flux Flux[5] must also be calculated.

[0073] Therefore, according to the data dependency of the above calculation processes, we can construct Figure 6 The asynchronous multi-stream laminar flow solution process is shown.

[0074] First, when the solver initially updates the flow field variable q[5] of the virtual unit, an additional initial module calculation flow is allocated. The initial module contains the calculation data that needs to be initialized in subsequent calculations, including gradients, limiters, fluxes, residuals, etc., which masks the CPU and GPU data transmission when the virtual unit flow field variable q[5] is updated and the MPI communication between CPUs.

[0075] Secondly, based on the node Gaussian gradient asynchronous multi-flow masking node value transfer constructed by variable decomposition, some modules of the limiter are migrated to the virtual unit gradient value update module after the gradient calculation is completed. The two calculation flows correspond to some limiter modules and the gradient value update module of the virtual unit respectively. Here, the limiter calculation mainly includes the maximum and minimum values ​​of the flow field variables and some limiter calculation processes that are not related to the gradient value. The two calculation flows can be executed concurrently to cover the CPU and GPU data transmission and the MPI communication between CPUs when the gradient value of the virtual unit is updated.

[0076] The calculation process then proceeds to the remaining calculation part of the limiter variable Limit[5]. There is no CPU-GPU data transmission and MPI communication until the final limiter variable Limit[5] virtual unit communication. At this time, if the laminar flow calculation involves fluid viscosity, the temperature gradient calculation is performed in parallel with multiple flows here, and the calculation process of the limiter variable Limit[5] is used to cover the CPU-GPU data transmission and MPI communication of the virtual unit node value in the temperature gradient calculation. After the calculation of the limiter variable Limit[5] is completed, the Limit[5] virtual unit communication begins. At this time, the time step module is used to cover the communication process on it. The calculation data of the time step module only depends on the flow field variable q[5]. The Limit[5] virtual unit communication flow and the time step module calculation flow can be executed concurrently.

[0077] After the above calculation process is completed, the laminar flow solution proceeds to the flux Flux[5] calculation module. The first thing to be calculated is the contribution of the fluid inviscidity to the flux Flux[5]. This calculation process does not involve communication. If the calculation involves fluid viscosity, the virtual unit communication after the temperature gradient calculation is completed is placed here and executed concurrently with the calculation module of the contribution of the fluid inviscidity to the flux Flux[5], covering up the temperature gradient virtual unit communication. If the calculation involves fluid viscosity, the contribution of the fluid viscosity to the flux Flux[5] is continued to be calculated. At this time, all information related to the temperature has been obtained, and the calculation will no longer involve virtual unit communication. The asynchronous multi-flow technology that allocates data according to the correlation of the calculation process and the asynchronous multi-flow technology that allocates data according to the correlation of the calculation variables can cover up all CPU and GPU data transmission and MPI communication modules on the laminar flow solver, so as to maximize the utilization of device resources and improve the overall performance.

[0078] Example 7

[0079] like Figure 7 As shown, Figure 7 Reconstruct the computational flow chart of asynchronous multi-flow turbulence solution for data correlation process.

[0080] The turbulence calculation module is similar to the laminar flow solution module in that it is divided into the right-hand side residual calculation module and the linear equations iterative update module. The solution variable is only a single turbulence variable. Before the turbulence solution module starts, the turbulence variable needs to be updated by a virtual unit. The turbulence variables remain unchanged in the remaining modules until the turbulence right-hand side residual calculation module ends.

[0081] The asynchronous multi-flow construction of the turbulence module starts with the update of the initial turbulence variable virtual unit of the turbulence module. At this time, the calculation flow of turbulent kinetic energy generation is executed synchronously. This calculation process only depends on laminar flow data and can be executed concurrently with the update of the initial turbulence variable virtual unit, masking the communication process.

[0082] Secondly, the turbulence gradient calculation module also requires the communication of virtual unit node values ​​and turbulence variable values, as well as the update of virtual unit gradient values ​​after the gradient calculation is completed. Unlike the laminar solution data dependence, the turbulence gradient calculation has no effect on the subsequent calculation of turbulent viscosity flux terms and source terms. The turbulent viscosity flux terms and source terms can be migrated to the turbulence gradient calculation module, and the two calculation flows are executed synchronously to achieve the purpose of concealing the communication.

[0083] After the gradient calculation is completed, it proceeds to the turbulent inviscid flux part. This calculation module has no virtual unit communication, and the calculation can be completed by a single flow.

[0084] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition, characterized in that: The method comprises the following: Start the CFD solver and initialize the CFD solver; use the asynchronous multi-flow method; Among them, the asynchronous multi-stream method includes: a method of decomposing according to the correlation of calculation variable data and a method of reconstructing the process according to the correlation of calculation process data; The method of decomposing by calculation variable data correlation is an asynchronous multi-flow optimization strategy for the node Gaussian gradient module in the CFD solver, which realizes the concurrent execution of gradient calculation by analyzing the data dependency between the gradient calculation module for calculating five variables and the symmetry surface correction calculation module for the velocity component; The method of decomposing by calculating the correlation of variable data includes the following: Step S100: identifying six calculation modules in the node Gaussian gradient module, the modules including gradient calculation modules of five variables and a symmetry plane correction calculation module of velocity, performing data dependency analysis on the six calculation modules, wherein the gradient calculation of the five variables is concurrent; Step S200: Allocate five computing flows to be submitted alternately and asynchronously, for parallel execution of GPU computing, CPU memory access, and MPI communication; The method of reconstructing the process according to the data correlation of the computing process allocates multiple computing flows according to the data correlation between each computing module, so that multiple computing flows can be processed concurrently, thereby completing the mutual covering of GPU computing and CPU memory access and MPI communication; The method of reconstructing the process according to the correlation of the calculation process data is initiated according to the calculation process and is divided into a laminar flow solution module and a turbulent flow solution module, including the following: Step T100: Construct a laminar flow solution module based on the data correlation dependency of the laminar flow module calculation process; wherein, the laminar flow solver iteratively updates and solves the flow field variable q[5], including a right-hand side residual calculation module and a linear equation group iterative update module. The linear equation group iteration will update the flow field variable q[5]. For multi-node MPI calculations, the virtual unit flow field variable q[5] needs to be updated before the laminar flow solver starts, and the remaining modules of the laminar flow solver will not be updated until the right-hand side residual calculation module is completed. The field variable q[5] remains unchanged; the gradient variables dqdx[5], dqdy[5], dqdz[5] are directly obtained from the flow field variable q[5], the limiter variable Limit[5] is obtained by combining the flow field variable q[5] and the gradient variables dqdx[5], dqdy[5], dqdz[5], and the flux Flux[5] calculation depends on the flow field variable q[5], the gradient variables dqdx[5], dqdy[5], dqdz[5] and the limiter variable Limit[5]; Where, dqdx[5] is the component of the gradient variable in the x direction, dqdy[5] is the component of the gradient variable in the y direction, and dqdz[5] is the component of the gradient variable in the z direction; Step T200: Construct a turbulence solution module, in which the turbulence solver is divided into a right-hand side residual calculation module and a linear equation group iterative update module, and the solution variable is only a single turbulence variable; before the turbulence solution module starts, the turbulence variable needs to be updated by a virtual unit, and the remaining modules remain unchanged until the turbulence right-hand side residual calculation module ends.

2. The CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition according to claim 1, characterized in that: The five calculation streams are: density calculation stream Stream rho, velocity component u calculation stream Stream u, velocity component v calculation stream Stream v, velocity component w calculation stream Stream w, and pressure calculation stream Stream p; The step S200 includes the following: Step S201: Use the default calculation stream Default Stream to enter the gradient calculation module; Step S202: Start the symmetry plane correction calculation of three velocity components, three velocity components u, v, w, submit the calculation stream Stream u, when Stream u reaches the communication between GPU and CPU and between CPUs, submit the Stream rho calculation flow, and start the density rho gradient calculation on the GPU; Step S203: After submitting the Stream rho calculation flow, when Stream rho encounters the memory access and MPI communication modules, the Stream p calculation flow is submitted to start the pressure gradient calculation on the GPU; Step S204: After submitting the calculation flow of Stream p, when Stream p encounters the GPU memory access and MPI communication module, the calculation flow of Stream u is submitted again. At this time, the GPU memory access and MPI communication have been completed on Stream u, and the symmetry surface correction calculation of the three velocity components is continued; Step S205: After submitting the Stream u computing flow, when Stream u reaches the GPU memory access and CPU-to-CPU MPI communication module again, the Stream rho computing flow is started again to keep the GPU computing core busy all the time; Similarly, the calculation streams of Stream v and Stream w are also submitted alternately.

3. The CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition according to claim 1, characterized in that: The step T100 includes: Step T101: When the laminar flow solver initially updates the virtual unit flow field variable q[5], an additional initial module calculation flow is allocated. The initial module contains the calculation data that needs to be initialized in the subsequent calculation, which masks the CPU and GPU data transmission and the MPI communication between the CPU and the CPU when the virtual unit flow field variable q[5] is updated; secondly, in the node Gaussian gradient asynchronous multi-flow masking node constructed by the method of calculating the correlation decomposition of the variable data Virtual unit node On the basis of value transfer, part of the limiter module is migrated to the virtual unit gradient value update module after the gradient calculation is completed. The two calculation flows correspond to part of the limiter module and the virtual unit gradient value update module respectively. The two calculation flows can be executed concurrently to cover the CPU and GPU data transmission and the MPI communication between CPUs when the virtual unit gradient value is updated. First, when the solver initially updates the flow field variable q[5] of the virtual unit, an additional initial module calculation flow is allocated. The initial module contains the calculation data that needs to be initialized in subsequent calculations, including gradients, limiters, fluxes, and residuals. It masks the CPU and GPU data transmission and the MPI communication between CPUs when the virtual unit flow field variable q[5] is updated. The limiter calculation includes the maximum and minimum values ​​of the flow field variables and some limiter calculation processes that are not related to the gradient value. The two calculation flows can be executed concurrently to cover the CPU and GPU data transmission and the MPI communication between CPUs when the gradient value of the virtual unit is updated; Step T102: When the calculation process proceeds to the remaining calculation part of the limiter variable Limit[5], there is no CPU-GPU data transmission and inter-MPI communication until the final limiter variable Limit[5] virtual unit communication. At this time, if the laminar flow calculation involves fluid viscosity, the temperature gradient calculation is performed in parallel with multiple flows here, and the calculation process of the limiter variable Limit[5] is used to cover the virtual unit node Node value, CPU-GPU data transmission and inter-MPI communication in the temperature gradient calculation; Step T103: After the calculation of the limiter variable Limit[5] is completed, the Limit[5] virtual unit communication begins. At this time, the time stepping module is used to cover the communication process on it. The calculation data of the time stepping module only depends on the flow field variable q[5]. The Limit[5] virtual unit communication flow and the time stepping module calculation flow can be executed concurrently; Step T104: Calculate the flux Flux[5].

4. The CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition according to claim 3, characterized in that: The step T103 includes: First, the contribution of the fluid inviscidity to the flux Flux[5] is calculated. This calculation process does not involve communication. If the calculation involves fluid viscosity, the virtual unit communication after the temperature gradient calculation is completed is placed here and executed concurrently with the calculation module of the fluid inviscid contribution to the flux Flux[5], covering up the temperature gradient virtual unit communication; If the calculation involves fluid viscosity, the contribution of fluid viscosity to flux Flux[5] will be calculated. At this time, all information related to temperature has been obtained and the calculation will no longer involve virtual unit communication.

5. The CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition according to claim 1, characterized in that: The asynchronous multi-flow construction method of the turbulence solution module in step T200 comprises the following steps: Step T201: the initial turbulence variable virtual unit of the turbulence module is updated, and the turbulence kinetic energy generation amount calculation flow is executed synchronously at this time. This calculation process only depends on the data of the laminar flow solution module and can be executed concurrently with the initial turbulence variable virtual unit update, covering up the communication process therein; Step T202: The turbulence gradient calculation module requires the communication of the virtual unit node value and the turbulence variable value, as well as the update of the virtual unit gradient value after the gradient calculation is completed. The turbulence viscosity flux term and the source term are migrated to the turbulence gradient calculation module, and the two calculation flows are executed synchronously, thereby masking the communication; Step T203: After the gradient calculation is completed, it proceeds to the turbulent inviscid flux part. This calculation module has no virtual unit communication and completes the calculation with a single flow.

6. The CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition according to claim 1, characterized in that: The gradient calculation module of the five variables includes three velocity components u, v, w, pressure p and density rho, and the gradient calculation of the five variables is concurrent. The symmetry surface correction of the three velocity components needs to be completed in advance, and initiating the calculation at the same time does not affect the calculation result.

7. The CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition according to claim 5, characterized in that: The turbulence gradient calculation in step T202 has no effect on the calculation of the turbulence viscosity flux term and the source term. The turbulence viscosity flux term and the source term are migrated to the turbulence gradient calculation module, so that the two calculation flows are executed synchronously.

8. The CFD multi-GPU asynchronous multi-stream optimization method based on data dependency decomposition according to claim 7, characterized in that: The two computing flows are divided into computing flow 0 and computing flow 1.

Images