Fluid high-performance cpu-gpu heterogeneous parallel simulation computing system

Abstract

The application provides a fluid high-performance CPU-GPU heterogeneous parallel simulation calculation system, and relates to the technical field of computers, aiming at realizing fast and accurate numerical simulation of CFD.The system comprises a first calculation module based on a CPU and a second calculation module based on a GPU, wherein the first calculation module is used for carrying out flow field preprocessing on a simulation file, transmitting a grid file obtained through flow field preprocessing to the second calculation module, judging whether iteration is ended according to flow field data, and carrying out flow field post-processing according to the flow field data in the case that iteration is ended; and the second calculation module is used for carrying out iterative calculation on the grid file and transmitting flow field data obtained through iterative calculation to the first calculation module.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a high-performance CPU-GPU heterogeneous parallel simulation computing system for fluid. BACKGROUND

[0002] With the increasing maturity of Computational Fluid Dynamics (CFD) and the rapid development of computer technology, CFD has made outstanding achievements in the fields of aerospace, meteorology, shipbuilding, automobile, etc.

[0003] In engineering practice, numerical simulation of CFD (for example, full machine simulation of aircraft), generally requires that the calculation grid of a single calculation state is in the order of ten million or even one hundred million, and the number of states to be calculated is hundreds or thousands. Such a huge amount of calculation urgently needs to improve the computing efficiency.

[0004] In the related art, parallel computing based on a Central Processing Unit (CPU) cluster is an effective way to solve large-scale scientific computing problems, and the performance of the parallel computing cluster depends largely on the upgrading of the CPU. However, due to the limitation of power consumption, the number of transistors integrated in the unit area of the CPU is difficult to further increase, resulting in a slower update speed of the CPU and a bottleneck in development. The parallel computing scheme based on the CPU cluster is difficult to adapt to the current rapid and accurate numerical simulation requirements of CFD. Therefore, how to quickly and accurately realize the numerical simulation of CFD is a technical problem to be solved. SUMMARY

[0005] In view of the above problems, the embodiments of the present application provide a high-performance CPU-GPU heterogeneous parallel simulation computing system for fluid, so as to overcome the above problems or at least partially solve the above problems.

[0006] The embodiments of the present application provide a high-performance CPU-GPU heterogeneous parallel simulation computing system for fluid, which comprises a first computing module based on CPU and a second computing module based on GPU, wherein:

[0007] The first computing module is configured to perform flow field preprocessing on a simulation file, and transmit a grid file obtained by flow field preprocessing to the second computing module, and determine whether to end iteration according to flow field data, and in the case of ending iteration, perform flow field post-processing according to the flow field data.

[0008] The second computing module is configured to perform iterative calculation on the grid file, and transmit flow field data obtained by iterative calculation to the first computing module.

[0009] Optionally, the iterative calculation on the grid file comprises a plurality of calculation steps, and the plurality of calculation steps comprise reconstruction, face gradient, inviscid flux, and viscous flux.

[0010] The second calculation module is specifically configured to perform: at each calculation step, placing a plurality of threads corresponding to the calculation step into a same thread block for calculation, and the thread block represents a group of simultaneously started threads.

[0011] Optionally, the second calculation module performs memory access in a structure body storage mode during execution of each calculation step.

[0012] The structure body storage mode refers to that the conservative quantity and the flux are both aligned structure bodies, and are written and read uniformly in a kernel function, and intermediate variables are stored in GPU registers in the form of temporary variables.

[0013] Optionally, the iterative calculation on the grid file comprises time advancement, and the time advancement is used to determine the speed of the next iteration calculation.

[0014] The second calculation module is specifically configured to perform: during the iterative calculation on the grid file, time advancement is performed based on a target CFL number and according to a target time advancement format.

[0015] The CFL number represents the ratio of the time step to the space step of the fluid passing through the grid element, and the target time advancement format is: an upper and lower relaxation implicit time advancement format, or a grid block granularity-upper and lower relaxation implicit time advancement format.

[0016] Optionally, the second calculation module is further configured to calculate a residual based on the flow field data obtained in the current iteration, and determine the stability of the calculation based on the residual, and in the case of stability, increase the target CFL number to obtain a new target CFL number, and the new target CFL number is used for the next iteration calculation.

[0017] Optionally, the time advancement step of the upper and lower relaxation implicit time advancement format comprises initialization, inner iteration, weight update, and conservative quantity update, and the iteration number of the inner iteration ranges from 5 to 20.

[0018] Optionally, the grid block granularity-upper and lower relaxation implicit time advancement format uses a Roe format Jacobi matrix for implicit time advancement, and the Roe format represents a flux format based on difference flux splitting.

[0019] The Jacobi matrix is obtained by projecting three-dimensional flux onto two mutually perpendicular three-dimensional basis vectors in a projection manner.

[0020] Optionally, the second computing module is specifically configured to perform: transmitting the flow field data calculated in the current iteration to the first computing module after every target number of iterations.

[0021] Optionally, the first computing module is specifically configured to perform: calculating a residual according to the flow field data, and judging whether an iteration end condition is met according to the residual, and in the case where the iteration end condition is met, ending the iteration calculation, and performing flow field post-processing according to the flow field data.

[0022] Optionally, the second computing module is further configured to, after receiving the grid file, assign boundary conditions, and perform iteration calculation on the grid file based on the assigned boundary conditions.

[0023] The embodiments of the present application include the following advantages:

[0024] The fluid high-performance CPU-GPU heterogeneous parallel simulation computing system provided by the embodiments of the present application combines the complementary hardware differences between CPU and GPU, allocates different tasks in the computational fluid dynamics process on different hardware for processing, so as to improve the calculation rate; by combining the characteristics of the CPU that the storage and control unit is more and the calculation unit is less, tasks such as flow field preprocessing, judging whether to end iteration and flow field post-processing are allocated to the first computing module based on CPU for processing; by combining the characteristics of the GPU that the storage and control unit is less and the calculation unit is more, the task of iteration calculation on the grid file is allocated to the second computing module based on GPU. Moreover, the CPU and GPU only perform data transmission before the iteration calculation starts and when the flow field data obtained by the iteration calculation is output, so that the independence of the iteration calculation in the GPU can be ensured, thereby greatly reducing the delay problem caused by transmission in the comprehensive calculation and improving the calculation efficiency. In this way, fast and accurate CFD numerical simulation can be realized based on the system. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0026] Figure 1 is a structural schematic diagram of a fluid high-performance CPU-GPU heterogeneous parallel simulation computing system provided by the embodiments of the present application;

[0027] Figure 2 is a schematic diagram of time consumption of different computing steps provided by the embodiments of the present application;

[0028] Figure 3 is a time consumption comparison result of different parallel topologies provided by an embodiment of the present application;

[0029] Figure 4 is a comparison result of array-structure memory access modes provided by an embodiment of the present application;

[0030] Figure 5 is a time consumption comparison result of different structure storage modes provided by an embodiment of the present application;

[0031] Figure 6 is a variation diagram of CFL numbers in BDPLUR format provided by an embodiment of the present application;

[0032] Figure 7 is a diagram of a target CFL number determination method provided by an embodiment of the present application;

[0033] Figure 8 is a convergence speed comparison result of different CFL numbers in BDPLUR format provided by an embodiment of the present application;

[0034] Figure 9 is a time consumption comparison result of different CFL numbers provided by an embodiment of the present application;

[0035] Figure 10 is a convergence speed comparison result of DPLUR and LUSGS provided by an embodiment of the present application;

[0036] Figure 11 is a speedup ratio comparison result of DPLUR and LUSGS provided by an embodiment of the present application;

[0037] Figure 12 is a speedup ratio comparison result of different computing system-algorithm combinations provided by an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the above objectives, characteristics and advantages of the present application more apparent, clear and easy to understand, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative work fall within the scope of protection of the present application.

[0039] In order to better illustrate and understand the technical solutions of the present application, the technical concepts in the embodiments of the present application are first described.

[0040] (1) CPU-GPU heterogeneous computing system:

[0041] Developing parallel computing based on Central Processing Unit (CPU) cluster is an effective way to solve large-scale scientific computing problems, and the performance of parallel computing cluster depends largely on the upgrading of CPU. Moore's law points out that the number of transistors that can be accommodated on a microprocessor will double about every 18 months, and the performance will also double. The power consumption of the processor is proportional to its area, while the speed of the processor is proportional to the square root of the area. Limited by power consumption, it is difficult to further reduce the size of the transistor, so it is also difficult to further increase the number of transistors integrated in the unit area of the CPU, which leads to the slow update speed of the CPU and the development bottleneck. The Moore's law of CPU computing performance gradually loses its effectiveness.

[0042] At the same time, Graphic Processing Units (GPU) have strong floating-point operation ability and storage bandwidth in data parallelism, and they have begun to enter people's field of vision as a new type of scientific computing device. Compared with traditional CPU high-performance computing, GPU computing has the advantages of low cost, low power consumption, high performance, etc.

[0043] Due to the complementary hardware differences between CPU and GPU (as shown in Table 1), the combination of CPU and GPU can not only complement each other, but also select the appropriate computing unit for calculation according to the actual application, thereby providing higher performance, lower energy consumption, greater flexibility, and faster development cycle, avoiding waste of hardware resources and time. At the same time, the CPU-GPU heterogeneous computing system can also adapt to diversified computing needs and improve the overall performance of the system. At present, this system has been widely used in artificial intelligence, high-performance computing, graphics processing, scientific computing, financial data analysis and risk prediction, cloud computing and big data, Internet of Things and edge computing, etc.

[0044] Table 1 Hardware characteristics of CPU and GPU

[0045]

[0046] (2) Fluid high-performance heterogeneous parallel format:

[0047] Computational fluid mechanics mainly studies two key problems: calculation time and calculation accuracy, which are mutually restricted and contradictory. That is, pursuing calculation accuracy usually increases calculation time and calculation amount, and vice versa. The main way to solve this contradiction is to develop high-performance parallel computing. At present, there are two main directions: grid block granularity parallel algorithm represented by CPU cluster and grid point granularity parallel algorithm represented by GPU single card or cluster.

[0048] Generally, in the selection of time advancing format, CPU usually adopts implicit format, while GPU usually adopts explicit format. This is because the CFL (Courant, Friedrichs, Lewy) number of implicit format can be theoretically infinite, thus having higher computational efficiency than explicit format (the CFL number of explicit format generally cannot exceed 1, otherwise the calculation does not converge). However, the traditional pipeline type implicit time advancing format (for example, LUSGS format) which is very suitable for CPU parallel calculation, is not suitable for GPU parallel calculation due to the hardware differences between CPU and GPU (as shown in Table 1), so it is forced to use the explicit time advancing format which is limited by the CFL number, which greatly limits the advantages of GPU hardware itself in parallel calculation. How to break through the limitation and develop an implicit time advancing format suitable for GPU has become a difficulty in parallel calculation.

[0049] Therefore, in order to fill the gap of computational fluid dynamics in CPU-GPU heterogeneous efficient parallel solver, and solve the problem that the traditional time advancing format cannot use the advantages of heterogeneous system as much as possible, the embodiments of the present application provide a fluid high-performance CPU-GPU heterogeneous parallel simulation calculation system (also can be called: heterogeneous parallel iterative solver) combining the complementary hardware differences between CPU and GPU. The system allocates different tasks in the process of computational fluid dynamics on different hardware for processing to improve the rate of calculation; by combining the characteristics of CPU that the storage and control unit is more and the calculation unit is less, tasks such as flow field preprocessing, judging whether to end iteration and flow field post-processing are allocated to the first calculation module based on CPU for processing; by combining the characteristics of GPU that the storage and control unit is less and the calculation unit is more, the task of iterative calculation on the grid file is allocated to the second calculation module based on GPU. Moreover, the system only performs data transmission between CPU and GPU before starting the iterative calculation and when outputting the flow field data obtained by the iterative calculation, so as to ensure the independence of the iterative calculation in GPU, thereby greatly reducing the delay problem caused by transmission in comprehensive calculation and improving the calculation efficiency.

[0050] In addition, in order to improve the solving speed of the system, the key factors affecting the calculation acceleration of the system and the countermeasures are studied. In the iterative calculation process, by reasonably allocating the number of threads in the thread block, reducing the data communication between the host and the device, fully utilizing the shared memory and efficiently solving the Navier-Stokes equation, the parallel efficiency of the system is improved. At the same time, the Data-Parallel Lower-Upper Relaxation (DPLUR) and the Block Data Parallel Upper Power Relaxation (BDPLUR) are used to reduce the calculation time of the system and enhance the calculation efficiency of the system. The system uses structured grids, and can customize the use of the laminar / turbulent, compressible / incompressible fluid mechanics simulation modules as needed.

[0051] The fluid high-performance CPU-GPU heterogeneous parallel simulation computing system provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0052] Referring to Figure 1 , it is a structural schematic diagram of a fluid high-performance CPU-GPU heterogeneous parallel simulation computing system provided by the embodiments of the present application. As Figure 1 shown, the fluid high-performance CPU-GPU heterogeneous parallel simulation computing system comprises a first computing module based on a CPU and a second computing module based on a GPU, wherein: Figure 1 The first computing module is configured to perform flow field preprocessing on a simulation file, transmit a grid file obtained by the flow field preprocessing to the second computing module, and determine whether to end iteration according to flow field data, and perform flow field post-processing according to the flow field data in the case of ending iteration.

[0053] The second computing module is configured to perform iterative calculation on the grid file, and transmit flow field data obtained by the iterative calculation to the first computing module.

[0054]

[0055] ​In the embodiments of the present application, considering that the hardware characteristics of the CPU are that the storage and control units are more, and the calculation unit is less, which is suitable for performing memory allocation, logical control and other tasks; while the hardware characteristics of the GPU are just the opposite, that is, the storage and control units are less, and the calculation unit is more, which is more skilled in high-density and high-concurrency calculation; in addition, the bandwidth of the communication bus PCIE between the CPU and the GPU is only 10-20 GB / s, while the memory bandwidth of the register in the GPU is 8 TB / s. Therefore, the key to improve the parallel efficiency of the GPU is to reduce the data communication between the CPU and the GPU as much as possible. Therefore, in order to reduce this time-consuming item, according to the hardware characteristics of the CPU and the GPU, the structure of the fluid high-performance CPU-GPU heterogeneous parallel simulation calculation system is divided into two parts, that is, the first calculation module based on the CPU (processing in the CPU end) and the second calculation module based on the GPU (processing in the GPU end).

[0056] The first calculation module is responsible for flow field preprocessing (flow field preprocessing includes parameter reading and mesh division of simulation file), judging whether to end iteration and flow field post-processing (flow field post-processing includes a series of processing and analysis work of flow field data) and other tasks; the second calculation module is responsible for the task of iterative calculation of the grid file (that is, iterative calculation in the finite volume method); wherein, the iterative calculation of the grid file includes the calculation steps of boundary condition assignment, reconstruction and flux, and time advancement.

[0057] Moreover, the CPU and the GPU only perform data transmission before the start of the iterative calculation and when the flow field data is output, so as to ensure the independence of the iterative calculation in the GPU, thereby greatly reducing the delay problem caused by transmission in the comprehensive calculation and improving the calculation efficiency.

[0058] Specifically, the working process of the system is as follows: for the simulation file needing CFD, first, the first calculation module performs flow field preprocessing on the simulation file to obtain the grid file, and transmits the grid file to the second calculation module; the second calculation module performs iterative calculation (for example, performs boundary condition assignment, reconstruction, flux and time advancement) according to the grid file, and transmits the flow field data obtained by the iterative calculation to the first calculation module; then, the first calculation module judges whether to end iteration according to the flow field data, and in the case of ending iteration, performs flow field post-processing according to the flow field data.

[0059] It should be noted that the iterative calculation of the second calculation module is independent of the first calculation module, and the iterative calculation of the second calculation module will not stop in the process of the first calculation module judging whether to end iteration, and the second calculation module will continue to perform the next iteration calculation. Only when the first calculation module determines that the iteration is ended, the second calculation module will stop the iterative calculation.

[0060] In practical applications, a fluid high-performance CPU-GPU heterogeneous parallel simulation computing system, i.e., a heterogeneous parallel fluid mechanics solver, can be constructed based on an NVIDIA CUDA (Compute Unified Device Architecture) programming model.

[0061] By adopting the technical solutions of the embodiments of the present application, different tasks in the process of computational fluid mechanics are allocated to different hardware for processing based on the complementary hardware differences between the CPU and the GPU, so as to improve the calculation rate; and the CPU and the GPU only perform data transmission before starting the iterative calculation and when outputting the flow field data obtained through the iterative calculation, so that the independence of the iterative calculation in the GPU can be ensured, thereby greatly reducing the delay problem caused by transmission in the comprehensive calculation and improving the calculation efficiency. In this way, fast and accurate CFD numerical simulation can be realized based on the system.

[0062] In an optional embodiment, the second computing module is specifically configured to perform: transmitting the flow field data obtained through the current iteration to the first computing module after every target number of iterations.

[0063] In the embodiments of the present application, in order to reduce the data transmission delay, the first computing module and the second computing module only perform flow field data transmission once after every target number of iterations. For example, if the target number is 10, the second computing module only transmits the current flow field data to the first computing module after every 10 iterations, and the first computing module determines whether to end the iteration according to the flow field data.

[0064] In this way, flow field data transmission is not required after each iteration, the number of data transmission between the first computing module and the second computing module is reduced, and the total data transmission delay is further reduced.

[0065] In an optional embodiment, the first computing module is specifically configured to perform: calculating a residual based on the flow field data, determining whether an iteration end condition is met based on the residual, ending the iterative calculation in the case where the iteration end condition is met, and performing flow field post-processing based on the flow field data.

[0066] In the embodiments of the present application, the first computing module determines whether to end the iterative calculation by calculating a residual, for example, if the iteration end condition is that the residual is less than a preset residual threshold, when the residual calculated by the first computing module based on the flow field data is less than the preset residual value, the iteration is ended at this time, and the first computing module starts to perform the flow field post-processing operation.

[0067] In an alternative embodiment, the second computing module is further configured to, after receiving the grid file, assign boundary conditions, and perform iterative computation on the grid file based on the assigned boundary conditions.

[0068] In the embodiments, the iterative computation needs to be performed within a certain range of boundary conditions. After receiving the grid file, the second computing module assigns boundary conditions, and performs iterative computation on the grid file based on the assigned boundary conditions, so as to ensure smooth and accurate iterative computation.

[0069] The fluid high-performance CPU-GPU heterogeneous parallel simulation computing system is described below through performance optimization of the system in Section 1.1 and comparison and optimization of different iterative formats in Section 1.2.

[0070] 1.1 Performance optimization of the system

[0071] In order to improve the solving speed of the system, the key factors affecting the calculation speed of the system and the countermeasures are studied. As shown in Figure 2 Figure 2 is a schematic diagram of time consumption of different computing steps provided by the embodiments, and it can be seen that the time consumption of each computing step in the iterative computation execution process is quite different. The main time-consuming items are reconstruction, surface gradient, inviscid flux, and viscous flux, which account for more than 70% of the total time cost. The embodiments consider that the absolute time consumption of the four items is quite different under different parallel topological modes, and therefore, it is necessary to study the influence of the parallel topological mode on each time-consuming item.

[0072] In combination with the above embodiments, in an embodiment, the embodiments also provide a fluid high-performance CPU-GPU heterogeneous parallel simulation computing system. In the system, the iterative computation on the grid file includes a plurality of computing steps, and the plurality of computing steps include reconstruction, surface gradient, inviscid flux, and viscous flux.

[0073] The second computing module is specifically configured to perform: in each computing step, the plurality of threads corresponding to the computing step are put into the same thread block for computation, and the thread block represents a group of simultaneously started threads.

[0074] ​In the embodiments of the present application, a fluid high-performance CPU-GPU heterogeneous parallel simulation computing system can be constructed based on a CUDA programming model. In the CUDA programming model, threads of a thread block (Block) can work cooperatively and communicate through shared memory. A Grid is composed of multiple Blocks, and there are at most 32 threads in a Block. CUDA can organize a three-dimensional Grid and Block. GridDim is the number of thread blocks in the grid, usually represented as (GridDim.x, GridDim.y, GridDim.z). BlockDim is the number of threads in each thread block, usually represented as (BlockDim.x, BlockDim.y, BlockDim.z). That is, six dimensions of the Grid and Block need to be determined in CUDA. Since the number of grids in the structured grid is a three-dimensional array, when point granularity parallelism is used, the thread block organization method in CUDA needs to be determined according to the dimensions of the three-dimensional array, and there are 120 filling combinations.

[0075] Each calculation includes multiple threads, and the second calculation module, when performing iterative calculation, places the multiple threads corresponding to the calculation step into the same thread block for calculation at each calculation step, so as to ensure that all threads in an SM (Streaming Multiprocessor) are in an active state, thereby increasing the internal thread occupancy rate and improving parallel efficiency.

[0076] To illustrate the performance of the system, the embodiments of the present application set up four experimental groups, in which the calculation amount is completely the same, but the parallel topology mode is completely different, as shown in Table 2.

[0077] Table 2 Parallel topology mode experiment configuration

[0078]

[0079] The time consumption of the four different parallel topology modes in Table 1 is analyzed through five repeated experimental groups (Test1, Test2, Test3, Test4, Test5), as shown in Table 3. Figure 3 Figure 3 is a time consumption comparison result of different parallel topology modes provided by the embodiments of the present application. From Figure 3 ​As shown in the results, the calculation time of the experimental groups 2, 3 and 4 is similar, and the calculation time of the experimental group 1 is about twice that of the other experimental groups. In theory, the reason for this phenomenon is that the number of threads in a single Block is too small, and the total number of Blocks is too large (that is, the number of started SMs is also large), so that there is a redundant calculation resource in each thread bundle, thereby increasing the waiting time during calculation. It is worth noting that when the grid is increased to 353x161x2, the calculation speed of the experimental group 1 is nearly 5 times slower than that of the experimental group 2, and the experimental groups 3 and 4 have a situation of insufficient registers. According to the above analysis, since the experimental group 1 only allocates the number of threads in the thread block to 6, which is much smaller than the maximum value 32, the actual active thread occupancy rate in the GPU is low, and the parallel efficiency is also low. Therefore, this experiment proves that placing as many threads as possible in a Block can improve the occupancy rate and program parallel efficiency.

[0080] By adopting the technical solutions of the embodiments of the present application, when the second calculation module executes each calculation step, the second calculation module places multiple threads corresponding to the calculation step into the same thread block for calculation, so as to ensure that all threads in one SM are in an active state, thereby increasing the internal thread occupancy rate, reducing the time consumption of the parallel topology mode, and improving the parallel efficiency of the fluid high-performance CPU-GPU heterogeneous parallel simulation calculation system.

[0081] In combination with the above embodiments, in an embodiment, the embodiments of the present application also provide a fluid high-performance CPU-GPU heterogeneous parallel simulation calculation system, in which the second calculation module performs memory access in a structure body storage mode during execution of each calculation step.

[0082] In the structure body storage mode, the conservative quantity and the flux are both aligned structure bodies, and are written and read uniformly in the kernel function, and the intermediate variable is stored in the GPU register in the form of a temporary variable.

[0083] In the embodiments of the present application, the method of storing memory variables includes array storage and structure body storage, as shown in Figure 4 Figure 4 is a comparison result of an array-structure body memory access mode provided by the embodiments of the present application, and it can be seen that, compared with array storage, structure body storage can reduce the number of operations (commands) for accessing shared memory. Therefore, the embodiments of the present application perform memory access in a structure body storage mode to reduce the number of memory access times of global variables.

[0084] ​In theory, since the structure contains 7 variables, the number of operations accessing shared memory can be reduced to 1 / 7. However, the numerical experiment results show that in the gradient calculation of the 64x64x64 grid size, the time consumption of one memory access of the structure storage is only 21.65% less than that of multiple memory accesses of the array storage, which is much lower than the theoretical prediction result. The reason is that the structure storage is different from the classified storage and the unified storage. The numerical experiments in the embodiments of the present application are calculated by using the finite volume method. Since the finite volume method generates a large number of intermediate variables in each calculation step, if they are stored in the same structure (i.e. unified storage), a large number of unnecessary memory accesses will be generated when the kernel function calls the global variable, thereby reducing the utilization of computing resources. This is the reason why the numerical experiment results show that the structure storage is only 1 / 5 faster than the array storage.

[0085] Therefore, in order to further improve the utilization of computing resources, the structure storage method adopted in the embodiments of the present application is a structure classified storage method, the basic principle of which is to reduce the memory access amount as much as possible when the kernel function is called. The gradient calculation numerical experiment is still used for verification. For the global variable, the difference between the structure classified storage and the unified storage is compared as shown in Figure 5 Figure 5 is a time consumption comparison result of different structure storage modes provided by the embodiments of the present application, each experimental group is respectively used for 6 repeated tests (Test1, Test2, Test3, Test4, Test5, Test6), and the average value of the time consumption of the 6 tests is taken as the final time consumption. From the experimental results, it can be seen that the calculation time consumption of the classified storage is reduced by 40.1% compared with the unified storage. Overall, the calculation time consumption of the structure classified storage can be reduced by 50% compared with the array storage.

[0086] By using the technical scheme of the embodiments of the present application, the second calculation module performs memory access according to the structure storage method in the process of executing each calculation step, so as to reduce the memory access times of the global variable, reduce the memory access time, and further improve the parallel efficiency of the fluid high-performance CPU-GPU heterogeneous parallel simulation computing system.

[0087] 1.2 Comparison and optimization of different iteration formats

[0088] In combination with the above embodiments, in an embodiment, the embodiments of the present application also provide a fluid high-performance CPU-GPU heterogeneous parallel simulation computing system, in which the iteration calculation on the grid file includes time advancement, and the time advancement is used to determine the speed of the next iteration calculation.

[0089] The second calculation module is specifically configured to perform: in the process of iteration calculation on the grid file, performing time advancement based on a target CFL number and according to a target time advancement format.​

[0090] wherein CFL number represents the ratio of time step and space step of fluid passing through the grid cell, and the target time advancing format is: a lower-upper symmetric Gauss-Seidel (LUSGS) implicit time advancing format, or a block-structured grid size-lower-upper symmetric Gauss-Seidel (BDPLUR) implicit time advancing format.

[0091] In the embodiments of the present application, it is considered that the traditional iterative format (for example, the LUSGS format) is more suitable for iterative calculation on a CPU, and cannot exert its calculation advantage on a CPU-GPU heterogeneous system. Therefore, the lower-upper symmetric Gauss-Seidel (LUSGS) implicit time advancing format or the block-structured grid size-lower-upper symmetric Gauss-Seidel (BDPLUR) implicit time advancing format is adopted for time advancing in the embodiments of the present application, so as to reduce the calculation time consumption of the fluid high-performance CPU-GPU heterogeneous parallel simulation calculation system and enhance the calculation efficiency of the system.

[0092] 1.2.1 Optimization of CFL number in advancing format:

[0093] In the embodiments of the present application, the CFL number represents the ratio of time step and space step of fluid passing through the grid cell. Generally, when the CFL number is within a certain range, the larger the CFL number, the faster the convergence of iterative calculation. However, since different calculation examples have different CFL number ranges, and some calculation examples may present a case that the optimal CFL number increases with the increase of the number of iterations. Therefore, setting a fixed CFL number for a certain calculation example is not conducive to achieving the best convergence speed and stability requirements. In addition, the block-structured grid size-lower-upper symmetric Gauss-Seidel (BDPLUR) implicit time advancing format itself adopts Roe format for implicit time advancing format in the calculation of inviscid flux, which has smaller dissipation than the lower-upper symmetric Gauss-Seidel (DPLUR) implicit time advancing format, so that the CFL number limit range of the BDPLUR format is narrower than that of the DPLUR format. Therefore, in actual calculation, in order to ensure convergence stability, the CFL number of the BDPLUR format is generally set to be smaller than that of the DPLUR format. For example, as shown in the three-dimensional Airfoil Near-Wake calculation example, the fixed CFL number range used by the DPLUR format is 1-1000, and the fixed CFL number range used by the BDPLUR format is only about 1-10. However, according to the actual situation, the CFL number of the BDPLUR format gradually increases from the initial 10 to the final 1000 with the increase of the number of iterations. Figure 6

[0094] Therefore, the embodiments of the present application provide a self-adaptive hill climbing algorithm for a floating CFL number, which determines whether the CFL number needs to be increased in the next state according to the residual value state in each state, so as to achieve the effect of the maximum convergence speed under the requirement of convergence stability, that is, to determine the target CFL number according to the residual value.

[0095] ​In combination with the above embodiments, in an embodiment, the application further provides a fluid high-performance CPU-GPU heterogeneous parallel simulation computing system, in which the second computing module is further configured to calculate a residual according to the flow field data obtained in the current iteration, and determine the stability of the calculation according to the residual, and in the case of stability, increase the target CFL number to obtain a new target CFL number, which is used for the next iteration calculation.

[0096] As shown in Figure 7 , Figure 7 is a schematic diagram of a method for determining a target CFL number provided by the application. After the second computing module completes each computing step of the current iteration calculation, the second computing module calculates a residual according to the current state (i.e. the flow field data obtained in the current iteration), and determines the stability of the calculation according to the residual. In the case of stability, the target CFL number is increased to obtain a new target CFL number, and the next state (i.e. the next iteration calculation) is entered, and the flow field data obtained in the current iteration is saved (information is maintained). In the case of instability, the target CFL number remains unchanged, and the flow field data in the previous state is read for iteration calculation. In this way, the second computing module determines whether the CFL number should be increased in the next state according to the residual value in each state, so as to achieve the maximum convergence speed under the requirement of convergence stability.

[0097] In order to illustrate the effectiveness of the adaptive determination of the target CFL number in the embodiments of the application, the following takes the BDPLUR format as an example for illustration. As shown in Figure 8 , Figure 8 is a comparison result of the convergence speed of different CFL numbers in the BDPLUR format provided by the application, in which BDP-Mixed-5 represents that the target CFL number is a fixed value of 5, BDP-Mixed-10 represents that the target CFL number is a fixed value of 10, BDP-Fixed-5 represents that the target CFL number is adaptively determined to be 5, and BDP-Fixed-10 represents that the target CFL number is adaptively determined to be 10. It can be seen that after the adaptive determination of the target CFL number is used, the convergence slope during calculation is greatly increased, and there are fewer burrs, so that the convergence process is more stable. Therefore, the adaptive determination of the target CFL number in the embodiments of the application has significant advantages in accelerating the calculation speed and increasing the convergence stability.

[0098] In addition, by analyzing the time consumption under different CFL numbers, as shown in Figure 9 , Figure 9BDP-10 represents the time consumption of the target CFL number of 10 in the DPLUR format, BDP-5 represents the time consumption of the target CFL number of 5 in the DPLUR format, BDP-20 represents the time consumption of the target CFL number of 20 in the DPLUR format, BDP-50 represents the time consumption of the target CFL number of 50 in the DPLUR format, DP-5 represents the time consumption of the target CFL number of 5 in the DPLUR format, and LUSGS-5 represents the time consumption in the LUSGS format. It can be seen that, compared with the fixed CFL number form, the method of adaptively adjusting the target CFL number according to the residual error in the second calculation module of the embodiment of the application makes the calculation efficiency of the BDP format increase by nearly 5 times. Therefore, this experiment proves that using the adaptive CFL number has an irreplaceable role in further improving the finiteness, specificity of the CFL number of the BDP-LUR format and improving the calculation speed.

[0099] 1.2.2 Optimization of the inner iteration number (JIIS number) of the DPLUR format:

[0100] In the embodiment of the application, the time marching step of the upper and lower relaxation implicit time marching format includes initialization, inner iteration, weight updating and conservative quantity updating, and the iteration number of the inner iteration ranges from 5 to 20.

[0101] The upper and lower relaxation implicit time marching format is different from the LUSGS format, and the LUSGS format only needs one inner iteration process, while the upper and lower relaxation implicit time marching format needs to reach the JIIS number (Jacobi InterIteration Steps) before terminating the inner iteration, that is, it has multiple inner iterations. However, due to the characteristics of the Jacobi iteration method, the upper and lower relaxation implicit time marching format needs to be iterated multiple times to achieve overall convergence. Therefore, in order to converge faster, the embodiment of the application analyzes the value of the JIIS number (i.e. the iteration number of the inner iteration) to determine that the efficiency is highest when the iteration number of the inner iteration is 5-20, and therefore the value range of the iteration number of the inner iteration is 5 to 20.

[0102] Specifically, the influence of the JIIS number on the time marching format DPLUR is analyzed and compared with the LUSGS format. The same GPU and the same CFL number (all taking 1000) are used to compare the convergence speed and the speedup ratio of the two formats. As shown in Figure 10 and Figure 11 , the convergence speed comparison result of DPLUR and LUSGS provided by the embodiment of the application is Figure 10 , the speedup ratio comparison result of DPLUR and LUSGS provided by the embodiment of the application is Figure 11 , and Figure 10 .Figure 11 In the present application, "X" in "DP-X" represents the JIIS value selected by the DPLUR format (for example, DP-5 represents that the JIIS value selected by the DPLUR format is 5). It can be seen that the system uses the DPLUR format for time advancement to overcome the front-back order dependence of LUSGS, can greatly release the GPU hardware point granularity parallel computing advantage, and through comparison with the NASA standard example, verifies the accuracy and reliability of the system, and the calculation rate is increased by more than 40 times, and the Jacobi inner iteration number is 5-20.

[0103] 1.2.3 Jacobi iteration matrix optimization for BDPLUR format:

[0104] In the embodiments of the present application, the grid block granularity-up and down relaxation implicit time advancement format uses the Roe format Jacobi matrix for implicit time advancement, and the Roe format represents a flux format based on difference flux splitting.

[0105] The Jacobi matrix is obtained by projecting three-dimensional flux onto two mutually perpendicular three-dimensional basis vectors in a projection manner.

[0106] Specifically, the Roe format represents a flux format based on difference flux splitting. Since it is obtained by approximation based on a more physical Riemann solver, its accuracy is significantly improved compared with the FVS format, and in particular, its shear wave resolution capability is outstanding, so it is often used in display flux calculation. In implicit calculation, the Jacobi matrix using the Roe format can obtain better convergence characteristics, but its stability is poor, and the selection of the CFL number cannot be infinite, and each example has its adaptive CFL number, which makes its application more limited.

[0107] Specific flux is expressed as follows:

[0108]

[0109] That is, the Roe format Jacobi matrix is:

[0110]

[0111] wherein the flux is used to calculate the numerical flux on the cell interface; and respectively represent the state variables on the left and right sides of the cell interface, and the state variables usually include density, momentum, energy and other conservative variables; represents a flux function; denotes the eigenvector matrix composed of the eigenvectors of the flux Jacobian matrix, which is used to project the difference of the state variables to the eigenspace; denotes the inverse matrix of the eigenvector matrix, which is used to convert the quantity in the eigenspace back to the physical space; denotes a diagonal matrix, which usually contains the absolute values of the eigenvalues of the flux Jacobian matrix, and it is used to handle the propagation direction of the wave in the eigenspace.

[0112] Since the BDPLUR format uses the Roe format Jacobian matrix for implicit time advancement, the Jacobian matrix needs to be implicit, otherwise it will seriously affect the convergence performance of the implicit time advancement format.

[0113] In the embodiment of the application, two mutually perpendicular three-dimensional basis vectors , are constructed according to the projection mode, and the original three-dimensional flux is projected onto the two basis vectors to obtain the Jacobian matrix.

[0114] For example, the Z- form is taken as an example, and the specific format is as follows:

[0115] ,

[0116] ,

[0117] where the rotation system ,

[0118] ,

[0119] .

[0120] At the same time, due to the specificity of different meshes, the embodiment of the application adopts a hybrid projection Jacobian matrix to ensure the stability of the convergence of the BDPLUR format, that is, the above , , are changed to the following format:

[0121] ,

[0122] where the vectors are all unit matrices.

[0123] In the embodiments of the present application, in order to verify the performance of the Jacobi matrix obtained based on the projection mode, the iteration speeds of four groups of matrices are compared through a three-dimensional Airfoil Near-Wake example. Through experimental analysis, it is obtained that after the calculation reaches a residual of 10-11, the original Roe format Jacobi matrix converges very slowly, and the time consumptions of the hybrid projection Jacobi matrix and the Jacobi matrices in the X, Y and Z directions of the separate projection are respectively: 177.64s, 186.844s, 192.656s and 185.609s. The time consumptions of the three Jacobi matrices of the separate projection are similar, and the calculation speed of the hybrid projection Jacobi matrix is increased by nearly 5% compared with the other three matrices.

[0124] Exemplarily, Figure 12 is a comparison result of acceleration ratios of different computing system-algorithm combinations provided by the embodiments of the present application, wherein BDP-GPU represents that the system of the embodiments of the present application adopts the DPLUR format for iteration, DP-GPU represents that the system of the embodiments of the present application adopts the DPLUR format for iteration, LUSGS-GPU represents that the system of the embodiments of the present application adopts the LUSGS format for iteration, and LUSGS-CPU represents that the traditional solver adopts the LUSGS format for iteration. According to Figure 12 It can be known that the test results of the same example and the same convergence degree show that in the three-dimensional case, the convergence speed of the iteration method of the system adopting the DPLUR format is the highest and can be increased by nearly 20 times compared with the iteration method of the traditional solver adopting the LUSGS format. And the convergence speed of the iteration method of the system adopting the BDPLUR format is the highest and can be increased by nearly 500 times compared with the iteration method of the traditional solver adopting the traditional LUSGS format.

[0125] In summary, the fluid high-performance CPU-GPU heterogeneous parallel simulation computing system of the embodiments of the present application combines the complementary hardware differences between the CPU and the GPU, allocates different tasks in the process of computational fluid dynamics on different hardware for processing, so as to improve the calculation rate; and the system only performs data transmission between the CPU and the GPU before starting the iteration calculation and when outputting the flow field data obtained by the iteration calculation, so as to ensure the independence of the iteration calculation in the GPU, thereby greatly reducing the delay problem caused by transmission in the comprehensive calculation and improving the calculation efficiency. In addition, the second computing module puts the multiple threads corresponding to the calculation steps into the same thread block for calculation in the iteration calculation process, reduces the data communication between the host and the device, and improves the parallel efficiency of the system; the upper and lower relaxation implicit time advancing format or the grid block granularity-upper and lower relaxation implicit time advancing format is used for time advancing, so as to reduce the calculation time consumption of the fluid high-performance CPU-GPU heterogeneous parallel simulation computing system and enhance the calculation efficiency of the system.

[0126] In practical applications, a "high-performance CPU-GPU heterogeneous parallel simulation computing system for fluid" can be written in C++, which can run on a Windows desktop platform and simulate the mechanical properties of compressible fluids. The main functions include: two-dimensional / three-dimensional compressible finite volume computational fluid dynamics (CFD) based on structured grid and parallel computing mode with NVIDIA CUDA programming environment. The system can be started by command line and input parameters by text.

[0127] Wherein, the main content of each folder in the system is as follows: head: contains the header file (.h file) of the program; mesh: contains the grid file of each example; source: contains the source file of the program; output: contains the results of the program calculation.

[0128] The hardware devices of the system can include: CPU: 2GHZ or above; GPU: NVIDIA GeForce RTX 3090 or above; memory: 2GM or above; hard disk: 80G or above; microcomputer or workstation: Windows10 operating system; CUDA programming environment: CUDA 11.2 version or above.

[0129] The software supported by the system can include: post-processing tool: Tecplot 2013 and later versions; development tool: Intel C++ compiler, Intel MKL high-performance mathematical library; development tool: Microsoft Visual Stutio; development driver: CUDA 11.2.

[0130] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between each embodiment can be referred to each other.

[0131] The embodiments of the present application are described with reference to flowcharts and / or block diagrams of systems according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal equipment realize a function implemented in the flowchart and / or block diagram. Figure 1 Each flow or multiple flows and / or blocks Figure 1apparatuses that perform the functions specified in a flowchart or multiple flows and / or a block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart or flowcharts Figure 1 a flowchart or multiple flows and / or a block or multiple blocks Figure 1 a flowchart or multiple flows and / or a block or multiple blocks Figure 1 a flowchart or multiple flows and / or a block or multiple blocks Figure 1 a flowchart or multiple flows and / or a block or multiple blocks

[0132] Although preferred embodiments of the application have been described, those skilled in the art will recognize that additional modifications and changes can be made thereto without departing from the scope of the application. Accordingly, the appended claims are intended to cover all such modifications and changes as fall within the scope of the application.

[0133] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and are more especially used for the purpose of distinction from other elements in the specification. Also, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0134] The above provides a fluid high-performance CPU-GPU heterogeneous parallel simulation computing system, and the principles and implementation manners of the application are described by specific examples. The above description of the embodiments is only used to help understand the method and core idea of the application; meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the application. In summary, the content of the specification should not be understood as a limitation of the application.

Claims

1. A fluid high-performance CPU-GPU heterogeneous parallel simulation computing system, characterized in that, The method comprises the following steps: a first computing module based on a CPU and a second computing module based on a GPU, wherein: the first computing module is configured to perform flow field preprocessing on a simulation file, transmit a mesh file obtained by the flow field preprocessing to the second computing module, and determine whether iteration is ended according to flow field data, and in the case where the iteration is ended, perform flow field post-processing according to the flow field data; the second computing module is configured to perform iterative calculation on the mesh file and transmit flow field data obtained by the iterative calculation to the first computing module; wherein the iterative calculation on the mesh file comprises time marching, and the time marching is configured to determine a speed of next iteration calculation; the second computing module is specifically configured to perform: in the process of the iterative calculation on the mesh file, time marching is performed according to a target time marching format based on a target CFL number; wherein the CFL number represents a ratio of a time step and a space step of fluid through a mesh element, and the target time marching format is: an upper and lower relaxation implicit time marching format, or a mesh block granularity-upper and lower relaxation implicit time marching format; the second computing module is specifically configured to perform: after every target number of iterations, the flow field data obtained by the current iteration is transmitted to the first computing module.

2. The system of claim 1, wherein, The iterative calculation on the mesh file comprises a plurality of calculation steps, and the plurality of calculation steps comprise: reconstruction, face gradient, inviscid flux, and viscous flux; the second computing module is specifically configured to perform: in each calculation step, a plurality of threads corresponding to the calculation step are put into a same thread block for calculation, and the thread block represents a group of threads started at the same time.

3. The system of claim 2, wherein, In the process of performing each calculation step, the second computing module performs memory access according to a structure body storage mode; wherein the structure body storage mode refers to: both conservative quantity and flux use aligned structure bodies, and are written and read uniformly in a kernel function, and intermediate variables are stored in GPU registers in the form of temporary variables.

4. The system of claim 1, wherein, The second computing module is further configured to calculate a residual based on the flow field data obtained by the current iteration, determine the stability of the calculation based on the residual, and in the case where the calculation is stable, increase the target CFL number to obtain a new target CFL number, and the new target CFL number is used for next iteration calculation.

5. The system of claim 1, wherein, The time marching step of the upper and lower relaxation implicit time marching format comprises: initialization, inner iteration, weight update, and conservative quantity update, and the iteration number of the inner iteration ranges from 5 to 20.

6. The system of claim 1, wherein, The mesh block granularity-upper and lower relaxation implicit time marching format uses a Roe format Jacobi matrix for implicit time marching, and the Roe format represents a flux format based on difference flux splitting. wherein the Jacobi matrix is obtained by projecting three-dimensional flux onto two mutually perpendicular three-dimensional basis vectors in a projection manner.

7. The system of claim 1, wherein, The first computing module is specifically configured to perform: calculating a residual based on the flow field data, determining whether an iteration end condition is met based on the residual, ending the iteration calculation in the case where the iteration end condition is met, and performing flow field post-processing according to the flow field data.

8. The system of any of claims 1-7, wherein, The second calculation module is further configured to, after receiving the grid file, assign boundary conditions, and perform iterative calculation on the grid file based on the assigned boundary conditions.

Images