A Parallel Control Method and Device for Watershed Water Environment and Aquatic Ecology Simulation

Abstract

The present invention discloses a parallel control method for water environment and water ecology simulation in a river basin, including: Step 1: Construct a multi-program parallelism and adopt a parallel control system that combines the message passing mode (MPI) and the shared memory parallel mode (OPENMP); Step 2: Construct a multi-program boundary exchange control method; Step 3: Set the multi-program calculation boundary exchange rules. By constructing a multi-program parallelism and a parallel control system that combines MPI and OPENMP, the present invention performs program partitioning on the hydrodynamic-water quality-water ecology calculation process and data, sets the communication rules between programs, constructs a multi-program boundary exchange control module, and sets the multi-program calculation boundary exchange rules, so as to achieve the precise extraction, asynchronous reading and writing, communication control, and automatic exchange of calculation boundaries of multi-program calculation data, thereby effectively improving the river basin simulation efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of simulation, and particularly relates to a multi-program efficient parallel control method for a multi-dimensional multi-interface multi-index whole-process model of the water environment and water ecology in a river basin. Background Art

[0002] With the continuous development of hydrodynamic calculations based on finite volume method numerical calculations and two- and three-dimensional hydrodynamic water quality mathematical models, the calculation units have gradually expanded from local to the basin scale. The demand for simulation calculation accuracy at the basin scale is getting higher and higher, the basin calculation grids are increasing, and the involved hydrodynamic, water quality, and water ecological processes are becoming more and more complex. Due to different hardware conditions and computing capabilities for simulation calculations, how to flexibly construct the parallel control system of the water environment and water ecology model in the river basin to adapt to the characteristics and calculation conditions of the river basin and maximize the calculation efficiency is an important prerequisite for the current large-scale water environment and water ecology simulation calculations in the river basin and also an important technical problem for improving the simulation ability and efficiency.

[0003] Since the simulation of the water environment and water ecology in the river basin is a process with extremely strong spatio-temporal correlation and intricate mechanisms, the calculation grids are tightly coupled, and there is a need for large-scale, multi-level, and whole-process data exchange between parallel sub-units, which poses requirements for the setting of parallel granularity.

[0004] Currently, parallel calculations at the thread level usually use the shared memory parallel mode (OPENMP), and distributed parallel calculations for multi-core and multi-machine usually use the message passing mode (MPI). The separate or combined use of the two can achieve data sharing between parallel sub-units, thus solving the spatio-temporal dependence problem of parallel units. Although the combination of MPI and OPENMP can solve the communication problems of a certain number of parallel units, when performing parallel calculations for water environment and water ecology simulations at the basin scale, since OPENMP mainly focuses on loop parallelization and is only applicable to the parallel calculation of the same type of indicators with strong independence at the bottom layer, and unit parallelization mainly uses MPI for control communication, the two are limited to parallelization within the same program. For complex hydrodynamic, water quality, and water volume coupling simulation calculations, the amount of exchanged data increases exponentially, the exchange method is single, and the communication load distribution is uneven, which greatly restricts the calculation efficiency and accuracy.

[0005] Therefore, the existing technologies need to be improved. Summary of the Invention

[0006] The object of the present invention is to provide a parallel control method for watershed water environment and water ecology simulation in view of the deficiencies of the prior art. By constructing a parallel control system combining multi-program parallelism, MPI and OPENMP, the hydrodynamic-water quality-water ecology calculation process and data are divided and processed, the communication rules between programs are set, a multi-program boundary exchange control module is constructed, and the multi-program calculation boundary exchange rules are set to achieve accurate extraction, asynchronous reading and writing, communication control and automatic exchange of calculation boundaries of multi-program calculation data, thereby effectively improving the efficiency of watershed simulation.

[0007] In a first aspect, an embodiment of the present invention provides a parallel control method for watershed water environment and water ecology simulation, the method comprising:

[0008] Step 1: Construct a parallel control system that combines multi-program parallelism and uses the message passing mode (MPI) and the shared memory parallel mode (OPENMP) in combination;

[0009] Step 2: Construct a multi-program boundary exchange control method;

[0010] Step 3: Set multi-program calculation boundary exchange rules.

[0011] As a further improved technical solution, step 1 specifically includes:

[0012] S11: Divide the river network into blocks, divide the parallel levels, and use the message passing mode (MPI) to control the data exchange and communication between different blocks, and use the number of blocks as the number of processes to be started;

[0013] S12: Perform secondary division on the grid blocks as parallel domains, each parallel domain is controlled by a program, and multi-program parallel control is used.

[0014] As a further improved technical solution, inside the grid, OPENMP is used to control the parallel calculation of multiple indicators, including the parallelism of hydrodynamic multiple indicators - water quality multiple indicators - water ecology multiple indicators.

[0015] As a further improved technical solution, select the combination of multiple adjacent blocks with geometric boundaries as the division standard of the multi-program parallel domain.

[0016] As a further improved technical solution, each parallel domain is controlled by a program, each program identification number is PPID, each program includes multiple processes, OPENMP is used to handle a single process, the parallel threads sharing the memory adopt a fork-join relationship, and the loop parallel method is used in the calculation. There is only one main thread before the start of the loop. When parallel calculation is required, several branch threads are spawned to execute the loop parallel tasks. After the parallel code is executed, the branch threads converge and hand over the control flow to a separate main thread.

[0017] As a further improved technical solution, step two specifically includes:

[0018] Step S21: Set the exchange data key-value;

[0019] Step S22: Call the set and get functions of Redis to perform storage and reading;

[0020] Step S23: Package the module and provide an interface.

[0021] As a further improved technical solution, step three specifically includes:

[0022] Step 31: Set the multi-program data exchange boundary, and based on the exchange rule file completed in the block step of step S11, set the multi-program exchange rule;

[0023] Step 32: Set the exchange scheme.

[0024] As a further improved technical solution, the exchange scheme includes: Invoke the exchange module interface in the main program and set two parallel modes of local or cross-machine: When the program is arranged on different machines, add a judgment on whether the sending program identifier SEND_PID is equal to the receiving program identifier REC_PID. If they are equal, that is, when the program is arranged on the same machine, perform storage and extraction in the local Redis database. If they are not equal, set the IP and port of the host Redis, and uniformly store the data in the host Redis database during cross-machine communication.

[0025] In a second aspect, the present invention provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements a parallel control method for water environment and water ecology simulation in a basin as described in any one of the above.

[0026] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. The computer program, when executed by a processor, implements a parallel control method for water environment and water ecology simulation in a basin as described in any one of the above.

[0027] Compared with the prior art, the embodiments of the present invention have the following advantages:

[0028] The present invention constructs a parallel control system combining multi-program parallelism, MPI, and OPENMP, performs program partitioning on the hydrodynamic-water quality-water ecology calculation process and data, sets the communication rules between programs, constructs a multi-program boundary exchange control module, sets the multi-program calculation boundary exchange rule, realizes the accurate extraction, asynchronous reading and writing, communication control, and automatic exchange of calculation boundaries of multi-program calculation data, thereby effectively improving the basin simulation efficiency.

[0029] By dividing the parallel domain, the present invention constructs a parallel control system that combines multi-program parallelism, MPI, and OPENMP, overcoming the limitation of both being limited to parallelism within the same program and improving the running efficiency.

[0030] The present invention facilitates data transmission between programs by setting a multi-program boundary exchange control method. By setting multi-program calculation boundary exchange rules, the present invention ensures the continuity of numbers among multi-program systems, simplifies the calculation process, and improves the operation efficiency.

[0031] The present invention sets an exchange scheme to ensure the correctness and efficiency of data access. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a module diagram of the parallel control method for basin water environment and water ecology simulation provided by the present invention;

[0033] Figure 2 is a schematic diagram of the parallel control system provided by the present invention;

[0034] Figure 3 is a schematic diagram of key-value pair storage and reading provided by the present invention;

[0035] Figure 4 is a flowchart of generating program exchange information provided by the present invention;

[0036] Figure 5 is a flowchart of generating cross-program exchange identification numbers provided by the present invention;

[0037] Figure 6 is a schematic diagram of cross-machine exchange reading and writing provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The present invention will be further described below with reference to the embodiments shown in the drawings.

[0039] Embodiment 1

[0040] The present invention provides a parallel control method for basin water environment and water ecology simulation, Figure 1 being a module diagram of the parallel control method for basin water environment and water ecology simulation provided by the present invention, as Figure 1 shown, the present invention includes the following steps:

[0041] Step 1: Construct multi-program parallelism and adopt a parallel control system that combines MPI and OPENMP.

[0042] As Figure 2 shown, Step 1 specifically includes:

[0043] S11: Divide the river network into blocks and divide it into parallel levels.

[0044] Specifically, the river network is divided into blocks based on the generalization of the actual river network relationship. Within the grid, OPENMP is used to control the parallel calculation of multiple indicators, including the parallel calculation of multiple hydrodynamic indicators (water level, flow, flow velocity, etc.) - multiple water quality indicators (conventional water quality, heavy metals, etc.) - multiple water ecological indicators (algae, fish, phytoplankton, etc.).

[0045] River network block is to split a complete watershed grid into multiple independent grids for parallel calculation. At the same time, in order to ensure the accuracy of the calculation, the message passing mode (MPI) is used to control the data exchange communication between different blocks to reduce the truncation error caused by the block. For the MPI parallel level, the grid block is used as the parallel domain division standard, and a new process is opened for each block. In this process, the two functions MPI_SEND and MPI_RECV are used to control the sending and receiving of data. The object identifier to be sent or received, that is, the block number, needs to be set in the function parameters to realize point-to-point data transmission.

[0046] When initializing MPI, the number of blocks is used as the number of processes to be opened, the send / receive process ID is IMCR, the current block number is MA, the send block ID is MA_SEND, and the receive block ID is MA_REC. IMCR follows the MPI settings and is numbered from 0, while MA is numbered from 1. In order to correspond one to one, MA needs to be reduced by 1, that is, the send process ID IMCR = MA_SEND-1, and the receive process ID IMCR = MA_REC-1.

[0047] S12: Divide the grid blocks into secondary blocks as parallel domains and use multi-program parallel control.

[0048] The parallel domain corresponds to the parallel level of multi-program parallelism, which is located at the upper level of MPI parallelism. Since MPI parallelism is mainly parallelism within the same program, it has poor flexibility when performing large-scale parallel computing, so a new parallel level is added to expand the applicability of parallel computing. The parallel domain refers to the combination of partial blocks, which corresponds to the level of multi-program parallelism. When parallel, the objects of parallelism are multiple programs, and the area where the programs act is within the parallel domain.

[0049] Specifically, the grid blocks are divided twice, which is usually determined according to the hardware and communication conditions, speed requirements and computational complexity. Preferably, a combination of multiple blocks adjacent to each other in geometric boundaries can be selected as the division criteria for the multi-program parallel domain, such as Figure 2 As shown, two parallel domains are constructed.

[0050] Each parallel domain is controlled by a program, each program has an identifier of PPID, each program includes multiple processes, OPENMP is used to handle a single process, the fork-join relationship is adopted among the parallel threads sharing the memory, for the same type of metrics, there is independence among them, and the calculation processes are similar. When calculating, the loop parallel method is used. Before the start of the loop, there is only one main thread. When parallel calculation is required, several branch threads are spawned to execute the loop parallel tasks. After the parallel code is executed, the branch threads converge and hand over the control flow to the single main thread. Specifically, usually the DO loop in Fortran is used for loop calculation. At this time, the parallel construction instruction of!$OMP PARALLEL DO in OPENMP is called, and the number of loop iterations can be distributed to different threads for separate execution..

[0051] Thus, a parallel granularity with multi-program parallelism and three levels of MPI and OPENMP is constructed, and the data exchange process occurs during the gap between each iterative calculation.

[0052] Step 2: Construct a multi-program boundary exchange control method.

[0053] The multi-program boundary exchange is the data exchange at the block boundary of the parallel domain. In order to minimize the influence of truncation error, usually the data exchange area is set near the block boundary.

[0054] Step 2 specifically includes:

[0055] Step S21: Set the exchanged data key-value.

[0056] Set a unique communication identifier for each process of each program, which consists of the sending program identifier SEND_PID, the sending process identifier SEND_MYID, the receiving program identifier REC_PID, the receiving process identifier REC_MYID and the sending data length LENC to form the string "LPUSH exchange SEND_PID SEND_MYID REC_PID REC_MYID LENC" as the uniquely specified sending key key, and the sending data SEND_VALUE corresponding to the given sending key key; similarly, the receiving key key is "BRPOP exchange SEND_PID SEND_MYID REC_PID REC_MYID LENC". After the key value matching is completed, the REC_VALUE array is used to receive the data corresponding to the key. LPUSH and BRPOP are artificially set identifiers to facilitate distinguishing and checking the data sending and receiving situations when the program outputs.

[0057] Step S22: Call the set and get functions of Redis to perform storage and reading.

[0058] As Figure 3 shown, call the redisConnect() function provided by the Redis database, input the database IP address and database access port, perform the database connection operation. After successfully connecting to the Redis database, assign values to the key-value pair. Then call the redisCommand() function, set the database storage object, and splice the key-value into the string commSet as the input parameter of the function, so as to execute the set method to store the data in Redis; similarly, when receiving, after completing the database connection and key assignment, splice the key into the string commGet, call redisCommand() to execute the get method to obtain the redisReply structure array, extract the exchange data DATA from it and assign it to the REC_VALUE array.

[0059] Step S23: Package the module and provide an interface.

[0060] Due to compatibility issues, use the C language to write the Redis sending and receiving function modules c_sendC.c and c_recvC.c to send / receive the data datas. Among them, the module input parameters include the data length lenr, the sending program number send_pidr, the sending process number send_myidr, the receiving program number rec_pidr, and the receiving process number rec_myidr; while the main program is written in Fortran. When calling, use the method of subroutine conversion to write two Fortran modules f_send.f90 and f_recv.f90, call the C language interface in them, and then call the Redis sending and receiving modules, and complete the transfer and conversion of Fortran variables to C language variables.

[0061] Step Three: Set the multi-program calculation boundary exchange rules.

[0062] Specifically include:

[0063] Step 31: Set the multi-program data exchange boundary, and based on the exchange rule file completed in the block step of Step S11, set the multi-program exchange rules.

[0064] The exchange rule file is to set the area of the exchange information and the specific content of the information during MPI communication.

[0065] See Figure 4, given the total number of blocks NU_MA and the total number of programs NU_PR, the exchange information file P_EXC.DAT for each program is calculated, which includes three parameters: program ID PPID, starting block ID EXC_ID of the program, and the number of blocks included in the program NU_EXC. Among them, the program ID PPID is given manually to facilitate the distinction of different programs, usually incrementing the number starting from 1. The starting block ID of the program is determined by the parallel domain division method. All the blocks included in the parallel domain have unique IDs. From the block ID, the starting block ID of the program can be known. The number of blocks included in the program is also determined by the parallel domain division method. NU_EXC is equal to the number of blocks included in the parallel domain.

[0066] NU_EXCC is essentially NU_EXC before rounding, obtained by dividing the total number of blocks NU_MA by the total number of programs NU_PR, which reflects the number of blocks included in each parallel domain and must be an integer. However, the result NU_EXCC obtained after division is not necessarily an integer, so it is necessary to determine whether to round. Then, a loop is carried out. Given a variable i starting from 1 and gradually increasing, representing the program ID increasing from 1, the program ID PPID and the starting block ID EXC_ID of each program are calculated from the values of i and NU_EXC until the program ID PPID is equal to the total number of programs NU_PR, at which point the entire calculation process ends.

[0067] When allocating processes, since the block numbers are continuous in multiple programs and share a set of block numbering systems, and due to the setting of MPI, the process numbers used by different programs need to start from 0 again for re-numbering, and the process numbering system for each program is independent.

[0068] To ensure the correspondence between block numbers and process numbers, as Figure 5 shown, during initialization, MA MA_SEND MA_REC_PPID NU_EXC EXC_ID are used as the input values of the program flow.

[0069] In multi-program parallelism, each program calls MPI for multi-process parallelism. The processes included in each program need to be re-numbered. Since the MPI process numbering is stipulated to start from 0, the total number of processes in the program is equal to the total number of blocks in this program.

[0070] In the data sending process, MA_SEND is equal to MA. In the receiving process, MA_REC is equal to MA.

[0071] For the data sending process, when calling MPI for data exchange, according to the settings of MPI, a unique digital identification number needs to be given to each process. Its value range starts from 0 and is less than the total number of processes. In this application, the sending process number SEND_MYID is the process identification number of MPI in each program. At the same time, to meet the above value range, SEND_MYID is set to the difference between MA_SEND and EXC_ID.

[0072] Set the initial value of the sending process number SEND_MYID to the difference between the sending block number MA_SEND and the program start block number EXC_ID, that is, SEND_MYID = MA_SEND - EXC_ID = MA - EXC_ID, and the receiving process number REC_MYID = MA_REC - EXC_ID. The initial value of the sending program number SEND_PID is equal to the current program number PPID.

[0073] For the data receiving process, set the initial value of the receiving process number REC_MYID to the difference between the receiving block number MA_REC and the program start block number EXC_ID, REC_MYID = MA_REC - EXC_ID, and the initial value of the sending process number SEND_MYID is SEND_MYID = MA_SEND - EXC_ID. The initial value of the receiving program number REC_PID is equal to the current program number PPID.

[0074] For the data sending process, see Figure 5 , considering that there are two cases of cross-program exchange and in-program exchange, the situation is judged according to the value of the sending process number SEND_MYID at this time. If it is less than 0, it means that the process sending data belongs to another program numbered before the receiving program. Then the sending program number SEND_PID is equal to the difference between the current SEND_PID and 1, and the sending process number is equal to the sum of the current SEND_MYID and NU_EXC. If the value of SEND_MYID is greater than or equal to 0 and less than NU_EXC, it means that the process sending data belongs to the current program, and the values of SEND_PID and SEND_MYID are equal to the initial values. If the value of SEND_MYID is greater than or equal to NU_EXC, it means that the process sending data belongs to another program numbered after the receiving program. Then the sending process number SEND_MYID is equal to the difference between the current SEND_MYID and NU_EXC, and the sending program number SEND_PID is equal to the sum of the current SEND_PID and 1.

[0075] The receiving program number REC_PID and the receiving process number REC_MYID are the same when sending data.

[0076] Through this setting method, it is ensured that:

[0077] (1) The block numbers SEND_MA / REC_MA are consecutive among multiple program systems;

[0078] (2) Each program process number starts from 0 and is less than NU_EXC. The total number of processes is equal to the total number of blocks, and the process numbers correspond to the block numbers.

[0079] Step 32: Set the exchange scheme.

[0080] When the program is deployed on different machines, read and write in the local Redis database; otherwise, uniformly store the data in the host database during cross-machine communication.

[0081] In the main program, call the exchange module interface and set two parallel modes: local or cross-machine. When the program is deployed on different machines, add a judgment to check if the sending program identifier SEND_PID is equal to the receiving program identifier REC_PID. If they are equal, that is, when the program is deployed on the same machine, perform storage and extraction in the local Redis database. If not, set the IP and port of the host Redis, and uniformly store the data in the host Redis database during cross-machine communication, as Figure 6 shown.

[0082] It should be understood that the parts not elaborated in detail in this specification all belong to the prior art.

[0083] The protection scope of the present invention is not limited to the above embodiments. Obviously, those skilled in the art can make various changes and deformations to the present invention without departing from the scope and spirit of the present invention. If these changes and deformations fall within the scope of the claims of the present invention and its equivalent technologies, the intention of the present invention also includes these changes and deformations.

Claims

1. A parallel control method for water environment and water ecology simulation in a river basin, characterized in that The method includes the following steps: Step 1: Construct a parallel control system that combines the message passing mode MPI and the shared memory parallel mode OPENMP for multi-program parallelism. Step 2: Construct a multi-program boundary exchange control method. Step 3: Set the multi-program calculation boundary exchange rules. Among them, Step 1 specifically includes: S11: Divide the river network into blocks, divide the parallel levels, use the message passing mode MPI to control the data exchange and communication between different blocks, and use the number of blocks as the number of processes to be started. S12: Perform secondary division on the grid blocks as parallel domains, and each parallel domain is controlled by a program, using multi-program parallel control. Inside the grid, use OPENMP to control the parallel calculation of multiple indicators, including the parallelism of hydrodynamic multi-indicators - water quality multi-indicators - water ecological multi-indicators. Step 2 specifically includes: Step S21: Set the exchange data key-value. Step S22: Call the set and get functions of Redis to perform storage and reading. Step S23: Package the module and provide an interface. Step 3 specifically includes: Step 31: Set the multi-program data exchange boundary, and based on the exchange rule file completed in the block division step of Step S11, set the multi-program exchange rules. Step 32: Set the exchange scheme. The exchange scheme includes: Invoke the exchange module interface in the main program and set two parallel modes: local machine or cross-machine. When the program is deployed on different machines, add a judgment to determine whether the sending program identifier SEND_PID is equal to the receiving program identifier REC_PID. If they are equal, that is, when the program is deployed on the same machine, perform storage and extraction in the local Redis database. If they are not equal, set the IP and port of the host Redis, and uniformly store the data in the host Redis database during cross-machine communication.

2. The method according to claim 1, wherein Select the combination of multiple adjacent blocks with geometric boundaries as the division criterion for the multi-program parallel domain.

3. The method according to claim 1, wherein Each parallel domain is controlled by a program, each program identification number is PPID, each program includes multiple processes, uses OPENMP to handle a single process, and the parallel threads in the shared memory adopt a fork-join relationship. When calculating, use the loop parallel method. There is only one main thread before the start of the loop. When parallel calculation is required, several branch threads are spawned to execute the loop parallel task. After the parallel code is executed, the branch threads converge and hand over the control flow to a single main thread.

4. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements a parallel control method for simulating the water environment and water ecology of a basin as described in any one of claims 1 to 3.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a parallel control method for simulating the water environment and water ecology of a basin as described in any one of claims 1 to 3.

Images