A communication method and device for electromagnetic transient real-time parallel simulation data

Abstract

The present application discloses a communication method and device for electromagnetic transient real-time parallel simulation data. The method includes: writing subnet messages into a shared memory managed by a first subnet process, obtaining subnet messages from a shared memory managed by a second subnet process, and realizing communication between first-layer subnet processes; aggregating messages of the first subnet process, forming data packets for transmission and reception, and placing the data packets into a bus for communication, and realizing communication between second-layer subnet processes; and establishing a cross-host shared memory area based on remote direct data access technology, writing messages of the first subnet process into a memory area managed by a remote target subnet process through a non-blocking write mechanism, and reading messages of the first subnet process through the remote target subnet process, and realizing communication between third-layer subnet processes.

Description

Technical Field

[0001] The present application relates to the field of power systems, and in particular to a communication method and device for real-time parallel simulation data of electromagnetic transients. Background Art

[0002] The use of subnet parallelism in electromagnetic transient real-time simulation has become the current mainstream trend. Subnet parallelism is to divide the entire network that needs to be simulated in electromagnetic transient real-time into multiple different subnets, and the subnets are connected by transmission lines. Different subnets will be assigned to different CPU cores of multi-core computers for simulation calculations, thereby achieving parallel acceleration of different subnets. At the same time, subnets need to communicate with each other during the simulation calculation process, that is, the voltage, current and other required information on the transmission line where the interactive subnets connect to each other, so as to achieve iterative progress of global simulation.

[0003] From the perspective of electromagnetic transient real-time simulation applications, the status of subnets is completely equal and the execution is also parallel. Therefore, the communication between subnets also has the following characteristics: complete equality and parallelism from one subnet to another (point-to-point). Ideally, the communication time of a single subnet should only be related to the point-to-point communication volume of a single subnet and will not be affected by other subnets. When the number of subnets is relatively small, the number of CPUs is relatively small, and the distribution meets the SMP architecture, and the hardware conditions are similar to the ideal situation.

[0004] However, as the scale of electromagnetic transient real-time simulation becomes larger and larger, the demand for scalability becomes higher and higher, while the computing power of a single CPU is limited, and the number of subnets has increased from a few to dozens or even hundreds. The rapid increase in the number of subnets has led to a significant increase in the demand for CPU cores that carry subnet operations. Large-scale electromagnetic transient real-time parallel simulation can only be achieved using large servers or supercomputer clusters. The CPU distribution of large servers and supercomputers uses the NUMA architecture, which is quite different from the SMP architecture and does not have the parity characteristics of CPU cores.

[0005] The increase in the number of subnets has led to a rapid increase in the amount of communication between subnets, and the dense point-to-point communication has directly led to congestion in the direct message transmission between subnets through the inter-core communication of the CPU, and the communication performance has dropped significantly. After testing, in the scenario of 250 CPUs, the communication time spent on the same amount of communication in each subnet is 20 times that in the scenario of two CPUs. This communication performance has become the biggest limitation of large-scale electromagnetic transient real-time simulation.

[0006] With respect to the technical problems existing in the above-mentioned prior art, as the scale of electromagnetic transient real-time simulation becomes larger and larger, the increase in the number of subnets leads to a rapid increase in the communication volume between subnets, and the intensive point-to-point communication directly leads to congestion of direct message transmission between subnets through CPU inter-core communication, and the communication performance is greatly reduced. No effective solution has been proposed yet. Summary of the invention

[0007] The embodiments of the present disclosure provide a communication method for electromagnetic transient real-time parallel simulation data, so as to at least solve the technical problems existing in the prior art that as the scale of electromagnetic transient real-time simulation becomes larger and larger, the increase in the number of subnets leads to a rapid increase in the communication volume between subnets, and the intensive point-to-point communication directly leads to congestion of direct message transmission between subnets through CPU inter-core communication, resulting in a significant decrease in communication performance.

[0008] According to one aspect of an embodiment of the present disclosure, a communication method for electromagnetic transient real-time parallel simulation data is provided, comprising: writing subnet messages into a shared memory that is managed by a first subnet process, and obtaining subnet messages from a shared memory that is managed by a second subnet process, so as to achieve communication between first-layer subnet processes; aggregating subnet information to form a data packet for sending and receiving, placing the data packet into a bus for communication, so as to achieve communication between second-layer subnet processes; and establishing a cross-host shared memory area based on remote direct data access technology, writing the message of the first subnet process into a memory area managed by a remote target subnet process through a non-blocking write mechanism, and reading the message of the first subnet process through the remote target subnet process, so as to achieve communication between third-layer subnet processes.

[0009] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is running, a processor executes any one of the methods described above.

[0010] According to another aspect of the embodiments of the present disclosure, there is also provided a communication device for electromagnetic transient real-time parallel simulation data, including: a first communication module, used to write subnet messages into a shared memory responsible for a first subnet process, and obtain subnet messages from a shared memory responsible for a second subnet process, so as to realize communication between first-layer subnet processes; a second communication module, used to aggregate subnet information, form a data packet for sending and receiving, put the data packet into a bus for communication, so as to realize communication between second-layer subnet processes; and a third communication module, used to establish a cross-host shared memory area based on remote direct data access technology, write the message of the first subnet process into the memory area managed by the remote target subnet process through a non-blocking write mechanism, read the message of the first subnet process through the remote target subnet process, so as to realize communication between third-layer subnet processes.

[0011] According to another aspect of the embodiments of the present disclosure, there is also provided a communication device for electromagnetic transient real-time parallel simulation data, comprising: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: writing subnet messages into a shared memory responsible for a first subnet process, and obtaining subnet messages from a shared memory responsible for a second subnet process, to achieve communication between first-layer subnet processes; aggregating messages of the first subnet process to form a data packet for sending and receiving, placing the data packet into a bus for communication, to achieve communication between second-layer subnet processes; and establishing a cross-host shared memory area based on remote direct data access technology, writing messages of the first subnet process into a memory area managed by a remote target subnet process through a non-blocking write mechanism, reading messages of the first subnet process through the remote target subnet process, to achieve communication between third-layer subnet processes.

[0012] In the disclosed embodiment, a multi-layer communication method is provided to support scalable real-time parallel simulation calculation of electromagnetic transient in power system, replacing the original parallel dense point-to-point communication. Through the deep integration of electromagnetic transient parallel simulation tasks and the server hardware responsible for calculation, on the basis of the NUMA architecture of the computing hardware, the communication volume, communication mechanism and communication time of the simulation tasks are controlled to achieve global communication optimization configuration, thereby solving the core problem of reduced communication performance. It is achieved that as the simulation scale expands, the overall communication performance is greatly improved synchronously, and it is no longer a key bottleneck for large-scale electromagnetic transient real-time parallel simulation.

[0013] This solves the technical problem in the prior art that as the scale of electromagnetic transient real-time simulation becomes larger and larger, the increase in the number of subnets leads to a rapid increase in the amount of communication between subnets, and the intensive point-to-point communication directly leads to congestion in direct message transmission between subnets through CPU inter-core communication, resulting in a significant decrease in communication performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0015] Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to Embodiment 1 of the present disclosure;

[0016] Figure 2 It is a flow chart of a method for communicating electromagnetic transient real-time parallel simulation data according to the first aspect of Embodiment 1 of the present disclosure;

[0017] Figure 3It is a schematic diagram of a three-layer communication structure for real-time parallel simulation calculation of electromagnetic transient according to the second aspect of Embodiment 1 of the present disclosure;

[0018] Figure 4 is a schematic diagram of a timing mechanism for message transmission of a layer 3 communication task according to the first aspect of Embodiment 1 of the present disclosure;

[0019] Figure 5 is a schematic diagram of a communication device for electromagnetic transient real-time parallel simulation data according to Embodiment 2 of the present disclosure; and

[0020] Figure 6 It is a schematic diagram of a communication device for electromagnetic transient real-time parallel simulation data according to Embodiment 3 of the present disclosure. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0023] Example 1

[0024] According to this embodiment, an embodiment of a communication method for electromagnetic transient real-time parallel simulation data is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0025] The method embodiment provided in this embodiment can be executed in a server or a similar computing device. Figure 1 The hardware structure block diagram of a computing device for implementing a communication method for electromagnetic transient real-time parallel simulation data is shown. Figure 1 As shown, the computing device may include one or more processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, and a transmission device for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0026] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computing device. As involved in the embodiments of the present disclosure, the data processing circuitry acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0027] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the communication method of electromagnetic transient real-time parallel simulation data in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the communication method of electromagnetic transient real-time parallel simulation data of the above-mentioned application is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0028] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computing device. In one example, the transmission device includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0029] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of the computing device.

[0030] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing devices described above.

[0031] According to a first aspect of the present embodiment, a method for communicating electromagnetic transient real-time parallel simulation data is provided. Figure 2 A schematic diagram showing the process of the method is shown in FIG. Figure 2 As shown, the method includes:

[0032] S202: writing the subnet message into the shared memory of the first subnet process, and obtaining the subnet message from the shared memory of the second subnet process, so as to realize communication between the subnet processes of the first layer;

[0033] S204: Aggregate the subnet messages to form a data packet for transmission and reception, determine a time slice that matches the time slice corresponding to the data packet among all time slices in the time-sharing bus according to the time slice corresponding to the data packet, put the data packet into the bus for communication, and realize communication between the second-layer subnet processes; and

[0034] S206: A cross-host shared memory area is established based on remote direct data access technology, and the message of the first subnet process is written into the memory area managed by the remote target subnet process through a non-blocking write mechanism. The message of the first subnet process is read through the remote target subnet process to realize communication between the third-layer subnet processes.

[0035] Specifically, in this embodiment, the subnet message is written into the shared memory responsible for the first subnet process, and the subnet message is obtained from the shared memory responsible for the second subnet process, so as to realize the communication between the first layer of subnet processes. The messages of the first subnet process are summarized and formed into a data packet for sending and receiving. According to the time slice corresponding to the data packet, the time slice matching the time slice corresponding to the data packet is determined among all the time slices in the time-sharing bus, and the data packet is put into the bus for communication, so as to realize the communication between the second layer of subnet processes. Based on the remote direct data access technology, a cross-host shared memory area is established, and through the non-blocking write mechanism, the message of the first subnet process is dropped into the memory area managed by the remote target subnet process, and the message of the first subnet process is read by the remote target subnet process, so as to realize the communication between the third layer of subnet processes.

[0036] For example, refer to Figure 3 As shown in the figure, there is a large-scale electromagnetic transient real-time parallel simulation case consisting of 32 subnets (subnet numbers are 0 to 31). The calculation time of each subnet is from t0, t1, t2..t 32 Among them, t0 is the largest, t 32 Minimum. The communication message from subnet i to subnet j is represented by m ij , each subnet needs to communicate with the other 7 subnets, so the communication message in the subnet is expressed as: m ij1 ...m ij7 The computing system uses two servers (Node 1 (i.e. Figure 3 The corresponding node on the left) and Node 2 (i.e. Figure 3 Each server has two physical CPUs (Intel Xeon), each with eight CPU cores. The servers are connected using the InfinineBand network.

[0037] 1) Subnet CPU task allocation. In order to average the computing time of each node machine, subnets 1, 5, 9, 13, 17, 21, 25, 29 are distributed on the physical CPU 1 of Node 1; subnets 3, 7, 11, 15, 19, 23, 27, 31 are distributed on the physical CPU 2 of Node 1; among them, subnets 2, 6, 10, 14, 18, 22, 26, 30 are distributed on the physical CPU 1 of Node 2; subnets 4, 8, 12, 16, 20, 24, 28, 32 are distributed on the physical CPU 2 of Node 2.

[0038] 2) Establish a three-layer communication system. The first layer is the subnet communication task between multiple CPU cores within the physical CPU, including 4 communication groups. The second layer is the subnet communication task between different CPU cores between different physical CPUs in the same host. The third layer is the subnet communication task between different CPU cores between different hosts. ij1 ...m ij7 . It is divided into different levels, where the first level communication task is a ij1 ...a ijx , the second layer communication task is b ij1 ...b ijx , the third layer communication task is c ij1 ...c ijx .

[0039] 3) Initialize the first layer communication system. The four CPUs 29, 30, 31, and 32 with the shortest execution time of computing tasks will serve as the main cores in the communication group of the first layer communication system, responsible for the allocation of 4 shared memories, and divide the shared memory into eight consecutive segments. Each segment is allocated to a CPU in the group. All other CPUs will obtain access rights to this shared memory.

[0040] 4) Initialize the second layer of communication, using the peer-to-peer topology search method. ij1 ...b ijx The communication task is to establish the message topology diagram between all CPU cores. First search the non-adjacent edges on the topology diagram as the task of the first time slice of bus time-sharing utilization, then continue to search the remaining non-adjacent edges as the task of the second time slice of bus time-sharing utilization, and then continue to search and divide the time slices of bus time-sharing utilization until all edges on the topology diagram are searched. Finally, the bus time-sharing utilization time slice number of each message communication is returned. Let b ijx The time slice number is tpx.

[0041] 5) Initialize the third layer of communication, use the MPI tool to complete the RDMA configuration of Node1 and Node2, and initialize MPI.

[0042] refer to Figure 4 As shown, 6) the first step of simulation begins.

[0043] 7) Each process performs layer 3 communication and ij1 ...c ijx The message is directly thrown into the memory area managed by the remote target subnet through MPI_Isend.

[0044] 8) Each subnet process enters the internal calculation link.

[0045] 9) Each subnet process performs the first layer communication task a in the internal computing link ij1 ...a ijx Write operation of the message on shared memory.

[0046] 10) Each subnet completes most of the internal calculations and all first-layer communication tasks. ij1 ...a ijx Write operation of the message on shared memory.

[0047] 11) Each subnet enters the blocking synchronization (Barrier) operation.

[0048] 12) Each subnet returns from the blocking synchronization (Barrier) and starts the second layer communication task b ij1 ...b ijx . According to the sorting of time slices. Query the current communication time slice number of the bus. If the communication task b with the subnet process ijx If the time slice number matches, the communication message is put into the bus communication.

[0049] 13) Complete all Layer 2 communication tasks.

[0050] 14) The subnet performs the read operation of the first-layer communication task and obtains the messages sent to this subnet by other subnets from the shared memory segment responsible for the other subnet processes.

[0051] 15) Each subnet process enters the remaining internal calculation links.

[0052] 16) Each subnet performs the third layer communication message receiving operation, queries the MPI_Wait mechanism, and determines c ij1 ...c ijx The message has not been completely received and parsed using the message.

[0053] 17) End all message reading and restart step (6).

[0054] Thus, a multi-layer communication method that supports scalable real-time parallel simulation of electromagnetic transients in power systems is provided to replace the original parallel dense point-to-point communication. Through the deep integration of electromagnetic transient parallel simulation tasks and the server hardware responsible for the calculation, the communication volume, communication mechanism and communication time of the simulation tasks are controlled on the basis of the NUMA architecture of the computing hardware, and the global communication optimization configuration is realized, thereby solving the core problem of reduced communication performance. As the simulation scale expands, the overall communication performance is greatly improved synchronously, and it is no longer a key bottleneck for large-scale electromagnetic transient real-time parallel simulation.

[0055] This solves the technical problem in the prior art that as the scale of electromagnetic transient real-time simulation becomes larger and larger, the increase in the number of subnets leads to a rapid increase in the amount of communication between subnets, and the intensive point-to-point communication directly leads to congestion in direct message transmission between subnets through CPU inter-core communication, resulting in a significant decrease in communication performance.

[0056] Optionally, the communication between the first-layer subnet processes is that the subnet processes communicate between various CPU cores inside the same physical CPU, wherein each physical CPU includes multiple CPU cores.

[0057] Optionally, the subnet message is written into the shared memory responsible for the first subnet process, and the subnet message is obtained from the shared memory responsible for the second subnet process to realize communication between the first-level subnet processes, including: using the shared memory transmission method to transmit the messages of all subnet processes inside the physical CPU, where the shared memory is a three-level cache.

[0058] Specifically, the message of all subnet processes inside the physical CPU is transmitted by the transmission method of shared memory, wherein the shared memory is a three-level cache. Since different CPU cores inside the physical CPU share the L3 level cache (Cache), the refresh speed of the cache is very fast. The present invention proposes that the message transmission method of all subnets inside the physical CPU will adopt the transmission of shared memory. During the message transmission process, the subnet process that sends the message first writes all the messages that need to be transmitted into the shared memory segment it is responsible for. At the same time, the messages transmitted to this subnet by other subnets are obtained from the shared memory segments that other subnet processes are responsible for. The read and write operations of the shared memory are all accesses to the local memory, so there is no restriction on the read and write time, and the read and write operations can occur at any time.

[0059] Optionally, the communication between the second-layer subnet processes is the communication between CPU cores of different physical CPUs in the same host by the subnet processes, wherein each host includes a plurality of different physical CPUs.

[0060] Optionally, the messages of the first subnet process are aggregated to form a data packet for sending and receiving, and the data packet is placed in a bus for communication to realize communication between the second-layer subnet processes, including: based on a transmission method of a time-sharing bus, the time length of the communication process is divided into different time slices, wherein the length of each time slice is the duration of a communication task through a certain link; according to the time slice corresponding to the data packet, a time slice that matches the time slice corresponding to the data packet is determined among all time slices in the time-sharing bus; and allocating the time slices for all information transmission of the second-layer communication structure to allocate an exclusive link for each message.

[0061] Specifically, based on the transmission method of the time-sharing bus, the time length of the communication process is divided into different time slices, where the length of each time slice is the duration of the communication task through a certain link; and the time slices for all information transmissions of the second-layer communication structure are allocated, and an exclusive link is allocated to each message.

[0062] This embodiment proposes a sequential bus message passing method based on point-to-point pairing. The basic method of message passing will be a non-blocking point-to-point communication method between processes. In terms of the execution time and coordinated control of communication tasks, a message passing method based on time-sharing bus utilization is proposed. The communication process is divided into different time slices. The length of each time slice can ensure the completion of a communication task passing through a certain link at one time. Then, all message transmissions in the second-layer communication are optimized in time slice allocation. In different time slices, a certain message will monopolize a link until the message transmission is completed. Based on the method of placing communication messages into bus communication, the communication bottleneck caused by bus congestion is avoided.

[0063] Optionally, the communication between the third-layer subnet processes is communication between CPU cores of different hosts.

[0064] Optionally, a cross-host shared memory area is established based on remote direct data access technology, and the message of the first subnet process is written into the memory area managed by the remote target subnet process through a non-blocking write mechanism. The message of the first subnet process is read by the remote target subnet process to realize communication between third-layer subnet processes, including: in terms of message transmission timing, using threads to send messages of the first subnet process; after the message of the first subnet process is sent, using threads to execute other computing tasks in electromagnetic transients, other computing tasks, communication tasks between first-layer subnet processes and communication tasks between second-layer subnet processes; and when reading the message sent by the remote target subnet process, using a query mechanism to determine whether the sent message has been completely received.

[0065] Specifically, a remote simulation message transmission method based on remote direct data access technology establishes a shared memory area across hosts through remote direct data access; utilizes a message sending side to drop a message to a memory area managed by a remote target subnet process through a non-blocking mechanism; and utilizes a message receiving side to determine whether a message is completely received through a query mechanism, and when the message is completely received, utilizes the message receiving side to read the message.

[0066] In terms of the message transmission method, the present invention proposes a remote simulation message transmission method based on RDMA technology (remote direct data access), and establishes a shared memory area across node machines through RDMA. The message sending side (is it a host or a node machine) directly throws the message into the memory area managed by the remote target subnet through a non-blocking write mechanism (such as MPI_Isend). The message receiving side uses a query mechanism to determine whether the message has been fully received (MPI_Wait), and reads the message after it is fully received.

[0067] refer to Figure 4 As shown in the figure, in a single-step iteration, the information required for the third-layer communication is prepared and sent out as early as possible. After the information is sent, the same thread turns to execute other electromagnetic transient computing tasks, first-layer communication tasks, synchronization and second-layer communication tasks, etc. When the program executes to the point where it needs to read the message sent by other subnets through the third-layer communication, the program will query whether the message inside RDMA has been received. Therefore, there is a lot of idle time (typically more than 20 microseconds) between the sending and receiving of communication messages, thereby avoiding the impact of RDMA data transmission on the real-time performance of the simulation.

[0068] In addition, reference Figure 1 As shown, according to the second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0069] Therefore, according to this embodiment, a multi-layer communication method that supports scalable real-time parallel simulation calculation of electromagnetic transients in power systems is provided to replace the original parallel dense point-to-point communication. Through the deep integration of electromagnetic transient parallel simulation tasks and the server hardware that undertakes the calculation, on the basis of the NUMA architecture of the computing hardware, the communication volume, communication mechanism and communication time of the simulation task are controlled to achieve global communication optimization configuration, thereby solving the core problem of reduced communication performance. It is achieved that as the simulation scale expands, the overall communication performance is greatly improved synchronously, and it is no longer a key bottleneck for large-scale electromagnetic transient real-time parallel simulation.

[0070] This solves the technical problem in the prior art that as the scale of electromagnetic transient real-time simulation becomes larger and larger, the increase in the number of subnets leads to a rapid increase in the amount of communication between subnets, and the intensive point-to-point communication directly leads to congestion in direct message transmission between subnets through CPU inter-core communication, resulting in a significant decrease in communication performance.

[0071] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0072] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0073] Example 2

[0074] Figure 5 A communication device 500 for electromagnetic transient real-time parallel simulation data according to this embodiment is shown. The device 500 corresponds to the method according to the first aspect of embodiment 1. Figure 5 As shown, the device 500 includes: a first communication module 510, which is used to write subnet messages into the shared memory responsible for the first subnet process, and obtain subnet messages from the shared memory responsible for the second subnet process, so as to realize communication between the first-layer subnet processes; a second communication module 520, which is used to summarize the subnet information, form a data packet for sending and receiving, put the data packet into the bus for communication, so as to realize communication between the second-layer subnet processes; and a third communication module 530, which is used to establish a cross-host shared memory area based on remote direct data access technology, write the message of the first subnet process into the memory area managed by the remote target subnet process through a non-blocking write mechanism, read the message of the first subnet process through the remote target subnet process, and realize communication between the third-layer subnet processes.

[0075] Optionally, the communication between the first-layer subnet processes is that the subnet processes communicate between various CPU cores inside the same physical CPU, wherein each physical CPU includes multiple CPU cores.

[0076] Optionally, the first communication module 510 includes: a transmission submodule, which is used to transmit messages of all subnet processes inside the physical CPU using a shared memory transmission method, wherein the shared memory is a three-level cache.

[0077] Optionally, the communication between the second-layer subnet processes is the communication between CPU cores of different physical CPUs in the same host by the subnet processes, wherein each host includes a plurality of different physical CPUs.

[0078] Optionally, the second communication module 520 includes: a division submodule, which is used to divide the time length of the communication process into different time slices based on the transmission method of the time-sharing bus, wherein the length of each time slice is the duration of the communication task through a certain link; a time slice determination submodule, which is used to determine the time slice that matches the time slice corresponding to the data packet among all the time slices in the time-sharing bus according to the time slice corresponding to the data packet; and an allocation submodule, which is used to allocate the time slices for all information transmission of the second-layer communication structure, and allocate an exclusive link for each message.

[0079] Optionally, the communication between the third-layer subnet processes is communication between CPU cores of different hosts.

[0080] Optionally, the third communication module 530 includes: a sending submodule, which is used to use threads to send messages of the first subnet process in terms of message transmission timing; an execution submodule, which is used to use threads to execute other computing tasks in electromagnetic transients after the message of the first subnet process is sent, and other computing tasks include communication tasks between first-layer subnet processes and communication tasks between second-layer subnet processes; and a reading submodule, which is used to determine whether the sent message has been completely received through a query mechanism when reading the message sent by the remote target subnet process.

[0081] Therefore, according to this embodiment, a communication device 500 for electromagnetic transient real-time parallel simulation data is provided to replace the original parallel dense point-to-point communication by providing a multi-layer communication method that supports scalable electromagnetic transient real-time parallel simulation calculations of power systems. Through the deep integration of electromagnetic transient parallel simulation tasks and the server hardware that undertakes the calculation, on the basis of the NUMA architecture of the computing hardware, the communication volume, communication mechanism and communication time of the simulation tasks are controlled to achieve global communication optimization configuration, thereby solving the core problem of reduced communication performance. It is achieved that as the simulation scale expands, the overall communication performance is greatly improved synchronously, and it no longer becomes a key bottleneck for large-scale electromagnetic transient real-time parallel simulation.

[0082] This solves the technical problem in the prior art that as the scale of electromagnetic transient real-time simulation becomes larger and larger, the increase in the number of subnets leads to a rapid increase in the amount of communication between subnets, and the intensive point-to-point communication directly leads to congestion in direct message transmission between subnets through CPU inter-core communication, resulting in a significant decrease in communication performance.

[0083] Example 3

[0084] Figure 6 A communication device 600 for electromagnetic transient real-time parallel simulation data according to this embodiment is shown. The device 600 corresponds to the method according to the first aspect of embodiment 1. Figure 6 As shown, the device 600 includes: a processor 610; and a memory 620, which is connected to the processor 610 and is used to provide the processor 610 with instructions for processing the following processing steps: writing the subnet message into the shared memory responsible for the first subnet process, and obtaining the subnet message from the shared memory responsible for the second subnet process, so as to realize communication between the first-layer subnet processes; aggregating the subnet information to form a data packet for sending and receiving, and putting the data packet into the bus for communication, so as to realize communication between the second-layer subnet processes; and establishing a cross-host shared memory area based on remote direct data access technology, and writing the message of the first subnet process into the memory area managed by the remote target subnet process through a non-blocking write mechanism, and reading the message of the first subnet process through the remote target subnet process, so as to realize communication between the third-layer subnet processes.

[0085] Optionally, the communication between the first-layer subnet processes is that the subnet processes communicate between various CPU cores inside the same physical CPU, wherein each physical CPU includes multiple CPU cores.

[0086] Optionally, the subnet message is written into the shared memory responsible for the first subnet process, and the subnet message is obtained from the shared memory responsible for the second subnet process to realize communication between the first-level subnet processes, including: using the shared memory transmission method to transmit the messages of all subnet processes inside the physical CPU, where the shared memory is a three-level cache.

[0087] Optionally, the communication between the second-layer subnet processes is the communication between CPU cores of different physical CPUs in the same host by the subnet processes, wherein each host includes a plurality of different physical CPUs.

[0088] Optionally, the messages of the first subnet process are aggregated to form a data packet for sending and receiving, and the data packet is placed in a bus for communication to realize communication between the second-layer subnet processes, including: based on a transmission method of a time-sharing bus, the time length of the communication process is divided into different time slices, wherein the length of each time slice is the duration of a communication task through a certain link; according to the time slice corresponding to the data packet, a time slice that matches the time slice corresponding to the data packet is determined among all time slices in the time-sharing bus; and allocating the time slices for all information transmission of the second-layer communication structure to allocate an exclusive link for each message.

[0089] Optionally, the communication between the third-layer subnet processes is communication between CPU cores of different hosts.

[0090] Optionally, a cross-host shared memory area is established based on remote direct data access technology, and the message of the first subnet process is written into the memory area managed by the remote target subnet process through a non-blocking write mechanism. The message of the first subnet process is read by the remote target subnet process to realize communication between third-layer subnet processes, including: in terms of message transmission timing, using threads to send messages of the first subnet process; after the message of the first subnet process is sent, using threads to execute other computing tasks in electromagnetic transients, other computing tasks, communication tasks between first-layer subnet processes and communication tasks between second-layer subnet processes; and when reading the message sent by the remote target subnet process, using a query mechanism to determine whether the sent message has been completely received.

[0091] Therefore, according to this embodiment, a communication device 600 for electromagnetic transient real-time parallel simulation data is provided to replace the original parallel dense point-to-point communication by providing a multi-layer communication method that supports scalable electromagnetic transient real-time parallel simulation calculations of power systems. Through the deep integration of electromagnetic transient parallel simulation tasks and the server hardware that undertakes the calculation, on the basis of the NUMA architecture of the computing hardware, the communication volume, communication mechanism and communication time of the simulation tasks are controlled to achieve global communication optimization configuration, thereby solving the core problem of reduced communication performance. It is achieved that as the simulation scale expands, the overall communication performance is greatly improved synchronously, and it is no longer a key bottleneck for large-scale electromagnetic transient real-time parallel simulation.

[0092] This solves the technical problem in the prior art that as the scale of electromagnetic transient real-time simulation becomes larger and larger, the increase in the number of subnets leads to a rapid increase in the amount of communication between subnets, and the intensive point-to-point communication directly leads to congestion in direct message transmission between subnets through CPU inter-core communication, resulting in a significant decrease in communication performance.

[0093] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0094] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0095] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0096] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0097] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0099] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A communication method for electromagnetic transient real-time parallel simulation data, characterized in that: include: Writing the subnet message into the shared memory of the first subnet process, and obtaining the subnet message from the shared memory of the second subnet process, so as to realize the communication between the subnet processes of the first layer; Aggregate the subnet messages to form a data packet for sending and receiving, put the data packet into the bus for communication, and realize communication between the second-layer subnet processes; as well as Based on remote direct data access technology, a cross-host shared memory area is established, and a message of the first subnet process is written into the shared memory area managed by the remote target subnet process through a non-blocking write mechanism, and the message of the first subnet process is read by the remote target subnet process to realize communication between the third-layer subnet processes; The first layer of communication between subnet processes is the communication between the CPU cores in the same physical CPU, where each physical CPU contains multiple CPU cores. The subnet message is written into the shared memory of the first subnet process, and the subnet message is obtained from the shared memory of the second subnet process, so as to realize the communication between the subnet processes of the first layer, including: using the transmission method of the shared memory to transmit the messages of all the subnet processes in the physical CPU, wherein the shared memory is a three-level cache; The second layer of communication between subnet processes is the communication between CPU cores of different physical CPUs in the same host, where each host contains multiple different physical CPUs; Aggregate the messages of the first subnet process to form a data packet for sending and receiving, put the data packet into the bus for communication, and realize the communication between the second-layer subnet processes, including: based on the transmission method of the time-sharing bus, divide the time length of the communication process into different time slices, wherein the length of each time slice is the length of the communication task through a certain link; according to the time slice corresponding to the data packet, determine the time slice matching the time slice corresponding to the data packet in all the time slices in the time-sharing bus; and allocate the time slices of all information transmission of the second-layer communication structure, and allocate an exclusive link for each message; Based on the transmission method of time-sharing bus, the time length of the communication process is divided into different time slices, where the length of each time slice is the length of the communication task through a certain link; and the time slices of all information transmissions of the second-layer communication structure are allocated, and an exclusive link is allocated to each message; The communication between the third-layer subnet processes is the communication between the CPU cores of different hosts; Based on the remote direct data access technology, a shared memory area across hosts is established. Through a non-blocking write mechanism, the message of the first subnet process is written into the memory area managed by the remote target subnet process. The message of the first subnet process is read through the remote target subnet process to realize the communication between the third-layer subnet processes, including: in terms of message transmission timing, the message of the first subnet process is sent by a thread; after the message of the first subnet process is sent, other computing tasks in electromagnetic transients are executed by a thread, other computing tasks are communication tasks between the first-layer subnet processes and the second-layer subnet processes; and when reading the message sent by the remote target subnet process, it is determined through a query mechanism whether the sent message has been completely received.

2. A storage medium, characterized in that: The storage medium stores a program, wherein when the program is executed, the processor executes the method according to claim 1.

3. A communication device for real-time parallel simulation data of electromagnetic transient, characterized in that: include: The first communication module is used to write the subnet message into the shared memory responsible for the first subnet process, and obtain the subnet message from the shared memory responsible for the second subnet process, so as to realize the communication between the first layer subnet processes; The second communication module is used to aggregate the subnet messages, form a data packet for sending and receiving, put the data packet into the bus for communication, and realize the communication between the second layer subnet processes; as well as The third communication module is used to establish a cross-host shared memory area based on remote direct data access technology, write the message of the first subnet process into the shared memory area managed by the remote target subnet process through a non-blocking write mechanism, and read the message of the first subnet process through the remote target subnet process to realize communication between the third-layer subnet processes; The first layer of communication between subnet processes is the communication between the CPU cores in the same physical CPU, where each physical CPU contains multiple CPU cores. The subnet message is written into the shared memory of the first subnet process, and the subnet message is obtained from the shared memory of the second subnet process, so as to realize the communication between the subnet processes of the first layer, including: using the transmission method of the shared memory to transmit the messages of all the subnet processes in the physical CPU, wherein the shared memory is a three-level cache; The second layer of communication between subnet processes is the communication between CPU cores of different physical CPUs in the same host, where each host contains multiple different physical CPUs; Aggregate the messages of the first subnet process to form a data packet for sending and receiving, put the data packet into the bus for communication, and realize the communication between the second-layer subnet processes, including: based on the transmission method of the time-sharing bus, divide the time length of the communication process into different time slices, wherein the length of each time slice is the length of the communication task through a certain link; according to the time slice corresponding to the data packet, determine the time slice matching the time slice corresponding to the data packet in all the time slices in the time-sharing bus; and allocate the time slices of all information transmission of the second-layer communication structure, and allocate an exclusive link for each message; Based on the transmission method of time-sharing bus, the time length of the communication process is divided into different time slices, where the length of each time slice is the length of the communication task through a certain link; and the time slices of all information transmissions of the second-layer communication structure are allocated, and an exclusive link is allocated to each message; The communication between the third-layer subnet processes is the communication between the CPU cores of different hosts; Based on the remote direct data access technology, a shared memory area across hosts is established. Through a non-blocking write mechanism, the message of the first subnet process is written into the memory area managed by the remote target subnet process. The message of the first subnet process is read through the remote target subnet process to realize the communication between the third-layer subnet processes, including: in terms of message transmission timing, the message of the first subnet process is sent by a thread; after the message of the first subnet process is sent, other computing tasks in electromagnetic transients are executed by a thread, other computing tasks are communication tasks between the first-layer subnet processes and the second-layer subnet processes; and when reading the message sent by the remote target subnet process, it is determined through a query mechanism whether the sent message has been completely received.

4. A communication device for real-time parallel simulation data of electromagnetic transient, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Write the subnet message into the shared memory of the first subnet process, and obtain the subnet message from the shared memory of the second subnet process, so as to realize the communication between the subnet processes of the first layer; Aggregate the subnet messages to form a data packet for sending and receiving, put the data packet into the bus for communication, and realize communication between the second-layer subnet processes; as well as Based on remote direct data access technology, a cross-host shared memory area is established, and a message of the first subnet process is written into the shared memory area managed by the remote target subnet process through a non-blocking write mechanism, and the message of the first subnet process is read by the remote target subnet process to realize communication between the third-layer subnet processes; The first layer of communication between subnet processes is the communication between the CPU cores in the same physical CPU, where each physical CPU contains multiple CPU cores. The subnet message is written into the shared memory of the first subnet process, and the subnet message is obtained from the shared memory of the second subnet process, so as to realize the communication between the subnet processes of the first layer, including: using the transmission method of the shared memory to transmit the messages of all the subnet processes in the physical CPU, wherein the shared memory is a three-level cache; The second layer of communication between subnet processes is the communication between CPU cores of different physical CPUs in the same host, where each host contains multiple different physical CPUs; Aggregate the messages of the first subnet process to form a data packet for sending and receiving, put the data packet into the bus for communication, and realize the communication between the second-layer subnet processes, including: based on the transmission method of the time-sharing bus, divide the time length of the communication process into different time slices, wherein the length of each time slice is the length of the communication task through a certain link; according to the time slice corresponding to the data packet, determine the time slice matching the time slice corresponding to the data packet in all the time slices in the time-sharing bus; and allocate the time slices of all information transmission of the second-layer communication structure, and allocate an exclusive link for each message; Based on the transmission method of time-sharing bus, the time length of the communication process is divided into different time slices, where the length of each time slice is the length of the communication task through a certain link; and the time slices of all information transmissions of the second-layer communication structure are allocated, and an exclusive link is allocated to each message; The communication between the third-layer subnet processes is the communication between the CPU cores of different hosts; Based on the remote direct data access technology, a shared memory area across hosts is established. Through a non-blocking write mechanism, the message of the first subnet process is written into the memory area managed by the remote target subnet process. The message of the first subnet process is read through the remote target subnet process to realize the communication between the third-layer subnet processes, including: in terms of message transmission timing, the message of the first subnet process is sent by a thread; after the message of the first subnet process is sent, other computing tasks in electromagnetic transients are executed by a thread, other computing tasks are communication tasks between the first-layer subnet processes and the second-layer subnet processes; and when reading the message sent by the remote target subnet process, it is determined through a query mechanism whether the sent message has been completely received.

Images