Star-shaped and tree-shaped hybrid measurement and control architecture and real-time synchronization method for array systems
By introducing a star-shaped and tree-shaped hybrid measurement and control architecture of array system in electromagnetic catapult system, combining Ethernet and FPGA high-speed transceiver, the real-time performance and real-time performance of star-shaped and tree-shaped networking in multi-node systems in the prior art are solved. The problem of insufficient synchronization achieves efficient and reliable measurement and control communication.
Patent Information
- Application Number
- CN202210567325.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-23
- Publication Date
- 2025-02-14
- Estimated Expiration
- 2042-05-23
AI Technical Summary
In the prior art, a simple star network has a large number of measured and controlled nodes and has real-time and synchronous measurement and control requirements, while a simple tree network is difficult to develop and maintain, and it is impossible to take into account synchronization. and real-time, reduce development difficulty and improve communication reliability.
A star-shaped and tree-shaped hybrid measurement and control architecture of array system is proposed, including a primary-level measurement and control server, multiple sets of secondary-level measurement and control servers and node groups under measurement, as well as a central switch and fiber optic data synchronization network, which can transmit and receive high-speed via Ethernet and FPGA The device forms a hybrid network to realize the coordinated work of star and tree networks.
It realizes the consideration of synchronization and real-time in a multi-node system, reduces the difficulty of software development, improves communication reliability, and improves the fault tolerance and efficiency of the system.
Smart Images

Figure CN115102652B_ABST
Abstract
Description
Background Art
[0002] The electromagnetic catapult system involves the coordinated work of the power generation subsystem, energy storage subsystem, discharge subsystem and multiple support modules. The power generation, energy storage and discharge subsystems are all array systems, which can achieve instantaneous discharge of high voltage and high current through series and parallel connection. How to coordinate the management of the power generation subsystem, energy storage subsystem, discharge subsystem and multiple support modules is the key to the normal operation of the electromagnetic catapult system.
[0003] Point-to-point or one-to-many measurement and control systems can be networked based on serial buses such as CAN, Ethernet, and serial ports to achieve a simple, efficient, and low-cost measurement and control network. However, when the number of nodes to be measured and controlled exceeds dozens or hundreds, factors such as network structure, development difficulty, and maintenance cost must be considered at the same time, and real-time and synchronization must also be taken into account. First of all, the electromagnetic catapult system belongs to the power control system, which has requirements for real-time control. Taking the generator as an example, the generator controller uses 100us as a control cycle, that is, the parameters in the system, such as current, voltage, speed, torque, etc., are updated in a cycle of 100us (100us is an example). The synchronous controller in the measurement and control network needs to receive the parameters uploaded by all generator controllers in the system in real time, that is, once the generator controller has data updated, the synchronous controller is required to obtain the updated data with the shortest time (need to be less than the motor controller operation cycle 100us) to achieve real-time control of the controlled motor controller by the synchronous controller. Secondly, the electromagnetic catapult system is an array-type power control system, which also requires synchronization of control. For example, if the delay time of uploading data from each generator controller to the core machine is different, or the delay time of the command issued by the core machine to each generator controller is different, it will seriously affect the synchronization of the generator controller array. In a network composed of Ethernet, CAN, serial port and other buses, there can only be one measured and controlled node data on the bus link at the same time. If there are 50 measured and controlled nodes, and the time for each measurement and control node to interact with data is 20us, the data uploaded by the core machine to node 1 and node 50 will differ by 1ms, and the opposite data direction will also differ by 1ms. This has a serious impact on the real-time and synchronization of the measurement and control system. The commonly used timestamp-based data synchronization method is not real-time and cannot meet the requirements of this system.
[0004] As a commonly used communication interface, Ethernet is easy to expand, low cost, fast communication rate, high stability and reliability. The security modules in the system, such as oscilloscopes, oscilloscope recorders, power analyzers, multi-channel data acquisition instruments, four-quadrant rectifiers, PLCs, etc. are equipped with Ethernet interfaces. Relying on Ethernet and using multi-port switches, the electromagnetic catapult system can be star-networked, and the network structure is simple. As a mature product, multi-port switches are easily available on the market. The characteristic of star networking is that the communication between any two terminal nodes needs to be forwarded through the central node, so when the number of terminal nodes is large, real-time issues need to be considered.
[0005] In the electromagnetic catapult system, it is very important to control the synchronization of energy flow in each generator in the power generation subsystem and each discharge path in the discharge subsystem. To achieve high-precision synchronization, the parallel execution characteristics of FPGA can be used to expand multiple communication buses to communicate synchronously with multiple nodes under test and control. Since one FPGA cannot expand enough communication buses, a tree network based on FPGA high-speed transceiver is established based on the above ideas. The communication bus used in the tree network based on FPGA high-speed transceiver is a non-standard self-protocol bus, but if all devices in the system are connected to the tree network, the development difficulty and maintenance cost are relatively high.
[0006] In general, when the number of terminal nodes is large, the real-time performance of a simple star network is low, while the development difficulty and maintenance cost of a simple tree network are high, and its application has great limitations. Summary of the invention
[0007] In order to solve the above-mentioned problems in the prior art, namely, the simple star-shaped networking and tree-shaped networking are limited in application when there are a large number of nodes to be measured and controlled and there are real-time and synchronous measurement and control requirements, and it is impossible to take into account synchronization, real-time performance, and reduce development difficulty and improve communication reliability. The present invention provides a star-shaped and tree-shaped hybrid measurement and control architecture of an array system, wherein the star-shaped and tree-shaped hybrid measurement and control architecture includes a set of primary measurement and control servers, N sets of secondary measurement and control servers, N groups of measured and controlled nodes corresponding to the N sets of secondary measurement and control servers, and a central switch;
[0008] The primary measurement and control server, the N sets of secondary measurement and control servers and the N groups of measured and controlled nodes serve as terminal nodes of a star network, the central switch serves as a central node of the star network, and the terminal nodes are connected to the central node via Ethernet to form a star network based on Ethernet;
[0009] The first-level measurement and control server serves as the root of the tree network, the N sets of second-level measurement and control servers serve as sub-tree roots of the tree network, and the system function nodes in the N measured and controlled node groups serve as leaf nodes of the tree network. The leaf nodes are connected to the corresponding sub-tree roots through the optical fiber data synchronization network, and the sub-tree roots are connected to the tree roots through the optical fiber data synchronization network, thereby forming a tree network of the optical fiber data synchronization network based on FPGA high-speed transceiver.
[0010] In some preferred embodiments, the first-level measurement and control server is further provided with a first-level monitoring PC terminal;
[0011] The first-level monitoring PC terminal is used for human-computer interaction and data management of the first-level measurement and control server.
[0012] In some preferred embodiments, each of the N sets of secondary measurement and control servers is further provided with a secondary monitoring PC terminal;
[0013] The secondary monitoring PC terminal is used for human-computer interaction and data management of the secondary measurement and control server.
[0014] In some preferred embodiments, the primary measurement and control server includes a primary synchronization controller and a primary measurement and control core machine, and the secondary measurement and control server includes a secondary synchronization controller and a secondary measurement and control core machine;
[0015] The first-level synchronization controller and the second-level synchronization controller are both multi-channel optical communication controllers based on FPGA master control, and are externally provided with multiple optical communication interfaces based on FPGA high-speed transceivers to form the optical fiber data synchronization network;
[0016] The primary measurement and control core machine and the secondary measurement and control core machine are both industrial computers based on an embedded real-time operating system, and are externally provided with a Gigabit Ethernet interface and an optical communication interface based on a high-speed transceiver of an FPGA;
[0017] The first-level measurement and control core machine and the second-level measurement and control core machine realize a 1-way optical communication interface based on FPGA high-speed transceiver by extending the FPGA board.
[0018] In some preferred embodiments, the central switch is a multi-port central switch;
[0019] The multi-port central switch is used for data forwarding and is connected to the multiple sets of secondary measurement and control servers via Ethernet.
[0020] In some preferred embodiments, the measured and controlled node group includes P measured and controlled nodes; the P measured and controlled nodes include M system function nodes and Q system guarantee nodes, P = M + Q;
[0021] The measured and controlled node is connected to the central node of the multi-port central switch via a network cable, and a local area network is formed through the multi-port central switch.
[0022] All nodes under measurement and control (including non-standard customized equipment such as generators and controllers, energy storage devices, pulse discharge controllers and other system function nodes, as well as standard shelf products such as oscilloscopes, oscilloscope recorders, power analyzers, multi-channel data acquisition instruments, four-quadrant rectifiers, PLCs and other system support nodes), a set of first-level measurement and control servers, and multiple sets of second-level measurement and control servers as terminal nodes of the star network architecture are connected to the central node, i.e. the central switch, through Ethernet to form a star network.
[0023] The star network consists of a central node and terminal nodes. The central node is a multi-port central switch. The terminal nodes include all the nodes to be measured and controlled (i.e., the group of nodes to be measured and controlled includes system function nodes and system guarantee nodes), a first-level measurement and control core machine in a first-level measurement and control server, and the second-level measurement and control core machines in all second-level measurement and control servers. All terminal nodes are configured with different IP addresses. All the nodes to be measured and controlled are connected to the central node through network cables, and form a local area network through the central switch. The data interaction between any two terminal nodes needs to be cached and forwarded by the central node, so the hardware networking has the significant characteristics of the star network topology; in the planning and design of data interaction, it can also have the characteristics of tree-like structure hierarchy. The first-level measurement and control server directly controls all the second-level measurement and control servers, that is, in software design, the first-level measurement and control server only allows all the second-level measurement and control servers to establish connections with it, and does not allow the measured and controlled nodes to directly establish connections with the first-level measurement and control server. The software development difficulty of the first-level measurement and control server is simplified in the above manner; the measured and controlled nodes are grouped, assuming that each P measured and controlled nodes (including M system function nodes and Q system security nodes) are divided into a group (i.e., measured and controlled node group), which is divided into N groups in total, and each group of measured and controlled nodes is equipped with a set of second-level measurement and control servers. The above-mentioned P measured and controlled nodes and a set of second-level measurement and control servers constitute a minimum measurement and control unit, and the second-level measurement and control server manages all the measured and controlled nodes in the group.
[0024] In some preferred embodiments, the primary synchronization controller is connected to multiple secondary synchronization controllers via the optical fiber data synchronization network.
[0025] FPGA-based high-speed transceivers participate in the construction of a two-level measurement and control server, with one set of first-level measurement and control servers as the tree root and multiple sets of second-level measurement and control servers as sub-tree roots; all system function nodes are connected to the second-level measurement and control server as leaf nodes, so that the first-level measurement and control server, the second-level measurement and control server, and the system function nodes form a tree network.
[0026] In some preferred embodiments, the optical fiber data synchronization network is networked in the following manner:
[0027] The FPGA high-speed transceiver is configured with a full-duplex optical communication interface;
[0028] The number of the full-duplex optical communication interfaces is determined according to the number N of the measured and controlled node groups and the number M of the system function nodes in each measured and controlled node group:
[0029] The FPGA high-speed transceiver in the first-level measurement and control server is configured with an N+1-channel full-duplex optical communication interface;
[0030] The FPGA high-speed transceiver in the secondary measurement and control server is configured with M+2 full-duplex optical communication interfaces;
[0031] N secondary synchronous controllers occupy N optical communication interfaces of the primary synchronous controller among the N+1 full-duplex optical communication interfaces, and the remaining one full-duplex optical communication interface is connected to the optical communication interface of the primary measurement and control core computer through optical fiber;
[0032] The M system function nodes in the same controlled node group are connected to the M optical communication interfaces in the secondary synchronization controller of the secondary measurement and control server corresponding to the controlled node group through M of the M+2 full-duplex optical communication interfaces, and of the remaining two full-duplex optical communication interfaces, one is connected to the optical communication interface in the primary synchronization controller of the primary measurement and control server through optical fiber, and the other is connected to the secondary measurement and control core machine of the secondary measurement and control server.
[0033] In some preferred embodiments, the hybrid measurement and control architecture synchronously sends control instructions to M×N measured and controlled system functional nodes through the first-level synchronization controller of the first-level measurement and control server, and synchronously receives feedback data uploaded by the M×N measured and controlled system functional nodes.
[0034] The star network and the tree network are independent of each other in function, but also back up and complement each other; they are also interconnected in hardware connection. The system function nodes are both terminal nodes of the star network and leaf nodes of the tree network.
[0035] The tree network designed based on FPGA high-speed transceiver consists of a tree root, several sub-tree roots and a large number of leaf nodes. The first-level measurement and control server is the tree root; multiple second-level measurement and control servers are sub-tree roots; all the nodes under measurement and control are leaf nodes. The nodes under measurement and control are grouped in the same way as the grouping of the nodes under measurement and control in the star network. The first-level synchronous controller and the second-level synchronous controller are designed based on FPGA. The FPGA high-speed transceiver is configured as a full-duplex optical communication interface. The number of optical communication interfaces that need to be configured is determined by the number of system function nodes M in each minimum measurement and control unit or the number of minimum measurement and control unit groups N. At the same time, the maximum number of optical communication interfaces that can be configured by FPGA is limited by the selected FPGA hardware resources. Based on the characteristics of FPGA parallel execution, the configured multiple optical communication interfaces are independent of each other and can perform optical communication synchronously. The M+2 high-speed transceivers of the FPGA in the second-level synchronous controller are configured as optical communication interfaces. The N+1 high-speed transceivers of the FPGA in the first-level synchronous controller are configured as optical communication interfaces. The M measured and controlled system function nodes in the same minimum measurement and control unit are connected to the M optical communication interfaces in the secondary synchronization controller in the secondary measurement and control server in the minimum measurement and control unit through optical communication interfaces through optical fibers; the remaining two optical communication interfaces in the secondary synchronization controller, one is connected to the optical communication interface of the first-level synchronization controller in the first-level measurement and control server through optical fibers, and the other is connected to the second-level measurement and control core machine in the second-level measurement and control server. N second-level synchronization controllers occupy a total of N optical communication interfaces of the first-level synchronization controllers, and the remaining one optical communication interface in the first-level synchronization controller is connected to the optical communication interface of the first-level measurement and control core machine through optical fibers. In this way, a tree network based on the design of FPGA high-speed transceivers is formed. Since all optical communication interfaces are designed by FPGA high-speed transceivers, based on the parallelization of FPGA and the full-duplex characteristics of high-speed transceivers, the first-level synchronization controller in the first-level measurement and control server can synchronously send control instructions to M*N measured and controlled system function nodes; synchronously receive feedback data uploaded by M*N measured and controlled system function nodes; and data sending and uploading have the ability to be performed simultaneously.
[0036] Another aspect of the present invention provides a real-time synchronization method for a star-shaped and tree-shaped hybrid measurement and control architecture of an array system. The real-time synchronization method is:
[0037] The first-level synchronous controller sends the same data frame to all the second-level synchronous controllers at a set clock cycle; the clock cycle is the same as the control cycle of the leaf nodes of the tree network, generally tens of microseconds per beat; the data frame contains information selected by the first-level synchronous controller to identify a specific leaf node.
[0038] The data frame sent down is the instruction data or invalid bytes that need to be sent down from the measurement and control core machine to a leaf node. When the measurement and control core machine needs to operate a leaf node, the data frame sent down by the first-level synchronization controller is the instruction data; when the measurement and control core machine has no operation on the leaf node, the data sent down by the first-level synchronization controller is an invalid byte.
[0039] All secondary synchronization controllers operate in the period of receiving data frames sent by the primary synchronization controller. After receiving the data, the secondary synchronization controller forwards it to all leaf nodes. All leaf nodes regard the moment of receiving the data frame sent by the primary synchronization controller as the beginning of a control beat.
[0040] All secondary synchronous controllers have the same software and hardware design. The delay error of all secondary synchronous controllers receiving the same data frame from the primary synchronous controller is about ten nanoseconds. This error is mainly introduced by the communication delay difference caused by the optical fiber from the primary synchronous controller to all secondary synchronous controllers. The delay error of all leaf nodes receiving the same data frame from the secondary synchronous controller is about ten nanoseconds. This error is mainly introduced by the communication delay difference caused by the optical fiber from the secondary synchronous controller to all leaf nodes. Therefore, the time error of the current beat start time of all leaf nodes is tens of nanoseconds.
[0041] The target leaf node selected by the first-level synchronization controller responds to the received data frame, and the remaining non-target leaf nodes do not respond to the received data frame.
[0042] The target leaf node receives and caches instruction data in the current control cycle, and participates in the control algorithm when the next control cycle runs. In each control cycle of the leaf node, all leaf nodes update variables; the variables include the measurement results of all sensors in the node and the calculation results of the control algorithm participated by the measurement results and the instruction data issued by the first-level synchronous controller.
[0043] After the target leaf node completes the control algorithm, the sensor measurement results and the control algorithm calculation results are uploaded to the secondary synchronous controller before the end of the current control cycle.
[0044] After receiving the data uploaded by the leaf nodes in the group, the secondary synchronization controller packages and forwards it to the primary synchronization controller.
[0045] The above process should meet the following conditions: the communication delay time from the first-level synchronous controller sending data frames to the leaf node plus the running time of the leaf node control algorithm plus the communication delay time from the leaf node uploading the updated variables to the first-level synchronous controller is less than the time of one control beat. In the above manner, all leaf nodes run at the same beat, the target leaf node receives the instruction data sent by the first-level synchronous controller in the current beat, and participates in the control algorithm in the next control beat cycle, and the first-level synchronous controller receives the update data of the leaf node in the current beat.
[0046] The functional nodes of the measured and controlled system have at least two external communication buses, one is Ethernet, which is connected to the central switch and participates in the formation of a star network; the other is an optical communication interface designed based on FPGA high-speed transceiver, which is connected to the matching secondary synchronization controller and participates in the formation of a tree network.
[0047] The two communication buses are completely independent and do not affect each other. These nodes can generate control beats by the local clock of the node and by the secondary synchronization controller.
[0048] In terms of software function planning, the star network and the tree network are both independent of each other and can back up and complement each other: the tree network designed based on FPGA high-speed transceiver has the ability to quickly and in parallel upload data from multiple leaf nodes to the root of the tree, and the instructions of the root of the tree are synchronously sent to all leaf nodes with low latency error. The measurement and control commands are sent and key state variables are uploaded, and they are transmitted to the measurement and control server through tree network interaction. The core machine in the server gives priority to processing data from the synchronous controller to ensure the real-time performance of the measurement and control system. All data in the interactive measurement and control system in the Ethernet-based star network are used to realize functions with low real-time requirements, such as data storage, node network timing, node parameter adjustment software upgrade, etc. Ethernet, as a conventional communication interface, is a necessary communication interface for PLC, test instruments, dynamometers, and large power supply cabinets. Through the Ethernet-based star network structure, it is very easy to connect the above-mentioned equipment to the measurement and control system to realize the status configuration and remote monitoring of the field system guarantee nodes.
[0049] The data of the star network has a larger delay than that of the tree network. The data of the star network is used to perform delay verification on the core data of the tree network interaction, which is used to conduct a health assessment of the communication network of the entire measurement and control system. Once the tree network fails, the star network can also realize the measurement and control function of the system, but the system delay is greater than when the tree network participates in the measurement and control.
[0050] A third aspect of the present invention provides a device, comprising:
[0051] at least one processor; and
[0052] a memory communicatively connected to at least one of the processors; wherein,
[0053] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the real-time synchronization method of the star-shaped and tree-shaped hybrid measurement and control architecture of the array system.
[0054] In a fourth aspect of the present invention, a computer-readable storage medium is proposed, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement a real-time synchronization method of the star-shaped and tree-shaped hybrid measurement and control architecture of the above-mentioned array system.
[0055] Beneficial effects of the present invention:
[0056] (1) The star-shaped and tree-shaped hybrid measurement and control architecture of the array system of the present invention has the function nodes of the measurement and control system connected to both the star network and the tree network. The dual network attribute of the node enables it to have the verification capability, thereby making the system more fault-tolerant.
[0057] (2) The star-shaped and tree-shaped hybrid measurement and control architecture of the array system of the present invention only performs tree-shaped networking on the nodes to be measured and controlled that have real-time and synchronization requirements, thereby reducing the number of tree-shaped nodes in the network and lowering the development cost.
[0058] (3) The star-shaped and tree-shaped hybrid measurement and control architecture of the array system of the present invention performs tree-shaped hierarchical networking management on the terminal nodes in the star-shaped network in software planning, which simplifies the measurement and control software structure design and reduces the development difficulty.
[0059] (4) The star-shaped and tree-shaped hybrid measurement and control architecture of the array system of the present invention differentiates the functions of star networking and tree networking with emphasis. The tree networking realizes synchronous measurement and control, while the star networking realizes functions with low real-time requirements such as data storage, node network timing, node parameter adjustment and software upgrade. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:
[0061] Figure 1 These are two common multi-node network structure diagrams in measurement and control networks;
[0062] Figure 2 It is a network diagram of a hybrid measurement and control architecture of an electromagnetic catapult system of an embodiment of a star-shaped and tree-shaped hybrid measurement and control architecture of an array system of the present invention. DETAILED DESCRIPTION
[0063] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.
[0064] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0065] like Figure 1 As shown in the figure, there are two common multi-node network structure diagrams in the measurement and control network. Figure 1 The left picture is a tree network. Figure 1 The right picture is a star network. When the number of terminal nodes is large, the tree network has the problems of difficulty in development and high maintenance cost, and the star network has the problem of low real-time networking. In order to solve the above problems, the present invention provides a star and tree hybrid measurement and control architecture of an array system, which overcomes the limitations of star networking and tree networking in the case of a large number of nodes to be measured and controlled and with real-time and synchronous measurement and control requirements (such as in the electromagnetic catapult system), and mixes star networking and tree networking to realize the measurement and control networking requirements of a multi-node system, while taking into account synchronization and real-time performance, reducing the difficulty of software development and improving communication reliability.
[0066] A star-shaped and tree-shaped hybrid measurement and control architecture of an array system of the present invention, the star-shaped and tree-shaped hybrid measurement and control architecture comprises a set of primary measurement and control servers, N sets of secondary measurement and control servers, N groups of measured and controlled nodes corresponding one to one with the N sets of secondary measurement and control servers, and a central switch;
[0067] The primary measurement and control server, the N sets of secondary measurement and control servers and the N groups of measured and controlled nodes serve as terminal nodes of a star network, the central switch serves as a central node of the star network, and the terminal nodes are connected to the central node via Ethernet to form a star network based on Ethernet;
[0068] The first-level measurement and control server serves as the root of the tree network, the N sets of second-level measurement and control servers serve as sub-tree roots of the tree network, and the system function nodes in the N measured and controlled node groups serve as leaf nodes of the tree network. The leaf nodes are connected to the corresponding sub-tree roots through the optical fiber data synchronization network, and the sub-tree roots are connected to the tree roots through the optical fiber data synchronization network, thereby forming a tree network of the optical fiber data synchronization network based on FPGA high-speed transceiver.
[0069] In order to more clearly illustrate the star-shaped and tree-shaped hybrid measurement and control architecture of the array system of the present invention, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0070] The star-shaped and tree-shaped hybrid measurement and control architecture of the array system of the first embodiment of the present invention includes a set of primary measurement and control servers, N sets of secondary measurement and control servers, N groups of measured and controlled nodes corresponding to the N sets of secondary measurement and control servers, and a central switch, which is described in detail as follows:
[0071] like Figure 2 As shown, it is a schematic diagram of the network of the hybrid measurement and control architecture of the electromagnetic catapult system of an embodiment of the star-shaped and tree-shaped hybrid measurement and control architecture of the array system of the present invention. There are 96 nodes to be measured and controlled. Figure 2 The grouping is shown in Figure 1, which is divided into 8 minimum measurement and control units, each of which contains 12 nodes to be measured and controlled. Taking the minimum measurement and control unit 1 as an example, nodes 1 to 6 are system function nodes, including 2 sets of power generation subsystem nodes, 2 sets of energy storage subsystem nodes and 2 sets of discharge subsystem nodes; nodes 7 to 12 are system guarantee nodes, including digital acquisition, oscilloscope, oscilloscope recorder, PLC, power analyzer and four-quadrant rectifier. The nodes to be measured and controlled in the minimum measurement and control unit 2 to the minimum measurement and control unit 8 are the same as those in the minimum measurement and control unit 1.
[0072] Among the above-mentioned nodes to be measured and controlled, the power generation subsystem nodes, energy storage subsystem nodes, and discharge subsystem nodes need to realize synchronous pulse discharge control of instantaneous large current and high voltage, and have high real-time and synchronous measurement and control requirements. As auxiliary nodes for power supply and testing of the system, the system guarantee nodes only need to realize functions such as configuration of working modes and display of measurement data, and have relatively low real-time and synchronous measurement and control requirements.
[0073] Therefore, in this embodiment, Gigabit Ethernet and optical communication interfaces based on FPGA high-speed transceivers are configured for the power generation subsystem nodes, energy storage subsystem nodes, and discharge subsystem nodes. The two communication buses are completely independent and do not affect each other. These nodes can generate control beats by the local clock of the node, and can also generate control beats by the secondary synchronization controller. At the same time, in order to improve the anti-interference ability of the Gigabit Ethernet of the above three types of nodes, photoelectric conversion is performed on them, and the electrical interface of the Gigabit Ethernet is converted into an optical interface. The system security nodes such as digital acquisition, oscilloscopes, oscilloscope recorders, PLCs, power analyzers, four-quadrant rectifiers, etc. are shelf products (shelf products represent general standard products, not customized products), and are equipped with Gigabit Ethernet.
[0074] In this embodiment, the first-level measurement and control server, 8 sets of second-level measurement and control servers and 8 groups of measured and controlled nodes serve as terminal nodes of the star network, the central switch serves as the central node of the star network, and the terminal nodes are connected to the central node via Ethernet to form a star network based on Ethernet.
[0075] The first-level measurement and control server includes 1 first-level measurement and control core machine and 1 first-level synchronization controller. The first-level measurement and control core machine is equipped with 2 Gigabit Ethernet (including 1 optical interface Gigabit Ethernet and 1 electrical interface Gigabit Ethernet) and 1 optical communication interface based on FPGA high-speed transceiver. The first-level synchronization controller is equipped with 9 optical communication interfaces based on FPGA high-speed transceivers to form a fiber-optic data synchronization network.
[0076] The first-level measurement and control server serves as the root of the tree network, 8 sets of second-level measurement and control servers serve as sub-roots of the tree network, and the system function nodes in the 8 measured and controlled node groups serve as leaf nodes of the tree network. The leaf nodes are connected to the corresponding sub-roots through the optical fiber data synchronization network, and the sub-roots are connected to the tree roots through the optical fiber data synchronization network, forming a tree network of the optical fiber data synchronization network based on FPGA high-speed transceiver.
[0077] 12 nodes to be measured and controlled and a set of secondary measurement and control servers form a minimum measurement and control unit. Each of the 8 minimum measurement and control units contains a set of secondary measurement and control servers. Each set of secondary measurement and control servers includes a secondary measurement and control core machine and a secondary synchronization controller. Each secondary measurement and control core machine is equipped with 2 Gigabit Ethernet (including 1 optical interface Gigabit Ethernet and 1 electrical interface Gigabit Ethernet) and 1 optical communication interface based on FPGA high-speed transceiver design. Each secondary synchronization controller is equipped with 8 optical communication interfaces based on FPGA high-speed transceivers.
[0078] The optical communication interface based on FPGA high-speed transceiver design on the first-level measurement and control core machine and the second-level measurement and control core machine is realized by expanding the FPGA board card, which has the function of realizing the mutual conversion between PXIE and high-speed transceiver.
[0079] One first-level measurement and control core machine of the star network is equipped with one first-level monitoring PC terminal, and each of the eight second-level measurement and control core machines is equipped with one second-level monitoring PC terminal to achieve human-computer interaction and data management.
[0080] The first-level monitoring PC terminal connected to the first-level measurement and control core machine has a two-way data interaction function, including the display of the measured and controlled node data, the issuance of control instructions for the power generation subsystem node, energy storage subsystem node, and discharge subsystem node, and the browsing and management of stored data.
[0081] The secondary monitoring PC connected to the secondary measurement and control core machine has a two-way data interaction function, including the display of the measured and controlled node data in the corresponding minimum measurement and control unit, the browsing and management of the stored data, and the configuration of the system guarantee node.
[0082] All measurement and control core machines (1 first-level measurement and control core machine and 8 second-level measurement and control core machines) run an embedded real-time operating system based on VxWorks, and all monitoring PC terminals (1 first-level monitoring PC terminal and 8 second-level monitoring PC terminals) run an operating system based on Windows.
[0083] The star and tree hybrid measurement and control architecture also includes a multi-port central switch. In this embodiment, the multi-port central switch includes at least 57 Gigabit Ethernet optical interfaces and 48 Gigabit Ethernet electrical interfaces. The ports of the multi-port central switch serve as central nodes of the first-level measurement and control servers, and are connected to multiple sets of second-level measurement and control servers as terminal nodes of the star network through Ethernet.
[0084] In this embodiment, the secondary measurement and control server includes 12 measured and controlled nodes (that is, one measured and controlled node group includes 12 measured and controlled nodes), of which 6 measured and controlled nodes are system function nodes, and 6 measured and controlled nodes are system security nodes. The measured and controlled nodes are connected to the central node of the multi-port central switch through a network cable, and a local area network is formed through the multi-port central switch.
[0085] In this embodiment, one set of primary synchronization controller is connected to eight sets of secondary synchronization controllers via a data synchronization network.
[0086] Optical fiber data synchronization network, its networking mode is:
[0087] The FPGA high-speed transceiver is configured with a full-duplex optical communication interface. The number of the full-duplex optical communication interfaces is determined according to the number N of the measured and controlled node groups and the number M of the system function nodes in each measured and controlled node group:
[0088] The FPGA high-speed transceiver in the first-level measurement and control server is configured with 9 full-duplex optical communication interfaces, and the FPGA high-speed transceiver in the second-level measurement and control server is configured with 8 full-duplex optical communication interfaces. The 8 second-level synchronous controllers occupy 8 of the 9 full-duplex optical communication interfaces of the first-level synchronous controller, and the remaining one full-duplex optical communication interface is connected to the optical communication interface of the first-level measurement and control core machine through optical fiber; the 6 system function nodes in the same measured and controlled node group are connected to the 6 optical communication interfaces in the second-level synchronous controller of the second-level measurement and control server corresponding to the measured and controlled node group through 6 of the 8 full-duplex optical communication interfaces, and one of the remaining two full-duplex optical communication interfaces is connected to the optical communication interface in the first-level synchronous controller of the first-level measurement and control server through optical fiber, and the other is connected to the second-level measurement and control core machine of the second-level measurement and control server.
[0089] The first-level synchronization controller of the first-level measurement and control server synchronously sends control instructions to 6×8 functional nodes of the measured and controlled system, and synchronously receives feedback data uploaded by 6×8 functional nodes of the measured and controlled system.
[0090] The real-time synchronization method of the star-shaped and tree-shaped hybrid measurement and control architecture of the array system of the second embodiment of the present invention is as follows:
[0091] The primary synchronization controller sends the same data frame to all secondary synchronization controllers at a set clock cycle; the clock cycle is the same as the control cycle of the leaf node of the tree network; the data frame contains information selected by the primary synchronization controller for identifying a specific leaf node;
[0092] All secondary synchronization controllers take the period of receiving the data frame sent by the primary synchronization controller as the operation cycle. After receiving the data, the secondary synchronization controller forwards it to all leaf nodes. All leaf nodes take the moment of receiving the data frame sent by the primary synchronization controller as the beginning of a control beat.
[0093] The target leaf node selected by the first-level synchronization controller responds to the received data frame, and the remaining non-target leaf nodes do not respond to the received data frame;
[0094] The target leaf node receives and caches the instruction data in the current control cycle, and participates in the control algorithm when the next control cycle is running. In each control cycle of the leaf node, all leaf nodes update variables; the variables include the measurement results of all sensors in the node and the operation results of the control algorithm participated by the measurement results and the instruction data issued by the first-level synchronous controller;
[0095] After the target leaf node completes the control algorithm, it uploads the sensor measurement results and the control algorithm calculation results to the secondary synchronous controller before the current control cycle ends;
[0096] After receiving the data uploaded by the leaf nodes in the group, the secondary synchronization controller packages and forwards it to the primary synchronization controller.
[0097] Figure 2In the network diagram of the hybrid measurement and control architecture of the electromagnetic catapult system of an embodiment of the star-shaped and tree-shaped hybrid measurement and control architecture of the array system of the present invention, a star-shaped network with relatively low measurement and control requirements for real-time and synchronization is built as follows. A multi-port switch is selected as the central node of the star-shaped measurement and control network. The switch contains at least 57 Gigabit Ethernet optical interfaces and 48 Gigabit Ethernet electrical interfaces. 96 nodes to be measured and controlled, 8 secondary measurement and control core machines, and 1 primary measurement and control core machine are connected to the switch via Gigabit Ethernet, wherein the power generation subsystem node, the energy storage subsystem node, the discharge subsystem node, the secondary measurement and control core machine, and the primary measurement and control core machine use Gigabit Ethernet optical interfaces, and the system security node uses Gigabit Ethernet point interfaces. In this way, a star-shaped measurement and control network with a multi-port switch as the central node and the nodes to be measured and controlled and the measurement and control core machine as terminal nodes is formed in the hardware layout, and the data interaction between any two terminal nodes must be forwarded through the central node. In order to further improve the working efficiency of the star network, the planning and design of data interaction are hierarchically designed according to the principle of tree structure. The first-level measurement and control core machine directly controls all the second-level measurement and control core machines, that is, in software design, the first-level measurement and control core machine only allows 8 second-level measurement and control core machines to establish connection with it, and the measured and controlled nodes are not allowed to directly establish connection with the first-level measurement and control core machine. In the above way, the software development difficulty of the first-level measurement and control core machine is simplified; the measured and controlled nodes in each minimum measurement and control unit only interact with the second-level measurement and control core machine in the minimum measurement and control unit, that is, the second-level measurement and control core machine in the minimum measurement and control unit 1 is not allowed to establish connection with any measured and controlled node in the minimum measurement and control unit 2 to the minimum measurement and control unit 7.
[0098] For nodes with high requirements for real-time and synchronization, the following tree networking is performed to form a data synchronization network. The power generation subsystem nodes, energy storage subsystem nodes, and discharge subsystem nodes equipped with optical communication interfaces designed based on FPGA high-speed transceivers serve as leaf nodes of the tree network. The secondary measurement and control server in each minimum measurement and control unit serves as the subtree root. The primary synchronization controller serves as the tree root. The 6 leaf nodes in each minimum measurement and control unit are connected to the secondary synchronization controller as the subtree root in this unit through the optical communication interface designed based on FPGA high-speed transceiver; the remaining 2 optical communication interfaces of the secondary synchronization controller, 1 is connected to the primary synchronization controller as the tree root, and 1 is connected to the secondary measurement and control core machine in this unit; 8 of the 9 optical communication interfaces of the primary synchronization controller have been connected to 8 secondary synchronization controllers, and the remaining 1 optical communication interface is connected to the primary measurement and control core machine.
[0099] Since all optical communication interfaces are designed with FPGA's high-speed transceivers, based on the FPGA's parallel and full-duplex characteristics of high-speed transceivers, the first-level synchronization controller can simultaneously send control instructions to 48 measured and controlled nodes, and synchronously receive feedback data uploaded by 48 measured and controlled nodes, with the ability to send and upload data simultaneously.
[0100] Taking the control command sent by the first-level synchronous controller to the No. 1 leaf node as an example, the real-time and synchronization of the tree network is explained. The first-level synchronous controller sends the same data frame to all the second-level synchronous controllers with a clock cycle of 50uS. Since the sending of command data to the node is event-triggered, when no data is sent to any leaf node, the data frame sent by the first-level synchronous controller is invalid data, which is used to provide the operation rhythm for the second-level synchronous controller and the leaf node; when the first-level synchronous controller sends a command to the No. 1 leaf node, the first-level synchronous controller sends the same command data to the 8 second-level synchronous controllers. The 8 second-level synchronous controllers use the cycle of receiving the data frame sent by the first-level synchronizer as the operation cycle. The delay error of the 8 second-level synchronous controllers receiving the same data frame from the first-level synchronous controller is about ten nanoseconds. This error is mainly introduced by the communication delay difference caused by the optical fiber from the first-level synchronous controller to all the second-level synchronous controllers. After receiving the data, the second-level synchronous controller forwards it to all leaf nodes. The leaf node uses the frequency of receiving the data frame sent by the second-level synchronous controller as the control frequency. The delay error of all leaf nodes receiving the same data frame from the secondary synchronous controller is about ten nanoseconds, which is mainly introduced by the communication delay difference caused by the optical fiber from the secondary synchronous controller to all leaf nodes. Therefore, the time error of all leaf nodes at the start of the current beat is tens of nanoseconds. The data frame contains information for identifying leaf node No. 1. Only leaf node No. 1 will respond to the data frame sent, and the other 47 leaf nodes will not respond to the received data frame. Leaf node No. 1 receives and caches instruction data in the current beat and participates in the control algorithm when the next beat runs. All leaf nodes start to receive the data frame sent by the primary synchronous controller as the beginning of a control beat. In each control beat of the leaf node, variables will be updated. These variables contain the measurement results of all sensors in the node, as well as the operation results of the control algorithm participated by the measurement results and the instruction data sent by the primary synchronous controller. After completing the control algorithm, the leaf node uploads the measurement results of the sensor and the operation results of the control algorithm to the secondary synchronous controller. After receiving the data uploaded by the leaf node in the group, the secondary synchronous controller packages and forwards it to the primary synchronous controller.
[0101] The above process should meet the following conditions: the communication delay time from the first-level synchronous controller sending data frames to the leaf node plus the running time of the leaf node control algorithm plus the communication delay time from the leaf node uploading the updated variables to the first-level synchronous controller is less than the time of one control beat.
[0102] In the above manner, all leaf nodes run at the same beat. Leaf node No. 1 receives the instruction data issued by the first-level synchronous controller in the current beat, and participates in the control algorithm in the next control beat cycle. The first-level synchronous controller receives the update data of all leaf nodes in the current beat.
[0103] In this embodiment, the star network and the tree network are planned as follows in terms of software functions, so that the two parts can be independent of each other and back up and complement each other: the tree network designed based on FPGA high-speed transceiver has the ability to quickly and in parallel upload data of multiple leaf nodes to the root of the tree, and the instructions of the root of the tree are synchronously sent to all leaf nodes with low latency error, and the measurement and control commands are sent and key state variables are uploaded. They are transmitted to the measurement and control server through the tree network interaction. The core machine in the server gives priority to processing data from the synchronization controller to ensure the real-time performance of the measurement and control system; all data in the interactive measurement and control system in the Ethernet-based star network are used to implement functions with low real-time requirements, such as data storage, node network timing, node parameter adjustment software upgrades, etc., as well as status configuration and remote monitoring of on-site system security nodes.
[0104] The data of the star network has a larger delay than that of the tree network. The data of the star network is used to perform delay verification on the core data of the tree network interaction, which is used to conduct a health assessment of the communication network of the entire measurement and control system. Once the tree network fails, the star network can also realize the measurement and control function of the system, but the system delay is greater than when the tree network participates in the measurement and control.
[0105] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the method described above can refer to the corresponding process in the aforementioned system embodiment, and will not be repeated here.
[0106] It should be noted that the star-shaped and tree-shaped hybrid measurement and control architecture and method of the array system provided in the above embodiment are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiment can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not regarded as improper limitations of the present invention.
[0107] A device according to a third embodiment of the present invention includes:
[0108] at least one processor; and
[0109] a memory communicatively connected to at least one of the processors; wherein,
[0110] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the real-time synchronization method of the star-shaped and tree-shaped hybrid measurement and control architecture of the array system.
[0111] A computer-readable storage medium according to a fourth embodiment of the present invention stores computer instructions, wherein the computer instructions are used to be executed by the computer to implement the real-time synchronization method of the star-shaped and tree-shaped hybrid measurement and control architecture of the above-mentioned array system.
[0112] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process and related instructions of the storage device and processing device described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.
[0113] Those skilled in the art should be able to appreciate that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two, and the programs corresponding to the software modules and method steps can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.
[0114] The terms "first", "second", etc. are used to distinguish similar objects rather than to describe or indicate a particular order or sequence.
[0115] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus / device.
[0116] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.
Claims
1. A star-shaped and tree-shaped hybrid measurement and control architecture of an array system, characterized in that: The star-shaped and tree-shaped hybrid measurement and control architecture includes a set of primary measurement and control servers, N sets of secondary measurement and control servers, N groups of measured and controlled nodes corresponding to the N sets of secondary measurement and control servers, and a central switch; The primary measurement and control server, the N sets of secondary measurement and control servers and the N groups of measured and controlled nodes serve as terminal nodes of a star network, the central switch serves as a central node of the star network, and the terminal nodes are connected to the central node via Ethernet to form a star network based on Ethernet; The first-level measurement and control server serves as the root of the tree network, the N sets of second-level measurement and control servers serve as sub-tree roots of the tree network, the system function nodes in the N groups of measured and controlled nodes serve as leaf nodes of the tree network, the leaf nodes are connected to the corresponding sub-tree roots through the optical fiber data synchronization network, and the sub-tree roots are connected to the tree roots through the optical fiber data synchronization network, forming a tree network of the optical fiber data synchronization network based on FPGA high-speed transceiver; The first-level measurement and control server includes a first-level synchronization controller and a first-level measurement and control core machine, and the second-level measurement and control server includes a second-level synchronization controller and a second-level measurement and control core machine; The first-level synchronization controller and the second-level synchronization controller are both multi-channel optical communication controllers based on FPGA master control, and are externally provided with multiple optical communication interfaces based on FPGA high-speed transceivers to form the optical fiber data synchronization network; The primary measurement and control core machine and the secondary measurement and control core machine are both industrial computers based on an embedded real-time operating system, and are externally provided with a Gigabit Ethernet interface and an optical communication interface based on a high-speed transceiver of an FPGA; The first-level measurement and control core machine and the second-level measurement and control core machine realize a 1-way optical communication interface based on FPGA high-speed transceiver by extending the FPGA board; The optical fiber data synchronization network is networked in the following manner: The FPGA high-speed transceiver is configured with a full-duplex optical communication interface; The number of the full-duplex optical communication interfaces is determined according to the number N of the measured and controlled node groups and the number M of the system function nodes in each measured and controlled node group: The FPGA high-speed transceiver in the first-level measurement and control server is configured with an N+1-channel full-duplex optical communication interface; The FPGA high-speed transceiver in the secondary measurement and control server is configured with M+2 full-duplex optical communication interfaces; N secondary synchronous controllers occupy N optical communication interfaces of the primary synchronous controller among the N+1 full-duplex optical communication interfaces, and the remaining one full-duplex optical communication interface is connected to the optical communication interface of the primary measurement and control core computer through optical fiber; The M system function nodes in the same controlled node group are connected to the M optical communication interfaces in the secondary synchronization controller of the secondary measurement and control server corresponding to the controlled node group through M of the M+2 full-duplex optical communication interfaces, and of the remaining two full-duplex optical communication interfaces, one is connected to the optical communication interface in the primary synchronization controller of the primary measurement and control server through optical fiber, and the other is connected to the secondary measurement and control core machine of the secondary measurement and control server.
2. The star-shaped and tree-shaped hybrid measurement and control architecture of the array system according to claim 1 is characterized in that: The first-level measurement and control server is also provided with a first-level monitoring PC terminal; The first-level monitoring PC terminal is used for human-computer interaction and data management of the first-level measurement and control server.
3. The star-shaped and tree-shaped hybrid measurement and control architecture of the array system according to claim 1 is characterized in that: The N sets of secondary measurement and control servers are also each provided with a secondary monitoring PC terminal; The secondary monitoring PC terminal is used for human-computer interaction and data management of the secondary measurement and control server.
4. The star-shaped and tree-shaped hybrid measurement and control architecture of the array system according to claim 3 is characterized in that: The central switch is a multi-port central switch; The multi-port central switch is used for data forwarding and is connected to the multiple sets of secondary measurement and control servers via Ethernet.
5. The star-shaped and tree-shaped hybrid measurement and control architecture of the array system according to claim 4 is characterized in that: The measured and controlled node group includes P measured and controlled nodes; the P measured and controlled nodes include M system function nodes and Q system security nodes, P=M+Q; The measured and controlled node is connected to the central node of the multi-port central switch via a network cable, and a local area network is formed through the multi-port central switch.
6. The star-shaped and tree-shaped hybrid measurement and control architecture of the array system according to claim 5 is characterized in that: The primary synchronization controller is connected to a plurality of secondary synchronization controllers via the optical fiber data synchronization network.
7. The star-shaped and tree-shaped hybrid measurement and control architecture of the array system according to claim 6 is characterized in that: The hybrid measurement and control architecture sends control instructions to M×N measured and controlled system function nodes synchronously through the first-level synchronization controller of the first-level measurement and control server, and synchronously receives feedback data uploaded by the M×N measured and controlled system function nodes.
8. A real-time synchronization method for a star-shaped and tree-shaped hybrid measurement and control architecture of an array system, based on a star-shaped and tree-shaped hybrid measurement and control architecture of an array system according to any one of claims 1 to 7, characterized in that: The real-time synchronization method is: The primary synchronization controller sends the same data frame to all secondary synchronization controllers at a set clock cycle; the clock cycle is the same as the control cycle of the leaf nodes of the tree network; The data frame includes information selected by the primary synchronization controller for identifying a specific leaf node; All secondary synchronous controllers take the period of receiving the data frame sent by the primary synchronous controller as the operation period. After receiving the data, the secondary synchronous controller forwards it to all leaf nodes. All leaf nodes take the moment of receiving the data frame sent by the primary synchronous controller as the beginning of a control beat. The target leaf node selected by the first-level synchronization controller responds to the received data frame, and the remaining non-target leaf nodes do not respond to the received data frame; The target leaf node receives and caches the instruction data in the current control cycle, and participates in the control algorithm when the next control cycle is running. In each control cycle of the leaf node, all leaf nodes update variables; the variables include the measurement results of all sensors in the node and the operation results of the control algorithm participated by the measurement results and the instruction data issued by the first-level synchronous controller; After the target leaf node completes the control algorithm, it uploads the sensor measurement results and the control algorithm calculation results to the secondary synchronous controller before the current control cycle ends; After receiving the data uploaded by the leaf nodes in the group, the secondary synchronization controller packages and forwards it to the primary synchronization controller.
Citation Information
Patent Citations
Array system synchronous measurement and control network architecture
CN111884750A
Multi-node test system and method for performing multi-node test
CN112003768A