A data synchronization method suitable for large-scale distributed cluster simulation
By introducing high-performance computing servers and RPC microservice architectures in distributed cluster simulation environments, the problem of vehicle motion synchronization is solved, the synchronization and real-time nature of simulation data is improved, and more efficient simulation data processing and more complexity are achieved. Access to the algorithm.
Patent Information
- Application Number
- CN202410260221.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-07
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-03-07
AI Technical Summary
The existing clustered drone simulation environment cannot guarantee the synchronization of the vehicle motion process in a distributed environment, resulting in the inability to provide reliable simulation data, and it is difficult to achieve low-latency simulation data synchronization when the number of nodes increases, synchronization and real-time. Poor sex.
Adding high-performance computing servers as dedicated servers for simulation data synchronization in distributed cluster simulation environments, using RPC microservice architecture to control the calculation and output frequency of vehicle motion state data, and centralized processing and storage through dedicated servers for simulation data synchronization to ensure that Data synchronization and real-time.
Improve the number of simulation nodes that can be accessed by the distributed cluster simulation system, ensure the synchronization and real-time nature of the simulation data, and enable the existing distributed cluster simulation environment to access more types or more complex cluster intelligent algorithms to be used as the algorithm. Provide more reliable simulation data.
Smart Images

Figure CN118158238B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of simulation technology, and in particular to a data synchronization method suitable for large-scale distributed cluster simulation. Background Art
[0002] At present, high-precision cluster UAV confrontation simulation environment requires detailed modeling of three-dimensional simulation scenes and vehicle models, and at the same time requires a high degree of restoration of the movement and confrontation process of the vehicle in the scene. The existing cluster UAV simulation environment is limited by the 3D rendering engine and cannot guarantee the synchronization of the movement process of each vehicle in the distributed simulation environment, resulting in the inability to provide reliable simulation data for existing cluster intelligence algorithms. At the same time, when the number of distributed simulation nodes increases to more than 100 nodes, the existing simulation environment is difficult to achieve low-latency simulation data synchronization function, and the synchronization and real-time performance are poor.
[0003] Therefore, it is necessary to provide a data synchronization method suitable for large-scale distributed cluster simulation to solve the above problems. Summary of the invention
[0004] The purpose of the present invention is to provide a data synchronization method suitable for large-scale distributed cluster simulation. The use of a dedicated server for simulation data synchronization can increase the number of simulation nodes that can be accessed by the distributed cluster simulation system and ensure the synchronization and real-time performance of the simulation data. At the same time, the existing distributed cluster simulation environment can be connected to more types or more complex cluster intelligence algorithms, providing more reliable simulation data for the algorithm.
[0005] To achieve the above object, the present invention provides a data synchronization method suitable for large-scale distributed cluster simulation, comprising the following steps:
[0006] S1: Add a high-performance computing server to the existing distributed cluster simulation environment as a dedicated server for simulation data synchronization;
[0007] S2: The simulation data synchronization dedicated server adopts the RPC microservice architecture as the basic framework, and uses the RPC request-response mechanism to control the calculation and output frequency of the vehicle motion state data in the distributed simulation nodes. The vehicle motion data includes position, posture, speed, and rotation angle of the moving parts;
[0008] S3: The dynamics simulation software of the distributed simulation node sends the vehicle motion state data to the dedicated server for simulation data synchronization, which is then centrally processed by the dedicated server for simulation data synchronization and stored in the synchronization simulation data queue;
[0009] S4: The rendering software in the distributed simulation node periodically requests the synchronized data from the simulation data synchronization dedicated server for rendering according to the rendering frequency, and renders the motion status of all vehicles into the three-dimensional simulation scene.
[0010] Preferably, in step S2, the simulation data synchronization dedicated server includes simulation node registration management, cluster data forwarding and data smoothing processing;
[0011] Node registration management includes simulation node identity management and node service status query. Node registration management assigns a unique identity to each simulation node and provides the node with a service status query function. Node registration management includes the following processes:
[0012] After registering with the server, the simulation node will obtain the data interaction port assigned by the server. During the simulation process, the simulation node will interact with the server through the data interaction port. The simulation node will determine whether to start the simulation calculation process by querying the simulation service status and output the simulation status data to the server. The service status query function determines whether the data of this node needs to be synchronized during the simulation process according to the service connection status of the node.
[0013] The functions of cluster data forwarding include data reception and transmission of all simulation nodes in the cluster and time synchronization of cluster simulation data according to time constraint rules. The server and all simulation nodes in the cluster have separate communication ports.
[0014] Preferably, in step S3, after the simulation starts, the server sends service status data for starting the simulation to all connected simulation nodes. After receiving the service status data, the simulation nodes start the simulation calculation process. The calculation frequency is adjusted in real time according to the cluster data synchronization function of the server. The server performs synchronization processing on the received cluster data according to the set synchronization period. The specific process is as follows:
[0015] S31: Obtain the number of simulation nodes in the server that are in a normal connection state, and use the number as a synchronization update comparison benchmark value A;
[0016] S32: waiting to receive simulation data from each simulation node within a data synchronization cycle, and after any data port receives simulation status data, the number of updated nodes B recorded in the trigger server is increased by one;
[0017] S33: Determine whether the updated node quantity B is equal to the synchronous update comparison benchmark value A;
[0018] When A=B, the simulation data received in this cycle is stored in the synchronized cluster data queue, B is reset to 0, and after the synchronization cycle ends, all simulation nodes are triggered to perform the next simulation operation, and the process returns to step S31;
[0019] When B < A, it is judged whether the waiting data times out. If it does not time out, continue to wait. If it has timed out, continue to execute downward;
[0020] S34: During a data synchronization period, when there is unupdated data of simulation nodes, obtain the cluster data of the previous period from the synchronized cluster data queue, compare the cluster data of the previous period with the cluster data received during the current period, and obtain the identity identifier list of the unupdated nodes;
[0021] S35: According to the identity identifier list of the unupdated nodes, obtain the simulation state data of the corresponding nodes within 10 synchronization periods from the synchronized cluster data queue, obtain the average speed of each simulation node within 10 synchronization periods from this data, and calculate the position data of each simulation node at the next moment according to the average speed and the synchronization period;
[0022] S36: Merge the position data of the unupdated nodes at the next moment calculated by the server with the simulation state data of the remaining nodes received, store them in the synchronized cluster data queue, trigger all simulation nodes to perform the next simulation operation, and return to step S31 to enter the next synchronization period.
[0023] Preferably, in step S4, the synchronization data queue stored by the server during the cluster data synchronization is used for the three-dimensional scene rendering of simulation nodes or complex cluster intelligent algorithm operations. The logic for synchronous rendering when the three-dimensional scene rendering software is enabled on multiple simulation nodes is as follows:
[0024] S41: Judge whether the set node rendering period in the server is an integer multiple of the data synchronization period in the server. If it is an integer multiple, obtain the latest synchronization data from the synchronization data queue of the server according to the rendering frequency and broadcast it to all node rendering software for rendering;
[0025] If it is not an integer multiple, obtain the two frames of synchronization data adjacent to Tb in the rendering synchronization data queue of the server according to the time point Tb of the current rendering frame. According to the time difference and the two frames of synchronization data, obtain the rendering data through smoothing processing in the server, estimate the rendering data at the time point Tb by the method of linear interpolation, and broadcast the estimated rendering data to the simulation nodes. Use continuous multiple frames of synchronization data during the process of estimating the rendering data at the time point Tb;
[0026] S42: When the rendering software in the simulation node receives the rendering data sent by the server, find the simulation state data of each vehicle according to the identity identifier of the vehicle in the scene;
[0027] S43: The rendering software respectively sets the position, attitude and movement angle of the vehicle components corresponding to the obtained simulation state data.
[0028] Therefore, the present invention adopts the above-mentioned data synchronization method suitable for large-scale distributed cluster simulation, and uses a dedicated server for simulation data synchronization to increase the number of simulation nodes that can be accessed by the distributed cluster simulation system, and ensure the synchronization and real-time performance of the simulation data. At the same time, the existing distributed cluster simulation environment can be connected to more types or more complex cluster intelligence algorithms, providing more reliable simulation data for the algorithm.
[0029] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 It is a flow chart of a data synchronization method suitable for large-scale distributed cluster simulation of the present invention;
[0031] Figure 2 This is a functional composition diagram of the simulation data synchronization dedicated server of the present invention;
[0032] Figure 3 It is a schematic diagram of the logic of obtaining rendering data when the rendering cycle of the present invention is an integer multiple of the synchronization cycle;
[0033] Figure 4 It is a schematic diagram of the logic of obtaining rendering data when the rendering cycle of the present invention is not an integer multiple of the synchronization cycle;
[0034] Figure 5 This is a diagram of the large-scale distributed cluster simulation architecture of the present invention;
[0035] Figure 6 It is a two-dimensional situation interface diagram of large-scale distributed cluster simulation of the present invention;
[0036] Figure 7 This is a rendering effect diagram of a large-scale distributed cluster simulation three-dimensional scene of the present invention. DETAILED DESCRIPTION
[0037] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.
[0038] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.
[0039] The words "include" or "comprises" and the like used in the present invention mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of also including other elements. The orientation or position relationship indicated by the terms "inside", "outside", "upper", "lower", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. When the absolute position of the described object changes, the relative position relationship may also change accordingly. In the present invention, unless otherwise clearly specified and limited, the terms "attachment" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral body; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.
[0040] Example
[0041] like Figure 1 As shown, the present invention provides a data synchronization method suitable for large-scale distributed cluster simulation. When 1000 simulation nodes are connected in the simulation environment, the synchronization time error does not exceed 5 milliseconds; the method comprises the following steps:
[0042] S1: Add a high-performance computing server to the existing distributed cluster simulation environment as a dedicated server for simulation data synchronization;
[0043] S2: The simulation data synchronization dedicated server uses the RPC microservice architecture as the basic framework, such as Figure 2 As shown, the RPC request-response mechanism is used to control the calculation and output frequency of the vehicle motion state data in the distributed simulation node, and the vehicle motion data includes the position, posture, speed and rotation angle of the moving parts; in step S2, the simulation data synchronization dedicated server includes simulation node registration management, cluster data forwarding and data smoothing processing;
[0044] Node registration management includes simulation node identity management and node service status query. Node registration management assigns a unique identity to each simulation node and provides the node with a service status query function. Node registration management includes the following processes:
[0045] After the simulation node registers with the server, it will obtain the data interaction port allocated by the server and conduct data interaction with the server through the data interaction port during the simulation process. The simulation node determines whether to start the simulation calculation process by querying the simulation service status and outputs the simulation status data to the server. The service status query function determines whether it is necessary to synchronize the data of this node during the simulation process based on the service connection status of the node;
[0046] The function of cluster data forwarding includes the data reception and transmission of all simulation nodes in the cluster and the time synchronization of cluster simulation data by time constraint rules. The server and all simulation nodes in the cluster have separate communication ports. Therefore, the number of cluster nodes that the server can access is only limited by the hardware performance and network environment.
[0047] S3: The dynamic simulation software of the distributed simulation node sends the vehicle motion state data to the dedicated server for simulation data synchronization. After being centrally processed by the dedicated server for simulation data synchronization, it is stored in the synchronized simulation data queue;
[0048] In step S3, after the simulation starts, the server sends the service status data of starting the simulation to all connected simulation nodes. After receiving the service status data, the simulation nodes start the simulation calculation process, and the calculation frequency is adjusted in real time according to the cluster data synchronization function of the server. The server synchronizes the received cluster data according to the set synchronization period. The specific process is as follows:
[0049] S31: Obtain the number of simulation nodes with normal status connections in the server, and use this value as the synchronization update comparison reference value A;
[0050] S32: During a data synchronization period, wait to receive the simulation data of each simulation node. After any data port receives the simulation status data, trigger the increase of the number of updated nodes B recorded in the server by one;
[0051] S33: Judge whether the number of updated nodes B is equal to the synchronization update comparison reference value A;
[0052] When A = B, store the simulation data received in this period in the synchronized cluster data queue, reset B to 0, wait until the end of this synchronization period, trigger all simulation nodes to perform the next simulation operation, and return to step S31;
[0053] When B < A, judge whether the waiting data times out. If it does not time out, continue to wait. If it has timed out, continue to execute downward;
[0054] S34: when there is simulation node data that has not been updated in a data synchronization cycle, cluster data of the previous cycle is obtained from the synchronized cluster data queue, and the cluster data of the previous cycle is compared with the cluster data received in the current cycle to obtain an identity list of the unupdated nodes;
[0055] S35: According to the identity list of the unupdated nodes, the simulation state data of the corresponding nodes within 10 synchronization cycles are obtained in the synchronized cluster data queue, the average speed of each simulation node within 10 synchronization cycles is obtained from the data, and the position data of each simulation node at the next moment is calculated according to the average speed and the synchronization cycle;
[0056] S36: Merge the position data of the unupdated nodes calculated by the server at the next moment with the received simulation status data of the remaining nodes and store them in the synchronized cluster data queue, trigger all simulation nodes to perform the next simulation operation, and return to step S31 to enter the next synchronization cycle.
[0057] S4: The rendering software in the distributed simulation node periodically requests the synchronized data from the simulation data synchronization dedicated server for rendering according to the rendering frequency, and renders the motion status of all vehicles into the three-dimensional simulation scene.
[0058] In step S4, the synchronization data queue stored by the server during the cluster data synchronization process is used for 3D scene rendering of the simulation node or complex cluster intelligent algorithm calculation. The logic of synchronous rendering when the 3D scene rendering software is turned on on multiple simulation nodes is as follows:
[0059] S41: Determine whether the node rendering cycle set in the server is an integer multiple of the data synchronization cycle in the server. If it is an integer multiple, obtain the latest synchronization data in the synchronization data queue of the server according to the rendering frequency, and broadcast it to all node rendering software for rendering;
[0060] If it is not an integer multiple, obtain the two frames of synchronization data adjacent to Tb in the rendering synchronization data queue in the server according to the time point Tb of the current rendering frame, that is, the synchronization data at the time points Ta and Tc. According to the time difference and the two frames of synchronization data, obtain the rendering data through smoothing processing in the server, estimate the rendering data at the time point Tb by linear interpolation, and broadcast the estimated rendering data to the simulation node. Use the synchronization data of multiple consecutive frames in the process of estimating the rendering data at the time point Tb; the interpolation method can also be replaced by other methods.
[0061] S42: When the rendering software in the simulation node receives the rendering data sent by the server, it finds the simulation state data of each vehicle according to the identity of the vehicle in the scene, that is, the unique identity registered and assigned by the server;
[0062] S43: The rendering software sets the position and posture of the corresponding vehicle and the movement angle of the vehicle components according to the obtained simulation state data.
[0063] Embodiment 2
[0064] Suitable for large-scale distributed cluster simulation architectures such as Figure 5 As shown in the figure, the large-scale distributed cluster two-dimensional situation interface is as follows Figure 6 As shown in the figure, the rendering effect of large-scale distributed cluster 3D scene is as follows Figure 7 As shown, a high-performance computing server is used as a dedicated server for simulation data synchronization, 1 computer is used as a simulation control terminal, 20 computers are used as distributed simulation nodes (each computer runs 10 simulation nodes), and all hardware is connected to the same local area network bus. The dynamics simulation software in the distributed simulation nodes uses PX4 and JSBSim to perform software-in-the-loop dynamics simulation, and the simulation rendering software uses independent simulation software developed by Unreal Engine (UE4). This example implements a distributed cluster simulation of 200 nodes, and performs 3D scene rendering based on the simulation data synchronized by the dedicated server for simulation data synchronization. The simulation effect is as follows: Figure 3 and Figure 4 shown. Figure 3 The medium-circle icons are ground vehicles, and the other shapes are aerial drone clusters.
[0065] The specific simulation process is as follows
[0066] (1) Set the network IP address of each hardware and deploy all software to the corresponding hardware platform.
[0067] (2) Configure the interface IP address of each simulation node and simulation control terminal and point it to the dedicated server for simulation data synchronization. Write the IP address of the simulation control terminal into the data interaction configuration file of PX4 in the dynamics simulation software.
[0068] (3) Start the simulation software in each hardware platform, and check the registration status of all simulation nodes in the simulation data synchronization dedicated server in the simulation control terminal software. After checking all registrations, set the simulation parameters and send the start simulation command.
[0069] (4) After receiving the start simulation instruction, all distributed simulation nodes calculate the motion state data once and upload it to the simulation data synchronization dedicated server through the RPC service interface specified by the server.
[0070] (5) The simulation data synchronization dedicated server synchronizes the received simulation node motion state data according to the number of registered simulation nodes and the waiting timeout period, and stores the synchronized simulation data into the synchronization data queue.
[0071] (6) After receiving the start simulation instruction, the simulation rendering software obtains the synchronized simulation data from the simulation data synchronization dedicated server through an asynchronous request, and renders the motion status of all vehicles in the three-dimensional scene. This cycle continues until the simulation ends.
[0072] Therefore, the present invention adopts the above-mentioned data synchronization method suitable for large-scale distributed cluster simulation, and uses a dedicated server for simulation data synchronization to increase the number of simulation nodes that can be accessed by the distributed cluster simulation system, and ensure the synchronization and real-time performance of the simulation data. At the same time, the existing distributed cluster simulation environment can be connected to more types or more complex cluster intelligence algorithms, providing more reliable simulation data for the algorithm.
[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.
Claims
1. A data synchronization method suitable for large-scale distributed cluster simulation, characterized in that: It includes the following steps: S1: Add a high-performance computing server to the existing distributed cluster simulation environment as a dedicated server for simulation data synchronization; S2: The dedicated server for simulation data synchronization uses the RPC microservice architecture as the basic framework, and uses the request-response mechanism of RPC to control the calculation and output frequency of vehicle motion state data in the distributed simulation nodes. The vehicle motion data includes position, attitude, speed, and the rotation angle of the moving parts; In step S2, the dedicated server for simulation data synchronization includes simulation node registration management, cluster data forwarding, and data smoothing processing; The node registration management includes simulation node identity management and node service status query. The node registration management assigns a unique identity identifier to each simulation node and provides the node with a service status query function. The node registration management includes the following processes: After the simulation node registers with the server, it will obtain the data interaction port assigned by the server and perform data interaction with the server through the data interaction port during the simulation process. The simulation node judges whether to start the simulation calculation process and output the simulation status data to the server by querying the simulation service status. The service status query function judges whether it is necessary to synchronize the data of this node during the simulation process according to the service connection status of the node; The function of cluster data forwarding includes receiving and sending the data of all simulation nodes in the cluster and synchronizing the cluster simulation data according to the time constraint rules. The server and all simulation nodes in the cluster have separate communication ports; S3: The dynamic simulation software of the distributed simulation node sends the vehicle motion state data to the dedicated server for simulation data synchronization, and after centralized processing by the dedicated server for simulation data synchronization, it is stored in the synchronized simulation data queue. In step S3, after the simulation starts, the server sends the service status data of starting the simulation to all connected simulation nodes. After receiving the service status data, the simulation nodes start the simulation calculation process, and the calculation frequency is adjusted in real time according to the cluster data synchronization function of the server. The server synchronizes the received cluster data according to the set synchronization period. The specific process is as follows: S31: Obtain the number of simulation nodes with normal status connections in the server, and use this value as the synchronization update comparison reference value A; S32: During a data synchronization period, wait to receive the simulation data of each simulation node. After receiving the simulation status data on any data port, trigger the increase of the number of updated nodes B recorded in the server by one; S33: Judge whether the number of updated nodes B is equal to the synchronization update comparison reference value A; When A = B, store the simulation data received in this period in the synchronized cluster data queue, reset B to 0, wait until the end of this synchronization period, trigger all simulation nodes to perform the next simulation operation, and return to step S31; When B < A, judge whether the waiting data times out. If it does not time out, continue to wait. If it has timed out, continue to execute downward; S34: when there is simulation node data that has not been updated in a data synchronization cycle, cluster data of the previous cycle is obtained from the synchronized cluster data queue, and the cluster data of the previous cycle is compared with the cluster data received in the current cycle to obtain an identity list of the unupdated nodes; S35: According to the identity list of the unupdated nodes, the simulation state data of the corresponding nodes within 10 synchronization cycles are obtained in the synchronized cluster data queue, the average speed of each simulation node within 10 synchronization cycles is obtained from the data, and the position data of each simulation node at the next moment is calculated according to the average speed and the synchronization cycle; S36: Merge the position data of the unupdated nodes calculated by the server at the next moment with the received simulation status data of the remaining nodes and store them in the synchronized cluster data queue, trigger all simulation nodes to perform the next simulation operation, and return to step S31 to enter the next synchronization cycle; S4: The rendering software in the distributed simulation node periodically requests the synchronized data from the simulation data synchronization dedicated server according to the rendering frequency for rendering, and renders the motion states of all vehicles into the three-dimensional simulation scene; In step S4, the synchronization data queue stored by the server during the cluster data synchronization process is used for 3D scene rendering of the simulation node or complex cluster intelligent algorithm calculation. The logic of synchronous rendering when the 3D scene rendering software is turned on on multiple simulation nodes is as follows: S41: Determine whether the node rendering cycle set in the server is an integer multiple of the data synchronization cycle in the server. If it is an integer multiple, obtain the latest synchronization data in the synchronization data queue of the server according to the rendering frequency, and broadcast it to all node rendering software for rendering; If it is not an integer multiple, obtain the two frames of synchronization data adjacent to Tb in the rendering synchronization data queue in the server according to the time point Tb of the current rendering frame, obtain the rendering data through smoothing processing in the server according to the time difference and the two frames of synchronization data, estimate the rendering data at the time point Tb by linear interpolation, and broadcast the estimated rendering data to the simulation node, and use the synchronization data of multiple consecutive frames in the process of estimating the rendering data at the time point Tb; S42: When the rendering software in the simulation node receives the rendering data sent by the server, it finds the simulation state data of each vehicle according to the identity of the vehicle in the scene; S43: The rendering software sets the position and posture of the corresponding vehicle and the movement angle of the vehicle components according to the obtained simulation state data.
