A distributed simulation method and system for multiple unmanned systems

Abstract

The present invention discloses a distributed simulation method and system for multiple unmanned systems. The method comprises: decomposing the distributed simulation tasks of multiple unmanned systems into a distributed virtual simulation network, a perception and control subnetwork, and a DDS-based collaborative communication service network according to the three logical concepts of virtual simulation, control perception, and task collaboration; deploying the virtual simulation tasks for the multi-unmanned system task collaboration on distributed nodes; providing rule arbitration and decision-making, map scene management, real-time state synchronization, multi-point session management, intelligent simulation, and real-time situation sharing services through the integrated server of the distributed virtual simulation network; and providing local computing functions including dynamics, collision interaction, damage simulation, and graphics rendering through the client of the distributed virtual simulation network. The system is used to implement the above method. The present invention has the advantages of simple principle, wide application range, and high functional integration.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of unmanned system simulation, and in particular to a multi-unmanned system distributed simulation method and system. Background Art

[0002] From structural design and module function development to integrated application, unmanned systems rely on simulation systems as a platform to provide continuous integration testing support. The main task flow of a single unmanned system includes the functional links of observation, judgment, decision-making, and execution. The continuous integration development process for unmanned systems requires phased testing through functional module-level model-in-the-loop testing, full-system software-in-the-loop testing, hardware-in-the-loop testing, and actual machine-in-the-loop testing. Beyond actual machine testing, all other phases rely on simulation. This shows that verification, from basic sub-functional logic modules to integrated complex functional systems, relies on a simulation environment that supports the continuous integration development of unmanned systems.

[0003] Unmanned system simulation platforms should include object models and environmental data, provide realistic simulation and process deduction capabilities, and support data interfaces for the input and output of the functional modules of the unmanned system under test, exchanging corresponding simulation data with these interfaces. Currently, the workflow of unmanned system simulation platforms is becoming increasingly standardized, and the technology is gradually becoming systematic. With the development of unmanned systems, especially robotics, universities, research institutions, and robot vendors have designed and developed a wide range of simulation test platform software. These include offline commercial simulation tools primarily designed for automated assembly lines: Robot Master, RobotArt, RobotWorks, Robcad, DELMIA, RobotStudio, and Robomove; simulation tools that provide dynamic simulation for control algorithm development: ROS / Gazebo, CoppeliaSim (formerly V-REP), Webots, and Adams; simulation tools that provide sensor-level environmental perception simulation: USARSim, FlightGoggle, Airsim, Flightmare, and Rflysim; and simulation environments for reinforcement learning, such as Isaac Gym. Some of these platforms, such as RobotArt, RobotWorks, and Robcad, simulate complex product operating environments. Others, such as Gazebo and Carla, accelerate algorithm iteration during development, verify logic during safety testing, and fine-tune parameters during optimization. Others, such as OpenAI-Gym, support rapid training of machine learning algorithms. Given the relatively limited capabilities of individual unmanned systems, a growing number of researchers are focusing on task collaboration among multiple unmanned systems.

[0004] Currently, distributed multi-unmanned system collaborative task simulation platforms remain immature. Most research on collaborative planning and control of heterogeneous unmanned systems simulates distributed collaborative control on centralized verification platforms, ignoring the spatiotemporal inconsistencies in situational information sharing across distributed platforms and oversimplifying the complexity of task collaboration under distributed conditions. Furthermore, due to the limited computing resources of a single platform, it is impossible to achieve high-fidelity simulation of multi-source sensor data or support scalable distributed simulation.

[0005] High-fidelity multi-source sensor data simulation can leverage years of development in various simulator technologies tailored to game development needs, as well as mature and comprehensive game engine technology. Perception simulation of physical objects in unmanned systems primarily involves simulating simple motion states using inertial measurement sensors, and simulating visible light imaging using computer graphics and image rendering engines. Common visible light image rendering engines include OSG, OpenGL, and Unreal Engine. With advancements in ray tracing technology within rendering engines, image rendering is increasingly refining realistic details. Image rendering simulation engines not only offer traditional ray tracing simulation, LiDAR, and depth camera simulation, but also currently support static infrared sensor imaging simulation. Regarding infrared thermal imaging sensors, current online real-time simulation relies solely on visible light imaging rendering methods, using temperature UV mapping. This approach is unable to simulate the temporal thermal imaging changes associated with heat exchange between physical objects. Physical-level simulation of the motion of physical objects in unmanned systems is provided by the simulation engine's physics engine. Mature and common physics engines include ODE, IKFast, Bullet, CopterSim, PhysX, and Mujoco. The physics model solving capabilities used in unmanned system simulation research primarily focus on rigid-body dynamics and collision simulation. However, as research deepens, a growing number of simulation requirements, such as flexible bodies and fluids, are being added to the physics engine development roadmap. Most physics simulation calculations for continuous system simulations utilize numerical optimization methods to solve constrained differential equations. However, physics engines employing different modeling and computational approaches have their own strengths and weaknesses in their specific areas.

[0006] The simulation of coordinated missions for distributed multi-unmanned systems relies on the simulation of communication services. The open-source projects ROS1 and ROS2 are particularly useful references. They each utilize different communication middleware: ROS1 requires a master to handle the publish-subscribe communication layer, while ROS2 uses DDS (Data-Distribution Service) based on the RTSP (Real-Time Publish-Subscribe) protocol as the middleware. DDS (Data-Distribution Service) is an industry standard for publish-subscribe communication in real-time and embedded systems, making the system more fault-tolerant and flexible. While ROS's supporting libraries and application software are comprehensive, they can be cumbersome and redundant for algorithm verification or simulation training of unmanned systems. Summary of the Invention

[0007] The technical problem to be solved by the present invention is: in response to the technical problems existing in the prior art, the present invention provides a multi-unmanned system distributed simulation method and system with simple principle, wide application range and high functional integration.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] A multi-unmanned system distributed simulation method, comprising:

[0010] The distributed simulation task of multiple unmanned systems is decomposed into a distributed virtual simulation network, a perception and control sub-network, and a DDS-based collaborative communication service network according to the three logical concepts of virtual simulation, control perception, and task collaboration.

[0011] Deploy virtual simulation tasks for multi-unmanned system task coordination on distributed nodes;

[0012] The integrated server of the distributed virtual simulation network provides rule arbitration, map scene management, real-time state synchronization, multi-point session management, intelligent simulation and real-time situation sharing services; the client of the distributed virtual simulation network is used to provide local computing functions including dynamics, collision interaction, damage simulation and graphics rendering.

[0013] As a further improvement to the method of the present invention: the selection and switching of map scenes in the map scene management are uniformly controlled by the virtual environment integrated server; when the virtual environment integrated server chooses to load the map, all clients participating in the current task will load the same virtual scene map at the same time. After all clients have completed the pre-loading of the map, the virtual environment integrated server will automatically complete the confirmation and then enter the virtual scene synchronously.

[0014] As a further improvement to the method of the present invention: the map scene management includes consistency management of map scene data; the map scene data includes terrain, buildings, weather, and virtual entity objects controlled by the server, as well as virtual concept objects of the character's default initial generation position, and the assignment and association of dynamic non-player character virtual entity objects.

[0015] As a further improvement of the method of the present invention: the process of real-time status synchronization includes:

[0016] Step S1: Obtain all client connections connected to the virtual environment integrated server. First, obtain the maximum number of connections synchronized per frame by the engine. If the number exceeds this limit, no connection will be maintained.

[0017] Step S2: Find the entity objects to be synchronized. Only the entity objects that are placed in the network synchronization object list will be considered.

[0018] Step S3: Find the role controlled by the client;

[0019] Step S4: Verify the entity object. Objects that are to be destroyed or have empty ownership will not be synchronized.

[0020] Step S5: Whether the synchronization time of the entity object has arrived. The synchronization frequency is set when the object is defined. Before each synchronization, the next synchronization time is calculated. If the time is not reached, the current synchronization is abandoned.

[0021] Step S6: If the object is set to be related only to the control client, it is put into a special list and then synchronized only to the client that belongs to it;

[0022] Step S7: Entity objects in the dormant state will not be synchronized, and those about to enter the dormant state must also be specially processed to close the synchronization channel;

[0023] Step S8: Check whether the current object has a channel. If not, check whether the object has been loaded into the scene. If not, skip the synchronization step. If it is in the scene, check whether it is network-related. Entities that are invisible or too far away will not be synchronized.

[0024] Step S9: Arrange all objects in priority order, and calculate the priority corresponding to the current object according to whether there is a control terminal and whether the distance is within the field of view. The higher the priority, the earlier the synchronization.

[0025] As a further improvement of the method of the present invention: the multipoint session management includes the following process:

[0026] Step S10: The virtual environment integrated server creates a session in the virtual simulation network, sets the session name, the number of participating clients, and the connection method, whether it is a local area network or the Internet. After clicking Create, the virtual environment integrated server will enter the preparation interface;

[0027] Step S20: After the virtual environment integrated server is created, other clients find the session on the virtual environment integrated server search interface and click Join to join the session created by the previous client.

[0028] Step S30: After joining the session, select a role. Each or each type of unmanned system is regarded as a role. The name of each connected client and the selected role type will be displayed in the list. The virtual environment integrated server has the function of forcibly kicking out the client. The simulation process is started by the virtual environment integrated server, and all clients will enter the scene synchronously.

[0029] As a further improvement of the method of the present invention: the real-time situation sharing is a service interface provided by the virtual environment integrated service end to the outside, allowing access to other situation systems, and after access, publishing the spatial position and status of the object to other situation systems; this function is a reserved functional interface, used to connect physical systems to form a semi-physical simulation environment, or to connect to other virtual systems to form a larger simulation platform.

[0030] A multi-unmanned system distributed simulation system, comprising:

[0031] The distributed virtual simulation network, the perception control sub-network, and the DDS-based collaborative communication service network constitute a three-layer distributed simulation network.

[0032] Connect nodes in different perception and control sub-networks with the collaborative communication service of DDS to provide communication middleware services for situation sharing and task collaboration modules distributed in various perception and control sub-networks according to the task grouping of unmanned systems;

[0033] The virtual simulation task of multi-unmanned system task coordination is deployed on distributed nodes; among them, the virtual integrated server is used to provide rule arbitration and judgment, map scene management, real-time status synchronization, multi-point session management, intelligent simulation and real-time situation services; the client is used to provide local computing functions including dynamics, collision interaction, damage simulation, and graphics rendering; the client is used to provide three access methods: human-in-the-loop operation simulation method; intelligent unmanned system algorithm simulation method; simulation deduction guidance and control method.

[0034] As a further improvement of the system of the present invention: the perception control subnetwork maps the interaction permissions to the AR / VR head-mounted display device and the human-machine device of the joystick in the human-in-the-loop mode according to the access mode, or maps it to the software-in-the-loop (SITL) or hardware-in-the-loop (HITL) algorithm platform in the intelligent unmanned mode.

[0035] As a further improvement of the system of the present invention: the DDS-based collaborative communication service network is used to provide communication middleware services for the situation sharing and task collaboration modules distributed in various perception and control subnetworks according to the task grouping of the unmanned system; and the collaborative communication is forwarded through the virtual network according to the collaborative network.

[0036] As a further improvement of the system of the present invention: the hardware-in-the-loop is connected to the virtual network client through a network connection or a serial port, and topic-based status / instruction forwarding is realized through a message bridge; it is directly connected to the host computer, and the virtual simulation client realizes the dynamic simulation of the entity, and the status is forwarded from the host computer to the lower computer of the unmanned platform, and then the lower computer runs in the hardware-in-the-loop mode to upload the filtered status to the host computer through the hardware interface, and the host computer performs algorithm processing; the software-in-the-loop mode, by integrating the DDS function package, realizes the distributed nodes of the collaborative algorithm on Python, and at the same time, the software end obtains the corresponding perception control authority of the virtual object status / instruction from the client's interface service through remote procedure call.

[0037] Compared with the prior art, the advantages of the present invention are:

[0038] 1. The multi-unmanned system distributed simulation method and system of the present invention has a simple principle, a wide range of applications, and a high degree of functional integration. It can provide simulation services including dynamics, collision interaction, damage simulation, graphics rendering, and communication interaction functions for the control, planning, and verification of distributed collaborative algorithms of multi-unmanned systems. The multi-unmanned system distributed simulation platform designed by the present invention has six basic features: a. The distributed simulation platform includes a three-layer distributed simulation network consisting of a distributed virtual simulation network, a perception control subnetwork, and a collaborative communication service network based on DDS; b. The virtual integrated server in the distributed virtual simulation network provides the client with 6 main services including ① rule arbitration and judgment service, ② map scene management service, ③ real-time status synchronization service, ④ multi-point session management service, ⑤ intelligent blue team simulation service, and ⑥ real-time situation sharing service; c. The client in the distributed virtual simulation network can provide three access methods including ① human-in-the-loop operation simulation mode, ② intelligent unmanned system algorithm simulation mode, and ③ simulation deduction guidance and control mode. d. The client provides three types of local computing for the virtual unmanned system object with bound permissions: dynamic simulation, collision interaction calculation, and graphics rendering. It also provides two types of interfaces: the sensor interface and the planning control interface of the unmanned system object to the perception and control subnetwork. e. The perception and control subnetwork maps the interaction permissions to the human-machine devices of the AR / VR headset and joystick in the human-in-the-loop mode, or to the software-in-the-loop (SITL) or hardware-in-the-loop (HITL) algorithm platform in the intelligent unmanned mode, based on the access mode. f. The DDS-based collaborative communication service network provides communication middleware services for the situation sharing and task collaboration modules distributed in each perception and control subnetwork according to the task grouping of the unmanned system. Collaborative communications can be forwarded through the virtual network as needed.

[0039] 2. The present invention's multi-unmanned system distributed simulation method and system features a clear and logical network structure, facilitating functional development and service expansion integration. Based on the functional and service characteristics of unmanned systems, the present invention decomposes the distributed simulation tasks of multiple unmanned systems into a three-layer network structure consisting of a distributed virtual simulation network, a perception and control subnetwork, and a DDS-based collaborative communication service network, based on the three logical concepts of virtual simulation, perception and control, and task collaboration. This facilitates the integration and expansion of platform system functions and services.

[0040] 3. The present invention's distributed simulation method and system for multiple unmanned systems balances simulation service load, ensuring scalability and real-time operation support. To ensure real-time operation and scalability of simulation services, the present invention breaks down the complex coupled model calculations of multiple unmanned systems into rule arbitration and decision-making, map scene management, real-time state synchronization, multi-point session management, intelligent blue team simulation, and real-time situation sharing running on the server; dynamic simulation, interactive computing, and graphics rendering running on the client; and intelligent perception and planning control algorithms running on the terminal.

[0041] 4. The distributed simulation method and system for multiple unmanned systems of the present invention offers flexible and diverse access modes, supporting both simulation training and algorithm optimization. Based on the three access requirements of human-in-the-loop, intelligent algorithms, and simulated guidance and control, the present invention implements a perception control subnetwork and hardware driver interface for human-in-the-loop operation simulation, intelligent unmanned system algorithm simulation, and simulated deduction guidance and control.

[0042] 5. The present invention's distributed simulation method and system for multiple unmanned systems operates in a distributed, parallel manner, validating information sharing and collaborative task protocols. By connecting nodes in different perception and control subnetworks using DDS collaborative communication services, it provides communication middleware services for situation sharing and task collaboration modules distributed across the perception and control subnetworks, grouped according to the unmanned system's tasks. This serves to validate the design of information sharing and collaborative task protocols. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic flow diagram of the method of the present invention.

[0044] Figure 2 It is a schematic diagram of the implementation principle of the present invention in a specific application example.

[0045] Figure 3 It is a schematic diagram of a control loop in a self-stabilizing mode in a specific application example of the present invention.

[0046] Figure 4 It is a schematic diagram of a control loop in a semi-self-stabilizing mode in a specific application example of the present invention.

[0047] Figure 5 It is a schematic diagram of a control loop in full manual mode in a specific application example of the present invention.

[0048] Figure 6 It is a schematic diagram of linear position tracking in a position tracking control mode in a specific application example of the present invention.

[0049] Figure 7 It is a schematic diagram of curve position tracking in the position tracking control mode in a specific application example of the present invention.

[0050] Figure 8 It is a schematic diagram of arc path tracking in a position tracking control mode in a specific application example of the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] The multi-unmanned system distributed simulation method and system of the present invention can provide simulation services including dynamics, collision interaction, damage simulation, graphics rendering and communication interaction functions for the control, planning and distributed collaborative control verification of multi-unmanned systems.

[0053] like Figure 1 and Figure 2 As shown, the multi-unmanned system distributed simulation system of the present invention includes a distributed virtual simulation network, a perception and control subnetwork and a DDS-based collaborative communication service network. The above networks constitute a three-layer distributed simulation network; the present invention decomposes and corresponds the distributed simulation tasks of the multi-unmanned system according to the three logical concepts of virtual simulation, control perception and task collaboration.

[0054] The present invention further connects the nodes in different perception and control subnetworks with the collaborative communication service of DDS, so that it can provide communication middleware services for the situation sharing and task collaboration modules distributed in various perception and control subnetworks according to the task grouping of the unmanned system.

[0055] In a distributed virtual simulation network, the present invention uses distributed simulation technology to deploy virtual simulation tasks of multi-unmanned system task collaboration on distributed nodes; among them, the server is used to provide six services: rule arbitration and judgment, map scene management, real-time state synchronization, multi-point session management, intelligent simulation and real-time situation sharing; the client is used to provide local computing functions including dynamics, collision interaction, damage simulation and graphics rendering.

[0056] In a specific application instance, the virtual integrated server in the distributed virtual simulation network provides multiple services to the client; the services include: ① rule arbitration and judgment service; ② map scene management service; ③ real-time status synchronization service; ④ multi-point session management service; ⑤ intelligent blue team simulation service; ⑥ real-time situation sharing service.

[0057] In a specific application example, the client in the distributed virtual simulation network can provide three access methods, including: 1) human-in-the-loop operation simulation method; 2) intelligent unmanned system algorithm simulation method; 3) simulation deduction guidance and control method.

[0058] In a specific application instance, the client in the distributed virtual simulation network provides three types of local calculations, namely dynamic simulation, collision interaction calculation and graphics rendering, for the virtual unmanned system objects with bound permissions, and provides two types of interfaces, namely sensor interface and planning control interface of the unmanned system objects, to the perception and control subnetwork.

[0059] In a specific application example, the perception control subnetwork maps interaction permissions to the AR / VR head-mounted display device and the human-machine device of the joystick in the human-in-the-loop mode according to the access mode, or maps it to the software-in-the-loop (SITL) or hardware-in-the-loop (HITL) algorithm platform in the intelligent unmanned mode.

[0060] In a specific application example, the DDS-based collaborative communication service network provides communication middleware services for situation sharing and task collaboration modules distributed in various perception and control subnetworks based on the task grouping of unmanned systems. Collaborative communication can be forwarded through a virtual network as needed.

[0061] In a specific application example, the present invention further designs and implements the perception control subnetwork and hardware driver interface of the human-in-the-loop operation simulation mode, the intelligent unmanned system algorithm simulation mode and the simulation deduction guidance and control mode according to the three access needs of human-in-the-loop, intelligent algorithm and simulation guidance and control.

[0062] In a specific application example, the distributed virtual simulation network includes a virtual environment integrated server and a virtual environment client.

[0063] The virtual environment integrated server can be deployed independently or one of the distributed hosts can serve as both the server and the client. The virtual environment integrated server is used to provide six services to the distributed virtual simulation network: rule arbitration, map scene management, real-time state synchronization, multi-point session management, intelligent blue team simulation, and real-time situation sharing. Among them:

[0064] The rule arbitration decision is processed by the virtual environment integrated server in consideration of consistency, and is mainly used for mission simulation involving multiple unmanned systems.

[0065] The selection and switching of map scenes in the map scene management are all uniformly controlled by the virtual environment integrated server. When the virtual environment integrated server chooses to load the map, all clients participating in the current task will load the same virtual scene map at the same time. After all clients have completed the pre-loading of the map, the virtual environment integrated server will automatically complete the confirmation and then enter the virtual scene synchronously. Further, as a preferred embodiment, the map scene management includes consistency management of map scene data. The map scene data includes terrain, buildings, weather, and virtual entity objects controlled by the server, as well as virtual concept objects of the default initial generation position of the character, and the assignment and association of dynamic blue side (non-player character) virtual entity objects.

[0066] As a preferred solution, large-scale map scenes in the map scene management adopt partitioned synchronous management. The large-scale map scene can be synchronously managed by multiple cores of the virtual environment integrated server. In order to reduce memory usage, the map can be divided into several sub-levels according to the needs of the actual application. The client's local map service dynamically loads sub-levels based on relevance and unloads them when not needed. Any changes made by the client to any object in the map scene (sub-level) will be temporarily saved by the map scene management of the virtual environment integrated server and synchronized to all clients.

[0067] In the present invention, real-time state synchronization means that the virtual environment integrated server will perform synchronization of entity object related content in each frame. In a specific application example, the present invention adopts the following process:

[0068] Step S1: Obtain all client connections connected to the virtual environment integrated server. First, obtain the maximum number of connections that the engine can synchronize per frame. If the number exceeds this limit, no connections will be maintained.

[0069] Step S2: Find the entity objects to be synchronized. Only the entity objects that are placed in the network synchronization object list will be considered.

[0070] Step S3: Find the character controlled by the client (usually bound to the virtual camera). The position of this character is the key factor in determining whether other physical objects are synchronized to the client;

[0071] Step S4: Verify the entity object. Objects that are to be destroyed or have empty ownership will not be synchronized.

[0072] Step S5: Check whether the synchronization time of the entity object has arrived. Usually, the synchronization frequency is set when the object is defined. Before each synchronization, the next synchronization time is calculated. If the time is not reached, the synchronization is abandoned.

[0073] Step S6: If the object is set to be related only to the control client, it will be placed in a special list and synchronized only to the client that belongs to it;

[0074] Step S7: Entity objects in the dormant state will not be synchronized, and those about to enter the dormant state must also be specially processed to close the synchronization channel;

[0075] Step S8: Check whether the current object has a channel. If not, check whether the object has been loaded into the scene. If not, skip the synchronization step. If it is in the scene, check whether it is network-related. Entities that are invisible or too far away will not be synchronized.

[0076] Step S9: The number of Actors to be synchronized may be very large, so it is necessary to prioritize all objects. Usually, the priority of the current object is calculated based on whether there is a control terminal and whether the distance is within the field of view. The higher the priority, the earlier the synchronization.

[0077] In the present invention, multipoint session management includes the following processes:

[0078] Step S10: The virtual environment integrated server first needs to create a session in the virtual simulation network, set the session name, the number of participating clients, and the connection method, whether it is a local area network or the Internet. After clicking Create, the virtual environment integrated server will enter the preparation interface.

[0079] Step S20: After the virtual environment integrated server is created, other clients can find the session on the virtual environment integrated server search interface and join the previously created session by clicking Join. Within the same local area network, service discovery can be used to find all virtual environment integrated servers with established sessions. Different virtual environment integrated servers can be distinguished by their name, map type, and other criteria.

[0080] Step S30: After joining the session, select a role. Each unmanned system or type of unmanned system is considered a role. The name and selected role type of each connected client are displayed in a list for easy management. The virtual environment integrated server also has the function of forcibly kicking a client. Finally, the virtual environment integrated server initiates the simulation process, and all clients enter the scene synchronously.

[0081] The intelligent object simulation can be used to provide physical objects with certain behavior patterns that can react to the environment and unmanned system targets. For example, the present invention uses a combination of behavior trees, blackboards, and behavior modules.

[0082] In specific application examples, real-time situation sharing is a service interface provided by the virtual environment integrated server, allowing access to other situation systems. Once connected, the spatial location and status of objects can be published to other situation systems. This function is a reserved interface that can be used to connect physical systems to form a semi-physical simulation environment, or to integrate other virtual systems into a larger simulation platform. The specific data interface can be displayed through the client's navigation window function.

[0083] In a specific application example, the client in the distributed virtual simulation network is used to provide three access methods: human-in-the-loop operation simulation, intelligent unmanned system algorithm simulation, and simulation-based guidance and control. Furthermore, the client provides three types of local computing for bound virtual unmanned system objects: dynamics simulation, collision interaction calculation, and graphics rendering. It also provides two types of interfaces to the perception and control subnetwork: the sensor interface and the planning and control interface of the unmanned system object.

[0084] In specific applications, unmanned system models include, but are not limited to, drones and autonomous vehicles. Input can be provided through the simulator or through the perception and control subnetwork, allowing the Python client to interact with the client in the virtual simulation environment. Input is mainly divided into two categories: motion control input and task execution input.

[0085] Table Input mapping table of unmanned system control channel

[0086]

[0087] In a specific application example, the perception control subnetwork maps interaction permissions to the AR / VR head-mounted display device and the human-machine device of the joystick in the human-in-the-loop mode according to the access mode, or maps it to the software-in-the-loop (SITL) or hardware-in-the-loop (HITL) algorithm platform in the intelligent unmanned mode.

[0088] In specific application examples, hardware-in-the-loop (HIL) primarily connects to a virtual network client via a network connection or serial port, implementing topic-based state / command forwarding via a message bridge (ROS-Integration). Typically, it connects directly to a host computer, with the virtual simulation client performing the entity's dynamics simulation and forwarding the state from the host computer to the unmanned platform's slave computer. The slave computer, operating in HIL mode, then uploads the filtered state to the host computer via a hardware interface for algorithmic processing.

[0089] In specific application examples, the software-in-the-loop method is mainly implemented in Python. By integrating the DDS function package, the distributed nodes of the collaborative algorithm are implemented on Python. At the same time, the software side obtains the corresponding perception and control authority of the virtual object status / instructions from the client's interface service (API-server) through remote procedure call (RPC).

[0090] In specific application cases, AR / VR headset access primarily serves human-in-the-loop simulation training. Through streaming, a visual data stream is generated by the client and streamed to a display device such as a monitor or AR / VR headset. Specifically, in the case of AR / VR, the position and posture data of the headset's camera, along with the video stream from the AR device's camera, are transmitted back to help generate a more integrated simulation training experience.

[0091] In a specific application example, the DDS-based collaborative communication service network provides communication middleware services for situation sharing and task collaboration modules distributed in various perception and control subnetworks based on the task grouping of unmanned systems. Collaborative communication can be forwarded through a virtual network as needed.

[0092] Taking drone control as an example, in a specific application example, the present invention further adopts a drone dynamics simulation algorithm:

[0093] The Newton-Euler formula is used to model the rigid body dynamics of the UAV. The UAV's rigid body analysis can be divided into the translational motion of the base point and the rotational displacement around an axis on the base point:

[0094]

[0095] The origin of the rigid body coordinate system (body coordinate system) is the center of mass of the rigid body, and the translation part is the net external force acting on the rigid body, where is momentum, is the linear velocity, m is the mass of the rigid body; for the rotating motion part, is the resultant moment acting on the rigid body, is the angular momentum, is the angular velocity of the rigid body, and J is the moment of inertia of the rigid body.

[0096] The net external force (ground coordinate system) mainly includes the lift generated by the rotor Air resistance caused by movement gravity The resultant moment (body coordinate system) mainly includes the lift moment M T , inertia counter torque M D , gyroscopic torque (negligible).

[0097] Among them, F i = T ρn i 2 D 4 ,i are the lift generated by the corresponding rotor rotation, n i Represents the rotation speed of the i rotor.

[0098]

[0099] Under the ground system, the air resistance of the drone is Proportional to the square of the flight speed, the projected area S in the plane perpendicular to each motion axis yz ,S xy S yx are proportional to each other, is the translational resistance coefficient in the corresponding direction, the resistance can be expressed as:

[0100]

[0101] In the geodetic coordinate system, the local gravitational acceleration is approximately g = g0 (1-2h / R), and the gravity acting on the body is expressed as:

[0102]

[0103] The combined formula gives the resultant force acting on the UAV in the ground coordinate system:

[0104]

[0105] In the body coordinate system, the rotational torque is described. Under simple assumptions such as the rigid body and the rotor axis are perfectly parallel, M is used. Tx ,M Ty ,M Tz Indicates the lift moment on the body at X b ,Y b ,Z b The components on the axis can be expressed in matrix form as follows:

[0106]

[0107] in are the projection vectors of the x and y axes of the rotor i in the body coordinate system, and the inertial anti-torque is M Di =C P ρF i 2 D 5 / (2pi), which is proportional to the square of the rotational speed.

[0108] Furthermore, the drone of the present invention adopts four control modes:

[0109] The first mode: self-stabilizing mode control law;

[0110] See also Figure 3 In the self-stabilizing mode, the control rate of the FPV includes two levels of control: the inner and outer loops. The outer loop is the position control loop, and the inner loop is the attitude control loop. The input of the control algorithm is the desired speed. and yaw rate The output is the speed of the four rotor motors (n1, n2, n3, n4). The position control loop is designed with three independent PID controllers to obtain the desired acceleration:

[0111]

[0112] in, are the velocity information fed back by the onboard sensor (virtual inertial measurement unit). The desired roll angle θ is calculated by the simultaneous solution r , pitch angle and throttle U:

[0113]

[0114] The inner loop contains the PID control loop for the attitude:

[0115]

[0116] in, θ s ,ψ s The throttle thrust U and the desired attitude angular acceleration calculated by the position control loop are inverted by the formula to calculate the desired motor speed n. i , and then distribute the power and output it to the motor control module.

[0117] The second mode: semi-self-stabilizing mode control law;

[0118] See also Figure 4 , semi-stabilized mode (also known as stunt mode), the control flow chart of this mode is shown in the figure, which only has the attitude angular velocity control loop. Compared with the self-stabilized mode, the difference is that there is no outer loop position control loop, and the attitude control loop uses the attitude angular velocity PID feedback control shown in the formula. The input channel definition reference table shows that the input corresponding to channel 0 of the remote control joystick is the pitch angular velocity, the input corresponding to channel 1 is the roll angular velocity, the input corresponding to channel 2 is the yaw angular velocity, and the input corresponding to channel 3 is the throttle amount. Therefore, the input of the entire control algorithm is The input of the motor control is also inverted by the formula and the throttle thrust U, which can be interpreted as the speed of the four rotor motors (n1, n2, n3, n4). The rotor speed is then controlled by the motor control module.

[0119]

[0120] The attitude response of semi-stable mode is faster and more sensitive than that of self-stable mode, but its positional stability is not as good as that of self-stable mode. If the control input maintains a continuous angular velocity input, the drone will continue to move around the axis, performing stunts such as flips.

[0121] The third mode: attitude tracking mode control law;

[0122] See also Figure 5 In this mode, the aircraft moves in proportion to the joystick movement in remote control mode. Manipulating the right joystick in any direction will cause the aircraft to tilt to the corresponding angle. Returning the joystick to the center position will return the aircraft to its original attitude. Sometimes, multiple adjustments to the joystick movement are required to return the aircraft to its original attitude.

[0123] In full manual mode, as shown in the figure, the values ​​of the remote controller's pitch channel, yaw channel, and roll channel do not pass through the position control loop, but only the attitude angle control loop. The reference input is first-order forward differentiated and the desired yaw rate Enter the attitude control loop (which is consistent with the inner attitude angle control loop of the self-stabilizing control mode). The input of the control algorithm of this mode is θ, and throttle thrust U1, and the output is the speed of the four motors (n1, n2, n3, n4).

[0124] The fourth mode: position tracking control mode;

[0125] See also Figure 6-Figure 8 The present invention adopts the L1 path following algorithm. The basic idea of ​​the L1 path following algorithm is to select a reference point on the desired trajectory and use it to generate a lateral acceleration. The acceleration is expressed as,

[0126]

[0127] The UAV flies an arc close to the desired trajectory under the influence of lateral acceleration. The formula is easy to derive. It can be seen that the acceleration is related to the current airspeed, the angle between the airspeed and the desired point L1, and the distance between the UAV and the desired point L1. Since the airspeed can be measured, solving for the lateral acceleration mainly requires determining the length of L1 and solving for the angle η.

[0128] For a straight line expected path:

[0129]

[0130] The solution to the expected acceleration of the arc is mainly some geometric deductions. The specific derivation can be seen in the paper. It mainly involves three angles: the angle η2 between the airspeed and the tangent at the point, the angle η1 between the line connecting the point to L1 and the chord corresponding to L1, and 2η3, which is equivalent to the central angle of the circle corresponding to L1.

[0131]

[0132] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A distributed simulation method for multiple unmanned systems, characterized in that: include: The distributed simulation task of multiple unmanned systems is decomposed into a distributed virtual simulation network, a perception and control sub-network, and a DDS-based collaborative communication service network according to the three logical concepts of virtual simulation, control perception, and task collaboration. Deploy virtual simulation tasks for multi-unmanned system task coordination on distributed nodes; The integrated server of the distributed virtual simulation network provides rule arbitration, map scene management, real-time state synchronization, multi-point session management, intelligent simulation, and real-time situation sharing services. The client of the distributed virtual simulation network is used to provide local computing functions including dynamics, collision interaction, damage simulation, and graphics rendering. The selection and switching of map scenes in the map scene management are all controlled by the virtual environment integrated server; when the virtual environment integrated server selects to load a map, all clients participating in the current task will load the same virtual scene map at the same time. After all clients have completed the map pre-loading, the virtual environment integrated server will automatically complete the confirmation and enter the virtual scene synchronously; The map scene management includes consistency management of map scene data; the map scene data includes terrain, buildings, weather, virtual entity objects controlled by the server, virtual concept objects of the character's default initial generation position, and the assignment and association of dynamic non-player character virtual entity objects.

2. The multi-unmanned system distributed simulation method according to any one of claim 1, characterized in that: The process of real-time status synchronization includes: Step S1: Obtain all client connections connected to the virtual environment integrated server. First, obtain the maximum number of connections synchronized per frame by the engine. If the number exceeds this limit, no connection will be maintained. Step S2: Find the entity objects to be synchronized. Only the entity objects that are placed in the network synchronization object list will be considered. Step S3: Find the role controlled by the client; Step S4: Verify the entity object. Objects that are to be destroyed or have empty ownership will not be synchronized. Step S5: Whether the synchronization time of the entity object is reached, the synchronization frequency is set when the object is defined, and the next synchronization time is calculated before each synchronization. If this time is not reached, the synchronization will be abandoned; Step S6: If the object is set to be related only to the control client, it is put into a special list and then synchronized only to the client that belongs to it; Step S7: Entity objects in the dormant state will not be synchronized, and those about to enter the dormant state must also be specially processed to close the synchronization channel; Step S8: Check whether the current object has a channel. If not, check whether the object has been loaded into the scene. If not, skip the synchronization step. If it is in the scene, check whether it is network-related. Entities that are invisible or too far away will not be synchronized. Step S9: Arrange all objects in priority order, and calculate the priority corresponding to the current object according to whether there is a control terminal and whether the distance is within the field of view. The higher the priority, the earlier the synchronization.

3. The multi-unmanned system distributed simulation method according to any one of claim 1, characterized in that: The multipoint session management includes the following processes: Step S10: The virtual environment integrated server creates a session in the virtual simulation network, sets the session name, the number of participating clients, and the connection method, whether it is a local area network or the Internet. After clicking Create, the virtual environment integrated server will enter the preparation interface; Step S20: After the virtual environment integrated server is created, other clients find the session on the virtual environment integrated server search interface and click Join to join the session created by the previous client. Step S30: After joining the session, select a role. Each or each type of unmanned system is regarded as a role. The name of each connected client and the selected role type will be displayed in the list. The virtual environment integrated server has the function of forcibly kicking out the client; the simulation process is started by the virtual environment integrated server, and all clients will enter the scene synchronously.

4. The multi-unmanned system distributed simulation method according to any one of claim 1, characterized in that: The real-time situation sharing is a service interface provided by the virtual environment integrated service end to the outside, allowing access to other situation systems, and after access, publishing the spatial position and status of the object to other situation systems; this function is a reserved functional interface, used to connect physical systems to form a semi-physical simulation environment, or to connect to other virtual systems to form a larger simulation platform.

5. A system based on the multi-unmanned system distributed simulation method according to any one of claims 1 to 4, characterized in that: include: The distributed virtual simulation network, the perception control sub-network, and the DDS-based collaborative communication service network constitute a three-layer distributed simulation network. Connect nodes in different perception and control sub-networks with the collaborative communication service of DDS to provide communication middleware services for situation sharing and task collaboration modules distributed in various perception and control sub-networks according to the task grouping of unmanned systems; The virtual simulation task of multi-unmanned system task coordination is deployed on distributed nodes; among them, the virtual integrated server is used to provide rule arbitration and judgment, map scene management, real-time status synchronization, multi-point session management, intelligent simulation and real-time situation services; the client is used to provide local computing functions including dynamics, collision interaction, damage simulation, and graphics rendering; the client is used to provide three access methods: human-in-the-loop operation simulation method; intelligent unmanned system algorithm simulation method; simulation deduction guidance and control method.

6. The multi-unmanned system distributed simulation system according to claim 5, characterized in that: The perception control subnetwork maps the interaction permissions to the AR / VR head-mounted display device and the human-machine device of the joystick in the human-in-the-loop mode, or to the software-in-the-loop (SITL) or hardware-in-the-loop (HITL) algorithm platform in the intelligent unmanned mode according to the access mode.

7. The multi-unmanned system distributed simulation system according to claim 5, characterized in that: The DDS-based collaborative communication service network is used to provide communication middleware services for situation sharing and task collaboration modules distributed in various perception and control subnetworks according to the task grouping of the unmanned system; collaborative communications are forwarded through the virtual network according to the collaborative network.

8. The multi-unmanned system distributed simulation system according to claim 6, characterized in that: The hardware-in-the-loop mode is connected to the virtual network client through a network connection or a serial port, and topic-based status / instruction forwarding is achieved through a message bridge; it is directly connected to the host computer, and the virtual simulation client implements the dynamic simulation of the entity, and forwards the status from the host computer to the lower computer of the unmanned platform. The lower computer then runs in the hardware-in-the-loop mode and uploads the filtered status to the host computer through the hardware interface, and the host computer performs algorithm processing; the software-in-the-loop mode implements the distributed nodes of the collaborative algorithm on Python by integrating the DDS function package, and at the same time, the software end obtains the corresponding perception control authority of the virtual object status / instruction from the client's interface service through remote procedure call.

Images