Virtual-real hybrid simulation system for large-scale unmanned cluster cooperative control

Abstract

The invention relates to the technical field of unmanned cluster cooperative control and simulation, in particular to a virtual-real hybrid simulation system for large-scale unmanned cluster cooperative control, which comprises a task management module, a node module, a motion model module and a communication interface module, the node module comprises a plurality of physical task node modules and virtual task node modules; the task management module receives the state quantity, sends task instruction information and forwards the state quantity; the physical task node module interacts with the neighbor physical task node module and sends the control quantity; the virtual task node interacts with the neighbor task node module and sends a control quantity; the motion model module receives the control quantity, obtains a state quantity and sends the state quantity to the task management module; the communication interface module is used for information communication among the modules; the advantages of a physical simulation system and a software simulation system can be combined, and comprehensive and efficient large-scale cluster simulation and verification are achieved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned cluster collaborative control and simulation technology, and in particular to a virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control. Background Art

[0002] With the continuous development of unmanned swarm technology, unmanned swarm systems have shown great potential in military and civilian fields. These systems can achieve goals that cannot be completed by a single individual through the collaboration of multiple autonomous or collaboratively controlled intelligent agents to perform complex tasks, thus bringing nonlinear performance growth. However, it is a huge challenge to achieve effective simulation and control of these complex tasks. Currently, there are two main methods for the simulation and control research of unmanned swarm systems: systems based on pure physical simulation and systems based on pure software simulation.

[0003] Pure physics-based simulation systems use real hardware devices to simulate the behavior of unmanned swarms. Such systems can more accurately reflect the physical conditions in actual operations, such as environmental interference and hardware limitations. Therefore, they are of great value in testing and verifying the practical application capabilities of unmanned swarms. However, pure physics simulation systems usually face the problem of limited hardware resources, which limits their scalability and makes it difficult to effectively simulate large-scale swarm behaviors. In addition, the debugging and testing cycles of such systems are long and the cost is high, which limits their widespread application in large-scale unmanned swarm simulation.

[0004] Pure software-based simulation systems rely on computer programs to simulate the collaborative behavior of unmanned swarms in a virtual environment. Such systems can flexibly simulate large-scale swarms without being limited by hardware resources, and have advantages in terms of test cycle and cost. However, pure software simulation systems lack direct interaction with the actual physical environment, resulting in possible deviations between simulation results and actual operations in the real environment, which may affect the reliability and accuracy of algorithms and systems in some cases.

[0005] In summary, when it comes to dealing with complex tasks of large-scale unmanned swarm systems, current simulation technologies, both physical-based and software-based simulation systems, have their own advantages and disadvantages and cannot fully meet the simulation requirements for complex unmanned swarm systems. Summary of the invention

[0006] In view of the above problems, the present invention provides a virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control, which solves the technical problem that the physical and software-based simulation systems in the prior art each have their own advantages and disadvantages and cannot fully and efficiently support large-scale cluster simulation and verification.

[0007] On the one hand, the present invention provides a virtual-real hybrid simulation system for large-scale unmanned cluster collaborative control, comprising: a task management module, a node module, a motion model module and a communication interface module, wherein the node module comprises a plurality of physical task node modules and a virtual task node module;

[0008] The task management module is used to receive the state quantity, send task instruction information to the node module and forward the state quantity; the task instruction information is used to control the start and stop of the unmanned cluster collaborative task; the state quantity includes the motion state of each node in the node module;

[0009] The physical task node module is used to simulate the physical UAV node; the physical task node module receives the task instruction information and the observation information of each node, and performs distributed interaction of state quantities with other physical task node modules that are adjacent in space; the physical task node receives the state quantity and calculates the control quantity, and sends the control quantity to the motion model module; the control quantity is the information that controls the motion state of the node module.

[0010] The virtual task node module is used to simulate multiple virtual drone nodes; the virtual task node module receives the task instruction information and the observation information of each node, and performs distributed interaction of state quantities with other node modules adjacent in space; the node module receives the state quantity and calculates the control quantity, and sends the control quantity to the motion model module;

[0011] The motion model module receives the control quantity, performs motion simulation to obtain the state quantity, and sends the state quantity to the task management module; the state quantity includes the motion state of the node module;

[0012] The communication interface module is used for information communication between modules.

[0013] Preferably, the virtual-reality hybrid simulation system for large-scale unmanned swarm collaborative control is used to perform simulation tasks of unmanned swarms, and the simulation tasks include: collaborative navigation and obstacle avoidance tasks, collaborative search planning tasks, collaborative target perception and tracking tasks, task allocation and decision-making tasks, and collaborative guidance tasks.

[0014] Preferably, the operation process of the task management module includes the following steps:

[0015] Step S1-1, the task management module sends all node initial state information to the motion model module, sends task information to the physical task node module and the virtual task node module, and the motion model module sends the state information to the physical and virtual task node modules.

[0016] Step S1-2, the task management module determines whether the current task is finished, and if not, waits and receives the state quantity sent by the motion model module; if finished, the task management module sends a program termination flag to the physical task node module, the physical task node module and the motion model module;

[0017] Step S1-3: If the task management module receives the state quantity sent by the motion model module, the task management module calculates the node neighbor information and the adjacency matrix information according to the state quantity; then the task management module sends the node neighbor information and the state quantity to the physical task node module, and sends the adjacency matrix information and the state quantity to the virtual task node module;

[0018] The node neighbor information records the numbers of other node modules with similar spatial distances to each node module; the adjacency matrix information records the numbers of physical task node modules that the virtual task node module needs to interact with;

[0019] Step S1-4, update the clock information, the clock information indicates the current time, and return to step S1-2.

[0020] Preferably, the operation process of the physical task node module includes the following steps:

[0021] Step S2-1, update clock information, receive node neighbor information, state quantity and program termination flag from the task management module;

[0022] Step S2-2, determine whether the task management module issues a program termination flag, if so, terminate the program, otherwise execute step S2-3;

[0023] Step S2-3, sending its own state quantity information to all neighbor physical task node modules and virtual task node modules, and then waiting to receive the state quantity information of all neighbor physical task node modules and virtual task node modules; the neighbor physical task node module is the physical task node module recorded in the node neighbor information;

[0024] Step S2-4: After receiving the status information of all neighbor node modules, select the control algorithm of the simulation task currently being executed, calculate the control quantity, and send the control quantity to the motion model module, then update the clock information and return to step S2-2.

[0025] Preferably, the operation process of the virtual task node module includes the following steps:

[0026] Step S3-1, update the clock information and determine whether the adjacency matrix information, state quantity and program termination flag are received from the task management module;

[0027] Step S3-2, determine whether the task management module issues a program termination flag, if so, terminate the program, otherwise execute step S3-3;

[0028] Step S3-3, obtaining the number of the physical task node that needs to be interacted with according to the adjacency matrix information received from the task management module, and receiving the state quantity of the physical task node with the corresponding number;

[0029] Step S3-4, after receiving the state quantities of the physical node modules in all the adjacency matrix information, splicing them with the state quantities of the virtual task node modules, the control algorithm of the simulation task currently being executed calculates the control quantities of all the virtual task nodes according to the adjacency matrix information and the spliced ​​state quantities;

[0030] Step S3-5: Send the control quantities of all virtual task nodes to the motion model module, update the clock information at the same time, and return to step S3-2.

[0031] Preferably, the operation process of the motion model module includes the following steps:

[0032] Step S4-1, the motion model module waits to receive the initialization information sent by the task management module, calculates the initial state quantities of all nodes according to the initialization information, and sends it to the task management module;

[0033] Step S4-2, the motion model module waits to receive the control quantity sent by the physical task node module and the virtual task node module, selects the motion model of the simulation task currently being executed, and obtains the state quantity from the control quantity;

[0034] Step S4-3, the motion model module sends the state information of all node modules to the task management module;

[0035] Step S4-4: Update the clock information and return to step S4-2.

[0036] Preferably, the operation process of the motion model module includes the following steps:

[0037] Step S5-1, the virtual task node module and the physical task node generate control quantities and publish them to the ROS topic, the motion model module subscribes to the topic, receives the control quantities and updates the state quantities at the next moment;

[0038] Step S5-2, the motion model module publishes the state quantity to the ROS topic, the task management module subscribes to the topic, displays and records the system status in real time; the virtual task node module and the physical task node module subscribe to the topic for the next moment control quantity calculation;

[0039] Step S5-3, the task management module acts as a publisher and sends the state quantity, node neighbor information and adjacency matrix information to the ROS topic; the virtual task node module and the physical task node module subscribe to the topic and calculate the control quantity at the next moment.

[0040] Preferably, when performing collaborative navigation and obstacle avoidance tasks, the physical task node module and the virtual task node module both calculate control quantities for navigation control and obstacle avoidance control; the motion model module calculates the state quantities of all nodes through the two-dimensional fixed-wing aircraft model based on the received control quantities;

[0041] When performing collaborative navigation and obstacle avoidance tasks, the physical task node module and the virtual task node module receive the waypoint information and state quantity sent by the task management module, and calculate the control quantity based on the waypoint information and state quantity; the motion model module calculates the state quantity of all nodes through the two-dimensional fixed-wing aircraft model according to the received control quantity;

[0042] When executing task allocation and decision-making tasks, the task management module uses an auction algorithm to obtain the task allocation result at the initial moment of the task, and sends the task allocation result to each physical task node and virtual task node;

[0043] When performing collaborative guidance tasks, the physical task node module and the virtual task node module receive neighbor information and state quantities, then receive the estimated strike time of the adjacent nodes based on the neighbor information, calculate the control quantity, and send it to the motion model module; the motion model module calculates the state quantity based on the control quantity through the three-dimensional aircraft model.

[0044] Preferably, it also includes a data storage and management module, in which the storage data types include state quantity storage, control quantity storage, algorithm process data storage and environment variable storage; wherein the state quantity storage specifically includes: node position information, node velocity information and node attribute information; the control quantity storage specifically includes: node acceleration information, node heading angle change rate information; the algorithm process data storage specifically includes: adjacent obstacle information, adjacent node information; the environment variable storage specifically includes: obstacle information, target point information and planned trajectory information.

[0045] On the other hand, the present invention provides a simulation method according to the virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control, comprising the following steps:

[0046] Step S10, determining the simulation task of the current unmanned cluster;

[0047] Step S20, determining the initialization parameter settings of the simulation task in the task management module;

[0048] Step S30, the task management module, the node module, the motion model module and the communication interface module respectively execute the simulation task according to their respective operation processes to obtain simulation results;

[0049] Step S40: The task management module displays the current status of each node module in real time and in a visual manner.

[0050] Compared with the prior art, the present invention has at least the following beneficial effects:

[0051] (1) The present invention adopts a virtual-real combination architecture design. Not only can the present invention significantly expand the simulation scale when hardware resources are limited, but it can also maintain the accuracy and reality of the simulation results, thus overcoming the limitation that pure physical systems cannot be effectively expanded.

[0052] (2) By adopting a distributed communication architecture, the present invention can support real-time data exchange and collaborative control between multiple physical nodes and virtual nodes in a complex environment with limited communication capabilities, thereby improving the adaptability and reliability of the system in large-scale cluster simulation.

[0053] (3) The present invention achieves a high degree of consistency between the physical task node module and the motion model and communication network of the real environment. It can not only quickly verify the cluster algorithm, but also directly apply the verification results to the actual platform, significantly accelerating the process of system development and deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings are only for the purpose of illustrating particular embodiments and are not to be construed as limiting the invention.

[0055] Figure 1 This is a diagram of the distributed communication architecture of the hardware system provided by the present invention.

[0056] Figure 2 A schematic diagram of the structure of a virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control provided by the present invention.

[0057] Figure 3 This is a structural diagram of the storage data type provided by the present invention.

[0058] Figure 4 This is a diagram of the virtual-real combination software architecture provided by the present invention.

[0059] Figure 5 This is a software flow chart of the task management module provided by the present invention.

[0060] Figure 6 This is a software flow chart of the physical task node module provided by the present invention.

[0061] Figure 7 This is a software flow chart of the virtual task node module provided by the present invention.

[0062] Figure 8 This is a software flow chart of the motion model module provided by the present invention.

[0063] Fig. 9 A schematic diagram of the “local clock” message update framework provided by the present invention.

[0064] Fig.10 A schematic diagram of the user interface of the cluster collaborative control semi-physical simulation demonstration system provided by the present invention.

[0065] Fig.11 This is a schematic diagram of initial parameter adjustment for cluster collaborative navigation provided by the present invention.

[0066] Fig.12 A schematic diagram of the task bar provided by the present invention confirming the initial communication establishment.

[0067] Fig.13a , Fig.13b and Fig.13c They are schematic diagrams of the collaborative navigation task management module user interface, the motion model module user interface and the virtual task node module user interface provided by the present invention respectively.

[0068] Fig.14 A schematic diagram of selecting a data playback folder provided by the present invention.

[0069] Fig.15 This is a schematic diagram of the cluster collaborative search task initialization interface provided by the present invention.

[0070] Fig.16a , Fig.16b and Fig.16c They are schematic diagrams of the collaborative search task management module user interface, the motion model module user interface and the virtual task node module user interface provided by the present invention respectively.

[0071] Fig.17a , Fig.17b They are schematic diagrams of the collaborative decision-making task allocation initial interface and the task allocation result interface provided by the present invention respectively.

[0072] Fig.18 This is a schematic diagram of the collaborative tracking task simulation initialization interface provided by the present invention.

[0073] Fig.19a , Fig.19b and Fig.19c Schematic diagrams of the collaborative decision-making task management module user interface, the motion model module user interface and the virtual task node module user interface provided by the present invention respectively.

[0074] Fig. 20This is a schematic diagram of the coordinated strike mission simulation initialization interface provided by the present invention.

[0075] Fig.21a , Figure 21b and Fig.21c Schematic diagrams of the collaborative decision-making task management module user interface, the motion model module user interface and the virtual task node module user interface provided by the present invention respectively. DETAILED DESCRIPTION

[0076] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein, and therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0077] In order to solve the problem of effective simulation and verification of large-scale unmanned cluster systems in complex environments, the present invention provides a virtual-real combined unmanned cluster simulation system, which realizes the simulation and control of large-scale unmanned clusters through the combination of physical task nodes and virtual task nodes. The system adopts a distributed communication architecture to support real-time data exchange and collaborative control between multiple task nodes. Specifically, physical task nodes are used to simulate the behavior of unmanned clusters in real physical environments, while virtual task nodes expand the cluster scale through computing models, thereby realizing the simulation of large-scale clusters under limited hardware resources.

[0078] In order to illustrate the effectiveness of the system provided by the present invention, the above technical solution of the present invention is described in detail below through a specific embodiment. Figure 2 , Figure 4 As shown, a virtual-real hybrid simulation system for large-scale unmanned cluster collaborative control is disclosed, including the following modules: a task management module, a node module, a motion model module and a communication interface module, wherein the node module includes a plurality of physical task node modules and a virtual task node module;

[0079] The task management module is used to receive the state quantity, send task instruction information to the node module and forward the state quantity; the task instruction information is used to control the start and stop of the unmanned cluster collaborative task; the state quantity includes the motion state of each node in the node module;

[0080] The physical task node module is used to simulate the physical UAV node; the physical task node module receives the task instruction information and the observation information of each node, and performs distributed interaction of state quantities with other physical task node modules that are spatially adjacent; the physical task node module obtains the control quantity and sends the control quantity to the motion model module; the control quantity is the information that controls the motion state of the node module.

[0081] The virtual task node module is used to simulate multiple virtual drone nodes; the virtual task node module receives the task instruction information and the observation information of each node, and performs distributed interaction of state quantities with other node modules adjacent in space; the control quantity is obtained based on the state quantity of the node module, and the control quantity is sent to the motion model module;

[0082] The motion model module receives the control quantity, performs motion simulation to obtain the state quantity, and sends the state quantity to the task management module; the state quantity includes the motion state of the node module;

[0083] The communication interface module is used for information communication between modules.

[0084] In some embodiments, Figure 1 As shown, the virtual-reality hybrid simulation system of the present invention may also include a data bus and a data storage and management module. The task management module receives the state quantity and sends the neighbor information to the data bus; the physical task node module receives the control quantity, performs distributed interaction between nodes according to the neighbor information, and sends the state quantity to the data bus; the virtual task node module receives the control quantity and sends the state quantity to the data bus; the motion model module receives the control quantity and sends the state quantity to the bus; the data storage and management module only receives the state quantities of all physical task nodes and virtual task nodes, and organizes and stores them.

[0085] In some embodiments, the control amount and information such as the control amount are stored in the data storage and management module, specifically including: Figure 3 As shown, in the data storage and management module, the storage data types include state quantity storage, control quantity storage, algorithm process data storage and environment variable storage. Among them, the state quantity storage specifically includes: node position information, node speed information and node attribute information; the control quantity storage specifically includes: node acceleration information, node heading angle change rate information; the algorithm process data storage specifically includes: adjacent obstacle information, adjacent node information; the environment variable storage specifically includes: obstacle information, target point information and planning trajectory information.

[0086] In the specific implementation of the present invention, the task management module, the motion model module, the physical task node module, the virtual task node module, and the communication interface module respectively use corresponding hardware devices, and the number and model of the devices are shown in Table 1:

[0087] Table 1

[0088] Modules model Memory processor quantity Task Management Module Lenovo Y9000P 64G i9-14900HX 1 Motion Model Module Lenovo Y9000P 32G i9-13900HX 1 Physical task node module ROC-RK3588S-PC 32G 4Cortex-A76+4Cortex-A55 8 Virtual task node module Lenovo Y9000P 32G i9-13900HX 1 Communication interface module AX9000 Router - Qualcomm IPQ8072A 1

[0089] The software architecture of the virtual-real unmanned cluster simulation system of the present invention realizes the collaborative simulation of large-scale clusters through the virtual-real architecture. The physical task nodes interact with the motion model module through real hardware devices, and the virtual task nodes coordinate with the physical task nodes through software simulation. The system adopts a distributed communication architecture, and realizes real-time data exchange between task nodes through the ROS (Robot Operating System) publish / subscribe model to ensure that the collaborative control and simulation of each node in a complex environment are carried out synchronously. In addition, the task management module manages and schedules the behavior of the entire cluster by combining centralized control with distributed collaboration, thereby realizing efficient simulation and verification of the system.

[0090] The design principles of the virtual-real unmanned cluster simulation system of the present invention are as follows: the physical task node module will be responsible for the control quantity calculation of a single task node; the virtual task node module will simulate multiple physical task nodes and be responsible for their control quantity calculation; the task management module will grasp the global information and further be responsible for partial centralized information processing; the motion model module will simulate the kinematic model of the real platform (calculate state quantity); the interface module will realize efficient information exchange and coordinated control of the cluster collaborative control semi-physical simulation demonstration system.

[0091] The virtual-reality combined unmanned swarm simulation system provided by the present invention is used to simulate tasks such as collaborative navigation and obstacle avoidance, collaborative search planning, collaborative target perception and tracking, and collaborative guidance of unmanned swarms. The operation process of each module of the virtual-reality combined unmanned swarm simulation system of the present invention and its functions in the above tasks are introduced below.

[0092] 1. Task management module

[0093] The task management module grasps the global information and is further responsible for some centralized information processing. The types of information centrally processed in different task links are also different, involving information interaction with the physical task node module, motion model module and virtual task node module.

[0094] (1) Software module operation process

[0095] The software module operation process is as follows Figure 5 The specific details are as follows:

[0096] 1) The task management module sends initialization information and status information to the physical task node module and the virtual task node module, and updates the clock information at the same time.

[0097] 2) The task management module determines whether the current task is completed. If not, it waits and receives the status quantity sent by the motion model module. If it is completed, the task management module sends a program termination flag to the physical task node module, the physical task node module and the motion model module.

[0098] 3) If the task management module receives the state quantity sent by the motion model module, the task management module calculates the node neighbor information and the adjacency matrix information according to the state quantity; then the task management module sends the node neighbor information and the state quantity to the physical task node module, and sends the adjacency matrix information and the state quantity to the virtual task node module;

[0099] The node neighbor information records the numbers of other node modules that are close in spatial distance to each node module; and the adjacency matrix information records the numbers of physical task node modules that the virtual task node module needs to interact with.

[0100] 4) Update the clock information and return to step 2).

[0101] (2) Module task function

[0102] In the task corresponding to the collaborative navigation and obstacle avoidance method based on Reynolds' three principles, individuals need to obtain neighbor information to calculate the control component at the next moment. The task management module here assumes the role of the real platform perception module, that is, the neighbor information of a single platform in the cluster is conveyed by the task management module, and the task management module therefore needs to know the information of all platforms in the cluster.

[0103] In the tasks corresponding to the collaborative search planning method based on parallel scan lines, all physical task node modules need to obtain all waypoints planned by the task management module. Therefore, at the initial moment, the task management module needs to plan the waypoints of all nodes in the collaborative search process according to the environment size, the detection range of the node, and the movement speed of the node, and send them to the physical task node module.

[0104] In the tasks corresponding to the task allocation and decision-making method based on revenue and distributed auction, the task management module undertakes the target allocation and decision-making functions of the ground station. At the initial moment of the task, the auction algorithm is used to obtain the task allocation result and send it to each physical task node and virtual task node.

[0105] In the corresponding tasks of the collaborative target perception and tracking method based on artificial potential field and the distributed collaborative guidance method considering simultaneous attack constraints, the task management module assumes the function of a ground station, that is, the neighbor information of a single platform in the cluster is conveyed by the task management module. Therefore, the task management module also needs to receive the status information of all platforms in the cluster sent by the motion model module, and send the neighbor information to all physical task node modules and virtual task node modules.

[0106] 2. Physical task node module

[0107] The physical task node module integrates the functions of receiving task node state information, interacting with neighbors, and calculating and sending task node control information. It involves information interaction with the task management module, motion model module, and virtual task node module.

[0108] (1) Software module operation process

[0109] The software module operation process is as follows Figure 6 The specific details are as follows:

[0110] 1) Update the clock information and receive node neighbor information, state quantity, obstacle information and program termination flag from the task management module.

[0111] 2) Determine whether the task management module issues a program termination flag. If so, terminate the program; otherwise, proceed to the next step.

[0112] 3) Send its own state quantity information to all neighboring physical task node modules and virtual task node modules, and then wait to receive the state quantity information of all neighboring physical task node modules and virtual task node modules.

[0113] 4) After receiving the state quantity information of all neighbor node modules, select the control algorithm of the current task, calculate the control quantity, and send it to the motion model module, then update the clock information and return to step 2).

[0114] (2) Module task function

[0115] In the task corresponding to the collaborative navigation and obstacle avoidance method based on Reynolds' three principles, individuals need to obtain their own and their neighbors' state quantities to calculate the control components at the next moment. The physical task node module is responsible for calculating the control quantities of a single task node (navigation control component, internal adjustment component, and obstacle avoidance control component) and sends the control quantities to the motion model module.

[0116] In the tasks corresponding to the collaborative search planning method based on parallel scan lines, all physical task node modules first receive the waypoint information planned by the task management module, and then receive the node state quantity sent by the motion model module, calculate the current time step control quantity according to the current time step node state quantity and the next target waypoint, and send the current time step control quantity to the motion model computer.

[0117] In the corresponding tasks of the collaborative target perception and tracking method based on artificial potential field, all physical task node modules first receive the neighbor information sent by the task management module, and then receive the node state quantity sent by the motion model module, calculate the current time step control quantity according to the current time step node state quantity and the neighbor information, and send the current time step control quantity to the motion model computer.

[0118] In the corresponding task of the distributed collaborative guidance method considering simultaneous attack constraints, all physical task node modules first receive the neighbor information sent by the task management module, then receive the node state quantity sent by the motion model module, and then receive the estimated attack time of the adjacent node according to the neighbor information, calculate the control quantity of the current time step and send it to the motion model computer.

[0119] 3. Virtual task node module

[0120] The virtual task node module integrates the functions of receiving all virtual task node state information, generating neighbor sets, and calculating and sending virtual task node control quantities. It involves information interaction with the task management module, motion model module, and physical task node module.

[0121] (1) Software module operation process

[0122] The software module operation process is as follows Figure 7 The specific details are as follows:

[0123] 1) Update the clock information and determine whether the adjacency matrix, state quantity, obstacle information and program termination flag are received from the task management module.

[0124] 2) Determine whether the task management module issues a program termination flag. If so, terminate the program; otherwise, proceed to the next step.

[0125] 3) Determine the required physical task node number according to the adjacency matrix information received from the task management module, and receive the state quantity of the physical task node with the corresponding number.

[0126] 4) After receiving the state quantities of the physical node modules in all the adjacency matrix information, they are spliced ​​with the state quantities of the virtual task node modules. The control algorithm of the simulation task currently being executed calculates the control quantities of all virtual task nodes based on the adjacency matrix information and the spliced ​​state quantities.

[0127] 5) Send the control quantities of all virtual task nodes to the motion model module, update the clock information at the same time, and return to step 2).

[0128] (2) Module task function

[0129] In the task corresponding to the collaborative navigation and obstacle avoidance method based on Reynolds' three principles, individuals need to obtain their own and their neighbors' state quantities to calculate the control components at the next moment. The virtual task node module simulates multiple physical task nodes, takes on the calculation of all virtual task node control quantities (navigation control components, internal adjustment components, and obstacle avoidance control components), and sends the control quantities to the motion model module.

[0130] In the corresponding task of the collaborative search planning method based on parallel scan lines, the virtual task node module first receives the waypoint information planned by the task management module, and then receives the node state quantity sent by the motion model module, calculates the current time step control quantity according to the current time step node state quantity and the next target waypoint, and sends the current time step control quantity to the motion model computer.

[0131] In the corresponding tasks of the collaborative target perception and tracking method based on artificial potential field, the virtual task node module first receives the neighbor information sent by the task management module, then receives the node state quantity sent by the motion model module, exchanges state quantity information with the adjacent physical task nodes, and finally calculates the current time step control quantity based on the current time step node state quantity and neighbor information, and sends the current time step control quantity to the motion model computer.

[0132] In the corresponding task of the distributed collaborative guidance method considering simultaneous attack constraints, the virtual task node module first receives the neighbor information sent by the task management module, then receives the node state quantity sent by the motion model module, and then receives the estimated strike time of the adjacent physical task node according to the neighbor information, calculates the control quantity of the current time step and sends it to the motion model computer.

[0133] 4. Motion model module

[0134] The motion model module integrates the calculation and sending functions of all physical task nodes and virtual task node state quantities, which involves information interaction with the task management module, physical task node module and virtual task node module.

[0135] (1) Software module operation process

[0136] The software module operation process is as follows Figure 8 The specific details are as follows:

[0137] 1) The motion model module waits to receive the initialization information sent by the task management module, calculates the initial state quantities of all nodes according to the initialization information, and sends it to the task management module.

[0138] 2) The motion model module waits to receive the control quantity information sent by the physical task node module and the virtual task node module, and selects the corresponding model to obtain the state quantity information from the control quantity information.

[0139] 3) The motion model module sends the state information of all task nodes to the task management module.

[0140] 4) Update the clock information and return to step 2).

[0141] (2) Module task function

[0142] In the tasks corresponding to the collaborative navigation and obstacle avoidance method based on Reynolds' three principles, the collaborative search planning method based on parallel scanning lines, and the collaborative target perception and tracking method based on artificial potential fields, the motion model module calculates the state quantities of all node modules based on the control quantities (speed change rate, speed direction change rate) sent by the physical task node module and the virtual task node module, and sends them to the task management module, the physical task node module, and the virtual task node module.

[0143] In the corresponding tasks of the distributed collaborative guidance method considering simultaneous attack constraints, the motion model module calculates the state quantities of all node modules based on the control quantities (yaw acceleration and pitch acceleration) sent by the physical task node module and the virtual task node module, and sends them to the task management module, the physical task node module, and the virtual task node module.

[0144] 5. Communication interface module

[0145] Under the condition that all modules are connected to the same local area network, the communication interface module of the present invention adopts a publish / subscribe model based on the Robot Operating System (ROS) to realize efficient information exchange and coordinated control of the cluster collaborative control semi-physical simulation demonstration system. ROS is a flexible software framework with the characteristics of high modularity, strong scalability, and easy integration. The mechanism of topic message update based on the ROS publish / subscribe model is that when a new message is published to the corresponding topic, the message response function of the corresponding topic will be automatically executed, and the data will be processed in the message response function for subsequent use.

[0146] The communication interface module of the present invention adopts the publish / subscribe model of ROS to realize message transmission between nodes through topics. A publisher is a node that publishes messages to a specific topic; a subscriber is one or more nodes that subscribe to a specific topic to receive messages; a topic is a channel for message transmission and has a unique name identifier. The communication interface module processes control instructions, system status and other data and processes as follows:

[0147] 1) Communication interface module processes control instructions

[0148] The virtual task node module and the physical task node generate control quantities and publish them to the corresponding topics; the motion model module subscribes to these topics, receives control instructions and updates the status at the next moment.

[0149] 2) Communication interface module handles system status

[0150] The motion model module publishes the current system status to specific topics; the task management module subscribes to these topics to display and record the system status in real time to facilitate system maintenance and management; the virtual task node module and the physical task node module subscribe to these topics for the calculation of the control quantity at the next moment.

[0151] 3) Communication interface module processes other data

[0152] As a publisher, the task management module sends some global information after calculation and sorting to specific topics; the virtual task node module and the physical task node module subscribe to these topics and calculate the control amount at the next moment according to the corresponding algorithm.

[0153] like Fig. 9 As shown, the cluster collaborative control semi-physical simulation demonstration system of the present invention involves the updating and application of multi-platform data, so the real-time guarantee before data application is crucial. In order to ensure the real-time of data used in the same time period, the cluster collaborative control semi-physical simulation demonstration system sets up a "local clock" message transmission framework, that is, for each platform, a local clock is set inside itself, and when sending data, the current local clock time information is bound to the data itself and sent. When other platforms use this information, the real-time of the entire system can be ensured by comparing the time information corresponding to the information with the local clock time information.

[0154] The present invention also discloses a simulation method according to the virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control, comprising the following steps:

[0155] Step S10, determining the simulation task of the current unmanned cluster;

[0156] Step S20, determining the initialization parameter settings of the simulation task in the task management module;

[0157] Step S30, the task management module, the node module, the motion model module and the communication interface module respectively execute the simulation task according to their respective operation processes to obtain simulation results;

[0158] Step S40: The task management module displays the current status of each node module in real time and in a visual manner.

[0159] Specifically, the present invention discloses a test case of a virtual-real unmanned swarm simulation system that performs collaborative navigation tasks, collaborative search tasks, collaborative decision-making tasks, collaborative tracking tasks, and collaborative strike tasks, and verifies the performance of the system in different mission scenarios. Fig.10 As shown, in the above five test tasks, it is used to control and display the status of each module and the progress of the task. The following describes the test effects of the present invention in different task scenarios in conjunction with the user interface.

[0160] (1) Collaborative navigation task

[0161] In the collaborative navigation task simulation preparation phase, confirm that all devices are connected to the same local area network and that the local ROS environment configuration information is correct. At the beginning of the experiment, Fig.11 As shown, first select the initialization parameter setting in the task management module to adjust the initial parameters such as obstacle position, cluster initial position, and platform number; secondly, select Start Simulation System in the task management module to connect for the first time and establish communication. The specific communication establishment status will be displayed in the task information bar on the left; finally, select Start Simulation in the task management module to display the full process information in real time. The task management module will draw the current state of the cluster in real time. It can be seen from Figure 13(a) that the current cluster can successfully avoid obstacles and reach the navigation end point. Figure 13(b) shows the user interface of the motion model module, which shows the current time step and task execution status of each physical task node module and motion model module. Figure 13(c) shows the user interface of the virtual task node module, from which the state and control quantities of all virtual task nodes can be seen, and the task information box displays "Task stage information has been received, and collaborative navigation tasks are executed."

[0162] (2) Collaborative search tasks

[0163] In the collaborative search task simulation preparation phase, confirm that all devices are connected to the same LAN and that the local ROS environment configuration information is correct. At the beginning of the experiment, as shown, first select the initialization parameter setting in the task management module to adjust the initial parameters such as simulation duration, sensor detection radius, number of platforms, and size of the area to be searched; after selecting Start Simulation in the task management module, the task management module will draw the current state of the cluster in real time. As can be seen from Figure 16(a), the current route can successfully cover the area to be searched, and the nodes can move normally along the route. Figure 16(b) shows the user interface of the motion model module, which displays the current time step and task execution status of each physical task node module and motion model module. Figure 16(c) shows the user interface of the virtual task node module, from which the state and control quantities of all virtual task nodes can be seen, and the task information box displays "Task stage information has been received, and collaborative search tasks are executed."

[0164] (3) Collaborative decision-making tasks

[0165] The collaborative decision-making task does not require communication and interaction between the task management module, physical task node module, virtual task node module and motion model module. The task management module directly allocates tasks based on the target information obtained in the collaborative search task stage, that is, each node can only obtain one assigned target, and one target can be assigned to multiple nodes. Figure 17(a) shows the initialization interface of the task management module before task allocation. After entering the parameters of "cluster formation number" and "assigned target number", as shown in Figure 17(b), select the "start allocation button" to obtain the target allocation result diagram corresponding to each node.

[0166] (4) Collaborative tracking tasks

[0167] In the collaborative tracking task simulation preparation phase, confirm that all devices are connected to the same local area network and that the local ROS environment configuration information is correct. At the beginning of the experiment, Fig.18 As shown, first select the initialization parameter settings in the task management module to adjust the simulation duration, turning radius, target speed, and initial parameters of the platform PID controller; after selecting Start Simulation in the task management module, the task management module will draw the current state of the node in real time. It can be seen from Figure 19(a) that each node successfully completes the tracking of the target for continuous positive turns, and can ensure that the target point is ultimately within the field of view of the node. Figure 19(b) shows the motion model module user interface, which displays the current time step and task execution status of each physical task node module and motion model module. Figure 19(c) shows the virtual task node module user interface, from which the state and control quantities of all virtual task nodes can be seen, and the task information box displays "Task stage information has been received, and collaborative tracking tasks are executed."

[0168] (5) Coordinated strike mission

[0169] In the preparation phase of the coordinated strike mission simulation, all devices are confirmed to be connected to the same LAN and the local ROS environment configuration information is confirmed to be correct. At the beginning of the experiment, Fig. 20 As shown, first select the initialization parameter setting in the task management module to adjust the simulation duration, cruising speed, maximum acceleration and expected guidance time parameters; after selecting to start the simulation in the task management module, the task management module will draw the current state of the node in real time. It can be seen from Figure 21(a) that each node successfully coordinates according to the estimated remaining time of the strike, and it can ensure that the target point is saturated by multiple nodes at the same time. Figure 21(b) shows the user interface of the motion model module, which displays the current time step and task execution status of each physical task node module and motion model module. Figure 21(c) shows the user interface of the virtual task node module, from which the state and control quantities of all virtual task nodes can be seen, and the task information box displays "Task stage information has been received, and the coordinated strike task is executed."

[0170] Although the specific embodiments of the present invention have been described in a specific order, each action or step should be understood to require that such action or step be performed in the specific order shown or in a sequential order, or require that all illustrated actions or steps should be performed to obtain the desired result. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination.

[0171] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control, characterized in that: include: A task management module, a node module, a motion model module and a communication interface module, wherein the node module includes a plurality of physical task node modules and a virtual task node module; The task management module is used to receive the state quantity, send task instruction information to the node module and forward the state quantity; the task instruction information is used to control the start and stop of the unmanned cluster collaborative task; the state quantity includes the motion state of each node in the node module; The physical task node module is used to simulate the physical drone node; the physical task node module receives the task instruction information and the observation information of each node, and performs distributed interaction of state quantities with other physical task node modules that are adjacent in space; the physical task node module receives the state quantity and calculates the control quantity, and sends the control quantity to the motion model module; the control quantity is the information that controls the motion state of the node module; The virtual task node module is used to simulate multiple virtual drone nodes; the virtual task node module receives the task instruction information and the observation information of each node, and performs distributed interaction of state quantities with other node modules adjacent in space; the node module receives the state quantity and calculates the control quantity, and sends the control quantity to the motion model module; The motion model module receives the control quantity, performs motion simulation to obtain the state quantity, and sends the state quantity to the task management module; the state quantity includes the motion state of the node module; The communication interface module is used for information communication between modules.

2. The virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control according to claim 1 is characterized in that: The virtual-reality hybrid simulation system for large-scale unmanned swarm collaborative control is used to perform simulation tasks of unmanned swarms, including collaborative navigation and obstacle avoidance tasks, collaborative search planning tasks, collaborative target perception and tracking tasks, task allocation and decision-making tasks, and collaborative guidance tasks.

3. The virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control according to claim 2 is characterized in that: The operation process of the task management module includes the following steps: Step S1-1, the task management module sends all node initial state information to the motion model module, sends task information to the physical task node module and the virtual task node module, and the motion model module sends the respective state quantities to the physical and virtual task node modules; Step S1-2, the task management module determines whether the current task is finished, and if not, waits and receives the state quantity sent by the motion model module; if finished, the task management module sends a program termination flag to the physical task node module, the physical task node module and the motion model module; Step S1-3: If the task management module receives the state quantity sent by the motion model module, the task management module calculates the node neighbor information and the adjacency matrix information according to the state quantity; then the task management module sends the node neighbor information and the state quantity to the physical task node module, and sends the adjacency matrix information and the state quantity to the virtual task node module; The node neighbor information records the numbers of other node modules with similar spatial distances to each node module; the adjacency matrix information records the numbers of physical task node modules that the virtual task node module needs to interact with; Step S1-4, update the clock information, the clock information represents the current time, and return to step S1-2.

4. The virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control according to claim 3 is characterized in that: The operation process of the physical task node module includes the following steps: Step S2-1, update clock information, receive node neighbor information, state quantity and program termination flag from the task management module; Step S2-2, determine whether the task management module issues a program termination flag, if so, terminate the program, otherwise execute step S2-3; Step S2-3, sending its own state quantity information to all neighbor physical task node modules and virtual task node modules, and then waiting to receive the state quantity information of all neighbor physical task node modules and virtual task node modules; the neighbor physical task node module is the physical task node module recorded in the node neighbor information; Step S2-4: After receiving the status information of all neighbor node modules, select the control algorithm of the simulation task currently being executed, calculate the control quantity, and send the control quantity to the motion model module, then update the clock information and return to step S2-2.

5. The virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control according to claim 4 is characterized in that: The operation process of the virtual task node module includes the following steps: Step S3-1, update the clock information and determine whether the adjacency matrix information, state quantity and program termination flag are received from the task management module; Step S3-2, determine whether the task management module issues a program termination flag, if so, terminate the program, otherwise execute step S3-3; Step S3-3, obtaining the number of the physical task node that needs to be interacted with according to the adjacency matrix information received from the task management module, and receiving the state quantity of the physical task node with the corresponding number; Step S3-4, after receiving the state quantities of the physical node modules in all the adjacency matrix information, splicing them with the state quantities of the virtual task node modules, the control algorithm of the simulation task currently being executed calculates the control quantities of all the virtual task nodes according to the adjacency matrix information and the spliced ​​state quantities; Step S3-5: Send the control quantities of all virtual task nodes to the motion model module, update the clock information at the same time, and return to step S3-2.

6. The virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control according to claim 5 is characterized by: The operation process of the motion model module includes the following steps: Step S4-1, the motion model module waits to receive the initialization information sent by the task management module, calculates the initial state quantities of all nodes according to the initialization information, and sends it to the task management module; Step S4-2, the motion model module waits to receive the control quantity sent by the physical task node module and the virtual task node module, selects the motion model of the simulation task currently being executed, and obtains the state quantity from the control quantity; Step S4-3, the motion model module sends the state information of all node modules to the task management module; Step S4-4: Update the clock information and return to step S4-2.

7. The virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control according to claim 6 is characterized in that: The operation process of the motion model module includes the following steps: Step S5-1, the virtual task node module and the physical task node generate control quantities and publish them to the ROS topic, the motion model module subscribes to the topic, receives the control quantities and updates the state quantities at the next moment; Step S5-2, the motion model module publishes the state quantity to the ROS topic, the task management module subscribes to the topic, displays and records the system status in real time; the virtual task node module and the physical task node module subscribe to the topic for the next moment control quantity calculation; Step S5-3, the task management module acts as a publisher and sends the state quantity, node neighbor information and adjacency matrix information to the ROS topic; the virtual task node module and the physical task node module subscribe to the topic and calculate the control quantity at the next moment.

8. The virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control according to claim 7 is characterized in that: When performing collaborative navigation and obstacle avoidance tasks, both the physical task node module and the virtual task node module calculate the control quantities used for navigation control and obstacle avoidance control; the motion model module calculates the state quantities of all nodes through the two-dimensional fixed-wing aircraft model based on the received control quantities; When performing collaborative navigation and obstacle avoidance tasks, the physical task node module and the virtual task node module receive the waypoint information and state quantity sent by the task management module, and calculate the control quantity based on the waypoint information and state quantity; the motion model module calculates the state quantity of all nodes through the two-dimensional fixed-wing aircraft model according to the received control quantity; When executing task allocation and decision-making tasks, the task management module uses an auction algorithm to obtain the task allocation result at the initial moment of the task, and sends the task allocation result to each physical task node and virtual task node; When performing collaborative guidance tasks, the physical task node module and the virtual task node module receive neighbor information and state quantities, then receive the estimated strike time of the adjacent nodes based on the neighbor information, calculate the control quantity, and send it to the motion model module; the motion model module calculates the state quantity based on the control quantity through the three-dimensional aircraft model.

9. The virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control according to claim 8 is characterized in that: Also includes data storage and management modules: In the data storage and management module, the storage data types include state quantity storage, control quantity storage, algorithm process data storage and environment variable storage; among which, the state quantity storage specifically includes: node position information, node velocity information and node attribute information; the control quantity storage specifically includes: node acceleration information, node heading angle change rate information; the algorithm process data storage specifically includes: adjacent obstacle information, adjacent node information; the environment variable storage specifically includes: obstacle information, target point information and planned trajectory information.

10. A simulation method for a virtual-reality hybrid simulation system for large-scale unmanned cluster collaborative control according to any one of claims 1 to 9, comprising the following steps: Step S10, determining the simulation task of the current unmanned cluster; Step S20, determining the initialization parameter settings of the simulation task in the task management module; Step S30, the task management module, the node module, the motion model module and the communication interface module respectively execute the simulation task according to their respective operation processes to obtain simulation results; Step S40: The task management module displays the current status of each node module in real time and in a visual manner.

Images