Virtual-real combination simulation system and method for unmanned system cluster

Abstract

The invention discloses an unmanned system cluster virtual-real combination simulation system and method, and the system comprises a modeling unit which is used for generating a physical model of an unmanned system cluster; the sensing and positioning unit is used for acquiring model data from the modeling unit and used for predicting, acquiring and processing sensor data so as to determine the position and attitude information of the unmanned system cluster; the communication and message transmission unit is used for realizing data exchange among the units in the system; the algorithm development and verification unit is used for providing an interface to receive and execute a control algorithm and outputting the control algorithm to act on the physical model; and the performance evaluation and analysis unit is used for collecting simulation process data, calculating performance indexes and generating an evaluation report. According to the method, a whole set of tool chains from modeling, perception, communication, algorithm to evaluation are integrated through a virtuality and reality combined simulation system, comprehensive and efficient verification of the cluster algorithm is realized, and the use threshold of a user is remarkably reduced.

Description

Technical Field

[0001] This application relates to the field of testing technology, and in particular to a virtual-real combined simulation system and method for unmanned system clusters. Background Technology

[0002] In the field of unmanned system swarm collaborative control technology, computer simulation and hardware-in-the-loop (HIL) technology are core means to support algorithm development, testing, and teaching verification. Several existing unmanned system swarm simulation platform implementation schemes already exist.

[0003] One example is a simulation platform based on Gazebo, which is widely used in the ROS ecosystem. It provides 3D environment modeling and physics engine support and can simulate the basic dynamic behavior of a single type of unmanned system. Another common approach is a simulation system based on V-REP (now called CoppeliaSim), which supports multi-robot simulation and script control, but focuses on general robot applications and is not specifically optimized for cluster collaboration scenarios.

[0004] Existing technologies have significant technical defects and bottlenecks: High development threshold: Existing platforms rely on high-precision modeling and complex configuration, requiring users to have in-depth ROS knowledge and programming experience. For example, the Gazebo platform requires users to manually write URDF files to define robot models and integrate sensor plugins.

[0005] High operational difficulty; low computational efficiency: When simulating large-scale clusters, the computational complexity of the physical engine of the existing platform for handling multi-body interactions can reach \(O(n^2)\) (n is the number of unmanned systems in the cluster), which causes the simulation running time to increase sharply with the growth of the cluster size, making it difficult to support the verification of complex algorithms in real time. High cost: Commercial simulation solutions (such as V-REP) or customized Gazebo systems require expensive software licenses and high-performance hardware resources, which are far beyond the budget of small and medium-sized research institutions and universities, thus limiting the popularization of the technology; Incomplete validation: There is a lack of integrated algorithm validation and performance evaluation tools, the teaching content is limited, most systems do not support heterogeneous cluster (such as a mixture of drones and unmanned vehicles) simulation, and the algorithm library is updated late, making it difficult to dynamically integrate cutting-edge top journal algorithms, which affects the accuracy and practicality of algorithm development.

[0006] These issues collectively contribute to the current state of unmanned system swarm control, characterized by high development barriers, low accessibility, and incomplete verification, hindering innovation and development in this field. Therefore, there is an urgent need for a low-cost, high-efficiency, low-barrier-to-entry, and comprehensively verified virtual-real hybrid simulation solution for unmanned system swarms. Summary of the Invention

[0007] This application addresses the technical problems existing in the prior art by providing a virtual-real hybrid simulation system for unmanned system clusters, comprising a basic layer, a functional layer, and an application layer. The functional layer includes: Modeling unit, used to generate physical models of unmanned system clusters; The perception and localization unit acquires model data from the modeling unit for prediction and acquires and processes sensor data to determine the position and attitude information of the unmanned system cluster. A communication and message transmission unit is used to realize data exchange between various units within the system; An algorithm development and verification unit is used to provide an interface to receive and execute control algorithms, and to apply the output of the control algorithms to the physical model. The performance evaluation and analysis unit is used to collect simulation process data, calculate performance indicators, and generate evaluation reports.

[0008] Furthermore, the modeling unit constructs an unmanned system model based on the Newton-Euler equations and / or Lagrange mechanics. For the kinematics of the unmanned vehicle, the following model is used:

[0009] in, and The coordinates of the autonomous vehicle's location. For heading angle, For driving speed, Wheelbase This is the steering angle.

[0010] Furthermore, the sensing and localization unit achieves position estimation through an extended Kalman filter (EKF), and its state estimation equation is as follows:

[0011] in, It is the state estimate at time k. It is a control input. These are sensor measurements. It is a measurement function. It is the Kalman gain.

[0012] Furthermore, the positioning accuracy of the location estimation is measured through the error covariance matrix. Quantization, error covariance matrix Calculated based on the following recursive formula: predict:

[0013] renew:

[0014] in: The prediction error covariance matrix at time k based on the state at time k-1; : The state transition matrix at time k-1; The error covariance matrix at time k-1; State transition matrix The transpose of the matrix; The process noise covariance matrix at time k-1; : Identity matrix; : The observation matrix at time k; : The error covariance matrix updated at time k.

[0015] Furthermore, the average message transmission delay of the communication and message transmission unit is evaluated using the following formula:

[0016] in, It is the average delay. It's the number of messages. and These are the sending and receiving timestamps of the i-th message, respectively.

[0017] Furthermore, the algorithm development and verification unit trains the algorithm based on reinforcement learning, and its reinforcement learning reward function... for:

[0018] in, It is the system status. It's an algorithmic action. , , These are weighting coefficients. The reward is for maintaining the formation. It's an obstacle avoidance penalty. It's a reward for completing the task.

[0019] Furthermore, the performance evaluation and analysis unit evaluates the algorithm performance through a quantitative index system, and its comprehensive performance scoring function... for:

[0020] in, It is an error control indicator. It's the task completion rate. It is a consistency indicator. It's about computational efficiency. , , , It is the weighting coefficient.

[0021] Furthermore, the foundational layer includes hardware and software infrastructure to enable the operation of a virtual-real hybrid simulation system for unmanned system clusters. It includes: servers, operating systems, ROS2 communication middleware, GPU acceleration middleware, sensing and positioning hardware, user interaction terminals, and optional controller hardware. Each piece of hardware forms a distributed system through wired or wireless networks.

[0022] Furthermore, the application layer provides a graphical user interface and task wizard function through a user interaction terminal to support users in inputting simulation parameters, adjusting algorithm parameters, monitoring the simulation process, viewing visualization results, and evaluation reports.

[0023] This application also provides a virtual-real combined simulation method for unmanned system clusters, including the following steps: S1: Load simulation parameters and initialize ROS2 nodes and simulation environment; S2: Generate dynamic and / or kinematic physical models of one or more unmanned systems based on the input parameters; S3: Acquire sensor data and calculate the position and attitude information of each unmanned system through a multi-sensor fusion algorithm; S4: Transmit the position and attitude information to the algorithm development and verification unit; S5: Run the user-defined control algorithm, generate control commands based on the received real-time data, and apply them to the physical model to drive the simulation. S6: Collect data during the simulation process and calculate predetermined performance indicators; S7: Generate and output a performance evaluation report, adjust the algorithm or parameters based on the report feedback, and repeat the modeling to evaluation steps until the performance requirements are met.

[0024] In practical applications, the units described in the systems and methods disclosed in this application can be deployed on a single target server, or each unit can be deployed independently on different target servers. In particular, as needed, to provide more powerful computing capabilities, the units can also be deployed on a cluster of target servers.

[0025] Therefore, the virtual-real hybrid simulation system for unmanned system clusters proposed in this application divides the system into a basic layer, a functional layer, and an application layer, constructing a novel virtual-real hybrid simulation system for unmanned system clusters. It decomposes and standardizes complex simulation tasks. This virtual-real hybrid simulation system integrates a complete toolchain from modeling, perception, communication, algorithms to evaluation, realizing comprehensive and efficient verification of cluster algorithms, significantly reducing the user's learning curve, and enabling users without in-depth ROS knowledge to get started quickly.

[0026] To provide a clearer and more comprehensive understanding of this application, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of the functional layer structure of an unmanned system cluster virtual-real combined simulation system according to an embodiment of this application.

[0029] Figure 2 This is a schematic diagram of the simulation process of an unmanned system cluster virtual-real combined simulation system according to an embodiment of this application. Detailed Implementation

[0030] This application adopts a "three-layer, five-unit" architecture design, including a basic layer, a functional layer, and an application layer, and constructs a novel virtual-real hybrid simulation system for unmanned system clusters. It decomposes and standardizes complex simulation tasks. This virtual-real hybrid simulation system integrates a complete toolchain from modeling, perception, communication, algorithms to evaluation, and supports the dynamic integration of heterogeneous clusters and cutting-edge algorithms. It achieves comprehensive and efficient verification of cluster algorithms, significantly reduces the user's learning curve, and enables users without in-depth ROS knowledge to get started quickly.

[0031] The unmanned system cluster in this application, also known as the unmanned machine cluster, refers to a cluster of machines composed of multiple single units that can operate autonomously without real-time human control. It generally refers to various clusters of multiple machines in the sea, land, and air, such as unmanned ships, AGV unmanned vehicles, and unmanned aircraft, which can work together in a certain space or scenario.

[0032] The technical solution of this application is: a virtual-real hybrid simulation system for an unmanned system cluster, comprising a basic layer, a functional layer, and an application layer, wherein the functional layer includes: Modeling units (i.e., modeling units for swarm dynamics and kinematics) are used to generate physical models of unmanned system swarms. The perception and localization unit acquires model data from the modeling unit for prediction and acquires and processes sensor data to determine the position and attitude information of the unmanned system cluster. A communication and message transmission unit is used to realize data exchange between various units within the system; An algorithm development and verification unit is used to provide an interface to receive and execute control algorithms, and to apply the output of the control algorithms to the physical model. The performance evaluation and analysis unit is used to collect simulation process data, calculate performance indicators, and generate evaluation reports.

[0033] The foundational layer of this application constitutes the physical and software basis for system operation, and mainly includes: High-performance servers: As the core computing unit, they employ multi-core CPUs and GPUs (such as NVIDIA series) to run simulation engines (Pybullet) and various algorithms.

[0034] Operating System: A domestically developed operating system (such as Linux-based Kylin OS or UnionTech UOS) is used to provide a stable operating environment.

[0035] ROS2 communication middleware: Deployed on servers and networks, it enables real-time and reliable communication between distributed nodes, using the DDS (Data Distribution Service) protocol.

[0036] GPU-accelerated middleware, such as CUDA, is used to accelerate physics calculations and graphics rendering in the Pybullet engine.

[0037] Sensing and positioning hardware includes the Apriltag visual tagging system, lighthouse positioning base station, camera, IMU (inertial measurement unit), etc., which are connected to the server via USB or Ethernet.

[0038] User interaction terminal: usually a PC workstation, connected to the server via a network (wired / wireless), running a graphical user interface (GUI).

[0039] It may also include controller hardware: for hardware-in-the-loop (HIL) simulation, such as a real driverless car controller, which interacts with the server via serial port or CAN bus.

[0040] These components form a distributed system via a network, with the server acting as the central node, responsible for coordinating all computing and communication tasks.

[0041] The application layer of this application is implemented through a graphical user interface (GUI) on a user interaction terminal. It can be developed using Qt or Web technologies, serving as the window for user interaction with the entire simulation system, providing a graphical user interface, and mainly including the following functional modules: Project / Scene Management: Create new projects and select or edit simulation environments (such as warehouses or neighborhoods).

[0042] Parameter configuration panel: Allows users to set simulation parameters (number, model, and environment type of unmanned vehicle), sensor parameters, and algorithm parameters (such as PID gain and reinforcement learning weights) in the form of forms and sliders.

[0043] Algorithm selection and loading: Select the control algorithm to be tested from the integrated algorithm library or user local files.

[0044] Simulation console: Provides buttons for starting, pausing, stopping, and resetting the simulation.

[0045] Real-time monitoring dashboard: Displays the status, key indicators, and simulation images of the unmanned system in real time in the form of digital displays, dashboards, 2D / 3D views, etc.

[0046] Report Viewer: Displays visual reports and charts generated by the performance evaluation unit.

[0047] The application layer receives user input operations and parameter settings, converts them into system instructions or parameter files, and sends them to the corresponding units on the server. It also receives real-time status data and final evaluation reports from the server (such as the performance evaluation unit) and renders them on the GUI.

[0048] The GUI provided by the application layer encapsulates the complex backend system into simple visual operations, enabling non-professional developers and students to easily use the system for algorithm verification and experimental teaching, truly achieving a "low barrier to entry".

[0049] Please see Figure 1 The functional layer of this application consists of five units, and the functions, implementation methods, and connection relationships of each unit are as follows: Modeling units, or swarm dynamics and kinematics modeling units, are used to generate physical models of unmanned systems (drones, unmanned vehicles, unmanned ships, etc.), support heterogeneous swarm modeling, and provide basic dynamics and kinematic data for simulation.

[0050] The modeling theory is based on the Newton-Euler equations and / or Lagrange mechanics to construct the model. The following model is adopted for the kinematics of the autonomous vehicle.

[0051] in , The coordinates of the autonomous vehicle's location (to be corrected through subsequent positioning). For the heading angle (initial value configured by the user), The driving speed (an unknown quantity, controlled by the algorithm output) Wheelbase (user-configurable, e.g., 0.5m) The steering angle is an unknown quantity, controlled by the algorithm output.

[0052] Real-time physical rendering of multibody dynamics is achieved through the Pybullet engine (such as version 3.2.5). It receives user-configured model parameters (mass, inertia, constraints, such as unmanned vehicle mass of 20kg and friction coefficient of 0.8) and generates corresponding unmanned system models (UAV dynamics model, unmanned vehicle kinematics model, etc.).

[0053] The modeling unit receives model parameters (such as unmanned system type, cluster size, etc.) configured by the application layer user, and then provides the perception and positioning unit with data such as model inertia and constraints.

[0054] The perception and localization unit employs Apriltag visual localization, beacon localization, and multi-sensor fusion technology. Position estimation is achieved through extended Kalman filtering (EKF), and the state estimation equation is:

[0055] in: State estimation at time k (unknown quantities, including position and velocity, such as...) ), This is the state estimate at time k-1 (the result calculated at the previous time). The control input at time k is the speed / steering command output by the algorithm. The Kalman gain at time k is an unknown quantity, calculated from the covariance matrix. The sensor measurement value at time k (such as Apriltag visual coordinates, lighthouse signal). The measurement function (based on sensor characteristics, such as linear mapping) ).

[0056] The goal of positioning accuracy quantization is to minimize the error covariance matrix. traces traces Defined as a matrix Sum of the elements on the main diagonal: ,in: is the dimension of the state vector, representing the sum of the variances of all state estimation variables.

[0057] The recursive calculation is as follows: Prediction steps:

[0058] in: Let k be the prediction error covariance matrix based on time k-1. This is the state transition matrix at time k-1 (describing the evolution of the state from time k-1 to time k, derived based on the dynamics and kinematics model of the unmanned system, such as...). , This is the error covariance matrix at time k-1 (the result of the previous time step update). for The transpose of the matrix, The process noise covariance matrix at time k-1 (quantifying the statistical characteristics of process noise such as system modeling errors and external disturbances, preset according to the simulation scenario and unmanned system type, is pre-defined as follows). ); Update steps:

[0059] in: An identity matrix (with dimensions consistent with the state vector, such as...) ), The observation matrix at time k (describes the linear / nonlinear mapping relationship between the state and the measured value, derived based on sensor characteristics, such as...) ), The error covariance matrix updated at time k (quantized) (Estimation uncertainty).

[0060] The communication and message transmission unit receives position / attitude information from the sensing and positioning unit, control commands from the algorithm development and verification unit, and metric requests from the performance evaluation and analysis unit. Based on the ROS2 distributed communication architecture, it adopts a "publish + subscribe" model, defining topics (e.g., / vehicle pose) and message types (e.g., geometry_msgs / PoseStamped), and then forwards position / attitude information to the algorithm development and verification unit, control commands to the modeling unit, and communication delay data to the performance evaluation and analysis unit.

[0061] Latency assessment: The average message transmission latency is calculated using the following formula (known quantities: Unknown quantity:

[0062]

[0063] in: For average latency (target <5ms), For the number of messages (e.g., 1000), The timestamp of the i-th message (recorded by the server). The timestamp for receiving the i-th message (recorded by the receiver); Reliability guarantee: Supports QoS (Quality of Service) policies (such as "reliable transmission") to avoid data loss.

[0064] The algorithm development and verification unit provides an algorithm development environment and interface, receives and executes control algorithms, generates control commands to act on the physical model, and supports dynamic algorithm updates.

[0065] The algorithm development and verification unit integrates reinforcement learning frameworks (such as the PPO algorithm), provides Python / C++ libraries and API interfaces (such as set_control_command()), and supports the development of formation control, swarm control, obstacle avoidance algorithms, etc. In this embodiment of the application, the formula for designing the reward function of reinforcement learning is as follows:

[0066] in: This refers to the system status (such as the location of the autonomous vehicle and the distance to obstacles). For algorithm actions (unknown quantities, such as speed adjustment instructions). Weighting coefficients (user-configurable, such as 0.4, 0.3, 0.3). Rewards for maintaining formation (e.g., -1 for deviating from the target formation, +2 for achieving the target formation). The penalty for obstacle avoidance is -5 when the distance to the obstacle is less than 1.5m, and 0 when it is safe. The reward for completing the task (e.g., +10 upon reaching the target point); In addition, the algorithm development and verification unit can also support algorithm updates: by integrating cutting-edge top journal algorithms through Git version control, it can support dynamic algorithm replacement.

[0067] The algorithm development and verification unit can significantly lower the barrier to algorithm development by providing a low-code development environment, enhance the learning framework to improve algorithm adaptability, and support cutting-edge technology verification through dynamic update functionality, thereby greatly shortening the algorithm iteration cycle.

[0068] The performance evaluation and analysis unit collects simulation data, calculates performance indicators, generates visual evaluation reports, and supports algorithm optimization and iteration. Its indicator system quantifies algorithm performance through a comprehensive performance scoring function, the formula of which is as follows:

[0069] in: To control error metrics (such as RMSE, Root Mean Square Error, such as 0.12m), This represents the task completion rate (e.g., 100% completion). For example, a formation consistency indicator (e.g., formation retention rate of 0.95). For computational efficiency (e.g., FPS=60). , , , Weighting coefficients (user-configurable, such as 0.3, 0.3, 0.2, 0.2). For comprehensive performance scoring (target value) Preset by the user, such as 15).

[0070] The performance evaluation and analysis unit generates trajectory plots, control error curves, and communication latency curves using visualization tools such as Matplotlib, or displays real-time data through a web interface (such as the Flask framework). This multi-dimensional indicator system enables comprehensive quantification of algorithm performance, and the visual reports intuitively present problems (such as excessive vehicle control error), providing accurate data support for iterative optimization and greatly improving evaluation efficiency.

[0071] Please see Figure 2 Based on the above embodiments, this application provides a virtual-real combined simulation method for unmanned system clusters, the steps of which include: S1: Load simulation parameters and initialize ROS2 nodes and simulation environment; This step involves system initialization and configuration. Simulation parameters are input via the user terminal to initialize the Pybullet simulation engine and GPU acceleration module, and distributed communication nodes are established. Initialization time is determined via… To ensure consistency, among which and These are the startup times for ROS2 and Pybullet, respectively.

[0072] S2: Generate dynamic and / or kinematic physical models of one or more unmanned systems based on the input parameters.

[0073] This step involves loading and modeling the unmanned system, executed by the swarm dynamics and kinematics modeling unit. Based on user input parameters, it generates the unmanned system's dynamics and kinematics model, including mass, inertia, and constraints. This model is then calculated in real-time using the Pybullet physics engine and provided to the perception and localization units.

[0074] S3: Acquire sensor data and calculate the position and attitude information of each unmanned system through a multi-sensor fusion algorithm.

[0075] This step involves setting up the simulation environment and initializing the perception, with the perception and localization unit as the main execution entity. It includes configuring sensors (cameras, IMUs, etc.), initializing the multi-sensor fusion algorithm, and initializing the state vector using Kalman filtering. Covariance It begins collecting environmental data and calculating the precise position and attitude of the unmanned system.

[0076] S4: Based on the ROS2 architecture, the position and attitude information is transmitted to the algorithm development and verification unit.

[0077] This step involves algorithm injection and execution, with the algorithm development and verification unit as the execution entity. The user-selected control algorithm is loaded via the API interface, integrating ROS2 message subscription and publishing functions. The control commands output by the algorithm are sent to the cluster dynamics and kinematics modeling unit through the communication and message transmission unit, and the algorithm's effects are visualized using the Pybullet engine.

[0078] S5: Run the user-defined control algorithm, generate control commands based on the received real-time data, and apply them to the physical model to drive the simulation.

[0079] This step involves real-time monitoring and data acquisition, executed by the performance evaluation and analysis unit. It monitors the simulation's running status, collects performance data such as control error, task completion time, collaboration efficiency, and communication latency, and calculates the comprehensive performance score in real time. And store the data.

[0080] S6: Collect data during the simulation process and calculate the predetermined performance indicators.

[0081] This step involves results analysis and visualization, executed by the performance evaluation and analysis unit and the user interaction terminal. Performance reports and visualization charts (trajectory graphs, indicator curves, etc.) are generated based on the collected data and displayed through the GUI of the user interaction terminal.

[0082] S7: Generate and output a performance evaluation report, adjust the algorithm or parameters based on the report feedback, and repeat the modeling to evaluation steps until the performance requirements are met.

[0083] This step involves feedback optimization and iteration, executed by the entire system. The user adjusts the algorithm parameters based on the evaluation report or automatic system adjustments, repeating steps S1 to S6 to minimize the loss function using gradient descent. Until performance requirements are met, closed-loop optimization is achieved; among which It is the target performance score.

[0084] The application of the technical solution of this application in a certain scenario is illustrated by combining various preferred embodiments and implementation methods.

[0085] This application uses an unmanned vehicle swarm formation control and obstacle avoidance algorithm in a warehouse logistics scenario as an example to simulate four unmanned vehicles (1 master vehicle and three slave vehicles) maintaining a diamond formation and avoiding collisions in the presence of dynamic obstacles.

[0086] Based on the warehouse logistics scenario and the number of vehicles, the system configuration is as follows: Hardware configuration: High-performance server (Intel Xeon 8-core CPU, 32GB RAM, NVIDIA RTX 3080 GPU (10GB VRAM)), Apriltag visual markers (0.1m×0.1m), lighthouse positioning base station, camera, IMU sensor, PC user interaction terminal.

[0087] Software configuration: Domestic Tongxin UOS V20 operating system, ROS2 Humble version, Pybullet 3.2.5 simulation engine, CUDA 11.7 GPU acceleration middleware, Matplotlib visualization tool.

[0088] Based on the above hardware and software configuration, the specific implementation steps of this embodiment are as follows: Step S1: System Initialization (Execution Entity: Server) Create a new project named "WarehouseFleetSim" in the PC terminal GUI and input the parameters (4 unmanned vehicles, warehouse environment, diamond formation, mission start / end point). The server loads Tongxin UOS V20 and ROS2 Humble, and initializes Pybullet 3.2.5. ) and GPU acceleration (CUDA 11.7), establishing distributed nodes ( ),pass Synchronous clock.

[0089] Initialization took 10 minutes, which is 85% more efficient than Gazebo (15 minutes).

[0090] Step S2: Generating the unmanned vehicle model (Execution subject: Modeling unit) The modeling unit loads the differential drive model, configures the parameters (mass 20kg, wheelbase 0.5m, friction coefficient 0.8), generates 4 unmanned vehicle models (master vehicle 1#, slave vehicles 2#-4#) using Pybullet, and imports them into the warehouse environment; Verification: The GUI displays the model's initial posture in real time, and the modeling unit outputs inertial data (such as rotational inertia) to the sensing unit. Modeling took 3 minutes, which is 90% more efficient than manual coding (30 minutes).

[0091] Step S3: Perception and Localization (Executor: Perception and Localization Unit) Configure a camera to capture Apriltag images (0.1m × 0.1m, attached to a warehouse shelf), and an IMU to collect attitude data; initialize the EKF ( , ); Calculate at time k=100. , (Speed ​​1.2 m / s, turning angle 0.05 rad) , Substituting into the EKF equation, we get , , Positioning error 0.03m < 0.05m, update frequency 30Hz.

[0092] Step S5: Algorithm Execution (Execution Entity: Algorithm Development and Verification Unit) Load the swarm control and obstacle avoidance algorithm, and set the reward function parameters. , , ; Algorithm objective: The main vehicle travels along a straight line (20,20), while the slave vehicles maintain a diamond shape (1m spacing), and avoid obstacles when the distance to the obstacle is less than 1.5m; Execution: When vehicle #2 deviates from the target position by 0.2m, The algorithm adjusts its speed to 1.6 m / s; when a dynamic shelf is detected (at a distance of 1.2 m), The algorithm generates steering instructions. Go around it.

[0093] Control commands are sent to the modeling unit through the communication and message transmission unit. Pybullet drives the movement of the physical model, and the GUI displays the formation movement in real time.

[0094] Step S6: Performance Evaluation (Executing Entity: Performance Evaluation and Analysis Unit) Control error (Master vehicle RMSE = 0.08m, slave vehicle average RMSE = 0.12m), Task completion time = 120s ), Coordination and Consistency (formation retention rate) ; Indicator Calculation: ; Then, the master vehicle trajectory diagram (straight line deviation <0.1m), slave vehicle control error curve (stable within 0.15m), and communication delay curve (3.2ms±0.5ms) are generated and displayed through the GUI.

[0095] Step S7: Iterative optimization (Execution subject: the entire system) The assessment report indicated "excessive vehicle control error," and the user made adjustments. (Enhanced formation maintenance reward), optimized obstacle avoidance distance threshold to 1.2m; Iterative optimization: Repeat S2-S6, second simulation loss function After the third adjustment , Once the performance requirements are met, the iteration stops.

[0096] Therefore, the technical effects achieved by this application are as follows: Significantly reduced costs: By adopting open-source ROS2, the lightweight Pybullet engine, and a domestically developed operating system, expensive commercial software licenses are avoided, hardware costs are significantly reduced, and the accessibility of the technology is improved.

[0097] Significantly improves efficiency and accuracy: Modular design and standardized interfaces allow users to get started quickly without needing in-depth ROS knowledge; the Pybullet engine reduces computational complexity; high-precision sensing and positioning units and low-latency communication units ensure the accuracy and real-time performance of simulations.

[0098] Comprehensive verification and easy iteration: The integrated performance evaluation and analysis unit provides quantitative and visual evaluation reports, supports dynamic integration of classic algorithms to cutting-edge top journal algorithms, and forms a complete closed loop of "development-verification-evaluation-optimization", which greatly accelerates the iteration cycle of algorithm research and development and teaching verification.

[0099] It should be noted that those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, which may include, but is not limited to, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0100] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles set forth herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A virtual-real hybrid simulation system for unmanned system clusters, characterized in that, It includes a base layer, a functional layer, and an application layer. The functional layer includes: Modeling unit, used to generate physical models of unmanned system clusters; The perception and localization unit is used to acquire model data from the modeling unit for prediction and to acquire and process sensor data to determine the position and attitude information of the unmanned system cluster. A communication and message transmission unit is used to realize data exchange between various units within the system; An algorithm development and verification unit is used to provide an interface to receive and execute control algorithms, and to apply the output of the control algorithms to the physical model. The performance evaluation and analysis unit is used to collect simulation process data, calculate performance indicators, and generate evaluation reports.

2. The unmanned system cluster virtual-real combined simulation system according to claim 1, characterized in that, The modeling unit constructs unmanned system models based on Newton-Euler equations and / or Lagrange mechanics. For the kinematics of unmanned vehicles, the following model is used: ； in, and The coordinates of the autonomous vehicle's location. For heading angle, For driving speed, Wheelbase This is the steering angle.

3. The unmanned system cluster virtual-real combined simulation system as described in claim 1, characterized in that, The sensing and localization unit achieves position estimation through extended Kalman filtering (EKF), and its state estimation equation is as follows: ； in, It is the state estimate at time k. It is a control input. These are sensor measurements. It is a measurement function. It is the Kalman gain.

4. The unmanned system cluster virtual-real combined simulation system as described in claim 3, characterized in that, The positioning accuracy of location estimation is measured by the error covariance matrix. Quantization, error covariance matrix Calculated based on the following recursive formula: predict: ; renew: ; in: The prediction error covariance matrix at time k based on the state at time k-1; : The state transition matrix at time k-1; The error covariance matrix at time k-1; State transition matrix The transpose of the matrix; The process noise covariance matrix at time k-1; : Identity matrix; : The observation matrix at time k; : The error covariance matrix updated at time k.

5. The unmanned system cluster virtual-real combined simulation system as described in claim 1, characterized in that, The average message transmission delay of the communication and message transmission unit is evaluated using the following formula: ； in, It is the average delay. It's the number of messages. and These are the sending and receiving timestamps of the i-th message, respectively.

6. The unmanned system cluster virtual-real combined simulation system as described in claim 1, characterized in that, The algorithm development and verification unit trains the algorithm based on reinforcement learning, and its reinforcement learning reward function... for: ； in, It is the system status. It's an algorithmic action. , , These are weighting coefficients. The reward is for maintaining the formation. It's an obstacle avoidance penalty. It's a reward for completing the task.

7. The unmanned system cluster virtual-real combined simulation system as described in claim 1, characterized in that, The performance evaluation and analysis unit evaluates algorithm performance through a quantitative index system, and its comprehensive performance scoring function... for: ； in, It is an error control indicator. It's the task completion rate. It is a consistency indicator. It's about computational efficiency. , , , It is the weighting coefficient.

8. The unmanned system cluster virtual-real combined simulation system as described in claim 1, characterized in that, The foundational layer includes hardware and software infrastructure to enable the operation of a virtual-real simulation system for unmanned system clusters. It includes: servers, operating systems, ROS2 communication middleware, GPU acceleration middleware, sensing and positioning hardware, user interaction terminals, and optional controller hardware. The hardware components form a distributed system through wired or wireless networks.

9. The unmanned system cluster virtual-real combined simulation system as described in claim 1, characterized in that, The application layer consists of a graphical user interface and task wizard functions provided through a user interaction terminal, which support users in inputting simulation parameters, adjusting algorithm parameters, monitoring the simulation process, viewing visualization results, and evaluation reports.

10. A virtual-real hybrid simulation method for unmanned system clusters, characterized in that, Including the following steps: S1: Load simulation parameters and initialize ROS2 nodes and simulation environment; S2: Generate dynamic and / or kinematic physical models of one or more unmanned systems based on the input parameters; S3: Acquire sensor data and calculate the position and attitude information of each unmanned system through a multi-sensor fusion algorithm; S4: Transmit the position and attitude information to the algorithm development and verification unit; S5: Run the user-defined control algorithm, generate control commands based on the received real-time data, and apply them to the physical model to drive the simulation. S6: Collect data during the simulation process and calculate predetermined performance indicators; S7: Generate and output a performance evaluation report, adjust the algorithm or parameters based on the report feedback, and repeat the modeling to evaluation steps until the performance requirements are met.

Images