Multi-robot confrontation scene digital twinning system based on Unity

Abstract

The invention provides a multi-robot confrontation scene digital twinning system based on Unity. In the system, an entity robot runs an ROS system and an ROSBridge server side; the Unity engine constructs a virtual confrontation scene and operates the ROSBridge client side; an ROSBridge communication channel between the virtual confrontation scene and each entity robot is established through a wireless local area network; the working modes of the Unity engine comprise an entity confrontation scene only observation mode, a manual intervention control mode and a historical game real machine playback mode, the entity confrontation scene only observation mode realizes'control by real ', the manual intervention control mode realizes'control by real', and the historical game real machine playback mode realizes control by real '. And the historical game real machine playback mode realizes complete and accurate reproduction of the historical game on the entity robot. The method not only can adapt to a high-dynamic multi-robot confrontation scene, but also can assist developers in analyzing and improving software and hardware design of the autonomous intelligent robot through accurate reproduction of historical games.

Description

Technical Field

[0001] The present invention relates to the technical field of digital twins, and more particularly to a digital twin system for multi-robot confrontation scenarios based on Unity. Background Art

[0002] Multi-agent systems involve multiple autonomous intelligent robots that solve complex problems through interaction and cooperation. Among them, the unmanned robot autonomous confrontation scenario has become one of the important research directions of multi-agent systems. However, due to the large number of interacting individuals in the multi-agent system, complex behavior patterns, and dynamic and variable environments, it is difficult to intuitively observe and analyze the operation of robots.

[0003] Digital twin is a new technology that realizes the monitoring and analysis of a physical system by creating a virtual copy of the system. However, the digital twin systems established for autonomous robots at the present stage are usually oriented to industrial production environments, where the operating states of robots are relatively stable, making it difficult to adapt to high-dynamic unmanned confrontation scenarios. Moreover, the reproduction of historical states only appears in the virtual environment, unable to meet the demand for accurate reproduction in the theoretical research and technological development of unmanned confrontation scenarios. Summary of the Invention

[0004] In view of this, the present invention provides a digital twin system for multi-robot confrontation scenarios based on Unity, which can not only adapt to high-dynamic multi-robot confrontation scenarios, but also assist developers in analyzing and improving the software and hardware designs of autonomous intelligent robots through the accurate reproduction of historical matches.

[0005] To solve the above technical problems, the present invention is implemented as follows.

[0006] A digital twin system for multi-robot confrontation scenarios based on Unity includes: a physical confrontation scenario and a virtual confrontation scenario; an obstacle-equipped confrontation field is set in the physical confrontation scenario, and physical robots run on the confrontation field; the physical robots run the Robot Operating System (ROS) and the ROSBridge server; the Unity engine constructs the virtual confrontation scenario and runs the ROSBridge client; a ROSBridge communication channel between the Unity engine and each physical robot is established through a wireless local area network.

[0007] The working modes of the Unity engine include: only observation mode for the physical confrontation scenario, manual intervention control mode, and historical match real machine playback mode.

[0008] In the only-observation mode, the Unity engine performs an observation operation once per frame; the observation operation includes: obtaining the running data of all entity robots, and correspondingly refreshing the visualization interface in the virtual confrontation scenario to update the actions and states of the virtual robots in real time, and at the same time writing the current running data into the historical data log file;

[0009] In the manual intervention control mode, the Unity engine detects the input robot control instructions, resolves them into control quantities, and sends them to one or more entity robots to be controlled to control the running actions of the entity robots. At the same time, the observation operation is performed every frame; after the control ends, it switches back to the only-observation mode;

[0010] In the historical game real machine replay mode, the Unity engine reads the historical data log file, generates continuous control instructions, and sends them to the corresponding entity robots to control the entity robots to reproduce the trajectories and actions in the historical game; at the same time, the observation operation is performed every frame; after the replay ends, it switches back to the only-observation mode;

[0011] After the Unity engine finishes refreshing each frame, it detects whether there is a mode switch.

[0012] Preferably, the Unity engine enters the only-observation mode first after startup.

[0013] Preferably, when the Unity engine receives a manual stop command, it immediately closes the communication connection with the entity robots and exits the virtual confrontation scenario interface.

[0014] Preferably, the mode switch instruction is input by an external device of the computer running the Unity engine; different entity robots correspond to different buttons, and the mode of a single entity robot is switched without affecting the states of other entity robots.

[0015] Preferably, the Unity engine pre-stores a JSON file, which stores the number and serial numbers of entity robots in the startup state in the actual confrontation scenario, the LAN IP addresses, port numbers, affiliated camps of the entity robots, and the names of the entity robots;

[0016] The way for the Unity engine to establish a one-to-many connection with the entity robots is as follows:

[0017] When the virtual confrontation scenario starts running, the Unity engine reads and parses the JSON file through a C# script, performs a prefab instantiation operation, creates a cloned virtual robot for each entity robot, and assigns the affiliated camp, name, and network connection parameters of the robot to the corresponding function modules mounted on the clone;

[0018] The ROSBridge connection script mounted on each physical robot is responsible for creating a new thread, using the Unity engine as a ROSBridge client, and attempting to establish a connection with the ROSBridge server on the physical robot side according to the network connection parameters using the WebSocket protocol;

[0019] The main script of the Unity engine starts running, calls the ROSBridge connection results of each physical robot according to the prefab instantiation parameters, and simultaneously establishes multiple parallel communication connections between the Unity engine and multiple physical robots; when both the physical robot and the Unity engine display successful connection parameters, the establishment of the ROSBridge communication connection between the virtual confrontation scenario and the physical robot is completed.

[0020] Preferably, in the observation-only mode, the Unity engine works as follows:

[0021] The Unity engine subscribes to the ROS messages sent by the physical robot through the script, receives and reads them at a certain frequency, and the ROS messages carry the robot's running data;

[0022] The Unity engine converts the data in the ROS messages into a format and assigns it to the virtual robot in the Update function for refreshing the virtual confrontation scenario;

[0023] When the Unity engine screen is refreshed, the running data of all virtual robots in the new frame will be synchronously updated and displayed in the latest state;

[0024] The Unity engine, according to the status record script, every fixed time interval, concatenates the running data of all physical robots in the physical confrontation scenario into a line of string, adds a timestamp, and writes them into their respective historical data log files.

[0025] Preferably, in the historical game real machine replay mode, the Unity engine works as follows:

[0026] In the initialization stage of the Unity engine, it automatically reads all historical data log files stored in the specified directory and generates a log data list in the chronological order of the recorded content;

[0027] When entering the historical game real machine replay mode, the Unity engine parses the log data list with timestamps and extracts the running data of the physical robots in each log entry in the order of timestamps;

[0028] For each physical robot, the Unity engine receives the running data of the physical robot at the current moment through the ROSBridge communication channel, uses the data in the current log entry as the target state for comparison, takes the difference between the two values, and obtains the control instructions required for the physical robot to run from the current state to the target state in the log entry;

[0029] The Unity engine converts the control instructions into ROS messages and sends them to the corresponding physical robot through the ROSBridge communication channel to control the operation of the physical robot;

[0030] When the next frame is generated, the Unity engine reads the running data of the physical robot at this time through the ROSBridge communication channel and compares it with the target state in the log entry. If the target state is not reached, control continues; if the target state is reached under the condition that the error is less than the set threshold, the target state is changed to the next log entry, and the physical robot is made to reproduce the trajectory and actions in the historical game by generating control instructions according to the target states given in each log entry.

[0031] Preferably, the control instructions generated by the Unity engine according to the log entry include chassis control instructions, pan-tilt control instructions, and shooting control instructions;

[0032] The chassis control instructions include the lateral and longitudinal target linear velocity control quantities of the chassis, the rotation direction and angular velocity control quantity of the chassis;

[0033] The generation method of the chassis angular velocity control quantity is as follows: Read the vehicle head orientation and chassis coordinates when the physical robot reaches the previous target state, take the difference between the orientation and the orientation of the current target state, and record the difference as θ; Connect the coordinates of the previous target point and the coordinates of the current target state, and record the length of the connection as D; Directly calculate the angular velocity control quantity V of uniform rotation ω = θ / T, where T is the log recording time interval;

[0034] The generation methods of the lateral and longitudinal target linear velocity control quantities of the chassis are as follows: Calculate the coordinate difference between the current physical robot coordinates and the target state coordinates to obtain the distances dx and dy that the physical robot should move in its own x-axis and y-axis directions and the included angle α; Use the included angle α to calculate the linear velocity control quantity V in the x-axis direction in real time x and the linear velocity control quantity V in the y-axis direction y as follows:

[0035]

[0036] The angle α changes with the frame refresh, and new chassis control instructions need to be calculated for each frame;

[0037] The generation method of the pan-tilt control command is as follows: The reference coordinate system of the pan-tilt of the physical robot is the chassis coordinate system. When calculating the pan-tilt control command, the difference between the angle of the target state and the current actual angle is divided by the logging time interval T to directly obtain the pan-tilt rotation speed.

[0038] The generation method of the shooting control command is as follows: Calculate the difference in the cumulative number of bullets fired in two adjacent log records as the number of bullets to be fired within the current time interval, and generate the shooting control command.

[0039] Preferably, the operation data of the physical robot includes: the robot camp and name, position coordinates; the chassis movement speed and direction; the pitch angle and yaw angle of the shooting pan-tilt on the robot, and the muzzle shooting state; the remaining blood volume and survival state of the robot.

[0040] Beneficial effects:

[0041] (1) The digital twin system designed by the present invention can not only collect the operation data of the physical robot from the physical confrontation scenario and display it in the virtual confrontation scenario, but also generate control commands according to the historical data log file to control the physical robot to reproduce the trajectory and actions in the historical game, realizing the complete and accurate reproduction of the historical game effect of robot confrontation on the physical robot, and assisting developers to analyze and improve the software and hardware design of autonomous intelligent robots.

[0042] (2) In the state of collecting the operation data of the physical robot and displaying it in the virtual confrontation scenario, it also has a virtual-real two-way interactive control function, that is, a communication server and a client are respectively deployed on the physical robot and the digital twin system to realize virtual-real two-way connection communication; it has the function of "controlling the virtual with the real", that is, after the physical system is deployed, the real-time data feedback can be mapped to the digital twin virtual scenario, facilitating personnel to monitor the system status online in real time; it has the function of "controlling the real with the virtual", that is, the intervention control command input manually can be transmitted in real time and act on the intelligent body in the real environment to realize functions such as instant and flexible control command.

[0043] (3) The present invention designs a solution for establishing a one-to-many ROSBridge communication channel between the Unity engine and multiple robots. The robot is used as the server side, and the Unity engine is used as the client side. Through the client-server architecture, the parallel interaction between the Unity engine and multiple robots is realized, and the operation data is quickly transmitted, so as to meet the requirements of high-dynamic scenarios with large changes in the motion state of the robot, and adapt to the situation that existing robots mostly use the ROS system. Description of the drawings

[0044] Figure 1 It is a schematic diagram of the composition of a digital twin system for multi-robot confrontation scenarios based on Unity of the present invention;

[0045] Figure 2 This is the flowchart of the virtual - real bidirectional interaction function operation of a digital twin system for multi - robot confrontation scenarios based on Unity in the present invention;

[0046] Figure 3 This is a schematic diagram of the calculation method of the chassis control instruction for the historical game replay function of the digital twin system described in the present invention. Specific embodiments

[0047] The following combines the accompanying drawings and gives examples to describe the present invention in detail.

[0048] The present invention provides a digital twin system for multi - robot confrontation scenarios based on Unity. This system can realize the virtual - real bidirectional interaction function. It can not only collect the running data of physical robots from the physical confrontation scenario and display it in the virtual confrontation scenario, but also input instructions in the virtual confrontation scenario to remotely control the physical robots. Moreover, it can generate control instructions according to the historical data log file to control the physical robots to reproduce the trajectories and actions in the historical games.

[0049] Figure 1 Shows the composition of the digital twin system for multi - robot confrontation scenarios based on Unity in the present invention, as Figure 1 shown. This system includes a physical confrontation scenario and a virtual confrontation scenario.

[0050] In the physical confrontation scenario, there is a confrontation field with obstacles, which is used to provide an environment and cover for the autonomous confrontation of robots and restrict the robots from crossing its boundary; physical robots run on the confrontation field; the physical robots run the Robot Operating System (ROS) and the ROSBridge server - side.

[0051] The Unity engine constructs the virtual confrontation scenario and runs the ROSBridge client; the virtual confrontation scenario includes a virtual confrontation field and virtual robots, which are displayed by a visualization interface and are used to dynamically reflect the real - time running situation of the robots in the field. The Unity engine has a virtual scene driver, which is written in C# language and includes a refresh program for the visualization interface and a robot running data processing program, which is used to drive the operation of various elements in the Unity scene.

[0052] The virtual-real bidirectional interaction function requires the support of communication protocols, wireless local area networks, and data sending, receiving, and processing programs. The communication protocol uses the ROSBridge protocol. The wireless local area network is the network carrier for the communication between the physical robot and Unity. Through the wireless local area network, a ROSBridge communication channel is established between the virtual confrontation scenario and each physical robot; the data sending, receiving, and processing program is used to convert the states and control instructions of the physical and virtual robots into formats that conform to the communication protocol, and perform the tasks of sending and receiving data.

[0053] The working modes of the Unity engine include three types: only observation mode, manual intervention control mode, and historical game real machine playback mode.

[0054] In the only observation mode, the Unity engine performs an observation operation once per frame; the observation operation includes: obtaining the running data of all physical robots, and correspondingly refreshing the visualization interface in the virtual confrontation scenario to update the actions and states of the virtual robots in real time, and at the same time writing the current running data into the historical data log file. The only observation mode realizes "controlling the virtual with the real".

[0055] In the manual intervention control mode, the Unity engine detects the input robot control instructions, resolves them into control quantities, and sends them to one or more physical robots to be controlled, controls the running actions of the physical robots, and at the same time performs the observation operation once per frame; after the control ends, it switches back to the only observation mode. The manual intervention control mode realizes "controlling the real with the virtual".

[0056] In the historical game real machine playback mode, the Unity engine reads the historical data log file, generates continuous control instructions, and sends them to the corresponding physical robots to control the physical robots to reproduce the trajectories and actions in the historical game; at the same time, the observation operation is performed once per frame; after the playback ends, it switches back to the only observation mode.

[0057] In the above three working modes, after the system starts, it first enters the only observation mode, and then the Unity engine performs mode switching judgment after each frame is refreshed, and performs corresponding mode switching.

[0058] The following combines Figure 2 To describe in detail the working process of the digital twin system for multi-robot confrontation scenarios based on Unity under this preferred implementation.

[0059] Step 1: On the server side of the robot operating system ROS and the ROSBridge communication protocol running on multiple physical robots, initialize the virtual confrontation scenario in the Unity engine and run the ROSBridge client, and establish ROSBridge communication channels between the virtual confrontation scenario and each physical robot through the wireless local area network.

[0060] After the system is connected, the Unity engine first enters the observation-only mode and performs observation operations.

[0061] Step 2: The Unity engine performs the observation operation once per frame. The observation operation includes: The Unity engine obtains the running data of all entity robots, correspondingly executes the algorithm program in the virtual scene, and updates the actions and states of the virtual robots in the visualization interface in real time to achieve "controlling the virtual with the real", and at the same time writes the current running data into the historical data log file.

[0062] Step 3: The Unity engine detects the mode switching instruction input manually, and selects the corresponding robot running mode according to the instruction: If the observation-only mode is maintained, after one frame is refreshed, return to Step 2 to enter the observation loop; if the manual intervention control mode is selected, enter Step 4; if the historical game real machine replay mode is selected, enter Step 5.

[0063] Preferably, the mode switching instruction can be input by an external device of the computer running the Unity engine. Different keys can be used to switch the mode of a single entity robot respectively without affecting the states of other robots.

[0064] Step 4: In the manual intervention control mode, the Unity engine detects the input robot control instruction, solves it through the control algorithm, and sends it to one or more entity robots to be controlled, controls the running actions of specific entity robots to achieve "controlling the real with the virtual". After each frame is refreshed, maintain the manual intervention control mode, return to Step 2 to perform the observation operation, thus entering the manual control loop. After the control is completed, switch back to the observation-only mode and return to Step 2.

[0065] Specifically, the Unity engine detects the instruction input by the keyboard, mouse (or other external input devices), solves it through the control program, and converts it into specific instructions such as the chassis steering, moving speed, pan-tilt angle, shooting, etc. of the robot, which are transmitted to the entity robot by ROSBridge, and then processed by the entity robot ROS system and converted into the action signals of each drive motor to drive the entity robot to execute the target action, thus realizing the function of controlling the entity scene in the virtual scene.

[0066] Step 5: In the historical game real machine replay mode, the Unity engine reads the running historical data log files of each entity robot previously stored in the preset directory, executes the analysis and processing algorithm, automatically generates continuous control instructions, and sends them to the corresponding entity robots to control the entity robots to reproduce the trajectories and actions in the historical game; after each frame is refreshed, maintain the historical game real machine replay mode and return to Step 2 to perform the observation operation, thus entering the replay loop; when it is detected that the replay process reaches the end, automatically exit the historical game real machine replay mode and re-enter the observation-only mode, and return to Step 2.

[0067] Step 6: If the Unity engine receives an exit command at any time during the above steps, it shall immediately close the communication connection with the physical robot and exit the virtual scene interface; specifically, click the operation to end the Unity scene operation. Unity will first cut off the ROSBridge communication protocol connection, release the network port occupied by the current connection, and then log off the ROSBridge client to end the operation of the virtual scene.

[0068] In a preferred embodiment, the specific steps of the above Step 1 are as follows:

[0069] Step S11: The user inputs the local area network IP address, port number, affiliated camp, and robot name of the robot into a specific JSON file and saves them according to the number and serial numbers of the robots in the startup state in the actual scene.

[0070] After the virtual confrontation scene starts running, a one-to-many ROSBridge connection is established in the manner of Steps S12 - S14.

[0071] Step S12: When the virtual confrontation scene starts running, the Unity engine reads and parses the JSON file through a C# script, performs the prefab instantiation operation, creates a cloned virtual robot for each physical robot, and assigns the affiliated camp, name, and network connection parameters of the robot to the corresponding functional modules mounted on this clone.

[0072] Step S13: The ROSBridge connection script mounted on each robot is responsible for creating a new thread, using the Unity engine as the ROSBridge client, and attempting to establish a connection with the ROSBridge server on the physical robot side using the WebSocket protocol according to the above network connection parameters.

[0073] Step S14: The main script of the Unity engine starts running, calls the ROSBridge connection results of each robot according to the prefab instantiation parameters, and simultaneously establishes multiple parallel communication connections between the Unity engine and multiple physical robots. When the successful connection parameters are displayed on both the physical robot command line terminal and the Unity console, the establishment of the ROSBridge communication connection between the virtual confrontation scene and the physical robot is completed.

[0074] In a preferred embodiment, the specific working steps of Step 2 for the Unity engine are as follows:

[0075] Step S21: The Unity engine subscribes to the ROS messages sent by the physical robot through scripts, receives and reads them at a certain frequency. The content of the ROS messages carries the running data of the robot, including the robot's camp and name, position coordinates, chassis movement speed, chassis orientation angle, pitch angle and yaw angle of the shooting gimbal on the robot, muzzle shooting state, remaining blood volume and survival state of the robot.

[0076] Step S22: Convert the data in the ROS messages and assign them to the virtual robot in the Update function of the Unity scene.

[0077] Step S23: When the Unity engine screen is refreshed, the running data of all virtual robots in the new frame will be synchronously updated and displayed as the latest state.

[0078] Step S24: There is a status recording script in the Unity engine. Every fixed time interval, the running data of all physical robots in the physical confrontation scene will be written into the corresponding historical data log file respectively. When writing, each running data is concatenated into a line of string and a timestamp is added to save a detailed record of the system running process.

[0079] In a preferred embodiment, the specific steps of step five are as follows:

[0080] Step S51: In the initialization stage of the Unity engine, automatically read all historical data log files stored in the specified directory and generate a log data list in the chronological order of the record content.

[0081] Step S52: When entering the historical game real - time replay mode, the Unity engine parses the log data list with timestamps, extracts the running data of the physical robots in each piece of data in the order of timestamps, including information such as position, angle, shooting, and blood volume.

[0082] Step S53: For each physical robot, the Unity engine receives the running data of the physical robot at the current moment through the ROSBridge communication channel, compares it with the data in the current log entry as the target state, calculates the difference between the two values, and through a series of mathematical operations, obtains the control instructions required to run from the current state to the target state of the log entry.

[0083] Step S54: The Unity engine converts the above - mentioned control instructions into ROS messages and sends them to the physical robot through the ROSBridge communication channel to control the running of the physical robot.

[0084] Step S55: After cycling through Step 2 and reaching the next frame generation time, read the operating data of the entity robot at this time through the ROSBridge communication channel, and compare it with the target state in the log entry. If the target state is not reached, immediately repeat Steps S53 and S54; if the target state is reached under the condition that the error is less than the set threshold, change the target state to the next log entry, and then return to Step S53.

[0085] The robot control quantities in Step S53 include chassis control instructions, pan-tilt control instructions, and shooting control instructions. Among them, the chassis control instructions include the chassis lateral and longitudinal target linear velocity control quantities, the chassis rotation direction and angular velocity control quantities; the pan-tilt control instructions are the pan-tilt rotation direction and speed control quantities; the shooting control instructions are the shooting action control quantities. The specific calculation methods for the corresponding control instructions are as follows:

[0086] ① Generation of chassis control instructions:

[0087] As Figure 2 shown, read the vehicle head orientation and chassis coordinates of the robot when it reaches the previous target point, subtract the orientation from the state orientation of the current target, and record the difference as θ; connect the coordinates of the previous target point and the current target point, and record the connection length as D. Since the lateral and longitudinal translations do not affect the chassis orientation, the angular velocity control quantity for uniform rotation can be directly calculated as follows, where T is the time interval recorded in the aforementioned log:

[0088] V ω = θ / T

[0089] Calculate the difference between the current robot coordinates and the target point coordinates to obtain the distances dx and dy and the angle α that the robot should move in the x-axis and y-axis directions in its own IMU carrier coordinate system. Since the actual direction of translation is affected by the rotation state, the linear velocity control quantities in the x and y axes need to be calculated in real time using the current actual angle α as follows:

[0090]

[0091] Here, the preset log recording time interval T is greater than the frame refresh time. The process of running to the target point consists of multiple frames. During this process, the above angle α changes with the frame refresh, while θ and D do not change. Steps ① and ② are repeated for each frame to calculate new control instructions.

[0092] ② Pan-tilt control instructions: The reference coordinate system of the robot pan-tilt is the chassis coordinate system. When calculating the pan-tilt rotation control instructions, subtract the target angle from the current actual angle and divide by the log recording time interval to directly obtain the pan-tilt rotation speed.

[0093] ③ Firing control instruction: Calculate the difference in the cumulative number of bullets fired in two adjacent log record messages as the number of bullets to be fired within the current time interval, and generate a firing control instruction.

[0094] In summary, the above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A digital twin system for multi-robot adversarial scenarios based on Unity, characterized in that, Including: Physical confrontation scenarios and virtual confrontation scenarios; In the physical confrontation scenario, a confrontation field with obstacles is set, and physical robots run on the confrontation field; The physical robots run the Robot Operating System (ROS) and the ROSBridge server side; the Unity engine constructs the virtual confrontation scenario and runs the ROSBridge client; A ROSBridge communication channel between the Unity engine and each physical robot is established through a wireless local area network; The working modes of the Unity engine include: only observation mode for the physical confrontation scenario, manual intervention control mode, and historical game real machine replay mode; In the only observation mode, the Unity engine performs an observation operation once per frame; the observation operation includes: obtaining the running data of all physical robots, and correspondingly refreshing the visualization interface in the virtual confrontation scenario to update the actions and states of the virtual robots in real time, and at the same time writing the current running data into the historical data log file; In the manual intervention control mode, the Unity engine detects the input robot control instructions, resolves them into control quantities, and sends them to one or more physical robots to be controlled to control the running actions of the physical robots, and at the same time performs the observation operation once per frame; after the control ends, it switches back to the only observation mode; In the historical game real machine replay mode, the Unity engine reads the historical data log file, generates continuous control instructions, and sends them to the corresponding physical robots to control the physical robots to reproduce the trajectories and actions in the historical game; at the same time, the observation operation is performed once per frame; after the replay ends, it switches back to the only observation mode; After each frame is refreshed by the Unity engine, it detects whether there is a mode switch.

2. The system according to claim 1, wherein After the Unity engine is started, it first enters the only observation mode.

3. The system according to claim 1, wherein When the Unity engine receives a manual stop command, it immediately closes the communication connection with the physical robots and exits the virtual confrontation scenario interface.

4. The system according to claim 2, wherein The mode switch instruction is input by an external device of the computer running the Unity engine; different physical robots correspond to different keys to perform mode switching on a single physical robot without affecting the states of other physical robots.

5. The system according to claim 2, wherein The Unity engine pre-stores a JSON file, which stores the number and serial numbers of the physical robots in the startup state in the actual confrontation scenario, the local area network IP addresses, port numbers, affiliated camps of the physical robots, and the names of the physical robots; The way for the Unity engine to establish a one-to-many connection with the physical robots is: When the virtual confrontation scenario starts running, the Unity engine reads and parses the JSON file through a C# script, performs a prefab instantiation operation, creates a cloned virtual robot for each physical robot, and assigns the affiliated camp, name, and network connection parameters of the robot to the corresponding function modules mounted on the clone respectively; The ROSBridge connection script mounted on each physical robot is responsible for creating a new thread, taking the Unity engine as a ROSBridge client, and attempting to establish a connection with the ROSBridge server side of the physical robot according to the network connection parameters using the WebSocket protocol; The main script of the Unity engine starts running, calls the ROSBridge connection results of each physical robot according to the prefab instantiation parameters, and simultaneously establishes multiple parallel communication connections between the Unity engine and multiple physical robots; when both the physical robot and the Unity engine display successful connection parameters, the establishment of the ROSBridge communication connection between the virtual confrontation scenario and the physical robot is completed.

6. The system according to claim 2, wherein In the observation-only mode, the Unity engine works as follows: The Unity engine subscribes to the ROS messages sent by the physical robot through a script, receives and reads them at a certain frequency, and the ROS messages carry the robot's running data; The Unity engine converts the data in the ROS messages into a format and assigns it to the virtual robot in the Update function for refreshing the virtual confrontation scenario; When the Unity engine screen is refreshed, the running data of all virtual robots in the new frame will be synchronously updated and displayed as the latest state; According to the status record script, the Unity engine concatenates the running data of all physical robots in the physical confrontation scenario into a line of string every fixed time interval, adds a timestamp, and writes them into their corresponding historical data log files respectively.

7. The system according to claim 2, wherein In the historical game real machine replay mode, the Unity engine works as follows: In the initialization stage, the Unity engine automatically reads all historical data log files stored in the specified directory and generates a list of log data in the order of the recording content; When entering the historical game real machine replay mode, the Unity engine parses the list of log data with timestamps, extracts the running data of the physical robot in each log entry in the order of timestamps; For each physical robot, the Unity engine receives the running data of the physical robot at the current moment through the ROSBridge communication channel, takes the data in the current log entry as the target state, makes a comparison, and subtracts the two values to obtain the control instructions required for the physical robot to run from the current state to the target state of the log entry; The Unity engine converts the control instructions into ROS messages and sends them to the corresponding physical robot through the ROSBridge communication channel to control the running of the physical robot; When the next frame is generated, the Unity engine reads the running data of the physical robot at this time through the ROSBridge communication channel and compares it with the target state in the log entry. If the target state is not reached, control continues; If the target state is reached under the condition that the error is less than the set threshold, the target state is changed to the next log entry, and the physical robot is made to reproduce the trajectory and actions in the historical game by generating control instructions according to the target states given in each log entry.

8. The system according to claim 7, wherein The control instructions generated by the Unity engine based on the log entries include chassis control instructions, pan-tilt control instructions, and shooting control instructions; The chassis control instructions include the lateral and longitudinal target linear velocity control amounts of the chassis, the rotation direction and angular velocity control amount of the chassis; The method for generating the chassis angular velocity control quantity is as follows: Read the vehicle head orientation and chassis coordinates of the physical robot at the moment when it reaches the previous target state, calculate the difference between the orientation and the orientation of the current target state, and record the difference as θ; Connect the coordinates of the previous target point and the coordinates of the current target state, and record the length of the connection as D; Directly calculate the angular velocity control quantity V of uniform rotation ω = θ / T, where T is the log recording time interval; The generation method of the lateral and longitudinal target linear velocity control amounts of the chassis is as follows: calculate the coordinate difference between the current coordinates of the entity robot and the target state coordinates to obtain the distances dx and dy that the entity robot should move in its own x-axis and y-axis directions and the angle α; use the angle α to calculate the linear velocity control amount V in the x-axis direction in real time x , and the linear velocity control amount V in the y-axis direction y as follows: The angle α changes with the frame refresh, and new chassis control instructions need to be calculated for each frame; The generation method of the pan-tilt control instructions is as follows: The reference coordinate system of the pan-tilt of the physical robot is the chassis coordinate system. When calculating the pan-tilt control instructions, the difference between the angle of the target state and the current actual angle is divided by the log recording time interval T to directly obtain the pan-tilt rotation speed; The generation method of the shooting control instructions is as follows: Calculate the difference in the cumulative number of bullets fired in two adjacent log records as the number of bullets to be fired within the current time interval, and generate shooting control instructions.

9. The system according to any one of claims 1-8, characterized in that, The running data of the physical robot includes: the robot's camp and name, position coordinates; the chassis movement speed and direction; the pitch angle and yaw angle of the shooting pan-tilt on the robot, the muzzle shooting state; the remaining blood volume and survival state of the robot.

Images