Robot cluster cooperative control method and device based on indoor positioning and virtual-real fusion simulation, equipment and medium

Abstract

The application discloses a robot cluster cooperative control method and device based on indoor positioning and virtual-real fusion simulation, equipment and medium, relates to the technical field of robot cluster control, and comprises the following steps: a unified coordinate system is established by arranging multiple optical dynamic capture cameras in an indoor environment, three-dimensional space coordinates and attitude information of a target object are obtained, and based on the data, data fusion display is realized in a virtual simulation environment. Subsequently, a cooperative control instruction is generated by using a cluster control algorithm, and is sent to a robot cluster through wireless communication to execute a specific task, thereby realizing high-precision positioning of the robot cluster and greatly improving the task execution efficiency and cooperative control ability of the robot cluster in an indoor environment.

Description

Technical Field

[0001] This invention relates to the field of robot swarm control technology, and in particular to a method, device, equipment and medium for collaborative control of robot swarms based on indoor positioning and virtual-real fusion simulation. Background Technology

[0002] Due to the lack of global navigation satellite system (GNSS) signal support in indoor spaces, traditional GPS-based control methods are difficult to apply in such scenarios. Currently, indoor positioning largely relies on lidar, UWB (ultra-wideband), or visual SLAM (simultaneous localization and mapping). These technologies can achieve the localization of static or dynamic targets under certain conditions and can solve the positioning problem of UAVs in indoor environments to some extent. However, for the motion coordination and task scheduling of robot swarms, they lack controllability and safety in terms of algorithm verification, emergency handling, and fault tolerance. Therefore, there is an urgent need for a robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation to achieve collaborative control of robot swarms in indoor environments and improve task execution efficiency. Summary of the Invention

[0003] The main purpose of this application is to provide a method, device, equipment and medium for collaborative control of robot swarms based on indoor positioning and virtual-real fusion simulation, aiming to solve the technical problem of how to improve the collaborative control capability and task execution efficiency of robot swarms in indoor environments.

[0004] To achieve the above objectives, this application proposes a robot swarm cooperative control method based on indoor positioning and virtual-real fusion simulation, comprising:

[0005] Multiple optical motion capture cameras are set up in an indoor space, and a unified coordinate system for the multiple cameras is obtained;

[0006] The camera is used to acquire image data of the target object;

[0007] The image data is calculated based on the unified coordinate system to obtain the three-dimensional spatial coordinates and pose information of the target object;

[0008] A virtual simulation environment is constructed based on the indoor space, and the three-dimensional spatial coordinates and posture information are synchronously mapped to the virtual simulation environment to obtain a virtual reality result, wherein the virtual reality result is the display result of data fusion and synchronization between the target object and the virtual simulation environment;

[0009] Based on the three-dimensional spatial coordinates and attitude information, combined with the virtual reality results, a cluster control algorithm is run to generate collaborative control commands;

[0010] The collaborative control commands are sent to the robot cluster via wireless communication, so that the robot cluster can perform task actions according to the collaborative control commands. The robot cluster includes drones and unmanned vehicles.

[0011] In one embodiment, the step of calculating the image data based on the unified coordinate system to obtain the three-dimensional spatial coordinates and pose information of the target object includes:

[0012] Select multiple reflective marker points on the surface of the target object based on image data;

[0013] By performing coordinate transformation on multiple reflective marker points using a unified coordinate system, the corresponding image coordinates are obtained;

[0014] The coordinates of the reflective marker points are calculated using a triangulation algorithm based on the image coordinates.

[0015] Construct a three-dimensional rigid body model based on the coordinates;

[0016] The attitude of the three-dimensional rigid body model is estimated by the minimum mean square error rigid body registration algorithm to obtain the rotation matrix and translation vector.

[0017] Based on the rotation matrix and translation vector, calculate the centroid coordinates and initial attitude information of the target object;

[0018] The centroid coordinates and initial attitude information are subjected to exponential sliding filtering and spherical linear interpolation to obtain three-dimensional spatial coordinates and attitude information.

[0019] In one embodiment, the step of constructing a virtual simulation environment based on the indoor space and synchronously mapping the three-dimensional spatial coordinates and attitude information to the virtual simulation environment to obtain a virtual reality result includes:

[0020] A three-dimensional virtual scene is created based on the dimensions of the indoor space, wherein the three-dimensional virtual scene is consistent with the structure of the indoor space;

[0021] Set indoor space calibration points and obtain the three-dimensional coordinates of the calibration points;

[0022] Based on the three-dimensional virtual scene, virtual marker points corresponding to the calibration points are created, and virtual coordinates are obtained;

[0023] The rotation matrix, scaling factor, and translation vector are calculated using a scaled optimization algorithm based on the three-dimensional coordinates of the calibration points and the virtual coordinates.

[0024] The three-dimensional spatial coordinates and attitude information are mapped onto the three-dimensional virtual scene through the rotation matrix, scaling factor, and translation vector to generate a virtual reality result.

[0025] In one embodiment, the step of calculating the rotation matrix, scaling factor, and translation vector using a scaled optimization algorithm based on the three-dimensional coordinates of the calibration point and the virtual coordinates includes the following steps:

[0026] Obtain the set of three-dimensional coordinates of all calibration points in the indoor space and the set of virtual coordinates of the corresponding calibration points in the virtual scene;

[0027] The mean of the three-dimensional coordinate set and the virtual coordinate set is calculated to obtain a first mean and a second mean, wherein the first mean corresponds to the three-dimensional coordinate set and the second mean corresponds to the virtual coordinate set;

[0028] Subtract the first mean from the set of three-dimensional coordinates to obtain the first centered coordinate set;

[0029] Subtract the second mean from the virtual coordinate set to obtain the second centered coordinate set;

[0030] The covariance matrix is ​​calculated by performing covariance matrix calculation on the first centralized coordinate set and the second centralized coordinate set;

[0031] Singular value decomposition is performed on the covariance matrix to obtain the decomposition result;

[0032] Construct a rotation matrix based on the decomposition results;

[0033] The scaling factor is calculated based on the covariance matrix and the second centralized coordinate set.

[0034] The translation vector is calculated based on the rotation matrix, scaling factor, first mean, and second mean.

[0035] In one embodiment, the step of running a cluster control algorithm based on the three-dimensional spatial coordinates and attitude information combined with the virtual reality results to generate cooperative control commands includes:

[0036] A global control problem is constructed based on the aforementioned three-dimensional spatial coordinates and attitude information;

[0037] The global control problem is decomposed into multiple local sub-problems by using behavioral rules, formation control, or distributed control strategies.

[0038] Construct an initial robot swarm dynamics model;

[0039] Based on the initial robot swarm dynamics model, rolling optimization is performed on multiple local sub-problems to obtain a local control sequence;

[0040] The local control sequence is optimized using the alternating direction multiplier method to obtain the optimized control sequence.

[0041] The optimized control sequence is transformed to obtain the task type;

[0042] Based on the task type, the three-dimensional spatial coordinates, and the posture information, combined with the virtual reality results, collaborative control commands are generated.

[0043] In one embodiment, the step of generating collaborative control commands based on the task type, the three-dimensional spatial coordinates, and the attitude information combined with the virtual reality results includes:

[0044] Set safe distance thresholds for drones and unmanned vehicles based on the task type;

[0045] Based on the three-dimensional spatial coordinates and attitude information combined with the virtual reality results, the relative distance between the drone and the unmanned vehicle is obtained;

[0046] The judgment result is obtained by judging based on the safe distance threshold and the relative distance;

[0047] When the relative distance is greater than the safe distance threshold, a cooperative control command is generated, wherein the cooperative control command is that the unmanned vehicle first moves along the planned path and reports its real-time position, and the drone adjusts its flight trajectory according to the real-time position of the unmanned vehicle.

[0048] In one embodiment, after the step of sending the cooperative control command to the robot swarm via wireless communication so that the robot swarm executes the task action according to the cooperative control command, the method further includes:

[0049] The task execution status information received from the robot is used to update the virtual simulation environment, resulting in an updated virtual scene.

[0050] The control commands are corrected based on the updated virtual scene to obtain calibration control commands;

[0051] The calibration control command is sent to the robot so that the robot can re-execute the task action according to the calibration control command to achieve closed-loop control.

[0052] Furthermore, to achieve the above objectives, this application also proposes a robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation, wherein the robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation includes:

[0053] The setting module is used to set up multiple optical motion capture cameras in an indoor space and obtain a unified coordinate system for the multiple cameras;

[0054] The acquisition module is used to acquire image data of the target object using the camera;

[0055] The calculation module is used to calculate the image data based on the unified coordinate system to obtain the three-dimensional spatial coordinates and pose information of the target object;

[0056] The simulation module is used to construct a virtual simulation environment based on the indoor space and synchronously map the three-dimensional spatial coordinates and posture information to the virtual simulation environment to obtain virtual reality results, wherein the virtual reality results are the display results of data fusion and synchronization between the target object and the virtual simulation environment;

[0057] The instruction generation module is used to run a cluster control algorithm based on the three-dimensional spatial coordinates and attitude information combined with the virtual reality results to generate collaborative control instructions;

[0058] The transmitting module is used to send the collaborative control command to the robot cluster via wireless communication, so that the robot cluster can perform task actions according to the collaborative control command. The robot cluster includes drones and unmanned vehicles.

[0059] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation as described above.

[0060] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation as described above.

[0061] This application constructs a unified coordinate system by setting up multiple optical motion capture cameras indoors to acquire the three-dimensional spatial coordinates and attitude information of the target object. Based on this data, data fusion and display synchronized with the actual environment are achieved in a virtual simulation environment. Subsequently, a swarm control algorithm is used to generate cooperative control commands, which are then sent to the robot swarm via wireless communication to execute specific tasks. This achieves high-precision positioning of the robot swarm and greatly improves the task execution efficiency and cooperative control capabilities of the robot swarm in indoor environments. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart illustrating the first embodiment of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation of this application.

[0064] Figure 2 This is an overall framework diagram of the first embodiment of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation of this application;

[0065] Figure 3 This is a flowchart illustrating the second embodiment of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation of this application.

[0066] Figure 4 This is a flowchart illustrating the third embodiment of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation of this application.

[0067] Figure 5 This is a schematic diagram of the module structure of the robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation, which is the first embodiment of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation of this application.

[0068] Figure 6 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation in the embodiments of this application.

[0069] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0070] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0071] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0072] Due to the lack of global navigation satellite system (GNSS) signal support in indoor spaces, traditional GPS-based control methods are difficult to apply in such scenarios. Currently, indoor positioning mostly relies on lidar, UWB (ultra-wideband), or visual SLAM (simultaneous localization and mapping). These technologies can achieve the localization of static or dynamic targets under certain conditions and can solve the positioning problem of UAVs in indoor environments to some extent. However, for the motion coordination and task scheduling of robot swarms, there is a lack of controllability and safety in terms of algorithm verification, emergency handling, and fault tolerance.

[0073] Therefore, this application proposes a robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation to solve the above problems. The main solution of this application embodiment is as follows: multiple optical dynamic capture cameras are set up in an indoor space, and a unified coordinate system of multiple cameras is obtained; image data of the target object is acquired using the cameras; the image data is calculated based on the unified coordinate system to obtain the three-dimensional spatial coordinates and posture information of the target object; a virtual simulation environment is constructed based on the indoor space, and the three-dimensional spatial coordinates and posture information are synchronously mapped to the virtual simulation environment to obtain a virtual reality result, wherein the virtual reality result is the display result of data fusion and synchronization between the target object and the virtual simulation environment; a swarm control algorithm is run according to the three-dimensional spatial coordinates and posture information combined with the virtual reality result to generate collaborative control commands; the collaborative control commands are sent to the robot swarm via wireless communication so that the robot swarm can execute task actions according to the collaborative control commands. The robot swarm includes drones and unmanned vehicles.

[0074] Based on the above, this application also provides a robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation according to this application. In this embodiment, as... Figure 2 The overall framework diagram shown is based on Figure 2 It can be seen that it is mainly divided into three parts: the UAV end, the unmanned vehicle end, and the control station end. The method of this application is applied to the control station end, which includes a desktop computer and multiple software platforms, such as XingYing, Unity3D, and QGC (ground station software). Data communication with the UAV end and the unmanned vehicle end is achieved through WLAN networking of the motion capture camera end, which also includes an optical motion capture camera, model MARS1.3H. 8. This system captures motion data from drones and unmanned vehicles (UAVs). A command control platform and virtual simulation platform then run on a desktop computer at the main control station to monitor and control the entire system. The UAV hardware platform includes an industrial computer with Nvidia Jetson Nano, STM32F1, and STM32H7 microcontrollers, and a PIX4 aircraft attitude controller. Sensors and devices include a compass, BeiDou navigation system, wide-angle camera, inertial navigation system, gyroscope, and barometer. Communication with the ground control station is achieved via a UART interface connected to an ESP32 module. The UAV exchanges data with the control station via WLAN networking. Similarly, the UAV uses Nvidia Jetson Nano and STM32H7 microcontrollers as the core processing unit of its industrial computer and is equipped with sensors and actuators such as cameras, motor drives, single-line LiDAR, and DC motors. The UAV also communicates with the control station via WLAN networking.

[0075] In this embodiment, the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation includes steps S10 to S60:

[0076] Step S10: Set up multiple optical motion capture cameras in the indoor space and obtain a unified coordinate system for the multiple cameras.

[0077] It should be noted that in this embodiment, to ensure the visibility and positioning accuracy of the target object throughout the entire working area, eight or more optical motion capture cameras will be deployed. These cameras will be installed in different corners of the room to cover the largest possible monitoring range and ensure sufficient overlap to improve positioning reliability. Obtaining a unified coordinate system involves the precise calibration of the spatial parameters of each camera, which involves determining intrinsic parameters (such as focal length, principal point position, lens distortion coefficient, etc.) and extrinsic parameters (such as the camera's position and orientation relative to the world coordinate system). Intrinsic parameters are calculated using image processing algorithms after taking multiple images from different angles using a known pattern (such as a checkerboard) as a calibration board; extrinsic parameters require identifying feature points on the calibration board and solving for the camera's rotation and translation matrices using analytical geometry. The camera layout is rationally planned according to the size and shape of the indoor space to ensure that the camera's field of view can cover the entire target area while avoiding mutual occlusion. Cameras are typically installed in the corners or at the top of the room to obtain the best viewing angle and coverage.

[0078] Further, step S10 also includes: performing single-camera calibration on multiple cameras to determine the intrinsic parameter matrix, distortion coefficients, and extrinsic parameter matrix of the multiple cameras. Specifically, single-camera calibration is performed on each camera. During the calibration process, images are captured from multiple angles using a checkerboard calibration board. The corner points of the checkerboard are extracted using image processing algorithms, and the intrinsic parameter matrix (including focal length, principal point coordinates, etc.) and distortion coefficients (such as radial distortion and tangential distortion) of each camera are calculated. These parameters describe the geometric model of camera imaging and are the basis for subsequent multi-camera joint calibration. Next, multi-camera joint calibration is performed using an L-shaped calibration rod based on the intrinsic parameter matrix and distortion coefficients to obtain unified calibration coefficients. Specifically, the L-shaped calibration rod has multiple reflective markers with known positional relationships. These markers form corresponding points in the images of different cameras. Through these corresponding points, combined with the intrinsic parameter matrix and distortion coefficients obtained from the single-camera calibration, the unified calibration coefficients of each camera can be calculated. Next, the extrinsic parameter matrix is ​​optimized using the least squares method to obtain the relative positions and attitude relationships between multiple cameras. Specifically, the least squares method optimizes the extrinsic parameter matrix by minimizing the error between the observation data of all cameras and the theoretical model, thus obtaining more accurate relative positions and attitude relationships between the cameras. Finally, a unified coordinate system for multiple cameras is calculated based on unified calibration coefficients and the relative positions and attitude relationships between multiple cameras. Specifically, by aligning the coordinate systems of all cameras to a global coordinate system, it is ensured that the measurement data of all cameras can be fused and processed in a unified coordinate system. This process involves rotation and translation transformations between coordinate systems. Through precise calculation and optimization, the consistency and accuracy of the measurement data of each camera in the global coordinate system are ensured. Ultimately, the obtained unified coordinate system provides a reliable geometric basis for the collaborative operation of the multi-camera system, enabling the system to achieve high-precision target localization and tracking in complex indoor environments.

[0079] Furthermore, taking the calibration of two cameras as an example, the projection matrices of the two cameras are respectively and ,in This represents the intrinsic parameter matrix of the first camera. Represents the identity matrix. Represents the zero vector. This represents the intrinsic parameter matrix of the second camera. Represents the rotation matrix. Let the translation vector be denoted by . For a point in space, For the light center of the first camera, For the optical center of the second camera, They are respectively At the projection points of the first and second cameras, It is the projection point of the optical center of the second camera onto the first camera. It is the projection point of the optical center of the first camera onto the second camera, specifically represented as:

[0080]

[0081] in, Representing the rotation matrix The transpose of . Let the fundamental matrices of the two cameras be represented as . For any point in space , corresponding to image points from two cameras All conditions are met:

[0082]

[0083] in express The transpose of . If the intrinsic parameter matrices of the two cameras are known, the image points of the two cameras are normalized and denoted as . and At this point, the following conditions can be met:

[0084]

[0085] in, This represents the essential matrix of the two cameras. By... and Solving these two formulas that meet the requirements will yield the spatial point that can be found. The three-dimensional coordinates of the point in space are obtained. By using the three-dimensional coordinates, a unified world coordinate system can be obtained.

[0086] Step S20: Use a camera to acquire image data of the target object.

[0087] It should be noted that the aforementioned cameras acquire infrared images of reflective markers on the surface of the target object at a high frame rate, capturing its motion trajectory in three-dimensional space. The system employs a multi-camera collaborative working mode, with each camera's field of view overlapping to ensure that the target object can be captured simultaneously by at least two cameras at any position, thereby improving positioning accuracy and reliability. During image acquisition, camera parameters such as exposure time and gain are dynamically adjusted according to indoor lighting conditions to ensure image quality. Simultaneously, the system monitors the camera synchronization signal in real time to ensure that all cameras acquire images at the same time, avoiding positioning errors caused by time deviations. The acquired image data is transmitted to the main control console via a high-speed data transmission interface for subsequent processing and analysis.

[0088] To further improve the usability of image data, the system preprocesses the acquired images, including noise removal and distortion correction. These preprocessing steps effectively improve image quality, providing a more accurate data foundation for subsequent 3D reconstruction and pose estimation. Through these optimization measures, the optical motion capture camera system can efficiently and accurately acquire image data of target objects, providing solid data support for indoor positioning and virtual-real hybrid simulation.

[0089] Step S30: Calculate the image data based on a unified coordinate system to obtain the three-dimensional spatial coordinates and pose information of the target object.

[0090] It should be noted that this process relies on infrared image data of reflective markers attached to the target object, captured by an optical motion capture camera.

[0091] Further, step S30 includes: selecting multiple reflective markers on the surface of the target object based on image data. These markers are typically spherical or circular reflective patches with high reflectivity, installed at fixed positions on the target object. In this embodiment, at least three non-collinear reflective markers are fixedly installed on the surface of the target object. The optical motion capture camera identifies the pixel coordinates of these markers in the two-dimensional image by emitting infrared light and capturing its reflection signal. Subsequently, the coordinates of the multiple reflective markers are transformed using a unified coordinate system to obtain the corresponding image coordinates. Specifically, the unified coordinate system here refers to the world coordinate system shared by all cameras, obtained through the camera calibration process. Image coordinates refer to the projected position of each marker in the two-dimensional image. Combining the camera's intrinsic matrix (focal length, principal point, distortion coefficient, etc.) and extrinsic matrix (position and orientation), the image coordinates can be converted into ray directions in three-dimensional space. Next, based on the image coordinates, a triangulation algorithm is used to calculate the 3D spatial coordinates of the reflective marker points. Specifically, triangulation is a 3D point reconstruction method based on multi-view geometry. It calculates the intersection points in 3D space, i.e., the true spatial coordinates of the marker point, using the projection information of the same marker point from multiple camera views. This process processes multiple marker points separately, ultimately obtaining a set of 3D spatial points. Then, a 3D rigid body model is constructed based on the 3D spatial coordinates. Specifically, a 3D rigid body model of the target object is constructed using the 3D spatial coordinates of multiple reflective marker points. The rigid body model assumes that the target object remains rigid during motion, i.e., the relative positional relationship between the marker points remains fixed. Optimization algorithms such as the least squares method can further improve the accuracy of the rigid body model, ensuring that the model can accurately reflect the actual shape and motion state of the target object. Then, the least mean square error rigid body registration algorithm is used to estimate the attitude of the 3D rigid body model, obtaining the rotation matrix and translation vector. Specifically, the least mean square error rigid body registration algorithm calculates the rotation matrix and translation vector of the target object by minimizing the error between the current attitude and the initial attitude of the target object. Specifically, the algorithm registers the current 3D coordinates of the reflective marker with their initial coordinates. By solving for the optimal transformation matrix, it obtains the rotation matrix and translation information of the target object. This process can be updated in real time, ensuring that the target object's attitude information accurately reflects its motion state. Next, based on the rotation matrix and translation vector, the centroid coordinates and initial attitude information of the target object are calculated. Specifically, the centroid coordinates and attitude information of the target object can be calculated using the rotation matrix and translation vector. The centroid coordinates can be obtained by weighted averaging of the 3D coordinates of the reflective marker, while the attitude information is directly represented by the rotation matrix and translation vector. The centroid coordinates and attitude information are accurate descriptions of the target object's position and orientation in 3D space, providing crucial data for subsequent control and simulation.Finally, the centroid coordinates and initial pose information are processed by exponential moving average (EMA) filtering and spherical linear interpolation to obtain the three-dimensional spatial coordinates and pose information. Specifically, to improve the robustness and continuity of the target object's pose estimation, the system filters the centroid coordinates and pose information. For the centroid coordinates, the exponential moving average (EMA) method is used for smoothing. This method reduces the impact of noise on the centroid coordinates through weighted averaging. For the pose information, the spherical linear interpolation (SLERP) method is used for filtering to avoid pose jitter. When pose loss occurs due to frame loss or signal occlusion, the system can also estimate the pose information through interpolation to ensure its continuity.

[0092] The above steps enable the accurate acquisition of the target object's three-dimensional spatial coordinates and attitude information, providing reliable data support for indoor positioning and virtual-real combined simulation.

[0093] Step S40: Construct a virtual simulation environment based on the indoor space, and synchronously map the three-dimensional spatial coordinates and attitude information to the virtual simulation environment to obtain the virtual reality result.

[0094] It should be noted that, in order to achieve virtual simulation of the indoor space and synchronous mapping of the target object's 3D spatial coordinates and posture information, a virtual simulation environment highly consistent with the real environment is first constructed on a simulation platform such as Unity3D or Gazebo, based on the size, structure, and layout of the indoor space. This process includes creating walls, floors, ceilings, and various obstacles and facilities in the virtual scene, ensuring that the virtual environment matches the real indoor space in terms of geometry and visual effects.

[0095] After the virtual simulation environment is constructed, the 3D spatial coordinates and attitude information of the target object are mapped to the virtual environment in real time. This mapping process is achieved through coordinate transformation, converting the position and attitude of the target object in the real world to the corresponding position and attitude in the virtual world. Specifically, the system uses a previously calibrated unified coordinate system to convert the 3D coordinates and attitude information of the target object into coordinates and attitude in the virtual environment. To achieve data fusion and synchronous display, a virtual model of the target object is generated in the virtual environment, and the position and attitude of the virtual model are dynamically updated based on the real-time acquired 3D spatial coordinates and attitude information. In this way, the virtual model can reflect the motion state of the target object in the real world in real time, achieving virtual-real synchronization. At the same time, other auxiliary information, such as path planning, collision detection, and task progress, can be added to the virtual environment to enhance the information content and interactivity of the virtual simulation environment, thereby obtaining virtual reality results. The above virtual reality results are the display results of data fusion and synchronization between the target object and the virtual simulation environment, presented to the user through a visual interface. Users can intuitively observe the movement of the target object in the virtual environment through this interface, while obtaining relevant task information and feedback. This combination of virtual and real-world display not only enhances the system's visualization but also provides important reference for subsequent control decisions.

[0096] Step S50: Based on the three-dimensional spatial coordinates and attitude information combined with the virtual reality results, run the cluster control algorithm to generate collaborative control commands.

[0097] It's important to note that this process first relies on accurately acquired position and attitude data of each target object (such as a drone or unmanned vehicle) in three-dimensional space. Based on this real-time updated information combined with virtual reality results, the main control console can use swarm control algorithms to plan and optimize the behavior of the entire robot swarm. Swarm control algorithms typically include several types, such as rule-based behavior, formation control, and distributed control strategies. For example, rule-based behavior methods can define simple basic behaviors such as obstacle avoidance, clustering, and dispersing, and combine these behaviors to achieve complex group dynamics. Formation control, on the other hand, focuses on maintaining specific geometries or relative distances to ensure the swarm maintains a predetermined formation structure when performing tasks. For more complex and flexible task requirements, distributed control strategies are widely used, especially distributed model predictive control (DMPC), which decomposes the global optimization problem into multiple local subproblems. Each subsystem only needs to communicate with its directly adjacent systems, without a central coordinator.

[0098] Furthermore, in the specific implementation, for each robot Its dynamic model is used to describe the changes in state and control input over time. Utilizing the definition of a local optimization problem, each robot in the prediction time domain... The internal optimization problem is solved to determine the optimal trajectory. This involves calculating the desired trajectory and then applying it through the weight matrix. and Balancing different objectives is crucial, such as the trade-off between tracking error and control energy consumption. Distributed optimization methods, such as the Alternating Direction Multiplier Method (ADMM), are used to solve local optimization problems by iteratively updating the state estimate, control input, and Lagrange multipliers. This ensures consistency among the various subsystems and achieves global optimization. Finally, the cooperative control commands calculated using the above algorithm are converted into specific action commands, such as speed adjustments and steering commands.

[0099] Step S60: The collaborative control command is sent to the robot cluster via wireless communication so that the robot cluster can perform task actions according to the collaborative control command.

[0100] It's important to note that the robot swarm includes drones and unmanned vehicles (UAVs). Cooperative control commands generated are transmitted to the swarm in real-time via wireless communication, ensuring each robot can accurately receive and execute task actions. Specifically, the drone swarm uses an ESP32 serial module for communication. The ESP32 module features low power consumption, high reliability, and fast response, efficiently forwarding control commands from the main control console to the drone's flight control system. Upon receiving the commands, the flight control system immediately parses and executes the corresponding flight actions, such as adjusting flight attitude, speed, or heading, to complete tasks like formation flying and target tracking. The UAV swarm establishes a communication link with the ground control station via WiFi. WiFi communication offers advantages such as wide coverage and high transmission rates, meeting the needs of autonomous movement and task execution in indoor environments. After receiving commands from the main control console, the ground control station sends the commands to each UAV via the WiFi network. Based on the received commands, the UAVs autonomously plan paths, avoid obstacles, and execute task actions, such as traveling along a preset path, following target objects, or cooperating with other UAVs.

[0101] Meanwhile, to ensure communication stability and reliability, the system employs redundant communication mechanisms and data verification algorithms. During communication, the main control console monitors the communication link status in real time. If signal loss or data error is detected, a retransmission mechanism is immediately initiated to ensure the accuracy of commands. Furthermore, the system dynamically adjusts the command transmission frequency and data volume based on communication latency and bandwidth to adapt to different communication environments.

[0102] Furthermore, after step S60, the system further includes: receiving task execution status information from the robot to update the virtual simulation environment, obtaining an updated virtual scene; correcting control commands based on the updated virtual scene, obtaining calibration control commands; and sending the calibration control commands to the robot so that the robot can re-execute task actions according to the calibration control commands to achieve closed-loop control. Specifically, during algorithm operation, the system dynamically updates the virtual simulation environment by receiving task execution status information from the robot cluster, thereby achieving real-time correction and optimization of control commands. During task execution, the robot cluster (including drones and unmanned vehicles) will continuously feed back its own status information (such as position, speed, attitude, etc.) and task execution status (such as whether the specified action has been completed, whether there is a deviation, etc.) to the system via a wireless communication module. This feedback information is a crucial basis for closed-loop control. Upon receiving the feedback information, it will immediately compare and synchronize it with the corresponding data in the virtual simulation environment. For example, if a drone experiences a positional deviation while performing a formation flight task, the system will reflect this deviation information in the virtual simulation environment in real time, updating the corresponding position and status of the drone in the virtual scene. In this way, the virtual simulation environment can reflect the task execution status in the real scene in real time, providing accurate information support for subsequent control decisions. Next, the system will correct the control commands based on the updated virtual scene. The correction process is usually based on preset control strategies and optimization algorithms. For example, if an obstacle is found in a task execution path, the system will recalculate the obstacle avoidance path through a path planning algorithm and generate new control commands. The corrected control commands are called calibration control commands, the purpose of which is to correct deviations in the robot's task execution, optimize the task execution path and actions, and ensure that the task can be performed smoothly as expected. Finally, the calibration control commands are sent to the corresponding robots through the wireless communication module. After receiving the calibration control commands, the robots will immediately stop their current task actions, readjust their position, speed, attitude and other parameters according to the new commands, and then continue to execute the task. This process is repeated continuously, forming a closed-loop control loop. Through this closed-loop control mechanism, the system can respond to various changes in the task execution process in real time, adjust the control strategy in a timely manner, and ensure that the robot swarm completes the task efficiently and accurately in complex indoor environments. This process not only improves the efficiency and accuracy of task execution, but also enhances the system's adaptability and robustness in complex indoor environments.

[0103] This embodiment utilizes multiple optical motion capture cameras set up in an indoor space to acquire a unified coordinate system. Image data of the target object is obtained using these cameras. The image data is then calculated based on the unified coordinate system to obtain the target object's three-dimensional spatial coordinates and attitude information. A virtual simulation environment is constructed within the indoor space, and the three-dimensional spatial coordinates and attitude information are synchronously mapped onto this environment to obtain a virtual reality result. This virtual reality result is a display of the data fusion and synchronization between the target object and the virtual simulation environment. A swarm control algorithm is run based on the three-dimensional spatial coordinates and attitude information to generate collaborative control commands. These commands are then sent to a robot swarm (including drones and unmanned vehicles) via wireless communication, enabling the swarm to execute tasks. By setting up multiple optical motion capture cameras indoors to construct a unified coordinate system and acquire the target object's three-dimensional spatial coordinates and attitude information, data fusion and display synchronized with the actual environment are achieved in the virtual simulation environment. Subsequently, a swarm control algorithm is used to generate collaborative control commands, which are then sent to the robot swarm via wireless communication to execute specific tasks. This achieves high-precision positioning of the robot swarm and significantly improves its task execution efficiency and collaborative control capabilities in indoor environments.

[0104] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 The robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation, step S40, further includes steps S201 to S205:

[0105] Step S201: Create a three-dimensional virtual scene based on the dimensions of the indoor space.

[0106] It should be noted that precise measurements of the interior space's dimensions, including length, width, and height, as well as the specific locations and dimensions of structures such as walls, doors, windows, and columns, are required. This data can be obtained through laser scanners, 3D modeling tools, or manual measurement to ensure that the 3D virtual scene matches the interior space's structure.

[0107] Next, using professional simulation platforms such as Unity3D or Gazebo, a 3D virtual scene is constructed based on the measurement data. During the modeling process, every detail is meticulously reproduced according to the proportions and layout of the real environment. For example, the thickness of walls, the size of doors and windows, the height of ceilings, and the positions of various obstacles are accurately mapped into the virtual scene. Furthermore, textures and materials can be added to the virtual scene to make the virtual environment more visually realistic and enhance the user's immersion.

[0108] To ensure a precise correspondence between the virtual scene and the real environment, the system employs coordinate alignment technology. By aligning the coordinate system of the virtual scene with the unified coordinate system of the real environment, the system ensures that the position and posture of virtual objects accurately reflect the state of real objects. This process typically involves operations such as coordinate transformation, scaling, and translation adjustments to achieve seamless integration between the virtual and real worlds.

[0109] After the virtual scene is built, the system will perform verification and calibration. It compares the position and pose of target objects in the real environment with their corresponding objects in the virtual scene to check for discrepancies. If a discrepancy is found, the system will automatically adjust the positions of objects in the virtual scene or recalibrate the coordinate system to ensure consistency between the virtual scene and the real environment.

[0110] Step S202: Set indoor space calibration points and obtain the three-dimensional coordinates of the calibration points.

[0111] It should be noted that several representative locations are selected as calibration points within the indoor space. These locations typically include room corners, wall intersections, and specific locations on the floor to ensure that the calibration points cover the entire work area. The number and distribution of calibration points should be determined based on the size and complexity of the indoor space, generally no fewer than four, and they should be distributed on different planes as much as possible to improve the accuracy and reliability of the positioning. Next, these calibration points are photographed using an optical motion capture camera, and the obtained image data is processed using specialized software to calculate the precise three-dimensional coordinates of each calibration point in the world coordinate system.

[0112] Furthermore, to ensure system stability and reliability, it is necessary to periodically verify and update the calibration points and their corresponding coordinates. Especially after environmental changes or system relocation, recalibration helps eliminate accumulated errors and ensures long-term accuracy. Simultaneously, using the ROS tf2 tool to broadcast coordinate transformations allows for convenient data conversion between different coordinate systems, supporting seamless integration between virtual and real coordinate systems.

[0113] Step S203: Create virtual marker points corresponding to the calibration points based on the 3D virtual scene, and obtain virtual coordinates.

[0114] It's important to note that in a virtual simulation environment (such as Unity3D or Gazebo), a 3D virtual scene is created based on the actual dimensions and layout of the interior space. During creation, ensure that the walls, floors, ceilings, and other important structures in the virtual scene are completely consistent with the real environment. Next, based on the positions of the real calibration points, corresponding virtual marker points are created in the virtual scene. These virtual marker points are typically represented by simple geometric shapes (such as spheres or cubes) for easy identification and manipulation.

[0115] To obtain the virtual coordinates of virtual marker points, a global coordinate system needs to be defined in the virtual scene. This global coordinate system typically corresponds to a unified coordinate system in the real environment, ensuring a consistent benchmark for coordinate transformations between virtual and real environments. In a virtual simulation environment, the position information of virtual marker points in the global coordinate system can be directly obtained through a programming interface (API). This position information includes the three-dimensional coordinates (x, y, z) of the virtual marker points and possible pose information (such as rotation angles). After obtaining the virtual coordinates, these coordinates need to be compared and calibrated with the three-dimensional coordinates of the real calibration points. By calculating the spatial transformation relationship between the real calibration points and the virtual marker points (such as rotation matrices and translation vectors), the position of the virtual marker points can be fine-tuned to ensure their accurate spatial correspondence with the real calibration points. This process typically involves optimization algorithms, such as the least squares method, to minimize the error between the real and virtual coordinates.

[0116] Ultimately, by precisely creating virtual marker points and obtaining their virtual coordinates, the system can achieve seamless integration between the virtual scene and the real environment. This precise alignment not only provides a solid foundation for subsequent virtual-real hybrid simulations but also offers a reliable reference for high-precision indoor positioning and collaborative robot control.

[0117] Step S204: Calculate the rotation matrix, scaling factor, and translation vector using a scaled optimization algorithm based on the three-dimensional coordinates and virtual coordinates of the calibration point.

[0118] It should be noted that this process involves using a scaled optimization algorithm (such as the Umeyama algorithm) to compute the rotation matrix, scaling factor, and translation vector, thereby achieving optimal alignment between the two coordinate systems. The goal of this algorithm is to minimize the loss function between the two point sets.

[0119] Further, step S204 includes: obtaining the set of three-dimensional coordinates of all calibration points in the indoor space. The set of virtual coordinates of the corresponding calibration points in the virtual scene Then, the mean values ​​of the 3D coordinate set and the virtual coordinate set are calculated to obtain the first mean and the second mean, where the first mean corresponds to the 3D coordinate set and the second mean corresponds to the virtual coordinate set, specifically expressed as follows:

[0120]

[0121]

[0122] in, This represents the total number of three-dimensional coordinates or virtual coordinates. Indicates the current number of 3D coordinates or virtual coordinates;

[0123] Subtracting the first mean from the set of three-dimensional coordinates yields the first centered coordinate set, specifically represented as follows: ;

[0124] Subtracting the second mean from the virtual coordinate set yields the second centered coordinate set, specifically represented as follows: ;

[0125] The covariance matrix is ​​calculated by applying the first and second centered coordinate sets, and is specifically represented as follows:

[0126]

[0127] in, This represents the total number of three-dimensional coordinates or virtual coordinates. This represents the transpose of the second centered coordinate set. Singular value decomposition is performed on the covariance matrix to obtain the decomposition results. Then, a rotation matrix is ​​constructed based on these results, specifically as follows:

[0128]

[0129] in, This represents the first orthogonal matrix of the singular value decomposition, whose column vectors are left singular vectors; Let represent the second orthogonal matrix of the singular value decomposition, whose column vectors are the right singular vectors; Let represent a diagonal matrix whose diagonal elements are 1, and The determinant value, The determinant function is used to calculate the determinant value of a matrix. Representation matrix The transpose of . Then, the scaling factor is calculated based on the covariance matrix and the second centralized coordinate set, specifically expressed as:

[0130]

[0131] in, This represents the sum of all diagonal elements of the matrix. The modulus represents the second centralized coordinate set, and the specific numerical value represents the value.

[0132] The translation vector is calculated based on the rotation matrix, scaling factor, first mean, and second mean, and is specifically represented as follows:

[0133]

[0134] By calculating the translation vector, it can be ensured that the virtual model and the real object are in the same position in space, thereby achieving closed-loop control combining virtual and real elements and high-precision task execution.

[0135] Step S205: The three-dimensional spatial coordinates and pose information are mapped onto the three-dimensional virtual scene through rotation matrix, scaling factor and translation vector to generate virtual reality results.

[0136] It should be noted that this process involves converting the precise position and orientation information of a target object in the real world into corresponding data in the virtual environment. Specifically, firstly, image data of reflective markers on the surface of the target object is acquired using an optical motion capture camera, and then this data is used to calculate the three-dimensional spatial coordinates and pose information of the target object.

[0137] Next, coordinate transformations are performed on this 3D information using rotation matrices, scaling factors, and translation vectors. The rotation matrix adjusts the object's orientation in the virtual environment, ensuring its pose matches its real-world counterpart; the scaling factor adjusts the object's size, matching its proportions in the virtual environment; and the translation vector determines the object's position in the virtual environment, ensuring it corresponds to its real-world position. Through these transformations, the target object's 3D spatial coordinates and pose information are accurately mapped into the 3D virtual scene, generating a virtual reality result. This result includes not only the object's geometry and size but also its position and orientation in space.

[0138] This embodiment creates a 3D virtual scene with a consistent structure by accurately measuring the dimensions of the indoor space; sets up calibration points to obtain the physical and virtual coordinates; uses a scale-optimized algorithm to calculate coordinate transformation parameters; and finally maps the 3D spatial coordinate information of the real object to the virtual scene, achieving precise alignment between the physical space and the virtual environment. It supports high-precision real-time monitoring and visual interaction of control commands, improving the collaborative efficiency and execution accuracy of robot clusters in complex tasks.

[0139] Based on the first embodiment of this application, in the third embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 The robot swarm collaborative control step S50 based on indoor positioning and virtual-real fusion simulation further includes steps S301 to S307:

[0140] Step S301: Construct a global control problem based on three-dimensional spatial coordinates and attitude information.

[0141] It's important to note that this process first involves accurately modeling the current state of each robot, including its position, velocity, acceleration, and attitude (such as orientation and rotation angle) in three-dimensional space. This information is typically acquired in real-time by sensors and transmitted to the central control system. The construction of the global control problem then needs to consider the behavior of the entire robot swarm as a whole, including their interrelationships and collaborative methods. The control system defines global objectives and constraints based on task requirements, such as formation flying, path planning, and target tracking. These objectives and constraints are then transformed into mathematical models, forming a global optimization problem.

[0142] Step S302: Decompose the global control problem into multiple local sub-problems by using behavioral rules, formation control, or distributed control strategies.

[0143] It's important to note that this process allows a complex global problem to be transformed into multiple, more manageable, and solvable local problems, each associated with a specific robot or a group of robots. Based on behavioral rules, each robot determines its behavior according to predefined rules. These rules, such as obstacle avoidance, target tracking, or cooperation rules, define how the robot should act in specific situations. In this way, each robot only needs to focus on its local environment and task, without needing to understand the global state of the entire system. Formation control strategies involve how robots maintain specific relative positions and distances to form an effective formation. This is particularly important in drone swarms or unmanned vehicle platoons. Formation control ensures that the robot swarm moves as a whole while maintaining the required formation and spacing. Distributed control strategies further distribute the control problem to each robot, making each robot an independent control unit responsible for solving the local problems directly related to it. This strategy allows robots to operate autonomously without a central coordinator, improving the system's robustness and scalability.

[0144] Through these strategies, the global control problem is decomposed into multiple local subproblems, each of which can be solved independently. The solved local solutions are then integrated through appropriate communication and coordination mechanisms to form a solution to the global problem.

[0145] Step S303: Construct the initial robot swarm dynamics model.

[0146] It's important to note that, firstly, a physical model is created for each robot, including its mass, size, shape, and power system, as these factors affect the robot's inertia and dynamic response. Secondly, a kinematic model is established to describe the robot's motion in space, such as position, velocity, and acceleration, and how these motion states can be altered through control inputs (e.g., motor speed, joint angles). Furthermore, the model should include the robot's interaction with its environment, such as ground friction and air resistance, which influence the robot's movement. For drone swarms, aerodynamic effects, such as lift, drag, and eddies, must also be considered. During model building, parameter identification is typically required, and experimental data is used to calibrate the model parameters to ensure accuracy and reliability, ultimately resulting in a dynamic model.

[0147] Step S304: Based on the initial robot swarm dynamics model, perform rolling optimization on multiple local sub-problems to obtain the local control sequence.

[0148] It should be noted that rolling optimization, also known as model predictive control (MPC), is a method that uses future predictions to optimize current control decisions. This method is particularly useful in the context of robot swarms because it allows each robot to consider the state and actions of its neighbors, resulting in more coordinated group behavior.

[0149] Specifically, in the rolling optimization process, each robot constructs a local optimization problem based on its own dynamics model, task requirements, and information received from neighboring robots. This problem typically includes predicting the robot's trajectory over a future period, along with cost functions associated with this trajectory, such as energy consumption, path deviation, and collision risk. By solving this optimization problem, each robot can generate a local control sequence—a series of control inputs designed to minimize the cost function while satisfying dynamic and environmental constraints. The generation of the local control sequence considers not only the performance of individual robots but also the synergistic effects of the entire swarm. For example, when drones fly in formation, they need to avoid collisions with neighboring drones while maintaining formation stability. Rolling optimization allows each robot to continuously update its control sequence based on real-time information to adapt to dynamically changing environmental and task requirements.

[0150] Furthermore, the rolling optimization strategy offers excellent flexibility and scalability. It can easily adapt to changes in the number of robots in the swarm, as well as the introduction of new task objectives and constraints. In this way, robot swarms can achieve efficient and coordinated operations in complex and ever-changing environments, improving the success rate and efficiency of task execution.

[0151] Step S305: The local control sequence is optimized by the alternating direction multiplier method to obtain the optimized control sequence.

[0152] It should be noted that the Alternating Direction Method of Multipliers (ADMM) is an efficient distributed optimization algorithm, particularly suitable for solving large, complex, and distributed optimization problems. In the field of robot swarm control, ADMM can be used to optimize the local control sequences of individual robots to achieve the cooperative goals of the entire swarm.

[0153] Specifically, within the ADMM framework, each robot in the swarm can independently solve its own local optimization problem while coordinating with other robots through shared multiplier variables. This approach allows robots to maintain a degree of autonomy while ensuring that the behavior of the entire swarm conforms to global task requirements. Specifically, each robot calculates an initial control sequence based on its own dynamics model and local observation data, and incorporates this sequence as part of the ADMM iterative process.

[0154] In each iteration of ADMM, each robot updates its control sequence to reduce its local cost function, while coordinating its interactions with other robots through a multiplier update step. This process is repeated until a preset convergence condition is met, such as the cost function change falling below a certain threshold, or the maximum number of iterations being reached.

[0155] The control sequence optimized by ADMM considers not only the performance of individual robots but also the interdependencies among all robots in the swarm. This method can handle coupling constraints between robots, such as collision avoidance and formation maintenance, thereby achieving smoother and more efficient swarm collaboration.

[0156] Step S306: Transform the optimized control sequence to obtain the task type.

[0157] It should be noted that the optimized control sequence contains the action instructions that each robot should execute within a specific time period, such as speed, acceleration, and turning angle. These instructions need to be parsed and converted according to the specific requirements of the mission. For example, in formation flying missions, the control sequence may need to be converted into instructions to maintain a specific relative position and distance; in search and rescue missions, it may need to be converted into instructions to cover a specific area or path.

[0158] Specifically, the transformation process typically involves task planning algorithms that understand task requirements and break them down into sub-tasks that the robot swarm can perform. For example, behavior trees or decision networks are used to determine the actions the robots should take in a specific environment. Furthermore, environmental factors and the robots' dynamic capabilities need to be considered to ensure that the transformed task type aligns with the task objectives and remains within the robots' capabilities. The transformed task type also needs to be validated and adjusted to ensure the robot swarm's behavior is consistent with expectations. This may involve simulation testing or real-time monitoring to evaluate the robot swarm's performance and make necessary adjustments. Through this transformation, the robot swarm can obtain clear action guidelines from the optimized control sequence, thereby achieving efficient and coordinated task execution.

[0159] Ultimately, this transformation process enables robot swarms to acquire task types based on different task requirements, improving the system's versatility and adaptability. Whether in industrial automation, environmental monitoring, or disaster response, this capability greatly enhances the ability of robot swarms to operate in complex environments.

[0160] Step S307: Based on the task type, three-dimensional spatial coordinates, and attitude information combined with the virtual reality results, generate collaborative control commands.

[0161] It's important to note that the system determines the required collaborative behavior based on the task type, such as formation flying, cargo handling, or area coverage. Then, using 3D spatial coordinates and attitude information, the system can accurately calculate the position and orientation of each robot in space. This information is crucial for ensuring the coordination and consistency of the robot swarm, as it determines the relative positions and trajectories of the robots.

[0162] Further, step S307 includes: setting a safe distance threshold between the drone and the unmanned vehicle based on the task type. Specifically, the safe distance threshold is set according to the specific task requirements and the characteristics of the operating environment. This threshold should consider various factors, including but not limited to the robot's size, movement speed, operational complexity, and emergency obstacle avoidance reaction time. For example, in high-speed moving tasks, a larger safe distance needs to be set to allow sufficient time to react. For static or low-speed tasks, the safe distance can be appropriately reduced. Subsequently, the relative distance between the drone and the unmanned vehicle is calculated based on the three-dimensional spatial coordinates and attitude information. Specifically, the relative distance between the drone and the unmanned vehicle is calculated using the three-dimensional spatial coordinates and attitude information obtained from the optical motion capture system. By comparing the centroid coordinates of the drone and the unmanned vehicle, the specific distance formula is:

[0163]

[0164] in, and These represent the position coordinates of the UAV and the unmanned vehicle at a certain moment. Next, a judgment is made based on a safe distance threshold and the relative distance. When the relative distance is greater than the safe distance threshold, a cooperative control command is generated. This command instructs the unmanned vehicle to move along the planned path and report its real-time position, while the UAV adjusts its flight trajectory based on the vehicle's real-time position. Specifically, once the relative distance is calculated, a judgment is made based on a preset safe distance threshold. If the relative distance is greater than the safe distance threshold, the current state is considered safe, and the original plan can continue. At this point, a cooperative control command is generated to guide the unmanned vehicle to move along the planned path and report its current position in real-time via WiFi communication. Simultaneously, the UAV receives this real-time updated position information through the ESP32 serial port module and adjusts its flight trajectory accordingly. When the UAV receives the new position of the unmanned vehicle, it recalculates the shortest distance from itself to the vehicle based on the new data. Combining this with its own flight path and speed parameters, it uses a distributed model predictive control (DMPC) algorithm to optimize its flight trajectory. DMPC allows the UAV to solve the optimal control problem in the prediction time domain to minimize the deviation from the target path while maintaining a safe distance from the vehicle.

[0165] This collaborative control strategy forms a closed-loop control system, where the unmanned vehicle acts as the lead unit responsible for ground exploration or specific task execution, while the drone provides aerial perspective support, such as environmental monitoring or rapid search. Changes in the state of either party trigger corresponding adjustments in the other, thus achieving a truly dynamic collaborative working mode. Furthermore, the simulation platform plays a crucial role in this process; it can not only simulate various real-world situations but also provide predictive analysis based on virtual reality results for actual operations, further enhancing the overall performance of the system.

[0166] This embodiment constructs a global control problem based on three-dimensional spatial coordinates and attitude information, and decomposes it into multiple local sub-problems through behavior rules, formation control, or distributed strategies. Next, an initial robot swarm dynamics model is built, and rolling optimization is performed on these sub-problems to obtain local control sequences. These sequences are further optimized using the alternating direction multiplier method and then converted into specific task types. Finally, cooperative control instructions are generated according to task requirements, achieving refined control from global to local levels and enhancing the coordination and flexibility of the robot swarm when performing complex tasks.

[0167] Based on the first embodiment of this application, this application also provides a robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation. Please refer to... Figure 5 The device includes:

[0168] The setting module 10 is used to set up multiple optical motion capture cameras in an indoor space and obtain a unified coordinate system for the multiple cameras.

[0169] The acquisition module 20 is used to acquire image data of the target object using a camera.

[0170] The calculation module 30 is used to calculate the image data based on a unified coordinate system to obtain the three-dimensional spatial coordinates and pose information of the target object.

[0171] The simulation module 40 is used to construct a virtual simulation environment based on the indoor space and synchronously map the three-dimensional spatial coordinates and posture information to the virtual simulation environment to obtain a virtual reality result, wherein the virtual reality result is the display result of data fusion and synchronization between the target object and the virtual simulation environment.

[0172] The instruction generation module 50 is used to run a cluster control algorithm based on three-dimensional spatial coordinates and attitude information to generate cooperative control instructions. The robot cluster includes drones and unmanned vehicles.

[0173] The transmitting module 60 is used to send collaborative control commands to the robot cluster via wireless communication, so that the robot cluster can perform task actions according to the collaborative control commands. The robot cluster includes drones and unmanned vehicles.

[0174] The robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation provided in this application, employing the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation in the above embodiments, can solve the technical problem of how to improve the collaborative control capability and task execution efficiency of robot swarms in indoor environments. Compared with the prior art, the beneficial effects of the robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation provided in this application are the same as the beneficial effects of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation provided in the above embodiments, and other technical features in the robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0175] In one embodiment, the calculation module 30 is further configured to: select multiple reflective marker points on the surface of the target object based on image data; perform coordinate transformation on the multiple reflective marker points using a unified coordinate system to obtain the corresponding image coordinates; calculate the coordinates of the reflective marker points using a triangulation algorithm based on the image coordinates; construct a three-dimensional rigid body model based on the coordinates; perform attitude estimation on the three-dimensional rigid body model using a minimum mean square error rigid body registration algorithm to obtain a rotation matrix and a translation vector; calculate the centroid coordinates and initial attitude information of the target object based on the rotation matrix and translation vector; and perform exponential sliding filtering and spherical linear interpolation on the centroid coordinates and initial attitude information to obtain three-dimensional spatial coordinates and attitude information.

[0176] In one embodiment, the simulation module 40 is further configured to create a three-dimensional virtual scene based on the dimensions of the indoor space, wherein the three-dimensional virtual scene is consistent with the structure of the indoor space; set indoor space calibration points and obtain the three-dimensional coordinates of the calibration points; create virtual marker points corresponding to the calibration points based on the three-dimensional virtual scene and obtain their virtual coordinates; calculate using a scale-based optimization algorithm based on the three-dimensional coordinates of the calibration points and the virtual coordinates to obtain a rotation matrix, scaling factor, and translation vector; and map the three-dimensional spatial coordinates and attitude information onto the three-dimensional virtual scene through the rotation matrix, scaling factor, and translation vector to generate a virtual reality result.

[0177] In one embodiment, the simulation module 40 is further configured to: acquire a set of three-dimensional coordinates of all calibration points in the indoor space and a set of virtual coordinates of the corresponding calibration points in the virtual scene; calculate the mean of the three-dimensional coordinate set and the virtual coordinate set to obtain a first mean and a second mean, wherein the first mean corresponds to the three-dimensional coordinate set and the second mean corresponds to the virtual coordinate set; subtract the first mean from the three-dimensional coordinate set to obtain a first centered coordinate set; subtract the second mean from the virtual coordinate set to obtain a second centered coordinate set; calculate the covariance matrix of the first centered coordinate set and the second centered coordinate set to obtain a covariance matrix; perform singular value decomposition on the covariance matrix to obtain a decomposition result; construct a rotation matrix based on the decomposition result; calculate a scaling factor based on the covariance matrix and the second centered coordinate set; and calculate a translation vector based on the rotation matrix, the scaling factor, the first mean, and the second mean.

[0178] In one embodiment, the instruction generation module 50 is further configured to: construct a global control problem based on three-dimensional spatial coordinates and attitude information; decompose the global control problem into multiple local sub-problems through behavior rules, formation control, or distributed control strategies; construct an initial robot swarm dynamics model; perform rolling optimization on the multiple local sub-problems based on the initial robot swarm dynamics model to obtain a local control sequence; optimize the local control sequence using the alternating direction multiplier method to obtain an optimized control sequence; transform the optimized control sequence to obtain a task type; and generate cooperative control instructions based on the task type, three-dimensional spatial coordinates, attitude information, and virtual reality results.

[0179] In one embodiment, the instruction generation module 50 is further configured to set a safe distance threshold between the UAV and the unmanned vehicle based on the task type; calculate the relative distance between the UAV and the unmanned vehicle based on the three-dimensional spatial coordinates and attitude information combined with the virtual reality results; make a judgment based on the safe distance threshold and the relative distance to obtain a judgment result; when the relative distance is greater than the safe distance threshold, generate a cooperative control instruction, wherein the cooperative control instruction is for the unmanned vehicle to first move along the planned path and report its real-time position, and the UAV to adjust its flight trajectory based on the real-time position of the unmanned vehicle.

[0180] In one embodiment, the sending module 60 is further configured to receive task execution status information fed back by the robot cluster to update the virtual simulation environment and obtain an updated virtual scene; modify the collaborative control instructions according to the updated virtual scene to obtain calibration collaborative control instructions; and send the calibration collaborative control instructions to the robot cluster so that the robot cluster can re-execute task actions according to the calibration collaborative control instructions to achieve closed-loop control.

[0181] This application provides a robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation. The robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation includes: at least one processor; and a memory communicatively connected to at least one processor; wherein the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable at least one processor to execute the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation in the above embodiment 1.

[0182] The following is for reference. Figure 6 This document illustrates a structural schematic diagram of a robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation, suitable for implementing embodiments of this application. The robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0183] like Figure 6As shown, the robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation to exchange data with other devices wirelessly or via wired communication. Although various robot swarm collaborative control devices based on indoor positioning and virtual-real fusion simulation are shown in the figures, it should be understood that implementation or possession of all of them is not required. More or fewer may be implemented alternatively.

[0184] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0185] The robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation provided in this application, employing the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation in the above embodiments, can solve the technical problem of how to improve the collaborative control capability and task execution efficiency of robot swarms in indoor environments. Compared with the prior art, the beneficial effects of the robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation provided in this application are the same as the beneficial effects of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation provided in the above embodiments, and other technical features in this robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0186] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0187] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0188] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the robot swarm cooperative control method based on indoor positioning and virtual-real fusion simulation in the above embodiments.

[0189] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible storage medium containing or storing a program that can be executed by instructions, used by a device, or used in conjunction with it. The program code contained on the computer-readable storage medium may be transmitted using any suitable storage medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0190] The aforementioned computer-readable storage medium may be included in a robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation; or it may exist independently and not be assembled into a robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation.

[0191] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a robot swarm collaborative control device based on indoor positioning and virtual-real fusion simulation, enable the device to write computer program code for performing the operations of this application in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0192] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using dedicated hardware-based implementations that perform the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions.

[0193] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0194] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the above-described robot swarm cooperative control method based on indoor positioning and virtual-real fusion simulation. This addresses the technical problem of how to improve the cooperative control capability and task execution efficiency of robot swarms in indoor environments. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the robot swarm cooperative control method based on indoor positioning and virtual-real fusion simulation provided in the above embodiments, and will not be elaborated upon here.

[0195] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation as described above.

[0196] The computer program product provided in this application can solve the technical problem of how to improve the collaborative control capability and task execution efficiency of robot swarms in indoor environments. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation provided in the above embodiments, and will not be repeated here.

[0197] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for cooperative control of a robot swarm based on indoor positioning and virtual-real fusion simulation, characterized in that, The application relates to a method for constructing a virtual simulation environment and generating a collaborative control instruction. The method comprises the following steps: a plurality of optical dynamic capture cameras are arranged in an indoor space, and a unified coordinate system of the plurality of cameras is obtained; image data of a target object is obtained by using the cameras; three-dimensional space coordinates and attitude information of the target object are obtained by calculating the image data based on the unified coordinate system; a virtual simulation environment is constructed based on the indoor space, and the three-dimensional space coordinates and the attitude information are synchronously mapped to the virtual simulation environment to obtain a virtual real scene result, wherein the virtual real scene result is a data fusion and synchronous display result of the target object and the virtual simulation environment; a swarm control algorithm is run according to the three-dimensional space coordinates and the attitude information in combination with the virtual real scene result to generate a collaborative control instruction; the collaborative control instruction is sent to a robot swarm in a wireless communication mode, so that the robot swarm executes a task action according to the collaborative control instruction, and the robot swarm comprises a drone and an unmanned vehicle; the step of running the swarm control algorithm according to the three-dimensional space coordinates and the attitude information in combination with the virtual real scene result to generate the collaborative control instruction comprises the following steps: a global control problem is constructed based on the three-dimensional space coordinates and the attitude information; the global control problem is decomposed into a plurality of local sub-problems by using a behavior rule, a formation control or a distributed control strategy; an initial robot swarm dynamics model is constructed; a local control sequence is obtained by rolling optimization of the plurality of local sub-problems based on the initial robot swarm dynamics model; the local control sequence is optimized by using an alternating direction multiplier method to obtain an optimized control sequence; the optimized control sequence is converted to obtain a task type; 2. The method of claim 1, wherein, the collaborative control instruction is generated based on the task type, the three-dimensional space coordinates, the attitude information and the virtual real scene result. the step of calculating the three-dimensional space coordinates and the attitude information of the target object based on the unified coordinate system comprises the following steps: a plurality of reflective marker points on a surface of the target object are selected according to image data; image coordinates of the plurality of reflective marker points are obtained by coordinate conversion of the plurality of reflective marker points based on the unified coordinate system; coordinates of the reflective marker points are calculated based on the image coordinates by using a triangulation algorithm; a three-dimensional rigid body model is constructed according to the coordinates; a rotation matrix and a translation vector are obtained by performing attitude estimation on the three-dimensional rigid body model by using a least mean square error rigid body registration algorithm; a center of mass coordinate and initial attitude information of the target object are calculated according to the rotation matrix and the translation vector; 3. The method of claim 1, wherein, the center of mass coordinate and the initial attitude information are subjected to exponential sliding filtering and spherical linear interpolation processing to obtain the three-dimensional space coordinates and the attitude information. the step of constructing a virtual simulation environment based on the indoor space and synchronously mapping the three-dimensional space coordinates and the attitude information to the virtual simulation environment to obtain a virtual real scene result comprises the following steps: a three-dimensional virtual scene is created according to the size of the indoor space, wherein the three-dimensional virtual scene is consistent with the structure of the indoor space; three-dimensional coordinates of indoor space calibration points are obtained. create a virtual marker point corresponding to the calibration point based on the three-dimensional virtual scene, and obtain a virtual coordinate; calculate a rotation matrix, a scaling factor and a translation vector using a scale optimization algorithm according to the three-dimensional coordinates of the calibration point and the virtual coordinate; map the three-dimensional space coordinates and the attitude information into the three-dimensional virtual scene through the rotation matrix, the scaling factor and the translation vector, and generate a virtual real scene result.

4. The method of claim 3, wherein, The step of calculating a rotation matrix, a scaling factor and a translation vector using a scale optimization algorithm according to the three-dimensional coordinates of the calibration point and the virtual coordinate comprises: obtain a three-dimensional coordinate set of all calibration points in the indoor space and a virtual coordinate set corresponding to the calibration points in the virtual scene; perform mean value calculation on the three-dimensional coordinate set and the virtual coordinate set to obtain a first mean value and a second mean value, wherein the first mean value corresponds to the three-dimensional coordinate set, and the second mean value corresponds to the virtual coordinate set; subtract the three-dimensional coordinate set from the first mean value to obtain a first centralized coordinate set; subtract the virtual coordinate set from the second mean value to obtain a second centralized coordinate set; perform covariance matrix calculation on the first centralized coordinate set and the second centralized coordinate set to obtain a covariance matrix; perform singular value decomposition on the covariance matrix to obtain a decomposition result; construct a rotation matrix according to the decomposition result; calculate a scaling factor according to the covariance matrix and the second centralized coordinate set; calculate a translation vector according to the rotation matrix, the scaling factor, the first mean value and the second mean value.

5. The method of claim 1, wherein, The step of generating a cooperative control instruction based on the task type, the three-dimensional space coordinates and the attitude information in combination with the virtual real scene result comprises: set a safety distance threshold of a UAV and an unmanned vehicle according to the task type; calculate the relative distance between the UAV and the unmanned vehicle according to the three-dimensional space coordinates and the attitude information in combination with the virtual real scene result; judge according to the safety distance threshold and the relative distance to obtain a judgment result; when the relative distance is greater than the safety distance threshold, generate a cooperative control instruction, wherein the cooperative control instruction is that the unmanned vehicle moves along a planned path first and feeds back a real-time position, and the UAV adjusts a flight trajectory according to the real-time position of the unmanned vehicle.

6. The method of claim 1, wherein, After the step of sending the cooperative control instruction to the robot cluster through a wireless communication mode to enable the robot cluster to perform a task action according to the cooperative control instruction, the following steps are included: update the virtual simulation environment according to the task execution state information fed back by the robot cluster to obtain an updated virtual scene; correct the cooperative control instruction according to the updated virtual scene to obtain a calibrated cooperative control instruction; send the calibrated cooperative control instruction to the robot cluster to enable the robot cluster to perform a task action again according to the calibrated cooperative control instruction to realize closed-loop control.

7. A robot swarm cooperative control device based on indoor positioning and virtual-real fusion simulation, characterized in that, The device comprises: The device comprises a memory, a processor, and a robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation stored on the memory and running on the processor, and the robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation is configured to implement the steps of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation according to any one of claims 1-6. The storage medium stores a robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation, and the robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation implements the steps of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation according to any one of claims 1-6 when executed by the processor. The device comprises a memory, a processor, and a robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation stored on the memory and running on the processor, and the robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation is configured to implement the steps of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation according to any one of claims 1-6. The storage medium stores a robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation, and the robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation implements the steps of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation according to any one of claims 1-6 when executed by the processor. The device comprises a memory, a processor, and a robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation stored on the memory and running on the processor, and the robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation is configured to implement the steps of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation according to any one of claims 1-6. The storage medium stores a robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation, and the robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation implements the steps of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation according to any one of claims 1-6 when executed by the processor.

8. A robot swarm cooperative control device based on indoor positioning and virtual-real fusion simulation, characterized in that, The device comprises a memory, a processor, and a robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation stored on the memory and running on the processor, and the robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation is configured to implement the steps of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation according to any one of claims 1-6.

9. A storage medium, characterized by The storage medium stores a robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation, and the robot swarm collaborative control program based on indoor positioning and virtual-real fusion simulation implements the steps of the robot swarm collaborative control method based on indoor positioning and virtual-real fusion simulation according to any one of claims 1-6 when executed by the processor.

Images