Virtual-real fusion asymmetric mapping method based on cooperative control of double unmanned aerial vehicles

Abstract

The invention relates to a virtual-real fusion asymmetric mapping method based on dual unmanned aerial vehicle cooperative control, and the method comprises the steps: building a dual-vehicle asymmetric mapping model for a single-agent task scene in a virtual-real fusion scene; the pose information of the two real agents is synchronized according to the task scene limitation and the task time, and the two real agents autonomously and alternately receive the information of the virtual agent sensor and alternately complete the specified task. The invention relates to synchronization of a virtual world coordinate system and a real world coordinate system and displacement rotation transformation when two real agent local coordinate systems alternately map the same virtual agent coordinate system. And setting a dual-computer alternative mapping mechanism to realize real-time mapping of a large-scale operation space. According to the method, the scene space scale limitation of the virtual-real fusion experiment when the physical space is insufficient is made up, and distortion caused by simple speed and length scaling transformation in order to realize mapping of a small space to a large space is avoided.

Description

Technical Field

[0001] This invention relates to the field of virtual-real fusion technology, and in particular to a virtual-real fusion asymmetric mapping method based on dual UAV cooperative control. Background Technology

[0002] Existing virtual-real fusion technologies are usually based on digital twins, and their main function is to monitor the actual task process in real time. For example, virtual-real fusion (digital twin) technology in a workshop monitors and records the production information in the workshop in real time, and the data is used to improve the production model. However, these two processes are independent of each other.

[0003] The virtual-real fusion technology and digital twin technology are fundamentally different. Digital twin mainly involves data transmission from reality to the virtual world, while digital twin not only involves this data transmission process, but also includes real-time feedback from the virtual world to the real world. In addition, human factors are involved, acting as a third world to intervene. Most existing virtual-real fusion technologies that apply this concept have achieved the same proportion of spatial mapping. At the same time, various information in the virtual environment is subject to the constraints and limitations of the real world.

[0004] Current drone control methods suffer from limitations in the spatial scale of virtual-real scene fusion when physical space is insufficient. Furthermore, the common practice of mapping large spaces to small ones often involves simple speed and length scaling transformations, which can easily lead to distortion. Currently, there is a lack of drone control methods that can solve or partially solve these problems. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a virtual-real fusion asymmetric mapping method based on dual-UAV cooperative control. It aims to use a dual-UAV cooperative approach to synchronize the pose of two real UAVs with one virtual UAV, and the two UAVs alternately receive the sensor information of the virtual UAV and alternately complete specific tasks, thereby expanding the applicability of virtual-real fusion under the condition of limited physical space.

[0006] The objective of this invention can be achieved through the following technical solutions: One aspect of the present invention provides a virtual-real fusion asymmetric mapping method based on dual-UAV cooperative control, which executes the task of a single virtual UAV in virtual space through the cooperation of two real UAVs. The dual-UAV cooperative control process includes the following steps: A virtual environment is constructed and the task path of the virtual drone is generated. The task path of the virtual drone is segmented and assigned to two real drones. Acquire the real-time pose information of the real drone and synchronize it to the virtual drone through motion capture; Set a boundary area in the physical space. When any real drone reaches the boundary area, the switching of the controlled real drone is triggered, so as to realize the asymmetric dynamic alternation mapping between the virtual drone and the controlled real drone. During the switching process, based on the pose information of the virtual drone, the starting position and attitude of the relaying actual drone are calculated, and the mapping is completed through rotation and translation to achieve a smooth connection of the motion trajectories of the two actual drones.

[0007] As a preferred technical solution, the physical space is divided from the outside to the inside as follows: When a real drone enters the boundary zone, it immediately stops the mission, rotates 180° in place and hovers, preparing for the next takeover mission. When one real drone is in the safe zone, the boundary zone serves as a waiting area for the other real drone. When the two real drones switch, the boundary zone serves as a transition area for the initial pose of the coordinate system. The waiting real drone senses the real drone that is about to arrive at the boundary zone and avoids collision. The buffer zone serves as a buffer area for speed synchronization between the two drones. When one actual drone reaches the boundary area and switches to another actual drone, the pose information of the virtual drone and the controlled real drone are smoothly connected through coordinate system transformation, and the speed information of the virtual drone remains synchronized with that of the real drone. The safe zone serves as a safe flight area for the actual drone during mapping. Only one drone operates within this zone during each mapping round. Within the safe zone, the pose changes of the virtual drone are synchronized with those of the real drone, meaning a symmetrical mapping with a transformed initial coordinate system is performed.

[0008] As a preferred technical solution, when both real drones are within the buffer zone, they synchronously adjust their speed and attitude to achieve a smooth transition of the virtual drone's motion state from one real drone to the other during switching, ensuring the continuity of the mission.

[0009] As a preferred technical solution, the process of segmenting the mission path of a virtual drone and assigning it to two real drones includes the following steps: Based on the complete task path planned in the virtual space, the complete task path is dynamically divided into several sub-trajectory segments according to the size of the physical space and the setting of the boundary area. The sub-trajectory segments are alternately and continuously assigned to two real drones to achieve mission continuity.

[0010] As a preferred technical solution, the asymmetric dynamic alternation mapping between the virtual drone and the controlled real drone includes the following steps: Forward mapping: The attitude of the real drone is mapped onto the virtual drone through translation and rotation transformation, thus coupling the motion trajectories of the two real drones and the virtual drone. Inverse mapping: The trajectory segments are alternately mapped to two real drones.

[0011] As a preferred technical solution, the forward mapping process includes the following steps: Acquire the attitude information of the virtual drone when it reaches the boundary, as well as the initial pose information of another drone, calculate the departure angle of the first real drone that has reached the boundary, and the direction of the second real drone waiting to depart. When the second real drone is in flight, the endpoint coordinates of the previous mapping process and the real drone are rotated around the z-axis to obtain the three-dimensional coordinates of the virtual drone. The rotation angle is calculated based on the actual departure angle of the second real drone. The quaternion and time of the virtual drone are obtained through the conversion between quaternions and Euler angles, thus realizing the mapping.

[0012] As a preferred technical solution, the reverse mapping process includes the following steps: Based on the actual UAV's trajectory and pose planning information and the trajectory information during the surveying process, control commands are generated and sent to the actual UAV. The mapping is achieved by obtaining the corresponding quaternion based on the control command.

[0013] As a preferred technical solution, the real-time pose information of the real drone is obtained based on an optical motion capture system, and the real-time pose information is aligned with events in the virtual environment.

[0014] In another aspect, an electronic device is provided, comprising: one or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for executing the aforementioned virtual-real fusion asymmetric mapping method based on dual UAV cooperative control.

[0015] In another aspect, the present invention provides a computer-readable storage medium including one or more programs executable by one or more processors of an electronic device, the one or more programs including instructions for performing the aforementioned virtual-real fusion asymmetric mapping method based on dual UAV cooperative control.

[0016] Compared with the prior art, the present invention has at least one of the following beneficial effects: (1) Expanding the scope of application of virtual-real fusion under the condition of limited physical space: This invention is aimed at the single-agent task scenario in the virtual-real fusion scenario. A virtual drone synchronizes the pose information of two real drones according to the task scenario constraints and task time. The two real drones autonomously and alternately receive the sensor information of the virtual drone and alternately complete the prescribed tasks (such as planning, navigation, mapping, exploration, etc.), which makes up for the limitation of scene space scale in virtual-real fusion experiments when physical space is insufficient.

[0017] (2) Reduce distortion: Each trajectory segment of the present invention can be regarded as an equal-scale mapping during operation. The motion state of the UAV will not be scaled. It can always be directly referenced with the motion state of the real UAV. This can effectively expand the applicable scope of virtual-real fusion experiments and avoid the problem of distortion caused by simple speed and length scaling transformation in order to achieve mapping of small space to large space.

[0018] (3) High security: This invention serves large-scale virtual-real fusion experimental scenarios. For some dangerous and complex scenarios, experiments can obtain relatively realistic result data and models through this invention, while avoiding damage to the agent during training and effectively reducing the safety cost of the experiment. Attached Figure Description

[0019] Figure 1 This is a flowchart of the virtual-real fusion asymmetric mapping method based on dual UAV cooperative control in the embodiment; Figure 2 This is a schematic diagram illustrating the implementation principle of dual-machine collaboration in the embodiment; Figure 3 This is a schematic diagram illustrating the synchronization of virtual and real poses of the UAV in the embodiment. Figure 4 This is a schematic diagram of the dual-machine switching in the embodiment; Figure 5 This is a schematic diagram of a unified timestamp in the embodiment; Figure 6 This is a schematic diagram showing the actual site area setup in the embodiment; Figure 7 This is a schematic diagram of the electronic device in the embodiment. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] Example 1 To address the problems of the aforementioned existing technologies, this embodiment provides a dual-UAV cooperative control method based on virtual-real fusion and asymmetric mapping. The method uses the VICON motion capture system to achieve attitude synchronization between virtual and real UAVs. Boundaries are set within a certain physical space, and a dual-UAV cooperative approach is adopted. One virtual UAV synchronizes the poses of two real UAVs, and the two UAVs alternately receive the sensor information of the virtual UAV and alternately complete specific tasks. This effectively expands the applicability of virtual-real fusion experiments under the condition of limited physical space.

[0022] See Figure 1 and Figure 2 The method includes the following steps: Step S1: Construct a virtual environment and generate a mission path for the virtual drone. Divide the mission path of the virtual drone into trajectory segments and assign them to two real drones.

[0023] Construct a virtual environment and define the task paths for the virtual agents. Model the virtual scene using Unity3D and import it into the ISAAC SIM platform for rendering.

[0024] Specifically, when segmenting and assigning trajectories, the complete mission path planned in the virtual space is dynamically divided into several sub-trajectory segments based on the size of the real physical space and the setting of boundary areas. These sub-trajectory segments will be alternately and continuously assigned to two real drones to ensure the integrity and seamless connection of the virtual mission.

[0025] Step S2: Obtain the real-time pose information of the real drone and synchronize it to the virtual drone through motion capture.

[0026] The system acquires the real-time pose information of the real drone and synchronizes it to the virtual drone via the VICON optical motion capture system to achieve attitude synchronization. The virtual path is then segmented and assigned to the two real drones.

[0027] Specifically, the spatial position and attitude of the real drone are acquired using the VICON optical motion capture system and the ROS communication module, and then transmitted to the virtual intelligent agent model via the VICON-BRIDGE interface. Step S3: Set the boundary area in the physical space. When any real drone reaches the boundary area, trigger the switching of the controlled real drone, realizing the asymmetric dynamic alternation mapping between the virtual drone and the controlled real drone. The overall process is as follows: Figure 4 As shown in (a) A physical space boundary region is defined, and a dual-drone collaborative strategy is adopted. When either real drone reaches the defined physical boundary region, a drone mapping switch event is triggered. A buffer region is set up to perform speed matching and attitude transition, ensuring the continuity of virtual task execution and the safety of the real space.

[0028] Specifically, the physical space is divided into a safe zone, a buffer zone, and a boundary zone. When the real drone performing the mission enters the boundary zone, it immediately stops executing the mission, rotates 180° in place, and hovers, preparing for the next takeover mission. The virtual drone's attitude will then switch to the attitude of the next real drone. The attitude of the next drone is rotated and translated based on the attitude recorded when the previous drone crossed the boundary, and then mapped onto the virtual drone. The above transformation process is as follows: Figure 4 As shown in (b) of the diagram.

[0029] The safe zone is the safe flight area for drones to perform mapping. Each mapping round ensures that only one drone operates in this zone. Within this zone, the pose changes of the virtual drone are synchronized with those of the real drone, which is equivalent to a symmetrical mapping that transforms the initial coordinate system.

[0030] The buffer zone serves as a buffer for speed synchronization between the two drones. When the previous drone reaches the boundary and switches to the next, its pose information can be smoothly transitioned through coordinate system transformation, but its speed information remains synchronized with the real drone. Without a buffer zone for the next drone to accelerate, the virtual drone's flight speed would momentarily drop to zero and then accelerate during the transition phase. The buffer zone incorporates speed matching and attitude transition mechanisms. During the switch between the two drones, the two real drones within the buffer zone will synchronously adjust their speed and attitude to ensure that at the switch point, the virtual drone's motion state (including position, speed, and orientation) smoothly transitions seamlessly from one real drone to the other, without any lag during the switch, thus avoiding abrupt or discontinuous phenomena during virtual task execution.

[0031] Within the boundary zone, when one drone is in a safe flight phase, this area serves as a waiting zone for the other drone; when the two drones are in the transition phase, this area serves as a transition zone for the initial pose of the coordinate system. At the same time, drones that have not yet departed can sense drones that are about to reach the boundary here to avoid collisions caused by the endpoint of the previous drone and the starting point of the next drone being too close.

[0032] During mission switching, the two real drones within the buffer zone will synchronously adjust their speed and attitude to ensure that at the switching point, the motion state (including position, speed and orientation) of the virtual drone can smoothly and seamlessly transition from one real drone to the other, avoiding abrupt or discontinuous phenomena during the execution of the virtual mission.

[0033] During the asymmetric mapping process, the pose information of the real UAV in the limited physical space is accurately mapped to the virtual UAV in the larger-scale virtual space through dynamic coordinate system transformation. At the same time, the information perceived in the virtual environment (such as obstacle information) is transmitted to the real UAV after inverse transformation, so that the real UAV can perform path planning and obstacle avoidance in the physical space based on the perceived information of the virtual world.

[0034] Step S4: During the switching process, based on the pose information of the virtual drone, the starting position and attitude of the relaying actual drone are calculated, and the mapping is completed through rotation and translation to achieve a smooth connection of the motion trajectories of the two actual drones.

[0035] Based on the pose information of the virtual intelligent agent, the starting position and attitude of the relay drone are calculated, and the mapping is completed through rotation and translation matrices to ensure the smooth connection of the drone's motion trajectory.

[0036] Specifically, the mappings in steps S3 and S4 include mappings from reality to virtuality and dual-drone collaborative mapping.

[0037] (1) The mapping from reality to virtuality.

[0038] Taking drones as an example, the process involves mapping real-world drones to virtual drones. In terms of attitude synchronization, all state variables of the drone need to be synchronized, including 3D position data. Attitude quaternion data and time dimension .

[0039] For the transformation of 3D position data, assume the transformation matrix from the virtual world coordinate system to the real world coordinate system is: Virtual coordinates are used Indicates that the actual coordinates are used The mapping relationship of the three-dimensional location data is as follows: (1) When the real-world coordinate system and the virtual-world coordinate system coincide, the mapping relationship satisfies: (2) For quaternion mapping, the relationship between the virtual and real-world coordinate systems, and the relationship between the drone's local coordinate system and the baseline coordinate system, also needs to be considered. After synchronizing the world coordinates, the virtual drone's local coordinate system needs to be rotated by a certain angle to obtain the desired result. Assume the real drone revolves around the world coordinate axes... and The quaternion relationships for rotation, roll, pitch, and yaw angles are shown in the following formulas.

[0040] (3) in yes It is a transformation matrix between the rotation of a real drone and the rotation of a virtual drone. It is a function that converts the rotation angle of Euler into a rotation quaternion.

[0041] Regarding time Because the virtual-real fusion experiment aims to connect various coordinate systems in the virtual world with those in the real world, both virtual and real coordinate systems have timestamp attributes. The relationship between two coordinate systems can only be bound when the timestamp difference is within a certain value. Otherwise, an error message will appear indicating that another coordinate system cannot be found within the specified timestamp range of one coordinate system, resulting in the loss of the relationship between the two coordinate systems. Figure 5 This is a schematic diagram of the unified timestamp process in this embodiment. Before conducting the experimental algorithm, the time in the simulation world needs to be aligned with the time in the real world, such as VICON, and then input into the planning algorithm for experimentation.

[0042] (2) Dual UAV collaborative mapping.

[0043] Assume the three-dimensional position parameters of the two drones are as follows: and The quaternion parameters are respectively and The time dimension parameter is... and The three-dimensional attitude parameters of the virtual drone are Quaternion parameters are The time dimension parameter is .

[0044] Asymmetric mappings are divided into forward and reverse mappings.

[0045] Forward mapping maps the attitude of a real drone onto a virtual drone through displacement and rotation transformation, making the motion trajectory of the virtual drone coupled with two real drones smooth and uninterrupted.

[0046] Reverse mapping is the process of dividing the computation results (such as the planned path) of the algorithm on the virtual drone into multiple trajectories based on specific conditions (such as safe flight distance) and alternately mapping them to two real drones.

[0047] The combination of forward and backward mappings enables the algorithms to be trained and applied in large-scale spaces. The following formula summarizes the process well: The location information of the virtual drone is calculated using the following formula: (4) The control commands derived from the algorithm results of real drones are as follows: in It contains the virtual agent's position information, attitude quaternion information, and time information. It is also information from a real intelligent agent. It is a transformation matrix related to the position of the actual drone. It is a probability parameter, defined as follows: For forward mapping, in terms of the venue, a safe flight zone was defined in the real experimental scenario, surrounded by a 30-centimeter-wide buffer zone. The outer edge of the buffer zone is the boundary of the safe zone. To ensure smooth drone connection, after the current drone leaves the boundary, when the mapped object becomes the next drone, the current drone returns to the boundary and rotates 180° to wait in place. Based on this setup, the actual departure angle of the drone at the boundary relative to 0° can only be one of four possibilities: 0°, +90°, -90°, and 180°. This situation is significant in subsequent trajectory mapping. Regarding the experimental setup, the following is defined... , and To record the attitude information of the virtual drone each time it reaches the boundary, the recorded position, attitude quaternion, and time are defined. , and To record the starting position, attitude, and time of another drone at this moment, and define... and This is used to record the current direction of the virtual drone and the actual departure angle of the next drone. They are numbered 1 and 2 drones respectively. Therefore, there are... , , and When the drone reaches the boundary, it is numbered 1; when it is waiting to depart, it is numbered 2. The parameter settings are as follows. The recorded parameters are: The starting parameters for the position are: It can be obtained : and for: When UAV 2 reaches the boundary and waits for UAV 1 to leave, the parameter settings are the same. When the second UAV is in flight, the corresponding mapped pose calculation for the virtual UAV is as follows. First, the mapping of three-dimensional coordinates. Here we only consider the rotational change in the heading angle yaw, because the aircraft's pitch and roll angles do not change significantly. Then, the endpoint coordinates of the previous mapping process are added to the three-dimensional coordinate changes of the actual aircraft rotating around the z-axis by a certain amount to obtain the three-dimensional coordinates of the virtual UAV, as shown in Formula 10 below. in yes At this point, using Formula 8, we first calculate the actual yaw angle of the UAV. and its actual departure yaw angle At this point, a new set of rotation angles will be obtained, where: Then, a new set of quaternions is obtained through the conversion relationship between quaternions and Euler angles. At this point, the quaternion and time parameters of the virtual drone are: Conversely, taking path planning as an example, after planning a trajectory based on the virtual environment, it is necessary to reasonably allocate the trajectory to two drones. This requires setting a series of variables to record the trajectory pose information, namely... , and At the same time, it is necessary to define variables to record the trajectory information of the actual drone during the mapping process, i.e. , and Their corresponding Euler angles of rotation are respectively , and When the mapping of drones changes, for example from drone 1 to drone 2, the corresponding variable is calculated as follows: The calculation is shown in Equation 8. The final position trajectory mapped to UAV 2 is shown in the following formula: in yes yes This is the change angle of the trajectory.

[0048] here These are the control commands sent to drone 2; similarly, the commands sent to drone 1 are... Since the 3D position information of the trajectory planned by the algorithm has been rotated, the attitude angles in the trajectory information should also be rotated. This is because the position information of the trajectory only exists in... Since the rotation is on the axis, the attitude angle only rotates by the yaw angle, resulting in a new yaw angle as follows: At this point, the initial attitude angle is: After the transformation, the new attitude quaternion can be obtained as follows: Then we can obtain the quaternion of the control command as: The following example validates this solution. Taking dual drones as an example, when applied to real-world drones, unlike simulation, to ensure timely coordinate switching, when a drone flies out of the boundary, it rotates 180° in place and waits, without needing to fly back to the origin first. To ensure a smooth and seamless transition in pose changes, the boundary in the real scene is a buffer zone, not a simple line. The two drones will synchronize their speeds within the buffer zone to ensure that the virtual drone does not lag during the switching process with the real drone. Simultaneously, to prevent collisions, if the current drone approaches the boundary and overlaps with or approaches the position of another drone, a collision risk exists. The drone waiting at the boundary will avoid collisions based on the distance between the two drones and its own distance from the boundary.

[0049] The real-world scenario in this example is an indoor floor, but the mapped virtual scene is much larger than the real-world scene. The target task remains path planning and obstacle avoidance, using the ego planner algorithm. The real-world location is set up as follows: Figure 6 As shown in the figure, the experimental site was divided into a buffer zone, a boundary zone, and a safe zone. Two drones surveyed the virtual drone in a large park within this area and collaboratively completed path planning.

[0050] Example 2 Based on Example 1, this example provides a dual-UAV cooperative control system based on virtual-real fusion and asymmetric mapping, used to implement the virtual-real fusion asymmetric mapping method based on dual-UAV cooperative control in Example 1. (See also...) Figure 3 The system uses the VICON optical motion capture system to locate the drone, and additionally equips the device with three infrared cameras. It uses the VICON-BRIDGE ROS software package to achieve communication between VICON and ROS, allowing the virtual drone model to receive attitude information from the real drone published by the VICON system, thus achieving attitude synchronization. Figure 3 The system and related data are shown below.

[0051] This invention overcomes the limitation of a one-to-one mapping between virtual and real worlds when conducting virtual-real fusion experiments in a limited real physical space. It enables arbitrary scaling of spatial and task scales from a small real space to a large virtual space, broadening the applicability of virtual-real fusion simulations. Specifically, for single-agent tasks in virtual-real fusion scenarios, the invention establishes a dual-machine asymmetric mapping model. Based on task scenario constraints and time constraints, the model synchronizes the pose information of two real agents, allowing them to autonomously and alternately receive sensor information from the virtual agent and perform designated tasks (planning, navigation, mapping, exploration, etc.). The model involves the synchronization of the virtual and real world coordinate systems and the displacement and rotation transformations when the local coordinate systems of the two real agents alternately map to the same virtual agent coordinate system. In actual tasks, by setting experimental scenario boundaries and corresponding functional areas, a dual-machine alternating mapping mechanism is implemented to achieve real-time mapping of large-scale work spaces. Simultaneously, additional sensing channels are set to enable agents to locate and avoid each other. This model compensates for the limitations of scene space scale in virtual-real fusion experiments when physical space is insufficient, and also avoids the distortion problem when performing simple speed and length scaling transformations to achieve mapping of small space to large space. It can effectively expand the applicable scope of virtual-real fusion.

[0052] Example 3 Based on the foregoing embodiments, this embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the virtual-real fusion asymmetric mapping method based on dual UAV cooperative control as described in Embodiment 1.

[0053] like Figure 7 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1The method described herein. Of course, in addition to software implementation, this invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0054] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0055] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0056] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A virtual-real fusion asymmetric mapping method based on cooperative control of two unmanned aerial vehicles, characterized in that, The task of a single virtual unmanned aerial vehicle in a virtual space is executed through cooperation of two real unmanned aerial vehicles, and the process of the double unmanned aerial vehicle cooperative control includes the following steps: A virtual environment is constructed, a task path of the virtual unmanned aerial vehicle is generated, the task path of the virtual unmanned aerial vehicle is trajectory segmented, and the two real unmanned aerial vehicles are assigned; Real-time pose information of the real unmanned aerial vehicles is acquired, and the virtual unmanned aerial vehicle is synchronized through motion capture; A boundary area in a physical space is set, when any real unmanned aerial vehicle reaches the boundary area, switching of the controlled real unmanned aerial vehicle is triggered, and dynamic asymmetric mapping of the virtual unmanned aerial vehicle and the controlled real unmanned aerial vehicle is realized; In the switching process, based on pose information of the virtual unmanned aerial vehicle, a starting position and an attitude of the actual unmanned aerial vehicle for taking over are calculated, mapping is completed through rotation and translation, and smooth connection of motion trajectories of the two real unmanned aerial vehicles is realized. 2.The virtual-real fusion asymmetric mapping method based on the cooperative control of dual unmanned aerial vehicles according to claim 1, wherein, The physical space is divided from outside to inside into: A boundary area, when a real unmanned aerial vehicle performing a task enters the boundary area, the task is immediately stopped, 180° in-place rotation is performed, and in-place hovering is performed, preparing for the next task taking over, when one real unmanned aerial vehicle is in a safe area, the boundary area is used as a waiting area for the other real unmanned aerial vehicle, when the two real unmanned aerial vehicles are switched, the boundary area is used as a conversion area of initial poses of coordinate systems, and the real unmanned aerial vehicle in the waiting state senses the real unmanned aerial vehicle about to reach the boundary area, and collision avoidance is realized; A buffer area, as a buffer area for speed synchronization of the two real unmanned aerial vehicles, when one real unmanned aerial vehicle reaches the boundary area and is switched to the other real unmanned aerial vehicle, pose information of the virtual unmanned aerial vehicle and the controlled real unmanned aerial vehicle is smoothly connected through coordinate system conversion, and speed information of the virtual unmanned aerial vehicle is kept synchronized with the real unmanned aerial vehicle; A safe area, as a safe flight area for mapping of the real unmanned aerial vehicle, only one real unmanned aerial vehicle runs in the safe area in each mapping round, and in the safe area, a pose change amount of the virtual unmanned aerial vehicle is kept synchronized with the real unmanned aerial vehicle, that is, symmetric mapping of the initial coordinate system is performed. 3.The virtual-real fusion asymmetric mapping method based on the cooperative control of dual unmanned aerial vehicles according to claim 1, characterized in that, When the two real unmanned aerial vehicles are in the buffer area, the speed and the attitude of each other are adjusted synchronously, motion state of the virtual unmanned aerial vehicle is smoothly transferred from one real unmanned aerial vehicle to the other real unmanned aerial vehicle in switching, and continuity of the task is ensured. 4.The virtual-real fusion asymmetric mapping method based on the cooperative control of dual UAVs according to claim 1, characterized in that, The process of trajectory segmentation of the task path of the virtual unmanned aerial vehicle and assignment to the two real unmanned aerial vehicles includes the following steps: Based on a complete task path planned in a virtual space, the complete task path is dynamically segmented into a plurality of sub-trajectory segments according to the size of the physical space and setting of the boundary area; The sub-trajectory segments are alternately and continuously assigned to the two real unmanned aerial vehicles for execution, and connection of the task is realized.

5. The virtual-real fusion asymmetric mapping method based on dual-UAV cooperative control according to claim 1, characterized in that, The dynamic asymmetric mapping of the virtual unmanned aerial vehicle and the controlled real unmanned aerial vehicle includes the following steps: Forward mapping: the attitude of the real unmanned aerial vehicle is mapped to the virtual unmanned aerial vehicle through displacement rotation conversion, so that motion trajectories of the two real unmanned aerial vehicles and the virtual unmanned aerial vehicle are coupled; Reverse mapping: the trajectory segments are alternately mapped to the two real unmanned aerial vehicles.

6. The virtual-real fusion asymmetric mapping method based on dual-UAV cooperative control according to claim 5, characterized in that, The forward mapping process includes the following steps: Obtaining the attitude information of the virtual UAV when reaching the boundary, and the initial pose information of the other UAV, calculating the departure angle of the first real UAV reaching the boundary, and the direction of the second real UAV waiting for departure; When the second real UAV is in flight, obtaining the three-dimensional coordinates of the virtual UAV by rotating the endpoint coordinates of the previous mapping process and the real UAV around the z-axis, calculating the rotation angle based on the actual departure angle of the second real UAV, and obtaining the quaternion of the virtual UAV through the conversion between the quaternion and the Euler angle, and the time, to realize the mapping.

7. The virtual-real fusion asymmetric mapping method based on dual-UAV cooperative control according to claim 5, characterized in that, The reverse mapping process includes the following steps: Based on the trajectory pose planning information of the real UAV and the trajectory information in the surveying and mapping process, the control command sent to the real UAV is generated; Based on the corresponding quaternion of the control command, the mapping is realized. 8.The virtual-real fusion asymmetric mapping method based on the cooperative control of dual UAVs according to claim 1, wherein, The real-time pose information of the real UAV is obtained based on an optical motion capture system, and the real-time pose information is aligned with the events in the virtual environment.

9. An electronic device, comprising: Comprise: One or more processors and a memory, the memory storing one or more programs, the one or more programs including instructions for performing the asymmetric mapping method of virtual-real fusion based on the cooperative control of double UAVs according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Comprise one or more programs for one or more processors of an electronic device to execute, the one or more programs including instructions for performing the asymmetric mapping method of virtual-real fusion based on the cooperative control of double UAVs according to any one of claims 1-8.

Images