Unmanned aerial vehicle task operation guiding device and system

Abstract

The utility model provides an unmanned aerial vehicle task operation guiding device and system, an image identification sensor is located on augmented reality glasses, and motion capture positioners are worn on index fingers and wrists of two hands of an unmanned aerial vehicle operator. The manual marking stickers are pasted to the edge vertexes of all the display screens of the triple folding display screen and used for marking the fusion boundary of a virtual picture displayed in the augmented reality glasses. The space positioning radar is installed at the top end of the triple folding display screen of the unmanned aerial vehicle ground station. The augmented reality glasses and the space positioning radar are in communication connection with the integrated operation terminal. And the task plug-in is connected with the integrated operation terminal. On the premise that hardware configuration and software functions of an existing unmanned aerial vehicle ground station are not changed, real-time guidance of training tasks and real-time correction of training operation can be achieved, unmanned aerial vehicle systems of various types and models can be universally adapted, and on the premise that the reliability and the service life of an original unmanned aerial vehicle ground control station system are not affected, real-time guidance of training tasks and real-time correction of training operation can be achieved. The training efficiency and the training quality are improved.

Description

Technical Field

[0001] The utility model relates to the technical field of aviation embedded training equipment, and in particular to a drone mission operation guidance device and system. Background Art

[0002] The current technology of UAV embedded training equipment mainly adds the function of ground model training to the related hardware and software systems of the UAV ground control station. Its implementation requires the ground station hardware to reserve computing resources and corresponding interactive display devices for training simulation, and the ground station software architecture needs to support function expansion and provide corresponding engineering source code. This technical solution has the following problems: First, adding software and hardware configurations for simulation training to the UAV ground station system that meets the actual flight requirements of UAV equipment will significantly increase the system development cost; second, adding training function modules to the ground control station system that meets the reliability requirements of UAV equipment will increase the system complexity and failure rate; finally, embedded ground simulation training cannot be implemented in the current UAV equipment that does not reserve hardware resources or the software architecture does not support expansion or cannot provide source code. Utility Model Content

[0003] In order to solve the problems of increased system cost, increased system complexity and failure rate in related technologies, and difficulty in implementation on the closed system of existing drones, the utility model provides a drone task operation guidance device and system, which can guide the drone operator to complete each step of the training task in real time through the virtual-real fusion scene in the augmented reality glasses, perceive each movement of the drone operator's hands in real time through motion capture and spatial positioning, and then provide real-time prompts and corrections through augmented reality glasses, so as to realize embedded simulation training based on the drone ground control station. The technical solution is as follows:

[0004] In the first aspect, a UAV mission operation guidance device is provided, comprising: augmented reality glasses, artificial marking stickers, image recognition sensors, motion capture locators, space positioning radars, integrated computing terminals and mission plug-ins,

[0005] The augmented reality glasses are worn by the drone operator, the image recognition sensor is located on the augmented reality glasses, and the motion capture locator is worn on the index fingers and wrists of the drone operator's hands; the manual marking stickers are pasted on the edge of the drone ground station display device, and the space positioning radar is installed on the top of the drone ground station display device; the augmented reality glasses, the space positioning radar and the integrated computing terminal are communicated and connected; the task plug-in is connected to the integrated computing terminal.

[0006] Optionally, the motion capture locator is worn on the index fingers and wrists of both hands of the drone operator via elastic rubber bands.

[0007] Optionally, the drone ground station display device is a triple folding display screen, which includes at least one display screen, and the manual marking stickers are pasted on the four edge vertices of each display screen of the triple folding display screen.

[0008] Optionally, the spatial positioning radar is mounted on top of the triple folding display screen via an adjustable bracket.

[0009] Optionally, the task plug-in is connected to the integrated computing terminal via a data bus slot.

[0010] Optionally, the augmented reality glasses and the spatial positioning radar are connected to the integrated computing terminal in a wireless communication manner.

[0011] In a second aspect, a UAV mission operation guidance system is provided, comprising: a UAV mission operation guidance device and a UAV ground control station, wherein the UAV mission operation guidance device comprises any device described in the first aspect.

[0012] In the utility model, when the UAV operator uses the ground control station to conduct embedded ground simulation training, the UAV task operation guidance device can realize real-time guidance of the training task and real-time correction of the training operation without changing the hardware configuration and software functions of the existing UAV ground station. It can be universally adapted to various types and models of UAV systems, and without affecting the reliability and service life of the original UAV ground control station system, it improves the training efficiency and training quality, reduces the system complexity and failure rate, and saves training costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of a UAV mission operation guidance device is provided for an embodiment of the utility model;

[0014] Figure 2 A schematic diagram of a UAV mission operation guidance system provided by an embodiment of the utility model;

[0015] Among them, 101-augmented reality glasses, 102-artificial marking stickers, 103-image recognition sensor, 104-motion capture locator, 105-space positioning radar, 106-adjustable bracket, 107-integrated computing terminal, 108-mission plug-in, 109-UAV ground control station, 110-triple folding display screen, 111-folding keyboard, 112-trackball, 113-UAV operator. DETAILED DESCRIPTION

[0016] With the continuous development of human-computer interaction technology, augmented reality technology based on 3D registration can dynamically add virtual entities to real scenes, and realize the fusion and synchronous interaction of virtual and real scenes. Applying it to the operation guidance of ground simulation flight missions of drones will effectively make up for the above problems of traditional drone embedded training equipment.

[0017] The present invention provides a UAV mission operation guidance device. Figure 1 As shown, it includes: augmented reality glasses 101, manual marking stickers 102, image recognition sensors 103, motion capture locators 104, space positioning radars 105, adjustable brackets 106, integrated computing terminals 107, and task plug-ins 108.

[0018] The augmented reality glasses 101 are worn by the drone operator 113 , the image recognition sensor 103 is located on the augmented reality glasses 101 , and the motion capture locator 104 is worn on the index fingers and wrists of both hands of the drone operator through elastic rubber bands.

[0019] The manual marking stickers 102 are pasted on the four edge vertices of each display screen of the triple folding display screen 110 in the drone ground station 109, and are used to mark the fusion boundary of the virtual picture displayed in the augmented reality glasses 101. The spatial positioning radar 105 is installed on the top of the drone ground station display device such as the triple folding display screen 110 through the adjustable bracket 106.

[0020] The augmented reality glasses 101 and the spatial positioning radar 105 are connected to the integrated computing terminal 107 in a wired or wireless manner.

[0021] The task plug-in 108 is connected to the integrated computing terminal 107 via a data bus slot.

[0022] In this embodiment, the task plug-in 108 can be configured according to the training requirements. The task plug-in 108 is configured with a simulation resource database. The functions of the simulation resource database include a basic operation guidance function and a task operation guidance function.

[0023] The utility model provides a UAV mission operation guidance system, such as Figure 2 As shown, including: Figure 1 The UAV mission operation guidance device and the UAV ground control station 109 shown in FIG.

[0024] The UAV ground control station 109 is provided with a triple folding display screen 110, a folding keyboard 111, and a trackball 112. The triple folding display screen 110 may include at least one display screen, such as Figure 2 In the embodiment, the triple folding display screen 110 includes three display screens.

[0025] The drone operator 113 wears the augmented reality glasses 101 and sits on a folding seat in the drone ground control station 109. The image recognition sensor 103 is located on the augmented reality glasses 101, and the motion capture locator 104 is worn on the index fingers and wrists of both hands of the drone operator through elastic rubber bands.

[0026] The artificial marking stickers 102 are pasted on the four edge vertices of each display screen in the triple folding display screen 110 to mark the fusion boundary of the virtual image displayed in the augmented reality glasses 101.

[0027] The spatial positioning radar 105 is installed on the top of the triple folding display screen 110 through an adjustable bracket 106.

[0028] The augmented reality glasses 101 and the spatial positioning radar 105 are connected to the integrated computing terminal 107 in a wired or wireless manner.

[0029] The task plug-in 108 is connected to the integrated computing terminal 107 via a data bus slot.

[0030] The use process of the UAV mission operation guidance device provided in this embodiment includes the following steps:

[0031] Step 1: The image recognition sensor 103 on the augmented reality glasses 101 recognizes the artificial marking stickers 102 pasted on the four edge vertices of each display screen in the triple folding display screen 110, and performs three-dimensional registration of the spatial position and structure of each marked display screen. The augmented reality glasses 101 then transmits the three-dimensional registration information to the integrated computing terminal 107 via a wireless or wired communication link; the process of three-dimensional registration can refer to the relevant technology;

[0032] Step 2, the integrated computing terminal 107 calls the simulation resource database in the task plug-in 108 according to the preset training task, uses the three-dimensional model of the UAV ground control station 109 in the simulation resource database and the operation behavior model of the UAV operator 113 to generate virtual scene data, retrieves the corresponding task operation prompt information from the prompt information rules in the simulation resource database, and transmits the generated virtual scene data and the retrieved task operation prompt information to the augmented reality glasses 101 through a wireless communication link. The UAV operator 113 observes the synthetic image in the virtual-real fusion scene through the augmented reality glasses 101. The synthetic image consists of two parts, one part is the virtual scene data and the task operation prompt information, and the other part is the display screen of each display screen in the triple folding display screen 110. The UAV operator 113 completes the corresponding operation on the folding keyboard 111 and the trackball 112 in the UAV ground control station 109 according to the task operation prompt information;

[0033] Step 3: The spatial positioning radar 105 installed on the adjustable bracket 106 on the top of the triple folding display screen 110 performs real-time spatial positioning and motion trajectory tracking of the motion capture locator 104 worn on the index fingers and wrists of the drone operator 113, and transmits the collected task operation data of the drone operator 113 to the integrated operation terminal 107 via a wireless or wired communication link. The integrated operation terminal 107 compares the operation rules of the corresponding task in the task plug-in 108 with the task operation data of the drone operator 113 to form operation correction prompt information. The integrated operation terminal 107 then transmits the operation correction prompt information to the augmented reality glasses 101 via a wireless or wired communication link to correct the task operation of the drone operator 113 in real time.

[0034] For example, when the drone operator is performing the monitoring operation of the drone mission payload and communication link, the above system will guide the drone operator's line of sight to the display where the interface of the drone ground station mission payload monitoring software and communication link monitoring software is located through the virtual arrows in the edge area of ​​the field of vision in the augmented reality glasses. Then, according to the training task steps, the prompt information of each step of the operation is superimposed in real time on the display screen, keyboard, trackball and other devices corresponding to the step of the operation, guiding the drone operator to perform the task according to the steps.

[0035] For example, the current mission operation is to control the mission payload to manually track and measure the distance to the mission target. The above-mentioned device will guide the UAV operator in real time to adjust the deflection angle of the mission payload gimbal on the trackball according to the operation prompt information and the correction information after each step of the operation to align it with the target, thereby completing all the operation steps of target tracking and ranging.

[0036] For example, the current task operation is to control the communication link to switch the main and backup communication links. The above-mentioned device will guide the drone operator in real time to complete all the operation steps of switching the main and backup links on the keyboard according to the operation prompt information and the correction information after each step of the operation.

[0037] The above only expresses the implementation mode of the utility model, and its description is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the utility model, which all belong to the protection scope of the utility model. In addition, the parts not detailed in the utility model are all conventional technologies.

Claims

1. A UAV mission operation guidance device, characterized in that: include: Augmented reality glasses, artificial marking stickers, image recognition sensors, motion capture locators, space positioning radars, integrated computing terminals and task plug-ins, The augmented reality glasses are worn by the drone operator, the image recognition sensor is located on the augmented reality glasses, and the motion capture locator is worn on the index fingers and wrists of the drone operator's hands; the manual marking stickers are pasted on the edge of the drone ground station display device, and the space positioning radar is installed on the top of the drone ground station display device; the augmented reality glasses, the space positioning radar and the integrated computing terminal are communicated and connected; the task plug-in is connected to the integrated computing terminal.

2. The device according to claim 1, characterized in that The motion capture locator is worn on the index fingers and wrists of the drone operator's hands via elastic rubber bands.

3. The device according to claim 1, characterized in that The display device of the drone ground station is a triple folding display screen, which includes at least one display screen, and artificial marking stickers are pasted on the four edge vertices of each display screen of the triple folding display screen.

4. The device according to claim 1, characterized in that The spatial positioning radar is installed on top of the triple folding display screen via an adjustable bracket.

5. The device according to claim 1, characterized in that The task plug-in is connected to the integrated computing terminal through the data bus slot.

6. The device according to claim 1, characterized in that The augmented reality glasses and the space positioning radar are connected to the integrated computing terminal in a wireless communication manner.

7. A UAV mission operation guidance system, characterized in that: include: An unmanned aerial vehicle mission operation guidance device and an unmanned aerial vehicle ground control station, wherein the unmanned aerial vehicle mission operation guidance device comprises the device described in any one of claims 1 to 6.

Images