Flight path guidance system, flight path guidance device and flight path guidance method

Through the flight path guidance system, the flight path of the unmanned aerial vehicle is displayed using a head-mounted display, which solves the problem that operators find it difficult to monitor the situation and flight path of the unmanned aerial vehicle, and realizes effective path monitoring and deviation correction of the unmanned aerial vehicle.

CN112204351BActive Publication Date: 2024-11-05RAKUTEN GROUP INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201880093649.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2018-11-21
Publication Date
2024-11-05
Estimated Expiration
2038-11-21

AI Technical Summary

Technical Problem

In the prior art, it is difficult for the operator or supervisor of an unmanned aerial vehicle to grasp the situation and flight path of the unmanned aerial vehicle in flight at the same time, and it is difficult to determine whether the unmanned aerial vehicle deviates from the flight path and returns.

Method used

The flight path guidance system is adopted, including the flight path information acquisition mechanism, a specific mechanism, an image generation mechanism and a display control mechanism, and the flight path image observed from a specific visual position and direction is displayed through an optical or image-penetrating head-mounted display. , and determine whether the unmanned aircraft is on the flight path through the determination mechanism, and perform image correction to display the correct flight path.

Benefits of technology

It is realized that the unmanned aerial vehicle operator or supervisor can monitor the position and flight path of the unmanned aerial vehicle at the same time, judge and correct deviations in a timely manner, and ensure that the unmanned aerial vehicle flies along the predetermined path.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112204351B_ABST
    Figure CN112204351B_ABST
Patent Text Reader

Abstract

This invention provides a flight path guidance system, a flight path guidance device, and a flight path guidance method capable of simultaneously monitoring the status of an unmanned aerial vehicle (UAV) in flight and its flight path. A flight path information acquisition unit (50) acquires flight path information representing the flight path of the UAV in actual space. An actual space position and direction determination unit (54) determines the viewing position and viewing direction in actual space. A flight path image generation unit (58) generates a flight path image based on the flight path information, showing the flight path as viewed from the viewing position and in the viewing direction. A display control unit (60) displays the flight path image on a display unit in a state aligned with the position as viewed from the viewing position and in the viewing direction in actual space.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a flight path guidance system, a flight path guidance device and a flight path guidance method. Background Art

[0002] Patent document 1 describes a transparent head-mounted display that allows the user to see virtual images such as a mark image that makes the self-driving vehicle easy to identify, a moving direction image that indicates the moving direction of the self-driving vehicle, and a predicted position image that indicates the predicted position after a specified time. According to the technology described in patent document 1, even if the self-driving vehicle moves far away or multiple unmanned aerial vehicles are in the field of vision, the user is unlikely to lose sight of the self-driving vehicle. In addition, it is also easy to grasp the moving direction or predicted position of the self-driving vehicle.

[0003] Patent Document 2 describes an operation management technology for correcting the flight path of an unmanned aerial vehicle according to weather conditions. By making the unmanned aerial vehicle fly along the flight path provided by the operation management technology described in Patent Document 2, the unmanned aerial vehicle can be flown more safely.

[0004] Background Art Literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2018-112809

[0007] Patent Document 2: Japanese Patent Application Publication No. 2018-081675 Summary of the invention

[0008] [Problems to be solved by the invention]

[0009] In the technology described in Patent Document 1 or Patent Document 2, the operator or supervisor of the UAV cannot simultaneously grasp both the situation of the UAV in flight and the flight path of the UAV. Therefore, sometimes it is difficult for the operator or supervisor of the UAV to grasp whether the UAV in flight can fly according to the flight path. In addition, there is a case where the operator or supervisor of the UAV cannot grasp how to return the UAV to the flight path when the UAV deviates from the flight path.

[0010] The present invention has been made in view of the above-mentioned problems, and one of its purposes is to provide a flight path guidance system, a flight path guidance device and a flight path guidance method that can simultaneously grasp the situation of an unmanned aerial vehicle in flight and the flight path of the unmanned aerial vehicle.

[0011] [Technical means to solve the problem]

[0012] In order to solve the above-mentioned problem, the flight path guidance system of the present invention includes: a flight path information acquisition mechanism, which acquires flight path information representing the flight path of an unmanned aerial vehicle in actual space; a specifying mechanism, which specifies a visual position and a visual direction in the actual space; an image generating mechanism, which generates, based on the flight path information, an image representing the flight path when the visual direction is observed from the visual position; and a display control mechanism, which displays the image on a display unit in a state where the image is aligned with the position of the visual direction when the visual position is observed in the actual space.

[0013] In one form of the present invention, the specific mechanism specifies the visual position corresponding to the position of the display unit, and the visual direction corresponding to the direction of the display unit, and the display unit is an optically transparent head-mounted display, so that the wearer of the head-mounted display can see the situation and the image of observing the visual direction from the visual position in the actual space.

[0014] Alternatively, the specific mechanism specifies the visual position corresponding to the position of a camera that captures the actual space, and the visual direction corresponding to the shooting direction of the camera, and the image generating mechanism generates the image in which the flight path when the visual direction is observed from the visual position is superimposed on the image captured by the camera.

[0015] In this form, the specific mechanism may specify the visual position corresponding to the position of the camera possessed by the image-penetrating head-mounted display, and the visual direction corresponding to the shooting direction of the camera, and the display control mechanism may display the image on the display unit possessed by the head-mounted display.

[0016] In addition, the head-mounted display can also be worn on the head of the operator of the unmanned aerial vehicle.

[0017] In addition, in one form of the present invention, it also includes a virtual space generating mechanism, which generates a virtual space in which virtual objects representing the flight path are configured, and the specific mechanism also specifies a visual position in the virtual space corresponding to the visual position in the actual space, and a visual direction in the virtual space corresponding to the visual direction in the actual space, and the image generating mechanism generates the image representing the situation of observing the visual direction in the virtual space from the visual position in the virtual space.

[0018] In addition, in one aspect of the present invention, the image generating means generates the image in a form corresponding to whether the unmanned aerial vehicle is on the flight path.

[0019] In one aspect of the present invention, a determination unit is further included for determining whether the unmanned aerial vehicle is on the flight path, and the display control unit controls whether to display an image with a corrected position reflecting the flight path based on a determination result of the determination unit.

[0020] In this aspect, the determination unit may determine whether the UAV is on the flight path based on the measured position of the UAV and the flight path information.

[0021] In one aspect of the present invention, the flight path information indicates, for each of a plurality of positions on the flight path, the time at which the unmanned aerial vehicle should be at the position, and the image generating means generates the image also showing the time.

[0022] In this aspect, the image generating means may generate the image for each of the plurality of positions on the flight path, showing whether the UAV should be at the position before or after the current time.

[0023] In one aspect of the present invention, the flight route information indicates a plurality of selectable flight routes, and the image generation unit generates the image capable of distinguishing the selected flight routes from the unselected flight routes.

[0024] In one aspect of the present invention, the image generation unit generates the image that expresses at least one of a planned flight path of the unmanned aerial vehicle indicated by the flight path information and an actual flight path of the unmanned aerial vehicle.

[0025] In addition, in one aspect of the present invention, the flight path information acquisition unit acquires the flight path information representing the flight paths of the plurality of unmanned aerial vehicles, and the image generation unit generates the image representing the flight paths of the plurality of unmanned aerial vehicles.

[0026] In this aspect, the image generation unit may generate the image that enables the flight path of the unmanned aerial vehicle being operated by an operator viewing the image to be distinguished from the flight paths of other unmanned aerial vehicles.

[0027] In addition, in one aspect of the present invention, the unmanned aerial vehicle has an autonomous flight function.

[0028] In addition, the flight path guidance device of the present invention includes: a flight path information acquisition mechanism, which acquires flight path information representing the flight path of an unmanned aerial vehicle in actual space; a specifying mechanism, which specifies a visual position and a visual direction in the actual space; an image generating mechanism, which generates an image representing the flight path when the visual direction is observed from the visual position based on the flight path information; and a display control mechanism, which displays the image on a display unit in a state aligned with the position of the visual direction observed from the visual position in the actual space.

[0029] In addition, the flight path guidance method of the present invention includes the following steps: acquiring flight path information representing the flight path of an unmanned aerial vehicle in real space; specifying a visual position and a visual direction in the real space; generating an image showing the flight path when the visual direction is observed from the visual position based on the flight path information; and displaying the image on a display unit in a state aligned with the position of the visual direction observed from the visual position in the real space. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a diagram showing an example of the overall configuration of a flight path guidance system according to an embodiment of the present invention.

[0031] Figure 2A This is a diagram showing an example of the configuration of a head mounted display according to an embodiment of the present invention.

[0032] Figure 2B It is a diagram showing an example of the configuration of an unmanned aerial vehicle according to one embodiment of the present invention.

[0033] Figure 3 This is a diagram showing an example of a virtual space.

[0034] Figure 4 This is a diagram showing an example of a flight path image.

[0035] Figure 5 The diagram shows an example of an unmanned aerial vehicle in real space viewed by a wearer of a head mounted display and a flight path image displayed on a display unit.

[0036] Figure 6 This is a diagram showing another example of an unmanned aerial vehicle in real space viewed by a wearer of a head mounted display and a flight path image displayed on a display unit.

[0037] Figure 7 This is a diagram showing another example of an unmanned aerial vehicle in real space viewed by a wearer of the head mounted display and a flight path image displayed on the display unit.

[0038] Figure 8 This is a functional block diagram showing an example of functions incorporated in a head mounted display according to an embodiment of the present invention.

[0039] Fig. 9 This is a flowchart showing an example of the flow of processing performed in the head mounted display according to one embodiment of the present invention. DETAILED DESCRIPTION

[0040] Hereinafter, one embodiment of the present invention will be described with reference to the drawings.

[0041] Figure 1 FIG. 1 is a diagram showing an example of the overall configuration of a flight path guidance system 1 according to an embodiment of the present invention. Figure 1 As shown, the flight path guidance system 1 of the present embodiment includes a head mounted display (HMD) 10 , an unmanned aerial vehicle 12 , an operation management system 14 , and an operation terminal 16 .

[0042] The HMD 10 is a display device that a user can wear on his head. In the following description, the HMD 10 of this embodiment is an optically transparent type (optically see-through type), and the wearer of the HMD 10 can see the situation in the real space when wearing it.

[0043] In the HMD 10 of this embodiment, for example Figure 2A As shown, it includes a processor 20 , a storage unit 22 , a communication unit 24 , a display unit 26 and a sensor unit 28 .

[0044] The processor 20 is, for example, a program-controlled device such as a microprocessor that operates according to a program installed in the HMD 10 .

[0045] The storage unit 22 is, for example, a storage element such as a ROM (Read Only Memory), a RAM (Random Access Memory), or a flash memory, etc. The storage unit 22 stores programs and the like executed by the processor 20 .

[0046] The communication unit 24 is a communication interface such as a wireless LAN (Local Area Network) module or a Bluetooth (registered trademark) module.

[0047] The display unit 26 is a display that allows the wearer of the HMD 10 to see the actual space and virtual images through a half mirror or the like, for example. The display unit 26 is arranged in front of the eyes of the wearer of the HMD 10. In addition, the display unit 26 of this embodiment is, for example, a binocular display that can display a three-dimensional image by displaying an image for the left eye and an image for the right eye. In addition, the display unit 26 may only display a two-dimensional image and cannot display a three-dimensional image.

[0048] The sensor unit 28 is a device including, for example, a GNSS (Global Navigation Satellite System) receiver such as a GPS (Global Positioning System), an inertial sensor such as an acceleration sensor or a gyro sensor, a geomagnetic sensor, and the like. The sensor unit 28 can measure the position or posture of the HMD 10. The sensor unit 28 can also output the measurement results such as the position or posture of the HMD 10 to the processor 20 at a specified sampling rate. In addition, the sensor unit 28 can also include a camera that captures an image of the field of view of the wearer of the HMD 10. Hereinafter, this image is referred to as a field of view image.

[0049] The HMD 10 is worn on the head of, for example, an operator of the UAV 12 or a supervisor who supervises the operator.

[0050] The unmanned aerial vehicle 12 is an unmanned aerial vehicle with no passengers on board, for example, an unmanned aerial vehicle (so-called drone) driven by a battery or an engine.

[0051] In the unmanned aerial vehicle 12 of this embodiment, for example Figure 2B As shown, it includes a processor 30, a storage unit 32, a communication unit 34 and a sensor unit 36. In addition, the unmanned aerial vehicle 12 also includes common hardware such as propellers, motors, and batteries, but they are omitted here.

[0052] The processor 30 is, for example, a program-controlled device such as a microprocessor that operates according to a program installed in the unmanned aerial vehicle 12 .

[0053] The storage unit 32 is, for example, a storage element such as a ROM, a RAM, or a flash memory, etc. The storage unit 32 stores programs and the like executed by the processor 30 .

[0054] The communication unit 34 is a communication interface such as a wireless LAN module or a Bluetooth module.

[0055] The sensor unit 36 ​​includes, for example, a GNSS receiver such as GPS, an air pressure sensor, a LIDAR (Light Detection and Ranging), an altitude sensor such as an acoustic sensor, a camera, an inertial sensor, a wind speed sensor, etc. The sensor unit 36 ​​can measure the position or posture of the UAV 12. The sensor unit 36 ​​can also output the measurement results such as the position or posture of the UAV 12 to the processor 30 at a specified sampling rate.

[0056] The operation management system 14 of the present embodiment is a computer system that performs operation management such as flight path management of the unmanned aerial vehicle 12 and includes one or more server computers.

[0057] The operation terminal 16 of the present embodiment is, for example, a device for operating the unmanned aerial vehicle 12 .

[0058] The unmanned aerial vehicle 12 of this embodiment may also have an autonomous flight function. In this case, when the operator or supervisor of the unmanned aerial vehicle 12 determines that the unmanned aerial vehicle 12 cannot fly according to the predetermined flight path during autonomous flight, the operator may also use the operation terminal 16 to operate the unmanned aerial vehicle 12. In this case, the HMD 10 may also be worn on the head of the supervisor who supervises the unmanned aerial vehicle 12 that is flying autonomously. Alternatively, the HMD 10 may also be worn on the head of the operator who operates the unmanned aerial vehicle 12 when it is determined that the unmanned aerial vehicle 12 cannot fly according to the predetermined flight path during autonomous flight.

[0059] like Figure 1 As shown, the operation terminal 16 and the UAV 12 can communicate with each other. In addition, the HMD 10 and the operation terminal 16 can communicate with each other. In addition, the HMD 10 and the operation management system 14 can communicate with each other via a network 18 such as the Internet. In addition, the HMD 10 and the UAV 12 can also communicate with each other directly without going through the operation terminal 16.

[0060] In the present embodiment, for example, flight path information indicating the flight path of the unmanned aerial vehicle 12 is pre-stored in the operation management system 14. The flight path information may be, for example, a list of position information indicating a plurality of positions along a predetermined flight path of the unmanned aerial vehicle 12. Here, the position information may also include three-dimensional coordinate values ​​indicating the position indicated by the position information.

[0061] The coordinate values ​​(e.g., X-coordinate values ​​and Y-coordinate values) representing the horizontal position in the three-dimensional coordinate values ​​may be absolute coordinate values ​​(e.g., latitude and longitude) in the geocentric coordinate system used in GPS and the like. In addition, the coordinate values ​​(e.g., X-coordinate values ​​and Y-coordinate values) representing the horizontal position in the three-dimensional coordinate values ​​may also be relative coordinate values ​​in a coordinate system based on the take-off point of the unmanned aerial vehicle 12. Similarly, the coordinate values ​​(e.g., Z-coordinate values) representing the height in the three-dimensional coordinate values ​​may also be the absolute coordinate values ​​or the relative coordinate values. In the following description, it is assumed that the coordinate values ​​representing the horizontal position are absolute coordinate values, and the coordinate values ​​representing the height are relative coordinate values. In addition, it is assumed that the wearer of the HMD 10, such as an operator or a supervisor, is near the take-off point of the unmanned aerial vehicle 12, and the height of the HMD 10 is the sum of the height of the take-off point and the height of the wearer. Here, the value representing the height may be a specified value or a value input by the wearer and the like.

[0062] In addition, the flight path information may also include time information indicating the time when the UAV 12 should be at each of the multiple locations. For example, the location information included in the flight path information may be associated with the time information indicating the time when the UAV 12 should be at the location indicated by the location information.

[0063] In this embodiment, for example, the flight path information is sent from the operation management system 14 to the HMD 10. The HMD 10 generates a Figure 3 The virtual space 40 is shown. Figure 3 The virtual space 40 shown is, for example, a virtual three-dimensional space in which a flight path object 42, which is a virtual three-dimensional object representing a flight path shown in the flight path information, is arranged. Here, positions in the virtual space 40 correspond to positions in the real space on a one-to-one basis. In addition, directions in the virtual space 40 correspond to directions in the real space on a one-to-one basis.

[0064] The flight path object 42 may be, for example, a virtual object representing a line, the line being obtained by connecting the three-dimensional coordinate values ​​representing the positions indicated by the position information in the order of the list of the position information included in the flight path information. Here, for example, the flight path object 42 may be a virtual object representing a line, the line being obtained by connecting the three-dimensional coordinate values ​​representing the positions indicated by the position information included in the flight path information in the order from early to late times indicated by the time information for establishing the association. In addition, for example, Figure 3 As shown, the flight path object 42 can also be a columnar or tubular virtual object with a line as the axis, and the line is obtained by connecting the three-dimensional coordinate values ​​representing the position shown in the position information in the list order of the position information contained in the flight path information.

[0065] Furthermore, in this embodiment, for example, the visual position and visual direction in the actual space are specified. Here, for example, the visual position and visual direction in the actual space can be specified based on the inference result obtained using a well-known inference method. Here, the visual position can be associated with the position of the wearer of HMD 10. In addition, the visual direction can be associated with the posture of the head of the wearer of HMD 10.

[0066] In addition, the visual position may correspond to the position of the display unit 26, for example. Specifically, for example, the visual position may be the center position of the display unit 26. In addition, the visual direction may correspond to the direction of the display unit 26, for example. Specifically, for example, the visual direction may be the normal direction relative to the display surface of the display unit 26.

[0067] Furthermore, in the present embodiment, for example, a visual position 44 in the virtual space 40 corresponding to the visual position in the real space is identified. In addition, a visual direction 46 in the virtual space 40 corresponding to the visual direction in the real space is identified. As described above, the position in the virtual space 40 is associated with the position in the real space in a one-to-one correspondence, and the direction in the virtual space 40 is associated with the direction in the real space in a one-to-one correspondence, so the visual position 44 and the visual direction 46 can be uniquely identified.

[0068] In addition, in the present embodiment, for example, an image is generated that shows the flight path when the visual direction in the real space is viewed from the visual position in the real space. Here, for example, an image that shows the visual direction 46 in the virtual space 40 is viewed from the visual position 44 in the virtual space 40 may be generated. Hereinafter, an image generated in this way is referred to as a flight path image. Figure 4 This is a diagram showing an example of a flight path image. Figure 4 The flight path image shown includes an image of a flight path object 42 .

[0069] Furthermore, in the present embodiment, the flight path image is displayed on the display unit 26 of the HMD 10 in a state of being aligned with the state in which the viewing direction is viewed from the viewing position in the actual space. Here, the state of being aligned means a state in which the position of the flight path object 42 in the flight path image is adjusted so that the wearer of the HMD 10 sees the actual space as if the flight path object 42 is arranged at the position where it should be. In this way, Figure 5 As shown, the wearer of the HMD 10 can simultaneously see the unmanned aerial vehicle 12 in the real space and the flight path image displayed on the display unit 26 while the unmanned aerial vehicle 12 is flying. Figure 5 The UAV 12 shown is a real image, and the flight path image is a virtual image displayed on the display unit 26 .

[0070] Here, for example, the field of view that the wearer of the HMD 10 can see through the display unit 26 may be specified based on the measurement result of the sensor unit 28 and the design value of the display unit 26. Furthermore, the HMD 10 may determine whether the field of view includes the flight path based on the field of view of the wearer and the flight path information. Furthermore, when it is determined that the field of view includes the flight path, the HMD 10 may display the flight path image on the display unit 26.

[0071] Here, when the wearer of the HMD 10, such as the operator or supervisor of the UAV 12, sees Figure 5 In the situation shown, the wearer of the HMD 10 can understand that the UAV 12 is flying along the flight path.

[0072] On the other hand, when the wearer of the HMD 10, such as the operator or supervisor of the UAV 12, sees Figure 6 In the situation shown, the wearer of the HMD 10 can understand that the UAV 12 deviates from the flight path.

[0073] As described above, according to the present embodiment, the operator or supervisor of the unmanned aerial vehicle 12 can easily understand whether the unmanned aerial vehicle 12 can fly according to the flight path.

[0074] In addition, when the wearer of HMD 10 sees Figure 6 In the situation shown, the wearer of the HMD 10 can easily understand that the unmanned aerial vehicle 12 can be returned to the flight path by moving the unmanned aerial vehicle 12 toward the upper left as viewed from the wearer.

[0075] However, in this embodiment, the estimated result of the visual position or visual direction may contain an error. This error may be caused not only by the processing accuracy of estimating the visual position or visual direction, but also by the error of the position or direction of the HMD 10 measured by the sensor unit 28 of the HMD 10 or the error related to the height of the wearer. In particular, when the direction of the HMD 10 is specified by integrating the measurement values ​​of the acceleration sensor or the gyro sensor, the direction error of the HMD 10 is easy to accumulate. In this case, although the unmanned aerial vehicle 12 actually flies along the flight path shown by the flight path information, the wearer of the HMD 10 may see that the unmanned aerial vehicle 12 is flying away from the flight path.

[0076] Accordingly, in the present embodiment, it is also possible to determine whether the UAV 12 is on the flight path. Here, for example, it is also possible to determine whether the UAV 12 is on the flight path based on the position of the UAV 12 and the flight path information measured by the sensor unit 36 ​​of the UAV 12. And, it is also possible to control whether to display the image after the position reflected by the flight path is corrected based on the determination result.

[0077] For example, in an uncorrected state, assuming that the wearer of HMD 10 sees Figure 6 In this scenario, when it is determined that the unmanned aerial vehicle 12 is not on the flight path, no correction is required. On the other hand, when it is determined that the unmanned aerial vehicle 12 is on the flight path, the correction may be performed as shown in FIG. Figure 7 The image displayed on the display unit 26 is corrected as shown. Here, for example, the generated flight path image may be corrected by parallel translation or rotation. This correction may be performed, for example, based on the relationship between the camera of the HMD 10 and the display unit 26 that has been calibrated in advance, the visual range image, and the position where the unmanned aerial vehicle 12 should be at the current moment in the flight path image.

[0078] Here, for example, the flight path image may be corrected so that the position of the unmanned aerial vehicle 12 in the flight path image at the current moment coincides with the position of the unmanned aerial vehicle 12 in the visual range image.

[0079] In addition, for example, the position on the flight path shown in the flight path image that is closest to the position of the unmanned aerial vehicle 12 in the field of view image can be identified. Then, the flight path image can be corrected so that the position on the flight path in the flight path image identified in this way coincides with the position of the unmanned aerial vehicle 12 in the field of view image.

[0080] Alternatively, for example, a new flight path image may be generated after correcting the visual position 44 or the visual direction 46 .

[0081] When the operator or supervisor of the UAV 12 or the wearer of the HMD 10 sees Figure 7 In the situation shown, the wearer of the HMD 10 can understand that the UAV 12 is flying along the flight path.

[0082] Hereinafter, the functions of the HMD 10 according to the present embodiment and the processing executed by the HMD 10 according to the present embodiment will be further described.

[0083] Figure 8 1 is a functional block diagram showing an example of functions installed in the HMD 10 of this embodiment. Figure 8 All features shown, in addition, can also be equipped Figure 8 Functions other than those shown.

[0084] like Figure 8 As shown, in the HMD 10 of this embodiment, functionally, for example, it includes a flight path information acquisition unit 50, a virtual space generation unit 52, an actual space position and direction specifying unit 54, a virtual space position and direction specifying unit 56, a flight path image generating unit 58, a display control unit 60 and a determination unit 62.

[0085] The flight path information acquisition unit 50 is mainly equipped with the processor 20 and the communication unit 24. The virtual space generation unit 52, the virtual space position and direction specifying unit 56, the flight path image generation unit 58, and the determination unit 62 are mainly equipped with the processor 20. The real space position and direction specifying unit 54 is mainly equipped with the processor 20 and the sensor unit 28. The display control unit 60 is mainly equipped with the processor 20 and the display unit 26.

[0086] The above functions can also be implemented by executing a program including instructions corresponding to the above functions installed in the HMD 10 as a computer by the processor 20. In addition, the program can also be provided to the HMD 10 via a computer-readable information storage medium such as an optical disk, a magnetic disk, a magnetic tape, or a magneto-optical disk, or via the Internet.

[0087] In the present embodiment, the flight path information acquisition unit 50 acquires, for example, flight path information representing the flight path of the unmanned aerial vehicle 12 in the actual space. Here, the flight path information acquisition unit 50 may also acquire flight path information representing the planned flight path of the unmanned aerial vehicle 12. As described above, the flight path information may include, for example, a list of three-dimensional coordinate values ​​representing a plurality of positions along the planned flight path of the unmanned aerial vehicle 12. In addition, the flight path information may also include time information, which indicates, for each of the plurality of positions on the flight path, the time when the unmanned aerial vehicle 12 should be at the position.

[0088] In addition, the flight path information acquisition unit 50 may receive the flight path information transmitted from the operation management system 14. In addition, the flight path information acquisition unit 50 may acquire the flight path information stored in the storage unit 22 of the HMD 10 in advance.

[0089] In addition, the flight path information acquisition unit 50 may also acquire flight path information indicating the actual flight path of the unmanned aerial vehicle 12. Here, for example, the communication unit 34 of the unmanned aerial vehicle 12 may also repeatedly send the position information indicating the actual position or direction of the unmanned aerial vehicle 12 measured by the sensor unit 36 ​​of the unmanned aerial vehicle 12 to the HMD 10 at a specified time interval. Furthermore, the flight path information acquisition unit 50 may also receive a series of position information sent from the unmanned aerial vehicle 12 at a specified time interval as the flight path information indicating the actual flight path of the unmanned aerial vehicle 12. Here, the flight path information may be received by the HMD 10 via the operation terminal 16, or may be received by the HMD 10 directly from the unmanned aerial vehicle 12.

[0090] In this embodiment, the virtual space generation unit 52 generates, for example, a virtual space 40 in which virtual objects corresponding to the flight path are arranged. Figure 3 The virtual space 40 shown in the figure has a flight path object 42 arranged therein. The flight path object 42 represents the flight path indicated by the flight path information acquired by the flight path information acquisition unit 50 .

[0091] In this embodiment, the real space position and direction specifying unit 54 specifies, for example, a visual position and a visual direction in the real space. Here, the real space position and direction specifying unit 54 may also specify a visual position corresponding to the position of the display unit 26 and a visual direction corresponding to the direction of the display unit 26, wherein the display unit 26 allows the wearer of the HMD 10 to see the visual direction observed from the visual position in the real space and the flight path image.

[0092] In the present embodiment, the virtual space position and direction specifying unit 56 specifies, for example, a visual position 44 in the virtual space 40 corresponding to the visual position in the real space specified by the real space position and direction specifying unit 54. In addition, in the present embodiment, the virtual space position and direction specifying unit 56 specifies, for example, a visual direction 46 in the virtual space 40 corresponding to the visual direction in the real space specified by the real space position and direction specifying unit 54.

[0093] In this embodiment, the flight path image generation unit 58 generates, for example, a flight path image based on the flight path information. Figure 4 The flight path image shown in the figure shows the flight path when the visual direction in the real space is observed from the visual position in the real space. Here, the flight path image generation unit 58 can also generate the following flight path image, which shows the situation of observing the visual direction 46 in the virtual space 40 from the visual position 44 in the virtual space 40.

[0094] In addition, the flight path image generation unit 58 may also generate flight path images for multiple positions on the flight path, respectively, further showing the time when the unmanned aerial vehicle 12 should be at the position. For example, a flight path image may be generated whose visibility, such as color, brightness, and transparency, is different according to the time. For example, the following flight path image may be generated, that is, the earlier the time shown in the associated time information, the lower the visibility (for example, the lighter it looks), and the later the time, the higher the visibility (for example, the darker it looks). In this way, the wearer of the HMD 10 can easily grasp the flight path at a later time.

[0095] In addition, the flight path image generating unit 58 may also generate a flight path image corresponding to the current moment. For example, the flight path image generating unit 58 may also generate a flight path image that shows, for each of the multiple positions on the flight path, whether the unmanned aerial vehicle 12 should be at the position before or after the current moment. For example, a flight path image may be generated in which the visibility of the flight path before the current moment is lower than the visibility of the flight path after the current moment. Alternatively, a flight path image may be generated that does not show the flight path before the current moment. In addition, a flight path image may be generated in which a marking image is arranged at the position where the unmanned aerial vehicle 12 should be at the current moment. In this way, the wearer of the HMD 10 can understand the position where the unmanned aerial vehicle 12 should be at the current moment. In addition, it is easier for the wearer of the HMD 10 to understand the path that the unmanned aerial vehicle 12 should fly later.

[0096] In addition, the flight path image generating unit 58 may also generate a flight path image that represents at least one of the planned flight path of the unmanned aerial vehicle 12 shown in the flight path information and the actual flight path of the unmanned aerial vehicle 12 (the course of the actual flight path). Here, the flight path image generating unit 58 may, for example, also generate a flight path image that represents the actual flight path in addition to or instead of the planned flight path of the unmanned aerial vehicle 12. Furthermore, in the case of generating a flight path image that represents both the planned flight path and the actual flight path, a flight path image that can distinguish between the planned flight path and the actual flight path may also be generated. For example, a flight path image in which the visibility is different between the planned flight path and the actual flight path may also be generated.

[0097] In the present embodiment, the display control unit 60 displays the flight path image on the display unit 26 in a state where the position is aligned with the state in which the visual direction is viewed from the visual position in the real space. That is, the flight path image is displayed on the display unit 26 in a state where the position of the flight path object 42 in the flight path image is adjusted so that the wearer of the HMD 10 sees the real space where the flight path object 42 is arranged as it should be. As a result, the wearer of the HMD 10 can understand Figure 5 or Figure 6 Here, the display control unit 60 may also display the flight path image generated by the flight path image generation unit 58 on the display unit 26 .

[0098] In addition, in addition to displaying the flight path image, the display control unit 60 can also display information related to the unmanned aerial vehicle 12, such as the speed of the unmanned aerial vehicle 12, the position of the unmanned aerial vehicle 12 (latitude, longitude and altitude, etc.), the distance from the operator to the unmanned aerial vehicle 12, the distance to the destination, and the remaining battery level.

[0099] In this embodiment, the determination unit 62 determines, for example, whether the UAV 12 is on the flight path. Here, the determination unit 62 may also determine whether the UAV 12 is on the flight path based on the measured position and flight path information of the UAV 12. Here, the position of the UAV 12 may also be measured by the sensor unit 28 of the UAV 12.

[0100] Furthermore, in this case, the display control unit 60 may also control whether to display the image of the corrected position of the flight path according to the determination result of the determination unit 62. Here, for example, when it is determined that the unmanned aerial vehicle 12 is on the flight path, the flight path image generation unit 58 may also generate a corrected flight path image. Here, the flight path image generation unit 58 may also generate a new flight path image after correcting the visual position 44 or the visual direction 46. Alternatively, the display control unit 60 may also perform corrections such as parallel translation or rotation on the flight path image generated by the flight path image generation unit 58.

[0101] Below, refer to Fig. 9 The flowchart shown in FIG. 1 is used to explain an example of the process flow executed by the HMD 10 of this embodiment. In this processing example, it is assumed that the flight path information has been acquired by the flight path information acquisition unit 50. Furthermore, it is assumed that the virtual space generation unit 52 has generated Figure 3 The virtual space 40 is illustrated in FIG.

[0102] In the present embodiment, for example, the flight path image is displayed at a designated frame rate on the display unit 26. Then, the processes shown in S101 to S110 are repeatedly executed at the designated frame rate.

[0103] First, the real space position and direction specifying unit 54 specifies the visual position in the real space and the visual direction in the real space ( S101 ).

[0104] Then, the virtual space position and direction specifying unit 56 specifies the visual position 44 in the virtual space 40 and the visual direction 46 in the virtual space 40 (S102). Here, the visual position 44 in the virtual space 40 corresponding to the visual position in the real space specified in the process shown in S101 is specified. In addition, the visual direction 46 in the virtual space 40 corresponding to the visual direction in the real space specified in the process shown in S101 is specified.

[0105] Then, the flight-path image generation unit 58 generates a flight-path image showing a state where the viewing direction 46 in the virtual space 40 is viewed from the viewing position 44 in the virtual space 40 ( S103 ).

[0106] Then, the determination unit 62 acquires the visual field range image captured by the camera included in the HMD 10 ( S104 ).

[0107] Then, the determination unit 62 specifies the position of the unmanned aerial vehicle 12 within the visual range image acquired in the process shown in S104 by using a well-known image recognition technology ( S105 ).

[0108] Then, the determination unit 62 determines whether the wearer of the HMD 10 sees that the UAV 12 has deviated from the flight path based on the flight path image generated in the process shown in S103 and the position of the UAV 12 identified in the process shown in S105 (S106). Here, this determination can be performed based on the relationship between the camera and the display unit 26 of the HMD 10 that has been calibrated in advance, the position of the UAV 12 detected from the field of view image, and the flight path shown in the generated flight path image.

[0109] When it is determined that the wearer of the HMD 10 sees that the unmanned aerial vehicle 12 has deviated from the flight path ( S106 : Yes), the flight path information acquisition unit 50 acquires position information indicating the current position of the unmanned aerial vehicle 12 ( S107 ).

[0110] In the process shown in S107, the flight path information acquisition unit 50 may also request the unmanned aerial vehicle 12 to send position information indicating the current position. Then, the flight path information acquisition unit 50 may also receive the position information indicating the current position of the unmanned aerial vehicle 12 sent by the unmanned aerial vehicle 12 in response to the request. Here, the position information may be received by the HMD 10 via the operation terminal 16, or may be directly received by the HMD 10 from the unmanned aerial vehicle 12. Alternatively, the flight path information acquisition unit 50 may also acquire the position information indicating the current position from the position information repeatedly sent from the unmanned aerial vehicle 12 as the flight path information.

[0111] Then, the determination unit 62 determines whether the unmanned aerial vehicle 12 is on the flight path based on the position information and flight path information obtained in the process shown in S107 (S108). Here, for example, the distance between the position represented by the three-dimensional coordinate value indicated by the position information obtained in the process shown in S107 and the position where the unmanned aerial vehicle 12 should be at the current moment represented by the three-dimensional coordinate value in the flight path information can also be specified. And when the value indicating the specified distance is below the specified threshold, it can be determined that the unmanned aerial vehicle 12 is on the flight path.

[0112] In addition, the method for determining whether the unmanned aerial vehicle 12 is on the flight path is not limited to the above example. For example, the distance between the position represented by the three-dimensional coordinate value indicated by the position information obtained in the process shown in S107 and the line representing the flight path specified based on the flight path information can also be specified. And, when the value representing the specified distance is below a specified threshold, it can be determined that the unmanned aerial vehicle 12 is on the flight path.

[0113] In addition, different thresholds may be used in the horizontal direction (e.g., X-coordinate value and Y-coordinate value) and the height direction (e.g., Z-coordinate value). For example, a point on a two-dimensional plane represented by the XY coordinate value shown in the position information obtained in the process shown in S107 and a point on a two-dimensional plane represented by the XY coordinate value of the position where the unmanned aerial vehicle 12 should be at the current moment expressed in the three-dimensional coordinate value in the flight path information may be identified. Furthermore, the distance between the two points may be identified as the first distance. Furthermore, for example, a point on a one-dimensional straight line represented by the Z coordinate value shown in the position information obtained in the process shown in S107 and a point on a one-dimensional straight line represented by the Z coordinate value of the position where the unmanned aerial vehicle 12 should be at the current moment expressed in the three-dimensional coordinate value in the flight path information may be identified. Furthermore, the distance between the two points may be identified as the second distance. Furthermore, when the value representing the first distance is below the first threshold value and the value representing the second distance is below the second threshold value, it can be determined that the unmanned aerial vehicle 12 is on the flight path.

[0114] When it is determined that the UAV 12 is on the flight path (S108: Yes), the display control unit 60 displays the following Figure 7 The flight path image generated in the process shown in S103 is corrected for the position of the flight path and is displayed (S109).

[0115] When it is determined that the UAV 12 is not on the flight path (S108: No), the display control unit 60 displays the flight path image generated in the process shown in S103 (S110). Similarly, when it is determined in the process shown in S106 that the wearer of the HMD 10 sees the UAV 12 on the flight path (S106: No), the display control unit 60 displays the flight path image generated in the process shown in S103 (S110). The process shown in S110 is an example of a process of displaying the flight path image on the display unit 26 in a state where the position is aligned with the state in which the visual direction is viewed from the visual position in the actual space.

[0116] Then, the process returns to S101.

[0117] In addition, in the above-described processing example, the processing shown in S104 to S106 may not be executed. In this case, the processing shown in S107 may be executed after the processing shown in S103.

[0118] If no correction is to be performed, the processes shown in S104 to S109 do not need to be performed. In this case, the process shown in S110 may be performed after the process shown in S103.

[0119] In this embodiment, for example, the flight path information may also show a plurality of selectable flight paths. In addition, the flight path information may also show a flight path that can be selected at the current time point, and a flight path that can be selected in the future but cannot be selected at the current time point (for example, a flight path that passes through a predetermined area that is currently unsafe but will become safe in the future, etc.). In addition, the flight path information may also show a plurality of flight paths branching from one flight path.

[0120] In the case where a plurality of flight paths are shown in the flight path information as described above, the flight path image generation unit 58 may generate a flight path image that can distinguish between a selected flight path and a flight path that is not selected. For example, a flight path image may be generated in which the visibility of the selected flight path and the flight path that is not selected is different. For example, a flight path image may be generated in which the visibility of the currently selected flight path is higher than that of the flight path that is not selected. In addition, a flight path image may be generated in which the visibility of the flight path that can be selected at the current time point is higher than that of the flight path that cannot be selected.

[0121] In addition, the flight path image generating unit 58 may also generate a flight path image of a form corresponding to whether the unmanned aerial vehicle 12 is on the flight path. For example, when it is determined that the unmanned aerial vehicle 12 is on the flight path, a flight path image including a mark indicating that the unmanned aerial vehicle 12 is flying along the flight path may also be generated. In addition, when it is determined that the unmanned aerial vehicle 12 is not on the flight path, a flight path image including a mark indicating that the unmanned aerial vehicle 12 is flying off the flight path may also be generated. With respect to the mark indicating that the unmanned aerial vehicle 12 is flying along the flight path and the mark indicating that the unmanned aerial vehicle 12 is flying off the flight path, both may be configurable, or one may be configurable. When it is possible to configure both the mark indicating that the unmanned aerial vehicle 12 is flying along the flight path and the mark indicating that the unmanned aerial vehicle 12 is flying off the flight path, it is desired to make the colors, etc. different so that these marks can be distinguished. In addition, the mark may also be configured at a position within the flight path image corresponding to the position where the unmanned aerial vehicle 12 should be at the current moment in the actual space.

[0122] Here, the flight path image generation unit 58 may generate an image including the image of the flight path object 42 and the image of the mark as an image showing the flight path when the viewing direction is viewed from the viewing position. In addition, the flight path image generation unit 58 may generate an image including only the mark without including the image of the flight path object 42 as an image showing the flight path when the viewing direction is viewed from the viewing position.

[0123] The flight path information acquisition unit 50 may acquire flight path information indicating the flight paths of the plurality of unmanned aerial vehicles 12 . Furthermore, the flight path image generation unit 58 may generate a flight path image indicating the flight paths of the plurality of unmanned aerial vehicles 12 .

[0124] Here, for example, the flight path image generation unit 58 may generate flight path images with different visibility of the flight paths of the plurality of unmanned aerial vehicles 12. For example, the flight path image generation unit 58 may generate a flight path image that can distinguish the flight path of the unmanned aerial vehicle 12 being operated by the wearer of the HMD 10, that is, the operator viewing the flight path image, from the flight paths of other unmanned aerial vehicles 12.

[0125] For example, a flight path image may be generated in which the visibility of the flight path of the unmanned aerial vehicle 12 currently operated by the wearer, i.e., the operator of the HMD 10 is higher than the visibility of the flight paths of other unmanned aerial vehicles 12. Alternatively, a flight path image may be generated in which the visibility of the flight path of the unmanned aerial vehicle 12 currently operated by the wearer, i.e., the operator of the HMD 10 is lower than the visibility of the flight paths of other unmanned aerial vehicles 12. In this way, the wearer of the HMD 10 can anticipate the possibility of contact between the unmanned aerial vehicle 12 operated by the wearer, i.e., the operator of the HMD 10 and other unmanned aerial vehicles 12. Therefore, the possibility of contact between the unmanned aerial vehicle 12 operated by the wearer, i.e., the operator of the HMD 10 and other unmanned aerial vehicles 12 can be reduced.

[0126] Furthermore, when the UAV 12 takes off, the position of the UAV 12 in the field of view image captured by the camera of the HMD 10 may be detected, and the relationship between the position of the UAV 12 and the position of the HMD 10 may be estimated. In this way, the position of the wearer of the HMD 10 in the coordinate system based on the take-off point of the UAV 12 can be estimated with higher accuracy.

[0127] In addition, the flight path image generation unit 58 may generate a flight path image corresponding to whether the unmanned aerial vehicle 12 is closer than the flight path or farther inside when viewed from the recognition position.

[0128] For example, the camera provided by the HMD 10 is a camera provided with a distance measuring unit capable of measuring the distance to the unmanned aerial vehicle 12. In this case, it is also possible to determine whether the unmanned aerial vehicle 12 is closer to or inside the flight path when observed from the visual position based on the distance to the unmanned aerial vehicle 12 and the flight path information measured by the distance measuring unit. Furthermore, when the unmanned aerial vehicle 12 is closer to the flight path when observed from the visual position, a flight path image that represents the flight path in red can be generated. In addition, when the unmanned aerial vehicle 12 is inside the flight path when observed from the visual position, a flight path image that represents the flight path in yellow can be generated. In addition, when the value representing the difference between the distance to the unmanned aerial vehicle 12 and the distance to the flight path when observed from the visual position is less than a specified value, a flight path image that represents the flight path in blue can be generated.

[0129] In addition, for example, in the case where the unmanned aerial vehicle 12 is equipped with an omnidirectional camera, the position of an object such as an obstacle can be determined by performing image recognition on the image captured by the camera. In addition, a flight path image corresponding to whether an object such as an obstacle or a person intersects with the flight path can be generated. For example, when an object such as an obstacle or a person intersects with the flight path, a red flight path image can be generated, and when this is not the case, a blue flight path image can be generated. In addition, for example, a flight path image in which the portion intersecting with an object such as an obstacle is red and the other portion is blue can be generated.

[0130] In addition, the present invention is not limited to the above-described embodiments.

[0131] For example, in the above description, HMD10 is set as an optically transparent (optical see-through) HMD10, but the present invention can also be applied when HMD10 is an image-transmitting (video see-through) HMD10. In this case, the actual space position and direction specifying unit 54 can also specify a visual position corresponding to the position of a camera provided in the image-transmitting HMD10, and a visual direction corresponding to the shooting direction of the camera. The camera is, for example, a camera that shoots the front of the HMD10. In addition, the flight path image generating unit 58 can also generate a flight path image in which the flight path when observing the visual direction from the visual position is superimposed on the image captured by the camera. For example, a flight path image generating unit 58 can also generate a flight path image in which the flight path when observing the visual direction from the visual position is superimposed on the image captured by the camera. Figure 4The flight path image shown in the figure. The flight path image generated in this way can be said to be an example of an image that expresses the situation in the actual space and the flight path in a state where the positions are aligned with the situation in the actual space when the viewing direction is observed from the viewing position. In addition, the display control unit 60 can also display the flight path image on the display unit 26 of the image-transmitting HMD 10. In this way, the flight path image is displayed on the display unit 26 in a state where the position is aligned with the situation in the actual space when the viewing direction is observed from the viewing position.

[0132] In addition, the scope of application of the present invention is not limited to the HMD 10. For example, the present invention can also be applied to a situation where an operator or supervisor of the unmanned aerial vehicle 12 looks at a display other than the HMD 10, where the display other than the HMD 10 displays images captured by a camera configured in the actual space and the images captured by the camera are displayed. Figure 4 The image is obtained by overlapping the flight path image shown. In addition, the camera and the display may also be separated. In this case, for example, the actual space position and direction specifying unit 54 may also specify a visual position corresponding to the position of the camera, and a visual direction corresponding to the shooting direction of the camera. Furthermore, the flight path image generating unit 58 may also generate an image in which the flight path when the visual direction is observed from the visual position is superimposed on the image captured by the camera. Furthermore, the display control unit 60 may also display the flight path image on a display other than the HMD 10.

[0133] In addition, in this embodiment, the unmanned aerial vehicle 12, the operation management system 14 or the operation terminal 16 may also be equipped with Figure 8 Some or all of the features shown.

[0134] In addition, the specific character strings or numerical values ​​described above and the specific character strings or numerical values ​​in the drawings are examples, and the present invention is not limited to these character strings or numerical values.

Claims

1. A flight path guidance system, characterized in that Include: A flight path information acquisition unit that acquires flight path information representing a flight path of the unmanned aerial vehicle in real space; A specific mechanism for specifying a visual position and a visual direction in the actual space; An image generating means for generating, based on the flight path information, an image representing the flight path when the visual direction is viewed from the visual position; A display control means for causing the image to be displayed on a display unit in a state where the image is aligned with the state in which the image is viewed from the viewing position in the real space in the viewing direction; and A determination mechanism, for determining whether the unmanned aerial vehicle is on the flight path; and The display control mechanism: when the determination mechanism determines that the unmanned aerial vehicle is not on the flight path, the image generated by the image generating mechanism is displayed; when the determination mechanism determines that the unmanned aerial vehicle is on the flight path, the image generated by the image generating mechanism is displayed as a parallel-moved image or a rotated image.

2. The flight path guidance system according to claim 1, characterized in that: The specific mechanism specifies the visual position corresponding to the position of the display unit, and the visual direction corresponding to the direction of the display unit. The display unit is an optically transparent head-mounted display, so that the wearer of the head-mounted display can see the situation and the image of observing the visual direction from the visual position in the actual space.

3. The flight path guidance system according to claim 1, characterized in that: The specifying means specifies the visual position corresponding to the position of a camera that captures the actual space, and the visual direction corresponding to the capturing direction of the camera, and The image generation unit generates the image in which the flight path when the viewing direction is viewed from the viewing position is superimposed on the image captured by the camera.

4. The flight path guidance system according to claim 3, characterized in that: The specifying means specifies the visual position corresponding to the position of the camera provided in the image-transmitting head mounted display and the visual direction corresponding to the shooting direction of the camera, and The display control unit causes the display unit included in the head mounted display to display the image.

5. The flight path guidance system according to claim 2, characterized in that: The head mounted display is worn on the head of the operator of the UAV.

6. The flight path guidance system according to claim 4, characterized in that: The head mounted display is worn on the head of the operator of the UAV.

7. The flight path guidance system according to claim 1, characterized in that: The method further comprises a virtual space generating unit for generating a virtual space in which virtual objects representing the flight path are arranged. The specifying means further specifies a visual position in the virtual space corresponding to the visual position in the real space, and a visual direction in the virtual space corresponding to the visual direction in the real space, and The image generation unit generates the image showing a state in which the viewing direction in the virtual space is viewed from the viewing position in the virtual space.

8. The flight path guidance system according to claim 1, characterized in that: The determination mechanism determines whether the UAV is on the flight path according to the measured position of the UAV and the flight path information.

9. The flight path guidance system according to claim 1, characterized in that: In the flight path information, the times when the UAV should be at each of the multiple locations on the flight path are respectively indicated, and The image generating mechanism generates the image also representing the moment.

10. The flight path guidance system according to claim 9, characterized in that: The image generating unit generates the image for each of the plurality of positions on the flight path, showing whether the time when the UAV should be at the position is before or after the current time.

11. The flight path guidance system according to claim 1, characterized in that: A plurality of selectable flight paths are shown in the flight path information, and The image generating means generates the image capable of distinguishing the selected flight path from the unselected flight paths.

12. The flight path guidance system according to claim 1, characterized in that: The image generation unit generates the image representing at least one of a planned flight path of the unmanned aerial vehicle indicated by the flight path information and an actual flight path of the unmanned aerial vehicle.

13. The flight path guidance system according to claim 1, characterized in that: The flight path information acquisition unit acquires the flight path information indicating the flight paths of the plurality of unmanned aerial vehicles, and The image generating mechanism generates the image representing the flight paths of the plurality of the UAVs.

14. The flight path guidance system according to claim 13, characterized in that: The image generation mechanism generates the image capable of distinguishing the flight path of the unmanned aerial vehicle being operated by an operator viewing the image from the flight paths of other unmanned aerial vehicles.

15. The flight path guidance system according to claim 1, characterized in that: The unmanned aerial vehicle has an autonomous flight function.

16. A flight path guidance device, characterized in that Include: A flight path information acquisition unit that acquires flight path information representing a flight path of the unmanned aerial vehicle in real space; A specific mechanism for specifying a visual position and a visual direction in the actual space; An image generating means for generating, based on the flight path information, an image representing the flight path when the visual direction is viewed from the visual position; A display control means for causing the image to be displayed on a display unit in a state where the image is aligned with the state in which the image is viewed from the viewing position in the real space in the viewing direction; and A determination mechanism, for determining whether the unmanned aerial vehicle is on the flight path; and The display control mechanism: when the determination mechanism determines that the unmanned aerial vehicle is not on the flight path, the image generated by the image generating mechanism is displayed; when the determination mechanism determines that the unmanned aerial vehicle is on the flight path, the image generated by the image generating mechanism is displayed as a parallel-moved image or a rotated image.

17. A flight path guidance method, characterized in that The following steps are included: a flight path information acquisition step of acquiring flight path information representing a flight path of the unmanned aerial vehicle in real space; A specifying step of specifying a visual position and a visual direction in the actual space; an image generating step of generating an image representing the flight path when the visual direction is viewed from the visual position based on the flight path information; A display control step of displaying the image on a display unit in a state where the image is aligned with the state in which the image is viewed from the visual position in the real space in the visual direction; and A determination step of determining whether the UAV is on the flight path; and In the display control step, when the determination step determines that the unmanned aerial vehicle is not on the flight path, the image generated in the image generation step is displayed; when the determination step determines that the unmanned aerial vehicle is on the flight path, the image generated in the image generation step is displayed after being parallel-moved or rotated.