Information Processing Method, Information Processing Apparatus, and Program

Abstract

To provide an information processing method, an information processing device, and a program that enable intuitive generation of information for moving a moving object.SOLUTION: An information processing device includes: a display control unit that controls display of a virtual object based on an object existing in a real space, on a display screen; and a movement information generation unit that generates movement information for controlling movement of a moving object. This enables intuitive generation of information for moving the moving object.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, mobile bodies such as drones that are controlled using a control device have been used. For example, a drone equipped with a camera is made to capture a landscape from above, and the captured image is utilized.

[0003] For example, Patent Document 1 describes a technique for performing efficient image transfer by switching the mode from an image capture mode to an image transfer mode when a predefined mode switching condition occurs.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] As a method for controlling the movement of a mobile body such as a drone, in addition to the method in which a user controls the movement of the mobile body using a control device, a method of presetting a path along which the mobile body moves in advance and moving the mobile body along the set path can be considered. In this case, for example, it is conceivable to set the movement path of the mobile body on a map.

[0006] However, in the method of setting the path of the mobile body on a two-dimensional map, when the mobile body moves three-dimensionally, it has been difficult to intuitively set the movement of the mobile body. The technique described in Patent Document 1 is not a technique intended to intuitively generate movement information for controlling the movement of a mobile body such as a drone.

[0007] Therefore, the present disclosure proposes a novel and improved information processing apparatus, information processing method, and program capable of intuitively generating information for the movement of a moving body.

Means for Solving the Problems

[0008] According to the present disclosure, there is provided an information processing apparatus including a display control unit that controls the display of a virtual object on a display screen based on an object existing in the real space, and a movement information generation unit that generates movement information for controlling the movement of a moving body.

[0009] Further, according to the present disclosure, there is provided an information processing method in which a processor controls the display of a virtual object based on an object existing in the real space on a display screen and generates movement information for controlling the movement of a moving body.

[0010] Further, according to the present disclosure, there is provided a program for causing a computer to realize a function of controlling the display of a virtual object based on an object existing in the real space on a display screen and a function of generating movement information for controlling the movement of a moving body.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Mode for Carrying Out the Invention

[0012] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, there are cases where a plurality of components having substantially the same functional configuration are distinguished by attaching different alphabets after the same reference numeral. For example, a plurality of configurations having substantially the same functional configuration are distinguished as the user terminal 10a and the user terminal 10b as necessary. However, when it is not necessary to particularly distinguish each of a plurality of components having substantially the same functional configuration, only the same reference numeral is attached. For example, when it is not necessary to particularly distinguish the user terminal 10a and the user terminal 10b, it is simply referred to as the user terminal 10.

[0013] Note that the description will be made in the following order. 1. Configuration 1.1 Configuration of the information processing system 1.2 Configuration of the user terminal 2. Generation of virtual objects 3. Operation examples 3.1 Generation of movement information 3.2 Generation of imaging information 3.3 Simulation of the movement of the moving body 4. Imaging method 4.1 Imaging method by manual operation 4.2 Imaging method by automatic flight 4.3 Imaging method using the map displayed on the display screen of the terminal 4.4 Imaging method according to the present disclosure 5. Effects 6. Hardware configuration 7. Supplementary notes

[0014] <1. Configuration> <<1.1 Configuration of the information processing system>> First, with reference to FIG. 1, the configuration of the information processing system 1 according to an embodiment of the present disclosure will be described. FIG. 1 is a diagram showing the configuration of the information processing system 1 according to an embodiment of the present disclosure. The information processing system 1 includes a user terminal 10 and a moving body 20. The user terminal 10 and the moving body 20 are communicably connected to each other.

[0015] The user terminal 10 may be, for example, a smartphone or a tablet terminal. The user terminal 10 generates movement information for controlling the movement of the moving body 20 in response to a user's operation, and transmits the movement information to the moving body 20. In addition, the user terminal 10 can also display virtual objects and the like, which will be described later, in response to a user's operation.

[0016] The moving body 20 is a device that moves based on the movement information generated by the user terminal 10. Here, the moving body 20 can be various devices that can move, but in the following, it will be described assuming that the moving body 20 is a drone. In addition, the moving body 20 may be equipped with an imaging device for imaging the scenery.

[0017] <<1.2. Configuration of User Terminal>> With reference to FIG. 2, the configuration of the user terminal 10 according to an embodiment of the present disclosure will be described. FIG. 2 is a functional block diagram showing the configuration of the user terminal 10 according to an embodiment of the present disclosure.

[0018] The user terminal 10 has a function of acquiring information such as image information, sensor information, or information based on an operation from a user, and outputting a result obtained by performing various processes on the acquired information. The functions of the user terminal 10 are realized by the cooperation of an information processing device 100, an imaging unit (first imaging device) 110, a sensor unit 120, an input unit 130, and a display unit 175 included in the user terminal 10.

[0019] The imaging unit 110 may be various known imaging devices for imaging an image. The imaging unit 110 has various known imaging elements such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) sensor. In addition to these imaging elements, the imaging unit 110 may have various members such as a lens for forming a subject image on the imaging element, and a light source for irradiating the subject with illumination light. The imaging unit 110 transmits the image information obtained by imaging to the information processing device 100.

[0020] The sensor unit 120 includes at least one of various known sensors such as a distance measurement sensor or an IMU (Inertial Measurement Unit), for example. The distance measurement sensor may be, for example, a stereo camera or a ToF (Time of Flight) sensor. The distance measurement sensor detects distance information regarding, for example, the distance between the user terminal 10 and an object existing around it, and transmits the detected distance information to the information processing device 100. Further, the IMU includes at least any one of, for example, an acceleration sensor, a gyro sensor, or a magnetic sensor. The IMU transmits the detected information as IMU information to the information processing device 100.

[0021] The input unit 130 has a function of generating input information based on an operation by the user. The input unit 130 can be, for example, a touch panel or the like. The input unit 130 generates input information based on various operations by the user, such as a touch operation, a drag operation, a pinch-out operation, or a pinch-in. The input unit 130 transmits the generated input information to the acquisition unit 140.

[0022] The information processing apparatus 100 has a function of performing various processes based on the acquired information and controlling the display on the display unit 175 based on the result of the process. The functions of the information processing apparatus 100 are realized by the cooperation of the acquisition unit 140, the processing unit 150, the display control unit 170, the storage unit 180, and the communication control unit 190.

[0023] The acquisition unit 140 acquires information input from at least any one of the imaging unit 110, the sensor unit 120, or the input unit 130. The acquisition unit 140 transmits the acquired information to the processing unit 150.

[0024] The processing unit 150 has a function of performing various processes based on the information transmitted from the acquisition unit 140. For example, the processing unit 150 has a function of generating information for controlling the moving body 20 based on the information transmitted from the acquisition unit 140. Further, the processing unit 150 generates information regarding the content displayed on the display screen of the display unit 175. The detailed configuration and functions of the processing unit 150 will be described later with reference to FIG. 3. The processing unit 150 transmits the generated information to the display control unit 170, the storage unit 180, or the communication control unit 190.

[0025] The display control unit 170 has a function of controlling the display on the display screen of the display unit 175. The display control unit 170 controls, for example, the display of virtual objects based on the objects existing in the real space on the display screen of the display unit 175 based on the information transmitted from the processing unit 150.

[0026] The display unit 175 is a display device having a function of displaying various known images. In the present embodiment, the display unit 175 and the aforementioned input unit 130 are integrated and configured as a touch panel. As will be described later, the user can cause the information processing apparatus 100 to generate movement information for controlling the movement of the moving body 20 in the user terminal 10 by performing a predetermined operation while referring to the display screen of the display unit 175.

[0027] The storage unit 180 has a function of storing various types of information such as information generated or acquired by the information processing apparatus 100. For example, the storage unit 180 may store information regarding a virtual object generated in advance. Note that a method for generating a virtual object will be described later. Further, the storage unit 180 may store movement information for controlling the movement of the moving body 20. More specifically, the storage unit 180 may store information (Waypoint information) regarding a specific point (also referred to as a "Waypoint") included in the route of the moving body 20. The route of the moving body 20 may be formed by connecting a plurality of Waypoints. Further, the storage unit 180 may store movement information or imaging information generated by the processing unit 150. The information stored in the storage unit 180 is referred to by the processing unit 150, the display control unit 170, or the communication control unit 190 as necessary.

[0028] The communication control unit 190 has a function of controlling the transmission of various types of information generated by the processing unit 150. The communication control unit 190 controls the transmission of movement information or imaging information generated by the processing unit 150. The movement information or imaging information is transmitted to the moving body 20. The moving body 20 can move or perform imaging based on the transmitted information.

[0029] Next, with reference to FIG. 3, the processing unit 150 included in the information processing apparatus 100 will be described in more detail. FIG. 3 is a functional block diagram showing the configuration of the processing unit 150. As shown in FIG. 3, the processing unit 150 includes a detection unit 151, a self-position calculation unit 154, a virtual object calculation unit 155, a generation unit 156, and a prediction unit 160.

[0030] The detection unit 151 has a function of performing various detections based on the information transmitted from the acquisition unit 140. The functions of the detection unit 151 are realized by the plane detection unit 152 and the object detection unit 153. The plane detection unit 152 has a function of detecting a plane included in an image based on the image information, the distance information, etc. The object detection unit 153 has a function of detecting a predetermined object included in the image based on the image information, the distance information, etc. The detection unit 151 transmits the detected result to the self-position calculation unit 154.

[0031] The self-position calculation unit 154 has a function of calculating the self-position of the user terminal 10. Here, the self-position of the user terminal 10 includes not only the position where the user terminal 10 exists but also the posture of the user terminal 10. Specifically, the self-position calculation unit 154 takes the image information, the distance information, and the IMU information as inputs, and calculates the position or posture of the user terminal 10 with respect to the environment or objects around the user terminal 10 by the SLAM (Simultaneous Localization And Mapping) technology. At this time, the self-position calculation unit 154 may determine the origin or scale in SLAM based on the object information, the plane information, etc. The self-position calculation unit 154 transmits the calculated result to the virtual object calculation unit 155 and the generation unit 156.

[0032] The virtual object calculation unit 155 generates information regarding virtual objects arranged on the display screen. More specifically, the virtual object calculation unit 155 calculates arrangement information (information regarding position or orientation, etc.) or scale information, etc. of the virtual objects to be arranged on the display screen of the display unit 175 based on the self-position of the user terminal 10 calculated by the self-position calculation unit 154, the result detected by the detection unit 151, the input information input to the input unit 130, information regarding virtual objects stored in the storage unit 180, and the like. Here, the virtual object calculation unit 155 calculates the scale of the virtual object based on the scale of the real space. More specifically, the virtual object calculation unit 155 appropriately expands or contracts the scale of the virtual object displayed on the display screen from the scale of the real object that is the basis of the virtual object, and generates scale information. The virtual object calculation unit 155 transmits the calculated result to the movement information generation unit 157 described later.

[0033] The generation unit 156 has a function of generating various types of information for controlling the moving body 20. More specifically, the generation unit 156 generates information for controlling the movement of the moving body 20, the operation of the imaging device (second imaging device) provided in the moving body 20, the display of the display unit 175, and the like. The functions of the generation unit 156 are realized by the movement information generation unit 157, the imaging information generation unit 158, and the display information generation unit 159 provided in the generation unit 156.

[0034] The movement information generation unit 157 generates movement information related to virtual objects for controlling the movement of the mobile body 20. Specifically, the movement information generation unit 157 generates the position and direction of the Waypoint as movement information based on the self-position of the user terminal 10, the input information to the input unit 130, the arrangement information of the virtual object, the scale information, and the Waypoint information. For example, the movement information generation unit 157 may generate the route of the mobile body 20 as movement information. At this time, the movement information generation unit 157 may generate movement information with a scale corresponding to the scale information of the virtual object, or may generate movement information by adjusting the movement information to the scale of the real space. In addition, the movement information generation unit 157 can also correct the movement information and generate new movement information based on operations such as input from the user to the input unit 130. Specific operations by the user will be described later. The movement information generation unit 157 transmits the generated movement information to the display information generation unit 159, the prediction unit 160, and the storage unit 180.

[0035] The imaging information generation unit 158 has a function of generating imaging information for controlling the imaging range of the imaging device provided in the mobile body 20 based on the operation of the user. More specifically, the imaging information generation unit 158 generates imaging information regarding the direction or the angle of view that the imaging device faces based on the input information generated by the input unit 130 and the like. The imaging information generation unit 158 transmits the generated imaging information to the prediction unit 160 and the storage unit 180.

[0036] The display information generation unit 159 has a function of generating display information regarding the content displayed on the display unit 175. More specifically, the display information generation unit 159 generates CG (Computer Graphics) for display on the display unit 175 as display information based on the position of the imaging unit 110, the arrangement information of virtual objects, scale information, Waypoint information, and the prediction result by the prediction unit 160 described later. The display information generation unit 159 can generate a video indicating the movement of the moving body 20 as display information. Also, the display information generation unit 159 can generate display information for displaying a simulation video captured by the imaging device provided in the moving body 20. The display information generation unit 159 transmits the generated display information to the display control unit 170.

[0037] The prediction unit 160 has a function of predicting the operation of the moving body 20. More specifically, the prediction unit 160 can predict the movement of the moving body 20 and the operation of the imaging device provided in the moving body 20. The functions of the prediction unit 160 are realized by the movement prediction unit 161 and the imaging prediction unit 162. The prediction unit 160 transmits the predicted result to the display information generation unit 159.

[0038] The movement prediction unit 161 has a function of predicting the movement of the moving body 20 based on movement information. For example, the movement prediction unit 161 can predict the route of the moving body 20. Also, the imaging prediction unit 162 predicts the imaging image captured by the imaging device provided in the moving body 20 based on the movement information and the imaging information. More specifically, the imaging prediction unit 162 predicts the image captured by the imaging device based on the route through which the imaging device passes and the posture of the imaging device.

[0039] <2. Generation of Virtual Object> In the present embodiment, a virtual object is displayed on the display unit 175, and the user can cause the information processing apparatus 100 to generate movement information, imaging information, or the like by performing a predetermined operation while viewing the display. Here, an example of a method for generating the virtual object will be described. Note that the method for generating the virtual object is not limited to the method described below, and the virtual object may be generated by any method.

[0040] A method for generating a virtual object will be described with reference to FIGS. 4 to 7. FIG. 4 is a diagram showing a state where user U1 operates the moving body 20 to image the tower 420 and the forest 430. Further, FIG. 5 is a diagram showing the virtual object 422 generated based on the imaged tower 420. Further, FIG. 6 is a diagram showing the virtual object 432 generated based on the imaged forest 430. Furthermore, FIG. 7 is a diagram showing the path 402 along which the moving body 20 has moved.

[0041] First, the method for generating a virtual object according to the present embodiment will be briefly described. Here, it is assumed that user U1 wants to image a video including the tower 420 with the imaging device 206. In the present embodiment, first, user U1 uses the control device 401 to operate the moving body 20 to cause the imaging device 206 to image the tower 420 existing in the real space and the forest 430 existing around the tower 420 in advance. Next, based on the captured image, a three-dimensional virtual object is generated based on various CG technologies. In the present embodiment, further, a Waypoint is set on the path through which the moving body 20 has passed, and movement information is generated. Hereinafter, with reference to FIGS. 4 to 7, the method for generating a virtual object will be described in more detail.

[0042] First, as shown in FIG. 4, user U1 uses the control device 401 to fly the moving body (drone) 20. The moving body 20 includes a fuselage 202, propellers 204, and an imaging device 206. By driving the propellers 204, the moving body 20 can fly. Further, when user U1 operates the propellers 204 and the like, the direction, attitude, speed, etc. of the fuselage 202 are controlled. Furthermore, the imaging device 206 provided in the moving body 20 images the scenery around the moving body 20. Here, an image including the tower 420 shown in FIG. 4 and the forest 430 around it is imaged by the imaging device 206.

[0043] Based on the captured image, virtual objects of the tower 420 and the forest 430 are generated using various known CG technologies. More specifically, a virtual object 422 of the tower 420 existing in the real space shown in FIG. 5 and a virtual object 432 of the forest 430 shown in the real space shown in FIG. 6 are generated. Information regarding the virtual objects generated in this way is stored in the storage unit 180 provided in the information processing apparatus 100.

[0044] Furthermore, the user U1 may cause the moving body 20 to realize the flight when the actually desired image is captured. More specifically, the user U1 operates the moving body 20 so that the moving body 20 passes through a path 402 that turns around the tower 420 shown in FIG. 4. The moving body 20 calculates the path 402 traveled by the sensors provided in the moving body 20 and records the calculated result in a recording medium or the like provided in the moving body 20. Here, a Waypoint may be set on the path 402 according to a predetermined rule. For example, a Waypoint may be set at every predetermined distance. Also, the density of the Waypoint may be adjusted according to the curvature of the path 402.

[0045] FIG. 7 shows an example of a path 402 with a Waypoint 406 set. Note that FIG. 7 shows 13 Waypoints 406a to 406m, but the present invention is not limited to this. Two or more and 12 or less Waypoints may be set on the path 402, or 14 or more Waypoints may be set. In the figures used in the following description, Waypoints may be shown, but the number of Waypoints is not limited to the numbers shown in those figures.

[0046] Information regarding the path 402 with the Waypoint set in this way may be stored as movement information in the storage unit 180 provided in the information processing apparatus 100.

[0047] <3. Operation Example> Here, based on the virtual object generated as described above, a specific example of an operation for causing the user to generate movement information, imaging information, or the like in the information processing apparatus 100 will be described.

[0048] <<3.1. Generation of Movement Information>> A case where movement information is generated based on a user operation will be described. First, with reference to FIGS. 8 to 12, a case where, when movement information is previously recorded in the information processing apparatus 100, the movement information is corrected and new movement information is generated will be described. FIG. 8 is a diagram showing a state in which a plane on a desk 500 existing in the real space is detected by the user terminal 10a. FIGS. 9 and 10 are diagrams showing a state in which Waypoint 408a is selected based on the operation of the user U2. FIGS. 11 and 12 are diagrams showing a state in which the position of Waypoint 408a is adjusted based on the operation of the user U2.

[0049] As shown in FIG. 8, it is assumed that a desk 500 exists in the real space in front of the eyes of the user U2. The user U2 holds the user terminal 10a, and a start button 602 for starting the display of the virtual object and the setting of the Waypoint is displayed on the display screen 610 of the user terminal 10a. Here, the display screen 610 also functions as an input unit 130 that receives a touch operation, a pinch operation, or the like from the user U2. When the user U2 touches the start button 602, the user terminal 10a detects the plane 506 on the desk 500.

[0050] Then, as shown in FIG. 9, on the display screen 610a of the user terminal 10a, an image 612a of the virtual object 422a of the tower generated in advance and an image 614a of the virtual path 404a of the moving body 20 are displayed. Here, the virtual path is a path in the virtual space where the virtual object is arranged. By appropriately scaling the scale of the virtual path, an actual path along which the moving body 20 actually moves is formed. Hereinafter, when the virtual path and the actual path are not particularly distinguished, these are also simply referred to as "paths". At this time, the virtual object 422a is displayed on the display screen 610a as if it were placed on the desk 500. Although the virtual object 422a is shown on the desk 500 in FIG. 9, the virtual object 422a is shown here for the purpose of explanation, and the virtual object 422a is not actually placed on the desk 500.

[0051] Here, the size of the image 614a of the virtual object 422a may be, for example, a size that can be placed on the desk 500 as shown in FIG. 9. Further, on the image 614a of the virtual path 404a, an image 616a of the Waypoint 408a for adjusting the virtual path 404a is displayed. In the present embodiment, the user U2 can also adjust the size of the image 612a of the virtual object displayed on the display screen 610a (that is, the distance from the user terminal 10a to the virtual object 422a) by performing a pinch operation or the like on the display screen 610a. At this time, the user terminal 10a may store the ratio between the size of the tower existing in the real space that is the basis of the virtual object 422a and the size of the virtual object 422a.

[0052] In the present embodiment, since the image 612a of the virtual object 422a is displayed on a plane (on the desk 500), the user U2 can feel as if the virtual object 422a is placed on the ground. Therefore, the user U2 can operate the user terminal 10a more intuitively.

[0053] Here, it is assumed that the moving body 20 has flown manually or the like in advance, and the virtual path 404a is generated based on the flight. Also, it is assumed that Waypoints are set in advance in the virtual path 404a. Specifically, as shown in FIG. 9, 13 Waypoints indicated by circles are set in the virtual path 404a. Furthermore, one of the 13 Waypoints, Waypoint 408a, corresponds to the image 616a of the Waypoint displayed on the display screen 610a.

[0054] The user U2 can select the Waypoint to be adjusted by touching the image 616a of the Waypoint displayed on the display screen 610a. Here, it is assumed that the user U2 has selected Waypoint 408a in the virtual path 404a. Also, when Waypoint 408a is selected, the user U2 can adjust the position of Waypoint 408a by performing a pinch operation or a drag operation or the like on the display screen 610a. Further, in the present embodiment, the user U2 can also adjust the orientation of the imaging device provided in the moving body 20 by operating the display screen 610a. For example, when Waypoint 408a is selected, by the user U2 performing a pinch operation or a drag operation or the like on the display screen 610a, the orientation of the imaging device of the moving body 20 when the moving body 20 passes through the position corresponding to the Waypoint 408a can be specified.

[0055] Next, with Waypoint408a selected, user U2 moves the position of user terminal 10a. For example, as shown in FIGS. 11 and 12, user U2 pulls user terminal 10a towards user U2. As a result, the position of the selected Waypoint408a moves in accordance with the movement of user terminal 10b. Also, virtual path 404a changes to virtual path 404b according to the position of Waypoint408b after the movement. Due to this change in virtual path 404, the path along which mobile body 20 actually moves is adjusted. Thus, in this embodiment, the path of mobile body 20 is adjusted based on the operation of moving user terminal 10a by user U2. In this way, the path of mobile body 20 is adjusted, and movement information representing the adjusted path is newly generated. Based on this movement information, mobile body 20 can fly around tower 420.

[0056] Here, the case where Waypoints are preset in virtual path 404a has been described. If Waypoints are not preset in virtual path 404a, user U2 can also set a Waypoint by touching a part of image 614 of virtual path 404 displayed on display screen 610. The Waypoint set in this way can also be adjusted by the method described above.

[0057] Thus, according to this embodiment, with Waypoints set, user U2 can finely adjust the Waypoints by operating the display screen 610 and moving user terminal 10a. Therefore, it becomes possible to more intuitively generate information for the movement of mobile body 20.

[0058] The above described a method of generating new movement information by adjusting the virtual path 404a of the moving body 20 when the virtual path 404a is preset. Next, two methods of setting the path of the moving body 20 when the path of the moving body 20 is not preset will be described with reference to FIGS. 13 and 14. FIGS. 13 and 14 are diagrams showing a state in which the path of the moving body 20 is newly set based on the operation of the user U2.

[0059] Note that both methods are methods of setting the path and Waypoint of the moving body 20 by the user U2 operating the user terminal 10. The Waypoint set by the method described with reference to FIGS. 13 and 14 may also be adjusted by the method described above with reference to FIGS. 8 to 12.

[0060] The first method of setting the path of the moving body 20 will be described with reference to FIG. 13. First, the user U2 touches a part of the display screen 610c of the user terminal 10c (for example, the designated portion 616c shown on the display screen 610c). While touching the designated portion 616c, the user U2 moves the user terminal 10c downward, for example, indicated by a dashed line. The user terminal 10c stores the trajectory of the movement of the user terminal 10c as the path of the moving body 20. At this time, the user terminal 10c may set a Waypoint on the path and store it together with the path.

[0061] Next, the second method of setting the path of the moving body 20 will be described with reference to FIG. 14. In the second method, as shown in the upper part of FIG. 14, the user U2 touches the display screen 610d of the user terminal 10d to set the Waypoint 408d. The image 616d of the set Waypoint 408d is displayed on the display screen 610d.

[0062] Next, as shown in the lower part of FIG. 14, the user U2 moves the position of the user terminal 10d and touches the display screen 610e of the user terminal 10e at the new position to set Waypoint 408e. Thereafter, the movement of the user terminal 10 and the setting of Waypoint 408 are repeated, and a path of the moving body 20 is generated by connecting the set plurality of Waypoints 408. The generated path and Waypoint 408 are stored in the user terminal 10.

[0063] In this way, according to the present embodiment, even when the path of the moving body 20 is not preset, the user U2 can set the path of the moving body 20 by operating the display screen 610 and moving the user terminal 10.

[0064] Here, the position of the Waypoint set by the operation of the user terminal 10 will be described in more detail with reference to FIGS. 15 to 18. FIG. 15 is a diagram showing a state where the position of the imaging unit 110a provided in the user terminal 10f is set as Waypoint 410. FIG. 16 is a diagram showing the display screen 610f when the position of the imaging unit 110a provided in the user terminal 10f is set as Waypoint 410. FIG. 17 is a diagram showing a state where a position separated from the user terminal 10g by a predetermined distance is set as Waypoint 412. Further, FIG. 18 is a diagram showing the display screen 610 when a position separated from the user terminal 10g by a predetermined distance is set as Waypoint 412.

[0065] First, with reference to FIGS. 15 and 16, the position of the set Waypoint will be described. As shown in FIG. 15, a virtual object 422a of a tower is arranged on the desk 500. The imaging unit 110a provided in the user terminal 10f is imaging the front of the imaging unit 110a. That is, the imaging unit 110a is imaging the range including the virtual object 422a. Here, the position of the imaging unit 110a is designated as a Waypoint.

[0066] At this time, as shown in FIG. 16, an image 612f of a virtual object of a tower placed on the desk 500 is displayed on the display screen 610f of the user terminal 10f. While looking at the display screen 610f, the user can set a Waypoint by touching the display screen 610f or the like. Also, the user can move the user terminal 10f and set a Waypoint 410 at a new position.

[0067] At this time, the image displayed on the display screen 610 may correspond to the image actually captured by the imaging device of the moving body 20 at the Waypoint. In this case, the user can check in advance the image captured by the imaging device provided in the moving body 20.

[0068] Next, with reference to FIGS. 17 and 18, a method for setting a position at a predetermined distance away from the imaging unit 110a by the user terminal 10g as a Waypoint will be described. Specifically, a position that is at a distance d in front of the imaging unit 110a and is slightly deviated downward from the optical axis 414 of the imaging unit 110a is set as the Waypoint 412.

[0069] At this time, as shown in FIG. 18, an image 612g of a virtual object of a tower and an image 616 of the Waypoint are displayed on the display screen 610g of the user terminal 10g. Further, a guide surface 617 connecting the user terminal 10g and the image 616 of the Waypoint is displayed on the display screen 610g. While looking at the image 616 of the Waypoint arranged on the guide surface 617, the user can set the Waypoint 412 by a touch operation or the like on the display screen 610g.

[0070] At this time, since the Waypoint 412 is located below the optical axis 414 of the imaging unit 110a, it is considered that the user can more easily recognize the position of the Waypoint 412 with reference to the display screen 610g.

[0071] The method of setting a Waypoint by operating the display screen 610 has been described above. Next, variations of the method of setting a Waypoint will be described with reference to FIGS. 19 and 20. Specifically, the method of setting a Waypoint using a designated object that designates the path of the moving body 20 will be described.

[0072] FIG. 19 is a diagram showing how a Waypoint is set using the designation bar 620. FIG. 20 is a diagram showing the display screen 610h when a Waypoint is set by the designation bar 620.

[0073] In the present embodiment, a spherical designated object 622 is provided at the tip of the designation bar 620. Here, the designation bar 620 may be a touch pen or the like that can touch the user terminal 10 to perform various operations. Also, it is assumed that the user terminal 10h is equipped with a sensor that can detect the three-dimensional position of the designated object 622 provided in the designation bar 620. Specifically, the user terminal 10h is equipped with a distance measuring sensor such as a ToF sensor or a stereo camera, for example. The user terminal 10 acquires the position information of the designated object 622 based on the sensor information of the distance measuring sensor. The position information of the designated object 622 is expressed in three dimensions of (x, y, z). Here, z is the direction of gravity (vertical direction). The x and y directions are directions perpendicular to the z direction and are orthogonal to each other.

[0074] Here, a Waypoint is set based on the position information. Specifically, for example, when the user performs a touch operation or the like on the display screen 610h, the user terminal 10 sets the position of the designated object 622 as a Waypoint.

[0075] At this time, an image 618 of the designation bar and an image 616 of the designated object are displayed on the display screen 610h of the user terminal 10h. Therefore, the user can set a Waypoint while confirming the position of the designated object 622 on the display screen 610h.

[0076] <<3.2. Generation of Imaging Information>> As described above, operations for generating movement information (more specifically, information including Waypoints) for controlling the movement of the mobile body 20 have been explained. Next, two methods for generating imaging information for controlling the range imaged by the imaging device provided in the mobile body 20 will be described with reference to FIGS. 21 and 22. FIG. 21 is a diagram showing a state in which the orientation of the imaging device of the mobile body 20 is set by moving the orientation of the user terminal 10i. Further, FIG. 22 is a diagram showing a state in which the field of view angle of the imaging device of the mobile body 20 is set by performing a pinch operation on the display screen of the user terminal 10j.

[0077] First, with reference to FIG. 21, a method for setting the direction in which the imaging device of the mobile body 20 images will be described. The user selects, for example, one of the Waypoints included in the route of the mobile body 20. In this state, as shown in FIG. 21, by the user moving the orientation of the user terminal 10i, the direction in which the imaging unit 110a provided in the user terminal 10i images (that is, the imaging ranges 134a, 134b) can be adjusted. The direction information regarding the adjusted direction in which the imaging unit 110a images is generated as imaging information. That is, the user terminal 10i can generate direction information based on the attitude information of the user terminal 10i. The imaging device of the mobile body 20 can image in the same direction as the adjusted imaging unit 110a at the set Waypoint based on the direction information.

[0078] Also, the user terminal 10 according to the present embodiment can generate field of view angle information for controlling the field of view angle of the imaging device of the mobile body 20 based on a pinch-out operation or a pinch-in operation on the display screen by the user.

[0079] Next, with reference to FIG. 22, a method for setting the angle of view of the imaging device of the mobile body 20 will be described. For example, the user selects one of the Waypoints included in the path of the mobile body 20 in advance. In this state, when the user performs a pinch-out operation or a pinch-in operation on the display screen of the user terminal 10j, the imaging range 134 of the imaging unit 110a is adjusted, so that the angle of view of the imaging device when the mobile body 20 passes through the selected Waypoint can be set. That is, the user terminal 10j can generate angle-of-view information for controlling the angle of view of the imaging device of the mobile body 20 based on the pinch-out operation or the pinch-in operation on the display screen by the user.

[0080] As described above, based on the user's operation, the generation of the direction information and the angle-of-view information by the user terminal 10 has been described. The imaging device of the mobile body 20 can perform imaging based on the direction information and the angle-of-view information. Note that the above direction information and angle-of-view information may be generated when the position of the Waypoint is set, or may be generated after the position of the Waypoint is set.

[0081] <<3.3. Simulation of the Operation of the Mobile Body>> Next, with reference to FIGS. 23 and 24, the simulation of the operation of the mobile body 20 by the user terminal 10 based on the movement information and the imaging information will be described. Specifically, the user terminal 10 simulates the movement of the mobile body 20 and the images captured by the imaging device provided in the mobile body 20. FIG. 23 is a diagram showing a display screen 611 for displaying the simulation result of the movement of the mobile body 20. FIG. 24 is a diagram showing a display screen 610k for displaying the simulation result of the image captured by the imaging device provided in the mobile body 20.

[0082] As shown in FIG. 23, on the display screen 611 of the user terminal 10, an image 612i of a virtual object of a tower placed on a table and an image 630 of a moving body shown schematically in a triangle are displayed. When the simulation of the movement of the image 630 of the moving body is started, the image 630 of the moving body moves along a virtual path 615 connected by the images 616a - m of the Waypoints. By checking the movement of the image 630 of the moving body, the user can predict how the actual moving body 20 moves.

[0083] Also, according to the user terminal 10k according to the present embodiment, it is also possible to simulate an image captured by the imaging device provided in the moving body 20. Specifically, when the moving body 20 flies through the Waypoints set as described above, the user terminal 10k can display an image predicted to be captured by the imaging device of the moving body 20. As shown in FIG. 24, an image predicted to be captured is displayed on the display screen 610k of the user terminal 10k. By looking at the display screen 610k, the user can predict an image captured by the imaging device provided in the moving body 20. Note that the user can also stop the video displayed on the display screen 610k by touching the stop button 619 shown at the center of the display screen 610k.

[0084] <4. Imaging Method> Hereinafter, a method of imaging a landscape using the moving body 20 will be described. First, three methods of imaging a landscape without using the technology of the present disclosure described above will be described. Then, a method of imaging a landscape using the moving body 20 using the technology of the present disclosure will be described.

[0085] In the following description, it is assumed that the mobile object 20 (e.g., a drone) flies around a building like a tower to capture, for example, an impressive commercial video. In such a case, it is necessary for the mobile object 20 to fly in a three-dimensional space. Therefore, an impressive video can be captured only by appropriately controlling various conditions such as the flight position, speed, and camera orientation of the mobile object 20. For this reason, an advanced technique for operating the mobile object 20 is required to capture an impressive video.

[0086] Also, it is difficult for the user to fly the mobile object 20 along the same trajectory manually multiple times. In addition, when imaging outdoors or over a large area, it is necessary to consider sunlight conditions or people coming and going, and the timing of imaging becomes important. Therefore, by repeating the imaging, a video with good conditions can be captured.

[0087] <<4.1. Imaging Method by Manual Operation>> First, with reference to FIG. 25, a method will be described in which the user operates the mobile object 20 by manual operation using a control device and causes the imaging device of the mobile object 20 to capture a landscape. Here, it is assumed that the movement of the mobile object 20 and the orientation of the imaging device provided in the mobile object 20 are operated by the control device. FIG. 25 is a flowchart showing the imaging method by manual operation. Hereinafter, the imaging method by manual operation will be described according to the flowchart shown in FIG. 25.

[0088] First, the user checks the difference in images according to the imaging conditions (step S101). More specifically, the user actually flies the mobile object 20 around the building by manual operation and checks the difference in the appearance of the images according to the imaging conditions such as the orientation of the imaging device of the mobile object 20 or the distance between the mobile object 20 and the building. It is preferable that the user is familiar with operating the mobile object 20.

[0089] Next, the user captures an image using the mobile body 20 (step S103). More specifically, the user manually controls the flight of the mobile body 20 and the orientation of the imaging device so that an impressive image is captured, and causes the imaging device of the mobile body 20 to capture an image.

[0090] At this time, the user may display a two-dimensional map screen together with the image captured by the imaging device provided in the mobile body 20 on various known mobile terminals such as a tablet terminal, and display the path along which the mobile body 20 is flying. Further, the user may set a Waypoint on the path based on a predetermined rule. Thereby, the user can set a Waypoint while checking the captured image.

[0091] Next, the user checks the image captured in step S103 (step S105). If the flight of the mobile body 20 and the orientation of the imaging device are controlled as intended (step S107: YES), the process proceeds to step S109. On the other hand, if the flight of the mobile body 20 and the orientation of the imaging device are not controlled as intended (step S107: NO), the process returns to step S103.

[0092] Even if the flight of the mobile body 20 and the orientation of the imaging device are controlled as intended (step S107: YES), for example, if there is a timing when the day is cloudy or an unintended person crosses in front of the imaging device, and an impressive image as intended is not captured (step S109: NO), the process returns to step S103. On the other hand, if an impressive image as intended is captured (step S109: YES), the imaging method shown in FIG. 25 ends.

[0093] The method of capturing an image by manual operation has been described above. According to such a method, in order to obtain an intended image, it is necessary to reproduce the flight of the same mobile body 20 and the orientation of the imaging device by manual operation many times. Therefore, in order to obtain a desired image, it takes time and effort, and also requires manual labor each time an image is captured.

[0094] <<4.2. Imaging Method by Automatic Flight>> Next, with reference to FIG. 26, a method of causing an imaging device to capture an image while causing a moving body 20 to fly automatically will be described. FIG. 26 is a flowchart showing a method of causing the moving body 20 to fly automatically and causing the imaging device to capture an image. Hereinafter, the description will be made along the flowchart shown in FIG. 26.

[0095] First, the processes of steps S201 to S207 are performed. Since the processes of steps S201 to S207 are substantially the same as the processes of steps S101 to S107, the description thereof will be omitted here.

[0096] When the flight of the moving body 20 and the orientation of the imaging device are controlled as intended (step S207: YES), the data of the imaging conditions is saved (step S209). More specifically, various imaging conditions such as the position, speed of the moving body 20 during flight, and the orientation of the imaging device when the flight of the moving body 20 and the orientation of the imaging device are controlled as intended are recorded. The imaging conditions are recorded on various known recording media provided in the moving body 20 or the like. Note that information regarding the position or speed of the moving body 20 is acquired by a GPS or IMU or the like provided in the moving body 20.

[0097] Next, the imaging operation is reproduced (step S211). More specifically, based on the imaging conditions recorded in step S209, the flight of the moving body 20 and the orientation of the imaging device are automatically reproduced. The image captured at this time is confirmed by the user.

[0098] If an image as intended is not captured (step S213: NO), the process returns to step S211 and the imaging operation is reproduced again. On the other hand, for example, when the sunlight conditions or the like are suitable and an image as intended is captured (step S213: YES), the imaging shown in FIG. 26 ends.

[0099] The imaging method by automatic flight has been described above. According to such a method, the user does not need to manually operate the same moving body 20, so the burden on the user is reduced.

[0100] Based on the data recorded in step S209, for example, a map can be displayed on a display screen such as a tablet terminal, and a virtual path of the moving body 20 can be depicted on the map and finely adjusted. However, when the virtual path is displayed on a two-dimensional map screen, for example, it is difficult to intuitively adjust the altitude of the virtual path.

[0101] Also, since the self-position of the moving body 20 is calculated using GPS (Global Positioning System) or IMU, etc., an error of about 50 cm to 1 m that depends on GPS occurs in the relative position with respect to a building or the like.

[0102] <<4.3. Imaging method using the map displayed on the terminal display screen>> In the above method, the user needs to go to the actual site where the moving body 20 flies and perform the operation of the moving body 20 and the setting of the Waypoint. Therefore, a method can be considered in which a path along which the moving body 20 flies is specified in advance using a map or the like displayed on a display screen such as a tablet terminal, and the moving body 20 is made to fly along the specified path.

[0103] More specifically, the user, for example, displays a map on the display screen of a tablet terminal and sets a Waypoint by touching the display screen or the like. Note that the position of the Waypoint may be set by longitude and latitude. At this time, the user may set the speed and altitude of the moving body 20 at the specified Waypoint. Further, the user can also set the direction of the moving body 20 at the specified Waypoint. For example, it can also be set so that the direction of the moving body 20 faces the traveling direction.

[0104] Similarly, a plurality of Waypoints are set, and by connecting these Waypoints, a path of the moving body 20 is set. The user can cause the moving body 20 to fly along the set path and cause the imaging device to capture an image by recording or transmitting the information representing the set path to the moving body 20.

[0105] According to such a method, the user can set the route along which the mobile body 20 flies before going to the site where the mobile body 20 flies. Therefore, the user can set the route along which the mobile body 20 flies while staying at a company, home, or the like. However, what kind of video will be captured by the imaging device of the mobile body 20 is not known until the mobile body 20 is actually flown and the imaging device is made to capture a video. Further, how the video captured by the imaging device changes when the position of the Waypoint is corrected is not known unless the mobile body 20 is actually flown and the imaging device is made to capture a video.

[0106] In addition, the method of installing the Waypoint by touching the map displayed on the display screen is convenient when the mobile body 20 is flown roughly over a wide area. However, when the imaging device of the mobile body 20 is made to capture the surroundings of a building in a jerky manner, it is considered difficult to finely set the route of the mobile body 20 and the like. Further, since a two-dimensional map is displayed on the display screen, it is necessary to set the altitude of the mobile body 20 numerically, and the Waypoint cannot be set intuitively.

[0107] <<4.4. Imaging Method According to the Present Disclosure>> Next, the imaging method according to the present disclosure will be described with reference to FIGS. 27 to 29. FIG. 27 is a flowchart showing the procedure until a virtual object is generated. FIG. 28 is a flowchart showing the procedure until a video is captured based on the generated movement information and imaging information. Further, FIG. 29 is a diagram showing the display process by the information processing device 100. Hereinafter, the imaging method according to the present disclosure will be described with reference to FIGS. 27 to 29. In the following description, FIGS. 2 to 24 described above will be appropriately referred to.

[0108] The operation of the user in step S301 shown in FIG. 27 is substantially the same as the operation of the user in step S101. However, in the imaging method according to the present disclosure, the user who operates the moving body 20 may be unfamiliar with the operation of the moving body 20.

[0109] Next, the user causes the imaging device of the moving body 20 to capture an image (step S303). The user causes the imaging device of the moving body 20 to capture a subject that is the basis of the virtual object. At this time, the user may manually control the flight of the moving body 20 and the orientation of the imaging device so that an impressive image is captured, and cause the imaging device of the moving body 20 to capture an image. For example, the user may cause the moving body 20 to turn around the tower 420 as shown in FIG. 4 and cause the imaging device 206 to capture an image. At this time, it is assumed that the imaging device 206 captures an image including the tower 420 and the forest 430.

[0110] Next, the user checks the captured image (step S305). More specifically, the user checks that the image captured by the imaging device 206 is the intended image.

[0111] Next, a virtual object is generated (step S307). More specifically, based on information such as the image captured in step S303, the position, orientation of the moving body 20, and the orientation of the imaging device 206 when the image was captured, information regarding a three-dimensional virtual object is generated using various known CG technologies. For example, information regarding the virtual object 422 of the tower shown in FIG. 5 and the virtual object 432 of the forest shown in FIG. 6 is generated.

[0112] Note that the position and orientation of the moving body 20 may be calculated with higher accuracy not only by using GPS and IMU, but also in the process of generating a three-dimensional virtual object based on the captured images by Bundle Adjustment. Thereby, the relative position or orientation of the moving body 20 with respect to the environment such as a building can be calculated more accurately. Here, Bundle Adjustment is a method for estimating various parameters with high accuracy from images. Information regarding the virtual object generated at this time is recorded in the storage unit 180 provided in the user terminal 10. At this time, information regarding the path 402 and the waypoint 406 along which the moving body 20 has moved as shown in FIG. 7 may be recorded in the storage unit 180.

[0113] As described above, with reference to FIG. 27, the process until the virtual object is generated has been described. Next, with reference to FIGS. 28 and 29, the procedure until the desired video is captured will be described. Note that the processes in steps S401 to S405 shown in FIGS. 28 and 29 are mainly processes performed by the information processing apparatus 100 according to an embodiment of the present disclosure.

[0114] The information processing apparatus 100 performs display processing of the virtual object (step S401). The display processing of the virtual object will be described with reference to FIG. 29. FIG. 29 is a flowchart showing the display processing of the virtual object. Hereinafter, the display processing of the virtual object will be described according to the flowchart shown in FIG. 29. The process shown in FIG. 29 is executed, for example, when the start button 602 displayed on the display screen 610 of the user terminal 10a is touched as described with reference to FIG. 8.

[0115] First, the acquisition unit 140 acquires image information and sensor information (step S501). More specifically, the acquisition unit 140 acquires image information including the desk 500 imaged by the imaging unit 110. Further, the acquisition unit 140 acquires IMU information detected by the sensor unit 120, distance information from the user terminal 10a to the desk 500, and the like. The acquisition unit 140 transmits the acquired image information and distance information to the detection unit 151 included in the processing unit 150. Also, the acquisition unit 140 transmits the acquired image information, distance information, and IMU information to the self-position calculation unit 154 included in the processing unit 150.

[0116] Next, the plane detection unit 152 detects a plane based on the image information and distance information transmitted from the acquisition unit 140 (step S503). Here, the plane detection unit 152 detects a flat plane 506 on the desk 500. The plane detection unit 152 transmits the detected result to the virtual object calculation unit 155.

[0117] Next, the self-position calculation unit 154 calculates the self-position of the user terminal 10 based on the image information, distance information, and IMU information (step S505). More specifically, the self-position calculation unit 154 calculates the position and orientation of the user terminal 10 with respect to the desk 500 or the surrounding environment. The self-position calculation unit 154 transmits the calculated result to the virtual object calculation unit 155.

[0118] Next, the virtual object calculation unit 155 calculates the position, direction, scale, etc. of the virtual object to be arranged based on the calculation result of the self-position calculation unit 154 and the information regarding the virtual object recorded in the storage unit 180 (step S507). The virtual object calculation unit 155 transmits the calculated result to the movement information generation unit 157.

[0119] Next, the movement information generation unit 157 sets the route of the moving body 20 based on the calculation result of the virtual object calculation unit 155 and the Waypoint information recorded in the storage unit 180 (step S509). For example, the movement information generation unit 157 sets a virtual route that turns around a virtual object placed on the desk 500. The movement information generation unit 157 transmits information regarding the set virtual route to the display information generation unit 159.

[0120] Next, the display information generation unit 159 generates display information (step S511). More specifically, the display information generation unit 159 generates display information for displaying the virtual route of the moving body 20 around the virtual object placed on the desk 500, and transmits the generated display information to the display control unit 170.

[0121] Next, the display control unit 170 controls the display of the display unit 175 so that an image of the virtual route is displayed around the virtual object placed on the desk 500 (step S513). As a result, on the display screen of the display unit 175, an image 612 of the virtual object 422 of the tower on the desk 500 existing in front of the user's eyes and an image 614 of the virtual route turning around it are displayed.

[0122] The display process of the virtual object has been described above with reference to FIG. 29. Next, returning to FIG. 28, the imaging method according to the present disclosure will be described.

[0123] The information processing apparatus 100 generates movement information and imaging information (step S403). For example, as described with reference to FIGS. 9 to 22, movement information such as Waypoint and imaging information such as the orientation and zoom ratio of the imaging device are generated based on an operation of moving the user terminal 10 by the user, and are transmitted to the prediction unit 160. Here, the processing of the information processing apparatus 100 in the operation described with reference to FIGS. 9 to 22 will be described.

[0124] (Processing for adjusting a preset Waypoint) Here, the processing of the information processing apparatus 100 in the process of adjusting the Waypoint described with reference to FIGS. 9 to 12 will be described. As shown in FIG. 9, when the user U2 touches the image 616a of the Waypoint displayed on the display screen 610a, the input unit 130 transmits input information indicating that the Waypoint 408a corresponding to the image 616a of the Waypoint has been selected to the movement information generation unit 157.

[0125] Next, as shown in FIGS. 11 and 12, when the user U2 pulls the user terminal 10 toward the user U2 side, the sensor unit 120 detects the movement of the user terminal 10 and transmits the detected sensor information to the self-position calculation unit 154. The self-position calculation unit 154 calculates the position and orientation of the user terminal 10 based on the sensor information. The self-position calculation unit 154 transmits the calculated result to the movement information generation unit 157.

[0126] Next, the movement information generation unit 157 corrects the virtual path of the moving body 20 so that the position of the selected Waypoint 408a is displaced by the distance that the user terminal 10 has moved. Thereby, information regarding the new virtual path is generated as movement information and transmitted to the prediction unit 160.

[0127] (Process of newly setting a Waypoint) Next, the processing performed by the information processing apparatus 100 in the operation described with reference to FIG. 13 or FIG. 14 will be described. The acquisition unit 140 acquires input information based on an operation on the display screen 610 by the user from the input unit 130. Further, the acquisition unit 140 acquires image information from the imaging unit 110, distance information and IMU information from the sensor unit 120. The acquisition unit 140 transmits the acquired information to the processing unit 150.

[0128] The self-position calculation unit 154 calculates the self-position of the user terminal 10 based on the transmitted sensor information, distance information, etc., and transmits the calculated result to the generation unit 156. The movement information generation unit 157 identifies the positions of the Waypoints based on the calculated result and input information, etc. The movement information generation unit 157 sets a virtual path of the moving body 20 by connecting the identified multiple Waypoints, and transmits the virtual path to the prediction unit 160 as movement information.

[0129] (Process of setting Waypoints using a designated rod) Next, the processing of the information processing apparatus 100 when setting Waypoints using the designated rod 620, which was described with reference to FIGS. 19 and 20, will be described. First, the acquisition unit 140 acquires IMU information and sensor information from the sensor unit 120. Further, the acquisition unit 140 acquires image information from the imaging unit 110 and transmits the image information to the object detection unit 153.

[0130] The object detection unit 153 detects the designated object 622 included in the image based on the image information. Further, the object detection unit 153 detects, based on the sensor information, for example, the distance and direction from the imaging unit 110 to the designated object 622, and transmits the detected result to the generation unit 156.

[0131] The movement information generation unit 157 identifies the position of the designated object 622 based on the result detected by the object detection unit 153, and sets the position as a Waypoint. The movement information generation unit 157 sets a virtual path by connecting the set multiple Waypoints, and transmits the virtual path to the prediction unit 160 as movement information.

[0132] (Process of setting the direction of the imaging device) Next, the processing of the information processing apparatus 100 when setting the direction of the imaging device, which was described with reference to FIG. 21, will be explained. The self-position calculation unit 154 calculates the posture of the user terminal 10 based on the sensor information, and transmits the calculated result to the generation unit 156. The imaging information generation unit 158 sets the direction of the imaging device based on the calculated posture. The imaging information generation unit 158 transmits the set direction of the imaging device to the prediction unit 160 as direction information.

[0133] (Processing for setting the angle of view of the imaging device) Next, the processing of the information processing apparatus 100 when setting the angle of view of the imaging device, which was described with reference to FIG. 22, will be explained.

[0134] The imaging information generation unit 158 acquires input information indicating that a pinch-in operation or a pinch-out operation by the user has been performed from the input unit 130. The imaging information generation unit 158 generates angle-of-view information representing the angle of view of the imaging device based on the input information, and transmits it to the prediction unit 160.

[0135] Next, the information processing apparatus 100 performs a simulation of the movement of the moving body 20 and the video imaged by the imaging device (step S405). For example, the prediction unit 160 simulates the movement of the moving body 20 on the display screen 611. Specifically, the movement prediction unit 161 predicts the movement of the moving body 20 based on the movement information, and transmits the prediction result to the display information generation unit 159.

[0136] Also, the prediction unit 160 simulates the video to be imaged. More specifically, the imaging prediction unit 162 predicts the video imaged by the moving body 20 based on the movement information and the imaging information, and transmits the prediction result to the display information generation unit 159.

[0137] Next, the display unit 175 displays the prediction result (step S407). More specifically, the display information generation unit 159 generates display information for displaying the prediction result based on the prediction result and transmits it to the display control unit 170. The display control unit 170 controls the display of the display unit 175 so that the display unit 175 displays the prediction result based on the display information. As a result, the prediction result is displayed on the display unit 175. More specifically, the prediction result of the movement of the moving body 20 shown in FIG. 23 or the prediction result of the video imaged by the imaging device of the moving body 20 shown in FIG. 24 is displayed.

[0138] Next, when the simulation as intended is performed (step S409: YES), the process proceeds to step S411. At this time, the storage unit 180 may store the movement information and imaging information used in the simulation. On the other hand, when the simulation as intended is not performed (step S409: NO), the process returns to step S403.

[0139] Next, the movement of the moving body 20 and the imaging by the imaging device are performed (step S411). For example, the virtual path of the moving body 20 formed in step S403 is converted into an actual path along which the moving body 20 actually moves by being converted into a coordinate system in the real space by the movement information generation unit 157. Information regarding the actual path is transmitted to the moving body 20 by the communication control unit 190. While the moving body 20 flies along the generated actual path, the imaging device images the scenery based on the imaging information.

[0140] When an impressive video as intended is imaged (step S413: YES), the imaging process shown in FIG. 28 ends. On the other hand, when an impressive video as intended is not imaged (step S413: NO), the process returns to step S411.

[0141] <5. Effects> The imaging method according to the present disclosure has been described above. The information processing apparatus 100 according to the present disclosure controls the display of virtual objects based on objects existing in the real space on the display screen, and generates movement information for controlling the movement of the moving body. For this reason, when the user wants to fly the moving body 20, such as a drone, around an object existing in the real space that serves as the basis for the virtual object, the user can specify the path of the moving body 20 while viewing the virtual object. Therefore, according to the information processing apparatus 100 according to the present embodiment, it is possible to generate movement information for more intuitively controlling the movement of the moving body 20.

[0142] Also, according to the information processing apparatus 100 according to the present embodiment, the movement information generation unit 157 generates movement information based on an operation by the user U2 who views the display screen 610. The user U2 can specify movement information such as the path of the moving body 20 while viewing the virtual object displayed on the display screen 610. For this reason, it is possible to generate movement information for more intuitively controlling the movement of the moving body 20.

[0143] Also, in the present embodiment, as shown in FIGS. 9 and 11 and the like, the display on the display screen 610 includes an image 614 of the virtual path of the moving body 20. For this reason, it becomes easier for the user U2 to imagine the path of the moving body 20.

[0144] Also, in the present embodiment, an image 616 of one or more Waypoints (adjustment parts) for adjusting the path of the moving body 20 is displayed on at least a part of the image 614 of the virtual path displayed on the display screen 610. The movement information generation unit 157 generates movement information based on an operation of moving the image 616 of the Waypoint. For this reason, the user U2 can specify the path of the moving body 20 only by moving the image 616 of the Waypoint that serves as a landmark for the path, so that it is possible to generate movement information for more intuitively controlling the movement of the moving body 20.

[0145] Also, in the present embodiment, as described with reference to FIG. 13, the movement information generation unit 157 moves the position of the Waypoint 408 based on an operation of moving the position of the display screen 610. Therefore, the user can more intuitively specify the route of the moving body 20.

[0146] Also, in the present embodiment, the image 612 of the virtual object is displayed superimposed on the image captured by the imaging unit 110 provided in the user terminal 10. Therefore, the user U2 can recognize the virtual object 422a as if it actually exists in the real space. Therefore, the user U2 can more intuitively specify the route of the moving body 20. Also, by making the viewpoint of the imaging unit 110 coincide with the Waypoint, it is also possible to display an image captured from the Waypoint on the display screen. Therefore, it is also possible to predict in advance how the captured image will change when the position of the Waypoint changes.

[0147] Also, in the present embodiment, as described with reference to FIGS. 15 to 18, the movement information generation unit 157 generates movement information based on an operation of the user U2 moving the viewpoint of the imaging unit 110. More specifically, the movement information generation unit 157 generates movement information based on the movement of a predetermined position with reference to the viewpoint. The display screen 610 includes an image captured by the imaging unit 110. Therefore, when the moving body 20 actually moves, the user U2 can easily imagine the scenery captured by the imaging device provided in the moving body 20, so that it becomes possible to cause the imaging device provided in the moving body 20 to capture a more desired video.

[0148] Note that, as described with reference to FIGS. 15 and 16, the path of the moving body 20 may be the viewpoint of the imaging unit 110. Further, the path of the moving body 20 may be a position separated from the viewpoint of the imaging unit 110 by a predetermined distance. For example, as described with reference to FIG. 17, a position that is separated from the viewpoint of the imaging unit 110 by a distance d in the forward direction and is lowered to such an extent that it fits within the viewing angle from the optical axis of the imaging unit 110 may be designated as the path of the moving body 20. In this case, as shown in FIG. 18, when a Waypoint or the like is displayed on the display screen 610g, the user U2 can more intuitively specify the path of the moving body 20.

[0149] Further, in the present embodiment, as described with reference to FIGS. 19 and 20, the movement information generation unit 157 generates movement information based on an operation of moving a designation object for designating the path of the moving body 20. More specifically, in the present embodiment, the path of the moving body 20 is designated based on the path along which the designation object 622 provided at the tip of the designation bar 620 has moved. Thereby, the user can designate the path of the moving body 20 by a simple operation of moving the designation object 622. Further, as shown in FIG. 20, an image 616 of the designation object 622 is displayed on the display screen 610h. Therefore, the user U2 can recognize the position of the designation object 622 via the display screen 610h. Therefore, the user U2 can more easily imagine the path of the moving body 20.

[0150] Further, in the present embodiment, the moving body 20 includes an imaging device. The imaging device captures the scenery around the moving body 20. Further, the information processing apparatus 100 according to the present embodiment includes an imaging information generation unit 158 that generates imaging information for controlling the range captured by the imaging device included in the moving body 20 based on a user operation. Therefore, the user can specify the range captured by the imaging device included in the moving body 20 based on various operations, so that a more appropriate video can be captured by the imaging device of the moving body 20.

[0151] In addition, in the present embodiment, the imaging information generation unit 158 generates direction information regarding the direction in which the imaging device included in the moving body 20 images as imaging information. Therefore, the user can cause the imaging device of the moving body 20 to capture a more appropriate video.

[0152] In addition, in the present embodiment, an image captured by the imaging unit 110 is displayed on the display screen. Also, as described with reference to FIG. 21, the imaging information generation unit 158 generates direction information based on an operation of moving the orientation of the imaging unit 110. Therefore, the user can perform direction information while speculating on the video captured by the imaging device of the moving body 20, and thus can cause the imaging device to capture a more appropriate video.

[0153] In addition, in the present embodiment, the imaging information generation unit 158 can generate, as imaging information, angle-of-view information for controlling the angle of view of the imaging device of the moving body 20 based on a pinch-out operation or a pinch-in operation on the display screen by the user U2. Therefore, the user can easily specify the range captured by the imaging device of the moving body 20.

[0154] In addition, in the present embodiment, the information processing device 100 further includes a movement prediction unit 161 that predicts the movement of the moving body 20 based on movement information. More specifically, the movement prediction unit 161 can simulate the movement of the moving body 20. Therefore, the user can confirm the route of the moving body 20 in advance based on the simulation of the movement of the moving body 20. In the present embodiment, as described with reference to FIG. 23, the simulation result (i.e., the prediction result) of the movement of the moving body 20 is displayed on the display screen 611. Therefore, the user can more easily confirm the route of the moving body 20 by looking at the display screen 611.

[0155] Further, in the present embodiment, the information processing apparatus 100 further includes an imaging prediction unit 162 that predicts an image captured by the imaging device of the moving body 20 based on the movement information and the imaging information. In the present embodiment, the imaging prediction unit 162 can simulate an image captured by the imaging device. The user can check the captured image based on the result of the simulation. Further, in the present embodiment, as described with reference to FIG. 24, the result of the simulation by the imaging prediction unit 162 is displayed on the display screen 610k. Therefore, the user can easily check the prediction result by the imaging prediction unit 162.

[0156] Further, in the present embodiment, the moving body 20 can move three-dimensionally. Therefore, the user can specify the path of the moving body 20 three-dimensionally. For this reason, the user can generate movement information for more intuitively controlling the movement of the moving body 20.

[0157] Further, according to the present embodiment, once the path of the moving body 20 is set, it is possible to cause the moving body 20 to fly along the same path many times without manual intervention, or to cause the imaging device to capture the same kind of video many times.

[0158] <6. Hardware Configuration> Next, with reference to FIG. 30, an example of the hardware configuration of the user terminal 10 that constitutes the information processing system 1 according to an embodiment of the present disclosure, like the user terminal 10 described above, will be described in detail. FIG. 30 is a functional block diagram showing a configuration example of the hardware configuration of the user terminal 10 that constitutes the information processing system 1 according to an embodiment of the present disclosure.

[0159] The user terminal 10 that constitutes the information processing system 1 according to the present embodiment mainly includes a CPU 901, a ROM 902, and a RAM 903. Further, the user terminal 10 further includes a host bus 904, a bridge 905, an external bus 906, an interface 907, an input device 908, an output device 909, a storage device 910, a drive 912, a connection port 914, and a communication device 916.

[0160] The CPU 901 functions as an arithmetic processing unit and a control unit, and controls all or part of the operations within the user terminal 10 according to various programs recorded in the ROM 902, the RAM 903, the storage device 910, or the removable recording medium 913. The ROM 902 stores programs, arithmetic parameters, etc. used by the CPU 901. The RAM 903 stores programs used by the CPU 901 and parameters that change as appropriate during program execution, etc. temporarily. These are interconnected by a host bus 904 formed by an internal bus such as a CPU bus. For example, the acquisition unit 140, the processing unit 150 (each functional unit shown in FIG. 3), the display control unit 170, and the communication control unit 190 shown in FIG. 2 may be constituted by the CPU 901.

[0161] The host bus 904 is connected to an external bus 906 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 905. Further, an input device 908, an output device 909, a storage device 910, a drive 912, a connection port 914, and a communication device 916 are connected to the external bus 906 via an interface 907.

[0162] The input device 908 is an operation means operated by the user, such as a mouse, a keyboard, a touch panel, buttons, switches, levers, and pedals. Further, the input device 908 may be, for example, a remote control means (so-called remote control) using infrared rays or other radio waves, or an external connection device 915 such as a mobile phone or a PDA corresponding to the operation of the user terminal 10. Furthermore, the input device 908 is constituted by, for example, an input control circuit that generates an input signal based on information input by the user using the above operation means and outputs it to the CPU 901. The user of the user terminal 10 can input various data to the user terminal 10 or instruct a processing operation by operating this input device 908.

[0163] The output device 909 is composed of a device capable of notifying the user of the acquired information visually or auditorily. Such devices include display devices such as CRT display devices, liquid crystal display devices, plasma display devices, EL display devices, and lamps, audio output devices such as speakers and headphones, and printer devices. The output device 909 outputs, for example, the results obtained by various processes performed by the user terminal 10. Specifically, the display device displays the results obtained by various processes performed by the user terminal 10 in text or image. On the other hand, the audio output device converts an audio signal composed of reproduced audio data, acoustic data, etc. into an analog signal and outputs it.

[0164] The storage device 910 is a data storage device configured as an example of the storage unit of the user terminal 10. The storage device 910 is composed of, for example, a magnetic storage unit device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. This storage device 910 stores programs executed by the CPU 901, various data, etc. For example, the storage unit 180 shown in FIG. 2 can be composed of the storage device 910.

[0165] Drive 912 is a reader / writer for a recording medium and is built into or externally attached to user terminal 10. Drive 912 reads information recorded on a removable recording medium 913 such as a mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs it to RAM 903. Further, drive 912 can also write recordings to a removable recording medium 913 such as a mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory. The removable recording medium 913 is, for example, a DVD medium, an HD-DVD medium, or a Blu-ray (registered trademark) medium, etc. Also, the removable recording medium 913 may be a CompactFlash (registered trademark) (CF), flash memory, or an SD memory card (Secure Digital memory card), etc. Further, the removable recording medium 913 may be, for example, an IC card (Integrated Circuit card) equipped with a non-contact type IC chip or an electronic device, etc.

[0166] Connection port 914 is a port for directly connecting to user terminal 10. Examples of connection port 914 include a USB (Universal Serial Bus) port, an IEEE 1394 port, and a SCSI (Small Computer System Interface) port, etc. Another example of connection port 914 includes an RS-232C port, an optical audio terminal, and an HDMI (registered trademark) (High-Definition Multimedia Interface) port, etc. By connecting an external connection device 915 to this connection port 914, user terminal 10 can directly acquire various data from external connection device 915 or provide various data to external connection device 915.

[0167] The communication device 916 is a communication interface composed of, for example, a communication device or the like for connecting to a communication network 917. The communication device 916 is, for example, a communication card for a wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), or WUSB (Wireless USB). Further, the communication device 916 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication. This communication device 916 can transmit and receive signals, for example, in accordance with a predetermined protocol such as TCP / IP, between the Internet and other communication devices. Further, the communication network 917 connected to the communication device 916 is composed of a network or the like connected by wire or wirelessly, and may be, for example, the Internet, a home LAN, infrared communication, radio wave communication, or satellite communication.

[0168] As described above, an example of a hardware configuration capable of realizing the functions of the user terminal 10 constituting the information processing system 1 according to an embodiment of the present disclosure has been shown. Each of the above-described components may be configured using general-purpose members, or may be configured by hardware specialized for the functions of each component. Therefore, it is possible to appropriately change the hardware configuration to be used according to the technical level at the time of implementing this embodiment. Although not shown in FIG. 30, the user terminal 10 constituting the information processing system 1 naturally includes various configurations corresponding thereto.

[0169] Note that it is possible to create a computer program for realizing each function of the user terminal 10 that constitutes the information processing system 1 according to the above-described embodiment and install it on a personal computer or the like. Further, it is possible to provide a computer-readable recording medium storing such a computer program. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed via a network, for example, without using a recording medium. Further, the number of computers for executing the computer program is not particularly limited. For example, the computer program may be executed by a plurality of computers (for example, a plurality of servers, etc.) in cooperation with each other.

[0170] <7. Supplementary Note> As described above, the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can conceive of various modification examples or correction examples within the scope of the technical idea described in the claims, and it is naturally understood that these also belong to the technical scope of the present disclosure.

[0171] For example, in the above embodiment, the user U1 flies the mobile body 20 in advance and causes the imaging device 206 mounted on the mobile body 20 to capture an image for generating a virtual object, but the present technology is not limited to such an example. For example, when a virtual object has been generated by some method, such as when a virtual object has been generated based on an image captured by a user different from the user U1 using the mobile body 20 in the past, the already generated virtual object may be used.

[0172] In the above-described embodiment, the mobile body 20 has been described as a drone. However, the mobile body 20 may be any movable device. For example, the technology of the present disclosure can also be applied to various flying objects that can fly like drones. Furthermore, the technology of the present disclosure can also be applied to a manipulator corresponding to a robot's hand or arm. In this case, the information processing apparatus may control, for example, the display of a virtual object to be handled by the manipulator on a display screen. Furthermore, the information processing apparatus can generate movement information for controlling the movement of the mobile body, taking, for example, the fingertip of the manipulator as the mobile body. As a result, it becomes possible to more intuitively generate movement information for controlling the movement of the fingertip or the like of the manipulator.

[0173] In the above-described embodiment, information regarding the virtual object and the Waypoint and the like is recorded in the information processing apparatus 100. However, this is not the only case, and information regarding the virtual object and the Waypoint and the like may be recorded in various servers connected to the network. In this case, the information processing apparatus 100 can receive the information recorded in the server as appropriate via the network and generate movement information, imaging information, and the like.

[0174] In the above-described embodiment, it has been assumed that the user terminal 10 is mainly a smartphone or a tablet terminal or the like. However, this is not the only case, and the user terminal 10 may be a general-purpose PC (Personal Computer), a game machine, a robot, or a wearable device such as an HMD (Head Mounted Display) or a smartwatch.

[0175] In addition, the steps shown in the flowchart of the above-described embodiment include not only the processes that are performed in chronological order along the described order, but also the processes that are not necessarily processed in chronological order and can be executed in parallel or individually. Needless to say, even in the steps that are processed in chronological order, the order can be changed as appropriate in some cases.

[0176] Also, the effects described in this specification are merely illustrative or exemplary and not limiting. That is, the technology according to the present disclosure may exhibit other effects that are apparent to those skilled in the art from the description of this specification, in addition to or instead of the above effects.

[0177] Note that the following configurations also belong to the technical scope of the present disclosure. (1) A display control unit that controls the display on the display screen of a virtual object based on an object existing in the real space, and A movement information generation unit that generates movement information for controlling the movement of a moving body. An information processing apparatus. (2) The movement information generation unit generates the movement information based on an operation by a user who views the display screen. The information processing apparatus according to (1) above. (3) The display includes the path of the moving body. The information processing apparatus according to (2) above. (4) At least a part of the path has one or more adjustment portions for adjusting the path displayed, The operation is an operation of moving the position of the adjustment portion displayed on the display screen. The information processing apparatus according to (3) above. (5) The virtual object is displayed superimposed on an image captured by a first imaging device. The information processing apparatus according to any one of (2) to (4) above. (6) The operation includes an operation of moving the viewpoint of the first imaging device, The movement information generation unit generates the movement information based on the movement of a predetermined position with reference to the viewpoint. The information processing apparatus according to (5) above. (7) The movement information generation unit generates the movement information based on an operation of moving a designated object that designates the path of the moving body. The information processing apparatus according to any one of (2) to (6) above. (8) The moving body includes a second imaging device that images a landscape, and further includes an imaging information generation unit that generates imaging information for controlling a range imaged by the second imaging device based on a user operation. The information processing apparatus according to any one of (1) to (7) above. (9) The imaging information generation unit generates direction information regarding the direction in which the second imaging device images as imaging information. The information processing apparatus according to (8) above. (10) An image captured by the first imaging device is displayed on the display screen, and the imaging information generation unit generates the direction information based on an operation of moving the orientation of the first imaging device. The information processing apparatus according to (9) above. (11) The imaging information generation unit generates angle-of-view information for controlling the angle of view of the second imaging device as the imaging information based on a pinch-out operation or a pinch-in operation on the display screen by the user. The information processing apparatus according to any one of (8) to (10) above. (12) The apparatus further includes an imaging prediction unit that predicts an image captured by the second imaging device based on the movement information and the imaging information. The information processing apparatus according to any one of (8) to (11) above. (13) The apparatus further includes a movement prediction unit that predicts the movement of the moving body based on the movement information. The information processing apparatus according to any one of (1) to (12) above. (14) The moving body is movable three-dimensionally. The information processing apparatus according to any one of (1) to (13) above. (15) The moving body is an aircraft. The information processing apparatus according to the above (14). (16) The processor is configured to control the display of virtual objects based on objects existing in the real space on the display screen, and generate movement information for controlling the movement of the moving body, and includes An information processing method. (17) A computer is caused to have a function of controlling the display of virtual objects based on objects existing in the real space on the display screen, and have a function of generating movement information for controlling the movement of the moving body, and A program for realizing the above.

Explanation of Signs

[0178] 10 User terminal 100 Information processing apparatus 110 Imaging unit 120 Sensor unit 130 Input unit 140 Acquisition unit 150 Processing unit 151 Detection unit 157 Movement information generation unit 158 Imaging information generation unit 159 Display information generation unit 161 Movement prediction unit 162 Imaging prediction unit 170 Display control unit 175 Display unit 20 Moving body 202 Airframe 204 Propeller 206 Imaging device 402 Route 404 Virtual route 406, 408, 410, 412 Waypoint 422, 432 Virtual object 610 Display screen 612 Image of virtual object 614 Image of virtual route Image of 616 Waypoint 622 Designated object

Claims

1. A method for detecting an object in real space using an imaging device mounted on a moving object; a movement information generating step of generating movement information for controlling the movement of the moving object in accordance with a scale of a real space by a processor; a display control step in which a processor displays, on a display screen, an object present in a real space photographed by the imaging device and a virtual route based on the generated movement information from a viewpoint of the imaging device while the moving object is moving; The virtual route is a real route along which the moving object travels. Information processing methods.

2. A method for detecting an object in real space using an imaging device mounted on a moving object; a movement information generating step of generating movement information for controlling the movement of the moving object in accordance with a scale of a real space by a processor; a display control step in which a processor displays, on a display screen, an object present in a real space photographed by the imaging device and a virtual route based on the generated movement information from a viewpoint of the imaging device while the moving object is moving; A plane detection step in which a processor detects a plane based on the photographed information; An information processing method comprising the steps of:

3. A movement information generating step in which a processor generates movement information for controlling the movement of a moving object according to a scale of a real space based on an operation by a user viewing a display screen; a display control step of a processor displaying, on the display screen, an object existing in a real space and a virtual route based on the generated movement information; the virtual route is displayed superimposed on an image captured by an imaging device mounted on the moving object, the operation includes an operation of moving a viewpoint of the imaging device, In the movement information generating step, the movement information is generated based on a movement of a predetermined position with respect to the viewpoint. Information processing methods.

4. In the movement information generating step, the movement information is generated based on an operation by a user viewing the display screen.

3. The information processing method according to claim 1 or 2.

5. At least one adjustment portion for adjusting the virtual route is displayed on at least a portion of the virtual route; the operation is an operation of moving the position of the adjustment part displayed on the display screen, The information processing method according to claim 4.

6. The virtual route is displayed superimposed on an image captured by the imaging device.

3. The information processing method according to claim 1 or 2.

7. In the movement information generating step, the movement information is generated based on an operation of moving a designated object that designates a route of the moving body. The information processing method according to claim 4.

8. The moving body is capable of moving three-dimensionally. The information processing method according to any one of claims 1 to 7.

9. The moving object is an aircraft. The information processing method according to claim 8.

10. A movement information generating unit that generates movement information for controlling the movement of a moving body equipped with an imaging device that captures an image of an object existing in real space, in accordance with the scale of the real space; a display control unit that displays, on a display screen, an object present in a real space photographed by the imaging device and a virtual route based on the generated movement information from a viewpoint of the imaging device while the moving object is moving; The virtual route is a real route along which the moving object travels. Information processing device.

11. Computer, a movement information generating unit that generates movement information for controlling the movement of a moving body equipped with an imaging device that captures an image of an object existing in the real space, in accordance with the scale of the real space; a display control unit that displays, on a display screen, an object present in a real space photographed by the imaging device and a virtual route based on the generated movement information from a viewpoint of the imaging device while the moving object is moving; The virtual route is a real route along which the moving object travels. program.

Images