Control method, program, and control system

Abstract

[Problem] To smoothly move a mobile body to a part of an object as intuitively instructed by a user. [Solution] Provided is a control method of a mobile body, the method comprising identifying a positional relationship between the mobile body and a terminal device, acquiring an input of a movement straight line from the terminal device to the object by the terminal device, and moving the mobile body toward the object along the movement straight line.

Description

Control method, program, and control system

[0001] The present disclosure relates to a control method, a program, and a control system.

[0002] As disclosed in Patent Document 1, head mounted displays (HMDs) that integrate a camera, a display optical system, and a sensor have become widespread. In HMDs, a virtual object can be displayed in an image of real space captured by a camera by arranging the virtual object in a coordinate system of real space sensed by a sensor.

[0003] International Publication No. 2020 / 235035

[0004] On the other hand, in recent years, the use of mobile objects such as drones has become common in the construction industry. For example, it has become common to use mobile objects to inspect or survey structures or facilities in order to improve work efficiency and ensure user safety.

[0005] However, the size and scale of objects to be inspected or surveyed in the construction industry are extremely large, and therefore, manually moving a mobile object such as a drone to various locations on the large object for inspection or survey has been a significant burden for users.

[0006] Therefore, there is a need for a mobile object to be smoothly moved to the location of an object that the user intuitively points to.

[0007] According to the present disclosure, a method for controlling a moving body is provided, which includes determining the relative positions of a moving body and a terminal device, obtaining an input of a movement line from the terminal device to an object at the terminal device, and moving the moving body toward the object along the movement line.

[0008] Furthermore, according to the present disclosure, a program is provided for causing a computer to function as a positional relationship determination unit that determines the respective positional relationships between a mobile body and a terminal device, an instruction acquisition unit that acquires input of a movement line from the terminal device to an object, and a mobile body control unit that moves the mobile body toward the object along the movement line.

[0009] Furthermore, according to the present disclosure, a control system is provided that includes a terminal device that acquires input of a line of movement toward an object, and a moving body whose positional relationship with the terminal device has been identified, and the moving body is controlled to move toward the object along the line of movement.

[0010] FIG. 1 is an explanatory diagram illustrating an overview of the technology according to the present disclosure. FIG. 2 is a block diagram illustrating the functional configuration of a control system that realizes the technology according to the present disclosure. FIG. 3 is a block diagram illustrating the functional configuration of a control system according to a modified example. FIG. 4 is a flowchart illustrating the flow of operation of the control system according to the present embodiment. FIG. 5 is an explanatory diagram illustrating a procedure for standardizing the coordinate systems of a terminal device and a moving body. FIG. 6 is an explanatory diagram illustrating an example of an image displayed on the display unit of a terminal device. FIG. 7 is an explanatory diagram illustrating movement of a moving body onto a moving line and movement along the moving line of a moving body. FIG. 8 is an explanatory diagram illustrating an example of an image displayed on the display unit of a terminal device when a moving body moves near an object. FIG. 9 is an explanatory diagram illustrating an example of an image displayed on the display unit of a terminal device when the position and orientation of a moving body are adjusted by a gesture from a user. FIG. 10 is a block diagram illustrating an example of the hardware configuration of an information processing device that realizes a terminal device.

[0011] Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0012] The description will be given in the following order: 1. Overview of the technology according to the present disclosure 2. Embodiment 2.1. Configuration example 2.2. Modification 2.3. Operation example 3. Hardware configuration example

[0013] 1. Overview of Technology According to the Present Disclosure An overview of the technology according to the present disclosure will be described with reference to Fig. 1. Fig. 1 is an explanatory diagram illustrating an overview of the technology according to the present disclosure.

[0014] 1 , the technology according to the present disclosure is a control method for controlling the movement of a moving body D at a work site F where an object S to be worked on, inspected, etc. is present. Specifically, the technology according to the present disclosure is a control method for acquiring, by a terminal device T, a movement line L to the object S indicated by a gesture G by a user U, and for moving the moving body D toward the object S along the acquired movement line L.

[0015] The site F is a location where inspections and the like are performed on the target object S. The site F may be a construction site, a work site, an inspection site, or the like.

[0016] The target object S is a facility, building, or structure present at a site F. A user U can inspect the target object S by capturing an image of the target object S using a mobile object D.

[0017] The mobile body D is, for example, a drone flying in the air. The mobile body D senses the environment of the site F using a sensor or the like, thereby creating an environmental map of the site F and estimating the mobile body D's own position within the environmental map. This allows the mobile body D to move autonomously or by remote control within the environment of the site F. The mobile body D may be a multicopter with multiple rotors, an aircraft with at least a pair of fixed wings, or an airship or balloon with a gas bag filled with a buoyancy gas that is lighter than the atmosphere.

[0018] The gesture G is a gesture or hand gesture of the user U that indicates the movement line L. The gesture G may be, for example, a pointing motion using the user U's finger, a direction indicating motion using the user U's hand or arm, or a gazing motion using the user U's line of sight.

[0019] The terminal device T is an information processing terminal carried by the user U. Specifically, the terminal device T includes a sensor or a camera and a display unit. The terminal device T detects a movement line L indicated by a gesture G made by the user U using the sensor or the camera, and can display an image of an object S captured by a moving body D moving along the movement line L on the display unit. The terminal device T may be, for example, an HMD device worn by the user U.

[0020] Furthermore, the terminal device T senses the environment of the site F using a sensor or a camera, thereby creating an environmental map of the site F and estimating the terminal device T's own position within the environmental map. This allows the terminal device T to place a virtual object in the real space of the site F based on the estimated own position. The virtual object placed in the real space is displayed on the display unit of the terminal device T as an augmented reality (AR) image by being superimposed on a see-through image of the real space, for example.

[0021] The terminal device T and the moving body D can identify their positional relationship by sharing a coordinate system of their estimated self-positions. This allows the moving body D to use the movement line L acquired by the terminal device T for movement control of the moving body D without converting the coordinate system, and therefore, can move more smoothly along the movement line L.

[0022] The technology disclosed herein acquires, with a terminal device T, a movement line L to a target object S indicated by a gesture G made by a user U, and transmits the acquired movement line L to the moving object D, thereby making it possible to move the moving object D along the movement line L to the target object S. This allows the user U to more intuitively move the moving object D to a location where inspection of the target object S will be performed.

[0023] 2. Embodiments> (2.1. Configuration Example) A configuration example of a control system that realizes the technology according to the present disclosure will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of a control system 1 that realizes the technology according to the present disclosure.

[0024] 2 , a control system 1 according to an embodiment of the present disclosure includes a terminal device T and a mobile object D. The terminal device T includes a sensor 111, a camera 112, a self-position estimation unit 120, an instruction acquisition unit 130, a communication unit 140, a positional relationship identification unit 150, and a display unit 160. The mobile object D includes a sensor 211, a camera 212, a motor 213, a self-position estimation unit 220, a communication unit 240, a positional relationship identification unit 250, and a mobile object control unit 270.

[0025] (Terminal Device T) The sensor 111 includes a sensor that senses the environment of the site F around the terminal device T, or a sensor that senses the position or attitude of the terminal device T. The results of sensing by the sensor 111 can be used, for example, to estimate the self-position of the terminal device T. Furthermore, when the sensor 111 senses a gesture G of the user U, the results of sensing by the sensor 111 can be used to detect an instruction of the user U represented by the gesture G. For example, the sensor 111 may include a LiDAR (Light Detection And Ranging) or a ToF sensor as a sensor that senses the environment of the site F. Furthermore, the sensor 111 may include an inertial measurement unit (IMU) or a GNSS (Global Navigation Satellite System) sensor as a sensor that senses the position or attitude of the terminal device T.

[0026] The camera 112 includes an imaging device that captures an image of a site F around the terminal device T. The image of the site F captured by the camera 112 can be used, for example, to estimate the self-position of the terminal device T. Furthermore, the image of a gesture G of the user U captured by the camera 112 can be used to detect an instruction of the user U represented by the gesture G.

[0027] The self-position estimation unit 120 estimates the self-position of the terminal device T based on information acquired by the sensor 111 or the camera 112. As one example, the self-position estimation unit 120 may estimate the self-position of the terminal device T by SLAM (Simultaneous Localization and Mapping) using information on the environment of the site F acquired by the sensor 111 or the camera 112. As another example, the self-position estimation unit 120 may estimate the self-position of the terminal device T based on information on the position or attitude of the terminal device T acquired by the sensor 111. As yet another example, the self-position estimation unit 120 may estimate the self-position of the terminal device T by combining the above two self-position estimation methods.

[0028] The instruction acquisition unit 130 detects a gesture G made by the user U, and thereby acquires an instruction of the user U represented by the detected gesture G. For example, the instruction acquisition unit 130 may acquire a movement line L indicated by the gesture G by detecting the gesture G of the user U with the sensor 111 or the camera 112. Specifically, the instruction acquisition unit 130 may acquire, as the movement line L, a line extending to the target object S in a direction indicated by the gesture G made by the user U using a finger, hand, arm, or line of sight. Furthermore, the instruction acquisition unit 130 may acquire an operation input to the terminal device T or the moving body D indicated by the user U via the gesture G.

[0029] The communication unit 140 is a wireless communication interface that transmits and receives data between the terminal device T and the mobile body D. As an example, the communication unit 140 may be a communication interface for a wireless LAN (Local Area Network), Wi-Fi (registered trademark), Bluetooth (registered trademark), or WUSB (Wireless USB) that performs wireless communication with the mobile body D. As another example, the communication unit 140 may be a router or modem for various types of communication that performs wireless communication with the mobile body D via a network such as an Internet communication network, an infrared communication network, or a mobile communication network.

[0030] For example, the communication unit 140 may receive from the moving body D the estimated self-position and coordinate system of the moving body D, as well as an image captured by the camera 212 of the moving body D. The communication unit 140 may also transmit to the moving body D the estimated self-position and coordinate system of the terminal device T, as well as the movement line L acquired by the instruction acquisition unit 130.

[0031] The positional relationship specifying unit 150 performs calibration to specify the positional relationship between the terminal device T and the moving object D.

[0032] As an example, the positional relationship identification unit 150 may identify the positional relationship between the terminal device T and the moving body D by making the coordinate system of the terminal device T and the coordinate system of the moving body D common to each other. The coordinate system of the terminal device T is the coordinate system used for estimating the self-position of the terminal device T, and the coordinate system of the moving body D is the coordinate system used for estimating the self-position of the moving body D. The positional relationship identification unit 150 may make the coordinate systems of the terminal device T and the moving body D common to each other by using the same object imaged by both the terminal device T and the moving body D as a reference.

[0033] As another example, the positional relationship identification unit 150 may identify the relative positions of the terminal device T and the moving body D to identify the positional relationship between the terminal device T and the moving body D. Specifically, the positional relationship identification unit 150 may identify the position of the moving body D in the coordinate system of the terminal device T by capturing an image of a marker attached to the moving body D with the camera 112 of the terminal device T.

[0034] In addition, when the terminal device T and the moving body D each have a positioning system that can constantly grasp the relative position of each other, the positional relationship identification unit 150 may identify the positional relationship between the terminal device T and the moving body D based on the positioning system. Examples of positioning systems that can constantly grasp the relative positions of each other include a positioning system using a UWB (Ultra Wide Band) sensor, a positioning system using an RTK (Real-time Kinematic) sensor and a GNSS sensor, and a positioning system using multiple cameras.

[0035] The display unit 160 displays an image captured by the camera 112 of the terminal device T as a see-through image. The display unit 160 may also display an image in which a virtual object is superimposed on the see-through image captured by the camera 112 of the terminal device T. Furthermore, the display unit 160 may display an image captured by the camera 212 of the moving body D alongside or in switchover with the see-through image captured by the camera 112 of the terminal device T. Specific examples of images displayed on the display unit 160 will be described later.

[0036] (Mobile body D) The sensor 211 includes a sensor that senses the environment of the site F around the mobile body D, or a sensor that senses the position or attitude of the mobile body D. The results of sensing by the sensor 211 can be used, for example, to estimate the self-position of the mobile body D. For example, the sensor 211 may include a LiDAR (Light Detection and Ranging) or a barometric pressure sensor as a sensor that senses the environment of the site F. Furthermore, the sensor 211 may include an inertial measurement unit (IMU) or a GNSS (Global Navigation Satellite System) sensor as a sensor that senses the position or attitude of the mobile body D.

[0037] The camera 212 includes an imaging device that captures an image of a site F around the moving body D. The image of the site F captured by the camera 212 can be used, for example, to estimate the self-position of the moving body D. In addition, the image of the object S captured by the camera 212 can be displayed on the display unit 160 of the terminal device T.

[0038] The self-position estimation unit 220 estimates the self-position of the moving body D based on information acquired by the sensor 211 or the camera 212. As one example, the self-position estimation unit 220 may estimate the self-position of the moving body D by SLAM (Simultaneous Localization and Mapping) using information about the environment of the site F acquired by the sensor 211 or the camera 212. As another example, the self-position estimation unit 220 may estimate the self-position of the moving body D based on information about the position or attitude of the moving body D acquired by the sensor 211. As yet another example, the self-position estimation unit 220 may estimate the self-position of the moving body D by combining the above two self-position estimation methods.

[0039] The communication unit 240 is a wireless communication interface that transmits and receives data between the mobile body D and the terminal device T. As an example, the communication unit 240 may be a communication interface for a wireless LAN (Local Area Network), Wi-Fi (registered trademark), Bluetooth (registered trademark), or WUSB (Wireless USB) that performs wireless communication with the terminal device T. As another example, the communication unit 240 may be a router or modem for various types of communication that performs wireless communication with the mobile body D via a network such as an Internet communication network, an infrared communication network, or a mobile communication network.

[0040] The communication unit 240 may receive the estimated self-position and coordinate system of the terminal device T, and the movement line L acquired by the instruction acquisition unit 130, from the terminal device T. The communication unit 240 may also transmit the estimated self-position and coordinate system of the moving body D, and an image captured by the camera 212 of the moving body D, to the terminal device T.

[0041] The positional relationship specifying unit 250 performs calibration to specify the positional relationship between the moving body D and the terminal device T.

[0042] As an example, the positional relationship identification unit 250 may identify the positional relationship between the moving body D and the terminal device T by making the coordinate system of the moving body D and the coordinate system of the terminal device T common to each other. The coordinate system of the moving body D is a coordinate system used for estimating the self-position of the moving body D, and the coordinate system of the terminal device T is a coordinate system used for estimating the self-position of the terminal device T. The positional relationship identification unit 250 may make the coordinate systems of the moving body D and the terminal device T common to each other by using the same object imaged by both the moving body D and the terminal device T as a reference.

[0043] As another example, the positional relationship identification unit 250 may identify the relative positions of the moving body D and the terminal device T, thereby identifying the positional relationship between the moving body D and the terminal device T. Specifically, the positional relationship identification unit 250 may identify the position of the terminal device T in the coordinate system of the moving body D by capturing an image of a marker attached to the terminal device T with the camera 212 of the moving body D.

[0044] In addition, when the moving body D and the terminal device T each have a positioning system that can constantly grasp the relative position of each other, the positional relationship identification unit 250 may identify the positional relationship between the moving body D and the terminal device T based on the positioning system. Examples of positioning systems that can constantly grasp the relative positions of each other include a positioning system using a UWB sensor, a positioning system using an RTK sensor and a GNSS sensor, and a positioning system using multiple cameras.

[0045] The motor 213 is a drive device that operates a movement mechanism (e.g., a rotor or a propeller) of the moving body D. The motor 213 may be, for example, a drive device that rotates a rotor or a drive device that rotates a propeller. The motor 213 is driven under the control of the moving body control unit 270 to operate the movement mechanism, thereby moving the moving body D to a desired position.

[0046] The mobile object control unit 270 controls the movement of the mobile object D by controlling the driving of the motor 213. For example, the mobile object control unit 270 may control the movement of the mobile object D based on the self-position of the mobile object D estimated by the self-position estimation unit 220 and an environmental map of the site F around the mobile object D.

[0047] When the mobile object control unit 270 receives from the terminal device T the movement line L indicated by the gesture G of the user U, it moves the mobile object D on the movement line L and then moves the mobile object D along the movement line L toward the object S. Subsequently, when the sensor 211 or the camera 212 detects approach to an obstacle including the object S, the mobile object control unit 270 stops the mobile object D moving along the movement line L. This allows the mobile object control unit 270 to smoothly move the mobile object D to the vicinity of the object S that is present on the movement line L indicated by the gesture G by the user U. Therefore, the control system 1 can smoothly move the mobile object D to the location of the object S to be inspected, etc., based on the intuitive instruction of the user U via the gesture G.

[0048] (2.2. Modification) Next, a modification of the control system 1 according to the present embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the functional configuration of a control system 1A according to the modification.

[0049] As shown in Fig. 3 , a control system 1A according to the modified example includes a terminal device T, an operation device C, and a mobile object D. The terminal device T is similar to the terminal device T shown in Fig. 2 except that the communication unit 140 is omitted. The mobile object D is similar to the mobile object D shown in Fig. 2. The operation device C includes an input unit 310 and a communication unit 340.

[0050] The operation device C is a controller for the moving body D to which movement instructions to the moving body D are input via the input unit 310. The input unit 310 may be, for example, an input device such as a touch panel, a button, a switch, or a lever that accepts input from the user U. The communication unit 340 is a wireless communication interface that transmits and receives data between the operation device C and the moving body D.

[0051] In the control system 1A according to the modified example, the terminal device T and the moving object D are connected via the operation device C. Specifically, the terminal device T and the operation device C are connected, for example, by a wired cable or wire, and the operation device C and the moving object D are connected by wireless communication via the communication units 240 and 340. Even in such a case, the control system 1A according to the modified example can move the moving object D toward the target S based on an intuitive instruction from the user U using a gesture G, similar to the control system 1 shown in FIG.

[0052] (2.3. Operational Example) An operational example of the control system 1 according to this embodiment will be described with reference to Figs. 4 to 9. Fig. 4 is a flowchart showing the flow of the operation of the control system 1 according to this embodiment. Figs. 5 to 9 are explanatory diagrams for explaining the details of each operation of the control system 1.

[0053] 4 , first, the self-positions of the moving body D and the terminal device T are estimated (S101). Specifically, in the terminal device T, the self-position estimation unit 120 estimates the self-position of the terminal device T based on information acquired by the sensor 111 or the camera 112. In addition, in the moving body D, the self-position estimation unit 220 estimates the self-position of the moving body D based on information acquired by the sensor 211 or the camera 212.

[0054] Next, the positional relationship between the moving body D and the terminal device T is identified (S102). The positional relationship between the terminal device T and the moving body D may be identified by the positional relationship identifying units 150 and 250 by using a common coordinate system for the terminal device T and the moving body D, respectively.

[0055] 5 is an explanatory diagram illustrating a procedure for standardizing the coordinate systems of the terminal device T and the moving body D. For example, as shown in FIG. 5, first, the same object Ob is captured by the camera 112 of the terminal device T worn by the user U and the camera 212 of the moving body D. The positional relationship identification units 150 and 250 can standardize the coordinate systems of the terminal device T and the moving body D by using the captured common object Ob as a reference. Therefore, the positional relationship identification units 150 and 250 can identify the positional relationship between the terminal device T and the moving body D.

[0056] Next, the instruction acquisition unit 130 of the terminal device T acquires the movement line L instructed by the user U (S103). Specifically, the sensor 111 or the camera 112 of the terminal device T detects the gesture G of the user U, and the instruction acquisition unit 130 acquires the movement line L instructed by the gesture G of the user U.

[0057] The acquired movement line L is superimposed on the see-through image of the terminal device T and displayed on the display unit 160 (S104).

[0058] FIG. 6 is an explanatory diagram illustrating an example of an image displayed on the display unit 160 of the terminal device T. For example, as shown in FIG. 6, the display unit 160 may display an image in which a movement line L is superimposed as a virtual object on a see-through image PT captured by the camera 112 of the terminal device T. Specifically, a line display L1 indicating the movement line L to the object S indicated by the user U's gesture G and a cone display L2 representing a cone shape with an apex angle of approximately 1 degree and centered on the movement line L are superimposed on the see-through image PT. By displaying the cone display L2 in addition to the line display L1, the display unit 160 can represent the location of the object S indicated by the user U's gesture G, taking into account an error resulting from the distance to the object S. The line display L1 and the cone display L2 are confirmed, for example, by input from the user U performing the gesture G, such as by voice or button press.

[0059] Next, the movement line L indicated by the gesture G of the user U is confirmed, and the moving object D moves on the movement line L (S105).

[0060] FIG. 7 is an explanatory diagram illustrating the movement of a moving object D on a moving line L and the movement of the moving object D along the moving line L. As shown in FIG. 7 , the moving object D may move on the moving line L via the shortest route. Specifically, if the moving object D is present in the air between the user U and the object S, the moving object D may move to an intersection R between the moving line L and a perpendicular line drawn from the current position Q of the moving object D to the moving line L (i.e., the shortest route between the current position Q and the moving line L). On the other hand, if the moving object D is not present in the air between the user U and the object S and is waiting on the ground, the moving object D may move to a point on the moving line L that is a predetermined distance (e.g., approximately 3 m) away from the user U. At this time, the moving object D may point the angle of view of the camera 212 toward the object S on the moving line L.

[0061] Thereafter, the moving body D moves toward the object S along the movement line L (S106). At this time, the moving body D determines whether or not an obstacle including the object S has been detected (S107).

[0062] 7, for example, the moving body D approaches the object S by moving along the movement line L. As a result, the moving body D detects the approach of the object S or another obstacle by the sensor 111 or the camera 112 as it moves along the movement line L. If an obstacle is detected (YES in S107), the moving body D stops moving to avoid contact with the obstacle (S108). On the other hand, if no obstacle is detected (NO in S107), the moving body D continues moving along the movement line L until an obstacle is detected (S106).

[0063] After the movement of the moving body D is stopped, the image captured by the camera 212 of the moving body D is displayed on the display unit 160 of the terminal device T (S109). Specifically, the image of the object S captured by the camera 212 of the moving body D is transmitted from the moving body D to the terminal device T, and the transmitted image of the object S is displayed alongside the see-through image on the display unit 160 of the terminal device T.

[0064] FIG. 8 is an explanatory diagram showing an example of an image displayed on the display unit 160 of the terminal device T when the moving body D moves near the target object S. For example, as shown in FIG. 8 , the display unit 160 may display a moving body image PD captured by the camera 212 of the moving body D alongside a see-through image PT captured by the camera 112 of the terminal device T. The display unit 160 may also display an image VD representing the angle of view of the camera 212 of the moving body D and an image DI indicating the status of the moving body D superimposed on the see-through image PT. The image DI indicating the status of the moving body D may display, for example, the speed of the moving body D, the remaining charge of the battery installed in the moving body D, or the route of the moving body D. According to the image shown in FIG. 8 , the user U can simultaneously check the position information of the moving body D within the environment of the site F, the status information of the moving body D, and the image information of the target object S captured by the moving body D on the display unit 160 of the terminal device T.

[0065] The see-through image PT captured by the camera 112 of the terminal device T and the moving body image PD captured by the camera 212 of the moving body D may be switched to full-screen display by an input operation from the user U. This allows the user U to flexibly check the position information of the moving body D, the status information of the moving body D, and the image information of the target object S captured by the moving body D in various display formats by switching between them.

[0066] Furthermore, the position and orientation of the moving object D are adjusted by input of a gesture G from the user U (S110).

[0067] FIG. 9 is an explanatory diagram showing an example of an image displayed on the display unit 160 of the terminal device T when the position and orientation of the moving body D are adjusted by a gesture G from the user U. For example, as shown in FIG. 9 , the user U can adjust the position and orientation of the moving body D so that the camera 212 of the moving body D captures a desired part of the target S by manipulating the moving body image PD captured by the camera 212 of the moving body D with the gesture G. Examples of gestures G on the moving body image PD include a swipe operation or a drag operation to move the moving body image PD up, down, left, or right, a pinch-out operation or a pinch-in operation to enlarge or reduce the moving body image PD, or a grab operation to grab and move the moving body image PD. Furthermore, the terminal device T may detect the movement of the head of the user U wearing the terminal device T and output an instruction to control the orientation of the angle of view of the camera 212 of the moving body D based on the movement of the user U's head.

[0068] Thereafter, the moving body D whose position and orientation have been adjusted performs an inspection of the target object S (S111).

[0069] According to the control system 1 according to the present embodiment that executes the above operations, the user U can move the moving body D to a location where inspection of the object S is to be performed, by an intuitive gesture G. Furthermore, the user U can adjust the position and orientation of the moving body D while checking the image of the object S captured by the camera 212 of the moving body D, which allows for smoother inspection of the object S.

[0070] 3. Hardware Configuration Example> The hardware configuration of an information processing device 900 that realizes the terminal device T according to this embodiment will be described below with reference to Fig. 10. Fig. 10 is a block diagram showing an example of the hardware configuration of the information processing device 900 that realizes the terminal device T.

[0071] The functions of the information processing device 900 that realizes the terminal device T may be realized by cooperation between software and hardware described below. The functions of the self-location estimation unit 120, the instruction acquisition unit 130, and the positional relationship determination unit 150 may be executed by, for example, the CPU 901. The functions of the display unit 160 may be executed by, for example, the output device 907. The functions of the communication unit 140 may be executed by the connection port 910 or the communication device 911.

[0072] As shown in FIG. 10, the information processing device 900 includes a CPU (Central Processing Unit) 901 , a ROM (Read Only Memory) 902 , and a RAM (Random Access Memory) 903 .

[0073] The information processing device 900 may further include a host bus 904a, a bridge 904, an external bus 904b, an interface 905, an input device 906, an output device 907, a storage device 908, a drive 909, a connection port 910, or a communication device 911. The information processing device 900 may include a processing circuit such as a DSP (Digital Signal Processor) or an ASIC (Application Specific Integrated Circuit) instead of or in addition to the CPU 901.

[0074] The CPU 901 functions as an arithmetic processing device or control device, and controls operations within the information processing device 900 in accordance with various programs recorded in the ROM 902, RAM 903, storage device 908, or a removable recording medium attached to the drive 909. The ROM 902 stores programs and calculation parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901, and parameters used during the execution of the programs.

[0075] The CPU 901, ROM 902, and RAM 903 are interconnected by a host bus 904a capable of high-speed data transmission. The host bus 904a is connected to an external bus 904b, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 904. The external bus 904b is connected to various components via an interface 905.

[0076] The input device 906 is a device that accepts input from a user, such as a mouse, keyboard, touch panel, button, switch, or lever. The input device 906 may also be a microphone that detects the user's voice. The input device 906 may also be, for example, a remote control device that uses infrared rays or other radio waves, or may be an externally connected device that supports operation of the information processing device 900.

[0077] The input device 906 further includes an input control circuit that outputs an input signal generated based on information input by the user to the CPU 901. By operating the input device 906, the user can input various data to the information processing device 900 or instruct the information processing device 900 to perform processing operations.

[0078] The output device 907 is a device capable of visually or audibly presenting information acquired or generated by the information processing device 900 to a user. The output device 907 may be, for example, a display device such as an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), an OLED (Organic Light Emitting Diode) display, a hologram, or a projector, or may be a sound output device such as a speaker or headphones, or a printing device such as a printer. The output device 907 can output information acquired by processing by the information processing device 900 as video such as text or an image, or sound such as voice or audio.

[0079] The storage device 908 is a data storage device configured as an example of a storage unit of the information processing device 900. The storage device 908 may be configured, for example, by a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 908 can store programs executed by the CPU 901, various data, various data acquired from the outside, and the like.

[0080] The drive 909 is a device for reading or writing data from or to a removable recording medium such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and is built into or externally attached to the information processing device 900. For example, the drive 909 can read information recorded on an attached removable recording medium and output the information to the RAM 903. The drive 909 can also write data to an attached removable recording medium.

[0081] The connection port 910 is a port for directly connecting an external device to the information processing device 900. The connection port 910 may be, for example, a Universal Serial Bus (USB) port, an IEEE 1394 port, or a Small Computer System Interface (SCSI) port. The connection port 910 may also be an RS-232C port, an optical audio terminal, or a High-Definition Multimedia Interface (HDMI) (registered trademark) port. By connecting the connection port 910 to an external device, various types of data can be transmitted and received between the information processing device 900 and the external device.

[0082] The communication device 911 is, for example, a communication interface configured with a communication device for connecting to the communication network 920. The communication device 911 may be, for example, a communication card for a wired or wireless LAN (Local Area Network), Wi-Fi (registered trademark), Bluetooth (registered trademark), or WUSB (Wireless USB). The communication device 911 may also be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication.

[0083] The communication device 911 can transmit and receive signals, for example, via the Internet or other communication devices using a predetermined protocol such as TCP / IP. The communication network 920 connected to the communication device 911 is a wired or wireless network, and may be, for example, an Internet communication network, a home LAN, an infrared communication network, a radio wave communication network, or a satellite communication network.

[0084] It is also possible to create a program that causes hardware such as the CPU 901, ROM 902, and RAM 903 built into a computer to perform functions equivalent to those of the information processing device 900. It is also possible to provide a computer-readable recording medium on which the program is recorded.

[0085] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0086] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0087] Note that the following configurations also fall within the technical scope of the present disclosure. (1) A method for controlling a moving body, comprising: identifying a positional relationship between a moving body and a terminal device; acquiring, at the terminal device, an input of a movement line from the terminal device to an object; and moving the moving body toward the object along the movement line. (2) The method for controlling a moving body described in (1), in which movement of the moving body toward the object is stopped based on a determination of proximity between the moving body and an obstacle including the object. (3) The method for controlling a moving body described in (2), further comprising, after moving the moving body toward the object, acquiring an input for adjusting a position of the moving body. (4) The method for controlling a moving body described in (3), in which at least one of the input of the movement line or the input for adjusting the position of the moving body is an input by gesture. (5) The method for controlling a moving body described in (4), in which the input by gesture is acquired by the terminal device. (6) The method for controlling a moving body according to any one of (1) to (5), wherein, when moving toward the target, the moving body turns the angle of view of an imaging device mounted on the moving body toward the target. (7) The method for controlling a moving body according to any one of (6), further comprising inspecting the target using the imaging device after moving the moving body toward the target. (8) The method for controlling a moving body according to any one of (1) to (7), wherein the moving body and the terminal device specify a mutual positional relationship by sharing a coordinate system of their estimated self-positions. (9) The method for controlling a moving body according to any one of (1) to (8), wherein the terminal device further includes a display unit that displays a see-through image captured by the terminal device. (10) The method for controlling a moving body according to (9), further comprising superimposing an image showing the movement line on the see-through image. (11) The method for controlling a moving body according to (10), wherein the image showing the line of movement is expressed in a cone shape with the terminal device as the apex and the line of movement as the center axis. (12) The method for controlling a moving body according to any one of (9) to (11), wherein the display unit displays the see-through image and an image of the moving body captured by an imaging device mounted on the moving body side by side.(13) The method for controlling a moving body according to any one of (9) to (12), wherein the terminal device is a head-mounted display device worn by a user. (14) A program for causing a computer to function as: a positional relationship specifying unit that specifies a positional relationship between a moving body and a terminal device; an instruction acquisition unit that acquires an input of a movement line from the terminal device to an object; and a moving body control unit that moves the moving body toward the object along the movement line. (15) A control system including: a terminal device that acquires an input of a movement line toward an object; and a moving body whose positional relationship with the terminal device has been specified, wherein movement of the moving body is controlled so that the moving body moves toward the object along the movement line.

[0088] 1, 1A Control system 111, 211 Sensor 112, 212 Camera 120, 220 Self-position estimation unit 130 Instruction acquisition unit 140, 240, 340 Communication unit 150, 250 Positional relationship determination unit 160 Display unit 213 Motor 270 Mobile object control unit 310 Input unit D Mobile object T Terminal device S Object C Operation device U User G Gesture L Movement line

Claims

1. A method for controlling a moving object, comprising: determining the relative positions of a moving object and a terminal device; acquiring an input of a movement line from the terminal device to an object at the terminal device; and moving the moving object along the movement line toward the object.

2. A method for controlling a moving body according to claim 1, wherein movement of the moving body toward the object is stopped based on a determination of proximity between the moving body and an obstacle including the object.

3. The method for controlling a moving object according to claim 2, further comprising obtaining an input for adjusting the position of the moving object after moving the moving object toward the target.

4. The method for controlling a moving body according to claim 3, wherein at least one of the input of the movement line and the input for adjusting the position of the moving body is input by gesture.

5. The method for controlling a moving object according to claim 4, wherein the gesture input is acquired by the terminal device.

6. The method for controlling a moving body according to claim 1, wherein, when moving toward the target, the moving body directs the angle of view of an imaging device mounted on the moving body toward the target.

7. The method for controlling a moving body according to claim 6, further comprising inspecting the object using the imaging device after moving the moving body toward the object.

8. A method for controlling a mobile body according to claim 1, wherein the mobile body and the terminal device identify their mutual positional relationship by sharing a coordinate system for their estimated self-positions.

9. The method for controlling a moving body according to claim 1, wherein the terminal device further comprises a display unit for displaying a see-through image captured by the terminal device.

10. The method for controlling a moving body according to claim 9, further comprising superimposing an image showing the movement line on the see-through image.

11. The method for controlling a moving body according to claim 10, wherein the image showing the line of movement is expressed in a cone shape with the terminal device as the apex and the line of movement as the central axis.

12. A method for controlling a moving body according to claim 9, wherein the display unit displays the see-through image and an image of the moving body captured by an imaging device mounted on the moving body side by side.

13. The method for controlling a moving object according to claim 9, wherein the terminal device is a head-mounted display device worn by a user.

14. A program for causing a computer to function as: a positional relationship determination unit that determines the respective positional relationship between a mobile object and a terminal device; an instruction acquisition unit that acquires input of a movement line from the terminal device to an object; and a mobile object control unit that moves the mobile object along the movement line toward the object.

15. A control system comprising: a terminal device that acquires input of a movement line toward an object; and a mobile body whose positional relationship with the terminal device has been identified; wherein the movement of the mobile body is controlled so that it moves toward the object along the movement line.

Images