Systems and methods for controlling a mobile object, a mobile object, and a program

Abstract

It stably controls the movement of a moving object based on the movement of the object it is following. [Solution] Information indicating the position and direction of movement of a person is acquired. A graph consisting of nodes and edges, showing the person's movement path, is acquired. Based on the person's position, a first edge of the graph corresponding to the person's position is determined. Based on the first edge or one or more edges located in the direction of the person's movement relative to the first edge, the target position of the moving object is determined.

Description

Technical Field

[0006] , ,

[0001] The present invention relates to a system and method for controlling a moving object, a moving object, and a program, and particularly to a moving object that moves according to the movement of a person.

Background Art

[0002] The development of autonomous mobile robots has been actively carried out. For example, Patent Document 1 discloses an autonomous mobile robot that moves following a following target such as a moving person. Patent Document 1 attempts to achieve a more natural following by setting a following target on the side or behind the current position of the following target.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Even with a method of controlling a moving object to move toward a following target set on the side or behind the current position of the following target as in Patent Document 1, the movement of the moving object may become unstable based on the movement of the following target. Also, with the method of Patent Document 1, it is difficult to move the moving object according to the movement of a person by a method other than following (for example, having a person lead the moving object).

[0005] An object of the present invention is to stably control the movement of a moving object that moves based on the movement of a following target.

Means for Solving the Problems

[0006] A system according to an embodiment of the present invention has the following configuration. That is, A system for controlling a moving object to move according to the movement of a person, Information acquisition means for acquiring information indicating a person's position and direction of movement, A graph acquisition means that acquires a graph showing a person's movement path, which is composed of nodes and edges, A determination means for determining the first edge of the graph corresponding to the position of the person, based on the position of the person, A determination means for determining the target position of the moving body based on the first edge or one or more edges located in the direction of movement of the person relative to the first edge, It is equipped with. [Effects of the Invention]

[0007] It is possible to stably control the movement of a moving object based on the movement of the object being followed. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram showing the external appearance of a mobile body according to one embodiment. [Figure 2] A diagram showing an example of the functional configuration of a mobile body according to one embodiment. [Figure 3] A flowchart of a processing method according to one embodiment. [Figure 4] A diagram explaining how to create a graph. [Figure 5] A diagram illustrating how to predict a user's destination. [Figure 6] A diagram illustrating the method for determining the target location. [Figure 7] A diagram illustrating the method for determining the target location. [Figure 8] A diagram illustrating the method for determining the target location. [Figure 9] A diagram illustrating the method for determining the target location. [Figure 10] A diagram explaining how to update the graph. [Modes for carrying out the invention]

[0009] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims, and not all combinations of features described in the embodiments are essential to the invention. Two or more of the features described in the embodiments may be combined in any way. Furthermore, identical or similar configurations will be given the same reference numeral, and redundant descriptions will be omitted.

[0010] <Mobile Unit Configuration> Referring to Figure 1, an example of the external configuration of a mobile body 100 according to one embodiment will be described. In Figure 1, arrow X indicates the front-to-back direction of the mobile body 100. F indicates the front, and R indicates the rear. Arrows Y and Z indicate the width direction (left-right direction) and the up-down direction of the mobile body 100, respectively. The mobile body 100 is capable of autonomous movement. For example, the mobile body 100 is equipped with a battery and moves mainly by motor power. The mobile body 100 may be used in the vicinity of amusement facilities, large commercial facilities, airports, parks, sidewalks, parking lots, etc. The mobile body 100 may be a vehicle that moves on the ground using wheels, an aircraft that moves in the air (e.g., a drone), or a robot that moves on the ground using two or more legs. Movement using wheels is also called driving. Movement in the air is also called flying. Movement using two or more legs is also called walking. The following description will focus on the case where the mobile body 100 is a vehicle, but similar descriptions apply to other forms of mobile bodies.

[0011] The mobile object 100 is capable of moving in accordance with the movement of a person (for example, the user of the mobile object 100, hereinafter simply referred to as the user). For example, the mobile object 100 is capable of moving in accordance with the user. To move in accordance with the user means that the mobile object 100 moves based on the user's proactive movement. For example, the mobile object 100 can move in a way that leads the user, alongside the user, or following the user. The following example describes a configuration in which no person rides in the mobile object 100, but the mobile object 100 may carry a person other than the user.

[0012] The mobile body 100 includes, for example, a pair of left and right front wheels 101 and rear wheels 102 included in the traveling unit 204 (FIG. 2). The traveling unit 204 may be in other forms, such as the form of a four-wheeled vehicle or a two-wheeled vehicle. Instead of or in addition to accompanying the user of the mobile body 100, the mobile body 100 may be capable of autonomously moving to a preset destination.

[0013] The mobile body 100 has a housing 110 capable of accommodating luggage. The front surface 111 of the housing is provided with a lid that can be opened and closed for accommodating luggage. This lid can have a locking mechanism. The locking mechanism is controlled by the mobile body 100. For example, when the user authentication of the mobile body 100 is successful, the locked state is released. Instead of this, the mobile body 100 may be capable of accommodating luggage in other manners.

[0014] A touch screen 120 is arranged on the upper surface 112 of the housing. The user can, via the touch screen 120, for example, change the settings of the mobile body 100 or check information regarding facilities. The sensor box 130 houses a detection unit 206 (FIG. 2) therein. The detection unit 206 includes sensors such as a camera. The detection unit 206 recognizes users and other targets included in the environment around the mobile body 100 through the front surface 131, side surfaces, and rear surface of the sensor box 130.

[0015] <Functional configuration example of the mobile body> Referring to FIG. 2, a functional configuration example of the mobile body 100 will be described. The mobile body 100 includes the components shown in FIG. 2. The mobile body 100 may include components not shown in FIG. 2, or may not include some of the components shown in FIG. 2.

[0016] The mobile body 100 is an electric autonomous mobile body provided with a traveling unit 204 and having a battery 205 as the main power source. The battery 205 is a secondary battery such as a lithium-ion battery, for example. Using the electric power supplied from the battery 205, the traveling unit 204 makes the mobile body 100 run autonomously.

[0017] The driving unit 204 accelerates and decelerates the mobile body 100 by changing the rotational speed of a pair of front wheels 101 using a motor as the drive source. The driving unit 204 may also include a braking mechanism for decelerating the mobile body 100. The driving unit 204 steers the mobile body 100 by making the rotational speeds of the pair of front wheels 101 different. The driving unit 204 can detect and output physical quantities representing the motion of the mobile body 100, such as the mobile body 100's speed, acceleration, steering angle, angular velocity of the mobile body 100's housing 110, and angular acceleration.

[0018] The mobile body 100 includes a detection unit 206 which contains one or more sensors. The detection unit 206 generates data for recognizing targets (including objects and people in the environment surrounding the mobile body 100) contained in the environment surrounding the mobile body 100. The detection unit 206 includes sensors such as an imaging device (camera), radar device, lidar (Light Detection and Ranging), and ultrasonic sensor, which have a detection range around the mobile body 100, and outputs sensor information. The imaging device may be configured to use a fisheye lens or to be configured to perform stereo imaging. Furthermore, the detection unit 206 includes a GNSS (Global Navigation Satellite system) sensor and receives GNSS signals to detect the current position of the mobile body 100. The detection unit 206 may detect the current position using Wi-Fi (Local Area Network) or Bluetooth signals. The imaging device may be an RGB camera or may further have a depth measurement function. For example, the mobile unit 100 may have RGB cameras with depth measurement capabilities on the front and rear sides of the sensor box 130, and RGB cameras without depth measurement capabilities on the right and left sides of the sensor box 130.

[0019] The mobile unit 100 includes a control unit (ECU (Electronic Control Unit)) 201. The control unit 201 functions as a control device for the mobile unit 100. The control unit 201 includes one or more processors 202, such as a CPU (Central Processing Unit), and a memory 203, which is a storage device such as a semiconductor memory. Therefore, the control unit 201 can also be called an information processing device or a computer. The memory 203 stores programs executed by the processors 202 and data used by the processors 202 for processing. Multiple sets of processors 202 and memory 203 may be provided according to the function of the mobile unit 100 and configured to communicate with each other.

[0020] The control unit 201 acquires physical quantities representing the motion output by the travel unit 204, detection results from the detection unit 206, input information from the touchscreen 120, and audio information input from the audio input device 207, and executes corresponding processing. For example, the control unit 201 controls the motor of the travel unit 204, controls the display on the touchscreen 120, and provides audio notifications to the surrounding environment.

[0021] The voice input device 207 picks up sounds from the environment surrounding the mobile body 100. The control unit 201 can recognize the input sounds and execute corresponding processing. The storage device 208 is a non-volatile large-capacity storage device that stores map information including roads that the mobile body 100 can travel on, areas where entry is restricted, landmarks, stores, etc. The storage device 208 may also store programs executed by the processor 202 and data used by the processor 202 for processing. The communication device 209 is a communication device that can connect to an external network via wireless communication such as fifth-generation mobile communication or wireless LAN.

[0022] The display device 210 displays (presents) a user interface screen to the user on the touchscreen 120, or outputs (presents) spoken audio to the environment surrounding the mobile body 100 via the microphone. The input device 211 may include, for example, a touch panel and be configured as an integral part of the touchscreen 120. The input device 211 receives operation input from the user via the touch panel.

[0023] The control unit 201 corresponds to a system or information processing device that predicts the movement of a person, or a system or information processing device that controls a mobile body to move in accordance with the movement of a person, according to one embodiment. The processor 202 of the control unit 201 realizes the functions of the information acquisition unit 221, graph acquisition unit 222, determination unit 223, selection unit 224, prediction unit 225, decision unit 226, control unit 227, and update unit 228 by executing a program stored in the memory 203 or storage device 208. On the other hand, at least some of these functions may be realized by an information processing device other than the mobile body 100. That is, an information processing device such as a computer equipped with a processor and memory can perform the processing of each of these units based on information transmitted from the mobile body 100 as needed by executing a program stored in memory. Furthermore, such an information processing device may remotely control the mobile body 100 based on the processing results. Moreover, at least some of these functions may be realized by a cloud service. The system that predicts the movement of a person or the system that controls a mobile body to move in accordance with the movement of a person, according to one embodiment, may be such an information processing device or cloud system.

[0024] The information acquisition unit 221 acquires information indicating the user's position and direction of movement. The information acquisition unit 221 can acquire information indicating the user's position and direction of movement. For example, the information acquisition unit 221 can acquire information indicating the user's current geographical location. The information acquisition unit 221 can also determine the user's direction of movement based on changes in the user's current geographical location. In another embodiment, the information acquisition unit 221 may acquire information indicating the user's current posture. The information acquisition unit 221 can determine the user's direction of movement based on the user's current posture. The information acquisition unit 221 can estimate a person's position based on the output from a sensor. For example, the information acquisition unit 221 can acquire this information based on the detection result of the detection unit 206 (e.g., an image captured by an imaging device).

[0025] The graph acquisition unit 222 acquires a graph showing the user's movement path. The graph consists of nodes and edges. The edges indicate the person's movement path. Nodes connect two edges. The graph may follow the world coordinate system. Alternatively, the graph may follow another coordinate system, such as a coordinate system based on the moving object 100. In any case, the nodes of the graph correspond to positions in real space. The edges of the graph correspond to paths in real space, and points on the edges correspond to positions on the paths. The edges may be straight or curved.

[0026] Figure 4 shows an example of a graph. In Figure 4, nodes are represented by circles, edges by solid lines, and structures such as buildings and roads by dashed lines. In one embodiment, the graph is created based on environmental features such as roads, buildings, and obstacles. For example, edges can be set along the outer edges of the area where a person can move. Edges can also be set along the outer edges of static obstacles that do not move. In Figure 4, edges 401 to 403 are set along the outer edges of buildings.

[0027] Nodes may be set at the intersection of two or more paths. Nodes may also be set at locations where people frequently pass through. For example, a node may be set at a building entrance, or at the top or bottom of a staircase. Furthermore, a node may be set at a location where people frequently stop. For example, a node may be set at the location of a vending machine. In Figure 4, node 411 is set at a building entrance.

[0028] The method for creating the graph is not particularly limited. For example, a user may manually create a graph based on environmental features. Alternatively, an information processing device may automatically create a graph based on environmental features. On the other hand, the graph may also be created based on pedestrian movement data. For example, data showing the movement trajectories of multiple pedestrians can be extracted based on video or the like. A user may manually create a graph while referring to the movement trajectories of multiple pedestrians. Alternatively, an information processing device may automatically create a graph based on the movement trajectories of multiple pedestrians.

[0029] The storage device 208 can store data for such graphs. In this case, the graph acquisition unit 222 can acquire the graph from the storage device 208. Alternatively, the graph acquisition unit 222 may acquire the graph from an external information processing device via the communication device 209.

[0030] The determination unit 223 determines the first edge of the graph corresponding to the user's position based on the user's position. For example, the determination unit 223 can determine the edge closest to the user's current geographical location as the first edge. In the example in Figure 5, the edge closest to the user's current position, represented by U, is edge 501. Therefore, the determination unit 223 can determine edge 501 as the first edge. The determination unit 223 can determine the first edge based, for example, on the user's position according to the world coordinate system and the graph according to the world coordinate system.

[0031] The determination unit 223 may determine the first edge based on the user's direction of movement. Furthermore, the determination unit 223 may determine the first edge from among multiple edges based on both the distance between each of the multiple edges constituting the graph and the user's position, and the similarity between the direction of each of the multiple edges and the user's direction of movement. For example, the determination unit 223 can set a first score for an edge based on the distance between the user's position and the edge. The determination unit 223 can also set a second score for an edge based on the similarity between the user's direction of movement and the direction of the edge. Then, the determination unit 223 can select the first edge based on the first score and the second score. For example, the first edge selected by the determination unit 223 may be the edge with the highest overall score. The overall score can be calculated based on the first score and the second score.

[0032] The prediction unit 225 can predict the user's destination as a position corresponding to a point on the first edge. In the example in Figure 5, the prediction unit 225 can predict the user's destination as a position corresponding to a point on edge 501, which is the first edge. Here, the prediction unit 225 can predict the user's destination as a position corresponding to a point reached along edge 501 in the direction of the user's movement from the point corresponding to the user's position on edge 501. The point corresponding to the user's position on the first edge can be the point closest to the user's position on the first edge. Furthermore, the prediction unit 225 can determine the user's direction of movement along the first edge according to the direction of movement indicated by the information acquired by the information acquisition unit 221. The prediction unit 225 can determine the user's direction of movement along the first edge such that the dot product of the vector of the user's direction of movement along the first edge at the point corresponding to the user's position on the first edge and the vector of the user's direction of movement indicated by the information acquired by the information acquisition unit 221 is positive. In the example shown in Figure 5, the user (U) is moving in the lower left direction as indicated by the arrow. In this case, the prediction unit 225 can determine that the direction of the user's movement along the edge 510 is from node 510 towards node 511.

[0033] Furthermore, the prediction unit 225 can predict the user's destination as a point on one or more edges located in the user's direction of movement relative to the first edge. An edge located in the user's direction of movement relative to the first edge may include a second edge that is connected to the first edge via a first node located ahead of the user's direction of movement along the first edge, among the two nodes to which the first edge is connected. Additionally, an edge located in the user's direction of movement relative to the first edge may include one or more further edges that are continuous with the second edge via one or more nodes. For example, suppose the first edge is edge 503 and the user is moving in the direction from node 511 to 512. In this case, the selection unit 224 can select node 512, located in the user's direction of movement, as the first node, among the nodes to which edge 503 is connected. The selection unit 224 can also select edge 505, which is connected to edge 503 at node 512, as the second edge. In this case, the prediction unit 225 can predict the position corresponding to a point on the second edge, edge 505, as the user's destination.

[0034] On the other hand, in the example in Figure 5, the selection unit 224 can select node 511, which is located in the user's direction of movement, as the first node, from among the nodes to which edge 501, the first edge, is connected. Here, node 511, the first node, is connected to multiple edges, namely edges 502, 503, and 504. In this case, the selection unit 224 can select a second edge from the multiple edges based on the orientation of each of the multiple edges and the orientation of the first edge or the user's direction of movement. The selection unit 224 can select an edge as the second edge that extends in a direction closer to the orientation of the first edge or the user's direction of movement. In the example in Figure 5, the selection unit 224 can select edge 502 as the second edge. In this case, the prediction unit 225 can predict the position corresponding to a point on edge 505, the second edge, as the user's destination. Note that the orientation of the edge may be the orientation of the edge at the connection point to node 511.

[0035] As a specific example, the selection unit 224 can select a second edge from among multiple edges based on the angle between the first edge and each of the multiple edges. Here, the angle between the first edge and the selected second edge may be the largest angle among the angles between the first edge and each of the multiple edges.

[0036] The selection unit 224 may also select a second edge based on the distance between the user's position and each edge. For example, the selection unit 224 may assign a third score to each of the multiple edges connected to the first edge at the first node, based on the similarity between the edge's orientation and the orientation of the first edge or the user's direction of movement. The third score can be higher the greater the similarity. The selection unit 224 may also assign a fourth score to each of the multiple edges, based on the distance between the edge and the user's position. The fourth score can be higher the shorter the distance. The selection unit 224 can then select a second edge based on the third and fourth scores. For example, the second edge selected by the selection unit 224 may be the edge with the highest overall score. The overall score can be calculated based on the third and fourth scores.

[0037] The prediction unit 225 can predict the user's destination as a point located a predetermined distance away from the point closest to the user's current position on the first edge, along one or more edges in the user's direction of movement. This predetermined distance may be a fixed value. Alternatively, this predetermined distance may be a distance corresponding to the user's movement speed.

[0038] The determination unit 226 determines the target position of the mobile body 100. The target position is the target location where the mobile body 100 will move. The following describes how the determination unit 226 determines the target position so that the mobile body 100 leads the user.

[0039] In one embodiment, the determination unit 226 can determine the target location of the movement based on the user's destination predicted by the prediction unit 225. Figure 6 shows an example of a method for determining the target location of the movement. Figure 6 shows the user's current location (U) and direction of movement (arrow). Figure 6 also shows the current location (R) of the moving object 100. Figure 6 further shows the target location of the moving object 100 (T) and the direction of movement (arrow) from the current location (R).

[0040] In this embodiment, the determination unit 226 determines a target movement position so that the mobile body 100 leads the user, and the movement of the mobile body 100 is controlled. In one embodiment, the determination unit 226 sets the target movement position (T) of the mobile body 100 in front of the user's destination. The determination unit 226 may set the target movement position (T) of the mobile body 100 based on the predicted destination of the user. For example, the target movement position (T) may be a position located along an edge away from the predicted destination of the user. Alternatively, the target movement position (T) may be a position offset in any other way from the predicted destination of the user. For example, the determination unit 226 may set the target movement position (T) so that the mobile body 100 leads diagonally in front of the user. In this case, the target movement position of the mobile body is determined based on a position corresponding to a point on one or more edges located in the direction of the user's movement (i.e., the predicted destination of the user). For example, the determination unit 226 can set a target movement position (T) such that the moving object 100 moves to a location in front of the user's destination after a predetermined time, as predicted by the prediction unit 225.

[0041] On the other hand, in another embodiment, the determination unit 226 may determine the target movement position so that the moving body 100 follows the user. In this case, the determination unit 226 can set the target movement position (T) of the moving body 100 behind the user's destination.

[0042] Furthermore, it is not essential to use the predicted user's destination to determine the target position of the moving body 100. The determination unit 226 can determine the target position of the moving body 100 based on one or more edges located in the user's direction of movement, including the first edge determined by the determination unit 223. For example, the determination unit 226 can determine the target position of the moving body 100 based on a position corresponding to a point on one or more edges located in the user's direction of movement. In the example in Figure 5, the determination unit 226 can set the target position of the moving body 100 at a position corresponding to a point on edge 501, which is the first edge, or edge 502, which is the second edge. The target position of the moving body 100 may be a position corresponding to a point located a predetermined distance away along one or more edges located in the user's direction of movement from the point closest to the user's current position on the first edge. This predetermined distance may be, for example, a distance corresponding to the user's speed of movement. Furthermore, the determination unit 226 may set the target position of the moving body 100 at a position offset from the position corresponding to a point on edge 501 or edge 502. With such a configuration, the moving body 100 can lead the user.

[0043] On the other hand, in the example shown in Figure 7(A), among the nodes to which the first edge corresponding to the user's position (U) is connected, the first node 710, which is located in the direction of the user's movement, has multiple edges connected to it, namely edges 701 and 702. In this situation, the user may move in the direction of edge 701 or in the direction of edge 702. Therefore, the determination unit 226 can determine the target position of the moving body 100 based on the multiple edges (for example, edges 701 and 702) connected to the first node 710.

[0044] For example, the determination unit 226 can evaluate each of the multiple edges connected to the first edge. The determination unit 226 can evaluate each edge based on the relationship between the user's direction of movement and the orientation of the edge. For example, the determination unit 226 can rate the edge higher the closer the user's direction of movement is to the orientation of the edge. Alternatively, the determination unit 226 may evaluate each edge based on the distance between the user's current position and the edge. The evaluation of an edge may also indicate the probability that the user will travel along the path corresponding to the edge. In the example in Figure 7(A), the determination unit 226 evaluates the probability that the user will travel along the paths corresponding to edges 701 and 702 to be 0.5, respectively. Furthermore, the determination unit 226 can determine the position on the path obtained by weighting and combining the paths corresponding to each of the multiple edges according to the evaluation of each edge as the target position of the moving object. In Figure 7(A), the path obtained by weighting and combining the paths corresponding to edges 701 and 702 according to their evaluations (both 0.5) is shown as a dashed line. The determination unit 226 then sets the target position (T) on the dashed line.

[0045] Figure 7(B) shows the user's position (U), the position of the moving object 100 (R), and the target location (T) at a time later than that shown in Figure 7(A). In the example in Figure 7(B), the decision unit 226 evaluates the probability that the user will travel along the paths corresponding to edge 701 and edge 702 to be 0.65 and 0.35, respectively. The decision unit 226 also sets the target location (T) on the path (dashed line) obtained by weighting and combining the paths corresponding to edge 701 and edge 702 according to the evaluation.

[0046] Figure 7(C) shows the user's position (U), the position of the moving object 100 (R), and the target location (T) at a time later than that shown in Figure 7(B). In the example in Figure 7(C), the decision unit 226 evaluates the probability that the user will travel along the paths corresponding to edge 701 and edge 702 to be 1.0 and 0.0, respectively. Therefore, the decision unit 226 sets the target location (T) on the path corresponding to edge 701.

[0047] The control unit 227 controls the movement of the mobile body 100 so that it moves toward the target position determined by the determination unit 226. For example, the control unit 227 can generate a trajectory from the current position of the mobile body 100 toward the target position. The control unit 227 can also generate a trajectory that avoids obstacles. The control unit 227 then moves the mobile body 100 along this trajectory. Depending on the current attitude and speed of the mobile body 100, this trajectory may or may not pass through the target position. The driving unit 204 can move the mobile body 100 according to the control of the control unit 227. For example, the control unit 227 can supply control signals to the driving unit 204 to move the mobile body 100.

[0048] Here, the control unit 227 can control the movement of the moving body 100 so that it moves in a straight line toward the target position. In the example shown in Figure 8(A), the control unit 227 controls the moving body 100 so that it moves in a straight line from its current position (R) to the target position (T). On the other hand, in the example shown in Figure 8(B), there is an obstacle area 800 between the current position (R) and the target position (T) of the moving body 100. Therefore, in response to the control unit 227 determining that there is an obstacle between the current position and the target position of the moving body 100, it can control the moving body 100 so that it moves along the path indicated by the edge. In the example shown in Figure 8(B), the control unit 227 sets a temporary target position (T') on the path indicated by the edge. The control unit 227 then controls the movement of the moving body 100 so that it moves toward the temporary target position (T'). On the other hand, the control unit 227 can control the mobile body 100 so that it moves in a straight line from its current position to the target position, depending on whether it has determined that there are no obstacles between the mobile body 100's current position and the target position. In this way, if it is determined that there are no obstacles between the mobile body 100's current position and the target position while the mobile body 100 is moving along the path indicated by the edge, the mobile body 100 can start moving in a straight line to the target position.

[0049] <Method for controlling a mobile object> Referring to Figure 3, the method by which the control unit 201 controls the mobile body 100 will be described. As described above, the control unit 201 can control the mobile body 100 to accompany the user. Each step of the method in Figure 3 may be performed by the processor 202 executing a program stored in memory 203 or storage device 208. Alternatively, at least some of the steps of the method in Figure 3 may be performed by a dedicated integrated circuit such as an ASIC (Application Specific Integrated Circuit). The method in Figure 3 may be started in response to the user instructing the mobile body 100 to accompany the user. The control unit 201 repeatedly executes the processes S301 to S307 (for example, with a 100ms cycle).

[0050] The method shown in Figure 3 is performed on a user of the mobile device 100. The control unit 201 identifies the user at the start of the method in Figure 3 and controls the mobile device 100 to accompany this user. The control unit 201 may identify a person positioned in front of the mobile device 100 in an image captured by the detection unit 206 (e.g., an imaging device) or a person who has previously registered as a user as the user. During the execution of the method in Figure 3, the control unit 201 continuously determines whether it can detect the user. If the user cannot be detected for a predetermined period of time since the last detection, the control unit 201 may determine that the user has been lost and interrupt the method in Figure 3. Subsequently, the control unit 201 may restart the method in Figure 3 depending on whether the user has been detected.

[0051] In steps S301 to S304, the control unit 201 predicts the user's destination.

[0052] In S301, the information acquisition unit 221 acquires information to be used in subsequent processing. As described above, the information acquisition unit 221 can acquire information indicating the user's position and direction of movement. The information acquisition unit 221 may also acquire moving object information and environmental information. The information acquisition unit 221 may store at least a portion of the acquired information in the memory 203 or storage device 208 for use in subsequent processing.

[0053] Mobile object information refers to information about the mobile object 100. Mobile object information may include the current speed of the mobile object 100, the current geographical location of the mobile object 100, and the current angular velocity of the mobile object 100. The control unit 201 may acquire mobile object information based on the output from the driving unit 204 or the detection results from the detection unit 206 (for example, GNSS positioning data or inertial sensor data).

[0054] Environmental information refers to information about the environment surrounding the mobile body 100. Environmental information may include the number, type, location, and size of targets included in the environment surrounding the mobile body 100. Targets may include static and dynamic targets. Static targets may include structures such as walls, guardrails, pillars, and steps. Static targets are targets that are fixed directly or indirectly to the ground and do not move. Static targets that may obstruct the movement of the mobile body 100 may be called static obstacles. Dynamic targets may include pedestrians, cyclists, autonomous mobile bodies, animals, etc. Dynamic targets are targets that can move relative to the ground. Dynamic targets that may obstruct the movement of the mobile body 100 may be called dynamic obstacles. The control unit 201 may acquire environmental information based on the detection results of the detection unit 206 (for example, images captured by the imaging device).

[0055] In S302, the graph acquisition unit 222 acquires the graph as described above.

[0056] In S303, the determination unit 223 determines the first edge of the graph corresponding to the user's position, as described above. The method for associating the user's position with the graph will be described later.

[0057] In S304, the prediction unit 225 predicts the user's destination as described above. At this time, the selection unit 224 may select the second edge as described above.

[0058] In steps S305-S306, the control unit 201 controls the movement of the mobile body 100.

[0059] In S305, the determination unit 226 determines the target position of the moving body 100 as described above.

[0060] In S306, the control unit 227 controls the movement of the moving body 100 so that it moves toward the target position, as described above.

[0061] Furthermore, while the method shown in Figure 3 is being executed, the control unit 201 can monitor the distance between the moving object 100 and any dynamic obstacles surrounding it. When this distance falls within a threshold (e.g., 1.5 m), the control unit 201 may take action (e.g., make a voice announcement) to ask the dynamic obstacle to yield its path. The control unit 201 may also stop the moving object 100 when the distance between the moving object 100 and any dynamic obstacles surrounding it falls within another threshold (e.g., 1 m).

[0062] After that, the process returns to S301, and the above process is repeated.

[0063] <Map Overlay> To perform movement control according to the graph, the information acquisition unit 221 can associate the position of the moving object 100 and the user's position with the graph. For example, if the graph follows a world coordinate system, the information acquisition unit 221 can use the position of the moving object 100 and the user's position that follow the world coordinate system. The information acquisition unit 221 may also acquire the user's position in a coordinate system based on the moving object 100. In this case, the information acquisition unit 221 can convert the user's position in a coordinate system based on the moving object 100's position and orientation to the world coordinate system. Conversely, the information acquisition unit 221 may convert the graph to a coordinate system based on the moving object 100. The orientation of the moving object 100 following the world coordinate system can be determined based on the results of imaging from an acceleration sensor or indicator.

[0064] The geographical location of a target such as a user can be determined as follows. For example, the information acquisition unit 221 can acquire observation data obtained by the detection unit 206. The detection unit 206 observes targets included in the environment surrounding the moving object 100. The observation data may include images taken by an imaging device (camera), which is an example of the detection unit 206. Alternatively, or in addition to this, the observation data may include data observed by a lidar, which is an example of the detection unit 206. The data observed by the lidar may include the direction of the target and the distance to the target. The information acquisition unit 221 also uses the observation data to determine the geographical location of the target. The geographical location may be represented by two-dimensional coordinate values ​​in the horizontal plane.

[0065] For example, the information acquisition unit 221 may determine the geographic location by homography transforming the image captured by the imaging device (camera). Alternatively, or in addition to this, the information acquisition unit 221 may determine the geographic location based on data observed by the LiDAR (i.e., the direction of the target and the distance to the target).

[0066] Furthermore, the information acquisition unit 221 may use SLAM technology to determine the position and orientation of the moving object 100, as well as the position of the target. For example, the information acquisition unit 221 detects feature points from the image captured by the detection unit 206. The information acquisition unit 221 also matches the feature points detected from the image captured at time t with the feature points detected from the image captured at time t-Δt. Based on the matching results, the information acquisition unit 221 estimates the amount and direction of movement of the moving object 100 between time t and t-Δt. The information acquisition unit 221 can also estimate the position and orientation of the moving object 100 based on the estimated amount and direction of movement of the moving object 100. Furthermore, the information acquisition unit 221 can determine the position of the target based on the position and orientation of the moving object 100 and the feature point detection results.

[0067] Using the method described above, the information acquisition unit 221 can estimate the position of the mobile body 100 based on the output from the detection unit equipped with the mobile body 100. The information acquisition unit 221 can also generate an environmental map showing the positions of targets such as the user and obstacles. Such an environmental map may be a map in a coordinate system based on the mobile body 100.

[0068] Here, the determination unit 226 can determine the target movement position of the mobile body 100 based on a graph that has been converted to a coordinate system based on the mobile body 100 and superimposed on the environment map. In other words, the determination unit 226 can determine the target movement position of the mobile body 100 in a coordinate system based on the mobile body 100. For example, the information acquisition unit 221 can convert the graph to a coordinate system based on the mobile body 100 and superimpose it on the environment map. With this configuration, the movement of the mobile body 100 can be controlled according to the position of the user or obstacle shown on the environment map.

[0069] Incidentally, when the position estimation result of the moving object 100 by the information acquisition unit 221 changes, the transformation parameters from the world coordinate system to the coordinate system based on the moving object 100 change. In this case, the transformation result of the graph to the coordinate system based on the moving object 100 also changes. In one embodiment, the information acquisition unit 221 can deform the graph and overlay it on the map in order to mitigate the change in the transformation result of the graph that accompanies the change in the coordinate system based on the moving object. For example, when the information acquisition unit 221 performs a coordinate transformation on the graph, it can apply a filter that reduces the change in the transformation result of the graph. With such a configuration, it is possible to suppress the rapid change in the target position of the moving object 100 due to a change in the position estimation result of the moving object 100.

[0070] <Graph Update> The update unit 228 can update the graph based on the detection results of obstacles. For example, the update unit 228 can update the graph to delete edges based on its determination that an obstacle exists on an edge. For example, Figure 10(A) shows the information acquisition unit 221 detecting an obstacle region 1000. Edges 1001 and 1002 pass through the obstacle region 1000. In this case, the update unit 228 can update the graph to delete edges 1001 and 1002 that pass through the obstacle region 1000. The update unit 228 may also delete edges based on its determination that a static obstacle exists on an edge. Static obstacles include semi-static obstacles such as desks or chairs that do not move autonomously but can be moved by external forces. Note that an obstacle existing on an edge means that the obstacle exists on the path in real space corresponding to the edge.

[0071] Furthermore, the update unit 228 may update the graph to offset the edge based on its determination that an obstacle exists on the edge. In this case, the update unit 228 can translate the edge so as not to pass through the obstacle. The update unit 228 may also move another edge that was connected to the edge before the offset via a node to connect with the offset edge. For example, Figure 10(B) shows that the information acquisition unit 221 has determined that a dynamic obstacle, a person (P), exists on edge 1011. In this case, the update unit 228 can offset edge 1011 to edge 1012. In Figure 10(B), the nodes at both ends of edge 1011 have also moved, and another edge that was connected to edge 1011 has also moved to connect with these nodes. The update unit 228 may also delete the edge based on its determination that a dynamic obstacle exists on the edge.

[0072] Furthermore, the update unit 228 may update the graph to modify or delete edges in a manner appropriate to the size of the obstacle present on the edge, based on its determination that an obstacle exists on the edge. For example, if the ratio of the length of the portion of the edge passing through the obstacle area to the total length of the edge is greater than a threshold, the update unit 228 can delete this edge. Alternatively, if the ratio of the length of the portion of the edge passing through the obstacle area to the total length of the edge is less than or equal to a threshold, the update unit 228 can offset this edge.

[0073] <Location control at the destination> The control unit 227 may change the method of controlling the movement of the moving body 100 depending on whether the user is located near the destination. In one embodiment, nodes corresponding to the person's destination are set in advance. The graph data may include information indicating whether each node corresponds to the person's destination. The control unit 227 can then change the relative position or orientation of the moving body with respect to the person at the target location, depending on whether the user is located within a predetermined range from the node corresponding to the destination.

[0074] Figure 9(A) shows an example where node 900 does not correspond to a person's destination. In this example, the mobile body 100(R) reaches the target location (T) facing away from the user (U), leading the user (U). Figure 9(B) shows an example where node 900 corresponds to a person's destination. In this example, the mobile body 100(R) reaches the target location (T) facing the user (U). In this configuration, it becomes easier for the user to operate the mobile body 100 at the destination. As mentioned above, luggage can be stored on the front of the mobile body 100. With this configuration, it becomes easier for the user to access the front of the mobile body 100 to store luggage.

[0075] The control unit 227 may further change the relative position or orientation of the moving body to the person at the target location, depending on the user's speed. For example, the control unit 227 can estimate that the user is about to stop at a node if the user is within a predetermined range from the node corresponding to the destination and the user's speed is below a threshold. In this case, the control unit 227 can control the movement of the moving body 100 (R) so that it reaches the target location (T) facing the user (U).

[0076] <Other Embodiments> Different graphs may be used depending on the environment. For example, a person's travel route may change depending on the time of day or weather. Therefore, different graphs may be used depending on the time of day or weather.

[0077] <Summary of Embodiments> (Item 1) A system (201) that controls a moving object to move in accordance with the movement of a person, Information acquisition means (221) that acquires information indicating the position and direction of movement of a person, A graph acquisition means (222) that acquires a graph showing a person's movement path, which is composed of nodes and edges, A determination means (223) that determines the first edge of the graph corresponding to the position (U) of the person based on the position of the person, A determination means (226) for determining the target position (T) of the moving body (100) based on the first edge or one or more edges located in the direction of movement of the person relative to the first edge, A system characterized by comprising the following features. This system allows for stable control of the movement of a moving object based on the movement of the object being followed. (Item 2) The system according to item 1, characterized in that the determination means (226) determines the target position (T) of the moving body (100) based on the position corresponding to a point on the first edge or one or more edges located in the direction of movement of the person relative to the first edge. This system allows for more precise control of the movement of a moving object. (Item 3) The system according to any one of items 1 to 2, wherein the determination means determines the target position (T) of the moving body (100) based on a plurality of edges (701, 702) connected to the first edge at a first node (710) located in the direction of movement of the person, among the nodes to which the first edge is connected. This system allows for more precise control of the movement of a moving object. (Item 4) The system according to item 3, characterized in that the determination means (226) evaluates the probability that the user will travel along each of the plurality of edges (701, 702) connected to the first edge, and determines the position on the path obtained by weighting and combining the paths corresponding to each of the plurality of edges (701, 702) according to the probability as the target position (T) of the moving body (100). This system allows for more precise control of the movement of a moving object. (Item 5) The system according to any one of items 1 to 4, characterized in that the determination means (226) determines the position of the moving target so that the moving body (100) leads the person or follows the person. This system allows for more accurate guidance of individuals or the ability to follow them. (Item 6) The system according to any one of items 1 to 5, further comprising an update means (227) that updates the graph to delete the edges (1001, 1002) based on the determination that an obstacle exists on the edge. This system allows for stable control of the movement of a moving object even when obstacles are present. (Item 7) The system according to any one of items 1 to 6, further comprising an updating means (227) that updates the graph to offset the edge (1011) based on the determination that an obstacle exists on the edge. This system allows for stable control of the movement of a moving object even when obstacles are present. (Item 8) The system according to any one of items 1 to 7, further comprising an updating means (227) that updates the graph in a manner corresponding to the size of the obstacle present on the edge, based on the determination that an obstacle is present on the edge. This system allows for stable control of the movement of a moving object even when obstacles are present. (Item 9) The system according to item 8, wherein the updating means updates the graph to delete the edge if the ratio of the length of the portion of the edge that passes through the area of ​​the obstacle to the length of the edge is greater than a threshold. This system allows for precise control of the movement of a moving object according to the size of the obstacle. (Item 10) The system according to item 9, wherein the updating means updates the graph to offset the edge when the ratio of the length of the portion of the edge that passes through the area of ​​the obstacle to the length of the edge is less than or equal to the threshold. This system allows for precise control of the movement of a moving object according to the size of the obstacle. (Item 11) The information acquisition means (221) estimates the position of the moving body (100) based on the output from the sensor provided by the moving body (100), and generates an environmental map in a coordinate system based on the moving body (100). The system according to any one of items 1 to 8, characterized in that the determination means (226) determines the target position (T) of the moving object (100) based on the graph which has been transformed into a coordinate system based on the moving object (100) and superimposed on the environment map. This system allows for stable control of the movement of a mobile object in accordance with its surrounding environment. (Item 12) The system according to item 11, wherein the information acquisition means (221) is characterized by deforming the graph and superimposing it onto the environment map so as to mitigate changes in the transformation result of the graph due to changes in the coordinate system based on the moving body (100). This system allows for stable control of the movement of a mobile object in accordance with its surrounding environment. (Item 13) A mobile body (100) comprising a system (201) described in any one of items 1 to 12, and a sensor (206), The information acquisition means (221) of the system (201) estimates the position of a person based on the output from the sensor (206), The aforementioned mobile body (100) Control means (227) for controlling the movement of the moving body (100) so that it moves toward the target moving position (T), A driving means (204) moves the mobile body (100) in accordance with the control by the control means (227), A mobile device equipped with these features. This mobile object can move stably based on the movement of the object it is following. (Item 14) The mobile body according to item 13, characterized in that the control means (227) controls the mobile body (100) so that it moves along the path indicated by the edge when it determines that an obstacle exists between the current position of the mobile body (100) and the target position (T) of the mobile body (100). This mobile device can move stably even when obstacles are present. (Item 15) The mobile body according to any one of items 13 to 14, characterized in that the control means (227) controls the mobile body (100) so that it moves in a straight line from its current position to the target position (T) when it determines that there are no obstacles between the current position of the mobile body (100) and the target position (T) of the mobile body (100). This mobile object can move stably based on the movement of the object it is following. (Item 16) A node (900) corresponding to the person's destination is pre-configured. The mobile body according to any one of items 13 to 15, characterized in that the control means (227) changes the relative position or orientation of the mobile body (100) with respect to the person at the target location (T), depending on whether the person is located within a predetermined range from the node (900) corresponding to the destination. This mobile device can be easily operated by the user at its destination. (Item 17) A method for controlling a moving object to move in accordance with the movement of a person, performed by an information processing device (201), The process of acquiring information indicating the position and direction of movement of a person (S301), The process involves obtaining a graph (S302) that consists of nodes and edges and shows the movement path of a person, A step (S303) to determine the first edge of the graph corresponding to the position of the person based on the position of the person, A step (S305) to determine the target position of the moving body based on the first edge or one or more edges located in the direction of movement of the person relative to the first edge, A method characterized by including the following. This method allows for stable control of the movement of a moving object based on the movement of the object being followed. (Item 18) A program that causes a computer to function as one of the systems described in items 1 through 12. This program allows for stable control of the movement of a moving object based on the movement of the object being followed.

[0078] The invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of the gist of the invention. [Explanation of Symbols]

[0079] 100: Mobile unit, 201: Control unit, 221: Information acquisition unit, 222: Graph acquisition unit, 223: Judgment unit, 224: Selection unit, 225: Prediction unit, 226: Decision unit, 227: Control unit, 228: Update unit

Claims

1. A system for controlling a moving object so that it moves in accordance with the movement of a person, Information acquisition means for acquiring information indicating a person's position and direction of movement, A graph acquisition means that acquires a graph showing a person's movement path, which is composed of nodes and edges, A determination means for determining the first edge of the graph corresponding to the position of the person, based on the position of the person, A determination means for determining the target position of the moving body based on the first edge or one or more edges located in the direction of movement of the person relative to the first edge, A system characterized by comprising the following features.

2. The system according to claim 1, characterized in that the determination means determines the target position of the moving body based on the first edge or a position corresponding to a point on one or more edges located in the direction of movement of the person relative to the first edge.

3. The system according to claim 1, characterized in that the determination means determines the target position of the moving body based on a plurality of edges connected to the first edge at the first node located in the direction of movement of the person, among the nodes to which the first edge is connected.

4. The system according to claim 3, characterized in that the determination means evaluates the probability that a user will travel along a path corresponding to each of the plurality of edges connected to the first edge, and determines the position on the path obtained by weighting and combining the paths corresponding to each of the plurality of edges according to the probability as the target position of the moving body.

5. The system according to claim 1, characterized in that the determination means determines the position of the moving target so that the moving body leads the person or follows the person.

6. The system according to claim 1, further comprising an update means for updating the graph to delete an edge based on the determination that an obstacle exists on the edge.

7. The system according to claim 1, further comprising an updating means for updating the graph to offset the edge based on the determination that an obstacle exists on the edge.

8. The system according to claim 1, further comprising an updating means that updates the graph in a manner corresponding to the size of the obstacle present on the edge, based on the determination that an obstacle is present on the edge.

9. The system according to claim 8, wherein the updating means updates the graph to delete the edge if the ratio of the length of the portion of the edge that passes through the area of ​​the obstacle to the length of the edge is greater than a threshold.

10. The system according to claim 9, wherein the updating means updates the graph to offset the edge when the ratio of the length of the portion of the edge that passes through the area of ​​the obstacle to the length of the edge is less than or equal to the threshold.

11. The information acquisition means estimates the position of the moving object based on the output from the sensors provided by the moving object, and generates an environmental map in a coordinate system based on the moving object. The system according to claim 1, characterized in that the determination means determines the target position of the moving object based on the graph which has been transformed into a coordinate system based on the moving object and superimposed on the environment map.

12. The system according to claim 11, wherein the information acquisition means is characterized by deforming the graph and superimposing it onto the environment map so as to mitigate changes in the transformation result of the graph due to changes in the coordinate system based on the moving object.

13. A mobile body comprising the system according to any one of claims 1 to 12 and a sensor, The information acquisition means provided in the system estimates the position of a person based on the output from the sensor, The aforementioned moving body is Control means for controlling the movement of the moving body so that it moves toward the aforementioned target position, A driving means for moving the movable body in accordance with the control by the control means, A mobile device equipped with [the necessary components].

14. The mobile body according to claim 13, characterized in that the control means controls the mobile body so that it moves along the path indicated by the edge in response to the determination that an obstacle exists between the current position of the mobile body and the target position of the mobile body.

15. The mobile body according to claim 14, characterized in that the control means controls the mobile body so that it moves in a straight line from its current position to the target position, in response to the determination that there are no obstacles between the current position of the mobile body and the target position of the mobile body.

16. A node corresponding to the person's destination is pre-configured. The moving body according to claim 13, characterized in that the control means changes the relative position or orientation of the moving body with respect to the person at the moving target position, depending on whether the person is located within a predetermined range from the node corresponding to the destination.

17. A method for controlling a moving object to move in accordance with the movement of a person, performed by an information processing device, A process of acquiring information indicating the position and direction of movement of a person, The process involves obtaining a graph consisting of nodes and edges that shows the movement path of a person, A step of determining a first edge of the graph corresponding to the position of the person based on the position of the person, A step of determining the target position of the moving body based on the first edge or one or more edges located in the direction of movement of the person relative to the first edge, A method characterized by including the following.

18. A program for causing a computer to function as the system described in any one of claims 1 to 12.

Images