System and method for controlling moving body, moving body, storage medium, and program product

Abstract

The invention relates to a system and method for controlling a moving body, a moving body, a storage medium, and a program product. The present invention stably controls the operation of a moving body that moves on the basis of the operation of a following object. Information indicating a position and a moving direction of a person is acquired. A graph which is composed of nodes and edges and represents a movement path of a person is acquired. A first edge of the graph corresponding to the position of the person is determined based on the position of the person. The moving target position of the moving body is determined on the basis of the first side or one or more sides located in the moving direction of the person with respect to the first side.

Description

Technical Field

[0001] This invention relates to systems, methods, mobile bodies, storage media, and program products for controlling mobile bodies, and particularly to mobile bodies that move according to the movement of a person. Background Technology

[0002] The development of autonomous mobile robots is gaining momentum. For example, Patent Document 1 discloses an autonomous mobile robot that moves by following objects such as people. In Patent Document 1, the target is set to the side or behind the current position of the object being followed, thereby achieving a more natural following.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2021-77088 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] As in Patent Document 1, even methods that control a moving body to be positioned to the side or behind the current position of the object being followed and facing the target can sometimes result in unstable movement of the moving body due to the movement of the object being followed. Furthermore, in the method of Patent Document 1, it is difficult to use methods other than following to make the moving body move according to the movement of the person (e.g., to make the moving body guide the person).

[0008] The purpose of this invention is to provide stable control over the movement of a moving body that moves based on the movement of a following object.

[0009] Solution for solving the problem

[0010] An embodiment of the present invention relates to a system for controlling a mobile body to move according to the movement of a person. The system includes: an information acquisition unit that acquires information representing the position and direction of movement of the person; a graph acquisition unit that acquires a graph consisting of nodes and edges representing the movement path of the person; a determination unit that determines a first edge of the graph corresponding to the position of the person based on the position of the person; and a decision unit that determines a target position for the movement of the mobile body based on the first edge or one or more edges located relative to the first edge in the direction of movement of the person.

[0011] An embodiment of the present invention relates to a mobile body comprising: a system according to the embodiment; and a sensor, wherein the information acquisition component of the system estimates the position of a person based on the output from the sensor, the mobile body comprising: a control component that controls the movement of the mobile body to move toward the target position; and a drive component that moves the mobile body according to the control of the control component.

[0012] An embodiment of the present invention relates to a method for controlling a mobile body to move according to the movement of a person. The method includes: a step of acquiring information representing the position and direction of movement of the person; a step of acquiring a graph composed of nodes and edges representing the movement path of the person; a step of determining a first edge of the graph corresponding to the position of the person based on the position of the person; and a step of determining a target position of movement of the mobile body based on the first edge or one or more edges located in the direction of movement of the person relative to the first edge.

[0013] One embodiment of the present invention relates to a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the steps of the method of one embodiment are implemented.

[0014] One embodiment of the present invention relates to a computer program product that includes a computer program, which, when executed by a processor, implements the steps of the method of one embodiment.

[0015] The effects of the invention

[0016] It can stably control the movement of a moving body that moves based on the action of the object it is following. Attached Figure Description

[0017] Figure 1 This is a diagram showing the appearance of a mobile body according to one embodiment.

[0018] Figure 2 This is a diagram illustrating an example of the functional structure of a mobile body according to one embodiment.

[0019] Figure 3 This is a flowchart of a processing method according to one embodiment.

[0020] Figure 4 This is a diagram illustrating how to create a graph.

[0021] Figure 5 This is a diagram illustrating the method for predicting users' mobile destinations.

[0022] Figure 6 This is a diagram illustrating the method for determining the location of a moving target.

[0023] Figure 7 This is a diagram illustrating the method for determining the location of a moving target.

[0024] Figure 8 This is a diagram illustrating the method for determining the location of a moving target.

[0025] Figure 9 This is a diagram illustrating the method for determining the location of a moving target.

[0026] Figure 10 This is a diagram illustrating the method for updating a graph.

[0027] Explanation of reference numerals in the attached figures

[0028] 100: Moving body; 201: Control component (sometimes also called system); 204: Driving component (sometimes also called drive component); 206: Detection component (sometimes also called sensor); 221: Information acquisition unit (sometimes also called information acquisition component); 222: Image acquisition unit (sometimes also called image acquisition component); 223: Decision unit (sometimes also called decision component); 224: Selection unit; 225: Prediction unit; 226: Decision unit (sometimes also called decision component); 227: Control unit (sometimes also called control component); 228: Update unit (sometimes also called update component). Detailed Implementation

[0029] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Furthermore, the following embodiments are not intended to limit the scope of the claims, and the present invention does not require a combination of all the features described in the embodiments. Alternatively, two or more features from the plurality of features described in the embodiments may be arbitrarily combined. Additionally, the same or identical structures will be given the same reference numerals, and repeated descriptions will be omitted.

[0030] <Structure of a Moving Body>

[0031] Reference Figure 1 This section describes an example of the external structure of a movable body 100 according to one embodiment. Figure 1In the diagram, the arrow symbol X indicates the forward / backward direction of the mobile body 100. F indicates forward, and R indicates backward. The arrow symbols Y and Z indicate the width (left-right) and height (up-down) directions of the mobile body 100, respectively. The mobile body 100 is capable of autonomous movement. For example, the mobile body 100 is equipped with a battery and primarily moves using the power of an electric motor. The mobile body 100 can also be used in entertainment facilities, large commercial facilities, airports, parks, sidewalks, parking lots, and other similar locations. The mobile body 100 can be a vehicle that moves on the ground using wheels, an aircraft that moves in the air (e.g., a drone), or a robot that moves on the ground using two or more legs. Movement using wheels is also called driving. Movement in the air is also called flying. Movement using two or more legs is also called walking. The following description focuses on the case where the mobile body 100 is a vehicle, but the same description applies to other types of mobile bodies.

[0032] The mobile body 100 can move according to the movement of a person (e.g., the user of the mobile body 100, hereinafter referred to as the user). For example, the mobile body 100 can move in a manner that travels alongside the user. Traveling alongside the user means that the mobile body 100 moves based on the user's leading movement. For example, the mobile body 100 can move side by side with the user in a manner that guides the user or move in a manner that follows the user. In the following examples, a configuration is described in which a person is not riding on the mobile body 100, but it is also possible for a person different from the user to ride on the mobile body 100.

[0033] The mobile body 100 is equipped with a driving component 204. Figure 2 The vehicle includes, for example, a pair of front wheels 101 and a rear wheel 102. The driving component 204 can also be in the form of a four-wheeled vehicle or a two-wheeled vehicle, or other forms. In addition to traveling with the user instead of the mobile body 100, the mobile body 100 can also autonomously move to a pre-set destination.

[0034] Mobile body 100 has a housing 110 for accommodating luggage. A closable cover is provided on the front surface 111 of the housing for accommodating luggage. This cover may have a locking mechanism. The locking mechanism is controlled by mobile body 100. For example, mobile body 100 may unlock the cover upon successful user authentication. Alternatively, mobile body 100 may be able to accommodate luggage in other ways.

[0035] A touchscreen 120 is disposed on the upper surface 112 of the housing. Users can use the touchscreen 120 to change settings of the moving body 100, or to check facility-related information. A detection component 206 is housed inside the sensor housing 130. Figure 2 The detection component 206 includes sensors such as cameras. The detection component 206 identifies user and other object identifiers in the environment surrounding the moving body 100 through the front surface 131, sides, and back of the sensor housing 130.

[0036] <Example of the functional structure of a moving entity>

[0037] Reference Figure 2 This describes an example of the functional structure of the mobile body 100. The mobile body 100 includes... Figure 2 The structural elements shown. The moving body 100 may include... Figure 2 Structural elements not shown may also be omitted. Figure 2 A portion of the structural elements shown.

[0038] The mobile body 100 includes a driving component 204 and is an electrically powered autonomous mobile body primarily powered by a battery 205. The battery 205 is, for example, a secondary battery such as a lithium-ion battery. Using the power supplied from the battery 205, the driving component 204 enables the mobile body 100 to move autonomously.

[0039] The driving assembly 204 uses an electric motor as a drive source to change the rotational speed of a pair of front wheels 101, thereby accelerating and decelerating the moving body 100. The driving assembly 204 may also include a braking mechanism for decelerating the moving body 100. The driving assembly 204 causes the moving body 100 to rotate by creating a difference in rotational speed between the pair of front wheels 101. The driving assembly 204 can detect and output physical quantities representing the motion of the moving body 100, such as its travel speed, acceleration, rotation angle, angular velocity of the housing 110 of the moving body 100, and angular acceleration.

[0040] The mobile object 100 includes a detection component 206 comprising one or more sensors. The detection component 206 generates data for identifying object markers (objects and people in the surrounding environment) of the mobile object 100. The detection component 206 includes sensors such as a camera, radar, lidar (Light Detection and Ranging), and ultrasonic sensors, which have a detection range around the mobile object 100, to output sensor information. The camera can be a fisheye lens or a structure capable of stereo photography. Furthermore, the detection component 206 includes a GNSS (Global Navigation Satellite System) sensor to receive GNSS signals and detect the current position of the mobile object 100. The detection component 206 can also use wireless LAN (Local Area Network) or Bluetooth signals to detect the current position. The camera can also be an RGB camera and may have depth measurement capabilities. For example, the moving body 100 may have RGB cameras with depth measurement function on the front and rear sides of the sensor box 130, and RGB cameras without depth measurement function on the right and left sides of the sensor box 130.

[0041] The mobile unit 100 includes a control component (ECU (Electronic Control Unit)) 201. The control component 201 functions as a control device for the mobile unit 100. The control component 201 includes one or more processors 202, such as a CPU (Central Processing Unit), and a memory 203, such as a semiconductor memory. Therefore, the control component 201 can also be referred to as an information processing device or a computer. The memory 203 stores programs executed by the processor 202, data used by the processor 202 during processing, etc. Alternatively, the processor 202 and the memory 203 can be configured as multiple groups according to the functions of the mobile unit 100 and capable of communicating with each other.

[0042] The control component 201 acquires physical quantities representing motion output from the driving component 204, detection results from the detection component 206, input information from the touchscreen 120, and sound information input from the sound input device 207, and performs corresponding processing. For example, the control component 201 controls the motor of the driving component 204, controls the display of the touchscreen 120, and notifies the surrounding environment by sound.

[0043] The sound input device 207 picks up ambient sounds from the surrounding environment of the mobile body 100. The control component 201 recognizes the input sound and can perform corresponding processing. The storage device 208 is a non-volatile, high-capacity storage device that stores map information, including information such as the routes the mobile body 100 can travel, restricted areas, landmarks, and shops. The storage device 208 can also store programs executed by the processor 202 and data used by the processor 202 during processing. The communication device 209 is, for example, a communication device that can connect to an external network via wireless communication such as fifth-generation mobile communication or wireless LAN.

[0044] The prompting device 210 displays a user interface screen on the touchscreen 120 or outputs voice prompts to the surrounding environment of the mobile body 100 via a microphone. The input device 211 may include, for example, a touch panel integrated with the touchscreen 120. The input device 211 accepts operation input from the user via the touch panel.

[0045] The control component 201 corresponds to a system or information processing device that predicts the movement of a person, or a system or information processing device that controls a mobile body to move according to the movement of a person, according to one embodiment. The processor 202 of the control component 201 executes a program stored in the memory 203 or storage device 208, thereby implementing the functions of the information acquisition unit 221, image acquisition unit 222, determination unit 223, selection unit 224, prediction unit 225, decision unit 226, control unit 227, and update unit 228. Alternatively, at least some of these functions may be implemented by an information processing device different from the mobile body 100. That is, an information processing device such as a computer equipped with a processor and memory executes a program stored in memory, thereby enabling the processing of these units as needed and based on information sent from the mobile body 100. Alternatively, such an information processing device may remotely control the mobile body 100 based on the processing results. Furthermore, at least some of these functions may be implemented via cloud services. One embodiment of the system that predicts the movement of a person or controls a moving body to move according to the movement of a person may also be such an information processing device or cloud system.

[0046] Information acquisition unit 221 acquires information indicating the user's location and direction of movement. Information acquisition unit 221 is capable of acquiring information indicating the user's location and direction of movement. For example, information acquisition unit 221 can acquire information indicating the user's current geographic location. Furthermore, information acquisition unit 221 can determine the user's direction of movement based on changes in the user's current geographic location. In other embodiments, information acquisition unit 221 may acquire information indicating the user's current posture. Information acquisition unit 221 can determine the user's direction of movement based on the user's current posture. Information acquisition unit 221 can estimate the person's location based on the output from sensors. For example, information acquisition unit 221 can acquire this information based on the detection results of detection component 206 (e.g., images captured by a camera device).

[0047] The graph acquisition unit 222 acquires a graph representing the user's movement path. The graph consists of nodes and edges. Edges indicate the character's movement path. Nodes connect two edges. Furthermore, the graph can be based on a world coordinate system. Alternatively, it can be based on other coordinate systems, such as a coordinate system based on the moving body 100. In either case, the nodes of the graph are associated with their positions in real space. Additionally, the edges of the graph are associated with their paths in real space, and the points on the edges are associated with their positions on the paths. Moreover, the edges can be straight or curved.

[0048] Figure 4 An example of a diagram is shown. Figure 4 In this diagram, nodes are represented by circles, edges by solid lines, and structures such as buildings and roads by dashed lines. In one embodiment, the diagram is created based on environmental features such as roads, buildings, and obstacles. For example, edges can be set along the outer edge of a character's movable area. Additionally, edges can be set along the outer edge of stationary, immovable obstacles. Figure 4 In the middle, edges 401 to 403 are set along the outer edge of the building.

[0049] Nodes can also be placed at the intersection of two or more roads. Additionally, nodes can be placed at locations where characters frequently pass through. For example, nodes can be placed at building entrances or at the top or bottom of stairs. Furthermore, nodes can be placed at locations where characters frequently stop. For example, nodes can be placed at the location of vending machines. Figure 4 In this context, node 411 is set at the building's entrance / exit.

[0050] There are no particular limitations on the method of map creation. For example, users can manually create maps based on environmental features. Alternatively, information processing devices can automatically create maps based on environmental features. On the other hand, maps can also be created based on pedestrian movement data. For example, data representing the movement trajectories of multiple pedestrians can be extracted from motion images. Users can also manually create maps while referring to the movement trajectories of multiple pedestrians. Alternatively, information processing devices can automatically create maps based on the movement trajectories of multiple pedestrians.

[0051] Storage device 208 is capable of storing such graph data. In this case, graph acquisition unit 222 can acquire the graph from storage device 208. Alternatively, graph acquisition unit 222 can also acquire the graph from an external information processing device via communication device 209.

[0052] The determination unit 223 determines the first edge of the graph corresponding to the user's location based on the user's location. For example, the determination unit 223 can determine the edge closest to the user's current geographical location as the first edge. Figure 5 In the example, the edge closest to the user's current position, represented by U, is edge 501. Therefore, the determination unit 223 can determine edge 501 as the first edge. The determination unit 223 can, for example, determine the first edge based on the user's position in the world coordinate system and the graph in the world coordinate system.

[0053] Furthermore, the determination unit 223 may determine the first edge based on the user's movement direction. Alternatively, the determination unit 223 may determine the first edge from among multiple edges based on both the distance between each edge of the graph and the user's position, and the similarity between the direction of each edge and the user's movement direction. For example, the determination unit 223 may assign a first score to the edge based on the distance between the user's position and the edge. Additionally, the determination unit 223 may assign a second score to the edge based on the similarity between the user's movement direction and the edge's direction. Moreover, the determination unit 223 may select the first edge based on both the first and second scores. For example, the first edge selected by the determination unit 223 may be the edge with the highest overall score. An overall score can be calculated based on the first and second scores.

[0054] The prediction unit 225 is able to predict the location corresponding to the point on the first side as the user's mobile destination. Figure 5In this example, the prediction unit 225 can predict the location corresponding to a point on the first edge, i.e., edge 501, as the user's destination. Here, the prediction unit 225 can predict the location corresponding to a point that has moved along edge 501 in the user's movement direction relative to the point corresponding to the user's location on edge 501. The point on the first edge corresponding to the user's location may be the point on the first edge closest to the user's location. Furthermore, the prediction unit 225 can determine the user's movement direction along the first edge based on the user's movement direction indicated by the information acquired by the information acquisition unit 221. The prediction unit 225 can determine the user's movement direction along the first edge by ensuring that the dot product of the vector of the user's movement direction along the first edge at the point corresponding to the user's location on the first edge and the vector of the user's movement direction indicated by the information acquired by the information acquisition unit 221 is positive. Moreover, Figure 5 In the example, user (U) is moving in the lower left direction as indicated by the arrow symbol. In this case, prediction unit 225 can determine that the user's movement direction along edge 501 is from node 510 to node 511.

[0055] Furthermore, the prediction unit 225 can predict the location corresponding to a point on one or more edges located in the user's movement direction relative to the first edge as the user's destination. The edges located in the user's movement direction relative to the first edge can include a second edge, which is an edge connected to the first edge via a first node located ahead of the user's movement direction along the first edge, one of the two nodes connected to the first edge. Additionally, the edges located in the user's movement direction relative to the first edge can also include one or more edges that are continuous with the second edge via one or more nodes. For example, assuming the first edge is edge 503, and the user is moving from node 511 towards node 512. In this case, the selection unit 224 can select node 512, located in the user's movement direction, among the nodes connected to the first edge (edge ​​503), as the first node. Furthermore, the selection unit 224 can select edge 505, which is connected to edge 503 at the first node (node ​​512), as the second edge. In this case, the prediction unit 225 predicts the location corresponding to a point on the second edge (edge ​​505) as the user's destination.

[0056] on the other hand, Figure 5In this example, the selection unit 224 can select node 511, which is located in the user's movement direction, from the nodes connected by the first edge, i.e., edge 501. Here, the first node, i.e., node 511, is connected to multiple edges, i.e., edges 502, 503, and 504. In this case, the selection unit 224 can select a second edge from the multiple edges based on the direction of each of the multiple edges and the direction of the first edge or the user's movement direction. The selection unit 224 can select an edge that extends in a direction closer to the direction of the first edge or the user's movement direction as the second edge. Figure 5 In this example, the selection unit 224 can select edge 502 as the second edge. In this case, the prediction unit 225 can predict the location corresponding to the point on the second edge, i.e., edge 502, as the user's destination. Moreover, the direction of the edge can be the direction of the edge connecting to node 511.

[0057] As a specific example, the selection unit 224 can select a second side from a plurality of sides based on the angle between the first side and each of the plurality of sides. Here, the angle between the first side and the selected second side can be the largest angle among the angles between the first side and each of the plurality of sides.

[0058] Furthermore, the selection unit 224 can also select a second edge based on the distance between the user's position and each edge. For example, the selection unit 224 can assign a third score to each of the multiple edges connected to the first edge at the first node based on the similarity between the edge's direction and the direction of the first edge or the user's movement direction. The higher the similarity, the higher the third score can be. In addition, the selection unit 224 can assign a fourth score to each of the multiple edges based on the distance between the edge and the user's position. The shorter the distance, the higher the fourth score can be. Moreover, the selection unit 224 can select a second edge based on both the third and fourth scores. For example, the second edge selected by the selection unit 224 can be the edge with the highest overall score. An overall score can be calculated based on both the third and fourth scores.

[0059] The prediction unit 225 can predict the user's destination location as a point along one or more edges located in the user's direction of movement, at a predetermined distance from the point on the first edge closest to the user's current location. This predetermined distance can be a fixed value. Alternatively, it can be a distance corresponding to the user's movement speed.

[0060] The decision unit 226 determines the target location of the mobile body 100. The target location refers to the location of the destination of the mobile body 100. The following describes the operation of the decision unit 226 in determining the target location by guiding the user with the mobile body 100.

[0061] In one embodiment, the decision unit 226 can determine the location of the moving target based on the user's predicted moving destination by the prediction unit 225. Figure 6 An example is shown illustrating the method for determining the location of a moving target. Figure 6 The text shows the user's current location (U) and direction of movement (arrow symbol). Figure 6 The current position (R) of the moving body 100 is also shown. Figure 6 The target position (T) of the moving body 100 and the direction of movement (arrow symbol) from the current position (R) are also shown.

[0062] In this embodiment, the decision unit 226 determines the target position of movement by guiding the user with the moving body 100, thereby controlling the movement of the moving body 100. In one embodiment, the decision unit 226 sets the target position (T) of the moving body 100 in front of the user's destination. Alternatively, the decision unit 226 sets the target position (T) of the moving body 100 based on the predicted destination of the user's movement. For example, the target position (T) may be a position away from the predicted destination of the user's movement along an edge. Alternatively, the target position (T) may be a position offset from the predicted destination of the user's movement by other methods. For example, the decision unit 226 may set the target position (T) by guiding the moving body 100 diagonally in front of the user. In this case, the target position of the moving body is determined based on the position corresponding to a point on one or more edges in the user's movement direction (i.e., the predicted destination of the user's movement). For example, the decision unit 226 can set the movement target position (T) in front of the user's movement destination after a predetermined time, as predicted by the prediction unit 225, so that the mobile body 100 moves to that location after the predetermined time.

[0063] On the other hand, in other embodiments, the decision unit 226 may determine the target location of the movement by having the moving body 100 follow the user. In this case, the decision unit 226 can set the target location (T) of the moving body 100 behind the user's destination.

[0064] Furthermore, it is not necessary to use the predicted user's destination to determine the target position of the mobile body 100. The decision unit 226 can determine the target position of the mobile body 100 based on one or more edges located in the user's movement direction, including the first edge determined by the decision unit 223. For example, the decision unit 226 can determine the target position of the mobile body 100 based on the position corresponding to a point on one or more edges located in the user's movement direction. Figure 5In this example, the decision unit 226 can set the target position of the moving body 100 at a location corresponding to a point on the first side (side 501) or the second side (side 502). The target position of the moving body 100 can be a location corresponding to a point that is a point located a predetermined distance away from the point on the first side closest to the user's current position along one or more sides located in the user's movement direction. This predetermined distance can be, for example, a distance corresponding to the user's movement speed. Alternatively, the decision unit 226 can set the target position of the moving body 100 at a position offset from the location corresponding to the point on side 501 or side 502. With this structure, the moving body 100 can guide the user.

[0065] on the other hand, Figure 7 In example (A), the first node 710, which is located in the user's movement direction and is connected to the first edge corresponding to the user's position (U), is connected to multiple edges, namely edges 701 and 702. In this situation, the user may move in the direction of edge 701 or in the direction of edge 702. Therefore, the decision unit 226 can determine the movement target position of the moving body 100 based on the multiple edges (e.g., edges 701 and 702) connected to the first node, namely node 710.

[0066] For example, the decision unit 226 can evaluate each of the multiple edges connected to the first edge. The decision unit 226 can evaluate each edge based on the relationship between the user's movement direction and the edge's direction. For example, the decision unit 226 can evaluate an edge by assigning it a higher rating the closer its movement direction is to the edge's direction. Alternatively, the decision unit 226 can evaluate each edge based on the distance between the user's current position and the edge. Alternatively, the edge evaluation can represent the probability that the user will traverse the path corresponding to the edge. Figure 7 In example (A), the decision unit 226 evaluates the probability of the user traversing the paths corresponding to edges 701 and 702 to 0.5 respectively. Furthermore, the decision unit 226 can determine the position on the path obtained by weighted synthesis of paths corresponding to multiple edges based on the evaluation of each edge as the target position for the moving body. Figure 7 In (A), the path obtained by weighted synthesis of the paths corresponding to edges 701 and 702 based on the evaluation (both are 0.5) is shown by dashed lines. Furthermore, the decision unit 226 sets the target position (T) of the movement on the dashed lines.

[0067] Figure 7 (B) shows Figure 7 The position of the user (U), the position of the moving body 100 (R), and the position of the moving target (T) at the time following (A). Figure 7In example (B), the decision unit 226 evaluates the probability of the user traversing the paths corresponding to edges 701 and 702 as 0.65 and 0.35, respectively. Furthermore, the decision unit 226 sets the target position (T) on the path (dashed line) obtained by weighted synthesis of the paths corresponding to edges 701 and 702 based on the evaluation.

[0068] Figure 7 (C) shows Figure 7 The position of the user (U), the position of the moving body 100 (R), and the position of the moving target (T) at the time following (B). Figure 7 In example (C), the decision unit 226 evaluates the probability of the user traversing the paths corresponding to edge 701 and edge 702 as 1.0 and 0.0, respectively. Therefore, the decision unit 226 sets the target position (T) on the path corresponding to edge 701.

[0069] The control unit 227 controls the movement of the mobile body 100 in a manner that moves towards the target position determined by the decision unit 226. For example, the control unit 227 can generate a track from the current position of the mobile body 100 to the target position. Furthermore, the control unit 227 can generate the track to avoid obstacles. The control unit 227 then moves the mobile body 100 along this track. Depending on the current attitude and speed of the mobile body 100, sometimes the track passes the target position, and sometimes it does not. The travel assembly 204 can move the mobile body 100 according to the control of the control unit 227. For example, the control unit 227 can supply a control signal to the travel assembly 204 to move the mobile body 100.

[0070] Here, the control unit 227 can control the movement of the mobile body 100 so that the mobile body 100 can directly move to the target location. Figure 8 In the example shown in (A), the control unit 227 controls the moving body 100 to move in a straight line from its current position (R) to its target position (T). On the other hand, Figure 8 In the example shown in (B), there is an area 800 where there is an obstacle between the current position (R) of the moving body 100 and the target position (T). Therefore, the control unit 227 can control the moving body 100 to move along the path shown in the side, based on the determination that there is an obstacle between the current position of the moving body 100 and the target position. Figure 8In the example shown in (B), the control unit 227 sets a temporary target position (T') on the path shown by the edge. Furthermore, the control unit 227 controls the movement of the mobile body 100 such that it moves towards the temporary target position (T'). On the other hand, the control unit 227 can control the mobile body 100 to move straight from its current position to the target position if it determines that there are no obstacles between the current position and the target position. Thus, when it is determined that there are no obstacles between the current position and the target position during the movement of the mobile body 100 on the path shown by the edge, the mobile body 100 can begin to move straight towards the target position.

[0071] <Control Methods for Moving Objects>

[0072] Reference Figure 3 This describes a method by which control component 201 controls mobile body 100. As described above, control component 201 can control mobile body 100 in a manner that moves alongside the user. Alternatively, processor 202 can execute a program stored in memory 203 or storage device 208, thereby performing... Figure 3 The steps of the method. Alternatively, it could be: Figure 3 At least some steps of the method are performed by an application-specific integrated circuit (ASIC). Alternatively, the method may begin by the user instructing the mobile body 100 to move in tandem with the user. Figure 3 The method is as follows. The control component 201 repeatedly (e.g., at 100ms cycles) executes the processes of S301 to S306.

[0073] Execute on the user of mobile body 100 Figure 3 The method. Control component 201 in Figure 3 The method determines the user at the start time and controls the mobile body 100 in a manner that allows the user to travel alongside it. Alternatively, the control component 201 may identify a person who is in front of the mobile body 100 in an image captured by the detection component 206 (e.g., a camera device) and who has previously registered as a user. During execution Figure 3 In this method, control component 201 continuously determines whether a user can be detected. If a user cannot be detected within a predetermined time after the last user was detected, control component 201 can determine that the user has been lost and interrupt the process. Figure 3 This method. Alternatively, control component 201 can restart based on the detection of the user. Figure 3 The method.

[0074] In S301 to S304, the control component 201 predicts the user's mobile destination.

[0075] In step S301, the information acquisition unit 221 acquires information used in subsequent processing. As described above, the information acquisition unit 221 can acquire information indicating the user's position and direction of movement. Alternatively, the information acquisition unit 221 may acquire information about the moving body and environmental information. Alternatively, the information acquisition unit 221 may store at least a portion of the acquired information in the memory 203 or storage device 208 for use in subsequent processing.

[0076] Mobile body information refers to information related to mobile body 100. Mobile body information may also include the current speed of mobile body 100, the current geographical location of mobile body 100, and the current angular velocity of mobile body 100. Control component 201 may also acquire mobile body information based on outputs from driving component 204 and detection results from detection component 206 (e.g., GNSS positioning data, inertial sensor data).

[0077] Environmental information refers to information related to the environment surrounding the mobile body 100. Environmental information may also include the number, type, location, and size of object markers in the environment surrounding the mobile body 100. Object markers may include static object markers and moving object markers. Static object markers may include structures such as walls, railings, pillars, and steps. Static object markers are those that are directly or indirectly fixed to the ground and do not move. Static object markers that may obstruct the movement of the mobile body 100 can be called static obstacles. Moving object markers may include pedestrians, cyclists, autonomous mobile bodies, animals, etc. Moving object markers are those that can move relative to the ground. Moving object markers that may obstruct the movement of the mobile body 100 can be called moving obstacles. The control component 201 may also acquire environmental information based on the detection results of the detection component 206 (e.g., images captured by a camera device).

[0078] In S302, the image acquisition unit 222 acquires the image as described above.

[0079] In S303, the determination unit 223 determines the first edge of the graph corresponding to the user's position as described above. Furthermore, the method for associating the user's position with the graph will be described later.

[0080] In S304, the prediction unit 225 predicts the user's mobile destination as described above. At this time, the selection unit 224 can also select the second option as described above.

[0081] In S305 to S306, the control component 201 performs movement control on the moving body 100.

[0082] In S305, the decision unit 226 determines the target position of the moving body 100 as described above.

[0083] In S306, the control unit 227 controls the movement of the moving body 100 in a manner that moves toward the target position as described above.

[0084] Moreover, control component 201 is executing Figure 3 In this method, the distance between the moving body 100 and moving obstacles around it can be monitored. Furthermore, if the distance is within a threshold (e.g., 1.5m), the control component 201 may perform an action (e.g., a sound broadcast) to request the moving obstacle to give way. Alternatively, if the distance between the moving body 100 and moving obstacles around it is within another threshold (e.g., 1m), the control component 201 may stop the moving body 100.

[0085] Then, the process returns to S301 and repeats the above process.

[0086] <Map Overlay>

[0087] In order to perform motion control based on the graph, the information acquisition unit 221 can associate the position of the moving body 100 and the user's position with the graph. For example, when the graph is based on a world coordinate system, the information acquisition unit 221 can use the position of the moving body 100 and the user's position based on the world coordinate system. The information acquisition unit 221 can also acquire the user's position in a coordinate system based on the moving body 100. In this case, the information acquisition unit 221 transforms the user's position in the coordinate system based on the moving body 100 to the user's position in the world coordinate system based on the position and attitude of the moving body 100 based on the world coordinate system. Conversely, the information acquisition unit 221 can also transform the graph to a coordinate system based on the moving body 100. The attitude of the moving body 100 based on the world coordinate system can be determined based on the camera results from an accelerometer or other sensors.

[0088] The geographical location of an object marker belonging to a user can be determined as follows. For example, the information acquisition unit 221 can acquire observation data obtained by the detection component 206. The detection component 206 observes object markers contained in the environment surrounding the mobile body 100. The observation data may also include images captured by a camera device (camera), which is an example of the detection component 206. Alternatively, or in addition, the observation data may include data observed by a lidar, which is an example of the detection component 206. The data observed by the lidar may also include the direction of the object marker and the distance to the object marker. Furthermore, the information acquisition unit 221 uses the observation data to determine the geographical location of the object marker. The geographical location may also be represented by two-dimensional coordinate values ​​in a horizontal plane.

[0089] For example, the information acquisition unit 221 may perform homography transformation on the image captured by the imaging device (camera) to determine the geographical location. Alternatively, or otherwise, the information acquisition unit 221 may determine the geographical location based on data observed by the lidar (i.e., the direction of the object marker and the distance to the object marker).

[0090] Alternatively, the information acquisition unit 221 can use SLAM (Simultaneous Localization and Mapping) technology to determine the position, orientation, and object marker position of the moving body 100. For example, the information acquisition unit 221 detects feature points based on the camera images from the detection component 206. Furthermore, the information acquisition unit 221 matches the feature points detected in the camera image at time t with the feature points detected in the camera image at time t-Δt. Based on the matching results, the information acquisition unit 221 estimates the amount and direction of movement of the moving body 100 during the period from time t to Δt. Furthermore, based on this estimated amount and direction of movement, the information acquisition unit 221 can estimate the position and orientation of the moving body 100. Finally, the information acquisition unit 221 can determine the object marker position based on the position and orientation of the moving body 100 and the feature point detection results.

[0091] Using the method described above, the information acquisition unit 221 can estimate the position of the mobile body 100 based on the output from the detection components provided by the mobile body 100. Furthermore, the information acquisition unit 221 can generate an environmental map showing the positions of objects such as users and obstacles. This environmental map can also be a map in a coordinate system based on the mobile body 100.

[0092] Here, the decision unit 226 can determine the target position of the moving body 100 based on a map that has been transformed to a coordinate system based on the moving body 100 and superimposed on the environment map. That is, the decision unit 226 can determine the target position of the moving body 100 in the coordinate system based on the moving body 100. For example, the information acquisition unit 221 can transform the map to a coordinate system based on the moving body 100 and superimpose it on the environment map. With this structure, the moving body 100 can be moved according to the position of the user or obstacles shown in the environment map.

[0093] However, when the information acquisition unit 221's estimation of the position of the moving body 100 changes, the transformation parameters from the world coordinate system to the coordinate system based on the moving body 100 change. In this case, the transformation result of the graph to the coordinate system based on the moving body 100 also changes. In one embodiment, the information acquisition unit 221 can deform the graph and overlay it on the map in a way that the change in the graph's transformation result is mitigated along with the change in the coordinate system based on the moving body. For example, when performing coordinate transformation on the graph, the information acquisition unit 221 can apply a filter that reduces the change in the graph's transformation result. With this structure, it is possible to suppress abrupt changes in the moving target position of the moving body 100 based on changes in the position estimation result of the moving body 100.

[0094] <Image Update>

[0095] The update unit 228 can update the graph based on the obstacle detection results. For example, the update unit 228 can update the graph by deleting edges if it is determined that an obstacle exists on an edge. Figure 10 (A) shows the case where the information acquisition unit 221 detects an obstacle in region 1000. Edges 1001 and 1002 pass through the obstacle in region 1000. In this case, the update unit 228 can update the graph by deleting edges 1001 and 1002 that pass through the obstacle in region 1000. Alternatively, the update unit 228 can delete edges based on the case where it is determined that a static obstacle exists on the edge. Static obstacles can include semi-static obstacles such as tables or chairs that move involuntarily but can be moved by external forces. Moreover, the existence of an obstacle on an edge means that the obstacle exists on the path in the real space corresponding to the edge.

[0096] Alternatively, the update unit 228 can update the graph by offsetting the edge based on the determination that an obstacle exists on the edge. In this case, the update unit 228 can translate the edge without passing through the obstacle. Alternatively, the update unit 228 can move other edges that were previously connected to the offset edge via nodes to be connected to the offset edge. For example, Figure 10(B) shows the situation where the obstacle (P) is present on edge 1011. At this time, the update unit 228 can shift edge 1011 towards edge 1012. Figure 10 In (B), the nodes at both ends of edge 1011 also move, so that other edges connected to edge 1011 also move in a way that connects to these nodes. Alternatively, the update unit 228 may delete the edge based on the case where a moving obstacle exists on the edge. Another possibility is, as... Figure 10 As shown in (C), edge 1021 is offset toward edge 1022 in accordance with the position of the user (U).

[0097] Alternatively, the update unit 228 may update the graph by modifying or deleting the edge based on the determination that an obstacle exists on the edge, using a method corresponding to the size of the obstacle present on the edge. For example, if the ratio of the length of the portion of the edge passing through the obstacle to the length of the edge is greater than a threshold, the update unit 228 may delete the edge. Furthermore, for example, if the ratio of the length of the portion of the edge passing through the obstacle to the length of the edge is less than a threshold, the update unit 228 may shift the edge.

[0098] <Location control at the destination>

[0099] Alternatively, the control unit 227 may change the movement control method of the mobile body 100 based on whether the user is near the destination. In one embodiment, nodes corresponding to the user's destination are preset. The data in the diagram may also include information indicating whether each node corresponds to the user's destination. Furthermore, the control unit 227 can change the relative position or direction of the mobile body relative to the user at the target location based on whether the user is within a predetermined range from the node corresponding to the destination.

[0100] Figure 9 (A) illustrates an example where node 900 does not correspond to a person's destination. In this example, the moving body 100 (R) guides the user (U) to the target location (T) while facing away from the user (U). Furthermore, Figure 9 (B) illustrates an example where node 900 corresponds to a person's destination. In this example, the mobile body 100 (R) arrives at the target location (T) facing the user (U). In this configuration, the user can easily operate the mobile body 100 at the destination. As mentioned above, luggage can be accommodated on the front surface of the mobile body 100. With this configuration, the front surface of the mobile body 100 is easily accessible for the user to accommodate luggage.

[0101] Alternatively, the control unit 227 may also change the relative position or direction of the moving body relative to the user at the target location based on the user's movement speed. For example, if the user is within a predetermined range from the node corresponding to the destination and the user's movement speed is less than a threshold, the control unit 227 can estimate that the user wants to stop at that node. In this case, the control unit 227 can control the movement of the moving body 100 (R) so that it reaches the target location (T) facing the user (U).

[0102] <Other Implementation Methods>

[0103] Alternatively, different maps can be used depending on the environment. For example, the movement path of a character can be changed according to the time of day or the weather. Therefore, different maps can also be used depending on the time of day or the weather.

[0104] <Summary of Implementation Methods>

[0105] (Project 1)

[0106] A system 201 for controlling a mobile body to move according to the movement of a person is characterized by comprising: an information acquisition unit 221 for acquiring information representing the position and direction of movement of the person; a graph acquisition unit 222 for acquiring a graph consisting of nodes and edges representing the movement path of the person; a determination unit 223 for determining a first edge of the graph corresponding to the position U of the person based on the position U of the person; and a decision unit 226 for determining the target position T of the mobile body 100 based on the first edge or one or more edges located in the direction of movement of the person relative to the first edge.

[0107] According to this system, it is possible to stably control the movement of a moving body that moves based on the movement of the object it is following.

[0108] (Project 2)

[0109] According to the system described in Project 1, the determining component 226 determines the target position T of the moving body 100 based on the position of a point corresponding to the first side or a point located on one or more sides relative to the first side in the direction of movement of the person.

[0110] This system allows for more reliable control of the movements of moving objects.

[0111] (Project 3)

[0112] The system described in any of Projects 1 to 2 is characterized in that the determining component determines the target position T of the moving body 100 based on a plurality of edges 701, 702 connected to the first edge at a first node 710 located in the direction of movement of the person in the node connected to the first edge.

[0113] This system allows for more reliable control of the movements of moving objects.

[0114] (Project 4)

[0115] According to the system described in Project 3, the determining component 226 evaluates the probability that the user passes through the paths corresponding to the plurality of edges 701 and 702 connected to the first edge, and determines the position on the path obtained by weighting and synthesizing the paths corresponding to the plurality of edges 701 and 702 according to the probability as the moving target position T of the moving body 100.

[0116] This system allows for more reliable control of the movements of moving objects.

[0117] (Project 5)

[0118] The system described in any of items 1 to 4 is characterized in that the determining component 226 determines the position of the moving target, such that the moving body 100 guides or follows the person.

[0119] This system allows for more reliable guidance or following of characters.

[0120] (Project 6)

[0121] The system described in any of items 1 to 5 is characterized by further comprising an updating component 228, which updates the graph by deleting the edges 1001 and 1002 based on the determination that an obstacle exists on the edge.

[0122] According to this system, the movement of a moving object can be stably controlled even in the presence of obstacles.

[0123] (Project 7)

[0124] The system described in any of items 1 to 6 is characterized by further comprising an updating component 228, which updates the graph by offsetting the edge 1011 based on the determination that an obstacle exists on the edge.

[0125] According to this system, the movement of a moving object can be stably controlled even in the presence of obstacles.

[0126] (Project 8)

[0127] The system described in any one of items 1 to 7 is characterized by further comprising an updating component 228, which updates the graph by modifying or deleting the edge in a manner corresponding to the size of the obstacle present on the edge, based on the determination that an obstacle exists on the edge.

[0128] According to this system, the movement of a moving object can be stably controlled even in the presence of obstacles.

[0129] (Project 9)

[0130] According to the system described in Project 8, if the length of the portion of the edge that passes through the obstacle is greater than a threshold relative to the length of the edge, the updating component updates the graph by deleting the edge.

[0131] According to this system, the movement of a moving body can be reliably controlled based on the size of the obstacle.

[0132] (Project 10)

[0133] According to the system described in Project 9, when the length of the portion of the edge that passes through the obstacle is less than the threshold relative to the length of the edge, the updating component updates the graph in a manner that causes the edge to shift.

[0134] According to this system, the movement of a moving body can be reliably controlled based on the size of the obstacle.

[0135] (Project 11)

[0136] The system described in any of Items 1 to 8 is characterized in that the information acquisition unit 221 estimates the position of the mobile body 100 based on the output of the sensors provided by the mobile body 100 and generates an environmental map in a coordinate system based on the mobile body 100, and the determination unit 226 determines the moving target position T of the mobile body 100 based on the map which is transformed to a coordinate system based on the mobile body 100 and superimposed on the environmental map.

[0137] This system enables stable control of the movement of a moving object based on its surrounding environment.

[0138] (Project 12)

[0139] According to the system described in Project 11, the information acquisition component 221 deforms the image and overlays it with the environment map in a manner that the change in the transformation result of the image is accompanied by a change in the coordinate system based on the moving body 100.

[0140] This system enables stable control of the movement of a moving object based on its surrounding environment.

[0141] (Project 13)

[0142] A mobile body 100 includes a system 201 as described in any of items 1 to 12; and a sensor 206, wherein the information acquisition unit 221 of the system 201 estimates the position of a person based on the output from the sensor 206, the mobile body 100 including: a control unit 227 that controls the movement of the mobile body 100 to move toward the target position T; and a drive unit 204 that moves the mobile body 100 according to the control of the control unit 227.

[0143] The moving object can move stably based on the actions of the object it is following.

[0144] (Project 14)

[0145] According to the moving body described in item 13, the control unit 227 controls the moving body 100 based on the determination that there is an obstacle between the current position of the moving body 100 and the target position T, so that the moving body 100 moves on the path represented by the edge.

[0146] The mobile object can move stably even in the presence of obstacles.

[0147] (Project 15)

[0148] According to any of the items 13 to 14, the moving body is characterized in that the control unit 227 controls the moving body 100 based on the determination that there is no obstacle between the current position of the moving body 100 and the target position T, so that the moving body 100 moves straight from the current position of the moving body 100 to the target position T.

[0149] The moving object can move stably based on the actions of the object it is following.

[0150] (Project 16)

[0151] According to the moving body described in any of items 13 to 15, the feature is that a node 900 corresponding to the destination of the person is preset, and the control unit 227 changes the relative position or direction of the moving body 100 at the moving target position T relative to the person based on whether the person is within a predetermined range from the node 900 corresponding to the destination.

[0152] Users can easily operate the mobile device at their destination.

[0153] (Project 17)

[0154] One method is a method of controlling a mobile body to move according to the movement of a person, performed by an information processing device 201, characterized by including: step S301 of acquiring information representing the position and direction of movement of the person; step S302 of acquiring a graph representing the movement path of the person, composed of nodes and edges; step S303 of determining a first edge of the graph corresponding to the position of the person based on the position of the person; and step S305 of determining the target position of the mobile body based on the first edge or one or more edges located in the direction of movement of the person relative to the first edge.

[0155] According to this method, it is possible to stably control the movement of a moving body that moves based on the movement of the object it is following.

[0156] (Project 18)

[0157] A program for enabling a computer to function as a system as described in any of items 1 to 12.

[0158] According to this program, it is possible to stably control the movement of a moving body that moves based on the action of the object it is following.

[0159] The invention is not limited to the above-described embodiments, and various modifications and alterations can be made within the scope of the invention's intent.

Claims

1. A system that controls a mobile body to move in accordance with a person's movement, the system characterized by comprising: information acquisition means that acquires information indicating a position and a moving direction of the person; map acquisition means that acquires a map composed of nodes and edges, the map indicating a moving path of the person; determination means that determines a first edge of the map corresponding to the position of the person, based on the position of the person; and decision means that decides a moving target position of the mobile body, based on the first edge or one or more edges located in the moving direction of the person with respect to the first edge.

2. The system according to claim 1, characterized in that the decision means decides the moving target position of the mobile body based on a position corresponding to a point on the first edge or one or more edges located in the moving direction of the person with respect to the first edge.

3. The system according to claim 1, characterized in that the decision means decides the moving target position of the mobile body based on a plurality of edges connected to the first edge at a first node located in the moving direction of the person among nodes connected by the first edge.

4. The system according to claim 3, characterized in that the decision means evaluates probabilities of paths corresponding to the plurality of edges connected to the first edge respectively by a user, and decides a position on a path obtained by weightedly combining the paths corresponding to the plurality of edges respectively in accordance with the probabilities as the moving target position of the mobile body.

5. The system according to claim 1, characterized in that the decision means decides the moving target position so that the mobile body guides the person or follows the person.

6. The system according to claim 1, characterized by further comprising update means that updates the map in a manner to delete an edge based on a case where it is determined that an obstacle exists on the edge.

7. The system according to claim 1, characterized by further comprising update means that updates the map in a manner to shift the edge based on a case where it is determined that an obstacle exists on the edge.

8. The system according to claim 1, characterized by further comprising update means that updates the map in a manner to correct or delete the edge by a method corresponding to a size of the obstacle existing on the edge based on a case where it is determined that an obstacle exists on the edge.

9. The system according to claim 8, characterized in that the update means updates the map in a manner to delete the edge in a case where a proportion of a length of a portion of the edge passing through a region of the obstacle with respect to a length of the edge is greater than a threshold value.

10. The system according to claim 9, characterized in that the update means updates the map in a manner to shift the edge in a case where the proportion of the length of the portion of the edge passing through the region of the obstacle with respect to the length of the edge is equal to or less than the threshold value.

11. The system according to claim 1, characterized in that ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The information acquisition means estimates the position of the mobile body based on output from a sensor possessed by the mobile body, and generates an environmental map in a coordinate system with the mobile body as a reference, The decision means decides the movement target position of the mobile body based on the map transformed to the coordinate system with the mobile body as a reference and superimposed on the environmental map.

12. The system according to claim 11, wherein The information acquisition means deforms the map and superimposes it on the environmental map in such a manner that a change in the transformation result of the map is moderated in conjunction with a change in the coordinate system with the mobile body as a reference.

13. A mobile body that comprises: the system according to any one of claims 1 to 12; and a sensor in the mobile body, The information acquisition means of the system estimates the position of a person based on output from the sensor, The mobile body comprises: A control means that controls the action of the mobile body so that it moves toward the movement target position; and A drive means that moves the mobile body in accordance with the control of the control means.

14. The mobile body according to claim 13, wherein The control means controls the mobile body so that it moves on the path represented by the edge in accordance with a case where it is determined that there is an obstacle between the current position of the mobile body and the movement target position.

15. The mobile body according to claim 14, wherein The control means controls the mobile body so that it straightens out from the current position of the mobile body to the movement target position in accordance with a case where it is determined that there is no obstacle between the current position of the mobile body and the movement target position.

16. The mobile body according to claim 13, wherein A node corresponding to a destination of a person is set in advance, The control means changes the relative position or direction of the mobile body with respect to the person at the movement target position in accordance with whether the person is within a predetermined range from the node corresponding to the destination.

17. A method of controlling a mobile body so that it moves in accordance with the movement of a person, the method characterized by comprising: a step of acquiring information indicating the position and movement direction of a person; a step of acquiring a map composed of nodes and edges to represent the movement path of a person; a step of determining a first edge of the map corresponding to the position of the person based on the position of the person; and a step of deciding the movement target position of the mobile body based on the first edge or one or more edges located in the movement direction of the person with respect to the first edge.

18. A computer-readable storage medium that stores a computer program, the computer-readable storage medium characterized by: when the computer program is executed by a processor, the steps of the method according to claim 17 are implemented.

19. A computer program product that includes a computer program, the computer program product characterized by: ​ The steps of the method according to claim 17 are implemented when the computer program is executed by the processor.

Images