Control device for mobile body, control method for mobile body, and program

Abstract

A control device for a mobile body according to an embodiment comprises: a recognition unit that recognizes a situation surrounding the mobile body and recognizes a group composed of at least one following target, on the basis of an image in which the situation surrounding the mobile body is imaged; and a control unit that executes following control in which at least the mobile body moves while following the group on the basis of the surrounding situation. The control unit sets a virtual obstacle surrounding the group, and executes the following control so as not to come into contact with the set virtual obstacle.

Description

MOBILE BODY CONTROL DEVICE, MOBILE BODY CONTROL METHOD, AND PROGRAM

[0001] The present invention relates to a control device for a mobile object, a control method for a mobile object, and a program.

[0002] In recent years, various technologies related to autonomously moving vehicles have been researched, and in this connection, a technology has been disclosed in which a plurality of vehicles communicate with each other and control other vehicles to follow a lead vehicle selected from the plurality of vehicles (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2021-28747

[0004] However, no consideration has been given to the control of a moving body such as a vehicle when it is made to follow multiple targets, and therefore, when making a moving body follow multiple targets, it may not be possible to appropriately control the movement of the moving body.

[0005] The present invention has been made in consideration of these circumstances, and one of its objectives is to provide a control device for a moving body, a control method for a moving body, and a program that can more appropriately control the movement of a moving body that is made to follow at least one or more tracking targets.

[0006] The control device for a moving body, the control method for a moving body, and the program according to the present invention employ the following configuration: (1): A control device for a moving body according to one aspect of the present invention includes a recognition unit that recognizes the situation around the moving body and a group made up of at least one or more targets to be followed based on an image of the situation around the moving body, and a control unit that executes tracking control to cause at least the moving body to move to follow the group based on the situation around the moving body, wherein the control unit sets a virtual obstacle around the group and executes the tracking control so as not to come into contact with the set virtual obstacle.

[0007] (2) In the above aspect (1), the virtual obstacle is set around the target to be followed that constitutes the group.

[0008] (3) In the above aspect (2), the virtual obstacles are set so as to surround the target objects constituting the group.

[0009] (4) In the above aspect (2), the virtual obstacle is set so as to surround the target and the moving object that constitute the group.

[0010] (5): In the above aspect (1), the moving body determines the tracking reference position based on the position of a person set as the user of the moving body among the tracking targets that make up the group.

[0011] (6) In the above aspect (3), the shape of the area surrounded by the virtual obstacle is rectangular, and the size of the area is set based on the state of the target objects that make up the group.

[0012] (7) In the above aspect (6), the closer the distance between the plurality of objects to be tracked that make up the group, the smaller the area is made.

[0013] (8) In the above aspect (6), when the group is moving, the area is adjusted according to the direction and speed of movement.

[0014] (9): In the above aspect (1), when the target to be tracked in the group uses a moving object, the control unit outputs the same information from the information output unit provided in each of the multiple moving objects in the group.

[0015] (10): In the above aspect (1), when the recognition unit is unable to recognize the target to be tracked within the group, the control unit determines the next target to be tracked within the group and causes the moving body to follow the determined target to be tracked.

[0016] (11) In the above aspect (10), the next person to be followed is the person to be followed that is closest to the previous person to be followed among the plurality of people to be followed that make up the group.

[0017] (12) In the aspect (10) above, the next person to be followed is a person having the largest feature value for each of the plurality of people to be followed that is recognized by the recognition unit, among the people to be followed that make up the group.

[0018] (13) In the aspect of (1) above, the recognition unit recognizes the group based on a recognition result by the recognition unit or on pre-registered group information.

[0019] (14) In the above aspect (1), the control unit sets the offset distance from the tracking reference position based on the status of the group.

[0020] (15): In the above aspect (1), the control unit follows the group at a position away from any of the following targets included in the group whose risk value based on the amount of movement over a specified period of time is equal to or greater than a threshold value.

[0021] (16): In the aspect (1) above, the control unit follows the group at a position where the recognition unit can easily recognize the following target, among the following targets included in the group, whose risk value based on the amount of movement over a predetermined period of time is equal to or greater than a threshold value.

[0022] (17): Another aspect of the present invention is a method for controlling a moving body, in which a computer of the moving body recognizes the surrounding situation of the moving body and a group consisting of at least one or more objects to be followed based on an image of the surrounding situation of the moving body, performs a following control in which at least the moving body moves to follow the group based on the surrounding situation, sets a virtual obstacle around the group, and performs the following control so as not to come into contact with the set virtual obstacle.

[0023] (18): A program according to another aspect of the present invention is a program that causes a computer of a moving body to recognize the situation around the moving body and a group consisting of at least one or more objects to be followed based on an image of the situation around the moving body, executes a following control such that at least the moving body moves to follow the group based on the situation around the moving body, sets a virtual obstacle around the group, and executes the following control so as not to come into contact with the set virtual obstacle.

[0024] According to the above aspects (1) to (18), it is possible to more appropriately control the movement of the moving object that is to be followed by at least one or more follow-up targets.

[0025] 1 is a diagram showing an example of the configuration of a mobile body system 1 including a mobile body M in the first embodiment. A diagram for explaining an example of a usage mode in which a mobile body M is made to follow a user U. A perspective view showing an example of the external configuration of a mobile body M. A diagram for explaining an example of the functional configuration of the mobile body M. A diagram for explaining group following control of a mobile body M in the first embodiment. A diagram for explaining an example of the contents of group information 226. A diagram for explaining setting of a virtual obstacle when multiple people in a group are using a mobile body. A diagram for explaining adjustment of a virtual obstacle during group movement. A diagram for explaining a case in which the next person to be followed in a group is followed. A flowchart showing an example of following processing in the first embodiment. A diagram showing an example of a control device 200A of a mobile body M in the second embodiment. A diagram for explaining setting a following reference position. A flowchart showing an example of following processing in the second embodiment.

[0026] Hereinafter, embodiments of a mobile object control device, a mobile object control method, and a program according to the present invention will be described with reference to the drawings.

[0027] 1 is a diagram showing an example of the configuration of a mobile system 1 including a mobile body M in a first embodiment. The mobile body system 1 includes, for example, a terminal device 2, a management device 10, an information providing device 20, and the mobile body M. These communicate with each other via, for example, a network NW. The network NW is, for example, any network such as a LAN (Local Area Network), a WAN (Wide Area Network), or an internet line.

[0028] [Terminal Device] The terminal device 2 is a computer device such as a smartphone or a tablet terminal. The terminal device 2 is used by a user of the mobile body system 1, and requests the use of the mobile body M to the management device 10 based on the user's operation, and obtains information indicating that the use of the mobile body M is permitted from the management device 10. The user may be, for example, a representative of a group consisting of at least one person. A person is an example of a "following target." In the embodiment, the following target may include, in addition to a person, other moving bodies (e.g., micromobility), a remotely controlled avatar robot, a pet that a person is carrying, etc. In the following description, the following target is mainly a person.

[0029] [Management Device] The management device 10 manages the usage status and usage reservations of mobile bodies M in the mobile body system 1. In response to a request obtained from the terminal device 2, the management device 10 sets usage authority for mobile bodies M that can be used by a user, and transmits information indicating that use of the set mobile body M is permitted to the terminal device 2 to provide the user with the information. The management device 10 also generates and manages schedule information that associates, for example, pre-registered user identification information with the date and time of reservation for use of the mobile body M. The management device 10 may also manage the usage status and usage reservations of a group consisting of at least one person including the user.

[0030] [Information Providing Device] The information providing device 20 provides the mobile object M and the terminal device 2 with the location of the mobile object M, the area in which the mobile object M is moving, map information around the area, etc. Furthermore, the information providing device 20 may generate a route from the current location of the mobile object M to a destination in response to a request from the mobile object M, and provide the generated route to the mobile object M. The management device 10 and the information providing device 20 may be, for example, a server device, or may be a device configured by cloud computing including one or more information processing devices.

[0031] [Mobile Body] The mobile body M is, for example, a mobile body capable of autonomous movement. Autonomous movement means moving the mobile body M by executing one or both of speed control and turning control of the mobile body M without a driving operation by a user. Turning control includes, for example, changing the direction of the mobile body M by spinning or turning, and steering control in the case where the mobile body M has a steering wheel. The mobile body M is, for example, a vehicle, but may also include other mobile bodies capable of autonomous movement (for example, a walking robot, etc.). Vehicles include not only four-wheeled vehicles but also general vehicles that can move on three wheels, two wheels, etc. The mobile body M has a structure that can carry and transport an object, for example, luggage. The object may include a person, such as a user.

[0032] The mobile object M is used by a user or a group based on, for example, a usage authority set by the management device 10. For example, the user or group loads luggage or the like onto the mobile object M and has it follow the user or group's movement or lead them to a destination. The location information, usage status, usage reservation status, etc. of the mobile object M are managed by the management device 10. Furthermore, the mobile object M acquires information from the management device 10 and the information providing device 20 and controls its movement based on the usage authority and the provided information.

[0033] The mobile body system 1 of the first embodiment may include a plurality of at least one of the terminal device 2, the management device 10, the information providing device 20, and the mobile body M. Furthermore, the mobile body system 1 of the embodiment may include the management device 10 and the information providing device 20 integrated together. Furthermore, the mobile body system 1 may not include the management device 10. In this case, at least some of the functions of the management device 10 are provided in the mobile body M, and the terminal device 2 communicates with the mobile body M via the network NW to manage usage rights and the like.

[0034] The mobile object M will be described in detail below. Note that the following description will mainly focus on a case where the mobile object M is used in a manner in which it moves by following the movement of a user or a group. The same applies to the second embodiment described later. FIG. 2 is a diagram for explaining an example of a usage manner in which the mobile object M follows the user U. The mobile object M is located, for example, at a predetermined location in a facility or town. When the user U wishes to use the mobile object M, the user U operates the operation unit of the mobile object M to set a usage authority and start using the mobile object M. Alternatively, the user U may operate the terminal device 2 to send a usage request to the management device 10 so that the mobile object M can be used, and start using the mobile object M based on the usage authority set by the management device 10.

[0035] For example, when user U goes shopping and wants to reduce the burden of carrying purchased luggage, the user considers leaving the luggage in mobile body M and operates terminal device 2 or an operation unit of mobile body M to set the usage authority of mobile body M and start usage. When starting usage, user authentication by mobile body M (e.g., vein authentication using palm information of user U) is performed. If, as a result of the authentication, it is determined that the user is authorized to use the mobile body M (a user permitted to use the mobile body M), the mobile body M stores user U's luggage and performs personal tracking control to autonomously follow the movement of user U. This allows user U to continue shopping or head to their next destination in a comfortable state without having to carry luggage.

[0036] The moving object M may not only move on roads (roadways or sidewalks) but also in indoor or outdoor facilities such as shopping centers, airports, parks, and theme parks, as well as on private property. In addition to the tracking control for following the user U as described above, the moving object M may also be capable of other operational controls, such as guidance control, evacuation control, and emergency control. Guidance control is, for example, control for guiding the user U to a destination specified by the user U, such as autonomously moving in front of the user U at a speed that matches the user U's movement to lead the user U. Furthermore, guidance control does not necessarily require the user U to set the destination. For example, the moving object M may predict the direction of movement of the user U based on the user U's orientation or a movement trajectory from a predetermined time ago, and autonomously move in front of the user U at a speed that matches the user U's movement. Evacuation control is, for example, control for moving to a predetermined evacuation area and waiting when the user U moves into a no-entry area that the moving object M cannot enter during the execution of tracking control or guidance control. Emergency control is, for example, control that moves autonomously to seek help from nearby people or facilities in order to help user U if something unusual happens to user U (for example, if user U collapses) while traveling with user U.

[0037] Furthermore, if there is at least one other person around the user U and the other person satisfies a predetermined condition, the moving body M determines that the user U and the other person form a group and executes group tracking control to follow the group.

[0038] 3 is a perspective view showing an example of the external configuration of the moving body M. In the description of FIG. 3, the forward direction of the moving body M is the positive x direction, the backward direction of the moving body M is the negative x direction, the width direction of the moving body M is the left direction based on the positive x direction, the right direction is the negative y direction, and the height direction of the moving body M, which is a direction perpendicular to the x direction and y direction, is the positive z direction.

[0039] In the example of FIG. 3 , the mobile body M includes, for example, a base body 110, a door unit 112 provided on the base body 110, and wheels (first wheel 120, second wheel 130, and third wheel 140) attached to the base body 110. For example, a user U can open the openable door unit 112 to put luggage into a storage compartment provided on the base body 110 or take luggage out of the storage compartment. The first wheel 120 and the second wheel 130 are drive wheels that rotate by power from a motor or the like. The third wheel 140 is an auxiliary wheel (driven wheel). Note that the mobile body M may be movable using a configuration other than wheels, such as caterpillars.

[0040] A cylindrical support body 150 extending in the positive z direction is provided on the surface of the base body 110 facing the positive z direction. A camera 180 that captures images of the surroundings of the moving body M at a wide angle (e.g., 360 degrees) is provided on the end of the support body 150 facing the positive z direction. The position at which the camera 180 is provided may be any position different from the above. The camera 180 may include multiple cameras that capture images of a predetermined area. The moving body M shown in FIG. 3 is an example, and other components may be added, or some components (e.g., the door portion 112) may be absent.

[0041] [Functional Configuration of Mobile Object] Fig. 4 is a diagram showing an example of the functional configuration of the mobile object M. In addition to the configuration shown in Fig. 3, the mobile object M includes, for example, a first motor 122, a second motor 132, a battery 134, a braking device 136, a steering device 138, an operation unit 160, an information output unit 170, a communication unit 190, and a control device 200. The first motor 122 and the second motor 132 are operated by power supplied to the battery 134. The first motor 122 drives the first wheel 120, and the second motor 132 drives the second wheel 130. The first motor 122 may be an in-wheel motor provided in the wheel of the first wheel 120, and the second motor 132 may be an in-wheel motor provided in the wheel of the second wheel 130.

[0042] The brake device 136 outputs a brake torque to each wheel based on an instruction from the control device 200. The steering device 138 includes an electric motor. The electric motor applies a force to a rack and pinion mechanism based on an instruction from the control device 200, for example, to change the direction of the first wheel 120 or the second wheel 130, thereby changing the course of the moving object M.

[0043] The operation unit 160 accepts input operations by the user U. The operation unit 160 includes, for example, at least one of a touch panel, a switch, a key, etc. The operation unit 160 may also have an audio input unit (such as a microphone) that accepts input operations by the user U through voice input or accepts voices of people around the user.

[0044] The information output unit 170 outputs predetermined information to the surroundings of the user U or the moving object M. The information output unit 170 includes, for example, a display unit 172 and an audio output unit 174. The display unit 172 is, for example, a liquid crystal display (LCD) or an organic electroluminescence (EL) display. The display unit 172 may also be integrated with the operation unit 160 as a touch panel. The display unit 172 displays images corresponding to various pieces of information in the embodiment. The display unit 172 may also include a light-emitting unit such as an LED (light-emitting diode) that emits a predetermined color. The audio output unit 174 outputs a sound corresponding to the movement of the moving object M or a sound corresponding to an image output by the display unit 172. The information output unit 170 is, for example, an example of an alarm unit that notifies the surroundings of the moving object M of predetermined information.

[0045] The communication unit 190 is a communication interface for communicating with the terminal device 2, the management device 10, or the information providing device 20 via the network NW. The communication unit 190 may also communicate with other mobile objects.

[0046] The control device 200 is housed within the base 110. The control device 200 includes, for example, a position identification unit 202, an information processing unit 204, a recognition unit 206, a virtual obstacle setting unit 208, a trajectory generation unit 210, a first control unit 212, a second control unit 214, and a storage unit 220. The position identification unit 202, the information processing unit 204, the recognition unit 206, the virtual obstacle setting unit 208, the trajectory generation unit 210, the first control unit 212, and the second control unit 214 are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be implemented by hardware (including circuitry) such as a large-scale integration (LSI), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a graphics processing unit (GPU), or may be implemented by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transitory storage medium) such as a hard disk drive (HDD) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device. Some or all of the information processing unit 204, the virtual obstacle setting unit 208, the trajectory generation unit 210, the first control unit 212, and the second control unit 214 are examples of a "control unit." Some or all of the functional configuration included in the control device 200 may be included in another device. For example, the other device and the moving object M may communicate with each other and cooperate to control the moving object M.

[0047] The storage unit 220 is realized by a storage device such as a HDD, flash memory, or RAM (Random Access Memory). The storage unit 220 stores control information 222 including a control program for controlling the behavior of the moving object M referenced by the first control unit 212 and the second control unit 214, map information 224, and group information 226. The map information 224 is, for example, map information provided by the information providing device 20, such as the location of the moving object M, the area in which the moving object M moves, and the surrounding area. The map information 224 may include information such as the location of products, the location of stores, and floor maps of facilities such as shopping centers, art galleries, and museums, associated with location information on the map (e.g., latitude and longitude). The map information 224 may also include detailed map information such as the width, gradient, and curvature of roads and passageways.

[0048] The group information 226 is information about a group consisting of at least one person (an example of at least one tracking target). The group information 226 may be acquired, for example, via the terminal device 2 or the management device 10, or may be acquired from the analysis results of an image captured by the camera 180 or the recognition results of the recognition unit 206. Details of the group information 226 will be described later. The storage unit 220 may also store characteristic information about the user U, characteristic information about a specific object (e.g., characteristic information based on shape, size, color, etc.), and information indicating a correspondence between the user U's actions (gestures) and the behavior (motion control) of the moving object M. At least a portion of the information stored in the storage unit 220 may be updated as needed by the communication unit 190 communicating with another device, such as the management device 10 or the information providing device 20.

[0049] The position identification unit 202 identifies the position of the mobile object M. The position identification unit 202 acquires position information of the mobile object M using a GPS (Global Positioning System) device (not shown) built into the mobile object M. The position information may be, for example, two-dimensional map coordinates or latitude and longitude information. The position identification unit 202 may also estimate the position information of the mobile object M by communicating with a communication device located within a predetermined distance via the communication unit 190 using a short-range wireless communication method such as Bluetooth (registered trademark) and acquiring position information of the communication device. The position identification unit 202 may also estimate the position of the mobile object M based on a specific structure (landmark, sign, sign) or the like included in a camera image captured by the camera 180. The position identification unit 202 may also estimate the position of the mobile object M simultaneously with the creation of an environmental map using a method such as so-called SLAM (Simultaneous Localization and Mapping) using a camera image captured by the camera 180 or a sensor such as LIDAR.

[0050] The information processing unit 204 manages, for example, information acquired from the terminal device 2, the management device 10, or the information providing device 20. Furthermore, based on information accepted by the operation unit 160, the information processing unit 204 transmits information to the terminal device 2, the management device 10, or the information providing device 20, outputs information received from each device, etc. (for example, control information 222, map information 224, group information 226, and characteristic information of the user U or a specific object) to each component of the control device 200, or stores the information in the storage unit 220.

[0051] Furthermore, the information processing unit 204 causes the information output unit 170 to output predetermined information. The predetermined information includes, for example, information such as an image (including a color image or a pattern), sound, or alarm according to the type of operation control being executed by the moving body M, information indicating that the user U has been recognized, information indicating that an individual or a group is being followed, information indicating the direction of forward movement, reverse movement, or turning, etc. The output information such as image or sound may be stored in the storage unit 220 or may be transmitted from the management device 10 or the information providing device 20.

[0052] The information processing unit 204 also performs processes for registering characteristic information of a user U who uses the mobile object M, registering group information (e.g., characteristic information for at least one person constituting a group), and authenticating the user U. For example, when registering the characteristic information of the user U, the information processing unit 204 generates characteristic information regarding the face, body shape (height), hair color, skin color, clothing color, etc. from a facial image or a full-body image of the user U captured by the camera 180 before the user U started using the mobile object M. The information processing unit 204 may also generate characteristic information regarding the posture and movement (walking behavior) of the user U. The information processing unit 204 may also acquire voice data or an image of the palm of the user U to generate characteristic information regarding fingerprints, veins, and voice. The information processing unit 204 may also generate information regarding the movement of the mobile object M corresponding to a gesture by the user U and register the generated information in the storage unit 220. The information processing unit 204 may also generate and register characteristic information for each person in the group, similar to that for the user U.

[0053] Furthermore, when performing user authentication, the information processing unit 204 generates feature information from the voice acquired from the image microphone capturing an image of the user U, references feature information previously stored in the storage unit 220 based on the generated feature information, and if matching feature information (feature information with a similarity equal to or greater than a threshold) is found, permits the user U to use the mobile object M. Furthermore, if matching feature information is not found in the storage unit 220, the information processing unit 204 does not permit use and causes the information output unit 170 to output an error message or a message prompting the user U to register feature information.

[0054] The recognition unit 206 recognizes the situation around the moving body M, for example, based on camera images captured by the camera 180. For example, the recognition unit 206 recognizes the positions (distance from the moving body M and direction relative to the moving body M) and states such as speed and acceleration of objects around the moving body M. Objects include people such as pedestrians, traffic participants such as cyclists, and obstacles present in facilities or on roads. The recognition unit 206 may also recognize feature information for each person in the vicinity, as well as the facial orientation, movement speed, and movement direction of each person. The feature information is, for example, information that can be obtained from the analysis results of the camera image, and specifically includes information about a person's body shape, height, facial feature information, color information, information about clothing such as clothes and hats, etc. The feature information may also include information about feature quantities. A feature quantity is, for example, the amount of information required to distinguish one person from another, and the more distinguishable features there are, the larger the value.

[0055] The recognition unit 206 also recognizes, for example, a user U using the moving object M and tracks (follows) the recognized user U. For example, the recognition unit 206 performs a matching process between the user's characteristic information recognized from the camera image captured by the camera 180 and the user's characteristic information registered in advance when the user U uses the moving object M or the characteristic information of the user U provided by the terminal device 2 or the management device 10, thereby identifying the user U using the moving object M from people in the vicinity. The recognition unit 206 may also identify the user U using AI (artificial intelligence) technology such as machine learning. When tracking the moving user U, the recognition unit 206 sets a future predicted trajectory based on the user U's past movement trajectory, and further compares the predicted trajectories with those of other people in the vicinity to identify the person moving along a smoother trajectory as the user U.

[0056] The recognition unit 206 may also recognize a gesture made by the user U. Gestures include, for example, a gesture of pointing to a specific position in the vicinity, a gesture of calling the moving object M, and a gesture of switching between operation controls (individual following control, group following control, guidance control, evacuation control, and emergency control) performed by the moving object M. The recognition unit 206 may also recognize a specific movement of the user U (for example, the movement of the user U falling down) based on a camera image captured by the camera 180, and may recognize that something unusual has happened to the user U when the recognition unit 206 recognizes the specific movement.

[0057] The moving body M may be provided with a detection unit, such as a radar device or a LIDAR (Light Detection and Ranging) device, that is different from the camera 180. In this case, the recognition unit 206 recognizes the situation around the moving body M using the detection results of the radar device or LIDAR instead of (or in addition to) the image.

[0058] Furthermore, the recognition unit 206 recognizes a group consisting of at least one person including the user U. The function of the recognition unit 206 to recognize a group will be described in detail later.

[0059] The virtual obstacle setting unit 208 sets a virtual obstacle around the group recognized by the recognition unit 206. The function of the virtual obstacle setting unit 208 will be described in detail later.

[0060] The trajectory generation unit 210 generates a target trajectory along which the moving body M should travel in the future, based on, for example, the position of the user U or group, surrounding objects, etc. For example, the trajectory generation unit 210 generates a target trajectory that allows the moving body M to move smoothly to a target point in accordance with an action based on a preset operation control (for example, individual following control, group following control, guidance control, evacuation control, emergency control).

[0061] For example, in the case of individual tracking control for tracking a specific person (user U), the trajectory generation unit 210 generates a target trajectory with a distance (offset distance) from the user U within a predetermined distance range and a target point behind the user U (not limited to directly behind the user U, but may be diagonally behind within a predetermined angle with the direction directly behind as the reference). Also, in the case of group tracking control for tracking a group made up of at least one person, the trajectory generation unit 210 generates a target trajectory for moving without coming into contact with virtual obstacles around the group set by the virtual obstacle setting unit 208. Also, the trajectory generation unit 210 may select a person to be followed from the people in the group, and generate a target trajectory with a distance from the selected person to be followed within a predetermined distance range and a target point behind the person to be followed.

[0062] In the case of guidance control, the trajectory generation unit 210 generates a target trajectory with a target point within a predetermined distance range from the user U and in front of the user U (not limited to directly in front of the user U, but may be diagonally forward within a predetermined angle from the front direction). In the case of evacuation control, the trajectory generation unit 210 generates a target trajectory with a predetermined evacuation area as a target point. In the case of emergency control, the trajectory generation unit 210 generates a trajectory for autonomous movement to seek help from nearby people or facilities.

[0063] Furthermore, the trajectory generation unit 210 generates a target trajectory corresponding to the behavior (or motion control) of the moving body M in response to the gesture, based on, for example, a predetermined correspondence between the gesture and the behavior of the user U, or generates a target trajectory (or corrects the generated target trajectory) for moving toward a destination while avoiding surrounding objects. For example, when the recognition unit 206 recognizes a gesture by the user U such that the user U beckons toward the moving body M, the trajectory generation unit 210 generates a target trajectory for the moving body M so that the distance between the moving body M and the user U becomes closer than a reference distance (for example, approximately 1 to 3 m) (for example, so that the distance between the moving body M and the user U becomes within 30 cm). Furthermore, when the motion control is switched in response to the gesture of the user U, the trajectory generation unit 210 generates a future trajectory corresponding to the switched motion control.

[0064] Furthermore, the trajectory generation unit 210 may generate a target trajectory along which the moving body should travel in the future, based on a destination set by the user U or the next destination of the user U estimated by the second control unit 214 described later. Note that the trajectory generation unit 210 may, for example, generate multiple trajectories according to the behavior of the moving body M, calculate the risk for each trajectory, and, if the total value of the calculated risks or the risk at each trajectory point meets a predetermined criterion (for example, if the total value is equal to or less than a first threshold value Th1 and the risk at each trajectory point is equal to or less than a second threshold value Th2), adopt the trajectory that satisfies the criterion as the target trajectory along which the moving body M should travel. For example, the risk tends to be higher the closer the distance between the trajectory (trajectory point on the trajectory) and an obstacle, and lower the closer the distance between the trajectory and the obstacle.

[0065] The first control unit 212 controls the motors (first motor 122, second motor 132), the braking device 136, and the steering device 138 so that the moving body M travels along the trajectory generated by the trajectory generating unit 210.

[0066] When the moving object M travels along the generated trajectory, the second control unit 214 controls the moving object M based on the recognition result of the recognition unit 206 so that the moving object M does not come into contact with surrounding objects and so that the distance between the user U and the moving object M is within a predetermined distance range. The predetermined distance range is a distance range between a predetermined shortest distance Dmin and a predetermined longest distance Dmax. The shortest distance Dmin and the maximum distance Dmax may be variable distances depending on, for example, the type of operation control (personal tracking control, guidance control, etc.) and surrounding conditions (shape of the moving path, degree of crowding), or may be fixed distances.

[0067] In addition, in group tracking control, when a virtual obstacle is set by the virtual obstacle setting unit 208, the second control unit 214 executes motion control of the moving body M so that the moving body M does not come into contact with the virtual obstacle.

[0068] [Movement Control of Moving Body] Next, a specific description will be given of the movement control of the moving body M according to the first embodiment. Note that the following description will be given of movement control relating to group following control in which the moving body M follows a group made up of multiple people including the user U. The people include, for example, pedestrians (adults, children, elderly people, etc.) and people who can move by a moving body such as a wheelchair (which may be manual or electric).

[0069] 5 is a diagram illustrating group tracking control of a moving object M in the first embodiment. In the example of FIG. 5, the moving object M is assumed to be following behind the user U at an offset distance D1. Here, the recognition unit 206 determines, based on the surrounding recognition result, whether or not there is a person (a person other than the user) who is within a predetermined distance from the user U and has been present for a predetermined period of time or longer. Then, when the recognition unit 206 determines that there is a person who is within a predetermined distance from the user U and has been present for a predetermined period of time or longer, it recognizes that the person and the user U constitute a group.

[0070] The recognition unit 206 may also calculate the average value of the distance or speed difference between the user U and other people over time, and recognize people whose average value is within a threshold as people who constitute a group. The recognition unit 206 may also recognize the gaze direction of people who are within a predetermined distance from the user U, and recognize that the person and the user U constitute a group if the gaze direction is directed toward the user U for a predetermined period of time or more. The recognition unit 206 may also analyze camera images captured by the camera 180 to analyze the mouth movements of people, and recognize a person who is conversing with the user U, and recognize that the person and the user U constitute a group. The recognition unit 206 may also recognize a person who is conversing with the user U based on voice input from the microphone of the operation unit 160, and recognize that the person and the user U constitute a group. This allows for more accurate group recognition.

[0071] Furthermore, for example, when information about a group is registered in advance in the group information 226 , the recognition unit 206 may recognize the group based on the registered group information 226 .

[0072] FIG. 6 is a diagram illustrating an example of the content of the group information 226. The group information 226 is, for example, information in which a group ID, which is identification information for identifying a group, is associated with the number of people, characteristic information, and user information. The number of people is the number of people in the group including user U. The characteristic information may be information indicating the characteristics of each person in the group (e.g., body shape, height, etc.) or information indicating the characteristics of the group as a whole (e.g., whether all members are women or whether all members wear the same uniform, hat, etc.). The characteristic information may also include a feature amount. The characteristic information may also include billing information, billing address, personal information, etc. for each person in the group. The user information may be information about user U who registered the group or information about the representative of the group. The user information may also include information about one or more users using the mobile object M.

[0073] The information processing unit 204 receives registration or modification of the group information 226 from, for example, the terminal device 2, the management device 10, or the operation unit 160, and stores the received group information in the storage unit 220. Note that the registration or modification of the group information 226 may be performed at any time.

[0074] For example, if user U has registered a group ID "GR100" in advance, and five people including user U are present within a predetermined distance for a predetermined period of time or longer, as shown in Fig. 5, the recognition unit 206 recognizes that user U and people P1 to P4 form a group. On the other hand, if user U has registered a group ID "GR200" in advance, the number of people is less than eight in the example of Fig. 5, and therefore user U and people P1 to P4 are recognized as not forming a group.

[0075] 5, the recognition unit 206 recognizes that the user U and the persons P1 to P4 form a group. Also, in the example of Fig. 5, the user U and the persons P1 to P4 are assumed to be stationary (their movement distance in a predetermined time is less than a predetermined distance).

[0076] The virtual obstacle setting unit 208 sets a virtual obstacle VOB1 so as to surround the periphery of the group. The virtual obstacle VOB1 is, for example, a virtual object that exists when an obstacle does not actually exist, but is assumed to exist. For example, the virtual obstacle setting unit 208 sets the virtual obstacle VOB1 so as to surround all the people (user U and people P1 to P4) constituting the group based on their positions (this may include a predetermined margin). The virtual obstacle setting unit 208 may also set the virtual obstacle VOB1 so as to surround all the people in the group and the moving object M.

[0077] The shape of the area surrounded by the virtual obstacle VOB1 may be a rectangle with length (in the front direction of the moving object M) W10 and width (in the horizontal direction of the moving object M) W20 as shown in Fig. 5, or a circle with radius R centered on the user U or the moving object M, or may be another polygon, ellipse, etc. For example, the virtual obstacle setting unit 208 may variably set the shape of the area depending on the number of people in the group, their positions, the shape of the movement path, etc.

[0078] The virtual obstacle setting unit 208 may also adjust the size of the area surrounded by the virtual obstacle VOB1 based on the status of the people in the group (e.g., the number of people in the group, the distance between the people, the movement status, etc.). For example, the virtual obstacle setting unit 208 makes the area larger the more people in the group there are, and makes the area smaller the shorter the distance between the people (i.e., the closer the people are).

[0079] Furthermore, when two or more people in a group are using the moving object M, the virtual obstacle setting unit 208 sets a virtual obstacle that surrounds all people in the group and all of the moving objects used by those people. Fig. 7 is a diagram for explaining the setting of a virtual obstacle when two or more people in a group are using the moving object M.

[0080] 7 shows an example in which, of the people (user U and people P1 to P4) who make up a group, user U uses moving body M1 and person P4 uses moving body M2. Also, in the example in FIG. 7, moving body M1 follows behind user U at an offset distance D1, and moving body M2 follows behind person P4 at an offset distance D1. In this case, when the recognition unit 206 recognizes that user U and people P1 to P4 make up a group, the virtual obstacle setting unit 208 of moving body M1 sets a virtual obstacle VOB2 that surrounds moving bodies M1 and M2 and the people who make up the group.

[0081] When the moving object M is performing tracking, the information processing unit 204 causes the information output unit 170 to output information indicating that tracking control is being performed. Furthermore, when there are multiple moving objects in a group as shown in FIG. 7 , the information processing unit 204 may cause the information output units 170 of the multiple moving objects M to output the same information. In this case, the information processing unit 204 causes the display units 172 of the moving objects M1 and M2 to emit light of similar colors or display images of similar patterns. For example, the information processing unit 204 may communicate with other moving objects in the group via the communication unit 190 to emit light of similar colors, or may emit light in response to control information (color information) from the management device 10 when the management device 10 is performing group management. Furthermore, instead of (or in addition to) emitting light of similar colors or displaying images of the same patterns, the information processing unit 204 may output a similar sound or display text information indicating that group tracking is being performed.

[0082] This makes it easier for people in the vicinity to understand that multiple moving objects are following the same group, making it easier for people in the vicinity to predict the path of moving object M and find a path that avoids contact with moving object M.

[0083] Furthermore, the virtual obstacle setting unit 208 may adjust the size of the area surrounded by the virtual obstacle depending on the movement status of the group (for example, the movement direction and movement speed). Fig. 8 is a diagram for explaining how the virtual obstacle is adjusted when the group moves. In the example of Fig. 8, it is assumed that a user U moves in the direction of the arrow at a speed UV, and persons P1 to P4 move in the direction of the arrow at their respective speeds PV1 to PV4.

[0084] In this case, the virtual obstacle setting unit 208 adjusts the length W11 and width W21 of the virtual obstacle based on the position information of the user U and the people P1 to P4, as well as their respective moving speeds and moving directions. For example, the virtual obstacle setting unit 208 sets the virtual obstacle VOB3 so as to surround the moving object M, the user U, and the people P1 to P4. The virtual obstacle setting unit 208 also extends the virtual obstacle VOB3 in the moving direction from the position of the person P3, who is moving at the forefront of the group, according to the moving speed PV3 (in the example of FIG. 8, the length W11 of the virtual obstacle VOB3 is lengthened). This prevents the people in the group from exceeding the virtual obstacle VOB3. Furthermore, if the people in the group are moving in the same direction (within a predetermined tolerance range), the virtual obstacle setting unit 208 narrows the width in the direction perpendicular to the moving direction of the people in the group (in the example of FIG. 8, the width W21 direction) compared to when the people are stationary. Since it is unlikely that people in a group will move in the direction perpendicular to the direction of travel (horizontal direction), narrowing the width allows for appropriate group tracking within virtual obstacle VOB3.

[0085] Furthermore, when following a moving group, the offset distance D2 from the user U may be set to a distance different from the offset distance D1 when the user U is stationary. In this case, the trajectory generation unit 210 sets, for example, the offset distance D2 from the user U when following a moving group to be shorter than the offset distance D1 when the user U is stationary. By making the offset distance D2 shorter when the user U is moving than when the user U is stationary, it is possible to make it less likely that the user U will lose sight of the user U.

[0086] Also, when the group is moving, the moving body M follows behind the user U recognized by the recognition unit 206, but if the user U moves position within the group during group following and the recognition unit 206 is no longer able to recognize the user U, the second control unit 214 determines the next person to be followed within the group and continues group following control.

[0087] 9 is a diagram illustrating a case where the next person to be followed in a group is followed. When the recognition unit 206 is unable to recognize the current person to be followed (e.g., user U), the second control unit 214 determines the next person to be followed from among the other people in the group. For example, based on the recognition result of the recognition unit 206, the second control unit 214 determines the person with the largest feature amount among people P1 to P4 other than user U who are present in the group as the person to be followed. This makes it possible to select a person who is easier to distinguish from other people (easier to recognize), making following easier.

[0088] Furthermore, the second control unit 214 may determine the person closest to the moving body M as the next person to be followed. In the example of Fig. 9, the user U has moved forward, making it difficult for the recognition unit 206 to recognize the person, and since the person P1 is closest to the moving body M, the person P1 is determined to be the next person to be followed.

[0089] Furthermore, the second control unit 214 may determine, based on the recognition result, the person closest to the user U (the previous person to be followed) as the next person to be followed. This makes it possible to suppress changes in the operation control of the moving object M when the person to be followed is changed from the user U to another person, and to continue smooth following control even when the person to be followed is changed. Furthermore, the second control unit 214 may determine, as the next person to be followed, the person specified by the user U via the terminal device 2 or the operation unit 160.

[0090] Furthermore, during group tracking control, the second control unit 214 may control the moving body M to perform group tracking at a position away from at least one person included in the group whose risk value based on the amount of movement in a predetermined time period is equal to or greater than a threshold. Note that when a group includes multiple people, a risk value is set for each person based on the amount of movement in a predetermined time period for each person. The risk value is, for example, an index value indicating that approach is not recommended (approach non-recommended index value).

[0091] For example, if the people in the group include children, there is a high possibility that the children will move in various directions, increasing the possibility of them coming into contact with the moving object M. Therefore, the second control unit 214 increases the risk value according to the amount of movement in a predetermined period of time, and causes the person to be followed in the group to follow from a position at least a distance away according to the risk value. In this case, in order to move away by more than the predetermined distance, the virtual obstacle setting unit 208 may increase the area surrounded by the virtual obstacle VOB3. This makes it possible to prevent contact between the moving object M and people in the group according to the risk for each person in the group.

[0092] Furthermore, the second control unit 214 may perform control so that a person whose risk value is equal to or greater than a threshold is tracked from a position (for example, directly behind the person) where the person can be easily recognized by the recognition unit 206. This allows group tracking to be performed at a more appropriate position while recognizing the position of a person who has moved a large amount.

[0093] [First Embodiment: Processing Flow] FIG. 10 is a flowchart showing an example of the tracking process in the first embodiment. The process shown in FIG. 10 may be repeatedly executed at a predetermined interval or at a predetermined timing. In the example of FIG. 10 , when the moving object M performs tracking control, the recognition unit 206 of the moving object M recognizes the situation around the user U (step S100). Next, the information processing unit 204 determines whether to perform group tracking control based on the presence of at least one person around the user U (step S110). If it is determined that group tracking control should be performed, the recognition unit 206 sets a group including at least one person around the user U (step S120). Next, the virtual obstacle setting unit 208 sets a virtual obstacle surrounding the group and the moving object (step S130).

[0094] Next, the trajectory generation unit 210 generates a trajectory for following the group without contacting the virtual obstacle (step S140). Furthermore, if it is determined in the processing of step S110 that group following control is not to be performed, individual following control is performed. In this case, the recognition unit 206 identifies the user U (step S150) and generates a trajectory for following the identified user U (step S160). After the processing of steps S140 and S160, the first control unit 212 and the second control unit 214 execute following control based on the generated trajectory (step S170). This ends the processing of this flowchart.

[0095] According to the first embodiment described above, the control device 200 for the moving body M includes a recognition unit 206 that recognizes the situation around the moving body M and a group consisting of at least one target to be followed based on an image of the situation around the moving body M, and a control unit that executes tracking control so that at least the moving body M moves to follow the group based on the situation around the moving body M. The control unit sets virtual obstacles around the group and executes tracking control so as to avoid contact with the set virtual obstacles, thereby more appropriately controlling the movement of the moving body that is to be followed by at least one target to be followed. Furthermore, according to the first embodiment, setting virtual obstacles around the group makes it easier for the moving body M to follow the group, and the moving body M can move together with the group.

[0096] Second Embodiment Next, a second embodiment of tracking control will be described. In the second embodiment, when performing group tracking control, a virtual obstacle is not set, but a tracking reference position is set based on the position information of at least one person recognized as a group, and the moving object M is caused to track based on the set tracking reference position, thereby executing group tracking control. An embodiment based on the above content will be described below. Note that the second embodiment differs from the functional configuration of the moving object M in the first embodiment shown in FIG. 4 in that a control device 200A is provided instead of the control device 200. Therefore, the following will mainly describe the configuration of the control device 200A. Furthermore, the same functional configurations as those in the first embodiment will be given the same names and symbols, and detailed description thereof will be omitted here.

[0097] 11 is a diagram illustrating an example of a control device 200A for a moving object M according to the second embodiment. The control device 200A includes a position identification unit 202, an information processing unit 204, a recognition unit 206, a tracking reference position setting unit 209, a trajectory generation unit 210, a first control unit 212, a second control unit 214, and a storage unit 220. The position identification unit 202, the information processing unit 204, the recognition unit 206, the tracking reference position setting unit 209, the trajectory generation unit 210, the first control unit 212, and the second control unit 214 are realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI, ASIC, FPGA, or GPU, or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device such as an HDD or flash memory (a storage device having a non-transitory storage medium), or may be stored in a removable storage medium (non-transitory storage medium) such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device. Some or all of the information processing unit 204, the tracking reference position setting unit 209, the trajectory generating unit 210, the first control unit 212, and the second control unit 214 are examples of a "control unit."

[0098] 11 differs from the control device 200 of the first embodiment in that it includes a following reference position setting unit 209 instead of the virtual obstacle setting unit 208. Therefore, the following description will focus on this difference. The following reference position setting unit 209 sets the center position of the group recognized by the recognition unit 206 as the following reference position.

[0099] 12 is a diagram for explaining the setting of the tracking reference position. When the recognition unit 206 recognizes that the user U and persons P1 to P4 shown in FIG. 12 constitute a group, the tracking reference position setting unit 209 derives the center position of the group from the average value of the position information for each person constituting the group, and sets the derived center position as the tracking reference position FP. The tracking reference position setting unit 209 may weight the position of user U, among the positions of each person constituting the group, more heavily than the positions of other people in the group. As a result, the tracking reference position FP is set to a position closer to user U than the position information based on the position information for each person constituting the group.

[0100] Furthermore, the tracking reference position setting unit 209 may increase the weight of a person who is easier to recognize or track based on the recognition result by the recognition unit 206, and adjust the tracking reference position FP so that it approaches the position of the person with the larger weight. For example, the tracking reference position setting unit 209 increases the weight of a person who has a larger characteristic amount compared to other people (an example of a person who is easy to recognize or easy to track) based on the recognition result by the recognition unit 206. This makes it possible to set the tracking reference position FP near a person with more distinctive features.

[0101] The following reference position setting unit 209 may also adjust the following reference position FP depending on the number of people in the group, the relative distance between people, the relative speed, etc. The following reference position setting unit 209 may also set a risk for each person in the group based on the amount of movement over a predetermined period of time, and set a small weight so that the following reference position is set at a position away from people with a high set risk. The risk is an index value indicating that approach is not recommended (approach non-recommended index value).

[0102] For example, if the people making up the group include children, there is a high possibility that the children will move in various directions, increasing the possibility that they will come into contact with the moving object M. Therefore, the tracking reference position setting unit 209 increases the risk value according to the amount of movement in a predetermined time, and sets the tracking reference position FP at a position at least a distance away according to the risk value. This makes it possible to prevent contact between the moving object M and people in the group according to the risk for each person making up the group.

[0103] Furthermore, the tracking reference position setting unit 209 may set the tracking reference position FP at a position where a person whose risk value is equal to or greater than a threshold value can be easily recognized by the recognition unit 206. This allows group tracking to be performed at a more appropriate position while recognizing the position of a person who moves a large amount.

[0104] Furthermore, the tracking reference position setting unit 209 may identify a person who is easiest to recognize or track among at least one person constituting a group, and set the position of the identified person as the tracking reference position FP. In this case, the tracking reference position FP is set to, for example, the position of the person closest to the moving object M or the position of the person with the largest feature amount.

[0105] Furthermore, when the distance between the members of a group becomes equal to or greater than a predetermined distance and the group splits, the recognition unit 206 may disband the group and terminate group tracking control. Furthermore, when the recognition unit 206 recognizes a group split, it may predict the future (predetermined time ahead) movement direction and movement amount based on the past movement history and surrounding conditions of each member of the group. Then, when it is predicted based on the prediction result that the group will temporarily split to avoid an obstacle and then the split will be resolved (the distance between the members will be less than the predetermined distance), the tracking reference position setting unit 209 sets a future tracking reference position after the split is resolved. This allows group tracking to continue based on the prediction result of the future movement direction and movement amount even if the group splits.

[0106] In the second embodiment, the trajectory generating unit 210 generates a target trajectory, for example, such that the distance from the tracking reference position set by the tracking reference position setting unit 209 is within a predetermined distance range and the target point is behind the tracking reference position. The first control unit 212 and the second control unit 214 move the moving object M along the generated target trajectory.

[0107] [Second embodiment: processing flow] Figure 13 is a flowchart showing an example of the tracking processing in the second embodiment. The tracking processing in the second embodiment differs from the tracking processing in the first embodiment shown in Figure 10 (the processing of steps S100 to S170) in that it includes processing of steps S230 and S240 instead of the processing of steps S130 and S140. Therefore, the following description will mainly focus on the processing of steps S230 and S240.

[0108] In the example of FIG. 13 , after determining that group tracking control is to be performed and setting a group, the tracking reference position setting unit 209 sets a tracking reference position based on the position information of at least one person belonging to the group (step S230). Next, the trajectory generation unit 210 generates a trajectory for tracking the group using the set tracking reference position as a reference (step S240). In the process of step S240, a trajectory is generated so that the moving object M does not come into contact with surrounding objects and the distance between the tracking reference position FP and the moving object M is within a predetermined distance range. The predetermined distance range is a distance range between a predetermined minimum distance Dmin and a maximum distance Dmax. The minimum distance Dmin and the maximum distance Dmax may be variable distances depending on, for example, surrounding conditions (shape of the moving path, degree of crowding), or may be fixed distances.

[0109] According to the second embodiment described above, the control device 200A for the moving body M includes a recognition unit 206 that recognizes the situation around the moving body M and a group consisting of at least one target to be followed based on an image of the situation around the moving body M, and a control unit that performs tracking control so that at least the moving body M moves to follow the group based on the situation around the moving body M. The control unit sets a tracking reference position based on the position of each person in the group and causes the moving body to follow the set tracking reference position, thereby more appropriately controlling the movement of the moving body that is to be followed by at least one target to be followed. Furthermore, according to the second embodiment, setting a tracking reference position according to the situation of the group allows more appropriate group tracking.

[0110] [Modification] One of the first and second embodiments described above may be combined with part or all of the other embodiment. For example, the virtual obstacle setting unit 208 in the first embodiment may set a virtual obstacle centered on the tracking reference position FP set in the second embodiment.

[0111] In the first and second embodiments, whether to perform group tracking or individual tracking may be switched by a command from the terminal device 2 by the user U. In the embodiments, when the target of use is switched from an individual to a group, each individual in the group may be charged a fee according to the number of people in the group. In this case, each individual is charged based on the personal information of the group members registered in the group information 226.

[0112] In addition, in group tracking control, when at least one person in a group uses a mobile object, the mobile objects may communicate with each other to form a convoy and follow the group. In this case, the mobile object at the front of the convoy follows the person to be followed (the person to be followed in the group or the following reference position), and the mobile objects at the rear follow the mobile object traveling in front.

[0113] Furthermore, the moving body M in the first and second embodiments may be one that the user U can ride on. In this case, the moving body M may be provided with driving controls such as a steering wheel, an accelerator pedal, and a brake pedal so that the user U can drive the moving body M. Furthermore, at least a part of the speed control and turning control of the moving body M may be controlled by the user's operation (driving operation).

[0114] In addition, in the first and second embodiments, group tracking control in tracking control has been mainly described, but in addition to (or instead of) tracking control, group guidance control may also be performed when the moving body M is performing guidance control.

[0115] The above-described embodiment can be expressed as follows: A control device for a moving body, comprising: a storage medium for storing computer-readable instructions; and a processor connected to the storage medium, wherein the processor executes the computer-readable instructions to: recognize the surrounding situation of the moving body and a group consisting of at least one or more targets to be followed based on an image of the surrounding situation of the moving body; perform tracking control so that at least the moving body moves to follow the group based on the surrounding situation; set a virtual obstacle around the group; and perform the tracking control so as not to come into contact with the set virtual obstacle.

[0116] The above-described embodiment can also be expressed as follows: A control device for a moving body, comprising: a storage medium for storing computer-readable instructions; and a processor connected to the storage medium, wherein the processor executes the computer-readable instructions to: recognize a surrounding situation of the moving body and a group made up of at least one or more targets to be followed based on an image of the surrounding situation of the moving body; perform tracking control to move at least the moving body to follow the group based on the surrounding situation; set a tracking reference position based on the position of each target to be followed that makes up the group; and cause the moving body to follow the set tracking reference position.

[0117] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0118] REFERENCE SIGNS LIST 1 Mobile system 2 Terminal device 10 Management device 20 Information providing device 110 Base 160 Operation unit 170 Information output unit 180 Camera 190 Communication unit 200, 200A Control device 202 Position identification unit 204 Information processing unit 206 Recognition unit 208 Virtual obstacle setting unit 209 Following reference position setting unit 210 Trajectory generation unit 212 First control unit 214 Second control unit 220 Memory unit M Mobile body

Claims

1. A control device for a moving body, comprising: a recognition unit that recognizes the situation around a moving body and a group consisting of at least one or more targets to be followed based on an image of the situation around the moving body; and a control unit that executes tracking control so that at least the moving body moves to follow the group based on the situation around the moving body, wherein the control unit sets virtual obstacles around the group and executes the tracking control so as not to come into contact with the set virtual obstacles.

2. The control device for a moving body according to claim 1, wherein the virtual obstacle is set around the target to be followed that constitutes the group.

3. The control device for a moving body according to claim 2, wherein the virtual obstacle is set so as to surround the target to be followed that constitutes the group.

4. The control device for a moving body according to claim 2, wherein the virtual obstacle is set so as to surround the moving body and the target to be followed that constitute the group.

5. The control device for a mobile body according to claim 1, wherein the mobile body determines a tracking reference position based on the position of a person set as a user of the mobile body among the tracking targets that make up the group.

6. The control device for a moving body according to claim 3, wherein the shape of the area surrounded by the virtual obstacles is rectangular, and the size of the area is set based on the state of the target to be tracked that constitutes the group.

7. The control device for a moving body according to claim 6, wherein the area is made smaller as the distance between the plurality of tracking targets constituting the group becomes shorter.

8. The control device for a moving body according to claim 6, wherein, when the group is moving, the area is adjusted according to the direction and speed of movement.

9. The control device for a moving body according to claim 1, wherein the control unit causes the information output unit provided in each of the multiple moving bodies in the group to output the same information when the tracking target in the group uses a moving body.

10. The control device for a moving body according to claim 1, wherein the control unit, when the recognition unit is unable to recognize the target to be followed within the group, determines the next target to be followed within the group and makes the moving body follow the determined target to be followed.

11. The control device for a mobile body according to claim 10, wherein the next target to be followed is the target to be followed that is closest to the previous target to be followed among the plurality of targets to be followed that make up the group.

12. The control device for a mobile body according to claim 10, wherein the next target to be followed is the target to be followed that has the largest feature value for each target to be followed recognized by the recognition unit among the multiple targets to be followed that make up the group.

13. The control device for a mobile body according to claim 1, wherein the recognition unit recognizes the group based on a recognition result by the recognition unit or on pre-registered group information.

14. The control device for a moving body according to claim 1, wherein the control unit sets an offset distance from the tracking reference position based on the status of the group.

15. The control device for a mobile body according to claim 1, wherein the control unit follows the group at a position away from any of the following targets included in the group whose risk value based on the amount of movement in a predetermined period of time is equal to or greater than a threshold value.

16. The control device for a mobile body according to claim 1, wherein the control unit follows the group at a position where the recognition unit can easily recognize the target to be followed that has a risk value greater than or equal to a threshold based on the amount of movement in a predetermined period of time.

17. A method for controlling a moving body, in which a computer of a moving body recognizes the surrounding situation of the moving body and a group consisting of at least one or more objects to be followed based on an image of the surrounding situation of the moving body, performs tracking control so that at least the moving body moves to follow the group based on the surrounding situation, sets a virtual obstacle around the group, and performs the tracking control so as not to come into contact with the set virtual obstacle.

18. A program that causes a computer of a moving body to recognize the situation around the moving body and a group consisting of at least one or more objects to be followed based on an image of the situation around the moving body, execute a following control so that at least the moving body moves to follow the group based on the situation around the moving body, set a virtual obstacle around the group, and execute the following control so as not to come into contact with the set virtual obstacle.

Images