Control device, control method, and robot

Abstract

The present technique relates to a control device, a control method, and a robot with which enables a robot to easily and safely follow a target to be followed. A recognition unit recognizes an object present around a carriage robot. A selection unit selects the object as a target to be followed on the basis of movement information that is information relating to the movement of the object recognized by the recognition unit. A drive control unit causes the carriage robot to follow the target to be followed. The present technique can be applied to, for example, a control device or the like incorporated in the carriage robot, the control device or the like recognizing an object present around the carriage robot, selecting the object as a target to be followed, and controlling the carriage robot so as to follow the target to be followed.

Description

Control device, control method, and robot

[0001] The present technology relates to a control device, a control method, and a robot, and in particular to a control device, a control method, and a robot that enable a robot to easily and safely follow a target to be followed.

[0002] In recent years, human-following robots have been developed that have a tracking function that allows them to run while following a human (see, for example, Patent Document 1). Human-following robots detect humans and follow them. However, when a human-following robot uses, for example, a 2D LiDAR (Light Detection and Ranging) sensor for detecting its own position or obstacles as a human detection sensor, it tends to mistakenly detect the legs of a desk or chair as a human leg because it determines a small point cloud as a human leg.

[0003] When there are multiple people or objects that are mistakenly detected as people in the vicinity, a human-following robot may follow an object that the user does not intend to follow. Therefore, people around the human-following robot cannot predict the direction in which the human-following robot will move, which is dangerous.

[0004] Therefore, a human-following robot has been devised that has a selection unit that, when multiple objects detected as humans exist, sets each of the detected multiple objects as a candidate target to be followed and selects the candidate target intended by the user as the target to be followed.This selection unit selects the target to be followed based on the temperature of the candidate target detected by an infrared camera, the similarity between the captured image of the candidate target and the image of the target to be followed that has been registered in advance, etc.

[0005] However, such a human-following robot requires an infrared camera, a new UI (User Interface), and user operation, which increases the cost of the human-following robot and the burden of user operation.

[0006] On the other hand, with conventional human-following robots, it is difficult for the user to determine whether the human-following robot is following an object that the user does not intend, because the user does not have a means for recognizing the object that the human-following robot is following. Therefore, if the human-following robot is following an object that the user does not intend, the user needs to quickly stop the robot, but if the user does not do so quickly, there is a possibility that the stopping operation will be delayed and pose a danger to people around.

[0007] Japanese Patent Application Laid-Open No. 2006-146551

[0008] For these reasons, there is a demand for a method that can make a robot, such as a human-following robot, follow a target easily and safely, but this demand has not yet been fully met.

[0009] The present technology has been made in view of such circumstances, and makes it possible to make a robot follow a target easily and safely.

[0010] A control device according to a first aspect of the present technology is a control device including: a recognition unit that recognizes objects present around a robot; a selection unit that selects the object as a target to be followed based on movement information that is information relating to the movement of the object recognized by the recognition unit; and a drive control unit that causes the robot to follow the target to be followed.

[0011] A control method according to a first aspect of the present technology is a control method including: a control device recognizing an object present around a robot; selecting the object as a target to be tracked based on movement information that is information relating to the movement of the object; and causing the robot to track the target to be tracked.

[0012] In a first aspect of the present technology, objects present around a robot are recognized, and based on movement information that is information regarding the movement of the object, the object is selected as a target to be followed, and the robot follows the target to be followed.

[0013] A robot according to a second aspect of the present technology is a robot including: a recognition unit that recognizes objects present in the surrounding area; a selection unit that selects the object as a target to be tracked based on movement information that is information relating to the movement of the object recognized by the recognition unit; and a drive unit that drives the robot to track the target to be tracked selected by the selection unit.

[0014] In a second aspect of the present technology, an object present in the surroundings is recognized, and based on movement information that is information regarding the movement of the object, the object is selected as a target to be tracked, and driving is performed to track the target to be tracked.

[0015] A control device according to a third aspect of the present technology is a control device including: a recognition unit that recognizes objects present around a robot; a selection unit that selects the object recognized by the recognition unit as a target to be followed; and a drive control unit that rotates the robot toward the target to be followed before causing the robot to follow the target.

[0016] In a third aspect of the present technology, an object present around a robot is recognized, the object is selected as a target to be followed, and the robot rotates toward the target to be followed before following the target.

[0017] The control device according to the first and third aspects of the present technology can be realized by causing a computer to execute a program.

[0018] In addition, in order to realize the control device of the first and third aspects of the present technology, the program to be executed by a computer can be provided by transmitting it via a transmission medium or by recording it on a recording medium.

[0019] 1 is a diagram illustrating an overview of a first embodiment of a mobility control system to which the present technology is applied; FIG. 2 is a block diagram illustrating an example of the configuration of the mobility control system of FIG. 1; FIG. 3 is a diagram illustrating object recognition by the recognition unit of FIG. 2; FIG. 4 is a diagram illustrating an example of a detection list; FIG. 5 is a flowchart illustrating detection list generation processing in the first embodiment; and FIG. 6 is a diagram illustrating an example of information stored in the storage unit of FIG. 2. pred1 is a diagram illustrating estimation of (x[m], y[m]). FIG. 1 is a diagram illustrating an example of a moving object list in the first embodiment. FIG. 2 is a diagram illustrating matching. FIG. 3 is a diagram illustrating another example of a moving object list in the first embodiment. FIG. 4 is a flowchart illustrating management processing. FIG. 5 is a flowchart illustrating details of movement information update processing. FIG. 6 is a diagram illustrating a configuration example of an operation device. FIG. 7 is a diagram illustrating an example transition of operation modes. FIG. 8 is a diagram illustrating examples of operations of the cart robot and an indicator. FIG. 9 is a diagram illustrating user operation in the first embodiment. FIG. 10 is a flowchart illustrating operation wait processing. FIG. 11 is a flowchart illustrating processing for selecting a target to be followed in the first embodiment. FIG. 12 is a flowchart illustrating processing for preparing for following in the first embodiment. FIG. 13 is a flowchart illustrating details of processing for updating a target to be followed. FIG. 14 is a flowchart illustrating processing for following. FIG. 15 is a diagram illustrating a first example of the operation of the cart robot in the first embodiment. FIG. 16 is a diagram illustrating a second example of the operation of the cart robot in the first embodiment. FIG. 17 is a diagram illustrating a third example of the operation of the cart robot in the first embodiment. FIG. 18 is a diagram illustrating a fourth example of the operation of the cart robot in the first embodiment. A block diagram illustrating a configuration example of a second embodiment of a movement control system to which the present technology is applied. FIG. 19 is a diagram illustrating an example of a recognition score. FIG. 20 is a diagram illustrating an example of a detection list with recognition scores. FIG. 21 is a flowchart illustrating processing for generating a detection list in the second embodiment. FIG. 10 is a diagram showing an example of a moving object list in the second embodiment. FIG. 11 is a diagram explaining the operation of a user in the second embodiment. FIG. 12 is a flowchart explaining the process of selecting a target to be followed in the second embodiment. FIG. 13 is a flowchart explaining the process of preparing to follow in the second embodiment. FIG. 14 is a diagram showing an example of the operation of a cart robot in the second embodiment. FIG. 15 is a block diagram showing an example of the configuration of computer hardware.

[0020] Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described. The description will be made in the following order: 1. First embodiment (control device that selects a target to be tracked based on an accumulated movement amount) 2. Second embodiment (control device that selects a target to be tracked based on a recognition score) 3. Computer

[0021] 1. First Embodiment Overview of a Mobility Control System FIG. 1 is a diagram showing an overview of a mobility control system according to a first embodiment of the present technology.

[0022] The movement control system 10 in Fig. 1 is composed of a cart robot 11 and an operating device 12. In the movement control system 10 in Fig. 1, a user 21 moves while holding the operating device 12, and the cart robot 11 follows the user 21.

[0023] Specifically, the cart robot 11 is a robot that can carry luggage and follows a movable object such as a human being. The cart robot 11 has a built-in control device 31, a sensor 32, an indicator 33, a drive unit 34, etc.

[0024] The control device 31 controls the drive unit 34 based on operation signals representing operations by the user 21 transmitted by wireless communication from the operation device 12, detection results of the sensor 32, etc., to rotate the cart robot 11 toward the user 21 and make it follow the user 21. The control device 31 controls the illumination of the indicator 33 according to the operation mode of the cart robot 11.

[0025] The sensor 32 is configured with a LiDAR, a camera, etc. Note that the sensor 32 may be any sensor as long as it can acquire sensor data capable of detecting the positions of surrounding objects. The sensor 32 acquires sensor data of objects such as the user 21 present around the cart robot 11.

[0026] The indicator 33 emits light under the control of the control device 31 to notify the user 21 of the operation mode of the cart robot 11. It should be noted that the cart robot 11 does not necessarily have to be equipped with the indicator 33. The drive unit 34 is composed of a motor, a motor control unit, etc., and drives the cart robot 11 under the control of the control device 31.

[0027] The operation device 12 is a remote controller that accepts operations of the user 21 on the cart robot 11. The operation device 12 accepts operations of the user 21 on the cart robot 11 and transmits operation signals representing the operations to the control device 31 via wireless communication. Note that the operation device 12 may not be a remote controller and may be integrated with the cart robot 11.

[0028] <Configuration Example of Mobility Control System> FIG. 2 is a block diagram showing a configuration example of the mobility control system 10 of FIG.

[0029] As shown in FIG. 2, the control device 31 of the movement control system 10 includes a recognition unit 50, a self-position estimation unit 51, a detection unit 52, a moving object management unit 53, a memory unit 54, a control unit 55, and a drive control unit 56.

[0030] The recognition unit 50 receives sensor data acquired by the sensor 32. Based on the sensor data, the recognition unit 50 recognizes predetermined objects, such as humans, present around the cart robot 11. Specifically, for example, if the sensor 32 is a LiDAR, the recognition unit 50 recognizes the predetermined object by clustering point cloud data as sensor data. If the sensor 32 is an RGB-D camera, the recognition unit 50 recognizes the predetermined object by bone detection using RGBD data as sensor data. Note that any algorithm can be used as the object recognition algorithm used by the recognition unit 50.

[0031] The recognition unit 50 generates a list of sensor coordinate system positions, which are positions in the sensor coordinate system of each recognized object, as a detection list, and supplies the list to the moving object management unit 53 .

[0032] The self-position estimation unit 51 receives sensor data acquired by the sensor 32 as input. The self-position estimation unit 51 uses the sensor data to estimate its own position and create an environmental map, i.e., performs SLAM (Simultaneous Localization and Mapping), for example. Based on the results of SLAM, the self-position estimation unit 51 generates conversion information used for conversion from the sensor coordinate system to a world coordinate system, a world coordinate position that is the position of the cart robot 11 in the world coordinate system, and the like. The self-position estimation unit 51 supplies the conversion information to the moving object management unit 53.

[0033] The detection unit 52 receives sensor data acquired by the sensor 32. The detection unit 52 uses the sensor data to create an occupancy grid map and detects obstacles. The detection unit 52 supplies obstacle information, including the position of the obstacle in the world coordinate system, to the drive control unit 56.

[0034] The moving object management unit 53 manages the moving object list stored in the memory unit 54 based on the detection list supplied from the recognition unit 50, the conversion information and world coordinate position of the cart robot 11 supplied from the self-position estimation unit 51, and the parameter list stored in the memory unit 54.

[0035] Specifically, the moving object management unit 53 converts the sensor coordinate system position of each object registered in the detection list into a world coordinate system position using the conversion information. The moving object management unit 53 matches each object based on the world coordinate system position of each object registered in the detection list and the world coordinate system position of each object registered in the moving object list. The moving object management unit 53 updates the moving object list based on the matching result, the detection list, and the parameter list. The moving object management unit 53 then notifies the control unit 55 of the update of the moving object list. The moving object list is a list of movement information that is information regarding the movement of the world coordinate system position, etc. of each object recognized by the recognition unit 50.

[0036] The storage unit 54 stores a moving object list, target information for identifying a target to be tracked supplied from the control unit 55, a parameter list, and the like.

[0037] The control unit 55 sets the operation mode based on the operation signal sent from the operation device 12, the notification supplied from the moving object management unit 53, and the parameter list stored in the memory unit 54. The operation modes include an operation wait mode, a target to be followed selection mode, a follow preparation mode, and a follow mode. The operation wait mode is an operation mode in which the robot waits to receive an operation signal representing an operation to start following. The target to be followed selection mode is an operation mode in which the target to be followed is selected. The follow preparation mode is an operation mode in which the robot rotates the cart robot 11 toward the target to be followed. The follow mode is an operation mode in which the cart robot 11 follows the target to be followed.

[0038] When the operation mode is the follow-up target selection mode, the control unit 55 selects a follow-up target based on the moving object list stored in the memory unit 54. Then, the control unit 55 supplies follow-up target information of the follow-up target to the memory unit 54 for storage. When the operation mode is the follow-up preparation mode, the control unit 55 calculates a rotation angle required to orient the cart robot 11 toward the follow-up target based on the moving object list, and supplies the calculated angle to the drive control unit 56. When the operation mode is the follow-up mode, the control unit 55 calculates a movement amount of the cart robot 11 required to cause the cart robot 11 to follow the follow-up target based on the moving object list, and supplies the calculated amount to the drive control unit 56.

[0039] The control unit 55 notifies the user 21 of the operation mode by controlling the light emission of the indicator 33 in accordance with the operation mode.

[0040] The drive control unit 56 performs PID (Proportional-Integral-Differential) control on the drive unit 34 based on the rotation angle supplied from the control unit 55, causing the cart robot 11 to rotate toward the target to be followed. The drive control unit 56 creates a plan for the trajectory of the cart robot 11 to follow the target to be followed while avoiding the obstacle, based on the obstacle information supplied from the detection unit 52 and the movement amount supplied from the control unit 55. The drive control unit 56 performs PID control on the drive unit 34 based on this plan, causing the cart robot 11 to follow the target to be followed while avoiding the obstacle.

[0041] <Explanation of Object Recognition by Recognition Unit> FIG. 3 is a diagram for explaining object recognition by the recognition unit 50 of FIG.

[0042] 3, the recognition unit 50 recognizes four objects 71 to 74 around the cart robot 11 as humans based on the sensor data, and assigns identification numbers to each of the objects 71 to 74 in order starting from 1. The recognition unit 50 detects the sensor coordinate system position (X[m], Y[m]) of each of the objects 71 to 74 based on the sensor data.

[0043] <Example of Detection List> FIG. 4 is a diagram showing an example of a detection list.

[0044] In the example of Fig. 4, objects 71 to 74 in Fig. 3 are recognized by the recognition unit 50. In this case, the recognition unit 50 generates a detection list in which sensor coordinate system positions (X[m], Y[m]) are registered in association with the identification numbers assigned to the objects 71 to 74, as shown in Fig. 4.

[0045] 4, the sensor coordinate system position (X[m], Y[m]) of object 71 assigned identification number "1" is (2.3, 0.4), and the sensor coordinate system position (X[m], Y[m]) of object 72 assigned identification number "2" is (0.8, -0.27). The sensor coordinate system position (X[m], Y[m]) of object 73 assigned identification number "3" is (0.3, 0.2), and the sensor coordinate system position (X[m], Y[m]) of object 74 assigned identification number "4" is (1.3, 1.2).

[0046] <Description of Detection List Generation Process> Fig. 5 is a flowchart illustrating the detection list generation process for generating a detection list by the recognition unit 50 in Fig. 2. This detection list generation process is started, for example, when sensor data is input from the sensor 32.

[0047] 5, the recognition unit 50 determines whether the operation mode is the operation waiting mode or not. If it is determined in step S11 that the operation mode is not the operation waiting mode, the process proceeds to step S12.

[0048] In step S12, the recognition unit 50 recognizes predetermined objects, such as humans, present around the cart robot 11 based on the sensor data. In step S13, the recognition unit 50 generates a detection list based on the sensor coordinate system position (X[m], Y[m]) of each object recognized by the processing in step S12, and supplies the list to the moving object management unit 53. Then, the detection list generation processing ends.

[0049] On the other hand, if it is determined in step S11 that the operation waiting mode is in effect, the processes of steps S12 and S13 are not performed, and the detection list generation process ends.

[0050] <Example of Information Stored in Storage Unit> FIG. 6 is a diagram showing an example of information stored in the storage unit 54 of FIG.

[0051] As shown in FIG. 6, the storage unit 54 stores a moving object list, tracking target information, and a parameter list.

[0052] The moving object list registers the movement information of each object recognized by the recognition unit 50. Specifically, the moving object list stores the initial position P init (x[m], y[m]), previous position P prev (x[m], y[m]), predicted value P pred (x[m], y[m]), current position P(x[m], y[m]), cumulative movement amount [m], relative distance to the cart robot 11 [m], following candidate flag, and lost flag are registered.

[0053] The moving object number is an identification number, for example, 0, that is assigned to each object recognized by the recognition unit 50 when the movement information of that object is registered in the moving object list.

[0054] Initial position P init (x[m], y[m]) is the world coordinate system position of the object when it is first recognized after the operation mode is set to a mode other than the operation waiting mode, and is, for example, (0.5, 0.3). prev (x[m], y[m]) is the world coordinate system position of the object when it was last recognized, for example (2.2, 0.5). init(x[m], y[m]) and previous position P prev (x[m], y[m]) is obtained by the moving object management unit 53 using the conversion information generated by the self-position estimation unit 51 to convert the sensor coordinate system position of the object registered in the detection list into a world coordinate system position.

[0055] Predicted value P pred (x[m], y[m]) is the predicted value of the object's world coordinate system position when it is recognized this time, for example (2.32, 0.45). The current position P(x[m], y[m]) is the actual world coordinate system position of the object when it is recognized this time, for example (2.3, 0.4).

[0056] The cumulative movement amount [m] is calculated by the previous position P prev (x[m], y[m]) and the current position P(x[m], y[m]), that is, the difference between the previous position P prev The relative distance [m] from the cart robot 11 is the distance [m] between the cart robot 11 and an object, calculated by the moving object management unit 53 based on the current position P(x[m], y[m]) and the world coordinate system position of the cart robot 11, and is, for example, 2.3.

[0057] The follow candidate flag is a flag that indicates whether the cumulative movement amount [m] has exceeded the follow preparation threshold [m] registered in the parameter list. The follow candidate flag is "true" if it indicates that the cumulative movement amount [m] has exceeded the follow preparation threshold [m], and is "false" if it indicates that the cumulative movement amount [m] has not exceeded the follow preparation threshold [m].

[0058] The lost flag is a flag that indicates whether or not an object is no longer recognized by the recognition unit 50. The lost flag is "true" if it indicates that the object is no longer recognized, and is "false" if it indicates that the object is not no longer recognized.

[0059] The tracking target information is the moving object number of the tracking target, which is 0, for example.

[0060] The parameter list registers an error judgment threshold [m], a tracking preparation threshold [m], and a tracking start threshold [m]. The error judgment threshold [m] is a threshold used by the moving object management unit 53 to determine whether or not the change in the world coordinate system position over time of an object assigned a moving object number is within an error range. The tracking preparation threshold [m] is a threshold used by the moving object management unit 53 to set a tracking candidate flag. The tracking start threshold [m] is a threshold used by the control unit 55 to determine whether or not to transition the operation mode to a tracking mode. The error judgment threshold [m], the tracking preparation threshold [m], and the tracking start threshold [m] are, for example, 0.05, 2.0, and 4.0, respectively.

[0061] <Explanation of Estimation of Predicted Value> FIG. 7 shows the predicted value P pred FIG. 10 is a diagram illustrating the estimation of (x[m], y[m]).

[0062] 7, the moving object list contains the moving information of objects 91 to 93. In this case, the moving object management unit 53 calculates the initial positions P init (x[m], y[m]), previous position P prev Based on (x[m], y[m]), etc., a predicted value P for each of the objects 91 to 93 is calculated using a method such as a Kalman filter. pred Estimate (x[m], y[m]).

[0063] For example, as shown in FIG. 7, the moving object management unit 53 determines the initial position P init (0.5,0.3), previous position P prev Based on (2.2,0.5), the predicted value P pred The moving object management unit 53 estimates the initial position P init (0.3,0.2), previous position P prev Based on (0.4, 0.3), the predicted value P pred The moving object management unit 53 estimates the initial position P init (0.8,-0.3), previous position P prev Based on (0.8,-0.3), the initial position P init and the previous position P prevThe same predicted value P pred The estimate is (0.8,-0.3).

[0064] <Example of Moving Object List> FIG. 8 shows the predicted values ​​P pred FIG. 10 is a diagram showing an example of a moving object list in which (x[m], y[m]) is registered.

[0065] In the example of FIG. 8, the moving object management unit 53 assigns moving object numbers 0 to 2 to the objects 91 to 93 in order.

[0066] Therefore, in the moving object list of FIG. 8, the initial position P init (x[m], y[m]), previous position P prev (x[m], y[m]), and the predicted value P pred As (x[m], y[m]), (0.5, 0.3), (2.2, 0.5), and (2.32, 0.45) are registered, respectively. Nothing is registered as the current position P(x[m], y[m]) corresponding to moving object number "0". 3.1 is registered as the accumulated movement amount [m] corresponding to moving object number "0", and 2.3 is registered as the relative distance [m] from the cart robot 11. "True" is registered as the following candidate flag corresponding to moving object number "0", and "false" is registered as the lost flag.

[0067] The initial position P corresponding to the moving object number "1" init (x[m], y[m]), previous position P prev (x[m], y[m]), and the predicted value P pred As (x[m], y[m]), (0.3, 0.2), (0.4, 0.3), and (0.42, 0.32) are registered, respectively. Nothing is registered as the current position P(x[m], y[m]) corresponding to moving object number "1". 0.28 is registered as the accumulated movement amount [m] corresponding to moving object number "1", and 0.5 is registered as the relative distance [m] from the cart robot 11. "false" is registered as the following candidate flag and lost flag corresponding to moving object number "1".

[0068] The initial position P corresponding to the moving object number "2" init(x[m], y[m]), previous position P prev (x[m], y[m]), and the predicted value P pred All of the values ​​(0.8, -0.3) are registered as (x[m], y[m]). Nothing is registered as the current position P(x[m], y[m]) corresponding to moving object number "2". 0 is registered as the accumulated movement amount [m] corresponding to moving object number "2", and 0.85 is registered as the relative distance [m] from the cart robot 11. "false" is registered as the following candidate flag and lost flag corresponding to moving object number "2".

[0069] <Explanation of Matching> FIG. 9 is a diagram for explaining matching by the moving object management unit 53 between the objects 91 to 93 registered in the moving object list of FIG. 8 and the objects registered in the detection list.

[0070] 9, the horizontal axis represents x [m] and the vertical axis represents y [m]. The predicted values ​​P pred (x[m], y[m]) is as shown in A of FIG.

[0071] In this case, when a detection list in which sensor coordinate system positions corresponding to the world coordinate system positions of the objects 102 to 104 shown in FIG. 9B is registered is supplied from the recognition unit 50, the moving object management unit 53 matches each of the objects 91 to 93 with each of the objects 102 to 104. Specifically, the moving object management unit 53 uses the conversion information to convert the sensor coordinate system positions of the objects 102 to 104 registered in the detection list into world coordinate system positions (0.8, -0.27), (2.3, 0.4), and (1.3, 1.2), respectively. The moving object management unit 53 calculates the predicted values ​​P pred Based on (2.32, 0.45), (0.42, 0.32), (0.8, -0.3) and the world coordinate system positions (0.8, -0.27), (2.3, 0.4), (1.3, 1.2) of the objects 102 to 104, matching (linking) is performed using a method such as GNN (Global Nearest-Neighbor).

[0072] As a result, in the example of FIG. 9, objects 93 and 102 are matched, and objects 91 and 103 are matched.

[0073] <Example of Updated Moving Object List> FIG. 10 is a diagram showing an example of a moving object list updated from the moving object list of FIG. 8 based on the matching results described with reference to FIG.

[0074] 9, the moving object management unit 53 recognizes the object 103 that has matched with the object 91 as the current object 91. Then, as shown in FIG. 10, the moving object management unit 53 registers the world coordinate system position (2.3, 0.4) of the object 103 as the current position P(x[m], y[m]) corresponding to the moving object number "0" of the object 91.

[0075] Here, the current position P (2.3, 0.4) of the object 91 is changed to the previous position P prev The distance dist[m] to (2.2, 0.5) is 0.14. Therefore, for example, if the value of the error judgment threshold [m] registered in the parameter list is 0.05 as shown in Figure 6, the distance dist[m] exceeds the error judgment threshold [m], and the moving object management unit 53 determines that the distance dist[m] is not an error. The moving object management unit 53 then adds the distance dist[m] to the cumulative movement amount [m], thereby updating the cumulative movement amount [m] to 3.24 (= 3.1 + 0.14).

[0076] The moving object management unit 53 updates the relative distance [m] to 2.3 from the cart robot 11 based on the current position P (2.3, 0.4) of the object 91 and the world coordinate system position of the cart robot 11. Since the following candidate flag of the object 91 is already "true" and the accumulated movement amount [m] exceeds the following preparation threshold [m], the moving object management unit 53 does not update the following candidate flag of the object 91.

[0077] 9, the moving object management unit 53 changes the lost flag corresponding to the moving object number "1" of the object 92 to "true." As a result, the movement information of the object 92 is finally deleted from the moving object list by the moving object management unit 53.

[0078] 9, the moving object management unit 53 recognizes the object 102 that has matched with the object 93 as the current object 93. Then, the moving object management unit 53 registers the world coordinate system position (0.8, -0.27) of the object 102 as the current position P(x[m], y[m]) corresponding to the moving object number "2" of the object 93, as shown in FIG.

[0079] Here, the current position P (0.8, -0.27) of the object 93 is changed to the previous position P prev The distance dist[m] to (0.8, -0.3) is 0.03. Therefore, for example, if the value of the error judgment threshold [m] registered in the parameter list is 0.05 as shown in FIG. 6, the distance dist[m] does not exceed the error judgment threshold [m], so the moving object management unit 53 determines that the distance dist[m] is an error and does not update the cumulative movement amount [m]. Therefore, the moving object management unit 53 does not update the tracking candidate flag either. The moving object management unit 53 updates the relative distance [m] to 0.85 from the cart robot 11 based on the current position P (0.8, -0.27) of the object 93 and the world coordinate system position of the cart robot 11.

[0080] Even if the distance dist [m] does not exceed the error determination threshold [m], the moving object management unit 53 does not need to update the relative distance [m] from the cart robot 11 .

[0081] 9, the moving object management unit 53 recognizes the object 104 that has not been matched with any of the objects 91 to 93 as a new object 104. Then, the moving object management unit 53 assigns a new moving object number "3" to this object 104, and adds the movement information of the object 104 to the movement information list as shown in FIG.

[0082] Specifically, the moving object management unit 53 determines the initial position P init The moving object management unit 53 registers the world coordinate system position (1.3, 1.2) of the object 104 as the previous position P(x[m], y[m]) and the current position P(x[m], y[m]). prev (x[m], y[m]) and predicted value P predNothing is yet registered as (x[m], y[m]). The moving object management unit 53 registers 0 as the cumulative movement amount [m]. The moving object management unit 53 registers 1.77 as the relative distance [m] to the cart robot 11 based on the current position P(1.3, 1.2) and the world coordinate system position of the cart robot 11. The moving object management unit 53 registers the initial value "false" as the following candidate flag and lost flag corresponding to the moving object number "4".

[0083] In this manner, the moving object management unit 53 updates the moving object list of FIG. 8 to the moving object list of FIG. 10 based on the matching results described with reference to FIG.

[0084] 11 is a flowchart illustrating the management process for managing the moving object list by the moving object management unit 53. This management process is started, for example, when a detection list is supplied from the recognition unit 50 to the moving object management unit 53 as a result of the detection list generation process of FIG.

[0085] 11, the moving object management unit 53 sets the count value i to 1. In step S32, the moving object management unit 53 matches the i-th object among N objects (N is an integer equal to or greater than 1) whose sensor coordinate system positions are registered in the detection list with each object whose movement information is registered in the moving object list.

[0086] In step S33, the moving object management unit 53 determines whether or not the i-th object matches any of the objects registered in the moving object list as a result of the processing in step S32. If it is determined in step S33 that they match, the processing proceeds to step S34.

[0087] In step S34, the moving object management unit 53 performs a movement information update process to update the movement information of the object registered in the moving object list and matched with the i-th object, as described in Fig. 10. Details of this movement information update process will be described later with reference to Fig. 12.

[0088] On the other hand, if it is determined in step S33 that there is no match, the process proceeds to step S35. In step S35, the moving object management unit 53 newly registers the movement information of the i-th object in the moving object list, as described in FIG.

[0089] After processing step S34 or S35, in step S36, the moving object management unit 53 determines whether or not the count value i is equal to or greater than N. If it is determined in step S36 that the count value i is not equal to or greater than N, that is, if the sensor system coordinate position of an object for which matching has not yet been performed is registered in the detection list, the processing proceeds to step S37.

[0090] In step S37, the moving object management section 53 increments the count value i by 1, and returns the process to step S32.

[0091] On the other hand, if it is determined in step S36 that the count value i is equal to or greater than N, i.e., if matching has been performed for all objects registered in the detection list, the processing proceeds to step S38. In step S38, the moving object management unit 53 updates the lost flag of any object registered in the moving object list that has not been matched with any object registered in the detection list to "true." In step S39, the moving object management unit 53 notifies the control unit 55 that the moving object list has been updated, i.e., that an object has been recognized. Then, the management processing ends.

[0092] Note that the lost flag of an object registered in the moving object list that has not been matched a predetermined number of times or more may be updated to "true." In this case, for example, an item "number of matching failures" is added to the moving object list described with reference to FIG. 8 and the like, and an initial value of 0 is set. Instead of the process of step S38, the moving object management unit 53 increments the number of matching failures by 1 for an object registered in the moving object list that has not been matched with any object registered in the detection list. Then, the moving object management unit 53 updates the lost flag of an object whose number of matching failures is a predetermined number or more to "true."

[0093] <Description of Movement Information Update Process> FIG. 12 is a flowchart illustrating the details of the movement information update process in step S34 of FIG.

[0094] In step S50 of Figure 12, the moving object management unit 53 updates the current position P (x[m], y[m]) of the object matched with the i-th object and the relative distance [m] from the cart robot 11 based on the world coordinate system positions of the i-th object and the cart robot 11.

[0095] In step S51, the moving object management unit 53 calculates the current position P(x[m], y[m]) of the object matched with the i-th object from the previous position P prev Calculate the distance dist[m] to (x[m], y[m]).

[0096] In step S52, the moving object management unit 53 determines whether the distance dist[m] exceeds the error determination threshold [m] registered in the parameter list. If it is determined in step S52 that the distance dist[m] exceeds the error determination threshold [m], the moving object management unit 53 determines that the distance dist[m] is not an error, and proceeds to step S53.

[0097] In step S53, the moving object management unit 53 adds the distance dist[m] to the cumulative movement amount [m] of the object matched with the i-th object, and proceeds to step S54.

[0098] On the other hand, if it is determined in step S52 that the distance dist[m] does not exceed the error determination threshold [m], the moving object management unit 53 determines that the distance dist[m] is an error and skips step S53. That is, the moving object management unit 53 does not update the cumulative movement amount [m]. The process then proceeds to step S54.

[0099] In step S54, the moving object management unit 53 determines whether the cumulative movement amount [m] of the object matched with the i-th object exceeds the tracking preparation threshold [m] registered in the parameter list. If it is determined in step S54 that the cumulative movement amount [m] exceeds the tracking preparation threshold [m], the process proceeds to step S55.

[0100] In step S55, the moving object management unit 53 sets the following candidate flag of the object matched with the i-th object to "true", and the process proceeds to step S56.

[0101] On the other hand, if it is determined in step S54 that the cumulative movement amount [m] does not exceed the tracking preparation threshold [m], the moving object management unit 53 skips step S55. That is, the tracking candidate flag of the object matched with the i-th object remains "false." Then, the process proceeds to step S56.

[0102] In step S56, the moving object management unit 53 calculates the previous position P prev (x[m], y[m]) is overwritten with the current position P(x[m], y[m]). In step S57, the moving object management unit 53 deletes the current position P(x[m], y[m]) of the object matched with the i-th object, returns the process to step S34 in Figure 11, and proceeds to step S36.

[0103] <Configuration Example of Operation Device> FIG. 13 is a diagram showing a configuration example of the operation device 12. As shown in FIG.

[0104] As shown in FIG. 13, the operation device 12 is provided with a cancel button 121, a start button 122, and a confirm button 123 as a UI.

[0105] The cancel button 121 is operated by the user 21, for example, when interrupting the following operation of the cart robot 11. The start button 122 is operated by the user 21, for example, when starting the following operation of the cart robot 11. The confirm button 123 is operated by the user 21, for example, when confirming the target to be followed.

[0106] <Example of Transition of Operation Mode> FIG. 14 is a diagram showing an example of transition of operation mode set by the control unit 55. In FIG.

[0107] 14, the operation modes include an operation waiting mode 141, a follow-up target selection mode 142, a follow-up preparation mode 143, and a follow-up mode 144. The operation waiting mode 141 is the initial value of the operation mode.

[0108] When the operation mode is the operation wait mode 141, the cart robot 11 waits in a stopped state until the start button 122 is operated by the user 21. When the start button 122 is operated by the user, the operation mode transitions to the follow-up target selection mode 142.

[0109] When the operation mode is the follow-up target selection mode 142, the cart robot 11 remains stopped and waits until at least one follow-up candidate flag in the moving object list becomes "true." When at least one follow-up candidate flag becomes "true," one object corresponding to the "true" follow-up candidate flag is selected as the follow-up target. Then, the operation mode transitions to a follow-up preparation mode 143. On the other hand, when an object unintended by the user 21 is selected as the follow-up target, for example, and the user 21 operates the cancel button 121, the operation mode transitions to an operation waiting mode 141. Note that when an object unintended by the user 21 is selected as the follow-up target, the user 21 can change the follow-up target by moving more.

[0110] When the operation mode is the follow-up preparation mode 143, the cart robot 11 rotates, while stopped, toward the target to be followed that was selected in the target to be followed selection mode 142. This allows the user 21 to recognize the target to be followed. When the relative distance [m] from the cart robot 11 exceeds the follow-up start threshold [m] or the user 21 operates the confirm button 123, the target to be followed is confirmed and the operation mode transitions to the follow mode 144. On the other hand, when the lost flag of the target to be followed becomes "true", the operation mode transitions to the target to be followed selection mode 142. When the user 21 operates the cancel button 121, the operation mode transitions to the operation wait mode 141.

[0111] When the operation mode is the follow mode 144, the cart robot 11 follows the object to be followed. Then, when the lost flag of the object to be followed becomes "true" or the user 21 operates the cancel button 121, the operation mode transitions to the operation waiting mode 141.

[0112] <Examples of Operation of Cart Robot and Indicator in Each Operation Mode> FIG. 15 is a diagram showing examples of operation of the cart robot 11 and the indicator 33 in each operation mode.

[0113] 15, the cart robot 11 is stopped when the operation mode is the operation waiting mode 141. In this case, the indicator 33 is turned off to notify the user 21 that the operation of the start button 122 is waiting.

[0114] When the operation mode is the follow-up target selection mode 142, the cart robot 11 is stopped, but the indicator 33 flashes yellow, thereby notifying the user 21 that a follow-up target is being selected.

[0115] When the operation mode is the follow-up preparation mode 143, the cart robot 11 rotates toward the target to be followed while remaining stationary, and maintains this state. In this case, the indicator 33 lights up yellow, indicating that the cart robot 11 is waiting for the target to be determined.

[0116] When the operation mode is the following mode 144, the cart robot 11 follows the target, and the indicator 33 lights up in green. This notifies the user 21 that the cart robot 11 is following the target.

[0117] <Explanation of User's Action> FIG. 16 is a diagram illustrating the action of the user 21 when the user 21 makes the cart robot 11 follow the user 21. In FIG.

[0118] As shown in A of Fig. 16, first, the user 21 stands in front of the cart robot 11 and operates the start button 122. This causes the operation mode to transition from an operation waiting mode 141 to a tracking target selection mode 142, and the detection list generation process of Fig. 5 and the management process of Fig. 11 are performed. As a result, a moving object list is generated in which the movement information of the user 21 is registered. At this time, the indicator 33 is flashing yellow.

[0119] Next, as shown in B of Fig. 16, the user 21 moves in a direction away from the cart robot 11. As a result, the moving object list is updated by the detection list generation process and management process in accordance with the movement of the user 21. When the cumulative movement amount [m] of the user 21 after the update exceeds the follow-up preparation threshold [m] and the follow-up candidate flag becomes "true", the user 21 is selected as the follow-up target. Then, the operation mode transitions to the follow-up preparation mode 143, and the indicator 33 starts to light up yellow.

[0120] When the user 21 recognizes that the operation mode has transitioned to the follow-up preparation mode 143 by the indicator 33 lighting up yellow, the user 21 moves left or right relative to the cart robot 11. For example, as shown in C of Fig. 16, when the user 21 moves leftward relative to the cart robot 11, the cart robot 11 rotates rightward relative to the user 21 while remaining stopped.

[0121] As a result, when the user 21 recognizes that he or she is the target to be followed, the user 21 moves in a direction away from the cart robot 11, as shown in D of Fig. 16. As a result, the moving object list is updated by the detection list generation process and management process in accordance with the movement of the user 21. When the relative distance [m] between the user 21 and the cart robot 11 after the update exceeds the following start threshold [m], the target to be followed is confirmed, and the operation mode transitions to the following mode 144. Then, the cart robot 11 starts following the user 21, and the indicator 33 lights up green.

[0122] When the user 21 recognizes that he or she is the target to be followed, he or she may operate the confirm button 123 instead of moving in a direction away from the cart robot 11. In this case, the target to be followed is confirmed, and the same processing as when the relative distance [m] from the cart robot 11 exceeds the following start threshold [m] is performed.

[0123] As described above, the operation mode transitions to the follow preparation mode 143 before the follow mode 144, and the cart robot 11 simply rotates toward the user 21 selected as the object to follow, and does not start following until the object to follow is confirmed and the operation mode transitions to the follow mode 144. This makes it possible to prevent the cart robot 11 from starting to follow an object unintended by the user 21.

[0124] The actions of the user 21 may be instructed by a speaker, a display unit, or the like (not shown) provided on the cart robot 11. In this case, for example, when the operation mode is the operation waiting mode 141, a message saying "Please stand in front of the cart robot" is output by image or sound. When the operation mode is the follow-up target selection mode 142, a message saying "Please move in a direction away from the cart robot" is output by image or sound. When the operation mode is the follow-up preparation mode 143, a message saying "Please move left or right" is output by image or sound.

[0125] 16 , when the operation mode is the follow-up target selection mode 142, the user 21 moves in a direction away from the cart robot 11, but the direction of movement of the user 21 is not limited to this direction. Note that when the user 21 moves in a direction away from the cart robot 11, it is possible to prevent the user 21 from moving out of the detection range of the sensor 32 and becoming unrecognized by the recognition unit 50.

[0126] 17 is a flowchart illustrating the operation waiting process performed by the control unit 55 when the operation mode is the operation waiting mode 141. This operation waiting process is performed, for example, at predetermined time intervals.

[0127] 17, the control unit 55 controls the illumination of the indicator 33 and turns off the indicator 33. In step S72, the control unit 55 determines whether or not the start button 122 has been operated, based on an operation signal supplied from the operation device 12. If it is determined in step S72 that the start button 122 has been operated, the process proceeds to step S73. In step S73, the control unit 55 sets the operation mode to the tracking target selection mode 142 and ends the operation waiting process.

[0128] On the other hand, if it is determined in step S72 that the start button 122 has not been operated, step S73 is skipped. That is, the operation mode remains in the operation waiting mode 141. Then, the operation waiting process ends.

[0129] 18 is a flowchart illustrating the process of selecting a target to be followed that is performed by the control unit 55 when the operation mode is the target to be followed mode 142. This process of selecting a target to be followed is started, for example, when a notification is sent from the moving object management unit 53 to the control unit 55 by the process of step S39 in FIG.

[0130] 18, the control unit 55 controls the light emission of the indicator 33 to cause the indicator 33 to flash in yellow. In step S92, the control unit 55 determines whether or not the cancel button 121 has been operated, based on an operation signal supplied from the operation device 12. If it is determined in step S92 that the cancel button 121 has been operated, the process proceeds to step S93. In step S93, the control unit 55 sets the operation mode to the operation wait mode 141, and ends the tracking target selection process.

[0131] On the other hand, if it is determined in step S92 that the cancel button 121 has not been operated, the control unit 55 selects, from the objects registered in the moving object list, an object whose tracking candidate flag is "true" as a tracking target candidate, based on the tracking candidate flag. Then, in step S94, the control unit 55 determines whether or not one or more tracking target candidates exist. If it is determined in step S94 that one or more tracking target candidates exist, the process proceeds to step S95.

[0132] In step S95, the control unit 55 (selection unit) selects, as the tracking target, the tracking target candidate with the largest cumulative movement amount [m] from the one or more tracking target candidates based on the cumulative movement amount [m] of the one or more tracking target candidates. For example, if there are multiple tracking target candidates, the control unit 55 selects, as the tracking target, the tracking target candidate with the largest cumulative movement amount [m] from the multiple tracking target candidates based on the cumulative movement amount [m] of the multiple tracking target candidates. The control unit 55 registers the moving object number of the tracking target in the parameter list as tracking target information. In step S96, the control unit 55 sets the operation mode to tracking preparation mode 143 and ends the tracking target selection process.

[0133] On the other hand, if it is determined in step S94 that one or more candidates for the follow-up target do not exist, steps S95 and S96 are skipped and the follow-up target selection process ends.

[0134] 19 is a flowchart illustrating the following preparation process performed by the control unit 55 when the operation mode is the following preparation mode 143. This following preparation process is started, for example, when the moving object management unit 53 notifies the control unit 55 by the process of step S39 in FIG.

[0135] 19, the control unit 55 controls the light emission of the indicator 33 to light up in yellow. In step S112, the control unit 55 determines whether or not the cancel button 121 has been operated, based on an operation signal supplied from the operation device 12. If it is determined in step S112 that the cancel button 121 has been operated, the process proceeds to step S113. In step S113, the control unit 55 sets the operation mode to the operation wait mode 141, and ends the tracking preparation process.

[0136] On the other hand, if it is determined in step S112 that the cancel button 121 has not been operated, the process proceeds to step S114. In step S114, the control unit 55 determines whether or not the lost flag corresponding to the moving object number indicated by the tracking target information registered in the moving object list, i.e., the lost flag of the tracking target, is "false."

[0137] If it is determined in step S114 that the lost flag is "false," the process proceeds to step S115. In step S115, the control unit 55 determines whether the distance to the cart robot 11 to be followed, i.e., the relative distance [m] to the cart robot 11 corresponding to the moving object number indicated by the information on the cart robot to be followed, exceeds the following start threshold [m].

[0138] If it is determined in step S115 that the tracking start threshold [m] is exceeded, the control unit 55 confirms the tracking target and proceeds to step S116. In step S116, the control unit 55 sets the operation mode to tracking mode 144 and ends the tracking preparation process.

[0139] On the other hand, if it is determined in step S115 that the tracking start threshold [m] has not been exceeded, the process proceeds to step S117. In step S117, the control unit 55 determines whether or not the confirmation button 123 has been operated, based on the operation signal supplied from the operation device 12. If it is determined in step S117 that the confirmation button 123 has been operated, the control unit 55 confirms the tracking target, and the process proceeds to step S116.

[0140] If it is determined in step S117 that the confirm button 123 has not been operated, the process proceeds to step S118. In step S118, the control unit 55 determines whether the number of tracking target candidates is 1. If it is determined in step S118 that the number of tracking target candidates is not 1, that is, if there are multiple tracking target candidates, the process proceeds to step S119.

[0141] In step S119, the control unit 55 performs a tracking target update process to reselect a tracking target from a plurality of tracking target candidates. Details of this tracking target update process will be described later with reference to FIG. 20. After the process of step S119, the process proceeds to step S120.

[0142] On the other hand, if it is determined in step S118 that the number of follow-up target candidates is one, the process proceeds to step S120.

[0143] In step S120, the control unit 55 rotates the cart robot 11 toward the target to be followed by supplying a rotation angle to the drive control unit 56. This allows the control unit 55 to prompt the user 21 to confirm whether the target to be followed is correct. Then, the following preparation process ends.

[0144] On the other hand, if it is determined in step S114 that the lost flag of the target to be followed is not "false," i.e., if the target to be followed is no longer recognized by the recognition unit 50, the process proceeds to step S121. In step S121, the control unit 55 deletes the target to be followed registered in the parameter list. In other words, the control unit 55 cancels the selection of the target to be followed. In step S122, the control unit 55 sets the operation mode to the target to be followed selection mode 142, and ends the follow-up preparation process.

[0145] <Explanation of the Follow-Up Target Update Process> FIG. 20 is a flowchart illustrating the details of the follow-up target update process in step S119 of FIG.

[0146] 20, the control unit 55 determines whether or not there is a candidate target to be followed whose cumulative movement amount [m] exceeds that of the target to be followed. If it is determined in step S131 that there is a candidate target to be followed whose cumulative movement amount [m] exceeds that of the target to be followed, the process proceeds to step S132.

[0147] In step S132, the control unit 55 selects the target candidate corresponding to the largest cumulative movement amount [m] from among the plurality of target candidates as the new target to be tracked, and updates the target information to be tracked. Specifically, the control unit 55 registers the moving object number of the target candidate newly selected as the target to be tracked in the parameter list as the target information to be tracked. Then, the process returns to step S119 in FIG. 19 and proceeds to step S120.

[0148] On the other hand, if it is determined in step S131 that there is no candidate target whose cumulative movement amount [m] exceeds that of the target, the process of step S132 is skipped. That is, the target is not changed. Then, the process returns to step S119 and proceeds to step S120.

[0149] As described above, by performing the target updating process of Fig. 20, the target to be followed is updated to the target candidate with the largest cumulative movement amount [m]. Therefore, when the cart robot 11 turns toward an object other than the user 21, the user 21 can change himself / herself to the target to be followed by moving further than the object.

[0150] 21 is a flowchart illustrating the tracking process performed by the control unit 55 when the operation mode is the tracking mode 144. This tracking process is started, for example, when a notification is sent from the moving object management unit 53 to the control unit 55 by the process of step S39 in FIG.

[0151] 21, the control unit 55 controls the light emission of the indicator 33 to light up in green. In step S152, the control unit 55 determines whether or not the cancel button 121 has been operated, based on an operation signal supplied from the operation device 12. If it is determined in step S152 that the cancel button 121 has been operated, the process proceeds to step S153. In step S153, the control unit 55 sets the operation mode to the operation wait mode 141, and ends the tracking process.

[0152] On the other hand, if it is determined in step S152 that the cancel button 121 has not been operated, the process proceeds to step S154. In step S154, the control unit 55 determines whether the lost flag of the tracking target is "false" as in the process of step S114 in FIG.

[0153] If it is determined in step S154 that the lost flag is "false," the process proceeds to step S155. In step S155, the control unit 55 supplies a movement amount to the drive control unit 56, thereby causing the cart robot 11 to follow the target while avoiding obstacles.

[0154] Specifically, the control unit 55 supplies the drive control unit 56 with, for example, the distance from the current position of the cart robot 11 to a position that is a predetermined distance (e.g., 1 m) away from the target to be followed as the amount of movement. Based on this amount of movement and the obstacle information, the drive control unit 56 creates a plan for the trajectory of the cart robot 11 that follows the target to be followed while avoiding the obstacle. Based on this plan, the drive control unit 56 performs PID control on the drive unit 34, causing the cart robot 11 to follow the target to be followed while avoiding the obstacle. After the processing of step S155, the following processing ends.

[0155] On the other hand, if it is determined in step S154 that the lost flag is not "false," the process proceeds to step S156. In step S156, the control unit 55 deletes the tracking target information registered in the parameter list. In step S157, the control unit 55 sets the operation mode to the operation waiting mode 141 and ends the tracking process.

[0156] As described above, by performing the following process of Figure 21, the cart robot 11 follows the target so that the distance between the target and the cart robot 11 remains constant until the cancel button 121 is operated or the lost flag of the target becomes "true".

[0157] <First Example of Movement of Cart Robot> FIG. 22 is a diagram showing an example of the movement of the cart robot 11 when two desks are present within the detection range of the sensor 32. In FIG.

[0158] In this case, as shown in A of Fig. 22, with desks 161 and 162 present within the detection range of sensor 32, user 21 stands in front of cart robot 11, thereby entering the detection range of sensor 32. Then, user 21 operates start button 122. This causes the operation mode to transition from operation wait mode 141 to follow-up target selection mode 142, and the detection list generation process of Fig. 5 and the management process of Fig. 11 are started.

[0159] In this case, when the recognition unit 50 recognizes a human as a specified object, it is desirable that the recognition unit 50 does not recognize the desks 161 and 162, but there is a possibility that the recognition unit 50 may mistakenly recognize the legs of the desks 161 and 162 as human legs and recognize the desks 161 and 162 as humans.

[0160] In this case, as a result of the detection list generation process and management process, a moving object list is generated in which movement information of the user 21 and the desks 161 and 162 is registered, and this is notified from the moving object management unit 53 to the control unit 55. In response to this notification, the tracking target selection process of FIG. 18 is started.

[0161] Next, as shown in B of FIG. 22 , the user 21 moves in a direction away from the cart robot 11. As a result, the moving object list is updated by the detection list generation process and management process in accordance with the movement of the user 21. At this time, the desks 161 and 162 are closer to the cart robot 11 than the user 21, but do not move. Therefore, the user 21 is the only object whose updated cumulative movement amount [m] exceeds the tracking preparation threshold [m] and whose tracking candidate flag is "true." As a result, the user 21 is selected as the tracking target in the tracking target selection process, and the operation mode transitions to the tracking preparation mode 143.

[0162] As described above, the control unit 55 selects an object whose cumulative movement amount [m] exceeds the tracking preparation threshold [m] as the object to be tracked, thereby preventing the erroneous selection of stationary desks 161 and 162, etc. as the object to be tracked.

[0163] <Second Example of Movement of Cart Robot> FIG. 23 is a diagram showing an example of the movement of the cart robot 11 when a person other than the user 21 is present within the detection range of the sensor 32. In FIG.

[0164] In this case, as shown in A of Figure 23, when a person 181 present in the detection range of the sensor 32 moves ahead of the user 21 during the tracking target selection mode 142, the moving object list is updated in accordance with that movement by the detection list generation process and management process. As a result, for example, the cumulative movement amount [m] of the person 181 after the update exceeds the tracking preparation threshold [m], and the tracking candidate flag becomes "true." As a result, the person 181 is selected as the tracking target in the tracking target selection process, and the operation mode transitions to the tracking preparation mode 143.

[0165] Then, as shown in B of FIG. 23 , in the following preparation process, the cart robot 11 rotates toward the human 181, and the user 21 recognizes that the target to be followed is another human 181. Therefore, the user 21 moves further away from the cart robot 11 than the human 181. As a result, the moving object list is updated by the detection list generation process and management process in accordance with the movement of the user 21. As a result, the cumulative movement amount [m] of the user 21 after the update exceeds the following preparation threshold [m], and the following candidate flag becomes "true." At this time, the moving object list is also updated in accordance with the movement of the human 181, but the cumulative movement amount [m] of the human 181 after the update is smaller than the cumulative movement amount [m] of the user 21 after the update. As a result, the target to be followed is changed to the user 21 in the following target update process.

[0166] As described above, the person 181 whose candidate flag becomes "true" for the first time after the operation mode switches to the target selection mode is selected as the target to be followed. However, the moving object list is updated as needed, and the target to be followed is updated based on the moving object list. Therefore, when the user 21 recognizes that the target to be followed is a person 181 other than himself, he or she can change the target to himself or herself by moving away from the person 181.

[0167] If the user 21 recognizes that the target to be followed is a human 181 other than himself / herself, he / she can restart the following operation of the cart robot 11 by operating the cancel button 121. In this case, the operation mode transitions to an operation waiting mode 141, and the cart robot 11 waits until the user 21 operates the start button 122.

[0168] <Third example of the movement of the cart robot> Figure 24 is a diagram showing an example of the movement of the cart robot 11 when a person other than the user 21 walks from behind the sensor 32 immediately after the user 21 operates the start button 122.

[0169] In this case, as shown in A of FIG. 24 , when the follow-up target selection mode 142 starts, a human 201 walking behind the cart robot 11 enters the detection range of the sensor 32 and moves ahead of the user 21, for example. As a result, the moving object list is updated by the detection list generation process and management process in accordance with the movement of the human 201. As a result, for example, the cumulative movement amount [m] of the human 201 after the update exceeds the follow-up preparation threshold [m], and the follow-up candidate flag becomes "true." As a result, the human 201 is selected as the follow-up target in the follow-up target selection process, and the operation mode transitions to the follow-up preparation mode 143. In the example of A of FIG. 24 , the human 201 is stationary when the operation mode transitions to the follow-up preparation mode 143.

[0170] 24B, when the cart robot 11 rotates toward the human 201 in the following preparation process, the user 21 recognizes that the target to be followed is another human 201. Therefore, the user 21 notifies the cart robot 11 that the target to be followed is incorrect by operating the cancel button 121, for example, and causes the cart robot 11 to redo the following operation. In this case, the operation mode transitions to an operation waiting mode 141, and the cart robot 11 waits until the user 21 operates the start button 122.

[0171] As described above, there is a case where the human 201 is mistakenly selected as the target to be followed because the human 201 walks from behind the sensor 32 immediately after the user 21 operates the start button 122. However, in this case, the user 21 recognizes the mistaken selection of the target to be followed by the rotation of the cart robot 11 and can operate the cancel button 121.

[0172] The operation device 12 may have an exclusion button that is operated to exclude the selected target to be followed from the candidates for the target to be followed. In this case, in B of FIG. 23 or B of FIG. 24, the user 21 operates the exclusion button when he or she recognizes that the target to be followed is a person 181 (201) other than the user 21. The control unit 55 changes the lost flag of the selected target to be followed from "false" to "true" based on an operation signal representing the operation of the exclusion button. As a result, in the following preparation process of FIG. 19, the operation mode transitions to the target to be followed selection mode, and the target to be followed is selected from the candidates for the target to be followed other than the target to be followed when the exclusion button was operated.

[0173] <Fourth Example of Movement of Cart Robot> FIG. 25 is a diagram showing an example of the movement of the cart robot 11 when the user 21 is outside the detection range of the sensor 32 when operating the start button 122. In FIG.

[0174] 25A, if the user 21 is too close to the sensor 32 when operating the start button 122 and is therefore outside the detection range of the sensor 32, the movement information of the user 21 is not registered in the moving object list by the detection list generation process and management process. On the other hand, the movement information of people 221 and 222 who are present within the detection range of the sensor 32 is registered in the moving object list. However, if the people 221 and 222 do not move, the following candidate flags of the people 221 and 222 remain "false."

[0175] After operating the start button 122, the user 21 moves in a direction away from the cart robot 11, as shown in B of FIG. 25 . Therefore, the user 21 enters the detection range of the sensor 32, and the moving object list is updated by the detection list generation process and management process. As a result, the movement information of the user 21 is registered in the updated moving object list. As the user 21 continues to move, the moving object list is further updated in accordance with the movement by the detection list generation process and management process. Then, the updated cumulative movement amount [m] of the user 21 exceeds the following preparation threshold [m], and the following candidate flag becomes "true." As a result, the user 21 is selected as the following target in the following target selection process, and the operation mode transitions to the following preparation mode 143.

[0176] As described above, even if the user 21 is not within the detection range of the sensor 32 when the start button 122 is operated, the user 21 can have the control unit 55 select the user 21 as the target to be followed by moving after operating the start button 122.

[0177] In the control device 31, the recognition unit 50 recognizes an object present around the cart robot 11, the control unit 55 selects the object as a target to be followed based on movement information of the object, and the drive control unit 56 causes the cart robot 11 to follow the target to be followed. Therefore, an object intended by the user 21 can be selected as a target to be followed from among objects present around the cart robot 11 without increasing the cost of the cart robot 11 or the operational burden on the user 21. As a result, the cart robot 11 can be caused to follow the target to be followed easily and safely.

[0178] The drive control unit 56 rotates the cart robot 11 toward the target to be followed before making the cart robot 11 follow the target. Therefore, the user 21 recognizes the target to be followed before the cart robot 11 starts following, and if the user 21 recognizes that they have erroneously selected the target to be followed, they can safely perform an action to reselect the target to be followed, such as moving away from the target to be followed or operating the cancel button 121. As a result, it is possible to prevent the cart robot 11 from following an object that the user 21 did not intend to select as the target to be followed, thereby preventing danger to people in the vicinity.

[0179] The movement information used to select a candidate target to be followed may include the cumulative movement amount [m], as well as the object's movement speed, movement direction, and a score based on the variance value of the shape and direction of the movement trajectory (gesture).The information used to determine the target to be followed may include the relative distance [m] from the cart robot 11, as well as the shape of the movement trajectory of the target to be followed.

[0180] 2. Second Embodiment Configuration Example of a Mobility Control System FIG. 26 is a block diagram showing a configuration example of a mobility control system according to a second embodiment to which the present technology is applied.

[0181] In the mobile control system 310 in Fig. 26, parts corresponding to those in the mobile control system 10 in Fig. 2 are assigned the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on parts that differ from the mobile control system 10. The mobile control system 310 differs from the mobile control system 10 mainly in that the control device 331 selects a target to be tracked based on a recognition score, and is otherwise configured in the same way as the mobile control system 10.

[0182] Specifically, the control device 331 differs from the control device 31 in that it has a recognition unit 350, a moving object management unit 353, and a control unit 355 instead of a recognition unit 50, a moving object management unit 53, and a control unit 55, but is otherwise configured in the same way as the control device 31.

[0183] The recognition unit 350 receives input of sensor data acquired by the sensor 32. Based on the sensor data, the recognition unit 350 recognizes predetermined objects, such as humans, present around the cart robot 311 and calculates a recognition score that indicates the accuracy (likelihood) of the recognized object. For example, the value of this recognition score increases as the accuracy of the recognized object increases. Any algorithm can be used as the object recognition algorithm used by the recognition unit 350. The recognition unit 350 generates a list of the sensor coordinate system position and recognition score of each recognized object as a detection list with recognition score, and supplies the list to the moving object management unit 353.

[0184] The moving object management unit 353 manages the moving object list stored in the memory unit 54 based on the detection list with recognition score, the conversion information and world coordinate position of the cart robot 311 supplied from the self-position estimation unit 51, and the parameter list stored in the memory unit 54.

[0185] Specifically, the moving object management unit 353 performs object matching in the same manner as the moving object management unit 53. The moving object management unit 353 updates the moving object list based on the results of this matching, the detection list with recognition scores, and the error determination threshold registered in the parameter list of the storage unit 54. The moving object management unit 353 then notifies the control unit 355 of the update of the moving object list.

[0186] The moving object list in the second embodiment differs from the moving object list in the first embodiment in that the movement information does not include a tracking candidate flag and a recognition score is newly registered, but otherwise the moving object list is configured in the same way as the moving object list in the first embodiment. The parameter list in the second embodiment differs from the parameter list in the first embodiment in that the tracking preparation threshold [m] is not registered, but otherwise the moving object list is configured in the same way as the parameter list in the first embodiment.

[0187] Like the control unit 55, the control unit 355 sets the operation mode based on the operation signal sent from the operation device 12, the notification supplied from the moving object management unit 353, and the tracking start threshold [m] registered in the parameter list of the memory unit 54.

[0188] When the operation mode is the follow-up target selection mode, the control unit 355 (selection unit) selects a predetermined object as a follow-up target based on the recognition score of each object registered in the moving object list in the storage unit 54. Then, the control unit 355 supplies follow-up target person information of the follow-up target to the storage unit 54 to store it.

[0189] When the operation mode is the follow-up preparation mode, the control unit 355 supplies the rotation angle to the drive control unit 56 based on the moving object list, similar to the control unit 55. When the operation mode is the follow-up mode, the control unit 355 supplies the movement amount to the drive control unit 56 based on the moving object list, similar to the control unit 55. The control unit 355 notifies the user 21 of the operation mode by controlling the emission of light from the indicator 33 in accordance with the operation mode, similar to the control unit 55.

[0190] <Example of Recognition Score> FIG. 27 is a diagram showing an example of a recognition score.

[0191] In the example of FIG. 27, the sensor 32 is an RGB-D camera, the sensor data is RGBD data, and the recognition unit 350 recognizes a human being as the predetermined object.

[0192] 27, when three people 371-1 to 371-3 are present in an image 370 corresponding to sensor data, the recognition unit 350 recognizes rectangular areas 372-1 to 372-3 that respectively include people 371-1 to 371-3 as predetermined objects. At this time, the recognition unit 350 calculates, for example, 0.3, 0.7, and 0.2 as the recognition scores for people 371-1 to 371-3, respectively.

[0193] <Example of Detection List with Recognition Score> FIG. 28 is a diagram showing an example of a detection list with recognition score.

[0194] In the example of Fig. 28, a cart robot 311 exists instead of the cart robot 11 in the example of Fig. 3. In this case, the recognition unit 350 generates a detection list with recognition scores, in which the sensor coordinate system positions (X[m], Y[m]) and recognition scores are registered in association with the recognition numbers assigned to the objects 71 to 74, as shown in Fig. 28.

[0195] In the example of Fig. 28, similar to the example of Fig. 4, recognition numbers "1" to "4" are assigned to objects 71 to 74 in order, and the sensor coordinate system positions (X[m], Y[m]) of objects 71 to 74 are the same as in Fig. 4. The recognition scores of objects 71 to 74 are 0.5, 0.3, 0.2, and 0.7, respectively.

[0196] <Description of Detection List Generation Process> Fig. 29 is a flowchart illustrating the detection list generation process by the recognition unit 350 in Fig. 26. This detection list generation process is started when sensor data is input from the sensor 32, for example.

[0197] 29, the recognition unit 350 determines whether the operation mode is the operation waiting mode. If it is determined in step S211 that the operation mode is not the operation waiting mode, the process proceeds to step S212.

[0198] In step S212, the recognition unit 350 recognizes a predetermined object, such as a human being, present around the cart robot 311 based on the sensor data, and calculates a recognition score for that object. In step S213, the recognition unit 350 generates a detection list with a recognition score based on the sensor coordinate system position (X[m], Y[m]) and recognition score of the object recognized by the processing in step S212, and supplies the detection list to the moving object management unit 353. Then, the detection list generation processing ends.

[0199] On the other hand, if it is determined in step S211 that the operation waiting mode is in effect, the processes of steps S212 and S213 are not performed, and the detection list generation process ends.

[0200] <Example of Moving Object List> FIG. 30 shows the predicted values ​​P of the objects 91 to 93 when a cart robot 311 is present instead of the cart robot 11 in FIG. 7, calculated by the moving object management unit 353.pred FIG. 10 is a diagram showing an example of a moving object list in which (x[m], y[m]) is registered.

[0201] In the example of FIG. 30, similarly to the example of FIG. 8, the moving object management unit 353 assigns moving object numbers 0 to 2 to the objects 91 to 93 in order.

[0202] Note that in the moving object list of Fig. 30, descriptions of parts corresponding to the moving object list of Fig. 8 will be omitted as appropriate, and the following description will focus on parts that differ from the moving object list of Fig. 8. The moving object list of Fig. 30 differs from the moving object list of Fig. 8 in that the movement information does not include a tracking candidate flag and a recognition score is newly registered, but is otherwise configured in the same way as the moving object list of Fig. 8.

[0203] Specifically, in the moving object list of Fig. 30, the recognition scores of the objects 91 to 93 are registered as recognition scores corresponding to the moving object numbers "0" to "2," respectively. In the example of Fig. 30, the recognition scores of the objects 91 to 93 are 0.3, 0.7, and 0.2, respectively.

[0204] The management process by the moving object management unit 353 differs from the management process of Fig. 11 in that a recognition score is registered together with movement information in step S35, and in that steps S54 and S55 are not performed in the movement information update process of Fig. 12, and a recognition score is registered instead. The rest of the process is configured in the same way as the management process of Fig. 11, so a description thereof will be omitted.

[0205] <Explanation of User's Action> FIG. 31 is a diagram illustrating the action of the user 21 when the user 21 makes the cart robot 311 follow the user 21. In FIG.

[0206] As shown in A of Fig. 31, first, the user 21 stands in front of the cart robot 311 and operates the start button 122. This causes the operation mode to transition from an operation wait mode to a follow-up target selection mode, and the detection list generation process and management process of Fig. 29 are performed. As a result, a moving object list is generated in which the movement information and recognition score of the user 21 are registered. At this time, the indicator 33 is flashing yellow. Then, the user 21 with the highest recognition score registered in the moving object list is selected as the follow-up target, the operation mode transitions to a follow-up preparation mode, and the indicator 33 starts to light up yellow.

[0207] When the user 21 recognizes that the operation mode has transitioned to the follow-up preparation mode by the indicator 33 lighting up in yellow, the user 21 moves left or right relative to the cart robot 311. For example, as shown in B of Fig. 31 , when the user 21 moves leftward relative to the cart robot 311, the cart robot 311 rotates rightward relative to the user 21 while remaining stopped.

[0208] As a result, when the user 21 recognizes that he or she is the target to be followed, the user 21 moves in a direction away from the cart robot 311, as shown in C of Fig. 31 . As a result, the moving object list is updated by the detection list generation process and management process in accordance with the movement of the user 21. When the updated relative distance [m] between the user 21 and the cart robot 311 exceeds the following start threshold [m], the target to be followed is confirmed, and the operation mode transitions to following mode. Then, the cart robot 311 starts following the user 21, and the indicator 33 lights up green.

[0209] As in the case of FIG. 16, when the user 21 recognizes that he or she is the target to be followed, the user 21 may operate the confirm button 123 instead of moving away from the cart robot 311.

[0210] As described above, in the second embodiment, as in the first embodiment, the operation mode transitions to the follow preparation mode before the follow mode, and the robot does not start following until the object to be followed is determined. This prevents the cart robot 311 from starting to follow an object unintended by the user 21. As in the first embodiment, the actions of the user 21 may be instructed by a speaker, a display, or the like (not shown) provided on the cart robot 311.

[0211] 32 is a flowchart illustrating the process of selecting a target to be followed that is performed by the control unit 355 when the operation mode is the target to be followed selection mode. This process of selecting a target to be followed is started, for example, when the moving object management unit 353 notifies the control unit 355 by the management process.

[0212] The processing in steps S291 to S293 in FIG. 32 is similar to the processing in steps S91 to S93 in FIG. 18, and therefore a description thereof will be omitted.

[0213] On the other hand, if it is determined in step S292 that the cancel button 121 has not been operated, the control unit 55 selects, based on the lost flag, an object whose lost flag is "false" from the objects registered in the moving object list as a candidate to be tracked. Then, in step S294, the control unit 355 determines whether or not one or more candidate to be tracked exists. If it is determined in step S294 that one or more candidate to be tracked exists, the process proceeds to step S295.

[0214] In step S295, the control unit 355 selects the tracking target candidate with the highest recognition score from the one or more tracking target candidates based on the recognition scores of the one or more tracking target candidates. The control unit 355 registers the moving object number of the tracking target in the parameter list as tracking target information. In step S296, the control unit 355 sets the operation mode to a tracking preparation mode and ends the tracking target selection process.

[0215] On the other hand, if it is determined in step S294 that one or more candidates for the follow-up target do not exist, steps S295 and S296 are skipped and the follow-up target selection process ends.

[0216] Note that, after the process of step S294, if the maximum value of the recognition scores of one or more follow-up target candidates does not exceed a predetermined threshold, the process may not proceed to step S295, and the follow-up target selection process may end.

[0217] 33 is a flowchart illustrating the following preparation process performed by the control unit 355 when the operation mode is the following preparation mode. This following preparation process is started, for example, when the moving object management unit 353 notifies the control unit 355 by the management process.

[0218] The processing in steps S311 to S317 in FIG. 33 is similar to the processing in steps S111 to S117 in FIG. 19, and therefore a description thereof will be omitted.

[0219] If it is determined in step S317 that the confirm button 123 has not been operated, the process proceeds to step S318. In step S318, the control unit 355 supplies a rotation angle to the drive control unit 56, thereby rotating the cart robot 311 toward the target to be followed. This allows the control unit 355 to prompt the user 21 to confirm whether the target to be followed is correct. Then, the following preparation process ends.

[0220] On the other hand, if it is determined in step S314 that the lost flag of the tracking target is not "false", the process proceeds to step S319. The processes of steps S319 and S320 are the same as the processes of steps S121 and S122 in Fig. 19, and therefore description thereof will be omitted.

[0221] 33, if it is determined in step S317 that the confirm button 123 has not been operated, the control unit 355 may determine whether or not there are multiple candidates for the tracking target. In this case, if it is determined that there are multiple candidates for the tracking target, the control unit 355 reselects the candidate for the tracking target with the largest recognition score as the tracking target.

[0222] The operation waiting process and the follow-up process by the control unit 355 are similar to the operation waiting process in FIG. 17 and the follow-up process in FIG. 21, and therefore a description thereof will be omitted.

[0223] <Example of Movement of Cart Robot> FIG. 34 is a diagram showing an example of the movement of the cart robot 311 when two people are present within the detection range of the sensor 32. In FIG.

[0224] In this case, as shown in A of Fig. 34, with people 401 and 402 present within the detection range of sensor 32, user 21 stands in front of cart robot 311, thereby entering the detection range of sensor 32. Then, user 21 operates start button 122. This causes the operation mode to transition from the operation wait mode to the follow-up target selection mode, and the detection list generation process and management process of Fig. 29 are started.

[0225] As a result, if the recognition unit 350 recognizes a human as the predetermined object, a moving object list is generated in which the movement information and recognition scores of the user 21 and the humans 401 and 402 are registered, and the moving object management unit 53 notifies the control unit 355. In response to this notification, the tracking target selection process of FIG. 32 is started. In the example of FIG. 34, the recognition scores of the human 401, the user 21, and the human 402 are 0.3, 0.5, and 0.7, respectively. Therefore, by the tracking target selection process of FIG. 32, the human 402 with the highest recognition score is selected as the tracking target, as shown in B of FIG. 34, and the operation mode transitions to a tracking preparation mode.

[0226] 34C, in the following preparation process, the cart robot 311 is rotated toward the human 402, and the user 21 recognizes that the person to be followed is another human 402. Therefore, the user 21 operates the cancel button 121 to make the cart robot 311 redo the following operation. In this case, the operation mode transitions to an operation waiting mode, and the cart robot 311 waits until the user 21 operates the start button 122.

[0227] The control unit 355 may select a target to be followed based on the relative distance [m] from the cart robot 311 instead of the recognition score. In this case, the control unit 355 selects, for example, a target to be followed that has a relative distance [m] from the cart robot 311 that is closest to a specific distance (for example, 1 [m] ahead) as the target to be followed. The control unit 355 may select a target to be followed based on a combination of the recognition score and movement information such as the relative distance [m] from the cart robot 311 and the accumulated movement amount [m].

[0228] The control unit 355 may determine the target to be followed based on other movement information, such as the accumulated movement amount [m], instead of the relative distance [m] from the cart robot 311. For example, the control unit 355 may determine the target to be followed when the accumulated movement amount [m] exceeds a predetermined threshold. The control unit 355 may also determine the target to be followed based on the elapsed time since the cart robot 311 rotated toward the target to be followed. For example, the control unit 355 may also determine the target to be followed when a predetermined time or more has elapsed since the cart robot 311 rotated toward the target to be followed. The control unit 355 may also determine the target to be followed based on both the movement information and the elapsed time.

[0229] As described above, in the control device 331, the recognition unit 350 recognizes an object present in the surrounding area, and the control unit 355 selects that object as the object to be followed. Then, the drive control unit 56 rotates the cart robot 311 toward the object to be followed before causing the cart robot 311 to follow the object to be followed.

[0230] Therefore, the user 21 can recognize the target to be followed before the cart robot 311 starts following the target, without increasing the costs of the operation device 12 or the cart robot 311. Therefore, if the user 21 recognizes that they have erroneously selected the target to be followed, they can operate the cancel button 121, for example, to stop the following operation of the cart robot 311. As a result, it is possible to prevent the cart robot 311 from following an object that the user 21 did not intend to select as the target to be followed, which could pose a danger to people in the vicinity. As described above, the control device 331 can easily and safely cause the cart robot 311 to follow the target to be followed.

[0231] On the other hand, if a display is provided on the operation device 12 or the cart robot 311 and the target to be followed is displayed on the display, the costs of the operation device 12 or the cart robot 311 increase. Also, the size of the operation device 12 or the cart robot 311 increases as space for the display needs to be secured.

[0232] If a display is provided on the operation device 12 or the cart robot 311, candidates for the target to be followed are displayed on the display, and the user 21 is allowed to select a desired candidate for the target to be followed as the target to be followed, the operation of the user 21 becomes cumbersome. Also, for example, if the user 21 does not have free hands due to carrying luggage, it is difficult for the user 21 to select the target to be followed.

[0233] After the drive control unit 56 rotates the cart robot 311 toward the target to be followed, the control unit 355 does not cause the cart robot 311 to follow the target to be followed until the target to be followed is confirmed. Therefore, the user 21 can operate the cancel button 121 without panicking and stop the cart robot 311 from following the target to be followed that is not intended by the user 21, compared to when the cart robot 311 starts following immediately after rotating.

[0234] In the first and second embodiments, the target that the user 21 intends to follow is the user 21, but it may be a movable object other than the user 21, such as another robot. The robot that follows the user 21 may be the cart robot 11 (311) or a robot that has a function other than carrying luggage.

[0235] The control unit 55 (355) may set the operation mode to the follow mode in response to another operation, such as a re-operation of the start button 122, a voice operation, or a gesture operation, other than the operation of the confirm button 123. Instead of operating the cancel button 123, the user may also perform a re-operation of the start button 122, a voice operation, a gesture operation, or the like.

[0236] The control device 31 (331) may be provided outside the cart robot 11 (311).

[0237] 3. Computer The series of processes of the control device 31 (331) described above can be executed by hardware or software. When the series of processes are executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, for example, that can execute various functions by installing various programs.

[0238] FIG. 35 is a block diagram showing an example of the hardware configuration of a computer that executes the series of processes of the control device 31 (331) described above by a program.

[0239] In the computer, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected by a bus 904.

[0240] An input / output interface 905 is also connected to the bus 904. An input unit 906, an output unit 907, a storage unit 908, a communication unit 909, and a drive 910 are connected to the input / output interface 905.

[0241] The input unit 906 includes a keyboard, a mouse, a microphone, etc. The output unit 907 includes a display, a speaker, etc. The storage unit 908 includes a hard disk, a non-volatile memory, etc. The communication unit 909 includes a network interface, etc. The drive 910 drives removable media 911 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0242] In a computer configured as described above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the memory unit 908 into the RAM 903 via the input / output interface 905 and the bus 904 and executing it.

[0243] The program executed by the computer (CPU 901) can be provided by being recorded on a removable medium 911 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0244] In a computer, the program can be installed in the storage unit 908 via the input / output interface 905 by inserting the removable medium 911 into the drive 910. The program can also be received by the communication unit 909 via a wired or wireless transmission medium and installed in the storage unit 908. Alternatively, the program can be installed in the ROM 902 or the storage unit 908 in advance.

[0245] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0246] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0247] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0248] For example, it is possible to adopt a configuration in which all or part of the above-described embodiments are combined.

[0249] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0250] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0251] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0252] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0253] The present technology may have the following configurations. (1) A control device comprising: a recognition unit that recognizes objects present around a robot; a selection unit that selects the object as a target to be followed based on movement information that is information regarding the movement of the object recognized by the recognition unit; and a drive control unit that causes the robot to follow the target to be followed. (2) The control device described in (1), wherein the drive control unit also rotates the robot toward the target to be followed before causing the robot to follow the target to be followed. (3) The control device described in (1) or (2), further comprising: a self-position estimator that generates a position of the robot, wherein the drive control unit causes the robot to follow the target to be followed when a distance between a position of the target to be followed among the objects recognized by the recognition unit and the position of the robot generated by the self-position estimator exceeds a predetermined threshold. (4) The control device described in (1) or (2), wherein the drive control unit causes the robot to follow the target to be followed in response to a user operation. (5) The control device according to any of (1) to (4), wherein the selection unit changes the target to be followed in response to a user operation. (6) The control device according to any of (1) to (5), wherein the selection unit cancels selection of the target to be followed when the target to be followed is no longer recognized by the recognition unit. (7) The control device according to any of (1) to (6), wherein the movement information is a cumulative value of the amount of movement of the object. (8) The control device according to any of (1) to (7), further comprising a notification unit that notifies that the robot is currently following the target to be followed. (9) The control device according to (8), wherein the notification unit also notifies that the selection unit is currently selecting the target to be followed. (10) A control method, comprising: a control device recognizing an object present around the robot; selecting the object as the target to be followed based on movement information that is information related to the movement of the object; and causing the robot to follow the target to be followed.(11) A robot comprising: a recognition unit that recognizes objects present in the surrounding area; a selection unit that selects the object as a target to be followed based on movement information that is information regarding the movement of the object recognized by the recognition unit; and a drive unit that drives the robot to follow the target to be followed selected by the selection unit. (12) A control device comprising: a recognition unit that recognizes objects present in the surrounding area of ​​a robot; a selection unit that selects the object recognized by the recognition unit as a target to be followed; and a drive control unit that rotates the robot toward the target to be followed before making the robot follow the target to be followed. (13) The control device according to (12), wherein the selection unit selects the target to be followed based on the accuracy of the object recognized by the recognition unit or the distance from the robot. (14) The control device according to (12) or (13), wherein the drive control unit also makes the robot follow the target to be followed after rotating the robot toward the target to be followed. (15) The control device according to (14), wherein the drive control unit causes the robot to follow the target to be followed based on at least one of movement information regarding the movement of the target to be followed and the elapsed time since the robot rotated towards the target to be followed. (16) The control device according to (14), wherein the drive control unit causes the robot to follow the target to be followed in accordance with a user operation. (17) The control device according to any of (12) to (16), wherein the selection unit changes the target to be followed in accordance with a user operation. (18) The control device according to any of (12) to (17), wherein the selection unit cancels selection of the target to be followed when the target to be followed is no longer recognized by the recognition unit. (19) The control device according to any of (12) to (18), further comprising a notification unit that notifies that the robot is following the target to be followed.

[0254] REFERENCE SIGNS LIST 11 Cart robot, 21 User, 31 Control device, 33 Indicator, 34 Drive unit, 50 Recognition unit, 51 Self-position estimation unit, 55 Control unit, 56 Drive control unit, 71 to 74 Object, 311 Cart robot, 331 Control device, 350 Recognition unit, 355 Control unit

Claims

1. A control device comprising: a recognition unit that recognizes objects present around a robot; a selection unit that selects the object as a target to be followed based on movement information that is information regarding the movement of the object recognized by the recognition unit; and a drive control unit that causes the robot to follow the target to be followed.

2. The control device according to claim 1, wherein the drive control unit also rotates the robot toward the target before causing the robot to follow the target.

3. The control device according to claim 1, further comprising a self-position estimation unit that generates a position of the robot, wherein the drive control unit causes the robot to follow the target when the distance between the position of the target to be followed among the objects recognized by the recognition unit and the position of the robot generated by the self-position estimation unit exceeds a predetermined threshold.

4. The control device according to claim 1, wherein the drive control unit causes the robot to follow the target in response to a user's operation.

5. The control device according to claim 1, wherein the selection unit changes the target to be followed in response to a user operation.

6. The control device according to claim 1, wherein the selection unit cancels the selection of the target to be followed when the target is no longer recognized by the recognition unit.

7. The control device according to claim 1, wherein the movement information is a cumulative value of the amount of movement of the object.

8. The control device according to claim 1, further comprising a notification unit that notifies that the robot is currently following the target to be followed.

9. The control device according to claim 8, wherein the notification unit also notifies that the selection unit is currently selecting the target to be followed.

10. A control method including: a control device recognizing an object present around a robot; selecting the object as a target to be followed based on movement information that is information regarding the movement of the object; and causing the robot to follow the target to be followed.

11. A robot comprising: a recognition unit that recognizes surrounding objects; a selection unit that selects the object as a target to be followed based on movement information that is information regarding the movement of the object recognized by the recognition unit; and a drive unit that drives the robot to follow the target to be followed selected by the selection unit.

12. A control device comprising: a recognition unit that recognizes objects present around a robot; a selection unit that selects the object recognized by the recognition unit as a target to be followed; and a drive control unit that rotates the robot toward the target to be followed before making the robot follow the target.

13. The control device according to claim 12, wherein the selection unit selects the target to be followed based on the accuracy of the object recognized by the recognition unit or the distance from the robot.

14. The control device according to claim 12, wherein the drive control unit also rotates the robot toward the target and then causes the robot to follow the target.

15. The control device according to claim 14, wherein the drive control unit causes the robot to follow the target based on at least one of movement information regarding the movement of the target and the elapsed time since the robot rotated toward the target.

16. The control device according to claim 14, wherein the drive control unit causes the robot to follow the target in response to a user's operation.

17. The control device according to claim 12, wherein the selection unit changes the target to be followed in response to a user operation.

18. The control device according to claim 12, wherein the selection unit cancels the selection of the target to be followed when the target is no longer recognized by the recognition unit.

19. The control device according to claim 12, further comprising a notification unit that notifies that the robot is currently following the target to be followed.

Images