Control device, control method, and program

Abstract

This control device controls a moving body which, while at least temporarily leading a following person, autonomously moves in a region in which another pedestrian walks. The control device comprises: a recognition unit which recognises objects including the following person and the other pedestrian; a prediction unit which predicts a future position of the recognised other pedestrian; an action plan generation unit which generates a route that the mobile body should follow in the future; and a drive control unit which controls a drive device attached to the mobile body such that the mobile body moves along the route. The action plan generation unit: sets a plurality of route candidates; selects one of the route candidates on the basis of a score, for each of said route candidates, that reflects an operation evaluation of a notification device that is mounted on the control device and notifies the other pedestrian; and sets said one candidate route as the above-mentioned route.

Description

Control device, control method, and program

[0001] The present invention relates to a control device, a control method, and a program.

[0002] In recent years, moving objects (called robots, micromobility, etc.) that move autonomously while following a user for purposes such as transporting the user's luggage have been put into practical use. An invention of a driving control device related to micromobility has been disclosed (Patent Document 1). Research is also being conducted on moving objects that not only follow a user but also move autonomously while leading the user.

[0003] Special Publication No. 2020-529050

[0004] “Social Force Model for Pedestrian Dynamics”D. Helbing and P. Molnar, , 20 May 1998.”Human Trajectory Forecasting in Crowds:A Deep Learning Perspective”, Parth Kothari, Sven Kreiss, and Alexandre Alahi, 11 January 2021

[0005] Conventional technologies sometimes fail to generate an appropriate route depending on the situation of the moving object. For example, they do not consider how to lead the moving object. Furthermore, moving objects may be equipped with notification devices such as speakers, buzzers, displays, and projectors, but they do not consider evaluating routes assuming that these devices are used or not.

[0006] The present invention has been made taking these circumstances into consideration, and one of its objectives is to provide a control device, control method, and program that can control a moving object to behave appropriately by reflecting the operation evaluation of an alarm device.

[0007] The control device, control method, and program according to the present invention employ the following configuration: (1): A control device according to one aspect of the present invention controls a mobile object that autonomously moves through an area where other pedestrians walk while at least temporarily leading a target person, and includes a recognition unit that recognizes objects including the target person and the other pedestrians, a prediction unit that predicts future positions of the recognized other pedestrians, a behavior plan generation unit that generates a future route for the mobile object, and a drive control unit that controls a drive unit attached to the mobile object so that the mobile object moves along the route, wherein the behavior plan generation unit sets a plurality of route candidates and, for each of the route candidates, selects one of the route candidates based on a score that reflects an operation evaluation of a notification device that is mounted on the control device and that notifies the other pedestrians, and selects the one of the route candidates as the route.

[0008] (2): In the above aspect (1), the behavior plan generation unit further includes an alarm control unit that generates an operation plan for the alarm device and controls the alarm device based on the operation plan for the alarm device.

[0009] (3): In the above aspect (1), the behavior plan generation unit calculates a first score based on the positional relationship between the moving body and a predetermined destination point, a second score based on the positional relationship between the leading subject and the other pedestrians from the present to a future time point, and a third score which is the result of an evaluation of the operation of the alarm device, and selects one of the route candidates based on the first score, the second score, and the third score to be the route.

[0010] (4): In the above aspect (3), the first score becomes a positive value the closer the moving body is to a predetermined destination point, and the second score becomes a positive value the closer the distance between the leading object and the other pedestrian is maintained.

[0011] (5) In the above aspect (1), the future time is a time when the moving object will reach the end of each of the plurality of route candidates.

[0012] (6): In the above aspect (3), the behavior plan generation unit selects one of the route candidates based on the score obtained by multiplying each of the first score, the second score, and the third score by a weight value.

[0013] (7) In the aspect of (6) above, the action plan generation unit is capable of changing the weight of the third score.

[0014] (8) In the above aspect (1), the notification device is a device that outputs sound, displays an image, and / or projects light.

[0015] (9): Another aspect of the present invention provides a control method executed by a processor of a control device that controls a moving body that moves autonomously through an area where other pedestrians are walking while at least temporarily leading a leading person, and includes the steps of: recognizing objects including the leading person and the other pedestrians; predicting the future positions of the recognized other pedestrians; generating a future route for the moving body; and controlling a drive unit attached to the moving body so that the moving body moves along the route. The step of generating the route includes setting a plurality of route candidates, and selecting one of the route candidates based on a score that reflects an operation evaluation of a notification device that is mounted on the control device and that notifies the other pedestrians, and setting the selected route as the route.

[0016] (10): Another aspect of the present invention provides a program that causes a processor of a control device that controls a mobile object that moves autonomously through an area where other pedestrians are walking while at least temporarily leading a lead subject to recognize objects including the lead subject and the other pedestrians, predict the future positions of the recognized other pedestrians, generate a future route for the mobile object, and control a drive unit attached to the mobile object so that the mobile object moves along the route, wherein generating the route includes setting a plurality of route candidates and selecting one of the route candidates based on a score that reflects an operation evaluation of an alarm device that is mounted on the control device and that alerts the other pedestrians, and setting the selected route as the route.

[0017] According to the aspects (1) to (10), the mobile object can be controlled to behave appropriately by reflecting the operation evaluation of the notification device.

[0018] FIG. 1 is a configuration diagram of a moving body. FIG. 2 is a configuration diagram of a control device. FIG. 3 is a diagram showing an example of a route candidate. FIG. 4 is a flowchart showing an example of the processing content of a prediction unit related to pedestrians. FIG. 5 is a diagram for explaining an example of a method for generating an ideal route. FIG. 6 is a diagram conceptually showing an operation plan of a notification device. FIG. 7 is a flowchart (part 2) showing an example of the processing content of a prediction unit related to pedestrians. FIG. 8 is a flowchart showing an example of the processing content of a prediction unit related to a user. FIG. 9 is a diagram showing an example of the characteristics of a fourth score. FIG. 10 is a diagram for explaining the calculation process of a fifth score. FIG. 11 is a diagram illustrating an example of a scene in which a moving body suppresses approach between a user and a pedestrian. FIG. 12 is a diagram illustrating an example of a scene in which a moving body suppresses approach between a user and a pedestrian. FIG. 13 is a flowchart showing an example of the flow of processing executed by a weight value adjustment unit.

[0019] [Overview] Below, with reference to the drawings, embodiments of a control device, a control method, and a program of the present invention will be described. The control device of the present invention controls the drive device of a mobile body to move the mobile body. In the present invention, a mobile body autonomously moves through an area where pedestrians walk, leading a lead person. The area where pedestrians walk includes sidewalks, public open spaces, floors within buildings, and may also include roadways. In the following description, it is assumed that no human occupants are present on the mobile body, but a human occupant may also be present on the mobile body. The lead person may be, for example, a pedestrian, but may also be a robot or an animal (hereinafter referred to as user U). For example, the mobile body moves slightly ahead of elderly user U while heading toward a predetermined destination, thereby preventing other pedestrians who may obstruct user U's movement from getting too close to user U (i.e., operating to clear a path for user U). User U is not limited to elderly people, but may also be people who have difficulty walking, children, people shopping at a supermarket, patients moving through a hospital, or pets on walks. Note that this behavior may not be constant, but may be temporary. For example, when a moving body is traveling alongside or following a user and detects a predetermined situation in the user's direction of travel (e.g., the presence of an obstacle or traffic congestion), the moving body may temporarily lead the user by executing the algorithm of the present invention.

[0020] 1 is a configuration diagram of a mobile object 1. The mobile object 1 is equipped with, for example, an HMI (Human Machine Interface) 10, an object detection device 20, a driving device 30, a sensor 40, a notification device 50, and a control device 100. These components are supported or housed by a base 5.

[0021] The HMI 10 presents various information to the user U and accepts input operations by the user U. The HMI 10 includes various display devices, speakers, buzzers, touch panels, switches, keys, short-range wireless communication devices, etc. For example, the HMI 10 accepts the setting of a destination point.

[0022] The object detection device 20 is a device that generates data for recognizing other pedestrians and the user U present around the moving body 1. The object detection device 20 includes, for example, a camera having an imaging range covering the periphery of the moving body 1. The object detection device 20 may include sensors such as a radar device, a LIDAR (Light Detection and Ranging), and an ultrasonic sensor, as well as an object recognition device that identifies an object by performing sensor fusion processing based on the outputs of these sensors.

[0023] The drive device 30 is a mechanism for moving the mobile body 1 including the base body 5 in any direction. The drive device 30 includes, for example, a plurality of wheels, a drive motor attached to one or more wheels, and a steering device attached to one or more wheels. There are no particular restrictions on the configuration of the drive device 30, and the drive device 30 may have any configuration. In principle, the drive device 30 moves the mobile body 1 while keeping the front surface of the base body 5 facing the traveling direction of the mobile body 1.

[0024] The sensor 40 is a sensor for detecting the behavior of the mobile body 1. The sensor 40 includes, for example, a wheel speed sensor for detecting the wheel speed, an acceleration sensor for detecting the acceleration acting on the mobile body 1, a yaw rate sensor attached near the center of gravity of the base body 5 in the horizontal direction, a steering angle sensor for detecting the steering angle of the steered wheels (steered wheels), and an orientation sensor for detecting the orientation of the mobile body 1 in the horizontal direction.

[0025] The notification device 50 is a device for alerting (giving an alert) to pedestrians, and includes a speaker, a buzzer, various display devices, a light projector, a short-range wireless communication device, etc. The notification device 50 may be shared in part or in whole with the HMI 10.

[0026] FIG. 2 is a configuration diagram of the control device 100. The control device 100 includes, for example, a recognition unit 110, a behavior plan generation unit 120, a prediction unit 130, a drive control unit 140, and a notification device control unit 150. The behavior plan generation unit 120 includes a route candidate setting unit 122, a score calculation unit 124, and a weight value adjustment unit 126. These components are implemented by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be implemented by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be implemented by a combination of software and hardware. The program may be stored in advance in a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device with a non-transitory storage medium), or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed in the storage device by inserting the storage medium into a drive device.

[0027] The recognition unit 110 recognizes objects, including other pedestrians (hereinafter simply referred to as pedestrian P) and the user U, based on the data output by the object detection device 20. When the object detection device 20 is a camera, the recognition unit 110 recognizes pedestrians by inputting camera images into a trained model for identifying pedestrians. The same applies to objects other than pedestrians. To distinguish between the pedestrian P and the user U, the recognition unit 110 may store multiple images of the user U captured by a camera in advance as templates in a storage unit (not shown) and identify the user U by comparing the templates with the camera images. Alternatively, the recognition unit 110 may recognize the location of the user U by using communication directivity to perform short-range wireless communication between the HMI 10 and a terminal device held by the user U.

[0028] [Route Candidates] The route candidate setting unit 122 of the behavior plan generation unit 120 generates multiple route candidates Pmc(i) that are candidates for the route Rm that the moving object 1 should travel in the future. Here, the operation of the alarm device 50 is not taken into consideration. i is an identifier for the route candidate and takes values ​​from 1 to n (n is a natural number greater than or equal to 2). Routes and route candidates are generated, for example, as a series of multiple (q) route points spaced at a predetermined distance. The route points represent points that the moving object 1 should reach at predetermined intervals (each step). FIG. 3 is a diagram showing an example of the route candidate Rmc(i). Here, n = 5 and q = 3. The route candidate setting unit 122 generates, for example, route candidates Rmc(1), Rmc(2), Rmc(4), and Rmc(5) that extend to the left and right of the route candidate Rmc(3), which extends linearly along the forward direction Dm of the moving object 1. The intervals between route points are, for example, constant, and each route candidate has a length of q steps. The route candidates are not generated so as to reach the destination point given to the mobile object 1, but are generated so as to have a length of a constant distance x q steps. For example, route candidate Rmc(2) is generated so as to turn left by a predetermined angle in each step. Route candidate Rmc(1) is generated so as to turn left by a larger predetermined angle in each step. The opposite is true for route candidates Rmc(4) and Rmc(5). This rule is merely an example, and route candidates may be generated based on other rules that show a similar trend (gradual widening). The route candidate setting unit 122 outputs the route candidate Rmc(i) to the prediction unit 130.

[0029] In the example of FIG. 3 , the intervals between the route points are constant, but the intervals may be variable. A long interval means that the route between the route points is moved at a relatively high speed, and a short interval means that the route between the route points is moved at a relatively low speed. In other words, the route Rm and the route candidates Rmc(i) may include a speed factor. For example, the route candidate setting unit 122 generates derivative route candidates with varying speeds based on route candidates generated with a constant interval between the route points.

[0030] [Prediction (Simulation)] When a route candidate Rmc(i) is given, the prediction unit 130 predicts the future positions of the pedestrian P and the user U for each route candidate Rmc(i) based on the route candidate Rmc(i). The prediction unit 130 predicts the future positions of the pedestrian P and the user U, for example, by any of the methods described in Non-Patent Documents 1 to 3, or a combination of these methods. In the following example, it is assumed that the prediction unit 130 predicts the future positions of the pedestrian P and the user U by a method based on the SFM (Social Force Model) described in Non-Patent Document 1.

[0031] 4 is a flowchart showing an example of the processing content of the prediction unit 130 regarding a pedestrian P. First, the prediction unit 130 performs goal prediction for each pedestrian P (S1). For example, the prediction unit 130 observes the movement trajectory of the pedestrian P in chronological order, and predicts that the goal (arrival point) of the pedestrian P is the destination of the movement vector of the pedestrian P in a state where there are no other pedestrians P, moving bodies 1, or users U in the vicinity.

[0032] Next, the prediction unit 130 performs the processes of S2 to S6 for each route candidate Rmc(i). The prediction unit 130 sequentially inputs the route candidates Rmc(i) for the moving body 1 one by one (S2), and generates an ideal route Ru#(i) for the user U to follow the moving body 1 in association with each of the route candidates Rmc(i) (S3). The ideal route Ru#(i) is a route along which the user U is expected to travel if it is assumed that a pedestrian P is not present in the vicinity of the moving body 1 or the user U. FIG. 5 is a diagram for explaining an example of a method for generating an ideal route Ru#(i). Here, an example is shown in which an ideal route Ru#(5) corresponding to the route candidate Rmc(5) is generated. For example, the prediction unit 130 generates the ideal route Ru#(5) of the user U so that the movement vector Vu-1 of the user U heading from step 0 (the calculation time point) to step 1 points toward the position Om-0 of the moving body 1 at step 0, so that the movement vector Vu-2 of the user U heading from step 1 to step 2 points toward the position Om-1 of the moving body 1 at step 1, and so that the movement vector Vu-3 of the user U heading from step 2 to step 3 points toward the position Om-2 of the moving body 1 at step 2, that is, so that the user U follows the moving body 1 with a one-step delay. This generation method is merely an example, and other generation methods may be adopted.

[0033] 4, the prediction unit 130 executes the processes of S4 and S5 for each pedestrian P (k = 1 to m) for q steps. First, assuming that the moving body 1 and user U move along the candidate route Rmc(i) for the moving body 1 and the ideal route Ru#(i) for the user U, various forces acting on the pedestrian P(k) in the current step are calculated (S4). These forces do not actually act, but are hypothetical forces that are considered to be generated by the psychological movements of the pedestrian P(k).

[0034] The virtual forces include, for example, (1) an influence F1 generated by the approach of another object (here, the moving body 1 and the user U), (2) a turning force F2 that corrects direction toward the goal after changing course mainly due to the influence F1, and (3) an acceleration / deceleration force F3 that adjusts speed to return to a desired speed after accelerating or decelerating mainly due to the influence F1. F1 is a translational force acting in any direction on a two-dimensional plane, F2 is a rotational force centered on the pedestrian, and F3 is a translational force parallel to the movement vector of the pedestrian. F1 to F3 are simply expressed, for example, as follows. In addition to these, integral elements, differential elements, etc. may be added to calculate the virtual forces.

[0035] F1 = (weight) x (coefficient) x (relative speed) / (distance to other objects) F2 = (moment of inertia) x (coefficient) x (angle difference between the most recent movement vector and the direction toward the goal) F3 = weight x (coefficient) x (difference between desired speed and most recent speed)

[0036] The prediction unit 130 predicts the position and velocity of pedestrian P(k) one step ahead, assuming that a virtual force continues to act for one step (S5). By executing this process for q steps for each pedestrian, a predicted route Rp(k,i) for each pedestrian P(k) is generated and output (S6). This predicted route Rp(k,i) is the result of predicting the future position of each pedestrian P(k) for each route candidate Rmc(i). The behavior plan generation unit 120 stores in memory a history of the results calculated during the process shown in FIG. 4.

[0037] Next, the prediction unit 130 calculates a predicted route Rp(k, n+1 to n+h) when the notification device 50 is activated at a predetermined timing. For example, the behavior plan generation unit 120 extracts the points and times at which the influence F1 exceeds a threshold during the process shown in FIG. 4 , and generates an activation plan for the notification device 50 so as to activate the notification device 50 at the extracted points and times. The notification device 50 may, for example, output a sound similar to a vehicle horn or a bicycle doorbell, or display an image message such as "Please make way." In response to this, the prediction unit 130 calculates a predicted route Rp(k, n+1 to n+h) for the pedestrian P on whom the mobile body 1 exerted the influence F1 at the points and times when the notification device 50 was activated, while correcting the influence F1 in an increasing direction. n+1 to n+h represent the range of possible values ​​for the identifier j of the additional route candidate Pmc(j) for the mobile body 1, assuming that the notification device 50 is activated. Hereinafter, j = n+1 to n+h. Among the route candidates Pmc(i) for the moving object 1, there may be some for which there was no point or time at which the influence F1 was equal to or greater than the threshold during the calculation process shown in FIG. 4 , so it is assumed that n≧h. FIG. 6 is a diagram conceptually illustrating an activation plan for the alarm device 50. The black circles in the diagram represent points at which the alarm device 50 is activated. In the diagram, it is determined that the alarm device 50 will be activated for the original route candidates Pmc(1), Pmc(2), and Pmc(4), and these are set as additional routes Pmc(6), Pmc(7), and Pmc(8). For route candidates Pmc(3) and Pmc(5), there was no point or time at which the influence F1 was equal to or greater than the threshold, so it is determined that the alarm device 50 will not be activated, and therefore additional route candidates corresponding to these candidates will not be set. The number of activations of the alarm device 50 is not limited to one, and may be set multiple times. Furthermore, the operation plan of the alarm device 50 is not limited to the above, and may be determined by any method as long as it has a similar directionality.

[0038] The prediction unit 130 performs the process shown in FIG. 7 based on the operation plan of the alarm device 50. FIG. 7 is a flowchart (part 2) showing an example of the processing content of the prediction unit 130 for pedestrian P. The goal prediction for each pedestrian can be based on the process calculated in S1 of FIG. 4. The prediction unit 130 performs the processes SS2 to SS6 for each additional route candidate Rmc(j) (j = n+1 to n+h). The prediction unit 130 sequentially inputs the additional route candidates Rmc(j) for the moving object 1 (SS2). The ideal route Ru# for user U can be based on the process calculated in S3 of FIG. 4. In other words, the ideal route Ru# corresponding to the route candidate that served as the basis for the additional route candidate can be used. Next, the prediction unit 130 performs the processes SS4 and SS5 for each pedestrian P (k = 1 to m) for q steps. First, assuming that the moving body 1 and user U are moving along the candidate additional route Rmc(j) for the moving body 1 and the ideal route Ru#(j) for the user U, various forces acting on the pedestrian P(k) in the current step are calculated (SS4). At this time, as described above, the influence F1 is corrected in an increasing direction for the point and time when the alarm device 50 is activated. The prediction unit 130 predicts the position and velocity of the pedestrian P(k) one step ahead, assuming that the virtual force continues to act for one step (SS5). By performing this process for q steps for each pedestrian, a predicted route Rp(k, j) for each pedestrian P(k) corresponding to the candidate additional route Rmc(j) is generated and output (SS6).

[0039] By correcting the influence F1 in an increasing direction, it is predicted that the target pedestrian P will move away from the moving object 1 with a larger influence F1 than usual. As a result, more space is created for the user U to move around, and as a result, it is expected that the various scores (except for the third score SC3) described below will improve.

[0040] Next, the prediction unit 130 predicts the future location of the user U according to the flowchart of Fig. 9. Fig. 8 is a flowchart showing an example of the processing content of the prediction unit 130 regarding the user U.

[0041] In the following description, the additional route candidate and the route candidate will not be distinguished from each other and will simply be referred to as the route candidate Pmc. The prediction unit 130 performs steps S11 to S14 for each route candidate Rmc(w). The parameter w is the combination of parameters i and j and can take values ​​from 1 to n+h. The prediction unit 130 inputs the predicted route of each pedestrian corresponding to the route candidate Rmc(w) (S11). Next, the prediction unit 130 executes steps S12 and S13 for q steps. The prediction unit 130 calculates various forces acting on the user U in the current step, assuming that the moving object 1 and each pedestrian P(w) will move according to the route candidate Rmc(w) and each pedestrian's predicted route Rp(k,w) (S12). The method for calculating these various forces may be the same as the process of S4 in FIG. 4. The prediction unit 130 predicts the position and velocity of the user U one step ahead, assuming that a virtual force continues to act during one step (S13). By executing this process for q steps, a predicted route Ru(w) for the user U for each route candidate Rmc(w) is generated and output (S14). This predicted route Ru(w) is the result of predicting the future location of the user U for each route candidate Rmc(w).

[0042] [Evaluation of Route Candidates] When the score calculation unit 124 obtains the predicted route Ru(w), ideal route Ru#(w), and predicted route Rp(k, w) corresponding to w = 1 to n + h, respectively, from the prediction unit 130 as prediction results, it calculates an overall score SC(w) for each route candidate Rmc(w). The overall score SC(w) is expressed by equation (1). α, β, and γ are coefficients, with SC1(w) being the first score, SC2(w) being the second score, and SC3(w) being the third score. The smaller the overall score SC(w), the better the evaluation, and the coefficients α, β, γ, and ζ and η, which will be described later, are assumed to be positive values.

[0043] SC(w) = α・SC1(w)+β・SC2(w)+γ・SC3(w)…(1)

[0044] The first score SC1(w) is a score based on the positional relationship between the moving body 1 and the destination point assigned to the moving body 1. For example, the first score SC1(w) is calculated so that it increases as the distance X1 between the end point (the point farthest from the moving body 1) of the route candidate Rmc(w) and the destination point assigned to the moving body 1 increases, and decreases as the distance X1 decreases. The first score SC1(w) may use the distance X1 as is, or may be the logarithm or exponent of the distance X1, or may be the output value of some function that uses the distance as an input value.

[0045] The second score SC2(w) is a score based on the positional relationship between the user U and the pedestrian P from the present to a future time (the time when the user U reaches the end of the route). The second score SC2(w) has a more positive value as the distance between the user U and the pedestrian P is maintained. For example, the second score SC2(w) is calculated based on equation (2). SC4(w) is the fourth score, and SC5(w) is the fifth score.

[0046] SC2(w)=ζ・SC4(w)+η・SC5(w)…(2)

[0047] For example, the fourth score SC4(w) is calculated by calculating the distance between the route points of the user U's predicted route Ru(w) and the route points of the predicted routes Rp(k,w) of all pedestrians P for each step. The fourth score SC4(w) increases as the shortest distance X2 across all steps (i.e., the distance at which the user U and any pedestrian P are closest to each other within q steps) decreases, and decreases as the distance X2 increases. However, if the distance X2 is sufficiently long (the distance between the user U and the pedestrian P is sufficiently maintained), the fourth score SC4(w) may indicate a constant value. For example, if the distance X2 is equal to or greater than a boundary value of approximately 2 m, the fourth score SC4(w) may indicate zero. FIG. 9 is a diagram illustrating an example of the characteristics of the fourth score SC4(w). It is preferable that the fourth score SC4(w) be calculated so that it rapidly increases as the distance X2 decreases in order to prevent the user U and the pedestrian P from getting too close. For example, a barrier function or an exponential function is preferably used, but any function showing a similar trend may be used.

[0048] The fifth score SC5(w) is calculated so that it increases as the degree of deviation between the predicted route Ru(w) and the ideal route Ru#(w) increases. For example, the score calculation unit 124 squares the distance between each route point of the predicted route Ru(w) and the ideal route Ru#(w) for each step, and calculates the square of the sum as the fifth score SC5(w). FIG. 10 is a diagram for explaining the calculation process of the fifth score SC5(w). In the diagram, r1 is the distance between the route point of the first step of the predicted route Ru(w) and the ideal route Ru#(w), r2 is the distance between the route point of the second step of the predicted route Ru(w) and the ideal route Ru#(w), and r3 is the distance between the route point of the third step of the predicted route Ru(w) and the ideal route Ru#(w). The fifth score SC5(w) can be calculated, for example, by √(r1 ２ +r2 ２ +r3 ２ ) can be calculated as

[0049] The third score SC3(w) is a score (a score obtained by evaluating the activation) for imposing a penalty on the activation of the alarm device 50. For example, the third score SC3(w) is set to a value that increases as the number of activations of the alarm device 50 increases. Alternatively (or in addition), the third score SC3(w) may be a sum of the amounts of behavioral changes of surrounding pedestrians P caused by the activation of the alarm device 50. In the latter case, a difference vector between the movement vectors of the pedestrian P for each step may be calculated, and the sum of the absolute values ​​(magnitudes) of the difference vectors may be used as the third score SC3(w).

[0050] The score calculation unit 124 calculates the overall score SC(w) for each route candidate Rmc(w), and then selects the route candidate Rmc(w) with the smallest overall score SC(w) and outputs it to the drive control unit 140 as the route Rm of the moving object 1. The drive control unit 140 controls the drive unit 30 attached to the moving object 1 so that the moving object 1 moves along the route Rm. Furthermore, if the selected route candidate Rmc(w) involves activation of the notification device 50, the notification control unit 150 activates the notification device 50 based on the activation plan for the notification device 50 described above.

[0051] By performing this processing, the mobile object 1 is controlled to simultaneously lead (guide) the user U to the given destination point by heading toward the destination point and prevent other pedestrians P from getting too close to the user U due to its own presence. For example, in a situation where the pedestrian P is approaching the user U, the overall score of a route candidate Rmc(w) in which the mobile object 1 gets between the pedestrian P and the user U to prevent the pedestrian P from approaching should be relatively high. Furthermore, by reflecting the activation evaluation of the notification device 50, the notification device 50 can be used effectively and excessive activation of the notification device 50 can be prevented from causing discomfort to the pedestrian P. As a result, the mobile object can be controlled to behave appropriately and have a high affinity with surrounding pedestrians P.

[0052] 11 and 12 are diagrams illustrating an example in which the moving body 1 inhibits the approach of the user U and the pedestrian P. In FIGS. 11 and 12, the left diagram shows the movement path of the pedestrian P when the coefficient β is set to 0 and the route Rm is selected, which simply takes into consideration the route to the destination. The right diagram shows the movement path of the pedestrian P when the coefficient β is set to a value sufficiently greater than 0. In the example on the right side of FIG. 11, the moving body 1 takes the route Rm that bulges to the right, thereby forcing the pedestrian P to take a route that is closer to the left (due to the influence F1), thereby inhibiting the pedestrian P from approaching the user U. In the example on the right side of FIG. 12, the moving body 1 deliberately moves at a slow speed, thereby forcing the pedestrian P to take a route that is closer to the right (due to the influence F1), thereby inhibiting the pedestrian P from approaching the user U. In this way, the control device 100 can generate a route for the moving body 1 so that the moving body 1 behaves appropriately toward the user U.

[0053] [Weight (Coefficient) Adjustment] The coefficients α, β, γ, ζ, and η may be fixed or adjusted by the weight value adjustment unit 126. The weight value adjustment unit 126, for example, fixes α and varies β, γ, ζ, and η to change the movement tendency of the moving object 1. Increasing the coefficients β and ζ causes the moving object 1 to prioritize preventing pedestrians P from approaching the user U, but on the other hand, increases the likelihood of the moving object 1 reaching the destination later. Therefore, adjusting the coefficients according to the situation of the moving object 1 and the user U allows for more appropriate route generation. Increasing the coefficient γ controls the moving object 1 to minimize activation of the alarm device 50. Conversely, decreasing the coefficient γ causes the moving object 1 to behave in a manner that prioritizes the user U's movement space and prevents pedestrians P from approaching too closely. Note that the weight value adjustment unit 126 may implement an operation mode in which the coefficient β is set to zero or an operation mode in which the coefficient η is set to zero in response to input to the HMI 10.

[0054] For example, the weight value adjustment unit 126 may adjust the coefficient β based on an index value that indicates the degree of influence that the pedestrian P has on the user U, such as the influence F1 described above. More specifically, when multiple pedestrians P are recognized, the weight value adjustment unit 126 may extract the largest influence F1 among the pedestrians P, and adjust the coefficient β so that the larger the largest influence F1, the larger the coefficient β. Furthermore, similar processing may be performed based on the sum of the influences F1 of the multiple pedestrians P.

[0055] Furthermore, the weight value adjustment unit 126 may adjust the coefficient β based on the number m of pedestrians P. More specifically, the weight value adjustment unit 126 may adjust the coefficient β so that the coefficient β increases as the number m of pedestrians P increases.

[0056] Furthermore, the weight value adjustment unit 126 may adjust the coefficient β based on both the influence F1 and the number m of pedestrians P. More specifically, the weight value adjustment unit 126 may perform the same processing as above based on the value obtained by multiplying the maximum value or total value of the influence F1 by the number m of pedestrians P.

[0057] Furthermore, the weight value adjustment unit 126 may adjust the coefficient β based on the distance between the moving body 1 and the user U. More specifically, the weight value adjustment unit 126 may adjust the coefficient β so that the coefficient β increases as the distance between the moving body 1 and the user U increases. This is because a long distance between the moving body 1 and the user U indicates that the user U is lagging behind in following the moving body 1 for some reason, and therefore it is inferred that it is necessary to move the moving body 1 closer to the user U to prevent the approach of pedestrians P.

[0058] Furthermore, the weight value adjustment unit 126 may adjust the coefficient β based on the speed of the user U. More specifically, the weight value adjustment unit 126 may adjust the coefficient β so that the coefficient β increases as the speed of the user U decreases. This is because a slow speed of the user U indicates that the user U is lagging behind in following the moving body 1 for some reason, and therefore it is inferred that it is necessary to move the moving body 1 closer to the user U to prevent the approach of the pedestrian P.

[0059] 13 is a flowchart showing an example of the flow of processing executed by the weight value adjustment unit 126 when all of the above-described elements are reflected. First, the weight value adjustment unit 126 determines whether the maximum value maxF1 of the influence F1 is equal to or greater than the threshold value Th1 (S20). If the maximum value maxF1 of the influence F1 is equal to or greater than the threshold value Th1, the weight value adjustment unit 126 increases the coefficient β by a first predetermined value A1 (S21). Here, the influence F1 may or may not include an increase correction due to the activation of the alarm device 50.

[0060] Next, the weight value adjuster 126 determines whether the number m of pedestrians P is equal to or greater than a threshold value Th2 (S22). If the number m of pedestrians P is equal to or greater than the threshold value Th2, the weight value adjuster 126 increases the coefficient β by a second predetermined value A2 (S23).

[0061] Next, the weight value adjuster 126 determines whether the distance Xmu between the moving object 1 and the user U is equal to or greater than a threshold value Th3 (S24). If the distance Xmu is equal to or greater than the threshold value Th3, the weight value adjuster 126 increases the coefficient β by a third predetermined value A3 (S25).

[0062] Next, the weight value adjuster 126 determines whether the speed Vu of the user U is less than a threshold value Th4 (S26). If the speed Vu is less than the threshold value Th4, the weight value adjuster 126 increases the coefficient β by a fourth predetermined value A4 (S27).

[0063] According to the embodiment described above, the moving body 1 can be controlled so as to behave appropriately, taking into account the operation evaluation of the alarm device 50 .

[0064] In the above description, the control device 100 is described as being mounted on the mobile body 1, but this is not limited to this. The control device 100 may be installed at a location away from the mobile body 1, acquire output data from the object detection device 20 via communication, and transmit a drive instruction signal to the drive unit 30, i.e., remotely control the mobile body 1.

[0065] The above-described embodiment can be expressed as follows: A control device for controlling a mobile body that autonomously moves in an area where other pedestrians walk while at least temporarily leading a target lead person, the control device comprising: one or more storage media for storing computer-readable instructions; and a processor connected to the one or more storage media, wherein the processor executes the computer-readable instructions to: recognize objects including the target lead person and the other pedestrians; predict future positions of the recognized other pedestrians; generate a future route for the mobile body to follow; and control a drive unit attached to the mobile body so that the mobile body moves along the route, wherein generating the route includes setting a plurality of route candidates and, for each route candidate, selecting one of the route candidates based on a score that reflects an operation evaluation of a notification device that is mounted on the control device and that notifies the other pedestrians, and setting the selected route candidate as the route.

[0066] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0067] REFERENCE SIGNS LIST 1 Mobile object 20 Object detection device 30 Drive unit 50 Notification unit 100 Control unit 110 Recognition unit 120 Action plan generation unit 122 Route candidate setting unit 124 Score calculation unit 126 Weight value adjustment unit 130 Prediction unit 140 Drive control unit 150 Notification control unit

Claims

1. A control device for controlling a moving body which moves autonomously in an area where other pedestrians walk while at least temporarily leading a leading subject, comprising: a recognition unit which recognizes objects including the leading subject and the other pedestrians; a prediction unit which predicts the future positions of the recognized other pedestrians; a behavior plan generation unit which generates a route which the moving body should take in the future; and a drive control unit which controls a drive unit attached to the moving body so that the moving body moves along the route, wherein the behavior plan generation unit sets a plurality of route candidates, and for each of the route candidates, selects one of the route candidates based on a score which reflects an operation evaluation of an alarm device mounted on the control device which alerts the other pedestrians, and sets this as the route.

2. The control device according to claim 1, wherein the behavior plan generation unit further comprises an alarm control unit that generates an operation plan for the alarm device and controls the alarm device based on the operation plan for the alarm device.

3. The control device as described in claim 1, wherein the action plan generation unit calculates a first score based on the positional relationship between the moving body and a predetermined destination point, a second score based on the positional relationship between the lead subject and the other pedestrians from the present to a future time point, and a third score which is a result of evaluating the operation of the alarm device, and selects one of the route candidates based on the first score, the second score, and the third score to set this as the route.

4. The control device as described in claim 3, wherein the first score is a positive value the closer the moving body is to a predetermined destination point, and the second score is a positive value the greater the distance between the leading object and the other pedestrians.

5. The control device according to claim 1, wherein the future time is a time when the moving object will reach an end point of each of the plurality of route candidates.

6. The control device according to claim 3, wherein the action plan generation unit selects one of the route candidates based on the score obtained by multiplying each of the first score, the second score, and the third score by a weighting value.

7. The control device according to claim 6, wherein the action plan generating unit is capable of changing a weight of the third score.

8. The control device according to claim 1, wherein the notification device is a device that outputs a sound, displays an image, and / or projects light.

9. A control method executed by a processor of a control device that controls a moving body that moves autonomously in an area where other pedestrians walk while at least temporarily leading a leading person, the control method comprising: recognizing objects including the leading person and the other pedestrians; predicting future positions of the recognized other pedestrians; generating a future route for the moving body; and controlling a drive unit attached to the moving body so that the moving body moves along the route, wherein generating the route includes setting a plurality of route candidates, and selecting one of the route candidates based on a score that reflects an operation evaluation of an alarm device mounted on the control device that alerts the other pedestrians, and setting the selected route as the route.

10. A program that causes a processor of a control device that controls a moving body that moves autonomously in an area where other pedestrians walk while at least temporarily leading a leading person, to execute the following operations: recognizing objects including the leading person and the other pedestrians; predicting the future positions of the recognized other pedestrians; generating a future route for the moving body; and controlling a drive unit attached to the moving body so that the moving body moves along the route, wherein generating the route includes setting a plurality of route candidates and selecting one of the route candidates based on a score that reflects an operation evaluation of an alarm device mounted on the control device that alerts the other pedestrians, and setting the selected route as the route.

Images