Filter creation method, filter creation device, program, and filter device

Abstract

A filter creation method according to the present invention comprises: acquiring a plurality of feature amounts of a plurality of pedestrians that serve as samples; acquiring a reference route that is a route of a moving body and generated by a function of a control device on the basis of the presence of the plurality of pedestrians; acquiring individual routes that are routes of the moving body and generated by a function of the control device on the basis of the presence of pedestrian groups in which pedestrians are excluded from the plurality of pedestrians one by one; extracting an individual route that has a smaller difference from the reference route as compared with the other individual routes; extracting a pedestrian that corresponds to the extracted individual route; and generating a trained model by machine learning such that, when the feature amount of the extracted pedestrian is input to the trained model, information indicating that the extracted pedestrian is a pedestrian to be excluded is output.

Description

FILTER CREATION METHOD, FILTER CREATION DEVICE, PROGRAM, AND FILTER DEVICE

[0001] The present invention relates to a filter creation method, a filter creation device, a program, and a filter device.

[0002] In recent years, moving objects (called robots, micromobility, etc.) that move autonomously while following a user for purposes such as transporting the user's luggage have been put into practical use. An invention of a driving control device related to micromobility has been disclosed (Patent Document 1). Research is also being conducted on moving objects that not only follow a user but also move autonomously while leading the user.

[0003] Special Publication No. 2020-529050

[0004] The above-mentioned mobile objects may move in areas where pedestrians are present (floors, sidewalks, open spaces, etc.). Unlike vehicles moving on roadways, when a mobile object moves through an area where there are many pedestrians, it must monitor many pedestrians as targets for collision avoidance, which may result in an excessive processing load on the control device. For this reason, it is desirable to narrow down the pedestrians to be monitored, but conventional technologies have not fully considered how to narrow down the pedestrians to be monitored.

[0005] The present invention has been made in consideration of these circumstances, and one of its objects is to provide a filter creation method, a filter creation device, a program, and a filter device that can assist a control device in appropriately narrowing down the pedestrians to be monitored.

[0006] The filter creation method, filter creation device, program, and filter device according to the present invention employ the following configuration: (1): A filter creation method according to one aspect of the present invention is a filter creation method in which a filter creation device creates a filter, the filter being a control device that controls a moving object that autonomously moves in an area where pedestrians walk, the control device recognizing objects including the pedestrians and generating a future path for the moving object based on the recognition result, and the control device is a filter for narrowing down pedestrians to be monitored, the filter creation method including: acquiring a plurality of feature amounts of a plurality of sample pedestrians; acquiring a reference route that is a path of the moving object generated by a function of the control device based on the presence of the plurality of pedestrians; acquiring individual routes that are paths of the moving object generated by a function of the control device based on the presence of a group of pedestrians obtained by excluding pedestrians one by one from the plurality of pedestrians; extracting an individual route that differs from the reference route less than other individual routes; extracting a pedestrian corresponding to the extracted individual route; and generating the trained model by machine learning such that, when the feature amounts of the extracted pedestrian are input to the trained model, information indicating that the pedestrian is a pedestrian to be excluded is output.

[0007] (2): In the above aspect (1), the plurality of features include an average value of the minimum distances between the pedestrian of interest and a predetermined number of other pedestrians in descending order of proximity to the pedestrian of interest during a predetermined observation period.

[0008] (3) In the aspect (1), the plurality of feature amounts include the speed of the pedestrian.

[0009] (4): In the above aspect (1), the control device generates a route along which the moving body will move while leading the leading person, and the plurality of feature quantities include the distance from the straight line connecting the moving body and the leading person to the pedestrian.

[0010] (5) In the aspect (1) above, the plurality of feature amounts include a length of time that the pedestrian stays in the monitored area.

[0011] (6) In the aspect (1) above, the plurality of feature amounts include a direction of the pedestrian as seen from the moving body.

[0012] (7) In the aspect (1) above, the plurality of feature amounts include the orientation of the pedestrian in a coordinate system based on the moving body.

[0013] (8) A filter creation device according to another aspect of the present invention is a filter creation device that creates a filter, wherein the filter is a control device that controls a moving body that autonomously moves in an area where pedestrians walk, and the control device recognizes objects including the pedestrians and generates a future path for the moving body based on the recognition result, and the control device is a filter for narrowing down pedestrians to be monitored, and the filter includes: a feature acquisition unit that acquires multiple feature amounts of multiple sample pedestrians; a reference path acquisition unit that acquires a reference path, which is the path of the moving body generated by a function of the control device based on the presence of the multiple pedestrians; an individual path acquisition unit that acquires individual paths, which are the paths of the moving body generated by a function of the control device based on the presence of a group of pedestrians obtained by excluding one pedestrian at a time from the multiple pedestrians; an extraction unit that extracts individual paths that have a smaller difference from the reference path than other individual paths and extracts the pedestrian corresponding to the extracted individual path; and a learning unit that generates the trained model by machine learning so that when the feature amounts of the extracted pedestrian are input to the trained model, information indicating that the pedestrian is a pedestrian to be excluded is output.

[0014] (9): A program according to another aspect of the present invention is a program to be executed by a filter creation device that creates a filter, the filter being a control device that controls a moving body that autonomously moves in an area where pedestrians walk, the control device recognizing objects including the pedestrians and generating a future path for the moving body based on the recognition results, the control device being a filter for narrowing down pedestrians to be monitored, the program causing the filter creation device to execute the following steps: acquire a plurality of feature amounts of a plurality of sample pedestrians; acquire a reference route that is the path of the moving body generated by a function of the control device based on the presence of the plurality of pedestrians; acquire individual routes that are the path of the moving body generated by a function of the control device based on the presence of a group of pedestrians obtained by excluding one pedestrian at a time from the plurality of pedestrians; extract an individual route that has a smaller difference from the reference route than other individual routes; extract a pedestrian corresponding to the extracted individual route; and generate the trained model by machine learning so that, when the feature amounts of the extracted pedestrian are input to the trained model, information indicating that the pedestrian is a pedestrian to be excluded is output.

[0015] (10): A filter device according to another aspect of the present invention is a filter device applied to a control device that controls a moving body that autonomously moves in an area where pedestrians walk, and that recognizes objects including the pedestrians and generates a future path for the moving body based on the recognition results, wherein a filter creation device acquires multiple feature amounts of multiple sample pedestrians, acquires a reference path that is the path of the moving body that is generated by a function of the control device based on the presence of the multiple pedestrians, acquires individual paths that are the path of the moving body that is generated by a function of the control device based on the presence of a group of pedestrians obtained by excluding one pedestrian at a time from the multiple pedestrians, extracts individual paths that have a smaller difference from the reference path than other individual paths, extracts pedestrians corresponding to the extracted individual paths, and generates the trained model by machine learning so that when the feature amounts of the extracted pedestrians are input to the trained model, information indicating that the pedestrian is a pedestrian to be excluded is output. The filter created by executing this process narrows down the pedestrians to be monitored by the control device.

[0016] According to aspects (1) to (10), it is possible to assist the control device in appropriately narrowing down the pedestrians to be monitored.

[0017] 1 is a diagram illustrating an example of the relationship between a moving body and a filter creation device. FIG. 2 is a diagram illustrating the configuration of a moving body. FIG. 3 is a diagram illustrating the configuration of a control device. FIG. 4 is a diagram illustrating an example of a route candidate. FIG. 5 is a flowchart illustrating an example of the processing content of a prediction unit related to a pedestrian. FIG. 6 is a diagram illustrating an example of a method for generating an ideal route. FIG. 7 is a flowchart illustrating an example of the processing content of a prediction unit related to a user. FIG. 8 is a diagram illustrating an example of the characteristics of a third score. FIG. 9 is a diagram illustrating the calculation process of a fourth score. FIG. 10 is a diagram illustrating an example of a scene in which a moving body inhibits approach between a user and a pedestrian. FIG. 11 is a diagram illustrating an example of a scene in which a moving body inhibits approach between a user and a pedestrian. FIG. 12 is a diagram illustrating the configuration of a filter creation device. FIG. 13 is a diagram illustrating each of feature amounts. FIG. 14 is a diagram illustrating the difference between a reference route and an individual route. FIG. 15 is a flowchart illustrating an example of the processing flow of a filter creation device. FIG. 16 is a diagram illustrating the contents of a trained model.

[0018] [Overview] Hereinafter, with reference to the drawings, embodiments of a filter creation method, a filter creation device, a program, and a filter device of the present invention will be described. The filter according to the present invention is applied to a mobile body control device that controls a driving device of the mobile body to move the mobile body. The mobile body, for example, autonomously moves through an area where pedestrians walk, leading a leading subject. Alternatively, the mobile body may move independently or follow a following subject at a certain distance. In the following description, the mobile body is assumed to lead a leading subject. The area where pedestrians walk may be a sidewalk, a public open space, a floor within a building, or the like, and may include a roadway. In the following description, it is assumed that no human occupants are present on the mobile body, but a human occupancy is also possible. The leading subject may be, for example, a pedestrian, but may also be a robot or an animal (hereinafter referred to as user U). For example, while heading toward a predetermined destination, the mobile body moves slightly ahead of an elderly user U to prevent other pedestrians who may obstruct the user U's movement from getting too close to the user U (i.e., operates to clear a path for the user U). The user U is not limited to elderly people, but may also be people who tend to have difficulty walking, children, people shopping at a supermarket, patients moving around in a hospital, pets on walks, etc. Note that such actions do not have to be performed constantly, but may be performed temporarily. For example, when a moving object runs alongside or pursues a user and detects a predetermined situation in the user's direction of travel (e.g., the presence of an obstacle or traffic congestion), the moving object may temporarily lead the user by executing a predetermined algorithm.

[0019] [Overall Configuration] Fig. 1 is a diagram showing an example of the relationship between a moving body 1 and a filter creation device 200. The filter creation device 200 includes, for example, a simulator 270 that simulates the functions of a control device 100 (described later) of the moving body 1. Alternatively, the filter device 200 may use an actual moving body 1 in place of the simulator 270. The filter creation device 200 acquires sample data 310, which is data on multiple pedestrians, and creates a filter to be used by the moving body 1 based on the sample data 310. The created filter (filter program) is installed in the moving body 1. Below, the moving body 1 will be described first, followed by a description of the filter creation device 200.

[0020] 2 is a configuration diagram of the mobile body 1. The mobile body 1 is equipped with, for example, an HMI (Human Machine Interface) 10, an object detection device 20, a driving device 30, a sensor 40, and a control device 100. These components are supported or housed by a base 5.

[0021] The HMI 10 presents various information to the user U and accepts input operations by the user U. The HMI 10 includes various display devices, speakers, buzzers, touch panels, switches, keys, short-range wireless communication devices, etc. For example, the HMI 10 accepts the setting of a destination point.

[0022] The object detection device 20 is a device that generates data for recognizing other pedestrians and the user U present around the moving body 1. The object detection device 20 includes, for example, a camera having an imaging range covering the periphery of the moving body 1. The object detection device 20 may include sensors such as a radar device, a LIDAR (Light Detection and Ranging), and an ultrasonic sensor, as well as an object recognition device that identifies an object by performing sensor fusion processing based on the outputs of these sensors.

[0023] The drive device 30 is a mechanism for moving the mobile body 1 including the base body 5 in any direction. The drive device 30 includes, for example, a plurality of wheels, a drive motor attached to one or more wheels, and a steering device attached to one or more wheels. There are no particular restrictions on the configuration of the drive device 30, and the drive device 30 may have any configuration. In principle, the drive device 30 moves the mobile body 1 while keeping the front surface of the base body 5 facing the traveling direction of the mobile body 1.

[0024] The sensor 40 is a sensor for detecting the behavior of the mobile body 1. The sensor 40 includes, for example, a wheel speed sensor for detecting the wheel speed, an acceleration sensor for detecting the acceleration acting on the mobile body 1, a yaw rate sensor attached near the center of gravity of the base body 5 in the horizontal direction, a steering angle sensor for detecting the steering angle of the steered wheels (steered wheels), and an orientation sensor for detecting the orientation of the mobile body 1 in the horizontal direction.

[0025] FIG. 3 is a configuration diagram of the control device 100. The control device 100 includes, for example, a recognition unit 110, a filter processing unit 115, a path generation unit 120, a prediction unit 130, and a drive control unit 140. The path generation unit 120 includes a path candidate setting unit 122 and a score calculation unit 124. These components are implemented by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be implemented by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be implemented by a combination of software and hardware. The program may be stored in advance in a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device with a non-transitory storage medium), or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed in the storage device by inserting the storage medium into a drive device.

[0026] The recognition unit 110 recognizes objects, including other pedestrians (hereinafter simply referred to as pedestrian P) and the user U, based on the data output by the object detection device 20. When the object detection device 20 is a camera, the recognition unit 110 recognizes pedestrians by inputting camera images into a trained model for identifying pedestrians. The same applies to objects other than pedestrians. To distinguish between the pedestrian P and the user U, the recognition unit 110 may store multiple images of the user U captured by a camera in advance as templates in a storage unit (not shown) and identify the user U by comparing the templates with the camera images. Alternatively, the recognition unit 110 may recognize the location of the user U by using communication directivity to perform short-range wireless communication between the HMI 10 and a terminal device held by the user U.

[0027] The filter processing unit 115 extracts one or more pedestrians P to be monitored (used as a basis for determining whether to control the moving object 1) from among multiple pedestrians P. The filter processing unit 115 operates according to a filter (filter program) created by the filter creation device 200. The filter processing unit 115 is an example of a "filter device." In the following description, a pedestrian P before processing by the filter processing unit 115 is referred to as a pedestrian candidate Pc, and a pedestrian P extracted by the filter processing unit 115 is referred to as a pedestrian P. First, the filter processing unit 115 derives multiple feature quantities for each pedestrian candidate Pc based on the recognition result of the recognition unit 110. The content of the feature quantities is the same as that described in the description of the filter creation device 200 below. The derived feature quantities are then input into the filter to determine whether or not to exclude the pedestrian candidate Pc from the monitoring targets. A pedestrian candidate Pc not excluded by the filter processing unit 115 is treated as a pedestrian P from now on.

[0028] [Route Candidates] The route candidate setting unit 122 of the route generation unit 120 generates multiple route candidates Pmc(i) that are candidates for the route Rm along which the moving object 1 will travel in the future. i is an identifier for the route candidate and takes values ​​from 1 to n (n is a natural number greater than or equal to 2). Routes and route candidates are generated, for example, as a series of multiple (q) route points spaced at a predetermined distance. The route points represent points that the moving object 1 should reach at predetermined intervals (each step). FIG. 4 shows an example of a route candidate Rmc(i). Here, n = 5 and q = 3. The route candidate setting unit 122 generates route candidates Rmc(1), Rmc(2), Rmc(4), and Rmc(5) that extend to the left and right, centered on route candidate Rmc(3), which extends linearly along the forward direction Dm of the moving object 1. The distance between route points is, for example, constant, and each route candidate has a length of q steps. The route candidates are not generated so as to reach the destination point given to the mobile unit 1, but are generated so as to have a length of a fixed distance x q steps. For example, route candidate Rmc(2) is generated so as to turn left by a predetermined angle in each step. Route candidate Rmc(1) is generated so as to turn left by a larger predetermined angle in each step. The opposite is true for route candidates Rmc(4) and Rmc(5). The above rule is merely an example, and route candidates may be generated based on other rules as long as they show a similar trend (gradual widening). The route candidate setting unit 122 outputs the route candidate Rmc(i) to the prediction unit 130.

[0029] In the example of FIG. 4 , the intervals between the route points are constant, but the intervals may be variable. A long interval means that the route between the route points is moved at a relatively high speed, and a short interval means that the route between the route points is moved at a relatively low speed. In other words, the route Rm and the route candidates Rmc(i) may include a speed factor. For example, the route candidate setting unit 122 generates derivative route candidates with varying speeds based on route candidates generated with a constant interval between the route points.

[0030] [Prediction (Simulation on Actual Machine)] When a route candidate Rmc(i) is given, the prediction unit 130 predicts the future positions of the pedestrian P and the user U for each route candidate Rmc(i) based on the route candidate Rmc(i). The prediction unit 130 predicts the future positions of the pedestrian P and the user U by, for example, any of the methods described in Non-Patent Documents 1 to 3, or a combination of these methods.

[0031] 5 is a flowchart showing an example of the processing content of the prediction unit 130 regarding a pedestrian P. First, the prediction unit 130 performs goal prediction for each pedestrian P (S1). For example, the prediction unit 130 observes the movement trajectory of the pedestrian P in time series, and predicts that the goal (arrival point) of the pedestrian P is the destination of the movement vector of the pedestrian P in a state where there are no other pedestrians P, moving bodies 1, or users U in the vicinity.

[0032] Next, the prediction unit 130 performs the processes of S2 to S6 for each route candidate Rmc(i). The prediction unit 130 sequentially inputs the route candidates Rmc(i) for the moving body 1 one by one (S2), and generates an ideal route Ru#(i) for the user U to follow the moving body 1 in association with each of the route candidates Rmc(i) (S3). The ideal route Ru#(i) is a route along which the user U is expected to travel if it is assumed that a pedestrian P is not present in the vicinity of the moving body 1 and the user U. FIG. 6 is a diagram for explaining an example of a method for generating an ideal route Ru#(i). Here, an example is shown in which an ideal route Ru#(5) corresponding to the route candidate Rmc(5) is generated. For example, the prediction unit 130 generates the ideal route Ru#(5) of the user U so that the movement vector Vu-1 of the user U heading from step 0 (the calculation time point) to step 1 points toward the position Om-0 of the moving body 1 at step 0, so that the movement vector Vu-2 of the user U heading from step 1 to step 2 points toward the position Om-1 of the moving body 1 at step 1, and so that the movement vector Vu-3 of the user U heading from step 2 to step 3 points toward the position Om-2 of the moving body 1 at step 2, that is, so that the user U follows the moving body 1 with a one-step delay. This generation method is merely an example, and other generation methods may be adopted.

[0033] 5, the prediction unit 130 executes the processes of S4 and S5 for each pedestrian P (k = 1 to m) for q steps. First, assuming that the moving body 1 and the user U move along the candidate route Rmc(i) for the moving body 1 and the ideal route Ru#(i) for the user U, respectively, various forces acting on the pedestrian P(k) in the current step are calculated (S4). These forces do not actually act, but are hypothetical forces that are considered to be generated by the psychological movements of the pedestrian P(k).

[0034] The virtual forces include, for example, (1) an influence F1 generated by the approach of another object (here, the moving body 1 and the user U), (2) a turning force F2 that corrects direction toward the goal after changing course mainly due to the influence F1, and (3) an acceleration / deceleration force F3 that adjusts speed to return to a desired speed after accelerating or decelerating mainly due to the influence F1. F1 is a translational force acting in any direction on a two-dimensional plane, F2 is a rotational force centered on the pedestrian, and F3 is a translational force parallel to the movement vector of the pedestrian. F1 to F3 are simply expressed, for example, as follows. In addition to these, integral elements, differential elements, etc. may be added to calculate the virtual forces.

[0035] F1 = (weight) x (coefficient) x (relative speed) / (distance to other objects) F2 = (moment of inertia) x (coefficient) x (angle difference between the most recent movement vector and the direction toward the goal) F3 = weight x (coefficient) x (difference between desired speed and most recent speed)

[0036] The prediction unit 130 predicts the position and velocity of pedestrian P(k) one step ahead, assuming that the virtual force continues to act for one step (S5). By performing this process for q steps for each pedestrian, a predicted route Rp(k,i) for each pedestrian P(k) is generated and output (S6). This predicted route Rp(k,i) is the result of predicting the future position of each pedestrian P(k) for each route candidate Rmc(i).

[0037] Next, the prediction unit 130 predicts the future location of the user U according to the flowchart of Fig. 7. Fig. 7 is a flowchart showing an example of the processing content of the prediction unit 130 regarding the user U.

[0038] The prediction unit 130 performs the processes of S11 to S14 for each route candidate Rmc(i). The prediction unit 130 inputs the predicted route of each pedestrian corresponding to the route candidate Rmc(i) (S11). Next, the prediction unit 130 executes the processes of S12 and S13 for q steps. The prediction unit 130 calculates various forces acting on the user U in the current step, assuming that the moving object 1 and each pedestrian P(k) will move according to the route candidate Rmc(i) and each pedestrian's predicted route Rp(k,i) (S12). The method for calculating these various forces may be the same as the process of S4 in FIG. 5. The prediction unit 130 predicts the position and velocity of the user U one step ahead, assuming that a virtual force continues to act during one step (S13). By executing this process for q steps, the prediction unit 130 generates and outputs a predicted route Ru(i) of the user U for each route candidate Rmc(i) (S14). This predicted route Ru(i) is the result of predicting the future location of the user U for each route candidate Rmc(i).

[0039] [Evaluation of Route Candidates] When the score calculation unit 124 obtains the predicted route Ru(i), ideal route Ru#(i), and predicted route Rp(k,i) corresponding to i = 1 to n as prediction results from the prediction unit 130, it calculates a total score SC(i) for each route candidate Rmc(i). The total score SC(i) is expressed by equation (1). α and β are coefficients, with SC1(i) ​​being the first score and SC2(i) being the second score. The smaller the value of the total score SC(i), the better the evaluation, and the coefficients α and β, as well as ζ and η, which will be described later, are assumed to be positive values.

[0040] SC(i)=α・SC1(i)+β・SC2(i)…(1)

[0041] The first score SC1(i) ​​is a score based on the positional relationship between the moving body 1 and the destination point assigned to the moving body 1. For example, the first score SC1(i) ​​is calculated so that it increases as the distance X1 between the end point (the point farthest from the moving body 1) of the route candidate Rmc(i) and the destination point assigned to the moving body 1 increases, and decreases as the distance X1 decreases. The first score SC1(i) ​​may use the distance X1 as is, or may be the logarithm or exponent of the distance X1, or may be the output value of some function that uses the distance as an input value.

[0042] The second score SC2(i) is a score based on the positional relationship between the user U and the pedestrian P from the present to a future time (the time when the user U reaches the end of the route). The second score SC2(i) has a more positive value as the distance between the user U and the pedestrian P is maintained. For example, the second score SC2(i) is calculated based on equation (2). SC3(i) is the third score, and SC4(i) is the fourth score.

[0043] SC2(i)=ζ・SC3(i)+η・SC4(i)…(2)

[0044] For example, the third score SC3(i) is calculated by calculating the distance between the route points of the user U's predicted route Ru(i) and the route points of all pedestrians P's predicted routes Rp(k,i) for each step. The third score SC3(i) increases as the shortest distance X2 across all steps (i.e., the distance at which the user U and any pedestrian P are closest to each other within q steps) decreases, and decreases as the distance X2 increases. However, if the distance X2 is sufficiently long (the distance between the user U and the pedestrian P is sufficiently maintained), the third score SC3(i) may indicate a constant value. For example, if the distance X2 is equal to or greater than a boundary value of approximately 2 m, the third score SC3(i) may indicate zero. FIG. 8 is a diagram showing an example of the characteristics of the third score SC3(i). It is preferable that the third score SC3(i) rapidly increases as the distance X2 decreases in order to prevent the user U and the pedestrians P from getting too close. For example, a barrier function or an exponential function is preferably used, but any function showing a similar trend may be used.

[0045] The fourth score SC4(i) is calculated so that it increases as the degree of deviation between the predicted route Ru(i) and the ideal route Ru#(i) increases. For example, the score calculation unit 124 squares the distance between each route point of the predicted route Ru(i) and the ideal route Ru#(i) for each step, and calculates the square of the sum as the fourth score SC4(i). FIG. 9 is a diagram for explaining the calculation process of the fourth score SC4(i). In the diagram, r1 is the distance between the route point of the first step of each of the predicted route Ru(i) and the ideal route Ru#(i), r2 is the distance between the route point of the second step of each of the predicted route Ru(i) and the ideal route Ru#(i), and r3 is the distance between the route point of the third step of each of the predicted route Ru(i) and the ideal route Ru#(i). The fourth score SC4(i) can be calculated, for example, by √(r1 ２ +r2 ２ +r3 ２ ) can be calculated as

[0046] The score calculation unit 124 calculates the overall score SC(i) for each route candidate Rmc(i), and then selects the route candidate Rmc(i) with the smallest overall score SC(i) and outputs it to the drive control unit 140 as the route Rm of the moving body 1. The drive control unit 140 controls the drive unit 30 attached to the moving body 1 so that the moving body 1 moves along the route Rm.

[0047] By performing the processing in this manner, the moving body 1 is controlled to simultaneously lead (guide) the user U to the given destination point by heading towards the destination point, and prevent other pedestrians P from getting too close to the user U due to its own presence. For example, in a scene where the pedestrian P is approaching the user U, the total score of a route candidate Rmc(i) in which the moving body 1 gets between the pedestrian P and the user U to restrain the pedestrian P from approaching should be relatively high.

[0048] 10 and 11 are diagrams illustrating an example in which the moving body 1 inhibits the approach of the user U and the pedestrian P. In FIGS. 10 and 11 , the left diagram shows the movement path of the pedestrian P when the coefficient β=0 and the route Rm that only takes into consideration heading to the destination are selected, and the right diagram shows the movement path of the pedestrian P when the coefficient β is set to a value sufficiently greater than 0. In the example on the right side of FIG. 10 , the moving body 1 takes the route Rm that bulges to the right, thereby forcing the pedestrian P to take a route that is closer to the left (due to the influence F1), thereby inhibiting the pedestrian P from approaching the user U. In the example on the right side of FIG. 11 , the moving body 1 deliberately moves at a slow speed, thereby forcing the pedestrian P to take a route that is closer to the right (due to the influence F1), thereby inhibiting the pedestrian P from approaching the user U. In this way, the control device 100 can generate a route for the moving body 1 so that the moving body 1 behaves appropriately toward the user U.

[0049] Each of the coefficients α, β, ζ, and η may be a fixed value or may be arbitrarily adjusted.

[0050] In the above description, the control device 100 is described as being mounted on the mobile body 1, but this is not limited to this. The control device 100 may be installed at a location away from the mobile body 1, acquire output data from the object detection device 20 via communication, and transmit a drive instruction signal to the drive unit 30, i.e., remotely control the mobile body 1.

[0051] [Filter Creation] FIG. 12 is a configuration diagram of a filter creation device 200. The filter creation device 200 includes, for example, a feature acquisition unit 210, a reference path acquisition unit 220, an individual path acquisition unit 230, an extraction unit 240, a learning unit 250, a simulator 270, and a storage unit 300. The components other than the storage unit 300 are realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI, ASIC, FPGA, or GPU, or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device such as an HDD or flash memory (a storage device with a non-transitory storage medium), or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed in the storage device by inserting the storage medium into a drive device.

[0052] The storage unit 300 is a RAM (Random Access Memory), a flash memory, a HDD, or the like. The storage unit 300 stores information such as sample data 310 and model setting information 320. The sample data 310 is, for example, time-series data such as the relative position, speed, and acceleration of a pedestrian as viewed from the moving body 1, and the position of a user U as viewed from the moving body 1. Alternatively, the sample data 310 may be time-series data in an absolute coordinate system. The sample data 310 may be data acquired when the moving body 1 moves in a real environment, or may be virtual data created based on probabilistic calculations. In the following description, as with the description of the filter processing unit 115, the pedestrian P included in the sample data 310 will be referred to as a pedestrian candidate Pc. The model setting information 320 is information that specifies the type of machine learning model that serves as the basis for the trained model that functions as a filter program, the number of layers, node connection information, and the like.

[0053] The feature acquisition unit 210 derives multiple feature amounts from the sample data. The multiple feature amounts are, for example, as follows. Some of the feature amounts may be derived by having the simulator 270 perform a simulation. FIG. 13 is a diagram schematically illustrating each of the feature amounts listed below. Due to space limitations, the pedestrian candidates Pc are divided into groups according to the feature amounts. (1) The average value of the minimum distance Dmin between the pedestrian candidate Pc and k other pedestrian candidates Pc (two in the figure) in order of their positions closest to the pedestrian candidate Pc during a predetermined observation period. (2) The speed V Pc of the pedestrian candidate Pc. (3) The distance D1 from the line connecting the user U and the moving body 1 to the pedestrian candidate Pc. (4) The length of time T1 that the pedestrian candidate Pc is in the monitoring area A1. (5) The direction (orientation) θ of the pedestrian candidate Pc as seen from the moving body 1. (6) The direction (movement vector or body orientation) φ of the pedestrian candidate Pc in the coordinate system of the moving body 1 (for example, a coordinate system with the central axis of the base 5 as one of the coordinate axes, represented by the X and Y axes in the figure). (7) The vibration amount V iv obtained by evaluating the future trajectory of the pedestrian candidate Pc according to a predetermined standard.

[0054] The reference path acquisition unit 220 acquires a reference path, which is a path of the moving object 1 generated by the function of the control device 1 based on the presence of multiple pedestrian candidates Pc. For example, the reference path acquisition unit 220 acquires the reference path by providing sample data 310 to the simulator 270. The simulator 270 is a device (e.g., a virtual machine or a physical processor) that has a path generation function equivalent to that of the control device 100 described above.

[0055] The individual path acquisition unit 230 selects pedestrian candidates Pc one by one from the sample data 310, and acquires, for each pedestrian candidate Pc, an individual path, which is the path of the moving object 1 generated by the function of the control device 1, based on the existence of remaining pedestrian candidates Pc excluding the selected pedestrian candidates Pc. For example, the reference path acquisition unit 220 acquires an individual path by providing the simulator 270 with data, from the sample data 310, excluding the pedestrian candidates Pc one by one.

[0056] The extraction unit 240 extracts individual routes whose difference from the reference route is smaller than that of other individual routes. For example, the extraction unit 240 extracts (N-M) pedestrian candidates Pc from the number N of pedestrian candidates Pc in order of decreasing difference from the reference route compared to other individual routes, so that M pedestrian candidates Pc remain. The "difference from the reference route" is, for example, a value obtained by calculating the root mean square deviation (RMSD) of the difference in position at each step when the reference route and an individual route are divided into a finite number of steps. FIG. 14 is a diagram for explaining the difference between the reference route and an individual route. In the diagram, RP is the reference route, IP is an individual route, and ΔY is the difference in position at several steps.

[0057] 15 is a flowchart showing an example of the processing flow of the above-described filter creation device 200. First, the feature acquisition unit 210 derives a plurality of feature amounts for each of a plurality of pedestrian candidates Pc from the sample data 310 (S20). Next, the reference path acquisition unit 220 generates a reference path using the simulator 270 (S21).

[0058] Next, the individual route acquisition unit 230 and the extraction unit 240 execute the processes of S22 and S23 for each pedestrian candidate Pc(u) included in the sample data 310. The parameter u is an identifier of the pedestrian candidate Pc, and u = 1 to N. First, the individual route acquisition unit 230 uses the simulator 270 to generate an individual route IP(u) based on the existence of the remaining pedestrian candidates Pc excluding the selected pedestrian candidate Pc(u) (S22). Next, the extraction unit 240 calculates the difference ΔP(u) between the individual route IP(u) and the reference route RP (S23).

[0059] Then, the extraction unit 240 extracts N−M pedestrian candidates Pc in ascending order of the difference ΔP, and sets them as pedestrian candidates to be excluded (S24).

[0060] The learning unit 250 generates a trained model that functions as a filter through machine learning. The learning unit 250 uses the feature amounts of the pedestrian candidate Pc (hereinafter referred to as pedestrian candidate Pc(p)) extracted by the extraction unit 240 as positive example training data, and the feature amounts of the pedestrian candidate Pc (hereinafter referred to as pedestrian candidate Pc(n)) that was not extracted as negative example training data. The learning unit 250 then generates the trained model through machine learning so that, when the feature amounts of the pedestrian candidate Pc(p) are input, information indicating that the pedestrian is a pedestrian to be excluded (for example, the probability that the pedestrian should be excluded or a flag indicating that the pedestrian is a pedestrian to be excluded) is output, and, when the feature amounts of the pedestrian candidate Pc(n) are input, information indicating that the pedestrian is not a pedestrian to be excluded is output.

[0061] The trained model is, for example, a random forest. FIG. 16 is a diagram schematically illustrating an example of the contents of the trained model. Alternatively, the trained model may be in the form of a deep neural network (DNN). Furthermore, the trained model is not limited to a model that, when the feature amounts of a single pedestrian candidate Pc are input, outputs information indicating that the pedestrian candidate is a pedestrian to be excluded. Alternatively, the trained model may, when the feature amounts of multiple pedestrian candidates Pc are input, output information indicating which pedestrian candidates Pc are pedestrians to be excluded. In the latter case, the trained model may be created so that the criteria (e.g., thresholds) for determining whether each pedestrian candidate Pc is to be excluded vary depending on the number of pedestrian candidates Pc whose feature amounts are input.

[0062] [Narrowing Down Feature Amounts] In the above description, the feature amounts are set as fixed amounts. However, the feature amounts may be narrowed down from a larger number of feature amounts prepared in advance. For example, in addition to the feature amounts described above, (8) the distance between the pedestrian candidate Pc and the moving body 1, (9) the distance between the pedestrian candidate Pc and the user U, (10) the distance between the pedestrian candidate Pc and the destination, (11) the angle between the orientation of the pedestrian candidate Pc and the line connecting the user U and the moving body 1, (12) the amount of vibration in the orientation of the pedestrian candidate Pc, and (13) the distance between the future position of the pedestrian candidate Pc and the moving body 1 may be prepared in advance. In this case, the filter creation device 200 may create trained models by excluding feature amounts one by one, similar to narrowing down the pedestrian candidates Pc, and sequentially exclude feature amounts corresponding to trained models in which the pedestrian candidate Pc to be excluded is not significantly different from the other trained models.

[0063] According to the embodiment described above, it is possible to assist the control device in appropriately narrowing down the number of pedestrians to be monitored.

[0064] The above-described embodiment can be expressed as follows: A filter creation device that creates a filter, wherein the filter is a control device that controls a moving body that autonomously moves in an area where pedestrians walk, the control device recognizing objects including the pedestrians and generating a future route for the moving body based on the recognition result, and the control device is a filter for narrowing down pedestrians to be monitored, the filter comprising: one or more storage media that store computer-readable instructions; and a processor connected to the one or more storage media, wherein the processor executes the computer-readable instructions to: acquire a plurality of feature amounts of a plurality of sample pedestrians; acquire a reference route that is a route of the moving body that is generated by a function of the control device based on the presence of the plurality of pedestrians; acquire individual routes that are routes of the moving body that are generated by a function of the control device based on the presence of a group of pedestrians obtained by excluding pedestrians one by one from the plurality of pedestrians; extract an individual route that has a smaller difference from the reference route than other individual routes; and extract a pedestrian corresponding to the extracted individual route. generating the trained model by machine learning so that, when the extracted features of the pedestrian are input to the trained model, information indicating that the pedestrian is an exclusion target is output.

[0065] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0066] REFERENCE SIGNS LIST 1 Mobile body 20 Object detection device 30 Drive device 100 Control device 110 Recognition unit 115 Filter processing unit 120 Path generation unit 122 Path candidate setting unit 124 Score calculation unit 130 Prediction unit 140 Drive control unit 200 Filter creation device 210 Feature amount acquisition unit 220 Reference path acquisition unit 230 Individual path acquisition unit 240 Extraction unit 250 Learning unit 300 Storage unit 310 Sample data 320 Model setting information

Claims

1. A filter creation method in which a filter creation device creates a filter, the filter being a filter for narrowing down pedestrians to be monitored by a control device that controls a moving body that autonomously moves in an area where pedestrians walk, the control device recognizing objects including the pedestrians and generating a future route for the moving body based on the recognition result, the filter creation method comprising: acquiring a plurality of feature amounts of a plurality of sample pedestrians; acquiring a reference route that is a route of the moving body that is generated by a function of the control device based on the presence of the plurality of pedestrians; acquiring an individual route that is a route of the moving body that is generated by a function of the control device based on the presence of a group of pedestrians obtained by excluding one pedestrian at a time from the plurality of pedestrians; extracting an individual route that has a smaller difference from the reference route compared to other individual routes; extracting a pedestrian corresponding to the extracted individual route; and generating the trained model by machine learning such that when the feature amounts of the extracted pedestrian are input into the trained model, information indicating that the pedestrian is a pedestrian to be excluded is output.

2. The filter creation method according to claim 1, wherein the plurality of feature quantities include an average value of the minimum distance between the pedestrian of interest and a predetermined number of other pedestrians in order of their proximity to the pedestrian of interest during a predetermined observation period.

3. The filter creation method according to claim 1, wherein the plurality of feature amounts includes a speed of the pedestrian.

4. The filter creation method according to claim 1, wherein the control device generates a route along which the moving body moves while leading a leading subject, and the plurality of feature amounts include a distance from a straight line connecting the moving body and the leading subject to the pedestrian.

5. The filter creation method according to claim 1, wherein the plurality of feature amounts includes a length of time that the pedestrian is in the monitored area.

6. The filter creation method according to claim 1, wherein the plurality of feature amounts include a direction of the pedestrian as viewed from the moving body.

7. The filter creation method according to claim 1, wherein the plurality of feature amounts include an orientation of the pedestrian in a coordinate system based on the moving body.

8. A filter creation device that creates a filter, the filter being a control device that controls a moving body that autonomously moves in an area where pedestrians walk, the control device recognizing objects including the pedestrians and generating a future route for the moving body based on the recognition result, the control device being a filter for narrowing down pedestrians to be monitored, the filter creation device comprising: a feature acquisition unit that acquires multiple feature amounts of multiple sample pedestrians; a reference path acquisition unit that acquires a reference path that is a route of the moving body generated by a function of the control device based on the presence of the multiple pedestrians; an individual path acquisition unit that acquires an individual path that is a route of the moving body generated by a function of the control device based on the presence of a group of pedestrians obtained by excluding one pedestrian at a time from the multiple pedestrians; an extraction unit that extracts an individual path that has a smaller difference from the reference path compared to other individual paths, and extracts the pedestrian corresponding to the extracted individual path; and a learning unit that generates the trained model by machine learning so that when the feature amounts of the extracted pedestrian are input to the trained model, information indicating that the pedestrian is a pedestrian to be excluded is output.

9. A program to be executed by a filter creation device that creates a filter, the filter being a control device that controls a moving body that autonomously moves in an area where pedestrians walk, the control device recognizing objects including the pedestrians and generating a future route for the moving body based on the recognition result, and is a filter for narrowing down pedestrians to be monitored, the program causing the filter creation device to execute the following: acquiring a plurality of feature amounts of a plurality of sample pedestrians; acquiring a reference route which is a route of the moving body generated by a function of the control device based on the presence of the plurality of pedestrians; acquiring an individual route which is a route of the moving body generated by a function of the control device based on the presence of a group of pedestrians obtained by excluding one pedestrian at a time from the plurality of pedestrians; extracting an individual route whose difference from the reference route is smaller than other individual routes; extracting a pedestrian corresponding to the extracted individual route; and generating the trained model by machine learning such that when the feature amounts of the extracted pedestrian are input into the trained model, information indicating that the pedestrian is a pedestrian to be excluded is output.

10. A filter device applied to a control device that controls a moving body that autonomously moves in an area where pedestrians walk, the control device recognizing objects including the pedestrians and generating a future route for the moving body based on the recognition result, wherein a filter creation device performs the following steps to narrow down pedestrians to be monitored by the control device using a filter generated by: acquiring a plurality of feature amounts of a plurality of sample pedestrians; acquiring a reference route which is a route of the moving body generated by a function of the control device based on the presence of the plurality of pedestrians; acquiring an individual route which is a route of the moving body generated by a function of the control device based on the presence of a group of pedestrians obtained by excluding one pedestrian at a time from the plurality of pedestrians; extracting an individual route whose difference from the reference route is smaller than other individual routes; extracting a pedestrian corresponding to the extracted individual route; and generating the trained model by machine learning so that information indicating that the pedestrian is a pedestrian to be excluded is output when the feature amounts of the extracted pedestrian are input to the trained model.

Images