Control system, control method, and program

Abstract

A control system for controlling a moving body that can autonomously move through an area in which pedestrians can move and moves so as to follow a target pedestrian, the control system comprising: a detection unit that is provided to the moving body and detects objects around the target pedestrian and the moving body; a recognition unit that recognizes the target pedestrian and the objects on the basis of the results of the detection by the detection unit; and a control unit that, when the moving body temporarily stops on the basis of the objects, causes the moving body to stop at a stop position in which the target pedestrian is included within the detection range of the detection unit so as to enable the recognition unit to recognize the target pedestrian.

Description

Control system, control method, and program

[0001] The present invention relates to a control system, a control method, and a program.

[0002] 2. Description of the Related Art Conventionally, robots that guide users to desired locations or transport luggage are known (see, for example, Patent Document 1).

[0003] JP 2012-111011 A

[0004] However, the above system does not take into consideration the position where the moving object stops.

[0005] The present invention has been made in consideration of these circumstances, and one of its objectives is to provide a control system, control method, and program that can stop a moving object at an appropriate position depending on the surrounding conditions.

[0006] The control system, control method, and program for a moving body according to the present invention employ the following configuration: (1): A control system for a moving body according to one aspect of the present invention controls a moving body that is capable of autonomously moving within an area in which pedestrians can move and that moves by following a target pedestrian, and includes: a detection unit provided on the moving body that detects the target pedestrian and objects in the vicinity of the moving body; a recognition unit that recognizes the target pedestrian and the object based on the detection results of the detection unit; and a control unit that, when the moving body temporarily stops based on the object, stops the moving body at a stopping position where the target pedestrian is included in the detection range of the detection unit so that the recognition unit can recognize the target pedestrian.

[0007] (2): In the above aspect (1), when the control unit assumes that the moving body has stopped at each of the multiple candidate stop positions, the control unit determines the candidate stop position as the stop position, within the detection range of each of the candidate stop positions, where the target pedestrian is located at the horizontal center of the detection range.

[0008] (3): In the aspect (1) above, when the control unit assumes that the moving body has stopped at each of the multiple candidate stop positions, the control unit determines as the stop position the candidate stop position that corresponds to the detection range of each of the candidate stop positions, in which the target pedestrian is not hidden by the object and a larger portion of the target pedestrian is included in the detection range.

[0009] (4): In the aspect (1) above, the control unit assumes that the moving body has stopped at each of the multiple candidate stopping positions, and determines the stopping position based on index information in which an index corresponding to the degree of deviation of the horizontal distance from the horizontal center of the detection range is associated, and the horizontal position of the target pedestrian that is not hidden by the object within the detection range of each of the candidate stopping positions.

[0010] (5): In the aspect (4) above, the index information is set to a lower index the greater the horizontal distance from the horizontal center of the detection range, and the control unit derives the sum of the indexes of the index information according to the horizontal position of the target pedestrian with respect to each of the detection ranges of the multiple stop position candidates, and determines the stop position by referring to each of the derived sums.

[0011] (6): In the above aspect (5), the control unit determines the stop position based on the sum of the indices for each of the detection ranges of the multiple stop position candidates and the distance from each of the multiple stop position candidates to the position of the target pedestrian.

[0012] (7): In any of the aspects of (5) above, the control unit determines the stop position based on the sum of the indices for each of the detection ranges of the multiple stop position candidates and the distance from each of the multiple stop position candidates to the position of the moving body.

[0013] (8): In any of the aspects of (5) above, the control unit determines the stop position based on the sum of the indices for each of the detection ranges of the multiple stop position candidates, an index based on the distance from each of the multiple stop position candidates to the position of the target pedestrian, and an index based on the distance from each of the multiple stop position candidates to the position of the moving body.

[0014] (9): In any of the above aspects (1) to (8), the control unit stops the moving body at a stopping position where the target pedestrian is included in the detection range of the detection unit in a narrow road where the moving body needs to move in a width direction to avoid another moving body when passing the other moving body.

[0015] (10) In any of the above aspects (1) to (8), the moving body can move autonomously in an area where vehicles cannot move but pedestrians can move.

[0016] (11): In one aspect of the control method of the present invention, a computer in a control system controls a mobile body that can move autonomously in an area where pedestrians can move and that moves by following a target pedestrian. The computer recognizes the target pedestrian and the object based on the detection results of a detection unit provided on the mobile body that detects the target pedestrian and objects around the mobile body, and when the mobile body stops temporarily based on the object, the computer stops the mobile body at a stopping position where the target pedestrian is included in the detection range of the detection unit so that the target pedestrian can be recognized.

[0017] (12): A program according to one aspect of the present invention causes a computer of a control system that controls a mobile body that can move autonomously within an area where pedestrians can move and that moves by following a target pedestrian to perform the following processes: recognizing the target pedestrian and the object based on the detection results of a detection unit provided on the mobile body that detects the target pedestrian and objects around the mobile body; and stopping the mobile body at a stopping position where the target pedestrian is included in the detection range of the detection unit so that the target pedestrian can be recognized when the mobile body stops temporarily based on the object.

[0018] According to the above aspects (1) to (11), when the control unit temporarily stops the moving body based on an object, the control unit stops the moving body at a stopping position where the target pedestrian is included in the detection range of the detection unit, thereby making it possible to stop the moving body at an appropriate position according to the surrounding conditions.

[0019] According to the above aspects (2) or (3), the control unit can more reliably track the user by determining the stop position so that the user being tracked is included in the detection range.

[0020] According to the above aspects (6) to (8), the control unit determines the stop position candidates by taking into account not only the sum of the indices corresponding to each of the stop position candidates, but also one or both of the distance from each of the stop position candidates to the position of the target pedestrian and the distance from each of the stop position candidates to the position of the moving body, thereby making it possible to determine more appropriate stop positions.

[0021] 1 is a diagram illustrating an example of the configuration of a mobile body system 1 including a mobile body 100. A diagram for explaining an example of a usage mode of the mobile body 100. A diagram for explaining a guidance mode. A perspective view showing the mobile body 100. A diagram for explaining an example of the functional configuration of the mobile body 100. A diagram for explaining avoidance control. A flowchart showing an example of the flow of processing executed by the control device 200. A diagram illustrating an example of the contents of cost information 228. A diagram (1) for explaining the cost related to the relationship between the field of view of the mobile body and the user. A diagram (2) for explaining the cost related to the relationship between the field of view of the mobile body and the user. A diagram (3) for explaining the cost related to the relationship between the field of view of the mobile body and the user. A diagram illustrating an example of index information 226 in which an index is associated with a position in the vertical direction.

[0022] Hereinafter, embodiments of a control system, a control method, and a program according to the present invention will be described with reference to the drawings.

[0023] Hereinafter, with reference to the drawings, embodiments of a mobile body control system, a control method, and a program according to the present invention will be described. The control system of the present invention controls the drive device of the mobile body to move the mobile body. The mobile body in this invention autonomously moves through an area where pedestrians walk, leading a lead subject and following the lead subject. The mobile body can move through areas where pedestrians can move, even if vehicles (cars, motorcycles, and light vehicles) cannot move through. The area where pedestrians move includes sidewalks, public open spaces, floors within buildings, and may also include roadways. In the following description, it is assumed that no human occupants are present on the mobile body, but a human occupancy is also possible. The lead subject is, for example, a pedestrian, but may also be a robot or an animal. For example, while heading toward a predetermined destination, the mobile body moves slightly ahead of an elderly user to prevent other pedestrians who may obstruct the user's movement from getting too close to the user (i.e., operates to clear a path for the user). The user is not limited to elderly people, but may also be a person who tends to have difficulty walking, a child, a person shopping at a supermarket, a patient moving around in a hospital, a pet taking a walk, etc. Note that such an action does not have to be performed constantly, but may be performed temporarily. For example, when a mobile object runs alongside or follows a user and detects a predetermined situation in the user's direction of travel (e.g., the presence of an obstacle or traffic congestion), the mobile object may temporarily lead the user by executing the algorithm of the present invention.

[0024] FIG. 1 is a diagram showing an example of the configuration of a mobile body system 1 including a mobile body 100. The mobile body system ("control system") 1 includes, for example, one or more terminal devices 2, a management device 10, an information providing device 30, and one or more mobile bodies 100. These communicate with each other, for example, via a network NW. The network NW is any network, such as a LAN, a WAN, or an internet line. Some of the functional components included in the information providing device 30 may be installed in the mobile body 100, and some of the functional components included in the mobile body 100 may be installed in the information providing device 30.

[0025] The terminal device 2 is a computer device such as a smartphone, a tablet terminal, etc. For example, the terminal device 2 requests the provision of authorization to use the mobile object 100 from the management device 10 based on a user's operation, and obtains information indicating that use has been permitted.

[0026] [Management Device] In response to a request from the terminal device 2, the management device 10 grants the user of the terminal device 2 the authority to use the mobile object 100 and manages reservations for use of the mobile object 100. The management device 10 generates and manages, for example, schedule information that associates pre-registered user identification information with the date and time of reservations for use of the mobile object 100.

[0027] [Information Providing Device] The information providing device 30 provides the mobile object 100 with the location of the mobile object 100, the area in which the mobile object 100 is moving, and map information about the surrounding area. In response to a request from the mobile object 100, the information providing device 30 may generate a route to the destination of the mobile object 100 and provide the generated route to the mobile object 100.

[0028] [Mobile Object] The mobile object 100 is used by a user in the following manner. FIG. 2 is a diagram illustrating an example of a manner in which the mobile object 100 is used. The mobile object 100 can autonomously move in an area where pedestrians can move. The mobile object 100 can, for example, move through an area where vehicles cannot pass. The mobile object 100 is located, for example, at a predetermined location in a facility or a city. When a user wants to use the mobile object 100, the user can start using the mobile object 100 by operating an operation unit (not shown) of the mobile object 100 or by operating the terminal device 2. For example, when a user goes shopping and has a lot of luggage, the user starts using the mobile object 100 and places the luggage in a storage unit of the mobile object 100. The mobile object 100 then moves together with the user, autonomously following the user. The user can continue shopping or head to their next destination with the luggage stored in the mobile object 100. For example, the mobile object 100 moves together with the user, moving along sidewalks and crosswalks on roadways. The mobile body 100 can move in areas where pedestrians can pass, such as roadways and sidewalks. For example, the mobile body 100 may be used in indoor or outdoor facilities or private land, such as shopping centers, airports, parks, and theme parks, and can move in areas where pedestrians can pass.

[0029] The moving body 100 may be capable of moving autonomously in modes such as a guidance mode or an emergency mode in addition to (or instead of) the following mode in which it follows the user as described above.

[0030] FIG. 3 is a diagram illustrating the guidance mode. The guidance mode is a mode in which the mobile object 100 guides the user to a destination specified by the user, autonomously moving ahead of the user at a speed that matches the user's movement speed to lead the user. As shown in FIG. 3 , when a user is searching for a specific product in a shopping center, the user requests the mobile object 100 to lead the user to the location of the specific product. The mobile object 100 then leads the user to the location of the product. This allows the user to easily find the specific product. When the mobile object 100 is used in a shopping center, the mobile object 100 or the information providing device 30 stores map information, including information on the locations of products, stores, and facilities within the shopping center, as well as map information for the shopping center. This map information includes detailed map information, such as the widths of roads and aisles.

[0031] The emergency mode is a mode in which the mobile object 100 moves autonomously to seek help from nearby people or facilities if something unusual happens to the user (for example, if the user falls) while moving with the user. In addition to (or instead of) following or guiding the user as described above, the mobile object 100 may move while maintaining a proper distance from the user.

[0032] 4 is a perspective view showing the moving body 100. In the following description, the forward direction of the moving body 100 is the positive x direction, the backward direction of the moving body 100 is the negative x direction, the width direction of the moving body 100, which is the left direction based on the positive x direction, is the positive y direction, the right direction based on the positive x direction, is the negative y direction, and the height direction of the moving body 100, which is a direction perpendicular to the x direction and y direction, is the positive z direction.

[0033] The mobile body 100 includes, for example, a base body 110, a door unit 112 provided on the base body 110, and wheels (first wheel 120, second wheel 130, and third wheel 140) attached to the base body 110. For example, a user can open the door unit 112 to put luggage into a storage compartment provided on the base body 110 or take luggage out of the storage compartment. The first wheel 120 and the second wheel 130 are driving wheels, and the third wheel 140 is an auxiliary wheel (driven wheel). The mobile body 100 may be movable using a configuration other than wheels, such as caterpillars.

[0034] A cylindrical support body 150 extending in the positive z direction is provided on the positive z direction surface of the base body 110. A camera 180 that captures images of the surroundings of the moving body 100 is provided on the end of the support body 150 in the positive z direction. The position at which the camera 180 is provided may be any position different from the above.

[0035] The camera 180 is, for example, a camera capable of capturing images of the periphery of the moving body 100 at a wide angle (e.g., 360 degrees). The camera 180 may include multiple cameras. The camera 180 may be realized by combining, for example, multiple 120-degree cameras or multiple 60-degree cameras. The camera 180 is an example of a "detection unit." The imaging direction or imaging range of the camera 180 changes, for example, depending on a change in the orientation of the moving body 100. For example, since the camera 180 alone does not rotate, the imaging range or imaging direction does not change unless the orientation of the moving body 100 changes.

[0036] 5 is a diagram showing an example of the functional configuration of the mobile body 100. In addition to the functional configuration shown in FIG. 4, the mobile body 100 further includes a first motor 122, a second motor 132, a battery 134, a brake device 136, a steering device 138, a communication unit 190, and a control device 200. The first motor 122 and the second motor 132 are operated by power supplied to the battery 134. The first motor 122 drives the first wheel 120, and the second motor 132 drives the second wheel 130. The first motor 122 may be an in-wheel motor provided in the wheel of the first wheel 120, and the second motor 132 may be an in-wheel motor provided in the wheel of the second wheel 130.

[0037] The brake device 136 outputs a brake torque to each wheel based on an instruction from the control device 200. The steering device 138 includes an electric motor. The electric motor applies a force to a rack and pinion mechanism based on an instruction from the control device 200, for example, to change the direction of the first wheel 120 or the second wheel 130, thereby changing the course of the mobile object 100.

[0038] The communication unit 190 is a communication interface for communicating with the terminal device 2 , the management device 10 , or the information providing device 30 .

[0039] [Control Device] The control device 200 includes, for example, a position identification unit 202, an information processing unit 204, a recognition unit 206, a processing unit 208, a path generation unit 210, a trajectory generation unit 212, a control unit 214, and a storage unit 220. The position identification unit 202, the information processing unit 204, the recognition unit 206, the path generation unit 210, the trajectory generation unit 212, and the control unit 214 are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device such as a hard disk drive (HDD) or flash memory (a storage device with a non-transitory storage medium), or may be stored in a removable storage medium (non-transitory storage medium) such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device. The storage unit 220 is realized by a storage device such as a HDD, flash memory, or RAM (Random Access Memory).

[0040] The storage unit 220 stores control information 222, which is a control program referenced by the control unit 214 for controlling the behavior of the mobile object 100, map information 224, index information 226, and cost information 228. The map information 224 is, for example, map information provided by the information providing device 30, such as the location of the mobile object 100, the area in which the mobile object 100 moves, and the surrounding area of ​​the area. Details of the index information 226 and the cost information 228 will be described later. Some or all of the functional configuration included in the control device 200 may be included in another device. For example, the other device and the mobile object 100 may communicate with each other and cooperate to control the mobile object 100.

[0041] The position identification unit 202 identifies the position of the mobile object 100. The position identification unit 202 acquires position information of the mobile object 100 using a GPS (Global Positioning System) device (not shown) built into the mobile object 100. The position information may be, for example, two-dimensional map coordinates or latitude and longitude information. The position identification unit 202 may estimate the position information of the mobile object M by communicating with a communication device located within a predetermined distance via the communication unit 190 using a short-range wireless communication method such as Bluetooth (registered trademark) and acquiring the position information of the communication device. The position identification unit 202 may also estimate the position of the mobile object M based on specific structures (landmarks, signs, signs, etc.) included in camera images captured by the camera 180. The position identification unit 202 may also estimate the position of the mobile object M simultaneously with creating an environmental map using a method such as SLAM using camera images captured by the camera 180 or a sensor such as Lidar.

[0042] The information processing unit 204 manages information acquired from, for example, the terminal device 2, the management device 10, or the information providing device 30.

[0043] The recognition unit 206 recognizes the positions (distance from the mobile body 100 and direction relative to the mobile body 100) and speed, acceleration, and other statuses of objects around the mobile body 100 based on, for example, images captured by the camera 180. Objects include traffic participants and obstacles within facilities or on roads. The recognition unit 206 recognizes and tracks the user of the mobile body 100. For example, the recognition unit 206 tracks the user based on an image (e.g., a user's facial image) captured of a user registered when the user uses the mobile body 100, or a user's facial image (or feature values ​​obtained from the user's facial image) provided by the terminal device 2 or the management device 10. The recognition unit 206 recognizes gestures made by the user. The mobile body 100 may be provided with a "detection unit" different from a camera, such as a radar device or LIDAR. In this case, the recognition unit 206 recognizes the situation around the mobile body M using detection results from the radar device or LIDAR instead of (or in addition to) images.

[0044] When the moving body 100 temporarily stops based on the object, the processing unit 208 determines a stopping position where the user to be followed is included in the detection range of the detection unit so that the user to be followed can be recognized by the recognition unit 206. Details of the processing by the processing unit 208 will be described later.

[0045] The route generation unit 210 generates a route to a destination or stopping position specified by the user. The destination may be the location of a product or a facility. In this case, the user specifies the product or facility, and the mobile object 100 sets the location of the specified product or facility as the destination. The route is a route that can reasonably reach the destination. For example, the distance to the destination, the time required to reach the destination, the ease of travel of the route, etc. are scored, and a route is derived in which each score and the combined score of the scores are equal to or greater than a threshold.

[0046] The trajectory generation unit 212 generates a trajectory along which the mobile object 100 should travel in the future based on, for example, a user's gesture, a destination set by the user, surrounding objects, the user's position, etc. The trajectory generation unit 212 generates a trajectory that allows the mobile object 100 to move smoothly to a target point. The trajectory generation unit 212 generates a trajectory according to the behavior of the mobile object 100 based on, for example, a correspondence between a predetermined gesture and an action, or generates a trajectory for heading toward a destination while avoiding surrounding objects. The trajectory generation unit 212 generates, for example, a trajectory for following a tracked user or a trajectory for leading a user. The trajectory generation unit 212 generates a trajectory according to an action based on, for example, a preset mode. The trajectory generation unit 212 generates multiple trajectories according to the behavior of the mobile object 100, calculates the risk for each trajectory, and if the total value of the calculated risks or the risk at each trajectory point meets a preset criterion (for example, if the total value is equal to or less than a threshold value Th1 and the risk at each trajectory point is equal to or less than a threshold value Th2), adopts the trajectory that meets the criterion as the trajectory for the mobile object 100. For example, the risk tends to be higher the closer the distance between the trajectory (trajectory point on the trajectory) and an obstacle, and lower the closer the distance between the trajectory and an obstacle.

[0047] The control unit 214 controls the motors (first motor 122, second motor 132), the brake device 136, and the steering device 138 so that the moving body 100 travels along a trajectory that satisfies a preset standard.

[0048] [Regarding Avoidance Control When Avoiding an Object] The control device 200 controls the mobile body 100, which is capable of autonomously moving in an area where pedestrians can move and moves by following a target pedestrian. For example, the mobile body 100 is capable of autonomously moving in an area where vehicles cannot move and pedestrians can move. The control device 200 detects the target pedestrian and objects around the mobile body 100, and recognizes the target pedestrian and objects based on the detection results. When the mobile body 100 temporarily stops based on the object, the control device 200 stops the mobile body 100 at a stopping position where the target pedestrian is included in the detection range of the detection unit. The temporary stop above refers to, for example, stopping at a position to avoid the object. This avoidance control will be described below.

[0049] The avoidance control is performed, for example, in a narrow passage. A narrow passage is a passage in which, when the moving body 100 passes another moving body, the moving body 100 must move in the width direction or into an area where the other moving body can be avoided. The avoidance control stops the moving body 100 in this narrow passage at a stopping position where the target pedestrian is included in the detection range of the detection unit.

[0050] When the control device 200 assumes that the moving body 100 has stopped at each of the multiple candidate stopping positions, the control device 200 may determine the candidate stopping position as the stopping position where the target pedestrian (e.g., the pedestrian to be followed) is located at the horizontal center of the detection range of each of the candidate stopping positions.

[0051] When it is assumed that the mobile object 100 has stopped at each of the stop position candidates, which are a plurality of stop position candidates, the control device 200 may determine, as the stop position, a stop position candidate corresponding to a detection range among the detection ranges of the stop position candidates in which the target pedestrian is not hidden by an object and a large portion of the target pedestrian is included in the detection range. For example, the control device 200 may determine, from each of the stop position candidates, which are a plurality of stop position candidates, a stop position such that a large portion of the target pedestrian is not hidden by an object in the detection range and the portion of the target pedestrian that is not hidden by an object is located at the center of the detection range. More specifically, the control device 200 may determine a stop position such that the sum of the index (3) described below is larger.

[0052] FIG. 6 is a diagram for explaining avoidance control. In reality, the moving body 100 performs control based on an image captured horizontally (in the X and Y directions) from the ground. However, for the sake of explanation, the description will be given with reference to FIG. 6 , which shows the ground viewed from above (from the positive Z direction to the negative Z direction). Hereinafter, the traveling direction of the moving body 100 and the user U will be referred to as direction d1, the direction opposite to direction d1 as direction d2, the direction obtained by rotating direction d1 90 degrees to the right of the horizontal direction as direction d3, and the direction opposite to direction d3 as direction d4. The moving body 100 and the user U are traveling in direction d1, and a pedestrian P (another moving body) ahead is traveling from direction d1 toward direction d2. To avoid interference with the pedestrian P, the moving body 100 and the user U need to enter one of areas AR1-AR3 and wait for the pedestrian P to pass by.

[0053] Area AR1 is located closer to direction d1 than areas AR2 and AR3, and closer to the narrow road in direction d4. Area AR2 is located closer to the narrow road in direction d4, and area AR3 is located closer to the narrow road in direction d3. Areas AR1, AR2, and AR3 are avoidance areas large enough for the moving body 100 or user U to enter. The arrow of the moving body 100 in Figure 6 indicates the direction of the center of the detection range (field of view) of the moving body 100.

[0054] At time T, the moving object 100 is located at position P1, just before the area AR2 and the area AR3 in the direction d2. At this time, the user U enters the area AR1 to avoid the pedestrian P. The control device 200 searches for the areas AR2 and AR3 in which the moving object 100 can retreat within the area AR, which is the search area. The control device 200 assumes that the moving object 100 has moved into each of the areas AR2 and AR3 at time T+1, and estimates the position of the user U within the detection range when the moving object 100 is pointed toward the user U. For example, the relationship between the position of the user U and the position of the moving object 100, which has been experimentally determined in advance, and information indicating the position of the user U within the detection range according to this relationship are stored in the storage unit 220. The control device 200 estimates the position of the user U within the detection range by referring to this information.

[0055] 6 , since a larger portion of the user U can be recognized from the area AR3, the control device 200 moves the mobile object 100 to the area AR3. Furthermore, when the mobile object 100 is directed toward the user U, the control device 200 moves the mobile object 100 so that the user U can be recognized closer to the center of the detection range. This process will be described in more detail below.

[0056] [Flowchart] FIG. 7 is a flowchart showing an example of the flow of processing executed by the control device 200. First, the control device 200 determines whether the passage in which the mobile object 100 is present is a narrow passage (step S100). The map information 224 associates information indicating whether the passage is a narrow passage with location information, an area, and the like. The control device 200 determines whether the passage is a narrow passage by referring to the map information 224. The control device 200 may detect a drivable area based on surrounding objects such as walls, and if the width of the detected drivable area is equal to or smaller than a threshold, determine that the drivable area is a narrow passage. The threshold is defined based on the width of the mobile object 100.

[0057] If the passage in which the moving body 100 is present is narrow, the control device 200 determines whether to stop the moving body 100 (step S102). For example, if the user U stops, the control device 200 stops the moving body 100. For example, if the future path of the moving body 100 and the future position of the pedestrian P will interfere with or approach each other, the control device 200 stops the moving body 100. For example, the control device 200 derives the risk of the moving body 100 approaching or interfering with the pedestrian P based on the future path of the moving body 100, the future position of the pedestrian P, and a predetermined algorithm for deriving the risk of interference, and if this risk is equal to or greater than a threshold, the control device 200 stops the moving body 100.

[0058] When the moving body 100 is stopped, the control device 200 detects candidate stopping positions (step S104). The control device 200 predicts the future position of the pedestrian P and determines, as candidate stopping positions, positions that do not interfere with the predicted future position of the pedestrian P or that do not come closer than a predetermined degree to the predicted future position of the pedestrian P.

[0059] Next, the control device 200 calculates the cost for each of the plurality of stop position candidates (step S106) and determines a stop position based on the calculated cost (step S108). The control device 200 then moves the mobile object 100 to the stop position (step S110). Details of the processing of steps S106 and S108 will be described later. This completes the processing of one routine of this flowchart.

[0060] Thereafter, when the user U starts moving and / or the risk of the moving body 100 approaching or interfering with the pedestrian P falls below a threshold, the control device 200 moves the moving body 100 from the stopped position.

[0061] [Calculation of Cost] The control device 200 derives the costs (1) to (3) below for each of the stop position candidates, and combines the derived costs to derive an integrated cost. The control device 200 may derive the integrated cost using a coefficient corresponding to each of the derived costs. For example, the stop position candidate with the smallest integrated score is determined as the stop position. The control device 200 may input each cost into a predetermined function or model to determine the stop position. Some of the costs (1) to (3) may be omitted.

[0062] The control device 200 generates cost information 228. Fig. 8 is a diagram showing an example of the contents of the cost information 228. The cost information 228 is information in which each of the costs (1) to (3) described below is associated with an integrated cost obtained by statistically processing and integrating the costs (1) to (3). In the cost information 228, information on the above-mentioned costs and integrated cost is generated for each stop position candidate.

[0063] (1) The control device 200 derives a cost related to the distance from the stop position candidate to the user U. For example, the longer the distance, the larger the cost that is derived. This is because if the distance is long, the cost of reaching a position suitable for tracking will be large if tracking is resumed after stopping.

[0064] (2) The control device 200 derives a cost related to the distance from the candidate stop position to the current location of the moving body 100. For example, the longer the distance, the greater the cost that is derived. This is because it is difficult to reach the candidate stop position if the distance is long.

[0065] (3) The control device 200 assumes that the mobile object 100 has stopped within each of the detection ranges of the multiple stop position candidates, and determines a stop position based on the index information 226 (graph G in FIG. 8 , described later) in which an index corresponding to the degree of deviation of the horizontal distance from the horizontal center of the detection range is associated with the mobile object 100, and the horizontal position of a target pedestrian who is not hidden by an object within each of the detection ranges of the stop position candidates. The index information 226 is set so that the index is lower the greater the deviation of the horizontal distance from the horizontal center of the detection range. The control device 200 derives the sum of the indexes in the index information 226 corresponding to the horizontal position of the target pedestrian within each of the detection ranges of the multiple stop position candidates, and determines the stop position by referring to each of the derived sums. The sum of the indexes in the index information 226 is the sum of the index values. The control device 200 may determine the stop position based on (3), for example, without using (1) or (2).

[0066] The control device 200 derives a cost related to the relationship between the field of view (detection range, imaging range) of the moving body 100 and the user U. The closer the user U is to the center of the field of view, and the larger the area or portion of the field of view that the user U occupies, the lower the derived cost. However, the area or portion of the user U that is not hidden by an object (the area or portion of the user U that can be recognized by the recognition unit 206) is taken into account in calculating the cost.

[0067] 9-11 are diagrams for explaining the index (In) for deriving the cost (3) above. The larger the index, the smaller the derived cost, and the smaller the index, the larger the derived cost. Graph G in FIG. 9-11 is a graph in which an index is associated with each region in the field of view. The index is highest at or near the center C of the field of view and decreases as it deviates from the center. The distribution of the index does not have to be a smooth mountain shape as shown in the figure, but may be a shape in which the index gradually decreases as it deviates from the center C. The distribution of the index may be set so that a predetermined region including the center is different from the edge regions, or the distribution may be the same in all regions.

[0068] For example, the control device 200 assumes that the moving body 100 is stopped at each of the stop position candidates so as to face the direction of the user U. Under this assumption, the control device 200 identifies the area of ​​the user U that appears in the field of view of the moving body 100. For example, in the example of Fig. 9, the user U is hidden by an obstacle, and only the arm of the user U is included at the edge of the field of view. The index at this time is the sum of the indexes of the width W1 of the area corresponding to the arm of the user U.

[0069] In the example of Fig. 10, user U is hidden by an obstacle, and only the arm of user U is included near the center of the field of view. The index in this case is the sum of the indices for the width W2 of the area corresponding to user U's arm. The index corresponding to Fig. 10 is larger than the index corresponding to Fig. 9. This is because in both Fig. 10 and Fig. 9, only the arm is included in the detection range without being hidden by an object, but in Fig. 10, the arm is not hidden by an object and is captured near the center of the detection range.

[0070] In the example of Figure 11, one arm of user U is hidden by an obstacle, and the area other than user U's arm is included near the center of the field of view. The index in this case is the sum of the indices for the width W3 of the area other than user U's arm. The index corresponding to Figure 11 is larger than the index corresponding to Figure 10. This is because in Figure 11, most of user U is not hidden by an object and is included in the detection range, and furthermore, the most of this is not hidden by an object and is located near the center of the detection range.

[0071] Furthermore, the index information 226 may be information in which an index is associated with a vertical position in addition to (or instead of) information in which an index is associated with a horizontal position. FIG. 12 is a diagram showing an example of index information 226 in which an index is associated with a vertical position. As shown in FIG. 12, the same index is associated with each vertical position. The control device 200 identifies the horizontal position index of the index information 226 corresponding to the vertical position of the user U and derives the sum of the identified vertical position indexes. This sum index is the sum of the indexes for width W4 in FIG. 12. The control device 200 may determine an index to be used for judgment, which is obtained by statistically processing this derived sum index and the horizontal sum index. This allows the control device 200 to determine a stop position that takes the user U's position into greater consideration.

[0072] The control device 200 sets the stop position candidate that minimizes the above-mentioned combined score as the stop position. The control device 200 generates a trajectory from the position of the moving body 100 to the stop position, and moves the moving body 100 along the generated trajectory. This trajectory is, for example, a trajectory that allows the moving body 100 to stop at the stop position while facing the direction of the user U. For example, this trajectory is preferably a trajectory that causes the moving body 100 to face the direction of the user U when stopped at the stop position, rather than a trajectory that causes the moving body 100 to stop at the stop position while facing a different direction from the moving body 100 and then turn at the stop position to face the direction of the user U. This achieves control that avoids turning at the stop position and allows the moving body 100 to smoothly face the desired direction and stop at the stop position.

[0073] Furthermore, the trajectory to the stopping position is a trajectory that allows the user U to be included in the detection range without being hidden by objects even while the moving body 100 is moving, or a trajectory that prevents the user U from being completely hidden by objects or being located outside the detection range for a predetermined period of time or longer. This further reduces the risk of losing sight of the user U.

[0074] As described above, the control device 200 sets a plurality of stop position candidates and determines a stop position based on the sum of the indices corresponding to each of the stop position candidates. The control device 200 further determines a stop position based on one or both of the sum of the indices corresponding to each of the stop position candidates, the distance from each of the stop position candidates to the position of the target pedestrian, and the distance from each of the stop position candidates to the position of the moving body. Specifically, the control device 200 determines a stop position based on an index based on the sum of the indices corresponding to each of the stop position candidates, an index based on the distance from each of the stop position candidates to the position of the target pedestrian, and an index based on the distance from each of the stop position candidates to the position of the moving body 100. This allows the moving body 100 to stop at an appropriate position according to the surrounding conditions.

[0075] In addition to or instead of some of the costs (1) to (3) described above, (4) cost related to the trajectory may be considered. (4) The control device 200 derives the cost related to the trajectory for stopping at the candidate stop position. The cost is derived higher when it is difficult for the moving body 100 to move along the trajectory. For example, the cost of a trajectory that makes sharp turns or small turns is higher. For example, the control device 200 may derive a higher cost the lower the likelihood that the user U (user U in area AR1) is included within the range captured by the camera 180, assuming that the moving body 100 travels along the trajectory. The likelihood that the user U is included within the range captured by the camera 180 may refer to the time during which the user U is included within the range, the percentage of the range, or a combination thereof. Furthermore, the cost of a trajectory that requires the moving body 100 to turn to face the user U at the stop position may be derived higher.

[0076] According to the embodiment described above, when the moving body 100 temporarily stops based on an object, the control device 200 stops the moving body at a stopping position where the target pedestrian is included in the detection range of the detection unit so that the recognition unit 206 can recognize the target pedestrian, thereby stopping the moving body at an appropriate position according to the surrounding conditions.

[0077] The above-described embodiment can be expressed as follows: A mobile body comprising: a storage medium storing computer-readable instructions of a control system for controlling a mobile body that is capable of autonomously moving within an area in which pedestrians can move and that moves in accordance with a target pedestrian; and a processor connected to the storage medium, wherein the processor executes the computer-readable instructions to recognize the target pedestrian and the object based on detection results from a detection unit provided on the mobile body that detects the target pedestrian and objects in the vicinity of the mobile body, and when the mobile body temporarily stops based on the object, the mobile body stops at a stopping position where the target pedestrian is included in the detection range of the detection unit so that the target pedestrian can be recognized.

[0078] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0079] REFERENCE SIGNS LIST 1 Mobile system 2 Terminal device 10 Management device 30 Information providing device 100 Mobile device 200 Control device 204 Information processing unit 206 Recognition unit 208 Processing unit 214 Control unit 220 Storage unit 226 Index information 228 Cost information

Claims

1. A control system for controlling a moving body that is capable of autonomously moving within an area in which pedestrians can move and that moves by following a target pedestrian, comprising: a detection unit provided on the moving body that detects the target pedestrian and objects in the vicinity of the moving body; a recognition unit that recognizes the target pedestrian and the objects based on the detection results of the detection unit; and a control unit that, when the moving body pauses based on the object, stops the moving body at a stopping position where the target pedestrian is included in the detection range of the detection unit so that the recognition unit can recognize the target pedestrian.

2. The control system of claim 1, wherein, when it is assumed that the moving body has stopped at each of the multiple candidate stop positions, the control unit determines as the stop position the candidate stop position in which the target pedestrian is located at the horizontal center of the detection range of each of the candidate stop positions.

3. The control system of claim 1, wherein, when it is assumed that the moving body has stopped at each of the multiple candidate stop positions, the control unit determines as the stop position the candidate stop position that corresponds to the detection range of each of the candidate stop positions in which the target pedestrian is not hidden by the object and a larger portion of the target pedestrian is included in the detection range.

4. The control system of claim 1, wherein the control unit assumes that the moving object has stopped at each of a plurality of candidate stopping positions, and determines the stopping positions based on index information in which an index corresponding to the degree of deviation of the horizontal distance from the horizontal center of the detection range is associated with the horizontal position of the target pedestrian that is not hidden by the object within the detection range of each of the candidate stopping positions.

5. The control system of claim 4, wherein the index information is set to a lower index the greater the horizontal distance from the horizontal center of the detection range, and the control unit derives a sum of indices of the index information corresponding to the horizontal position of the target pedestrian with respect to each of the detection ranges of a plurality of the stop position candidates, and determines the stop position by referring to each of the derived sums.

6. The control system of claim 5, wherein the control unit determines the stopping position based on the sum of the indices for each of the detection ranges of the plurality of stopping position candidates and the distance from each of the plurality of stopping position candidates to the position of the target pedestrian.

7. The control system of claim 5, wherein the control unit determines the stop position based on the sum of the indices for each of the detection ranges of the multiple stop position candidates and the distance from each of the multiple stop position candidates to the position of the moving body.

8. The control system of claim 5, wherein the control unit determines the stopping position based on the sum of the indices for each of the detection ranges of the multiple stopping position candidates, an index based on the distance from each of the multiple stopping position candidates to the position of the target pedestrian, and an index based on the distance from each of the multiple stopping position candidates to the position of the moving body.

9. A control system as claimed in any one of claims 1 to 8, wherein the control unit stops the moving body at a stopping position where the target pedestrian is included in the detection range of the detection unit in a narrow road where the moving body needs to move in a width direction to avoid another moving body when passing the other moving body.

10. The control system according to any one of claims 1 to 8, wherein the moving body is capable of autonomously moving in an area in which vehicles cannot move and pedestrians can move.

11. A control method in which a computer of a control system that controls a moving body that can move autonomously in an area where pedestrians can move and that moves by following a target pedestrian recognizes the target pedestrian and the object based on the detection results of a detection unit provided on the moving body that detects the target pedestrian and objects in the vicinity of the moving body, and when the moving body pauses based on the object, stops the moving body at a stopping position where the target pedestrian is included in the detection range of the detection unit so that the target pedestrian can be recognized.

12. A program for causing a computer of a control system that controls a moving body that is capable of autonomously moving within an area in which pedestrians can move and that moves by following a target pedestrian, to execute the following process: recognizing the target pedestrian and the object based on the detection results of a detection unit provided on the moving body that detects the target pedestrian and objects in the vicinity of the moving body; and, when the moving body pauses based on the object, stopping the moving body at a stopping position that includes the target pedestrian within the detection range of the detection unit so that the target pedestrian can be recognized.

Images