Control system, control method, and storage medium

Abstract

The invention provides a control system, a control method, and a storage medium capable of determining a stop position corresponding to an environment. A control system that controls a moving body that can autonomously move in a region in which pedestrians can move, the control system being provided with: a detection unit that detects objects in the vicinity of the moving body; a recognition unit that recognizes, on the basis of a result of the detection by the detection unit, traffic participants that are included in the object and that are estimated to be interleaved with the moving body, and a stop position to which the moving body can be retracted; and a control unit that stops the moving body at a stop position such that the moving body is included in the field of view of the traffic participant when it is estimated that the traffic participant and the moving body intersect each other.

Description

Technical Field

[0001] The invention relates to a control system, a control method and a storage medium. Background Art

[0002] Conventionally, there is known a robot that guides a user to a desired location or transports goods (for example, Japanese Patent Application Laid-Open No. 2012-111011).

[0003] However, in the above-mentioned system, the stop position of the moving body has not been sufficiently studied.

[0004] The present invention has been made in consideration of such circumstances, and one of its objects is to provide a control system, a control method, and a storage medium capable of determining a stop position according to the environment. Summary of the invention

[0005] The control system, control method, and storage medium according to the present invention employ the following structures.

[0006] (1): One aspect of the present invention relates to a control system for controlling a moving body, wherein the moving body can move autonomously in an area where pedestrians can move, wherein the control system comprises: a detection unit for detecting objects in the vicinity of the moving body; an identification unit for identifying, based on the detection result of the detection unit, traffic participants included in the objects who are presumed to have crossed the moving body, and a stopping position to which the moving body can retreat; and a control unit for stopping the moving body at the stopping position in such a way that the moving body is included in the field of vision of the traffic participants when it is presumed that the traffic participants have crossed the moving body.

[0007] (2): Based on the scheme of (1) above, when there are a plurality of the traffic participants and a plurality of stop position candidates, the control unit determines a stop position candidate among the plurality of stop position candidates in which the moving body is included in the field of vision of each of the plurality of traffic participants as the stop position, and stops the moving body at the stop position.

[0008] (3): Based on the scheme of (1) above, when it is estimated that the traffic participant and the moving body are intersecting, the control unit assumes that the moving body stops at each of a plurality of stop position candidates, i.e., stop position candidates, and determines as the stop position the stop position candidate in which the moving body is located at the center in the horizontal direction of the field of view or a position close to the center.

[0009] (4): Based on the scheme of (1) above, when it is presumed that the traffic participant crosses the moving body, the control unit assumes that the moving body stops at each of a plurality of stop position candidates, and determines as the stop position the stop position candidate corresponding to the field of view of a larger portion of the moving body that is not hidden by the object and includes the moving body.

[0010] (5): Based on the scheme of (1) above, when it is estimated that the traffic participant and the moving body are intersecting and there are multiple traffic participants, the control unit, assuming that the moving body stops at each of the multiple stop position candidates, obtains an index based on the size of the area in the field of view of each of the multiple traffic participants under each of the stop position candidates in which the moving body is not hidden by objects but can be recognized, and the control unit further determines the stop position candidate with the largest index among the multiple indexes obtained after statistical processing of the indexes of each of the multiple traffic participants under each of the stop position candidates as the stop position.

[0011] (6): Based on the scheme of (1) above, when it is presumed that the traffic participant crosses the moving body, the control unit assumes that the moving body stops at each of a plurality of stop position candidates, and determines the stop position based on indicator information that establishes a corresponding relationship with an indicator corresponding to the degree of deviation of the horizontal distance relative to the horizontal center of the field of view, and the horizontal position of the moving body in the field of view that corresponds to each of the stop position candidates and is not hidden by the object.

[0012] (7): Based on the scheme of (6) above, in the index information, the greater the deviation in the horizontal direction relative to the horizontal center of the field of view, the lower the index is set, and the control unit derives the sum of the indicators of the index information corresponding to the horizontal position of the moving body for each of the field of view of multiple traffic participants, and determines the stop position with reference to each of the derived sums.

[0013] (8): Based on any one of the above schemes (1) to (7), in a narrow road where the moving body needs to move in the width direction in order to avoid the traffic participant when the moving body crosses the traffic participant, the control unit stops the moving body at the stop position in such a way that the moving body is included in the field of vision of the traffic participant.

[0014] (9): Based on any one of the above schemes (1) to (7), after the control unit moves the moving body to the stop position or near the stop position, as the traffic participant approaches the moving body, the control unit moves the moving body in the width direction in a direction opposite to the direction in which the traffic participant is present.

[0015] (10): Based on any one of the above (1) to (7), the moving object can move autonomously in an area where vehicles cannot move but pedestrians can move.

[0016] (11): Another embodiment of the present invention relates to a control method, wherein a control system controls a moving body, and the moving body can move autonomously in an area where pedestrians can move. The control method causes a computer of the control system to perform the following processing: based on the detection result of a detection unit that detects objects around the moving body, identifying a traffic participant included in the object who is presumed to have crossed the moving body, and a stopping position to which the moving body can retreat; and when it is presumed that the traffic participant has crossed the moving body, stopping the moving body at the stopping position in a manner such that the moving body is included in the field of vision of the traffic participant.

[0017] (12): Another embodiment of the present invention relates to a storage medium storing a program, wherein a control system controls a moving body that can move autonomously in an area where pedestrians can move, and the program is used to cause a computer of the control system to perform the following processing: based on the detection result of a detection unit that detects objects around the moving body, identifying traffic participants included in the objects that are presumed to have crossed with the moving body, and a stopping position to which the moving body can retreat; and when it is presumed that the traffic participants have crossed with the moving body, stopping the moving body at the stopping position in a manner that allows the moving body to be included in the field of vision of the traffic participants.

[0018] [Effects of the invention]

[0019] According to the schemes (1) to (12), it is possible to determine a stop position according to the environment.

[0020] According to the scheme (9), after the control device moves the moving body to a stop position or near a stop position, as traffic participants approach the moving body, the control device moves the moving body in the width direction in a direction opposite to the direction in which the traffic participants exist, thereby being able to control the moving body to be easily recognizable by traffic participants and not to interfere with traffic participants. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1This is a diagram showing an example of a configuration of a moving object system including a moving object.

[0022] Figure 2 This is a diagram for explaining an example of the use form of a moving object.

[0023] Figure 3 It is a diagram for explaining the induction mode.

[0024] Figure 4 It is a three-dimensional diagram showing a moving object.

[0025] Figure 5 This is a diagram showing an example of a functional structure of a moving object.

[0026] Figure 6 This is a diagram for explaining avoidance control.

[0027] Figure 7 This is a flowchart showing an example of the flow of processing executed by the control device.

[0028] Figure 8 This is a diagram showing an example of the content of index information.

[0029] Fig. 9 This is a diagram for explaining the index (In) for deriving the above-mentioned index.

[0030] Fig.10 This is a diagram for explaining Modification Example 1.

[0031] Fig.11 : is a diagram showing an example of the index information of the modification example 1 including the cumulative index which is a cumulative index.

[0032] Fig.12 This is a diagram for explaining index information according to Modification 3.

[0033] Fig.13 This is a diagram for explaining the processing of Modification Example 3. DETAILED DESCRIPTION

[0034] Hereinafter, the control system, control method and storage medium of the present invention will be described with reference to the accompanying drawings. The control system of the mobile body of the present invention controls the driving device of the mobile body to move the mobile body. The mobile body of the present invention leads the guide object in the area where pedestrians walk, and moves autonomously following the object. The mobile body can move in the following area, which is an area where pedestrians can move even if the vehicle (motor vehicle, motor two-wheeled vehicle, light vehicle) cannot move. The area where pedestrians move is a sidewalk, an open space, the floor of a building, etc., and may also include a lane. In the following description, it is assumed that a person is not riding on the mobile body, but it may also be a person riding on the mobile body. The guide object is, for example, a pedestrian, or a robot or an animal. The mobile body moves slightly in front of the user who is an elderly person while going to a pre-assigned destination, thereby moving in a way that other pedestrians who hinder the user's movement will not be too close to the user (that is, in a way that creates a road for the user). The user is not limited to the elderly, but may also be a person who has a tendency to have difficulty walking, a child, a person shopping in a supermarket, a patient moving in a hospital, a pet walking, etc. In addition, the user does not necessarily need to decide the destination in advance, and the direction of the user's movement can be predicted, and the user can move autonomously in front of the user in accordance with the user's moving speed. It should be noted that such an action may not be performed all the time, but temporarily. For example, it is also possible that the mobile body is traveling side by side with the user or catching up with the user, and when a specified condition (such as the presence of an obstacle, congestion in traffic conditions, etc.) is detected in the direction of the user's travel, the mobile body temporarily leads the user by executing the algorithm of the present invention.

[0035] Figure 1 1 is a diagram showing an example of the structure of a mobile system 1 including a mobile body 100. The mobile system ("control system") 1 includes, for example, one or more terminal devices 2, a management device 10, one or more camera units 20, an information providing device 30, and one or more mobile bodies 100. They communicate, for example, via a network NW. The network NW is, for example, any network such as a LAN, a WAN, or an Internet line. A part of the functional structure included in the information providing device 30 may also be mounted on the mobile body 100, and a part of the functional structure included in the mobile body 100 may also be mounted on the information providing device 30.

[0036] [Terminal device]

[0037] The terminal device 2 is a computer device such as a smartphone or a tablet terminal. The terminal device 2 requests the management device 10 to provide permission to use the mobile object 100 or obtains information indicating that permission to use has been obtained from the management device 10, for example, based on an operation by a user.

[0038] [Management Device]

[0039] The management device 10 grants the user of the terminal device 2 the right to use the mobile body 100 or manages the reservation for use of the mobile body 100 in response to the request of the terminal device 2. For example, the management device 10 generates and manages schedule information that associates the identification information of the pre-registered user with the date and time of the reservation for use of the mobile body 100.

[0040] [Camera Department]

[0041] The imaging unit 20 is a camera that captures the scenery of the target area. The imaging unit 20 provides the information providing device 30 with images captured at predetermined intervals.

[0042] [Information providing device]

[0043] The information providing device 30 provides the location of the mobile body 100, the area where the mobile body 100 moves, and map information of the surrounding area to the mobile body 100. The information providing device 30 may also generate a route to the destination of the mobile body 100 in response to a request from the mobile body 100, and provide the generated route to the mobile body 100. Details of the information providing device 30 will be described later.

[0044] [Mobile body]

[0045] The mobile object 100 is used by the user in the following usage forms. Figure 2 1 is a diagram for explaining an example of the use form of the mobile body 100. The mobile body 100 can move autonomously in an area where pedestrians can move. For example, the mobile body 100 can pass through an area where vehicles cannot pass. The mobile body 100 is configured, for example, at a specified location of a facility or street. When a user wants to use the mobile body 100, he can operate an operation unit (not shown) of the mobile body 100 to start using it, or operate the terminal device 2 to start using the mobile body 100. For example, when a user goes out shopping and has a lot of goods, he starts using the mobile body 100 and puts the goods into the storage unit of the mobile body 100. And the mobile body 100 moves with the user in a manner of autonomously following the user. The user can continue shopping or go to the next destination while storing the goods in the mobile body 100. For example, the mobile body 100 moves while moving with the user on a crosswalk on a sidewalk or a lane. The mobile body 100 can move in areas where pedestrians can pass, such as lanes and sidewalks. For example, the moving object 100 can be used in indoor or outdoor facilities or private land such as shopping malls, airports, parks, and theme parks, and can move in areas where pedestrians can pass.

[0046] The moving object 100 may be capable of autonomously moving in a mode such as a guidance mode or an emergency mode in addition to (or instead of) the following mode in which the moving object 100 follows the user as described above.

[0047] Figure 3 is a diagram for explaining the guidance mode. The guidance mode is a mode for guiding the user to the destination specified by the user. In this mode, the vehicle moves autonomously in front of the user at the user's moving speed to lead the user. Figure 3 As shown, in a shopping mall, when a user is looking for a specified commodity, if the user requests the mobile body 100 to lead the way to the location of the specified commodity, the mobile body 100 leads the user to the location of the commodity. In this way, the user can easily find the specified commodity. It should be noted that when the mobile body 100 is used in a shopping mall, the mobile body 100 or the information providing device 30 maintains information such as the location of the commodity, the location of the store, the location of the facilities in the shopping mall, etc., which establish a corresponding relationship with the map information, as well as the map information of the shopping mall. The map information includes detailed map information, which includes the width of roads and passages, etc.

[0048] The emergency mode refers to a mode in which, when an abnormal situation occurs to the user during movement with the user (e.g., when the user falls), the mobile body 100 autonomously moves to seek help from nearby people or facilities in order to help the user. In addition to following and guiding as described above, the mobile body 100 may also (or instead of) move while keeping a distance from the user.

[0049] Figure 4 : is a perspective view showing the moving body 100. In the following description, the front direction of the moving body 100 is set as the positive x direction, the rear direction of the moving body 100 is set as the negative x direction, the width direction of the moving body 100 and the left direction with reference to the positive x direction are set as the positive y direction, the width direction of the moving body 100 and the right direction with reference to the positive x direction are set as the negative y direction, and the direction perpendicular to the x direction and the y direction and the height direction of the moving body 100 is set as the positive z direction.

[0050] The mobile body 100 includes, for example, a base body 110, a door portion 112 provided on the base body 110, and wheels (a first wheel 120, a second wheel 130, and a third wheel 140) assembled on the base body 110. For example, a user can open the door portion 112 to put goods into a storage portion provided on the base body 110, or take goods out of the storage portion. The first wheel 120 and the second wheel 130 are driving wheels, and the third wheel 140 is an auxiliary wheel (driven wheel). The mobile body 100 may also be movable using a structure other than wheels, such as tracks.

[0051] A cylindrical support body 150 extending in the positive z direction is provided on the surface of the base body 110 in the positive z direction. A camera 180 for photographing the surroundings of the moving body 100 is provided at the end of the support body 150 in the positive z direction. The position where the camera 180 is provided may be any position different from the above.

[0052] The camera 180 is, for example, a camera capable of capturing images of the surroundings of the moving object 100 at a wide angle (eg, 360 degrees). The camera 180 may include a plurality of cameras. The camera 180 may be implemented by combining a plurality of 120-degree cameras or a plurality of 60-degree cameras, for example.

[0053] Figure 5 FIG. 1 is a diagram showing an example of a functional configuration of the mobile object 100. The mobile object 100 includes Figure 4 In addition to the functional structure shown, the first motor 122, the second motor 132, the battery 134, the brake device 136, the steering device 138, the communication unit 190 and the control device 200 are provided. The first motor 122 and the second motor 132 are operated by the power supplied to the battery 134. The first motor 122 drives the first wheel 120, and the second motor 132 drives the second wheel 130. Alternatively, the first motor 122 is an in-wheel motor provided in the wheel portion of the first wheel 120, and the second motor 132 is an in-wheel motor provided in the wheel portion of the second wheel 130.

[0054] The brake device 136 outputs a braking torque to each wheel based on an instruction from the control device 200. The steering device 138 includes an electric motor. The electric motor, for example, applies a force to a rack-and-pinion mechanism based on an instruction from the control device 200 to change the direction of the first wheel 120 or the second wheel 130, thereby changing the forward path of the moving body 100.

[0055] The communication unit 190 is a communication interface for communicating with the terminal device 2 , the management device 10 , or the information providing device 30 .

[0056] [Control device]

[0057] The control device 200 includes, for example, a position determination unit 202, an information processing unit 204, an identification unit 206, a processing unit 208, a path generation unit 210, a track generation unit 212, a control unit 214, and a storage unit 220. The position determination unit 202, the information processing unit 204, the identification unit 206, the processing unit 208, the path generation unit 210, the track generation unit 212, and the control unit 214 are implemented by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be implemented by hardware (including a circuit unit: circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be implemented by the cooperation of software and hardware. The program may be stored in advance in a storage device such as a HDD (Hard Disk Drive) or a flash memory (a storage device having a non-temporary storage medium), or may be stored in a removable storage medium such as a DVD or a CD-ROM (a non-temporary storage medium), and installed by assembling the storage medium in a drive device. The storage unit 220 is implemented by a storage device such as a HDD, a flash memory, or a RAM (Random Access Memory). The structure in which the processing unit 208 and the control unit 214 are combined is an example of a "control unit".

[0058] The storage unit 220 stores control information 222, a control program for controlling the actions of the mobile body 100, which is referenced by the control unit 214, map information 224, indicator information 226, and field of view information 228. The map information 224 is, for example, map information such as the location of the mobile body 100, the area where the mobile body 100 moves, and the surrounding area provided by the information providing device 30. Details of the indicator information 226 and the field of view information 228 will be described later. Part or all of the functional structure included in the control device 200 may also be included in other devices. For example, other devices may communicate with the mobile body 100 and cooperate to control the mobile body 100.

[0059] The position determination unit 202 determines the position of the mobile body 100. The position determination unit 202 obtains the position information of the mobile body 100 through a GPS (Global Positioning System) device (not shown) built into the mobile body 100. The position information may be, for example, two-dimensional map coordinates or longitude and latitude information. In addition, the position determination unit 202 may estimate the position of the mobile body 100 simultaneously with the creation of the environmental map by using a camera image captured by the camera 180 or a so-called SLAM method using a sensor such as Lidar.

[0060] The information processing unit 204 manages information acquired from, for example, the terminal device 2 , the management device 10 , or the information providing device 30 .

[0061] The recognition unit 206 recognizes the position (distance from the mobile body 100 and direction relative to the mobile body 100) and the state such as speed and acceleration of the object in the vicinity of the mobile body 100 based on the image captured by the camera 180, for example. The so-called object includes traffic participants, obstacles existing in facilities and roads, etc. The recognition unit 206 recognizes and tracks the user of the mobile body 100. For example, the recognition unit 206 tracks the user based on the image (for example, the user's face image) captured when the user uses the mobile body 100, the user's face image provided by the terminal device 2 or the management device 10 (or the feature amount obtained from the user's face image). The recognition unit 206 recognizes the gesture made by the user. It should be noted that the mobile body 100 may be provided with a detection unit other than the camera, such as a radar device or LIDAR. The detection unit may also be provided on an object, a passage, etc. different from the mobile body 100. In this case, the recognition unit 206 uses the detection results of the radar device or LIDAR instead of (or in addition to) the image to recognize the state of the vicinity of the mobile body 100.

[0062] When it is estimated that the traffic participant crosses the moving body 100, the processing unit 208 determines the stopping position so that the moving body 100 stops at the stopping position so that the moving body 100 is included in the visual range of the traffic participant. For example, the processing unit 208 determines the stopping position candidate where the traffic participant can easily visually recognize the moving body 100 as the stopping position from among a plurality of stopping position candidates. The processing of the processing unit 208 will be described in detail later.

[0063] The route generation unit 210 generates a route to a destination specified by the user. The destination may also be a location of a product or a location of a facility. In this case, the user specifies a product or a facility, and the mobile body 100 sets the location of the specified product or facility as the destination. The route is a route that can reasonably reach the destination. For example, the distance to the destination, the time to reach the destination, the ease of the route, etc. are scored, and each score and a route having a score above a threshold value obtained by combining the scores are derived.

[0064] The track generation unit 212 generates a track that the moving body 100 should travel in the future, for example, based on the user's gesture, the destination set by the user, the surrounding objects, the user's position, etc. The track generation unit 212 generates a track that allows the moving body 100 to smoothly move to the target location. The track generation unit 212 generates a track corresponding to the movement of the moving body 100, or generates a track for going to the destination while avoiding surrounding objects, for example, based on the correspondence relationship between the gesture and the movement set in advance. The track generation unit 212 generates a track for following the user being tracked or a track for leading the user. The track generation unit 212 generates a track corresponding to the movement obtained based on a preset pattern, for example. The track generation unit 212 generates a plurality of tracks corresponding to the movement of the moving body 100, and obtains the risk of each track. When the total value of the obtained risk and the risk of each track point meet the preset reference (for example, when the total value is less than the threshold value Th1 and the risk of each track point is less than the threshold value Th2), the track that meets the reference is adopted as the track for the moving body 100. For example, the risk tends to be higher as the distance of the obstacle from the track (track point of the track) is smaller, and the risk tends to be lower as the distance of the obstacle from the track is larger.

[0065] The control unit 214 controls the motors (the first motor 122 and the second motor 132 ), the brake device 136 , and the steering device 138 so that the moving body 100 travels along a track that satisfies a preset reference.

[0066] [About evasion control when dodging objects]

[0067] The control device 200 controls the moving body that can move autonomously in an area where pedestrians can move. The control device 200 identifies traffic participants (e.g., pedestrians, other traffic participants) that are estimated to cross the moving body 100 and the stopping position to which the moving body 100 can retreat, included in the objects around the moving body 100 that are set on the moving body 100, and stops the moving body 100 at the stopping position so as to be included in the field of vision of the traffic participants when it is estimated that the traffic participants cross the moving body 100.

[0068] The visual field of a traffic participant refers to the visual field of the traffic participant when the traffic participant arrives at a preset location. The preset location refers to a stop position candidate that serves as a reference in the direction of travel of the moving body 100 or the traffic participant or a position at a predetermined distance relative to the current position of the moving body 100. The stop position candidate that serves as a reference may be, for example, each of the stop position candidates, or may be a specific stop position candidate (for example, the stop position candidate closest to the traffic participant). In the case where the stop position candidate that serves as a reference is each of the stop position candidates, it is determined whether the traffic participant can recognize the moving body 100 in the visual field corresponding to each of the stop position candidates.

[0069] When there are multiple traffic participants and multiple stop position candidates, the control device 200 determines the stop position candidate in which the moving body is included in the field of vision of each of the multiple traffic participants as the stop position, and stops the moving body at the stop position. For example, when the moving body 100 is not included in the field of vision of each of the traffic participants with respect to other stop position candidates, and when the moving body 100 is included in the field of vision of each of the traffic participants with respect to a specified stop position candidate, the control device 200 determines the specified stop position candidate as the stop position. The avoidance control described above will be described below.

[0070] The avoidance control is executed, for example, when the moving body 100 temporarily stops. The avoidance control is executed, for example, in a narrow road. A narrow road is a passage where the moving body 100 needs to move in the width direction or to an area where it can avoid other moving bodies in order to avoid other moving bodies when the moving body 100 crosses other moving bodies. In the narrow road, the avoidance control stops the moving body 100 at the stop position so that the moving body 100 is included in the field of vision of traffic participants.

[0071] The control device 200 may also determine as the stop position a stop position candidate in which the moving body 100 is located at the center or near the center of the horizontal direction of the visual field under the visual field of each of the stop position candidates, assuming that the moving body 100 stops at each of the multiple stop position candidates, as the stop position. For example, when there are multiple traffic participants and multiple stop position candidates, the control device 200 determines as the stop position a stop position candidate in which the moving body 100 is included in the visual field of each of the multiple traffic participants among the multiple stop position candidates, and stops the moving body at the stop position. Alternatively, stop position candidates in which the moving body 100 is included in multiple visual fields in the visual field of each of the multiple traffic participants, stop position candidates in which the moving body 100 is included in more visual fields, etc. may be determined as the stop position.

[0072] Alternatively, when it is estimated that a traffic participant crosses the moving body 100, the control device 200 assumes that the moving body 100 stops at each of the multiple stop position candidates, and determines the stop position candidate corresponding to the field of view in which the moving body 100 is not hidden by the object and includes more parts of the moving body 100 among the stop position candidates as the stop position. For example, the control device 200 may determine the stop position from each of the multiple stop position candidates, i.e., the stop position candidates, in such a manner that the moving body 100 is not hidden by the object in the field of view and includes more parts of the moving body 100, and the part of the moving body 100 that is not hidden by the object is located at the center of the field of view in the horizontal direction. More specifically, the control device 200 may determine the stop position in such a manner that the sum of the indicators described later is larger.

[0073] Figure 6 is a diagram for explaining avoidance control. In reality, the moving body 100 is controlled based on an image captured from the ground in the horizontal direction (XY direction), but for the sake of explanation, the image obtained by observing the ground from above (from the positive Z direction to the negative Z direction) is referred to. Figure 6 In the following examples, the user followed by the mobile body 100, the user guided by the mobile body 100, or the user moving together with the mobile body 100 is omitted.

[0074] Hereinafter, the moving direction of the moving body 100 is referred to as direction d1, the opposite direction to direction d1 is referred to as direction d2, the direction obtained by rotating direction d1 90 degrees to the right in the horizontal direction is referred to as direction d3, and the opposite direction to direction d3 is referred to as direction d4. The moving body 100 moves in direction d1, and pedestrians P1-P3 (traffic participants) in front are moving from direction d1 to direction d2. In order to avoid interfering with pedestrians P1-P3, the moving body 100 needs to enter any of areas AR1 and AR2 and wait for pedestrians P to pass. Area AR1 or area AR2 is an example of a "retreat area".

[0075] The area AR1 is located on the side of the narrow road in the direction d3. The area AR2 is located on the side of the narrow road in the direction d4. The area AR1 and the area AR2 are avoidance areas of a size that the moving object 100 can enter. Figure 6 The arrows of traffic participants P1-P3 are the center directions of the visual ranges of traffic participants P1-P3. For example, the visual range of traffic participant P1 is Figure 6The visual field V of the area AR1 is formed by the wall on the direction d1 side, which is not arranged to extend in the direction d4, but is formed to be inclined relative to the direction d1. Therefore, the traffic participant P1 can visually recognize all or most of the area AR1 in the visual field V. The wall on the direction d1 side, which is arranged to extend in the direction d4, which is formed in the area AR2. Therefore, the traffic participant P1 can visually recognize a part of the area AR2 in the visual field V. In other words, when the moving body 100 exists in the area AR1, the traffic participant P1 can recognize a larger area of ​​the moving body 100 than when the moving body 100 exists in the area AR2. This is because: when the moving body 100 exists in the area AR2, the wall on the direction d1 side, which forms the area AR2, becomes an obstacle to visual recognition, but when the moving body 100 exists in the area AR1, the wall on the direction d1 side, which forms the area AR1, does not become an obstacle to visual recognition.

[0076] At time T, the moving object 100 is located at position P0 in front of the area AR1 and the area AR2 in the direction d2. At this time, the moving object 100 needs to avoid the traffic participants P1-P3 coming toward the moving object 100. The control device 200 searches the area AR as the search area for the area AR1 and the area AR2 to which the moving object 100 can retreat.

[0077] The control device 200 assumes that the moving body 100 moves to each area in the area AR1 and the area AR2 at time T+1. The control device 200 estimates the positions of the traffic participants P1-P3 at time T+1, and estimates the visual field of each of the traffic participants P1-P3 based on the estimated positions. For example, information indicating the relationship between the positions of the traffic participants and the positions of the moving body 100 obtained in advance through experiments, and the visual field corresponding to the relationship is stored in the storage unit 220. The control device 200 refers to the information and an object that becomes an obstacle (such as a wall) to estimate the visual field and the position of the moving body 100 in the visual field.

[0078] exist Figure 6 In the example, when the moving body 100 is located in the area AR1, more parts of the moving body 100 can be recognized from the visual field of each of the traffic participants P1-P3, so the control device 200 moves the moving body 100 to the area AR1. Alternatively, the control device 200 moves the moving body 100 in such a manner that the moving body 100 slightly emerges from the area AR1 and stops, and is located on the center side of the visual field. The position where the moving body 100 is exposed and stopped is, for example, a position that is several tens of centimeters to the center side of the visual field relative to the reference position. The position on the center side is, for example, a position in the area AR1 where the moving body 100 does not interfere with the traffic participants P1-P3 even if the traffic participants P1-P3 are moving.

[0079] [flow chart]

[0080] Figure 7 2 is a flowchart showing an example of the process flow performed by the control device 200. First, the control device 200 determines whether the passage where the mobile body 100 is located is a narrow road (step S100). In the map information 224, a correspondence is established between the position information, the area, and the information on whether it is a narrow road. The control device 200 refers to the map information 224 to determine whether the passage is a narrow road. The control device 200 may also detect the drivable area based on the surrounding objects such as walls, and determine that the drivable area is a narrow road when the width of the detected drivable area is less than a threshold. The threshold is determined based on the width of the mobile body 100. In addition, the control device 200 may detect objects such as surrounding traffic participants and obstacles and detect the drivable area, and may determine that the drivable area is a narrow road when the detected drivable area is narrower than the actual area of ​​the passage. This is because: in the case where it is determined that it is better to temporarily retreat the mobile body 100 in the case where the passage is crowded, it may be determined that the passage is a narrow road for convenience even if it is actually a sufficiently wide passage. The determination may be made, for example, by comparing the actual area of ​​the passage with the detected area of ​​the drivable region, or based on the number of obstacles or other traffic participants per unit area.

[0081] When the passage where the moving body 100 is located is a narrow road, the control device 200 determines whether to stop the moving body 100 (step S102). For example, when the future path of the moving body 100 interferes with or approaches the position of the future pedestrian P, the control device 200 decides to stop the moving body 100. The control device 200 derives the risk of the moving body 100 approaching or interfering with the pedestrian P based on the future path of the moving body 100, the future position of the pedestrian P, and a predetermined algorithm for deriving the risk related to the interference, and stops the moving body 100 when the risk is greater than a threshold value.

[0082] When it is determined that the moving body 100 is to be stopped, the control device 200 detects a stop position candidate (step S104). The control device 200 predicts the future position of the pedestrian P and determines as a stop position candidate a position that does not interfere with the predicted future position of the pedestrian P or is not close to the predicted future position of the pedestrian P to a predetermined degree or more when the moving body 100 is moving.

[0083] Next, the control device 200 calculates an index for each of the plurality of stop position candidates (step S106), and determines the stop position based on the calculated index (step S108). Furthermore, the control device 200 moves the moving body 100 to the stop position (step S110). The details of the processing of step S106 and step S108 will be described later. Thus, the processing of routine 1 of this flowchart ends.

[0084] Thereafter, when a traffic participant passes the moving body 100 and / or the risk of the moving body 100 approaching or interfering with a pedestrian P is less than a threshold value, the control device 200 moves the moving body 100 from the stop position.

[0085] [Calculation of indicators]

[0086] The control device 200 derives an index for each stop position candidate, and derives a comprehensive index obtained by integrating the derived indexes. For example, the stop position candidate with the largest comprehensive index is determined as the stop position. The control device 200 may also input each index into a predetermined function or model to obtain the stop position.

[0087] The control device 200 generates index information 226 . Figure 8 2 is a diagram showing an example of the content of the index information 226. The index information 226 is information that establishes a correspondence between the index for each of the traffic participants P1-P3, the information indicating whether each index is above a threshold value, and the comprehensive index obtained by statistically processing (for example, summing up) the above-mentioned multiple indexes. The index information 226 is generated for each stop position candidate. In the above-mentioned indexes, a weight may also be assigned to each traffic participant. For example, the index may be adjusted in a manner that emphasizes the index with a high risk of interference with the moving body 100. For example, the index with a low risk of interference may be assigned a small weight. The method for deriving the above-mentioned index will be described later.

[0088] When it is estimated that a traffic participant and the moving body 100 are intersecting and there are multiple traffic participants, the control device 200 assumes that the moving body stops at each of the multiple stop position candidates. The control device 200 obtains an index based on the size of the area in the field of view of each of the multiple traffic participants under each stop position candidate where the moving body 100 is not hidden by an object and can be recognized as the moving body 100. As described above, Figure 8As shown in FIG. 1 , the control device 200 assumes that the moving body 100 exists in the area AR1 and determines the position of the moving body 100 in the field of vision of each of the traffic participants P1-P3 at that time. The same is true for the area AR2. The control device 200 further determines the stop position candidate with the largest index among the processed multiple indexes (comprehensive index) obtained by statistically processing the indexes of each of the multiple traffic participants under each stop position candidate as the stop position.

[0089] When it is estimated that a traffic participant crosses the moving body 100, the control device 200 assumes that the moving body stops at each of the plurality of stop position candidates, and determines the stop position based on the visual field information 228 that establishes a corresponding relationship with an index corresponding to the degree of deviation of the horizontal distance relative to the horizontal center of the visual field, and the horizontal position of the moving body 100 in the visual field that is not hidden by the object and corresponds to each of the stop position candidates. In the visual field information 228, the greater the deviation of the horizontal distance relative to the horizontal center of the visual field, the lower the index is set (see the following description). Fig. 9 The control device 200 derives the sum of the indicators of the field of view information 228 corresponding to the horizontal position of the moving body 100 for each of the field of view of a plurality of traffic participants, and determines the stop position with reference to each of the derived sums.

[0090] The control device 200 derives an index related to the relationship between the field of view of the traffic participant and the moving object 100. The more the moving object 100 is present in the center of the field of view, the larger the index is derived as the user U occupies a larger area or portion in the field of view. However, the area or portion of the moving object 100 that is not hidden by the object (the area or portion of the moving object 100 that the traffic participant can visually recognize or identify) is considered in the calculation of the index.

[0091] Fig. 9 This is a diagram for explaining the index (In) for deriving the above-mentioned index. Fig. 9 The graph G is the field of view information 228 in which each area in the field of view has a corresponding relationship with the indicator. Regarding the indicator, the center C of the field of view or near the center is the highest, and it becomes smaller as it deviates from the center. The distribution of the indicator may not be a smooth mountain shape as shown in the figure, but a shape in which the indicator gradually decreases as it deviates from the center C. The distribution of the indicator may be set in a manner that the prescribed area including the center is different from the area at the end, or the same distribution may be set in all areas.

[0092] The control device 200 assumes, for example, that the moving body 100 stops at each of the stop position candidates. In this assumption, the control device 200 determines the area of ​​the moving body 100 that is visible to the traffic participants. Fig. 9 In the example of FIG. 1 , most of the moving object 100 is hidden by the obstacle, and a part of the moving object 100 is included in the center of the visual field so that a part of the moving object 100 can be visually recognized. The index in this case is the sum of the indexes of the width W1 of the area corresponding to the part of the moving object 100.

[0093] Furthermore, the indicator information 226 may be information that establishes a correspondence relationship with the indicator relative to the position in the horizontal direction (or instead of this), and may also be information that establishes a correspondence relationship with the indicator relative to the position in the vertical direction. It is also possible that the positions in the vertical direction are each associated with the same indicator. The control device 200 determines the indicator corresponding to the position in the vertical direction of the moving body 100, and derives the sum of the determined vertical indicators. The control device 200 may also determine the indicator obtained by statistically processing the derived summed indicator and the summed indicator in the horizontal direction as an indicator to be used for judgment. Thus, the control device 200 can determine the stop position by taking the position of the moving body 100 in the field of view into greater consideration.

[0094] The control device 200 sets the stop position candidate where the above-mentioned comprehensive index becomes the maximum as the stop position. More specifically, the control device 200 sets the stop position candidate where the above-mentioned comprehensive index becomes the maximum and the index is not less than the threshold as the stop position. The control device 200 generates a track from the position of the moving body 100 to the stop position, and moves the moving body 100 along the generated track. The track is, for example, a track on which the moving body 100 can stop at the stop position in the direction of the user U. For example, the track is not a track on which the moving body 100 stops at the stop position in a direction different from that of the traffic participant or user and turns at the stop position in the direction of the traffic participant or user, but preferably, a track on which the moving body 100 turns in the direction of the traffic participant or user when stopping at the stop position. Thus, turning at the stop position is avoided, and control such that the moving body 100 moves in the desired direction and stops at the stop position is smoothly achieved.

[0095] As described above, when it is estimated that a traffic participant crosses the moving body 100 , the control device 200 stops the moving body 100 at the stopping position so that the moving body 100 is included in the field of vision of the traffic participant, thereby being able to determine the stopping position according to the environment.

[0096] <Variation 1>

[0097] In the above example, the control device 200 determines the stop position based on the visual field of the traffic participant when the traffic participant arrives at a preset point. In Modification 1, the control device 200 determines the stop position in consideration of the future position of the traffic participant.

[0098] Fig.10 2 is a diagram for explaining Modification 1. The control device 200 predicts that the traffic participants P1-P3 will move in the direction d2 in the future, and predicts the positions of the traffic participants P1-P3 at one or more or more times in the future ( Fig.10 The control device 200 derives the sum of the indices corresponding to the visual field corresponding to each position of the future traffic participants P1-P3. The control device 200 accumulates the indices of the future visual field for each traffic participant.

[0099] Fig.11 1 is a diagram showing an example of index information 226A of variant example 1 including the accumulated index, i.e., the accumulated index. The index information 226A is information that establishes a corresponding relationship between the accumulated index relative to the positions of the traffic participants P1-P3 at each time, information indicating whether each index is above a threshold value, and a comprehensive index obtained by statistically processing (e.g., summing) the above-mentioned multiple indexes. The index information 226A is generated for each stop position candidate. The control device 200 refers to the above-mentioned index information 226A to determine the stop position candidate with the largest comprehensive index and no accumulated index below the threshold value.

[0100] As described above, the control device 200 determines the stop position based on the visual field corresponding to the positions of one or more or a plurality of future road users P1 to P3, thereby being able to determine the stop position corresponding to the environment.

[0101] <Variation 2>

[0102] In the above example, the index corresponding to the visual field is considered, but in Modification 2, indices of other target items are also considered. The corresponding items considered are, for example, (1) the distance between the stop position candidate and the user, (2) the distance between the stop position candidate and the moving object 100, and (3) the visual field.

[0103] Fig.12 This is a diagram for explaining the index information 226B of the modification example 3. The index information 226B is information that establishes a correspondence relationship with the sub-indicators corresponding to each of the above-mentioned (1) to (3), and the comprehensive index obtained by statistically processing the sub-indicators. The index information 226B is generated for each stop position candidate. The sub-indicator of the field of view of the index information 226B is an index obtained based on the field of view of each traffic participant.

[0104] Regarding (1) above, for example, the shorter the distance, the larger the sub-index is derived. This is because the longer the distance, the greater the cost of reaching a position suitable for following when following is resumed after stopping.

[0105] Regarding the above (2), for example, the shorter the distance, the larger the sub-index is derived. This is because the longer the distance, the more difficult it is to reach the candidate stop position.

[0106] The control device 200 determines the stop position using one or more of the sub-indicators (1) to (3) above. Thus, the moving body 100 can stop at an appropriate position according to the surrounding conditions.

[0107] In addition to the sub-indicators (1) to (3) described above, (4) the cost associated with the track may be considered, or the cost associated with the track may be considered instead of some of the sub-indicators (1) to (3) described above. (4) The control device 200 derives a sub-indicator associated with the track for stopping at the candidate stop position. When the difficulty of the moving body 100 moving along the track is high, the sub-indicator is derived to be small. For example, the sub-indicator of a track that curves at an acute angle or turns in a sporadic manner is small.

[0108] As described above, the control device 200 determines the stop position based on the above-mentioned sub-indicators, thereby being able to determine the stop position that is more appropriate to the environment.

[0109] <Variation 3>

[0110] In the above example, the behavior of the moving body 100 after the moving body 100 moves to the stop position is not considered. In the third variation, after the control device 200 moves the moving body 100 to the stop position or near the stop position, as the traffic participant approaches the moving body 100, the control device 200 moves the moving body 100 in the width direction in the direction opposite to the direction in which the traffic participant exists.

[0111] Fig.13 This is a diagram for explaining the processing of Modification Example 3. It is assumed that at time T, the moving body 100 moves to area AR1#, which is a stop position. Traffic participants P1-P3 are moving in the direction d2. At time T+1, when the traffic participants P1-P3 are closer to the moving body 100 than at time T, the control device 200 moves the moving body 100 in the direction d3. At time T+2, when the traffic participants P1-P3 are closer to the moving body 100 than at time T+1, the control device 200 moves the moving body 100 further in the direction d3. The position of the moving body 100 at time T+3 is a position that does not interfere with the traffic participants P1-P3 when the traffic participants P1-P3 pass through the moving body 100.

[0112] As described above, the moving body 100 is located at a position where the traffic participants P1-P3 can recognize most of the moving body 100 when the traffic participants P1-P3 are far away, and moves in a manner that the moving body 100 moves away from the traffic participants P1-P3 in small steps in the width direction as the traffic participants P1-P3 approach the moving body. As a result, the traffic participants P1-P3 can recognize the moving body 100 and can pass the moving body 100 smoothly.

[0113] The above-described embodiment can be expressed as follows.

[0114] A control system controls a moving body, the moving body being able to move autonomously in an area where pedestrians can move, wherein:

[0115] The control system comprises:

[0116] a storage medium that stores computer-readable instructions; and

[0117] One or more processors connected to the storage medium,

[0118] The one or more processors execute the following processing by executing the instructions that can be read by the computer:

[0119] identifying, based on a detection result of a detection unit that detects an object around the moving body, a traffic participant estimated to have crossed the moving body and included in the object, and a stop position to which the moving body can retreat; and

[0120] When it is estimated that the road participant has crossed the moving object, the moving object is stopped at a stop position so that the moving object is included in the field of vision of the road participant.

[0121] Although specific embodiments of the present invention have been described above using the embodiments, the present invention is not limited to such embodiments at all, and various modifications and substitutions can be made without departing from the gist of the present invention.

Claims

1. A control system for controlling a moving body, wherein the moving body can move autonomously in an area where pedestrians can move, wherein: The control system comprises: a detection unit configured to detect an object around the moving body; a recognition unit that recognizes, based on the detection result of the detection unit, a traffic participant included in the object that is estimated to have crossed the moving body, and a stop position to which the moving body can retreat; as well as The control unit stops the moving body at a stop position so that the moving body is included in a field of vision of the moving body, when it is estimated that the moving body crosses the moving body.

2. The control system according to claim 1, wherein: When there are a plurality of the traffic participants and a plurality of stop position candidates, the control unit determines a stop position candidate in which the moving body is included in the field of vision of each of the plurality of traffic participants as a stop position and stops the moving body at the stop position.

3. The control system according to claim 1, wherein: When it is estimated that the traffic participant crosses the moving body, the control unit assumes that the moving body stops at each of a plurality of stop position candidates, i.e., stop position candidates, and determines as the stop position the stop position candidate in which the moving body is located at the center in the horizontal direction of the field of view or a position close to the center.

4. The control system according to claim 1, wherein: When it is estimated that the traffic participant crosses the moving body, the control unit assumes that the moving body stops at each of a plurality of stop position candidates, and determines as the stop position the stop position candidate corresponding to the field of view that includes a larger portion of the moving body that is not hidden by the object.

5. The control system according to claim 1, wherein: When it is estimated that the traffic participant crosses the moving body and there are a plurality of the traffic participants, the control unit obtains, for each of the plurality of traffic participants at each of the plurality of stop position candidates, an index based on the size of an area in which the moving body is not hidden by an object and can be recognized in the field of view of each of the plurality of traffic participants at each of the stop position candidates, assuming that the moving body stops at each of the plurality of stop position candidates. The control unit further determines, as a stop position, a stop position candidate having a maximum index among a plurality of processed indices obtained by statistically processing the respective indices of the plurality of traffic participants under each of the stop position candidates.

6. The control system according to claim 1, wherein: When it is estimated that the traffic participant has crossed the moving body, the control unit assumes that the moving body has stopped at each of a plurality of stop position candidates, and determines the stop position based on indicator information that establishes a corresponding relationship with an indicator corresponding to the degree of deviation of the horizontal distance relative to the horizontal center of the field of view, and the horizontal position of the moving body in the field of view that corresponds to each of the stop position candidates and is not hidden by the object.

7. The control system according to claim 6, wherein: In the index information, the index is set lower as the distance in the horizontal direction deviates from the center of the field of view range in the horizontal direction. The control unit derives a total of indices of the index information corresponding to the horizontal position of the moving body for each of the visual fields of a plurality of traffic participants, and determines the stop position with reference to each of the derived totals.

8. The control system according to any one of claims 1 to 7, wherein: In a narrow road where the moving body needs to be moved in a width direction in order to avoid the traffic participant when the moving body passes by the traffic participant, the control unit stops the moving body at a stop position so that the moving body is included in the field of vision of the traffic participant.

9. The control system according to any one of claims 1 to 7, wherein: After moving the moving body to the stop position or the vicinity of the stop position, the control unit moves the moving body in a direction opposite to the direction in which the traffic participant is present in the width direction as the traffic participant approaches the moving body.

10. The control system according to any one of claims 1 to 7, wherein: The moving object can autonomously move in an area where vehicles cannot move but pedestrians can move.

11. A control method, wherein: The control system controls the moving body, which can move autonomously in the area where pedestrians can move. The control method enables the computer of the control system to perform the following processing: identifying, based on a detection result of a detection unit that detects an object around the moving body, a traffic participant that is estimated to have crossed the moving body and that is included in the object, and a stop position to which the moving body can retreat; as well as When it is estimated that the road participant has crossed the moving object, the moving object is stopped at a stop position so that the moving object is included in the field of vision of the road participant.

12. A storage medium storing a program, wherein: The control system controls the moving body, which can move autonomously in the area where pedestrians can move. The program is used to make the computer of the control system execute the following processing: identifying, based on a detection result of a detection unit that detects an object around the moving body, a traffic participant that is estimated to have crossed the moving body and that is included in the object, and a stop position to which the moving body can retreat; as well as When it is estimated that the road participant crosses the moving object, the moving object is stopped at a stop position so that the moving object is included in the field of vision of the road participant.

Images