Mobile body control device, mobile body control method, program, and mobile body control system

Abstract

This mobile body control device according to an embodiment of the present invention is a control device for a mobile body, and comprises: a recognition unit that recognizes the surrounding situation of the mobile body on the basis of a detection result on the surrounding situation of the moving body; and a determination unit that determines, on the basis of the recognized surrounding situation, a target relative position of the mobile body with respect to a user to be either a first target relative position or a second target relative position.

Description

Mobile body control device, mobile body control method, program, and mobile body control system

[0001] The present invention relates to a mobile body control device, a mobile body control method, a program, and a mobile body control system.

[0002] Conventionally, robots that guide users to desired locations or transport luggage are known (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2012-111011

[0004] Conventional robots move with the user while maintaining a preset fixed relative positional relationship, such as in front of or behind the user. If the relative position of the robot with respect to the user can be changed according to the surrounding situation during movement, further effective utilization of the robot is expected.

[0005] The present invention has been made in consideration of such circumstances, and one of its objectives is to provide a mobile body control device, a mobile body control method, and a program that can appropriately control the relative position of the mobile body with respect to the user according to the surrounding situation.

[0006] The mobile body control device, mobile body control method, program, and mobile body control system according to this invention employ the following configurations. (1): The mobile body control device according to one aspect of the present invention includes a recognition unit that recognizes the surrounding situation of the mobile body based on the detection result of the surrounding situation of the mobile body, and a determination unit that determines the target relative position of the mobile body with respect to the user as either a first target relative position or a second target relative position based on the recognized surrounding situation.

[0007] (2): In the aspect of (1) above, the first target relative position is the lateral position of the user, and the second target relative position is the front position or the rear position of the user.

[0008] (3) In the embodiment of (1) or (2) above, the recognition unit recognizes, as the surrounding conditions, traffic participants in front of or behind the user and the moving body, and the determination unit determines the target relative position based on the recognized traffic participants.

[0009] (4) In the embodiment of (3) above, the determination unit determines the target relative position based on the number of traffic participants.

[0010] (5) In the embodiment of (3) above, the determination unit determines the target relative position based on the population density of the traffic participants.

[0011] (6) In the embodiment of (3) above, the determination unit determines the target relative position based on at least one of the relative position and relative speed of the traffic participant with respect to the user.

[0012] (7) In the embodiment of (6) above, the determination unit determines the target relative position based on a risk value that indicates the risk of the traffic participant colliding with the user, which is calculated based on the relative position and the relative speed.

[0013] (8) In the embodiment of (3) above, the determination unit determines the target relative position based on whether or not a flow of people by traffic participants is occurring in the direction of movement of the moving body.

[0014] (9) In the embodiment of (1) or (2) above, the recognition unit recognizes the movable area in the direction of movement of the user and the moving body as the surrounding conditions, and the determination unit determines the target relative position based on the result of comparing the width of the recognized movable area with a width threshold.

[0015] (10): In the embodiment of (2) above, the recognition unit recognizes the presence or absence of a curved road in the direction of movement of the user and the moving body as the surrounding conditions, and the determination unit determines the target relative position to the second target relative position when a curved road is recognized.

[0016] (11): In the embodiment of (2) above, the first target relative position is selectable by the user to be to the left or to the right of the user.

[0017] (12): In the embodiment of (2) above, the first target relative position is the one with a larger drivable area, whichever is to the left or right of the user.

[0018] (13): In the embodiment of (1) or (2) above, the system further comprises a setting unit that changes the range of the area where the moving body is prohibited from entering around the user, based on the determined relative position of the target.

[0019] (14): In the embodiment of (13) above, when the target relative position is determined to be the first target relative position, the setting unit sets the no-entry area of ​​a predetermined width in front of or behind the user.

[0020] (15): In the embodiment of (13) above, the setting unit sets the no-entry area such that its width narrows toward the front or rear of the user when the target relative position is determined to be the second target relative position.

[0021] (16): In the embodiment of (1) or (2) above, the detection result includes at least one of an image captured by a camera, a detected value detected by a radar device, and a detected value detected by a LIDAR.

[0022] (17): A method for controlling a mobile body according to another aspect of the present invention, wherein the computer of the control device for the mobile body recognizes the surrounding conditions of the mobile body based on the detection results of the surrounding conditions of the mobile body, and determines the target relative position of the mobile body to the user to be either a first target relative position or a second target relative position based on the recognized surrounding conditions.

[0023] (18): A program according to another aspect of the present invention causes the computer of a control device for a mobile body to recognize the surrounding conditions of the mobile body based on the detection results of the surrounding conditions of the mobile body, and to determine the target relative position of the mobile body to the user as either a first target relative position or a second target relative position based on the recognized surrounding conditions.

[0024] (19): A control system for a moving body according to another aspect of the present invention comprises: a recognition unit that recognizes the surrounding conditions of the moving body based on the detection results of the surrounding conditions of the moving body; and a determination unit that determines the target relative position of the moving body to the user to be either a first target relative position or a second target relative position based on the recognized surrounding conditions.

[0025] According to embodiments (1) to (19), the relative position of the moving object to the user can be appropriately controlled according to the surrounding conditions.

[0026] This figure shows an example of the configuration of a mobile system 1 including a mobile body 100 according to an embodiment. This figure illustrates an example of how the mobile body 100 is used according to an embodiment (follow mode). This figure illustrates an example of how the mobile body 100 is used according to an embodiment (guidance mode). This is a perspective view showing an example of the mobile body 100 according to an embodiment. This figure shows an example of the functional configuration of the mobile body 100 according to an embodiment. This figure illustrates the basic rules for controlling the relative position of the mobile body 100 with respect to the user U according to this embodiment. This figure illustrates an example of controlling the relative position of the mobile body 100 according to an embodiment (first movement rule: population density). This figure illustrates an example of controlling the relative position of the mobile body 100 according to an embodiment (second movement rule: risk value). This figure illustrates an example of controlling the relative position of the mobile body 100 according to an embodiment (third movement rule: environmental objects). This figure illustrates another example of controlling the relative position of the mobile body 100 according to an embodiment (third movement rule: environmental objects). This figure illustrates an example of controlling the relative position (left and right direction) of the mobile body 100 according to an embodiment. This figure illustrates an example of controlling the relative position of the mobile body 100 according to an embodiment (fourth movement rule: human flow). This figure illustrates an exception to the control of the relative position of the moving body 100 according to the embodiment (fourth movement rule: pedestrian flow). This figure illustrates an example of the relationship between the movement candidate (movable range) of the moving body 100 according to the embodiment and the user U's no-entry zone (collision radius). This flowchart is an example of the flow of the control process for the relative position of the moving body 100 according to the embodiment.

[0027] Hereinafter, embodiments of the control device for a mobile body, the control method for a mobile body, the program, and the control system of the present invention will be described with reference to the drawings.

[0028] Figure 1 shows an example of the configuration of a mobile system 1 including a mobile body 100. The mobile system 1 includes, for example, one or more terminal devices 2, a management device 10, an information providing device 20, and one or more mobile bodies 100. These communicate, for example, via a network NW. The network NW is any network such as a LAN (Local Area Network), WAN (Wide Area Network), or Internet connection.

[0029] [Terminal device] Terminal device 2 is, for example, a computer device such as a smartphone or a tablet terminal. Terminal device 2, for example, requests the management device 10 to grant permission to use the mobile device 100 based on the operation of a user who wishes to use the mobile device 100, and obtains information from the management device 10 indicating that permission to use has been granted.

[0030] [Management Device] The management device 10 grants users of the terminal device 2 the right to use the mobile device 100 in response to requests from the terminal device 2, and manages reservations for the use of the mobile device 100. For example, the management device 10 generates and manages schedule information that associates pre-registered user identification information with the date and time of reservations for the use of the mobile device 100.

[0031] [Information Provisioning Device] The information provisioning device 20 provides the mobile object 100 with information on the location of the mobile object 100, the area in which the mobile object 100 is moving, and map information of the area surrounding the area. The information provisioning device 20 may also generate a route to the destination of the mobile object 100 in response to a request from the mobile object 100 and provide the generated route to the mobile object 100. Note that each function of the information provisioning device 20 may be incorporated into the management device 10.

[0032] [Mobile Unit] The mobile unit 100 is, for example, a mobile robot that moves using power supplied from a battery. Alternatively, the mobile unit 100 may be a vehicle-type mobile unit or a mobile unit with any other arbitrary structure. The mobile unit 100 is used in various ways, such as a mode in which it moves autonomously following the user (hereinafter referred to as "follow mode"), or a mode in which it moves autonomously in front of the user (leading) to guide the user (hereinafter referred to as "guidance mode" or "leading mode").

[0033] Figure 2 is a diagram illustrating an example of how the mobile device 100 can be used (follow mode). The mobile device 100 is, for example, placed in a predetermined location in a facility or city. When a user wants to use the mobile device 100, they can start using it by operating the control unit (not shown) of the mobile device 100, or by operating the terminal device 2. For example, when a user goes shopping and has a lot of luggage, they can start using the mobile device 100 and put their luggage into the storage compartment of the mobile device 100. The mobile device 100 then moves with the user, autonomously following them. The user can continue shopping or head to their next destination with their luggage stored in the mobile device 100. For example, the mobile device 100 moves with the user, crossing sidewalks and road crosswalks. The mobile device 100 can move in areas accessible to pedestrians, such as roadways and sidewalks. For example, the mobile device 100 may be used in indoor or outdoor facilities such as shopping centers, airports, parks, and theme parks, or on private property.

[0034] Figure 3 is a diagram illustrating another example of how the mobile unit 100 can be used (guidance mode). The guidance mode is, for example, a mode in which the mobile unit guides the user to a destination specified by the user, and in this mode, the mobile unit autonomously moves in front of the user at the user's speed to guide the user. As shown in Figure 3, when a user is looking for a specific product in a shopping center, the user requests the mobile unit 100 to guide them to the location of the product, and the mobile unit 100 guides the user to the location of the product. This allows the user to easily find the specific product. When the mobile unit 100 is used in a shopping center, the mobile unit 100 or the information providing device 20 holds information that associates the locations of products, stores, and facilities within the shopping center with map information, as well as map information of the shopping center. This map information includes detailed map information, including the width of roads and passages. The locations of products, stores, and facilities within the shopping center may also be included in the map information. In this case, the mobile unit 100 or the information providing device 20 does not need to hold map information of the shopping center.

[0035] Furthermore, the guidance mode may be a mode in which the user is guided to a destination estimated based on map information and information such as the user's actions (including direction, speed, behavior, etc.), even if the user does not specify a destination. For example, the mobile body 100 or the information providing device 20 may detect the user's direction from an image captured by the camera 180 (described later), set a straight line representing the detected user's direction, and estimate the destination to be a place that intersects with or is closest to that straight line among the locations registered in the map information. Alternatively, for example, the mobile body 100 or the information providing device 20 may pre-register multiple gestures (for example, a gesture for drinking a beverage or a gesture for charging a mobile phone), compare the user's behavior detected from the image with the registered gestures, and estimate the destination to be a place among the locations stored in the map information that satisfies the requirements of the gesture (for example, a restaurant or a charging facility). Alternatively, for example, the mobile body 100 or the information providing device 20 may estimate the destination to be a place among the facilities stored in the map information that has been set as a destination most frequently by past users.

[0036] In addition to (or instead of) the follow mode and guidance mode described above, the mobile unit 100 may also be capable of autonomous movement in other modes, such as an emergency mode. The emergency mode is a mode in which the mobile unit autonomously moves to seek help from nearby people or facilities in order to assist the user if something happens to the user while moving with the user (for example, if the user falls). Alternatively, as a form of the follow mode as described above, the mobile unit 100 may move while maintaining a distance from the user that is neither too close nor too far. In this case, the mobile unit 100 may, for example, recognize the user's position based on images captured by the camera 180 (for example, in real time) and move in front of (or diagonally in front of), to the side of, or behind (or diagonally behind) the user in order to maintain a distance that is neither too close nor too far from the user.

[0037] Figure 4 is a perspective view showing an example of the structure of the mobile body 100. In the following description, the forward direction of the mobile body 100 is the positive x direction, the backward direction of the mobile body 100 is the negative x direction, the width direction of the mobile body 100 is the positive y direction to the left and the negative y direction to the right, and the height direction of the mobile body 100, which is perpendicular to the x and y directions, is the positive z direction.

[0038] The mobile body 100 comprises, for example, a base 110, a door 112 provided on the base 110, and wheels (first wheel 120, second wheel 130, and third wheel 140) mounted on the base 110. For example, a user can open the door 112 to put luggage into a storage compartment provided on the base 110, or to take luggage out of the storage compartment. The first wheel 120 and the second wheel 130 are drive wheels, and the third wheel 140 is an auxiliary wheel (driven wheel). The mobile body 100 may also be movable using components other than wheels, such as tracks.

[0039] A cylindrical support 150 extending in the positive z direction is provided on the surface of the base 110 in the positive z direction. A camera 180 for imaging the area around the moving body 100 is provided at the end of the support 150 in the positive z direction. The position where the camera 180 is provided may be any position other than that described above.

[0040] Camera 180 is, for example, a camera capable of capturing images of the area around the moving object 100 in a wide angle (for example, 360 degrees). Camera 180 may include multiple cameras. Camera 180 may be implemented by combining, for example, multiple 120-degree cameras or multiple 60-degree cameras.

[0041] FIG. 5 is a diagram showing an example of the functional configuration of the mobile body 100. In addition to the functional configuration shown in FIG. 4, the mobile body 100 further includes, for example, a first motor 122, a second motor 132, a battery 134, a brake device 136, a steering device 138, a communication unit 190, and a control device 200. The first motor 122 and the second motor 132 are operated by electric power supplied from the battery 134. The first motor 122 drives the first wheel 120. The first motor 122 may be an in-wheel motor provided on the wheel of the first wheel 120. The second motor 132 drives the second wheel 130. The second motor 132 may be an in-wheel motor provided on the wheel of the second wheel 130.

[0042] The brake device 136 outputs braking torque to each wheel based on an instruction from the control device 200. The steering device 138 includes an electric motor. The electric motor, for example, acts on a rack and pinion mechanism based on an instruction from the control device 200 to change the direction of the first wheel 120 or the second wheel 130, thereby changing the traveling route of the mobile body 100.

[0043] The communication unit 190 is a communication interface for communicating with the terminal device 2, the management device 10, and the information providing device 20 to transmit and receive various information. The communication unit 190 includes, for example, a network card, a NIC (Network Interface Controller), and the like.

[0044] (Control Device) The control device 200 controls the overall operation of the mobile body 100. The control device 200 includes, for example, a control unit 210 and a storage unit 220. The control unit 210 includes, for example, an acquisition unit 211, a recognition unit 212, a route generation unit 213, a drive control unit 214, a determination unit 215, and a setting unit 216. The control device 200 is an example of a "control device". That is, the control device 200 is a control device for a mobile body that leads a user or follows a user.

[0045] Each functional unit of the control unit 210 is realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit), or by the cooperation of software and hardware. The program may be stored in advance in a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device equipped with a non-transient storage medium), or it may be stored in a removable storage medium such as a DVD or CD-ROM (a non-transient storage medium) and installed when the storage medium is inserted into a drive device.

[0046] The storage unit 220 is implemented by a storage device such as an HDD, flash memory, or RAM (Random Access Memory). The storage unit 220 stores map information 222 that is referenced by the mobile body 100. The map information 222 is, for example, map information provided by the information providing device 20, such as the location where the mobile body 100 is located, the area in which the mobile body 100 moves, and the area surrounding the area.

[0047] The acquisition unit 211 acquires various information from the terminal device 2, the management device 10, and the information providing device 20 via the network NW. The acquisition unit 211 also acquires images of the surrounding environment of the moving object 100 captured by the camera 180.

[0048] The recognition unit 212 recognizes the surrounding situation of the moving body 100 based on the detection result of the surrounding situation of the moving body 100. For example, the recognition unit 212 recognizes the surrounding situation of the moving body 100 based on an image of the surrounding situation of the moving body 100 captured by the camera 180. The recognition unit 212 recognizes, for example, the position of an object in the vicinity (distance from the moving body 100 and direction with respect to the moving body 100), and states such as speed and acceleration. The object includes, for example, traffic participants (persons, vehicles, other moving bodies), and obstacles existing in facilities or on roads. In addition, the recognition unit 212 recognizes and tracks the user of the moving body 100. For example, based on an image of the user registered when the user uses the moving body 100 (for example, the user's face image), or an image of the user provided by the terminal device 2 or the management device 10 (for example, the user's face image, or a feature amount obtained from the user's face image), the recognition unit 212 tracks the user. Alternatively, the recognition unit 212 recognizes gestures made by the user. Note that the moving body 100 may be provided with a detection unit different from a camera, such as a radar device or LIDAR. In this case, instead of (or in addition to) the image, the recognition unit 212 uses the detection results of the radar device or LIDAR to recognize the situation around the moving body 100. The recognition unit 212 is an example of a "recognition unit". The recognition unit 212 recognizes, as the surrounding situation, traffic participants, environmental things (such as the presence or absence of a curved road), a drivable area, etc. in at least one of the front or rear of the user and the moving body 100. The detection result includes at least one of an image captured by a camera, a detection value detected by a radar device, and a detection value detected by LIDAR.

[0049] The route generation unit 213 generates a route to the destination based on the surrounding conditions of the mobile body 100 recognized by the recognition unit 212. The destination, when the mobile body 100 is in follow mode, represents the user itself that is being followed, or a point within a predetermined range from the user. For example, the route generation unit 213 may set the destination as a predetermined point diagonally behind the user so that the mobile body 100 can follow the user and remain visible to the user. Alternatively, for example, the route generation unit 213 may determine the destination to maintain a predetermined distance from the user based on the user's walking speed, in order to prevent the mobile body 100 from getting too far away from the user. In this case, the destination does not necessarily have to be behind the user; for example, the destination may be a predetermined point diagonally in front of the user, and the mobile body may travel in front of the user.

[0050] On the other hand, when the mobile unit 100 is in guidance mode, the destination refers to, for example, the location of a product or facility set by the user. In this case, the user specifies the location of the product or facility, and the mobile unit 100 compares the specified location of the product or facility with the map information 222. As a result of the comparison, the identified location of the product or facility is set as the destination. Furthermore, if the location set by the user is far from the current location of the mobile unit 100, the route generation unit 213 may set the location set by the user as the final destination and set a location within a predetermined range from the current location as a provisional destination.

[0051] The route is one that allows the moving body 100 to reach its destination in a reasonable manner, taking into account the forward direction of the moving body 100 (i.e., the x-direction of the moving body 100). The route generation unit 213 generates a number of waypoints to reach the destination from the current location and generates a route by connecting these waypoints.

[0052] The drive control unit 214 controls the motors (first motor 122, second motor 132), brake device 136, and steering device 138 so that the mobile body 100 travels along the path generated by the path generation unit 213.

[0053] The determination unit 215 determines the target relative position of the moving object 100 relative to the user (hereinafter also simply referred to as "relative position") to be either a first target relative position or a second target relative position, based on the surrounding conditions recognized by the recognition unit 212. The first target relative position is the user's lateral position, and the second target relative position is the user's position in front of or behind the user. Details of the processing of the determination unit 215 will be described later. The determination unit 215 is an example of a "determination unit". The determination unit 215 determines the target relative position based on traffic participants, environmental objects, drivable areas, etc., recognized by the recognition unit 212.

[0054] The setting unit 216 changes the range of the area around the user where the moving object is prohibited from entering, based on the target relative position determined by the determination unit 215. Details of the processing of the setting unit 216 will be described later. The setting unit 216 is an example of a "setting unit".

[0055] [Control of Relative Position] In follow mode, the mobile body 100 basically moves with the user, positioned behind (or diagonally behind) the user. On the other hand, in guidance mode, the mobile body 100 basically moves with the user, positioned in front of (or diagonally in front of) the user. In follow mode, the user may not be able to see the mobile body 100 positioned behind them, and may become anxious about whether the mobile body 100 is following them. On the other hand, in guidance mode, the user may get too close to the mobile body 100 moving in front of them, making it difficult to walk, or even accidentally kicking the mobile body 100. To avoid such situations, in follow mode and guidance mode, it is also possible to set the relative position of the mobile body 100 to the user's lateral position (including diagonally in front and diagonally behind).

[0056] As described above, when the mobile body 100 is positioned to the side of the user and accompanying them, depending on the surrounding conditions of the mobile body 100 and the user (for example, if there are many traffic participants, the road is narrow, etc.), it may be better to move the mobile body 100 in front of or behind the user rather than maintaining its lateral position. In this embodiment, the relative position of the mobile body 100 with respect to the user is controlled according to the surrounding conditions.

[0057] Figure 6 illustrates the basic rules for controlling the relative position of the mobile body 100 with respect to the user U. In Figure 6, the explanation takes the example of a case where the mobile body 100 is positioned laterally to the user U and moves forward toward the target location TP (i.e., in the x-direction of the mobile body 100). As shown in Figure 6(A), if there are no objects (e.g., traffic participants) in the surrounding area (in front of the user U and the mobile body 100), or if there are so few objects that they do not obstruct the user U's walking, the mobile body 100 will move parallel to the user U so as not to obstruct the user U's walking space. In this case, the relative position of the mobile body 100 with respect to the user U will be the lateral position (hereinafter referred to as the "lateral position").

[0058] On the other hand, as shown in Figure 6(B), if there are objects (for example, traffic participants P1 to P4) in the surrounding area (in the direction in front of user U and the mobile body 100) to the extent that they obstruct user U's walking, the mobile body 100 moves from a lateral position to a position in front of user U. At this time, the relative position of the mobile body 100 with respect to user U becomes a forward position (hereinafter referred to as the "forward position"). In this way, by changing the relative position of the mobile body 100 from a lateral position to a forward position, the mobile body 100 acts as a wall for user U, and it is expected that traffic participants P1 to P4 in front will avoid the mobile body 100 (and user U). This prevents collisions between user U and traffic participants P1 to P4 and secures walking space for user U.

[0059] (First movement rule: Population density) The following describes the specific movement rules that change the relative position of the moving object 100 from a lateral position to a forward position. In the first movement rule, the relative position of the moving object 100 is controlled based on the population density of traffic participants located in the forward direction of the moving object 100 and user U.

[0060] Figure 7 illustrates an example of controlling the relative position of the moving object 100 (first movement rule: population density). For example, a determination area AR1 of a predetermined size and shape (e.g., rectangle) is set in the forward direction of the moving object 100 or user U. The determination unit 215 calculates the population density (e.g., population density = number of traffic participants / area of ​​determination area AR1) for traffic participants included in this determination area AR1, and determines the relative position of the moving object 100 based on the comparison result of the calculated population density with a threshold. For example, if the calculated population density is greater than or equal to the threshold, the determination unit 215 changes the relative position of the moving object 100 from a lateral position to a forward position. On the other hand, if the calculated population density is less than the threshold, the determination unit 215 maintains the relative position of the moving object 100 in a lateral position.

[0061] Furthermore, the determination unit 215 may control the relative position of the moving object 100 based simply on the comparison result between the number of traffic participants and a threshold, instead of the population density described above. For example, in the example in Figure 7, the number of traffic participants included in the determination area AR1 is three people, traffic participants P1 to P3. If "3 people" is defined as the threshold, the determination unit 215 determines that the calculated number is greater than or equal to the threshold and changes the relative position of the moving object 100 from a lateral position to a forward position. On the other hand, if the calculated number is less than the threshold, the determination unit 215 maintains the relative position of the moving object 100 in the lateral position.

[0062] In other words, the determination unit 215 determines the target relative position based on the number of traffic participants. The determination unit 215 determines the target relative position based on the population density of traffic participants.

[0063] (Second movement rule: Risk value) In the second movement rule, the relative position of the moving object 100 is controlled based on the risk value caused by traffic participants present in the forward direction of the moving object 100 and user U. The risk value is an index value calculated based on the position (relative position), speed (relative speed), direction of movement, etc., of traffic participants present in the forward direction. The risk value is, for example, an index value indicating the risk of a traffic participant colliding with user U.

[0064] Figure 8 illustrates an example of controlling the relative position of the moving object 100 (second movement rule: risk value). The risk value is calculated for each traffic participant located in the forward direction of the moving object 100 and user U. Figure 8 shows an example where the magnitude of the calculated risk values ​​is in the order of traffic participants P6, P4, P2, P1, P5, P3, and P7. For example, traffic participant P6 has a high risk value because it is relatively close to user U and has a high relative speed. On the other hand, traffic participant P2 is relatively close to traffic participant P6, but has a lower relative speed, so its risk value is calculated to be lower than that of traffic participant P6. The risk value for each traffic participant is calculated at each point in time from the present to a future point in time (for example, the risk value at the present time, the risk value 1 second later, the risk value 2 seconds later, etc.). Alternatively, the risk value for each traffic participant may be calculated only at the present time.

[0065] The determination unit 215 determines the relative position of the moving body 100 based on the risk value calculated as described above. The determination unit 215 controls the relative position of the moving body 100 based on, for example, the comparison result of the maximum risk value calculated for each traffic participant with a threshold, the comparison result of the average risk value for each traffic participant with a threshold, the comparison result of the sum of the risk values ​​for each traffic participant with a threshold, the number of traffic participants with a risk value equal to or greater than the threshold, etc.

[0066] The risk value is represented, for example, by interaction force. Interaction force is an index used in social force models, which are used in pedestrian flow simulations, for example. A social force model is a model that simulates moving towards a goal while avoiding surrounding pedestrians and obstacles, based on the sum of the attractive force to the goal and the repulsive force received from surrounding pedestrians and obstacles, given a set goal. Interaction force represents this repulsive force received from surrounding pedestrians and obstacles. Interaction force is higher the closer the relative position and the faster the relative speed. Interaction force is calculated for each traffic participant at each point in time from the present to a future point in time. The determination unit 215 controls the relative position of the moving body 100 based on the interaction force calculated in this way.

[0067] Alternatively, the risk value may be expressed, for example, by the probability of collision. The probability of collision is an index based on the relative position (distance) between the traffic participant and user U. The determination unit 215 calculates a risk value for each traffic participant based on the relative distance between the traffic participant and user U. The risk value is defined to increase as the relative distance decreases. The determination unit 215 controls the relative position of the moving body 100 based on the comparison result of the collision probability calculated in this way with a threshold.

[0068] In other words, the determination unit 215 determines the target relative position based on at least one of the relative position and relative speed of the traffic participant with respect to the user. The determination unit 215 determines the target relative position based on the risk of the traffic participant colliding with the user, which is calculated based on the relative position and relative speed.

[0069] (Third movement rule: environmental objects) In the third movement rule, the relative position of the moving object 100 is controlled based on environmental objects in the forward direction of the moving object 100 and the user U. Environmental objects include, for example, the structure of the road (narrow roads, corners (curves, bends), width of the road on the left and right), changes in the user's destination, etc.

[0070] Figure 9A illustrates an example of controlling the relative position of the moving body 100 (third movement rule: environmental objects). Figure 9A shows an example where a corner exists as an environmental object in the direction in front of the moving body 100 and the user U. At the corner, there is no spatial margin for the moving body 100 and the user U to move side by side. Also, when two people are walking side by side, it is natural for one of them to move behind the other before reaching a corner so that they can turn the corner in a vertical line. For this reason, if the determination unit 215 detects the presence of a corner in the direction in front of the moving body 100 and the user U, it changes the relative position of the moving body 100 from a lateral position to a forward position.

[0071] Figure 9B illustrates another example of controlling the relative position of the moving body 100 (third movement rule: environmental objects). Figure 9B shows an example where a narrow path exists as an environmental object in the direction in front of the moving body 100 and the user U. In the narrow path, there is no spatial margin for the moving body 100 and the user U to move side by side. Therefore, if the determination unit 215 detects the presence of a narrow path in the direction in front of the moving body 100 and the user U, it changes the relative position of the moving body 100 from a lateral position to a forward position.

[0072] Figure 10 illustrates an example of controlling the relative position (left-right direction) of the mobile body 100. Figure 10 shows a situation where the mobile body 100 and user U are moving side-by-side on a sidewalk sandwiched between boundary B1 and boundary B2. User U is moving at a position shifted to the left (Y direction) from the center of the sidewalk. Here, the width of the space to the right of user U on the sidewalk, W2, is wider than the width of the space to the left, W1. In this case, the mobile body 100 is controlled to move in the wider space on the right. In this way, the determination unit 215 controls the relative position of the mobile body 100 based on the position (movement) of environmental objects and user U. Note that the relative position (left or right) of the mobile body 100 may be specified by user U.

[0073] In other words, the recognition unit 212 recognizes the movable area in the direction of movement of the user U and the mobile body 100 as part of the surrounding environment. The determination unit 215 determines the target relative position based on the comparison result between the width of the recognized movable area and the width threshold. The width threshold corresponds to the minimum width that allows the mobile body 100 and the user U to move side by side. The width threshold is, for example, the sum of the width of the mobile body 100 and the width of a typical user U, plus a predetermined margin. For example, as shown in Figure 9B, if a narrow path exists as an environmental object in the direction in front of the mobile body 100 and the user U, and the width of this narrow path (width of the movable area) is less than or equal to the width threshold, the relative position of the mobile body 100 is changed from a lateral position to a forward position. The first target relative position is the one with the larger drivable area, either to the left or to the right of the user U. Alternatively, the first target relative position can be selected by the user U to be either to their left or to their right.

[0074] (Fourth movement rule: pedestrian flow) In the fourth movement rule, the relative position of the moving body 100 is controlled based on whether or not a pedestrian flow of traffic participants is occurring in the direction in front of the moving body 100 and the user U.

[0075] Figure 11A illustrates an example of controlling the relative position of the moving body 100 (fourth movement rule: pedestrian flow). Figure 11A shows an example where a pedestrian flow by traffic participants P1 to P7 is occurring in the direction in front of the moving body 100. In this case, if the moving body 100 continues to move forward, there is a risk that the moving body 100 will be caught in the pedestrian flow and collide with traffic participants P1 to P7. Also, there is a risk that the movement of user U may be hindered as a result of traffic participants P1 to P7 avoiding the moving body 100. For this reason, if the determination unit 215 detects the presence of a pedestrian flow in the direction in front of the moving body 100, it changes the relative position of the moving body 100 from a lateral position to a forward position.

[0076] Figure 11B illustrates an example (exception) of controlling the relative position of the moving body 100 (fourth movement rule: pedestrian flow). Figure 11B shows an example where pedestrian flow by traffic participants P1 to P6 is occurring in the direction in front of the moving body 100. In this example, considering the direction and / or speed of movement of traffic participants P1 to P6, it is possible to avoid the pedestrian flow by controlling the direction and / or speed of movement of the moving body 100 without changing the relative position of the moving body 100 from a lateral position to a forward position. In this case, if the determination unit 215 recognizes the presence of pedestrian flow in the direction in front of the moving body 100 and determines that the pedestrian flow can be avoided by controlling the moving body 100, it controls the relative position of the moving body 100 to be maintained in a lateral position (so that the relative position is not changed from a lateral position to a forward position).

[0077] In other words, the determination unit 215 determines the target relative position based on whether or not a flow of people due to traffic participants is occurring in the direction of movement of the moving body 100.

[0078] Figure 12 illustrates an example of the relationship between the candidate movement range (movable range) of the mobile body 100 and the user U's restricted area (collision radius). Figure 12(A) shows the relationship between the candidate movement range MC of the mobile body 100 and the user U's restricted area CA1 when the user U and the mobile body 100, which is located laterally to the user U, are stationary. In this case, the restricted area CA1 of the user U is defined as a circular area surrounding the user U. The mobile body 100 moves in such a way that its candidate movement range MC does not enter the restricted area CA1.

[0079] Figure 12(B) shows the relationship between the candidate movement paths of the mobile body 100 and the user U's restricted area CA2 while the user U and the mobile body 100 are moving together from side to side. In this case, the user U's restricted area CA2 is defined as a series of circular areas around the user U along the user U's movement path. The mobile body 100 moves in a way that avoids entering the restricted area CA2 among its candidate movement paths MC.

[0080] Figure 12(C) shows the relationship between the candidate movement MC of the mobile body 100 and the user U's restricted area CA3 when the relative position of the mobile body 100 changes from a lateral position to a forward position while the user U and the mobile body 100 are moving side to side. In this case, the restricted area CA3 for the user U is defined as an egg-shaped region (a region that narrows towards the front) around the user U. The mobile body 100 determines a destination from its candidate movement MCs that does not enter the restricted area CA3 and moves to the forward target position FTP. By making the restricted area CA3 an egg-shaped region, the mobile body 100 can move smoothly in the forward direction of the user U without affecting (causing discomfort to) the user U's walking.

[0081] In other words, when the target relative position is determined to be the first target relative position (lateral position), the setting unit 216 sets an intrusion restriction area of ​​a predetermined width in front of or behind the user U. Also, when the target relative position is determined to be the second target relative position (forward position), the setting unit 216 sets the intrusion restriction area so that its width narrows toward the front of or behind the user U.

[0082] [Processing Flow] Next, the flow of the control process for the relative position of the mobile body 100 will be explained. Figure 13 is a flowchart showing an example of the flow of the control process for the relative position of the mobile body 100. The process shown in Figure 13 is executed repeatedly while the mobile body 100 is accompanying user U.

[0083] First, the acquisition unit 211 acquires a surrounding image of the moving object 100 captured by the camera 180 (step S101).

[0084] Next, the recognition unit 212 recognizes the surrounding conditions of the moving object 100 based on the acquired surrounding image (step S103). The recognition unit 212 recognizes, for example, traffic participants, obstacles, and environmental objects present in the surroundings.

[0085] Next, the determination unit 215 determines the target relative position based on the recognized surrounding conditions and decides whether or not to change the target relative position (step S105). For example, the determination unit 215 determines whether or not to change the target relative position by comparing the recognized surrounding conditions with at least one of the first movement rule (population density), second movement rule (risk value), third movement rule (environmental objects), and fourth movement rule (people flow).

[0086] If it is determined that the target relative position should be changed (step S105; YES), the path generation unit 213 generates a path for the moving body 100 to move from its current location to the changed target relative position (step S107). Here, the path generation unit 213 generates the path taking into account the user U's restricted access area set by the setting unit 216. Next, the drive control unit 214 controls the motor, brake device 136, and steering device 138 so that the moving body 100 travels along the generated path (step S109). As a result, the relative position of the moving body 100 is changed, for example, from a lateral position to a forward position.

[0087] If the relative position of the moving body 100 is changed in step S109 above, or if it is determined that the relative position of the target is not to be changed (step S105; NO), the control unit 210 determines whether the moving body 100 has arrived at the destination (step S111). If it is determined that the moving body 100 has not arrived at the destination (step S111; NO), the process returns to step S101 above and the subsequent processing is repeated. On the other hand, if it is determined that the moving body has arrived at the destination (step S111; YES), the processing of this flowchart ends.

[0088] As described above, according to this embodiment, the target relative position of the moving object with respect to the user is controlled based on the recognized surrounding conditions. This makes it possible to appropriately control the relative position of the moving object with respect to the user according to the surrounding conditions.

[0089] In the above explanation, a configuration was used as an example in which the relative position of the moving body 100 to the target is switched between a lateral position and a forward position. However, the relative position of the moving body 100 to the target may also be configured to switch between a lateral position and a position behind the user (referred to as the "rear position"). In this case, the relative position of the moving body 100 to the target may be controlled based on the surrounding conditions behind the user (in the opposite direction of movement).

[0090] Furthermore, some or all of the functional configurations included in the control device 200 mounted on the mobile unit 100 may be included in other devices. For example, the mobile unit 100 may communicate with and cooperate with another device (server) to control the mobile unit 100. In this case, the combination of the other device (server) and the mobile unit 100 (control device 200) may be configured as a control system.

[0091] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention.

[0092] 1 Mobile System 2 Terminal Device 10 Management Device 20 Information Provision Device 100 Mobile Unit 110 Base 112 Door 120 First Wheel 122 First Motor 130 Second Wheel 132 Second Motor 134 Battery 136 Brake Device 138 Steering Device 140 Third Wheel 150 Support 180 Camera 190 Communication Unit 200 Control Device 210 Control Unit 211 Acquisition Unit 212 Recognition Unit 213 Route Generation Unit 214 Drive Control Unit 215 Decision Unit 216 Setting Unit 220 Storage Unit NW Network

Claims

1. A control device for a mobile body, comprising: a recognition unit that recognizes the surrounding conditions of the mobile body based on the detection results of the surrounding conditions of the mobile body; and a determination unit that determines the target relative position of the mobile body to the user to be either a first target relative position or a second target relative position based on the recognized surrounding conditions.

2. The control device for a moving body according to claim 1, wherein the first target relative position is the user's lateral position, and the second target relative position is the user's forward or backward position.

3. The control device for a mobile body according to claim 1 or 2, wherein the recognition unit recognizes, as the surrounding conditions, traffic participants in front of or behind the user and the mobile body, and the determination unit determines the target relative position based on the recognized traffic participants.

4. The control device for a moving body according to claim 3, wherein the determination unit determines the target relative position based on the number of traffic participants.

5. The control device for a moving body according to claim 3, wherein the determination unit determines the target relative position based on the population density of the traffic participants.

6. The control device for a moving body according to claim 3, wherein the determination unit determines the target relative position based on at least one of the relative position and relative speed of the traffic participant with respect to the user.

7. The control device for a moving body according to claim 6, wherein the determination unit determines the target relative position based on a risk value indicating the risk of the traffic participant colliding with the user, which is calculated based on the relative position and the relative speed.

8. The control device for a moving body according to claim 3, wherein the determination unit determines the target relative position based on whether or not a flow of people by traffic participants is occurring in the direction of movement of the moving body.

9. The control device for a mobile body according to claim 1 or 2, wherein the recognition unit recognizes a movable area in the direction of movement of the user and the mobile body as the surrounding conditions, and the determination unit determines the target relative position based on the result of comparing the width of the recognized movable area with a width threshold.

10. The control device for a mobile body according to claim 2, wherein the recognition unit recognizes the presence or absence of a curved road in the direction of movement of the user and the mobile body as the surrounding conditions, and the determination unit determines the target relative position to the second target relative position when a curved road is recognized.

11. The control device for a moving body according to claim 2, wherein the first target relative position can be selected by the user to be to the left or to the right of the user.

12. The control device for a mobile body according to claim 2, wherein the first target relative position is the one with a larger drivable area, either to the left or to the right of the user.

13. A control device for a mobile body according to claim 1 or 2, further comprising a setting unit that changes the range of the restricted area for the mobile body around the user based on the determined relative position of the target.

14. The control device for a mobile body according to claim 13, wherein the setting unit sets the no-entry area of ​​a predetermined width in front of or behind the user when the target relative position is determined to be the first target relative position.

15. The control device for a mobile body according to claim 13, wherein the setting unit sets the no-entry area so that its width narrows toward the front or rear of the user when the target relative position is determined to be the second target relative position.

16. The control device for a mobile body according to claim 1 or 2, wherein the detection result includes at least one of an image captured by a camera, a detection value detected by a radar device, and a detection value detected by a LIDAR.

17. A method for controlling a mobile body, wherein the computer of the control device of the mobile body recognizes the surrounding conditions of the mobile body based on the detection results of the surrounding conditions of the mobile body, and determines the target relative position of the mobile body to the user to be either a first target relative position or a second target relative position based on the recognized surrounding conditions.

18. A program that causes the computer of a control device for a mobile object to recognize the surrounding conditions of the mobile object based on the detection results of the surrounding conditions of the mobile object, and to determine the target relative position of the mobile object to the user as either a first target relative position or a second target relative position based on the recognized surrounding conditions.

19. A control system for a mobile body, comprising: a recognition unit that recognizes the surrounding conditions of the mobile body based on the detection results of the surrounding conditions of the mobile body; and a determination unit that determines the target relative position of the mobile body to the user to be either a first target relative position or a second target relative position based on the recognized surrounding conditions.

Images