Mobile body control device, mobile body control method, and program

Abstract

A control device for a moving body is provided with: a recognition unit that recognizes one or more objects present around the moving body and a user of the moving body on the basis of an image in which a situation around the moving body is captured; a generation unit that generates a target trajectory for following the user on the basis of an index value relating to the recognized position of the one or more objects; and a control unit that controls a movement mechanism of the moving body so as to move the moving body along the target trajectory. The generation unit selects an object taking into account the index value on the basis of which of a first region extending between the moving body and the user and a second region located outside the first region the recognized one or more objects exist, and generates the target trajectory.

Description

Technical Field

[0001] This invention relates to a control device for a moving body, a control method for a moving body, and a control program for a moving body. Background Technology

[0002] Previously, robots were known to guide users to desired locations or transport luggage (for example, see Patent Document 1). The aforementioned robot references a database that establishes a correspondence between a maximum moving speed and various areas within the environment, so as to move at a speed that is the upper limit of the set maximum moving speed.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2012-111011 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] However, in existing technologies, there are many parameters to consider when driving autonomously (e.g., avoiding pedestrians), which sometimes makes it difficult to efficiently generate the robot's target trajectory.

[0008] This invention was made with the consideration of such circumstances in mind, and one of its objectives is to provide a control device, a control method, and a program for a moving body capable of efficiently generating a target trajectory for a robot.

[0009] Methods for solving problems

[0010] The control device, control method and program for the mobile body of the present invention adopt the following structure.

[0011] (1): A control device for a mobile body according to one aspect of the present invention includes: an identification unit that identifies one or more objects existing around the mobile body and a user of the mobile body based on an image of the surrounding conditions of the mobile body captured by the device; a generation unit that generates a target track for following the user based on index values ​​related to the positions of the identified one or more objects; and a control unit that controls the movement mechanism of the mobile body in a manner that causes the mobile body to move along the target track, wherein the generation unit filters objects considering the index values ​​based on which of the identified one or more objects exists in a first region extending between the mobile body and the user and a second region located outside the first region, and generates the target track.

[0012] (2): Based on the above (1) scheme, when one of the identified objects exists in the first region, the generation unit considers the index value related to the position of the object to generate the target trajectory, regardless of whether the object is stationary or moving.

[0013] (3): Based on the above (1) scheme, when one of the identified objects exists in the second region, the generation unit generates the target trajectory by considering the index value related to the position of the object only when the object is moving.

[0014] (4): Another aspect of the present invention provides a method for controlling a mobile body, which causes a computer to perform the following processing: based on an image of the surrounding conditions of the mobile body, identify one or more objects existing around the mobile body and the user of the mobile body; generate a target track for following the user based on index values ​​related to the positions of the identified one or more objects; control the movement mechanism of the mobile body in a manner that causes the mobile body to move along the target track; and generate the target track based on which of the identified one or more objects exists in a first region extending between the mobile body and the user and a second region located outside the first region.

[0015] (5): In another aspect of the present invention, a program storing a program causes a computer to perform the following processing: based on an image of the surrounding conditions of a moving body captured by the computer, to identify one or more objects present around the moving body and a user of the moving body; to generate a target track for following the user based on index values ​​related to the positions of the identified one or more objects; to control the moving mechanism of the moving body in a manner that causes the moving body to move along the target track; and to generate the target track based on which of the identified one or more objects is present in a first region extending between the moving body and the user and a second region located outside the first region.

[0016] Invention Effects

[0017] According to the schemes (1) to (5) above, the target trajectory of the robot can be generated efficiently. Attached Figure Description

[0018] Figure 1 This is a diagram illustrating an example of the structure of a mobile body system 1 including a mobile body 100.

[0019] Figure 2 This is a diagram illustrating an example of how the moving body 100 is used.

[0020] Figure 3 This is a diagram used to illustrate the boot mode.

[0021] Figure 4 This is a perspective view showing the moving body 100.

[0022] Figure 5 This is a diagram illustrating an example of the functional structure of the mobile body 100.

[0023] Figure 6 This is a diagram showing an outline of the processing performed by the risk distribution prediction unit 204 and the track generation unit 206.

[0024] Figure 7 This diagram shows an example of the first region AR1 and the second region AR2 set by the track generation unit 206 when generating a track.

[0025] Figure 8 This is a flowchart illustrating an example of the process executed by the control device 200. Detailed Implementation

[0026] Hereinafter, embodiments of the control device, control method and program for the mobile body of the present invention will be described with reference to the accompanying drawings.

[0027] Figure 1 This diagram illustrates an example of the structure of a mobile system 1 including mobile bodies 100. The mobile system 1 includes, for example, one or more terminal devices 2, a management device 10, an information providing device 20, and one or more mobile bodies 100. These communicate, for example, via a network NW. The network NW is, for example, any network such as a LAN, WAN, or Internet connection.

[0028] [Terminal Device]

[0029] Terminal device 2 is, for example, a computer device such as a smartphone or tablet. Terminal device 2 requests access permission for mobile body 100 from management device 10 or obtains information indicating that access is permitted, based on user operations.

[0030] [Management Device]

[0031] According to the request of the terminal device 2, the management device 10 grants the user of the terminal device 2 permission to use the mobile device 100 or manages the reservation for the use of the mobile device 100. For example, the management device 10 generates and manages schedule information that establishes a correspondence between the identification information of pre-registered users and the date and time of the reservation for the use of the mobile device 100.

[0032] [Information Providing Device]

[0033] The information providing device 20 provides the mobile body 100 with the location of the mobile body 100, the area the mobile body 100 is moving in, and the map information of the surrounding area. The information providing device 20 can also generate a track for the mobile body 100 to its destination according to the request of the mobile body 100, and provide the generated track to the mobile body 100.

[0034] [Moving Object]

[0035] The mobile body 100 is used by the user in the following manner. Figure 2 This diagram illustrates one example of how the mobile body 100 is used. The mobile body 100 is, for example, positioned in a designated location within a facility or street. When a user wants to use the mobile body 100, they can operate the operation unit (not shown) of the mobile body 100 to begin use, or operate the terminal device 2 to begin use. For example, when a user goes shopping and has accumulated luggage, they can use the mobile body 100 to place their luggage into its storage compartment. Then, the mobile body 100 moves with the user autonomously, following their movements. The user can continue shopping or proceed to their next destination while their luggage is stored in the mobile body 100. For example, the mobile body 100 can move with the user on sidewalks or crosswalks in driveways. The mobile body 100 can move in areas accessible to pedestrians, such as driveways and sidewalks. For example, the mobile body 100 can be used in indoor or outdoor facilities or private spaces such as shopping malls, airports, parks, and theme parks, and can move in areas accessible to pedestrians.

[0036] The mobile body 100 can also move autonomously in modes such as guided mode and emergency mode, in addition to (or instead of) the following mode that follows the user as described above.

[0037] Figure 3 This diagram illustrates a guided mode. A guided mode is a method of directing a user to a user-specified destination, and it autonomously moves in front of the user based on the user's movement speed to guide them. For example... Figure 3 As shown, in a shopping mall, when a user is looking for a specific item, if the user requests guidance from the mobile device 100 to the location of that item, the mobile device 100 will guide the user there. This allows the user to easily find the desired item. It should be noted that when the mobile device 100 is used in the shopping mall, the mobile device 100 or the information providing device 20 maintains information that establishes a correspondence between the locations of the items, shops, and facilities within the shopping mall and map information, as well as map information of the shopping mall. This map information includes detailed map information such as the width of roads and passageways.

[0038] Emergency mode is a mode in which the mobile body 100 moves autonomously to seek help from nearby people or facilities in order to assist the user in the event of an abnormality (such as a fall) during movement with the user. In addition, the mobile body 100 can also move while maintaining a distance from the user, in addition to following and guiding as described above (or instead).

[0039] Figure 4 This is a perspective view showing the movable body 100. In the following description, the forward direction of the movable body 100 is taken as the positive x-direction, the rearward direction of the movable body 100 is taken as the negative x-direction, and for the width direction of the movable body 100, the left direction is taken as the positive y-direction and the right direction is taken as the negative y-direction, with the positive x-direction as the reference. For directions orthogonal to the x-direction and y-direction, the height direction of the movable body 100 is taken as the positive z-direction.

[0040] The mobile body 100 includes, for example, a base 110, a door 112 provided on the base 110, and wheels (first wheel 120, second wheel 130, and third wheel 140) assembled on the base 110. For example, a user can open the door 112 to put luggage into or take luggage out of the storage compartment provided on the base 110. The first wheel 120 and the second wheel 130 are drive wheels, and the third wheel 140 is an auxiliary wheel (driven wheel). The mobile body 100 may also be able to move using a structure other than wheels, such as tracks.

[0041] A cylindrical support 150 extending in the positive z-direction is provided on the surface of the base 110. A camera 180 for capturing images around the moving body 100 is provided at the end of the support 150 in the positive z-direction. The position of the camera 180 can also be any position different from the one described above.

[0042] Camera 180 is, for example, a camera capable of capturing the periphery of moving object 100 with a wide angle (e.g., 360 degrees). Camera 180 may also include multiple cameras. Camera 180 may also be implemented by combining multiple 120-degree cameras or multiple 60-degree cameras.

[0043] Figure 5 This is a diagram illustrating an example of the functional structure of the moving body 100. Besides... Figure 4In addition to the functional structure shown, the mobile body 100 also includes a first motor 122, a second motor 132, a battery 134, a braking device 136, a steering device 138, a communication unit 190, and a control device 200. The first motor 122 and the second motor 132 operate using power supplied to the battery 134. The first motor 122 drives the first wheel 120, and the second motor 132 drives the second wheel 130. The first motor 122 is an in-wheel motor installed on the rim of the first wheel 120, and the second motor 132 can also be an in-wheel motor installed on the rim of the second wheel 130.

[0044] The braking device 136 outputs braking torque to each wheel based on the instruction of the control device 200. The steering device 138 is equipped with an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism based on the instruction of the control device 200 to change the orientation of the first wheel 120 or the second wheel 130, thereby changing the forward path of the moving body 100.

[0045] The communication unit 190 is a communication interface used to communicate with the terminal device 2, the management device 10, or the information providing device 20.

[0046] [Control Device]

[0047] The control device 200 includes, for example, an identification unit 202, a risk distribution prediction unit 204, a track generation unit 206, a drive control unit 208, and a storage unit 220. The identification unit 202, risk distribution prediction unit 204, track generation unit 206, and drive control unit 208 are implemented, for example, by executing a program (software) using a hardware processor such as a CPU (Central Processing Unit). Some or all of these components can be implemented using hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit), or through the coordinated use of software and hardware. The program can be pre-stored on a storage device such as an HDD (Hard Disk Drive) or flash memory (a storage device with a non-transitory storage medium), or stored on a removable storage medium such as a DVD or CD-ROM (a non-transitory storage medium), and installed by mounting the storage medium onto the drive device. The storage unit 220 is implemented using storage devices such as HDD, flash memory, and RAM (Random Access Memory). The storage unit 220 stores map information 222 referenced by the mobile body 100. The map information 222 may include, for example, map information provided by the information providing device 20, such as the location of the mobile body 100, the area the mobile body 100 is moving in, and the surrounding area. Some or all of the functional structures included in the control device 200 may also be included in other devices. For example, other devices may communicate with the mobile body 100 and cooperate to control the mobile body 100.

[0048] The identification unit 202 identifies, for example, the position (distance from the mobile body 100 and direction relative to the mobile body 100) and states such as speed and acceleration of objects located around the mobile body 100 based on images captured by the camera 180. Objects include traffic participants (pedestrians), obstacles within facilities, and roads. The identification unit 202 identifies and tracks users of the mobile body 100. For example, it tracks users based on images captured of users registered when they use the mobile body 100 (e.g., user facial images), user facial images provided by the terminal device 2 or management device 10 (or feature quantities obtained from user facial images). The identification unit 202 also identifies gestures performed by the user. It should be noted that a detection unit different from a camera, such as a radar device or LIDAR, may also be installed on the mobile body 100. In this case, the identification unit 202 uses the detection results of the radar device or LIDAR instead of images (or based on images) to identify the conditions around the mobile body 100.

[0049] The risk distribution prediction unit 204 sets an index value, or risk, for the object (and its surrounding area) identified by the identification unit 202, indicating the degree to which the moving body 100 should not enter or approach. The higher the risk value, the less the moving body 100 should enter or approach; the closer the value is to zero, the more preferable it is for the moving body 100 to pass. Therefore, generally speaking, the closer to the identified object's location, the higher the risk value; conversely, the farther away from the identified object's location, the lower the risk value. However, this relationship can also be reversed. Risk is an example of a "location-related index value" in the technical solution.

[0050] The risk distribution prediction unit 204 can set risks not only for the current time point t, Δt later (time t + Δt), 2Δt later (time t + 2Δt), ..., but also for future time points defined at certain time intervals. The risk distribution prediction unit 204 predicts the risks at future time points based on the changes in the position of the moving object target continuously identified by the identification unit 202.

[0051] The track generation unit 206 generates a track to the destination based on the risk predicted by the risk distribution prediction unit 204. Here, when the mobile body 100 is in follower mode, the destination represents the user being followed or a designated location near the user. On the other hand, in guide mode, for example, it represents the location of goods or facilities set by the user. In this case, the user specifies the location of the goods or facilities, and the mobile body 100 compares the specified location with map information 222, setting the determined location as the destination. Furthermore, in guide mode, if the user-set location is far from the mobile body 100's current location, the user-set location can be set as the final destination, and a location within a specified range from the current location can be set as a provisional destination. Additionally, in guide mode, the user may not necessarily set the destination; the guide mode can predict the user's movement direction and autonomously move ahead of the user based on the user's movement speed. In this case, the destination of the mobile body 100 can be set as a location within a specified range ahead of the user.

[0052] The track generation unit 206 generates multiple tracks for reaching the destination, calculates the risk of each track point constituting each track, and calculates the total value of the calculated risks as the track risk. The track generation unit 206 uses the track that meets the preset benchmarks as the target track for the movement of the mobile body 100, provided that the risks of each track point meet preset benchmarks (e.g., the risks of each track point are below threshold Th1) and the total risk of the track meets preset benchmarks (e.g., the total risk is below threshold Th2).

[0053] The drive control unit 208 controls the moving mechanisms such as the motors (first motor 122, second motor 132), the braking device 136, and the steering device 138, so that the moving body 100 travels along the track generated by the track generation unit 206.

[0054] Figure 6 This is a diagram illustrating a summary of the processes performed by the risk distribution prediction unit 204 and the trajectory generation unit 206. As an example, Figure 6 This illustrates a scenario where the mobile body 100 travels in follow mode, i.e., the user U itself is set as the destination. For example... Figure 6As shown, as an example, the risk distribution prediction unit 204 sets the risk on the bird's-eye view using ellipses or circles based on the direction and speed of travel as contour lines for objects other than user U and moving body 100, and sets a certain risk value for immovable areas BD such as walls. R(OB1) is the risk of stationary object (person standing in place) OB1, R(OB2) is the risk of moving object (pedestrian) OB2, R(OB3) is the risk of moving object (pedestrian) OB3, and R(OB4) is the risk of moving object (pedestrian) OB4. Since pedestrians are constantly moving, risks are set for future time points at positions different from the current time. R(OB2)_t is the risk of object OB2 in a certain control cycle, R(OB2)_t+Δt is the risk of object OB2 in the next control cycle, and R(OB2)_t+Δ2t is the risk of object OB2 in the control cycle after that. R(BD) is the risk of immovable area BD. In the figure, the density of the shading represents the risk value; the denser the shading, the greater the risk.

[0055] The track generation unit 206 generates multiple tracks from the moving body 100 to the destination (i.e., user U). Figure 6 In this example, orbits TP1 and TP2 are generated. The orbit generation unit 206 calculates the risk of each orbital point constituting orbits TP1 and TP2, and the sum of the calculated risks is taken as the risk of the orbit. Figure 6 In the case of orbit TP1, the orbit generation unit 206 calculates the sum of the risk values ​​r(TP1)_t+Δt at time point t+Δt and r(TP1)_t+2Δt at time point t+2Δt as the risk value. Similarly, for orbit TP2, the unit calculates the sum of the risk values ​​r(TP2)_t+Δt at time point t+Δt and r(TP2)_t+2Δt at time point t+2Δt as the risk value. The orbit generation unit 206 compares the risk values ​​of orbit TP1 and orbit TP2. Since the risk value of orbit TP1 is positive and the risk value of orbit TP2 is zero, orbit TP2 is selected as the target orbit.

[0056] In this way, the risk distribution prediction unit 204 sets the risk for the objects and their surrounding areas identified by the identification unit 202, and the track generation unit 206 selects the track with the lowest risk value from among the multiple candidate tracks. At this time, while the track generation unit 206 performs path calculations considering the risks of all objects identified by the identification unit 202, it also performs calculations for objects with significantly low contact risks, such as objects located relatively far from the moving body 100 and stationary objects, which is inefficient.

[0057] [Settings for Zone 1 and Zone 2]

[0058] Considering the above, the trajectory generation unit 206 sets a first region AR1 extending between the moving body 100 and the destination and a second region AR2 located outside the first region AR1. Based on which region in the first region AR1 and the second region AR2 the identified object is located, it filters out objects that take into account risk, and generates a target trajectory TP based on the risk of the filtered objects.

[0059] Figure 7 This diagram illustrates an example of the first region AR1 and the second region AR2 set by the track generation unit 206 when generating a track. Figure 7 As an example, the identification unit 202 identifies objects OB1 to OB4. Figure 7 As shown, the trajectory generation unit 206, for example, will take the moving body 100 as the starting point and the destination (in...) Figure 7 The region with user U as the endpoint is set as the first region AR1, and the region whose width is extended by a specified distance d based on the set first region AR1 is set as the second region AR2.

[0060] The trajectory generation unit 206 generates a target trajectory for objects identified as existing in the first region AR1, regardless of whether the object is stationary or moving, taking into account the object's risk value. Conversely, for objects existing in the second region AR2, it only considers the object's risk value to generate a target trajectory if the object is moving. This is because, as described above, the first region AR1 is defined as the region originating from the moving body 100 and ending at its destination; therefore, if an object outside the first region AR1 is stationary, it is assumed that considering its risk value is less necessary. Therefore, in Figure 7 In the case of the trajectory generation unit 206, the risk values ​​of the moving object OB1 and the stationary object OB2 within the first region AR1 and the risk value of the moving object OB3 within the second region AR2 are considered (in Figure 7 In the diagram, these objects are represented by diagonal lines. Without considering the risk value of the stationary object OB4 in the second region AR2, the risk values ​​of multiple orbits are calculated, and the orbit with the lowest calculated risk value is used as the target orbit TP.

[0061] [Processing flow]

[0062] Next, refer to Figure 8 The process flow of the processing performed by the control device 200 is described. Figure 8 This is a flowchart illustrating an example of the process executed by the control device 200. Figure 8The process shown is performed when the mobile body 100 is traveling in follow mode, that is, when the user U itself is set as the destination. However, even when the mobile body 100 is traveling in guide mode, the process shown in this flowchart is performed in the same way as long as the destination is recognized by the identification unit 202.

[0063] First, the identification unit 202 identifies one or more objects surrounding the mobile body 100 and the user of the mobile body 100, for example, based on images captured by the camera 180 (step S100). Next, the risk distribution prediction unit 204 sets a risk for the one or more objects identified by the identification unit 202 (step S102). Next, the track generation unit 206 sets a first region AR1 and a second region AR2 based on the identified user and the position of the mobile body 100 (step S104).

[0064] Next, the track generation unit 206 classifies which of the first region AR1 and the second region AR2 the identified object belongs to (step S106). Then, the track generation unit 206 generates a target track based on the risk of all objects contained in the first region AR1 and the risk of moving objects contained in the second region (step S108). Next, the drive control unit 208 controls the movement mechanism to move the moving body 100 along the generated target track (step S110). Thus, the processing of this flowchart ends.

[0065] It should be noted that in the above flowchart processing, after the risk distribution prediction unit 204 sets the risk for the identified objects, the track generation unit 206 classifies the objects with set risks into a first region AR1 and a second region AR2. However, the present invention is not limited to such a structure. The track generation unit 206 may also first classify the identified objects into a first region AR1 and a second region AR2, and then the risk distribution prediction unit 204 sets the risk for all objects contained in the first region AR1 and moving objects contained in the second region AR2.

[0066] Furthermore, in the above-described embodiment, as an example, the track generation unit 206 generates multiple candidate tracks and selects the track with the lowest risk value as the target track. However, the present invention is not limited to such a structure. The track generation unit 206 may generate the target track using any algorithm that at least considers the risk value of the object to generate the track.

[0067] Furthermore, in the above-described embodiment, as an example, the track generation unit 206 sets the first region AR1 and the second region AR2 as rectangular regions. However, the present invention is not limited to such a structure. The track generation unit 206 may at least set the first region AR1 as a predetermined region from the moving body 100 to the destination, and set the second region AR2 as a different predetermined region existing outside the first region AR1.

[0068] According to the implementation method described above, a target trajectory is generated based on which region—either a first region extending between the moving body and the user or a second region located outside the first region—is identified. This allows for the efficient generation of the robot's target trajectory.

[0069] The implementation methods described above can be performed as follows.

[0070] A mobile body, comprising:

[0071] Storage medium, which stores computer-readable instructions; and

[0072] The processor, which is connected to the storage medium,

[0073] The processor performs the following processing by executing computer-readable instructions:

[0074] Based on images captured of the surroundings of the moving object, identify one or more objects present around the moving object and the user of the moving object;

[0075] Based on the index values ​​related to the position of the identified one or more objects, a target trajectory for following the user is generated;

[0076] The moving mechanism controls the movement of the moving body in a manner that causes the moving body to move along the target track; and

[0077] The target trajectory is generated based on which region, either a first region extending between the mobile body and the user or a second region located outside the first region, the identified one or more objects are respectively located in.

[0078] The above describes specific embodiments of the present invention, but the present invention is not limited to such embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

[0079] Explanation of reference numerals in the attached figures:

[0080] 100 moving bodies

[0081] 200 control device

[0082] 202 Identification Department

[0083] 204 Risk Distribution Forecasting Department

[0084] 206 Orbit Generation Department

[0085] 208 Drive Control Unit

[0086] 220 Storage Department.

Claims

1. A control device for a moving body, wherein, The control device for the moving body includes: The identification unit identifies one or more objects present around the moving object and the user of the moving object based on images of the surrounding environment captured by the unit. The generation unit generates a target trajectory to the destination based on index values ​​related to the positions of the identified one or more objects. as well as The control unit controls the movement mechanism of the moving body in a manner that causes the moving body to move along the target track. The generation unit filters objects that take into account the index value based on which region the identified one or more objects are located in, either a first region extending between the moving body and the destination or a second region located outside the first region, and generates the target trajectory.

2. The control device for the moving body according to claim 1, wherein, When one of the identified objects exists in the first region, the generation unit considers the index value related to the position of the object to generate the target trajectory, regardless of whether the object is stationary or moving.

3. The control device for the moving body according to claim 1, wherein, When one of the identified objects exists in the second region, the generation unit generates the target trajectory by considering index values ​​related to the position of the object, only if the object is moving.

4. A method for controlling a moving body, wherein, The control method for the moving body causes the computer to perform the following processing: Based on images captured of the surroundings of the moving object, identify one or more objects present around the moving object and the user of the moving object; Based on the index values ​​related to the position of the identified one or more objects, a target trajectory for following the user is generated; The moving mechanism of the moving body is controlled in a manner that causes the moving body to move along the target track; as well as The target trajectory is generated by filtering objects based on which region, either a first region extending between the mobile body and the user or a second region located outside the first region, the identified one or more objects are respectively located in.

5. A program, wherein, The program causes the computer to perform the following processes: Based on images captured of the surroundings of the moving object, identify one or more objects present around the moving object and the user of the moving object; Based on the index values ​​related to the position of the identified one or more objects, a target trajectory for following the user is generated; The moving mechanism of the moving body is controlled in a manner that causes the moving body to move along the target track; as well as The target trajectory is generated by filtering objects based on which region, either a first region extending between the mobile body and the user or a second region located outside the first region, the identified one or more objects are respectively located in.

Images