Control device, control method, and program

Abstract

Provided is a control device for operating a mobile body by switching between a following mode for following a first user, a leading mode for leading the first user, and a remote operation mode in which a second user remotely operates a mobile body, the control device comprising an acquisition unit for acquiring a remote operation request that the mobile body be remotely operated, a mode-switching unit for switching the operation mode of the mobile body to the remote operation mode in response to the acquisition of the remote operation request while the mobile body is operating in the following mode or the leading mode, a generation unit for generating a target trajectory of the mobile body on the basis of the switched operation mode, and a control unit for controlling the mobile body on the basis of the target trajectory.

Description

Control device, control method, and program

[0001] The present invention relates to a control device, a control method, and a program.

[0002] Conventionally, a leading robot, which is an autonomous mobile body that leads a user in a supermarket, a shopping mall, etc., has been proposed (Patent Document 1). This leading robot generates a route to follow taking into account the surrounding congestion.

[0003] WO 2003 / 189105

[0004] However, while conventional mobile bodies such as leading robots have a leading mode in which they lead a user, they do not have a following mode in which they follow a user or a remote control mode in which a user remotely controls the mobile body. For this reason, conventional mobile bodies sometimes cannot switch between the following mode or leading mode and the remote control mode to travel.

[0005] The present invention has been made in consideration of these circumstances, and one of its objects is to provide a control device, a control method, and a program that can cause a moving object to travel by switching between a following mode or a leading mode, and a remote control mode.

[0006] The control device, control method, and program according to the present invention employ the following configuration: (1): A control device according to one aspect of the present invention is a control device that operates a moving object by switching between a following mode in which the moving object follows a first user, a leading mode in which the moving object leads the first user, and a remote operation mode in which a second user remotely operates the moving object, and includes: an acquisition unit that acquires a remote operation request to remotely operate the moving object, a mode switching unit that switches the operation mode of the moving object to the remote operation mode in response to the remote operation request being acquired while the moving object is operating in the following mode or the leading mode, a generation unit that generates a target trajectory of the moving object based on the switched operation mode, and a control unit that controls the moving object based on the target trajectory.

[0007] (2): In the above aspect (1), the device further includes a pairing unit that pairs the first user with the moving object, and a limiting unit that limits the range in which the second user can operate the moving object to within a predetermined range from the position of the first user with whom the pairing has been performed.

[0008] (3): In the above aspect (2), the control unit controls the mobile body so that, when the pairing is performed, the distance between the first user and the mobile body is shorter than when the pairing is not performed.

[0009] (4): In the above aspect (1), the mode switching unit displays an image of the second user on a display unit provided on the mobile object in response to the remote control request being acquired, and then switches the operating mode of the mobile object to the remote control mode in response to a predetermined condition being satisfied.

[0010] (5) In the aspect (1) above, the mode switching unit switches the operation mode of the moving object to either the following mode or the leading mode depending on the behavior of the first user.

[0011] (6): In the above aspect (5), when the mode switching unit detects that the first user is moving in a direction deviating from a predetermined route, it switches the operation mode of the moving body to the leading mode.

[0012] (7) In the aspect (5) above, when the mode switching unit detects that the first user has lost his / her way, it switches the operation mode of the moving object to the leading mode.

[0013] (8) In the aspect (5) above, when the mode switching unit detects that the first user is about to overtake the moving body, it switches the operation mode of the moving body to the following mode.

[0014] (9): Another aspect of the present invention provides a mobile body system comprising a mobile body and an information processing device capable of communicating with the mobile body via a network, wherein the mobile body comprises a control device that switches between a following mode in which the mobile body follows a first user, a leading mode in which the mobile body leads the first user, and a remote control mode in which a second user remotely controls the mobile body to operate the mobile body, and the control device comprises an acquisition unit that acquires a remote control request to remotely control the mobile body from the information processing device, a mode switching unit that switches the operation mode of the mobile body to the remote control mode in response to the remote control request being acquired while the mobile body is operating in the following mode or the leading mode, a generation unit that generates a target trajectory of the mobile body based on the switched operation mode, and a control unit that controls the mobile body based on the target trajectory.

[0015] (10): In another aspect of the present invention, a control method is provided in which a control device that operates a moving body by switching between a following mode in which the moving body follows a first user and a remote control mode in which the moving body is remotely controlled by a second user executes the following processes: acquiring a remote control request to remotely control the moving body; switching the operation mode of the moving body from the following mode to the remote control mode in response to the remote control request being acquired while the moving body is operating in the following mode; generating a target trajectory for the moving body based on the switched operation mode; and controlling the moving body based on the target trajectory.

[0016] (11): A program according to another aspect of the present invention causes a processor of a control device that operates a moving body by switching between a following mode in which the moving body follows a first user and a remote control mode in which a second user remotely controls the moving body to execute the following processes: acquiring a remote control request to remotely control the moving body; switching the operation mode of the moving body from the following mode to the remote control mode in response to the remote control request being acquired while the moving body is operating in the following mode; generating a target trajectory for the moving body based on the switched operation mode; and controlling the moving body based on the target trajectory.

[0017] According to the aspects (1) to (11), the moving body can be caused to travel by switching between a following mode or a leading mode and a remote control mode.

[0018] 1 is a diagram showing an example of the configuration of a mobile body system 1 including a mobile body 100. FIG. 2 is a perspective view showing the mobile body 100. FIG. 3 is a diagram showing the configuration of the mobile body 100 equipped with a control device 200. FIG. 4 is a diagram showing an example of the configuration of the control device 200. FIG. 5 is a diagram showing an example of a target trajectory generated by a generation unit 260. FIG. 6 is a diagram for explaining an environmental target risk function and an environmental target-based benefit function. X ｅ_ｌ 1 is a diagram illustrating an example of an environmental target risk function and an environmental target-based benefit function in an axial direction. ｅ_ｌ 1 is a diagram showing an example of a potential function in an axial direction; FIG. 2 is an image diagram of an environmental target-based benefit function when environmental targets are detected at multiple detection positions; FIG. 3 is a diagram for explaining a traffic participant / obstacle risk function; FIG. 4 is a diagram for explaining a traffic participant / obstacle risk function based on traffic participant U3; X ｔｐ_ｌ 1 is a diagram illustrating an example of a traffic participant / obstacle risk function in the axial direction. ｔｐ_ｌ1 is a diagram illustrating an example of a traffic participant / obstacle risk function in an axial direction. FIG. 2 is a diagram illustrating a follow mode benefit function. FIG. 3 is a diagram illustrating an example of a movement amount of the moving body 100. FIG. 4 is a diagram illustrating the field of view of a user (person to be followed). FIG. 5 is a diagram illustrating an example of a plurality of target positions. FIG. 6 is a diagram illustrating an example of a state in which a follow mode benefit function is set at a plurality of target positions. FIG. 7 is a diagram illustrating an example of a follow mode benefit function in the X-axis direction. FIG. 8 is a diagram illustrating an example of a follow mode benefit function in the Y-axis direction. FIG. 9 is a diagram illustrating a leading mode benefit function. FIG. 10 is a diagram illustrating a process for estimating a predicted trajectory of a user U1. FIG. 11 is a diagram illustrating an example of a state in which a leading mode benefit function is set at a plurality of target positions. FIG. 12 is a diagram illustrating an example of a leading mode benefit function in the X-axis direction. FIG. 13 is a diagram illustrating a state in which a moving body 100 is traveling automatically in a remote control mode. FIG. 14 is a diagram illustrating a state in which a user U2 gives an instruction to move the moving body 100 to the left. FIG. 15 is a diagram illustrating an example of a remote control benefit function. 1 is a diagram for explaining how the traveling direction of the moving body 100 is changed when a remote-control benefit function is set. FIG. 2 is a flowchart showing an example of processing executed by the control device 200. FIG. 3 is a flowchart showing a second example of processing executed by the control device 200. FIG. 4 is a flowchart showing a third example of processing executed by the control device 200.

[0019] Hereinafter, with reference to the drawings, embodiments of a control device, a control method, and a program of the present invention will be described. The control device of the present invention controls the movement mechanism of a mobile object to move the mobile object. The mobile object of the present invention autonomously moves in an area where pedestrians walk. The area where pedestrians walk includes sidewalks, public open spaces, floors within buildings, etc., and may also include roadways. In the following description, it is assumed that no person rides on the mobile object, but a person may ride on the mobile object. The mobile object operates in a following mode in which it follows a user, a leading mode in which it leads a user, or a remote control mode (avatar mode) in which the user remotely controls the mobile object. The user to be followed or led is, for example, a pedestrian, but may also be a robot or an animal.

[0020] [Mobile System] Fig. 1 is a diagram showing an example of the configuration of a mobile system 1 including a mobile object 100. The mobile object system 1 includes, for example, the mobile object 100, a user terminal device 300, and an information processing device 400. These communicate with each other via, for example, a network NW. The network NW is any network such as a LAN, a WAN, or an internet connection. Note that the mobile object 100 and the user terminal device 300 may communicate directly via short-range wireless communication without going through the network NW.

[0021] [User Terminal Device] The user terminal device 300 is, for example, a portable terminal device such as a smartphone or tablet device operated by the user U1. The user terminal device 300 receives instructions from the user U1 and transmits the received instructions to the mobile object 100. For example, the user terminal device 300 may be equipped with a touch panel and may receive instructions from the user U1 input using the touch panel. The user terminal device 300 may also be equipped with a voice recognition function and may receive instructions based on the voice of the user U1 recognized using the voice recognition function. The user terminal device 300 may also have a display unit that displays information received from the mobile object 100.

[0022] [Information Processing Device] The information processing device 400 is, for example, a computer that remotely controls the moving body 100 based on the operation of the user U2. The information processing device 400 may include an input device such as a joystick. For example, the user U2 may control the traveling direction of the moving body 100 by operating the joystick. Specifically, the user U2 can instruct the moving body 100 to move leftward relative to the traveling direction by tilting the joystick to the left. Furthermore, the user U2 can instruct the moving body 100 to move rightward relative to the traveling direction by tilting the joystick to the right.

[0023] The information processing device 400 transmits operation information based on the operation of the user U2 to the moving body 100. The joystick may be provided with a start button for instructing the moving body 100 to start moving and a stop button for instructing the moving body 100 to stop moving.

[0024] The information processing device 400 also receives image information transmitted from the mobile object 100. The image information is an image captured by a camera installed on the mobile object 100. The information processing device 400 may include a display unit such as a liquid crystal display or an organic EL display. The information processing device 400 may display the image information received from the mobile object 100 on the display unit. The user U2 operates an input device such as a joystick based on the image of the area around the mobile object 100 displayed on the display unit 330.

[0025] User U2 may control the angle of view of the camera of moving body 100 via information processing device 400. For example, user U2 may control the pan, tilt, and zoom of the camera provided on moving body 100 by operating information processing device 400. Furthermore, information processing device 400 may be a wearable device such as VR goggles. In this case, the user may control the camera in accordance with the movement of the VR goggles.

[0026] 2 is a perspective view showing the mobile body 100. The mobile body 100 includes, for example, a base body 10, a door unit 60 provided on the base body 10, and wheels (first wheel 20, second wheel 30, and third wheel 40) attached to the base body 10. For example, a user U1 can open the door unit 60 to put luggage into a storage compartment provided on the base body 10 or take luggage out of the storage compartment. The first wheel 20 and the second wheel 30 are driving wheels, and the third wheel 40 is an auxiliary wheel (driven wheel).

[0027] A cylindrical support body 50 extending upward is provided on the upper surface of the base body 10. A camera 80 that captures images of the surroundings of the moving body 100 is provided at the end of the support body 50. The position at which the camera 80 is provided may be any position different from the above.

[0028] The camera 80 is, for example, a camera that can capture images of the periphery of the moving body 100 at a wide angle (for example, 360 degrees). The camera 80 may include multiple cameras. For example, the camera 80 may be realized by combining multiple 120-degree cameras or multiple 60-degree cameras.

[0029] 3 is a diagram showing the configuration of a mobile object 100 equipped with a control device 200. The mobile object 100 includes, for example, an HMI 110, a detection device 120, a position identification device 130, a communication device 170, a base 160 equipped with the control device 200, a moving mechanism 140 attached to the base 160, and a sensor 150 attached to the moving mechanism 140, etc. The base 160 may be the same as the base 10 in FIG.

[0030] The HMI 110 presents various information to the user U1 and accepts input operations by the user U1. The HMI 110 includes various display devices (display units), speakers, microphones, buzzers, touch panels, switches, keys, and the like.

[0031] The detection device 120 is a device that generates data for recognizing objects and the user U1 present around the mobile body 100. The detection device 120 includes, for example, an object recognition device that recognizes objects based on the output of the camera 80. Note that the detection device 120 may recognize objects by using sensors such as a radar device, a LIDAR (Light Detection and Ranging), and an ultrasonic sensor in addition to the camera 80, and by performing sensor fusion processing based on the outputs of these sensors.

[0032] The positioning device 130 is a device that determines the position of the mobile body 100. The positioning device 130 includes, for example, a Global Navigation Satellite System (GNSS) receiver that determines the position of the mobile body 100 based on signals received from GNSS satellites. The positioning device 130 may determine or supplement the position of the mobile body 100 using an Inertial Navigation System (INS) that uses the output of a sensor 150, which will be described later. The positioning device 130 may also have an electromagnetic wave receiving function and determine or supplement the position of the mobile body 100 based on the intensity of electromagnetic waves arriving from surrounding electromagnetic wave sources (whose positions are known).

[0033] The movement mechanism 140 is a mechanism for moving the moving body 100 in any direction. The movement mechanism 140 includes, for example, a plurality of wheels (first wheel 20, second wheel 30, and third wheel 40), a drive motor attached to one or more of the wheels, and a steering device attached to one or more of the wheels. There are no particular restrictions on the configuration of the movement mechanism 140, and the movement mechanism 140 may include components other than wheels, such as pseudo feet for bipedal walking or caterpillars.

[0034] The sensor 150 is a sensor for detecting the behavior of the mobile body 100. The sensor 150 includes, for example, a wheel speed sensor for detecting the speed of the wheels, an acceleration sensor for detecting the acceleration acting on the mobile body 100, a yaw rate sensor attached near the center of gravity of the base body 160 in the horizontal direction, a steering angle sensor for detecting the steering angle of the steered wheels (steered wheels), and an orientation sensor for detecting the orientation of the mobile body 100 in the horizontal direction.

[0035] The communication device 170 is a wireless communication module that performs wireless communication with a wireless base station connected to the network NW. The communication device 170 communicates with the user terminal device 300 and the information processing device 400 via the network NW. The communication device 170 may also directly communicate with the user terminal device 300 by short-range wireless communication.

[0036] FIG. 4 is a diagram illustrating an example of the configuration of the control device 200. The control device 200 includes, for example, a first detection unit 210, a first determination unit 220, an estimation unit 225, a second detection unit 230, a second determination unit 240, a calculation unit 250, a generation unit 260, a control unit 270, an acquisition unit 280, a mode switching unit 285, a pairing unit 290, and a restriction unit 295. These components are implemented by, for example, a hardware processor such as a central processing unit (CPU) executing a program (software). Some or all of these components may be implemented by hardware (including circuitry) such as a large-scale integration (LSI), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a graphics processing unit (GPU), or may be implemented by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as a hard disk drive (HDD) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed in the storage device by inserting the storage medium into a drive device. Note that the control device 200 may store map information including at least a local map of the location where the mobile object 100 operates in the storage device.

[0037] The first detection unit 210 detects the position of user U1 based on information input from the detection device 120. When the moving body 100 is operating in a following mode or a leading mode, the first detection unit 210 repeatedly detects the position of user U1 at predetermined time intervals. The following mode is a mode in which the moving body 100 follows a person to be followed. The leading mode is a mode in which the moving body 100 leads a person to be led. The first detection unit 210 also detects an operation performed on the moving body 100 by user U2 based on operation information received by the communication device 170 from the information processing device 400.

[0038] The second detection unit 230 detects the position of an interfering object around the mobile body 100 that the mobile body 100 should avoid, based on information input from the detection device 120. An interfering object is an object that interferes with the traveling of the mobile body 100. For example, the interfering object is an environmental object, a traffic participant, an obstacle, or other object around the mobile body 100 that the mobile body 100 should avoid. An environmental object is, for example, an object at the boundary of the traveling lane that the mobile body 100 cannot enter, such as a wall or a lawn. A traffic participant is, for example, a pedestrian or a vehicle. An obstacle is, for example, a stationary object on the traveling lane that obstructs the traveling of the mobile body 100. The mobile body 100 repeatedly detects the position of the interfering object at predetermined time intervals to avoid contact with the interfering object while traveling.

[0039] The estimation unit 225 estimates a predicted trajectory of the user U1 based on the current and past positions of the user U1 detected by the first detection unit 210. Details of the process of estimating the predicted trajectory of the user U1 will be described later.

[0040] In the following mode, the first determination unit 220 determines a following mode benefit function indicating a degree to which it is recommended that the moving object 100 should move, based on the current and past positions of the user U1 detected by the first detection unit 210. In the leading mode, the first determination unit 220 determines a leading mode benefit function indicating a degree to which it is recommended that the moving object 100 should move, based on the predicted trajectory estimated by the estimation unit 225. In the remote control mode, the first determination unit 220 determines a remote control benefit function indicating a degree to which it is recommended that the moving object 100 should move, based on the operation of the user U2 on the moving object 100 detected by the first detection unit 210. In addition, the first determination unit 220 determines an environmental landmark-based benefit function based on the positions of environmental landmarks detected by the second detection unit 230. Details of the following mode benefit function, leading mode benefit function, remote control benefit function, and environmental landmark-based benefit function will be described later.

[0041] The second determination unit 240 determines a risk function indicating the degree of risk of interference between the mobile body 100 and the interfering object, based on the position of the interfering object detected by the second detection unit 230. Specifically, the second determination unit 240 determines the environmental object risk function based on the position of the environmental object detected by the second detection unit 230. The second determination unit 240 also determines a traffic participant / obstacle risk function based on the positions of the traffic participants and obstacles detected by the second detection unit 230. The environmental object risk function and the traffic participant / obstacle risk function will be described in detail later.

[0042] In the following mode, the calculation unit 250 calculates an evaluation function for evaluating a route along which the mobile object 100 will travel, based on the following mode benefit function and the environmental landmark-based benefit function determined by the first determination unit 220, and the environmental landmark risk function and the traffic participant / obstacle risk function determined by the second determination unit 240. In addition, in the leading mode, the calculation unit 250 calculates an evaluation function for evaluating a route along which the mobile object 100 will travel, based on the leading mode benefit function and the environmental landmark-based benefit function determined by the first determination unit 220, and the environmental landmark risk function and the traffic participant / obstacle risk function determined by the second determination unit 240. In addition, in the remote control mode, the calculation unit 250 calculates an evaluation function for evaluating the route along which the mobile object 100 will travel, based on the remote control benefit function and the environmental landmark-based benefit function determined by the first determination unit 220, and the environmental landmark risk function and the traffic participant / obstacle risk function determined by the second determination unit 240. Details of the evaluation function will be described later.

[0043] The generation unit 260 generates a target trajectory for the moving body 100 based on the evaluation function calculated by the calculation unit 250. For example, the generation unit 260 generates the target trajectory for the moving body 100 using a model that represents a portion of the periphery of a geometric shape. The model may be an arc model that represents an arc. The generation unit 260 generates the target trajectory for the moving body 100 by modeling the trajectory of the moving body 100 using the arc model. The generation process of the target trajectory by the generation unit 260 will be described below.

[0044] [Process for generating target trajectory] Fig. 5 is a diagram showing an example of a target trajectory generated by the generation unit 260. As shown in Fig. 5, the generation unit 260 generates a target trajectory for the moving body 100 by combining first to third predicted trajectories, which are arc-shaped.

[0045] 5, an XY coordinate system is defined with the center of the current position of the moving body 100 as the origin, the X axis in front of the moving body 100 as the front, and the Y axis to the left of the moving body 100 as the left.

[0046] Then, the position after the first predicted time from the origin in this XY coordinate system is defined as the first predicted position Z ｍ１ and the first predicted position Z ｍ１ The trajectory up to is defined as a first predicted trajectory in the form of an arc.

[0047] The radius of curvature of the first predicted trajectory is R ｍ１ The rotation angle of the first predicted trajectory is θ ｍ１ The rotation angle θ ｍ１ corresponds to the angle between the Y axis and the Y' axis, which will be described later.

[0048] Also, the first predicted position Z ｍ１ The predicted position when the trajectory is divided into three equal parts is calculated by dividing the trajectory from the origin to the first predicted position Z ｍ１ , three predicted positions Z ｍ１１ , Z ｍ１２ , Z ｍ１３ (=Z ｍ１ )

[0049] Next, the second predicted trajectory will be described. ｍ１ (= predicted position Z ｍ１３ ) is the origin, and the first predicted position Z ｍ１ An X'-Y' coordinate system is defined in which the tangent direction at is the X' axis and the direction perpendicular to this tangent line is the Y' axis.

[0050] Then, the origin of this X'-Y' coordinate system (i.e., the first predicted position Z ｍ１ ) and the position after the second predicted time is set as the second predicted position Z ｍ２ and the first predicted position Z ｍ１ to the second predicted position Z ｍ２ The trajectory up to is defined as a second predicted trajectory in the form of an arc.

[0051] The radius of curvature of the second predicted orbit is R ｍ２ The rotation angle of the second predicted orbit is θ ｍ２ The rotation angle θ ｍ２ corresponds to the angle between the Y′ axis and the Y″ axis described below.

[0052] Next, the third predicted trajectory will be described. ｍ２ is set as the origin, and the second predicted position Z ｍ２An X''-Y'' coordinate system is defined in which the tangent direction at is the X'' axis and the direction perpendicular to this tangent line is the Y'' axis.

[0053] The origin of this X'-Y' coordinate system (i.e., the second predicted position Z ｍ２ ) and the position after the third predicted time is the third predicted position Z ｍ３ and the second predicted position Z ｍ２ to the third predicted position Z ｍ３ The trajectory up to is defined as a third predicted trajectory in the form of an arc.

[0054] As described above, the first predicted orbit is determined as a combination of three predicted orbits that are shorter than the second and third predicted orbits. ｍ１１ The trajectory to and predicted position Z ｍ１１ From predicted position Z ｍ１２ The trajectory to and predicted position Z ｍ１２ From predicted position Z ｍ１３ The first predicted trajectory is determined as a combination of the trajectories up to

[0055] This is because the first predicted trajectory is closer to the moving body 100 than the second predicted trajectory and the third predicted trajectory, and therefore it is necessary to generate a predicted trajectory that can more reliably avoid interference with an interfering object. Note that the generation unit 260 may determine the first predicted trajectory as a combination of two or four or more predicted trajectories.

[0056] The generator 260 calculates the rotation angle θ ｍ１ ~θ ｍ３ and the radius of curvature R ｍ１ ~R ｍ３A plurality of trajectories are generated by determining a combination of a plurality of patterns of the above. Then, the generation unit 260 evaluates each of the generated plurality of trajectories using the evaluation function calculated by the calculation unit 250. Thereafter, the generation unit 260 determines one of the generated plurality of trajectories as a target trajectory for the moving body 100 based on the evaluation result using the evaluation function. By evaluating the generated plurality of trajectories based on the evaluation function, the generation unit 260 can generate a target trajectory for the moving body 100 that has a low risk of interfering with an interfering object and is suitable for the following mode or the leading mode. Details of the evaluation process using the evaluation function will be described later.

[0057] Returning to the explanation of Fig. 4, the control unit 270 controls the movement mechanism 140 so that the moving body 100 moves along the target trajectory generated by the generation unit 260. The control unit 270 controls the drive motor and steering device so that the position and behavior of the moving body 100 obtained from the output of the sensor 150 approach the target trajectory.

[0058] The acquisition unit 280 acquires a remote operation request to remotely operate the mobile object 100. For example, the acquisition unit 280 may acquire the remote operation request from the information processing device 400. Specifically, the information processing device 400 may transmit a login request to the mobile object 100 based on an operation by the user U2, for logging in to the mobile object 100, and the acquisition unit 280 may acquire the login request received from the information processing device 400 as the remote operation request.

[0059] The mode switching unit 285 selects one of a plurality of modes (e.g., a following mode, a leading mode, and a remote control mode) in response to an input from the user U1. For example, the mode switching unit 285 may select one of the plurality of modes in response to an instruction from the user U1 input via the HMI 110. The mode switching unit 285 may also receive an instruction to change the mode from the user terminal device 300 using the communication device 170 and select a mode based on the received instruction. The mode switching unit 285 switches the operation mode of the mobile object 100 to the remote control mode in response to a remote control request being acquired while the mobile object 100 is operating in the following mode or the leading mode. Details of this mode switching will be described later.

[0060] The pairing unit 290 performs pairing between the user U1 and the moving body 100. Pairing is a process for pairing the user U1 with the moving body 100 by authenticating the user U1. Note that the user U1 paired with the moving body 100 becomes a follower in the follow mode and a leading person in the leading mode. Details of the process of the pairing unit 290 will be described later.

[0061] In the remote operation mode, the limiting unit 295 limits the range in which the user U2 can operate the moving object 100 to a predetermined range from the position of the paired user U1. Details of the processing by the limiting unit 295 will be described later.

[0062] [Environmental Target Risk Function and Environmental Target-Based Benefit Function] Next, the environmental target risk function and the environmental target-based benefit function will be described. The environmental target risk function is a function that indicates the degree of risk of interference between the mobile object 100 and an environmental target (e.g., a wall, a lawn, etc.). Conversely, the environmental target-based benefit function is a function that indicates the degree of non-interference between the mobile object 100 and an environmental target.

[0063] 6 is a diagram for explaining the environmental target risk function and the environmental target-based benefit function. As shown in FIG. 6, the traveling direction of the mobile object 100 is the Y axis, and the direction perpendicular to the Y axis is the X axis. Since there are multiple environmental targets R1 to R3 around the mobile object 100, the mobile object 100 needs to travel in a manner that does not interfere with these multiple environmental targets R1 to R3.

[0064] In the example shown in Fig. 6, the second detection unit 230 detects the position of the environmental target R2 based on information input from the detection device 120. Here, the detected position of the environmental target R2 is designated as E_l. In Fig. 6, only one detected position E_l is shown, but in reality, the environmental target R2 is detected at multiple locations.

[0065] In the XY coordinate system defined by the X and Y axes, the coordinates of the detected position E_l are defined as (xe_l, ye_l), where the subscript l represents the number of the detected environmental object, and l=1, 2, ..., n. ｅ Let us say that ｅ represents the total number of detected positions of environmental targets.

[0066] Furthermore, the direction from the center of the moving body 100 toward the detection position E_l is defined as X ｅ_ｌ axis, X ｅ_ｌ The direction perpendicular to the axis is Y ｅ_ｌ The X axis and the X ｅ_ｌ The angle between the axis and the ｅ_ｌ Then, ψ ｅ_ｌ is expressed as the following equation (1).

[0067] ψ ｅ_ｌ =tan －１ (ye_l / xe_l) …(1)

[0068] Furthermore, the coordinates in the XY coordinate system of the predicted position on the trajectory generated by the generation unit 260 are expressed as Z ｍｊ (X ｍｊ , Y ｍｊ The subscript j represents the number of the predicted position on the trajectory generated by the generator 260, where j=1, 2, ..., n ｍ Let us say that ｍ represents the total number of predicted positions on the trajectory generated by the generation unit 260. In the example shown in FIG. 5, five predicted positions (Z ｍ１１ , Z ｍ１２ , Z ｍ１３ (=Z ｍ１ ), Z ｍ２ , Z ｍ３ ) is obtained, so n ｍ =5.

[0069] Furthermore, the coordinate Z of the predicted position is calculated based on the following equations (2) and (3): ｍｊ (X ｍｊ , Y ｍｊ ) to X ｅ_ｌ Axis and Y ｅ_ｌ X axis defined by ｅ_ｌ -Y ｅ_ｌ Coordinate Z in the coordinate system ｍｊ_ｅｌ (X ｍｊ_ｅｌ , Y ｍｊ_ｅｌ) to

[0070] X ｍｊ_ｅｌ = (X ｍｊ -xe_l) cos ψ ｅ_ｌ + (Y ｍｊ -ye_l) sinψ ｅ_ｌ …(2)

[0071] Y ｍｊ_ｅｌ =-(X ｍｊ -xe_l) sinψ ｅ_ｌ + (Y ｍｊ -ye_l) cos ψ ｅ_ｌ …(3)

[0072] FIG. 7 shows the X ｅ_ｌ 7 is a diagram showing an example of an environmental target risk function and an environmental target-based benefit function in the axial direction. As shown in FIG. 7, the first determination unit 220 defines the environmental target-based benefit function Pe_x_b with the detection position E_l of the environmental target R2 as the origin. Also, the second determination unit 240 defines the environmental target-based benefit function Pe_x_b with the detection position E_l of the environmental target R2 as the origin. ｅ_ｌ Define the environmental target risk function Pe_x_r in the axial direction.

[0073] The second determination unit 240 determines X so that it has a gradient in the first sign direction (positive direction). ｅ_ｌ The first determination unit 220 defines an environmental target risk function Pe_x_r in the axial direction, and defines an environmental target-based benefit function Pe_x_b to have a gradient in a second sign direction (negative direction) opposite to the first sign direction (positive direction).

[0074] As shown in FIG. ｅ_ｌ The environmental target risk function Pe_x_r in the axial direction is ｅ_ｌ The larger the coordinate value on the X axis, the larger the risk function value. ｅ_ｌ The axis value is set only in the positive area. ｅ_ｌ The environmental target-based benefit function Pe_x_b in the axial direction is ｅ_ｌ The smaller the coordinate value of the X axis, the smaller the benefit function value.ｅ_ｌ The axis values ​​are set only in the negative range. Note that the larger the risk function value, the higher the risk of the moving object 100 interfering with the interfering object, and the smaller the benefit function value, the lower the risk of the moving object 100 interfering with the interfering object.

[0075] FIG. 8 shows the Y ｅ_ｌ 8 is a diagram illustrating an example of a potential function in the axial direction. As shown in FIG. 8, the potential function Pe_y is a function whose origin is the detected position E_l of the environmental object R2. The second determination unit 240 determines the potential function Pe_y by ｅ_ｌ It is defined as the environmental target risk function in the axial direction.

[0076] Y ｅ_ｌ The environmental target risk function (potential function Pe_y) in the axial direction is expressed as Y ｅ_ｌ The closer the coordinate of the axis is to 0, the larger the risk function value becomes. As shown in FIGS. 7 and 8, ｅ_ｌ The environmental target risk function Pe_x_r in the axial direction and Y ｅ_ｌ The environmental target risk function Pe_y in the axial direction has a different shape from the environmental target risk function Pe_y in the axial direction. ｅ_ｌ The environmental target risk function Pe_x_r in the axial direction is defined as X ｅ_ｌ By defining the risk function value so that it increases as the coordinate value of the axis increases, it is possible to prevent the target trajectory of the moving body 100 from being generated toward a position beyond the environmental target. ｅ_ｌ The environmental target risk function Pe_y in the axial direction is also expressed as X ｅ_ｌ If it is defined as a function similar to the environmental target risk function Pe_x_r in the axial direction, the calculation load on the control device 200 will increase. ｅ_ｌ The environmental object risk function Pe_y in the axial direction is defined so that the position of the origin (i.e., the position where the environmental object R2 is detected) is at its peak.

[0077] Next, a method for calculating the environmental target risk function value will be described. As described above, the coordinate Z of the predicted position on the predicted trajectory in the XY coordinate system is ｍｊ (X ｍｊ , Y ｍｊ ) is Xｅ_ｌ -Y ｅ_ｌ Coordinate Z of the predicted position in the coordinate system ｍｊ_ｅｌ (X ｍｊ_ｅｌ , Y ｍｊ_ｅｌ At the current time k, the X coordinate of the predicted position X ｍｊ_ｅｌ The risk function value based on the above is Pe_x_r_j_l(k), and the Y coordinate of the predicted position Y ｍｊ_ｅｌ The risk function value based on this is defined as Pe_y_j_l(k). Pe_x_r_j_l(k) can be found from Pe_x_r shown in Fig. 7. Pe_y_j_l(k) can be found from Pe_y shown in Fig. 8.

[0078] In this case, the risk function value Pe_r_j_l(k) for each detection position on the environmental target can be expressed as in equation (4). That is, Pe_r_j_l(k) can be obtained by multiplying Pe_x_r_j_l(k) and Pe_y_j_l(k).

[0079] Pe_r_j_l(k)=Pe_x_r_j_l(k)×Pe_y_j_l(k)...(4)

[0080] Furthermore, the total risk function value Pe_r_j(k) for the coordinates of the predicted position on the predicted trajectory can be expressed as in equation (5). That is, the risk function values ​​Pe_r_j_l(k) (l=1, 2, ..., n) for all detected positions on the environmental targets can be expressed as ｅ ) can be used to find Pe_r_j(k).

[0081] Pe_r_j(k) = Σ ｌ＝１ ｎｅ (Pe_r_j_l(k)) …(5)

[0082] The environmental target risk function value Pe_r(k) can be expressed as in equation (6). That is, the total risk function value Pe_r_j(k) (j=1, 2, ..., n) for the coordinates of all predicted positions on the predicted trajectory is ｍ ) can be summed to obtain the environmental target risk function value Pe_r(k).

[0083] Pe_r(k) = Σｊ＝１ ｎｍ (Pe_r_j(k)) …(6)

[0084] Next, a method for calculating the environmental target-based benefit function value will be described. As described in FIG. 7, the first determination unit 220 defines the environmental target-based benefit function Pe_x_b with the detected position E_l of the environmental target R2 as the origin. In addition, the first determination unit 220 calculates the potential function Pe_y shown in FIG. 8 by multiplying Y ｅ_ｌ It is defined as the environmental target-based benefit function in the axial direction.

[0085] As mentioned above, the coordinate Z of the predicted position on the predicted trajectory in the XY coordinate system ｍｊ (X ｍｊ , Y ｍｊ ) is X ｅ_ｌ -Y ｅ_ｌ Coordinate Z of the predicted position in the coordinate system ｍｊ_ｅｌ (X ｍｊ_ｅｌ , Y ｍｊ_ｅｌ At the current time k, the X coordinate of the predicted position X ｍｊ_ｅｌ The benefit function value based on the above is Pe_x_b_j_l(k), and the Y coordinate of the predicted position Y ｍｊ_ｅｌ The benefit function value based on the above is defined as Pe_y_j_l(k). Pe_x_b_j_l(k) can be found from Pe_x_b shown in Fig. 7. Pe_y_j_l(k) can be found from Pe_y shown in Fig. 8.

[0086] In this case, the benefit function value Pe_b_j_l(k) for each detection position on the environmental object can be expressed as in equation (7). That is, Pe_b_j_l(k) can be obtained by multiplying Pe_x_b_j_l(k) and Pe_y_j_l(k).

[0087] Pe_b_j_l(k)=Pe_x_b_j_l(k)×Pe_y_j_l(k)...(7)

[0088] Furthermore, the total benefit function value Pe_b_j(k) for the coordinates of the predicted position on the predicted trajectory can be expressed as in equation (8). That is, the benefit function values ​​Pe_b_j_l(k) (l=1, 2, ..., n) for all detected positions on the environmental targets can be expressed as ｅ ) can be obtained as Pe_b_j(k).

[0089] Pe_b_j(k)=max Pe_b_j_l(k) =max Pe_x_b_j_l(k)×Pe_y_j_l(k)...(8)

[0090] 9 is an image diagram of the environmental target-based benefit function when environmental targets are detected at multiple detection positions. As shown in Fig. 9, for example, assume that there are walls on both the left and right sides, and environmental targets are detected on the right and left sides of the traveling direction of the mobile body 100. Furthermore, the environmental target-based benefit function set based on the detection position on the left side of the traveling direction of the mobile body 100 is denoted as Pe_x_b_1, and the environmental target-based benefit function set based on the detection position on the right side of the traveling direction of the mobile body 100 is denoted as Pe_x_b_2.

[0091] In this case, the first determination unit 220 determines the maximum value of the multiple benefit function values ​​(Pe_x_b_1 and Pe_x_b_2) as X ｅ_ｌ The environmental target-based benefit function value in the axial direction is set as Pe_b_j(k). The maximum value is selected because the benefit function should be set taking into consideration the balance with all environmental targets. For this reason, in the above-mentioned equation (8), Pe_b_j(k) is calculated taking into consideration the maximum value.

[0092] The environmental object-based benefit function value Pe_b(k) can be expressed as in equation (9). That is, the total benefit function value Pe_b_j(k) (j=1, 2, ..., n) for the coordinates of all predicted positions on the predicted trajectory is ｍ ) to obtain the environmental object-based benefit function value Pe_b(k).

[0093] Pe_b(k) = Σ ｊ＝１ ｎｍ (Pe_b_j(k)) …(9)

[0094] [Traffic Participant / Obstacle Risk Function] Next, the traffic participant / obstacle risk function will be described. The traffic participant / obstacle risk function is a function that indicates the degree of risk of interference between the mobile object 100 and a traffic participant or an obstacle.

[0095] 10 is a diagram for explaining the traffic participant / obstacle risk function. As shown in FIG. 10, the traveling direction of the mobile object 100 is the Y axis, and the direction perpendicular to the Y axis is the X axis. Since multiple traffic participants U3 and U4 exist around the mobile object 100, the mobile object 100 needs to travel in a manner that does not interfere with these multiple traffic participants U3 and U4.

[0096] 10 , the second detection unit 230 detects the positions of traffic participants U3 and U4 based on information input from the detection device 120. In addition, the second determination unit 240 sets a traffic participant / obstacle risk function to the positions of traffic participants U3 and U4 detected by the second detection unit 230.

[0097] FIG. 11 is a diagram for explaining the traffic participant / obstacle risk function based on the traffic participant U3. The following describes the process of determining the traffic participant / obstacle risk function based on the traffic participant U3 by the second determination unit 240. In FIG. 11, the detected position of the traffic participant U3 is designated as TP_l. Although FIG. 11 only shows the detected position TP_l of the traffic participant U3, in reality, the traffic participant U4 is also detected. Note that if there are obstacles on the road in addition to the traffic participants, the second detection unit 230 will also detect the obstacles.

[0098] In the XY coordinate system defined by the X and Y axes, the coordinates of the detected position TP_l are defined as (xtp_l, ytp_l), where the subscript l represents the number of the detected position of the traffic participant / obstacle, and l = 1, 2, ..., n. ｔ Let us say that ｔrepresents the total number of detected positions of traffic participants and obstacles. The traveling direction of traffic participant U3 is the Ytp_l axis, and the direction perpendicular to the Ytp_l axis is the Xtp_l axis.

[0099] Furthermore, the coordinates in the XY coordinate system of the predicted position on the trajectory generated by the generation unit 260 are expressed as Z ｍｊ (X ｍｊ , Y ｍｊ ) is defined as follows. Using the same method as in the above equations (2) and (3), the coordinate Z of the predicted position is calculated. ｍｊ (X ｍｊ , Y ｍｊ ) is X ｔｐ_ｌ Axis and Y ｔｐ_ｌ X axis defined by ｔｐ_ｌ -Y ｔｐ_ｌ Coordinate Z in the coordinate system ｍｊ_ｔｐｌ (X ｍｊ_ｔｐｌ , Y ｍｊ_ｔｐｌ ) is converted to

[0100] FIG. 12 shows the X ｔｐ_ｌ 12 is a diagram illustrating an example of a traffic participant / obstacle risk function in the axial direction. As shown in FIG. 12, the second determination unit 240 determines the detected position TP_l of the traffic participant U3 as the origin, and calculates the X ｔｐ_ｌ The traffic participant / obstacle risk function Ptp_x in the axial direction is defined. The second determination unit 240 determines X so that it has a gradient in the first sign direction (positive direction). ｔｐ_ｌ Define a traffic participant / obstacle risk function Ptp_x in the axial direction.

[0101] As shown in FIG. ｔｐ_ｌ The risk function Ptp_x for traffic participants and obstacles in the axial direction is ｔｐ_ｌ The closer the axis coordinate is to 0, the larger the risk function value becomes.

[0102] FIG. 13 shows the Y ｔｐ_ｌ 13 is a diagram illustrating an example of a traffic participant / obstacle risk function in the axial direction. As shown in FIG. 13, the second determination unit 240 determines the detected position TP_l of the traffic participant U3 as the origin, and calculates the Y ｔｐ_ｌ The traffic participant / obstacle risk function Ptp_y in the axial direction is defined. The second determination unit 240 determines Y so that it has a gradient in the first sign direction (positive direction). ｔｐ_ｌDefine a traffic participant / obstacle risk function Ptp_y in the axial direction.

[0103] As shown in FIG. ｔｐ_ｌ The risk function Ptp_y for traffic participants and obstacles in the axial direction is ｔｐ_ｌ The closer the axis coordinate is to 0, the larger the risk function value becomes.

[0104] Next, we will explain how to calculate the traffic participant / obstacle risk function value. As mentioned above, the coordinate Z of the predicted position on the predicted trajectory in the XY coordinate system is ｍｊ (X ｍｊ , Y ｍｊ ) is X ｔｐ_ｌ -Y ｔｐ_ｌ Coordinate Z of the predicted position in the coordinate system ｍｊ_ｔｐｌ (X ｍｊ_ｔｐｌ , Y ｍｊ_ｔｐｌ At the current time k, the X coordinate of the predicted position X ｍｊ_ｔｐｌ The risk function value based on the above is Ptp_x_j_l(k), and the Y coordinate of the predicted position is Y ｍｊ_ｔｐｌ The risk function value based on this is defined as Ptp_y_j_l(k). Ptp_x_j_l(k) can be found from Ptp_x shown in Fig. 12. Ptp_y_j_l(k) can be found from Ptp_y shown in Fig. 13.

[0105] In this case, the risk function value Ptp_r_j_l(k) for each detected position of a traffic participant / obstacle can be expressed as in equation (10). That is, Ptp_r_j_l(k) can be obtained by multiplying Ptp_x_j_l(k) and Ptp_y_j_l(k).

[0106] Ptp_r_j_l(k)=Ptp_x_j_l(k)×Ptp_y_j_l(k)...(10)

[0107] The total risk function value Ptp_r_j(k) for the coordinates of the predicted position on the predicted trajectory can be expressed as in equation (11). That is, the risk function values ​​Ptp_r_j_l(k) (l=1, 2, ..., n) for all detected positions of traffic participants and obstacles can be expressed as ｔ) can be used to find Ptp_r_j(k).

[0108] Ptp_r_j(k)=Σ ｌ＝１ ｎｔ (Ptp_r_j_l(k)) …(11)

[0109] The traffic participant / obstacle risk function value Ptp_r(k) can be expressed as in equation (12). That is, the total risk function value Ptp_r_j(k) (j=1, 2, ..., n) for the coordinates of all predicted positions on the predicted trajectory is ｍ ) can be used to calculate the traffic participant / obstacle risk function value Ptp_r(k).

[0110] Ptp_r(k) = Σ ｊ＝１ ｎｍ (Ptp_r_j(k)) …(12)

[0111] [Follow-up Mode Benefit Function] Next, the follow-up mode benefit function will be described. The follow-up mode benefit function is a function that indicates the degree to which it is recommended that the mobile object 100 should travel in the follow-up mode, and is determined based on the current and past positions of the user U1 (the person to be followed).

[0112] 14 is a diagram for explaining the following mode benefit function. As shown in FIG. 14, the traveling direction of the moving body 100 is the Y axis, and the direction perpendicular to the Y axis is the X axis. Since multiple environmental targets R4 and R5 exist around the moving body 100, the moving body 100 needs to follow the user U1 without interfering with these multiple environmental targets R4 and R5.

[0113] The first detection unit 210 detects the position Vu_0(k) of the user U1 at the current time k based on the detection result of the detection device 120. The first detection unit 210 repeatedly detects the position Vu_0(k) of the user U1 at every preset control time. The position Vu_0(k) of the user U1 is detected as the relative position of the user U1 with respect to the moving body 100 (position in the X-Y coordinate system). In FIG. 14, Vu_1(k) indicates the position of the user U1 one control time before Vu_0(k), and Vu_2(k) indicates the position of the user U1 one control time before Vu_1(k). Vu_n ｕ (k) indicates the most recent position of user U1.

[0114] The first detector 210 detects the current and past positions Vu_0(k), Vu_1(k), Vu_2(k), ..., Vu_n of the user U1. ｕ At this time, since the moving object 100 is actually moving, the previously detected position of the user U1 needs to be corrected according to the amount of movement of the moving object 100. This point will be described below.

[0115] FIG. 15 is a diagram showing an example of the movement amount of the moving body 100. In FIG. ０ indicates the position of the moving object 100 one control time before. ｍｖ is the time when the moving object 100 is at position Z ０ Δy indicates the distance traveled in the X direction by the moving body 100 when moving from the current position (the origin position in the XY coordinate system). ｍｖ is the time when the moving object 100 is at position Z ０ The control device 200 calculates the distance Δx in one control time based on the detection result of the sensor 150. ｍｖ and Δy ｍｖ Calculate.

[0116] Here, the position Vu_0(k) of the user U1 at the current time k is defined as in equation (13).

[0117] Vu_0(k) = [xu_0(k) yu_0(k)] …(13)

[0118] In this case, the control device 200 calculates the movement trajectories Vu_1(k), Vu_2(k), ..., Vu_m of the user U1. ｕ (k) is the movement amount Δx of the moving body 100 ｍｖ and Δy ｍｖ Based on this, the calculation is performed according to the following formulas (14) to (16). ｕ is n ｕ The following integers:

[0119] Vu_1(k) = [xu_1(k) yu_1(k)] = [xu_0(k-1)-Δx ｍｖ (k) yu_0(k-1)-Δy ｍｖ (k)] …(14)

[0120] Vu_2(k) = [xu_2(k) yu_2(k)] = [xu_1(k-1)-Δx ｍｖ (k) yu_1(k-1)-Δy ｍｖ (k)] …(15)

[0121] Vu_m ｕ (k) = [xu_m ｕ (k) yū_m ｕ (k)] = [xu_m ｕ −1(k−1)−Δx ｍｖ (k) yū_m ｕ −1(k−1)−Δy ｍｖ (k)] …(16)

[0122] FIG. 16 is a diagram illustrating the field of view of a user (a person to be followed). When the moving object 100 follows the user U1, it is preferable that the moving object 100 travels while remaining within the field of view of the user U1. This is to allow the user U1 to easily confirm the position of the moving object 100 without turning around. Therefore, the first determination unit 220 determines the following mode benefit function based on the position of the user U1 and information related to the field of view of the user U1. The information related to the field of view of the user U1 may be, for example, information related to the field of view of the user U1.

[0123] For example, the first determination unit 220 determines the relative position that the moving body 100 should follow with respect to the user U1 based on the position of the user U1 and information about the user U1's field of view, and sets the following mode benefit function. Specifically, the first determination unit 220 calculates an offset amount Δxt in the X-axis direction and an offset amount Δyt in the Y-axis direction based on the information about the user U1's field of view. Then, the first determination unit 220 calculates a position that is offset by Δxt in the X-axis direction and Δyt in the Y-axis direction from the position Vu_0(k) of the user U1 at the current time k, as the target position Vt_0(k).

[0124] 17 is a diagram illustrating an example of a plurality of target positions. As shown in FIG. 17, the first determination unit 220 determines the past positions Vu_0(k), ..., Vu_n of the user U1 from the present. ｕ (k), a plurality of target positions Vt_0(k), ..., Vt_n ｕ (k) is calculated from a plurality of target positions Vt_0(k), ..., Vt_n. ｕ (k) represents the past positions Vu_0(k), ..., Vu_n of the user U1 from the present ｕ This is a position offset by Δxt in the X-axis direction and Δyt in the Y-axis direction from (k).

[0125] 18 is a diagram showing an example of a state in which the tracking mode benefit functions are set for a plurality of target positions. As shown in FIG. 18, the first determination unit 220 calculates the tracking mode benefit functions for the plurality of target positions Vt_0(k), ..., Vt_n ｕ A tracking mode benefit function is set for each position of the target positions Vt_0(k), ..., Vt_n. ｕ It is recommended to drive around (k).

[0126] As described above, the coordinates of the predicted position on the trajectory generated by the generation unit 260 are Z ｍｊ (X ｍｊ , Y ｍｊ ) and the coordinates of each of the multiple target positions and the coordinate Z of the predicted position. ｍｊ (X ｍｊ , Yｍｊ ) and the benefit function value is calculated based on the above.

[0127] 19 is a diagram showing an example of a tracking mode benefit function in the X-axis direction. The horizontal axis Δxu_b in FIG. 19 represents the X coordinate of the target position and the X coordinate X of the predicted position on the trajectory generated by the generation unit 260. ｍｊ The first determination unit 220 defines the following mode benefit function Pc_b_x in the X-axis direction so that it has a gradient in the second sign direction (negative direction).

[0128] As shown in FIG. 19, the following mode benefit function Pc_b_x in the X-axis direction is a function whose benefit function value decreases as Δxu_b approaches 0.

[0129] 20 is a diagram showing an example of a tracking mode benefit function in the Y-axis direction. The horizontal axis Δyu_b in FIG. 20 represents the Y coordinate of the target position and the Y coordinate Y of the predicted position on the trajectory generated by the generation unit 260. ｍｊ The first determination unit 220 defines the following mode benefit function Pc_b_y in the Y-axis direction so that it has a gradient in the second sign direction (negative direction).

[0130] As shown in FIG. 20, the following mode benefit function Pc_b_y in the Y-axis direction is a function whose benefit function value decreases as Δyu_b approaches 0.

[0131] Next, a method for calculating the follow-up mode benefit function value will be described. First, a plurality of target positions Vt_l′ (l′=0, 1, . . . , n) at the current time k are calculated. ｕ ) are calculated based on equations (17) to (19).

[0132] Vt_0(k) = [xt_0(k) yt_0(k)] = [xu_0(k)+Δxt yu_0(k)+Δyt]…(17)

[0133] Vt_1(k) = [xt_1(k) yt_1(k)] = [xu_1(k)+Δxt yu_1(k)+Δyt]…(18)

[0134] Vt_n ｕ (k) = [xt_n ｕ (k) yt_n ｕ (k)] = [xu_n ｕ (k)+Δxt yu_n ｕ (k)+Δyt]…(19)

[0135] Next, the coordinates Z of the predicted positions on the trajectory generated by the generation unit 260 based on the plurality of target positions Vt_l′ are calculated. ｍｊ (X ｍｊ , Y ｍｊ ) (j=1, 2,..., n ｍ ) to calculate the benefit function value.

[0136] Specifically, the X coordinate of the predicted position on the trajectory generated by the generating unit 260 is expressed as X ｍｊ (k) and the X coordinate of the target position is xt_l'(k), the difference Δxu_b_l'(k) therebetween is expressed as in equation (20).

[0137] Δxu_b_l'(k)=X ｍｊ (k)-xt_l'(k)...(20)

[0138] The first determination unit 220 calculates Δxu_b_l′(k) based on equation (20). Furthermore, the first determination unit 220 obtains the benefit function value Pc_b_x_j_l′(k) corresponding to the calculated Δxu_b_l′(k) from the following mode benefit function in the X-axis direction shown in FIG.

[0139] Also, the Y coordinate of the predicted position on the trajectory generated by the generation unit 260 is expressed as Y ｍｊ (k) and the Y coordinate of the target position is yt_l'(k), the difference Δyu_b_l'(k) therebetween is expressed as in equation (21).

[0140] Δyu_b_l'(k)=Yｍｊ (k)-yt_l'(k)...(21)

[0141] The control device 200 calculates Δyu_b_l′(k) based on equation (21). The control device 200 also obtains the benefit function value Pc_b_y_j_l′(k) corresponding to the calculated Δyu_b_l′(k) from the following mode benefit function in the Y-axis direction shown in FIG.

[0142] Furthermore, the benefit function value Pc_b_j_l'(k) for each of the multiple target positions can be expressed as in equation (22). That is, Pc_b_j_l'(k) can be obtained by multiplying Pc_b_x_j_l'(k) and Pc_b_y_j_l'(k). Note that the benefit function value Pc_b_j_l'(k) may be signed so that it becomes a negative value.

[0143] Pc_b_j_l'(k)=Pc_b_x_j_l'(k)×Pc_b_y_j_l'(k)...(22)

[0144] Furthermore, the total benefit function value Pc_b_j(k) for the coordinates of the predicted positions on the predicted trajectory can be expressed as in equation (23). That is, the benefit function values ​​Pc_b_j_l′(k) (l′=0, 1, ..., n) for all target positions can be expressed as ｕ ) can be used to find Pc_b_j(k).

[0145] Pc_b_j(k) = Σ ｌ’＝０ ｎｕ (Pc_b_j_l'(k)) ...(23)

[0146] The following mode benefit function value Pc_b(k) can be expressed as in equation (24). That is, the total benefit function value Pc_b_j(k) (j=1, 2, ..., n) for the coordinates of all predicted positions on the predicted trajectory is expressed as ｍ ) to obtain the follow-up mode benefit function value Pc_b(k).

[0147] Pc_b(k) = Σ ｊ＝１ｎｍ (Pc_b_j(k)) …(24)

[0148] [Leading Mode Benefit Function] Next, the leading mode benefit function will be described. The leading mode benefit function is a function that indicates the degree to which it is recommended that the moving body 100 should travel in the leading mode, and is determined based on the current and past positions of the user U1 (the target person to lead).

[0149] 21 is a diagram for explaining the leading mode benefit function. As shown in FIG. 21, the traveling direction of the moving body 100 is the Y axis, and the direction perpendicular to the Y axis is the X axis. Because multiple environmental targets R6 and R7 exist around the moving body 100, the moving body 100 needs to lead the user U1 so as not to interfere with these multiple environmental targets R6 and R7.

[0150] In the leading mode, the mobile body 100 may lead a weak gaiter, such as an elderly person or a person with a leg injury, as the user U1. These weak gaiters may have difficulty moving quickly from side to side and may have difficulty passing through crowds. To enable such weak gaiters to walk through crowds safely, the mobile body 100 leads the weak gaiter as the user U1, thereby ensuring a smooth walking space for the weak gaiter with minimal side-to-side movement. The mobile body 100 also estimates the direction the weak gaiter wants to go from the movement trajectory and leads the weak gaiter in the estimated direction. Note that the mobile body 100 is not limited to a weak gaiter, but may also lead a pedestrian who is unsure of the way to their destination in a specific facility (such as a station, airport, or shopping mall).

[0151] The first detection unit 210 detects the position Vu_0(k) of the user U1 at the current time k based on the detection result of the detection device 120. The first detection unit 210 repeatedly detects the position Vu_0(k) of the user U1 at every preset control time. The position Vu_0(k) of the user U1 is detected as the relative position of the user U1 with respect to the moving object 100 (position in the X-Y coordinate system). In FIG. 21, Vu_1(k) indicates the position of the user U1 one control time before Vu_0(k), and Vu_2(k) indicates the position of the user U1 one control time before Vu_1(k). Vu_n ｕ (k) indicates the most recent position of user U1.

[0152] The first detector 210 detects the current and past positions Vu_0(k), Vu_1(k), Vu_2(k), ..., Vu_n of the user U1. ｕ At this time, since the moving body 100 is also actually moving, the past detected position of the user U1 needs to be corrected according to the amount of movement of the moving body 100. The method for correcting the past detected position of the user U1 is the same as the method described in FIG. 15 and equations (13) to (16), and therefore the description thereof will be omitted.

[0153] 22 is a diagram for explaining the process of estimating a predicted trajectory of the user U1. The estimation unit 225 estimates the past positions Vu_0(k), Vu_1(k), ..., Vu_n of the user U1 from the present to the past. ｕ The estimation unit 225 estimates the predicted trajectory of the user U1 based on (k). The ideal route for the moving body 100 to travel while leading is a curved route with minimal lateral movement. Therefore, the estimation unit 225 estimates the predicted trajectory of the user U1 using a model that represents a portion of the periphery of a geometric shape based on the current and past positions of the user U1. Specifically, the estimation unit 225 may estimate a circularly arced predicted trajectory using an arc model. For example, the estimation unit 225 may estimate the predicted trajectory of the user U1 in the leading mode by combining three arcs, as described above with reference to FIG. 5 .

[0154] 22, the estimation unit 225 estimates the predicted trajectory of the user U1 by combining first to third arc-shaped predicted trajectories. The estimation unit 225 determines the first to third predicted trajectories by appropriately setting the radius of curvature ρm1 of the first predicted trajectory, the radius of curvature ρm2 of the second predicted trajectory, and the radius of curvature ρm3 of the third predicted trajectory.

[0155] Specifically, the estimation unit 225 determines the first predicted trajectory so as to reduce the deviation from the past movement trajectory of the user U1 and reduce the risk of the moving body 100 interfering with surrounding traffic participants U5 to U7, etc. The estimation unit 225 also determines the second predicted trajectory and the third predicted trajectory so as to reduce the risk of the moving body 100 interfering with surrounding traffic participants U5 to U7, etc. For example, the estimation unit 225 may determine the predicted trajectory by determining the curvature or radius of curvature of the predicted trajectory so as to change the degree of risk obtained by a risk function set to have a gradient in a first sign direction (positive direction) at the positions of the traffic participants U5 to U7, in a second sign direction (negative direction).

[0156] 22, the estimation unit 225 calculates the end of the third predicted trajectory as the target position Vt_0(k) at the current time k. The estimation unit 225 also calculates the target positions Vt_j(k) (j=1, 2, ..., m) taking into account the movement speeds and control periods of the traffic participants U5 to U7. ｕ ) along the first predicted orbit, the second predicted orbit, and the third predicted orbit.

[0157] 23 is a diagram showing an example of a state in which leading mode benefit functions are set for a plurality of target positions. As shown in FIG. 23, the first determination unit 220 determines the leading mode benefit functions for a plurality of target positions Vt_0(k), ..., Vt_m on the predicted trajectory estimated by the estimation unit 225. ｕ A leading mode benefit function is set at each position of the target positions Vt_0(k), ..., Vt_m. ｕ It is recommended to drive around (k).

[0158] As described above, the coordinates of the predicted position on the trajectory generated by the generation unit 260 are Z ｍｊ (X ｍｊ , Y ｍｊ ) and the coordinates of each of the multiple target positions and the coordinate Z of the predicted position. ｍｊ (X ｍｊ , Y ｍｊ ) and the benefit function value is calculated based on the above.

[0159] 24 is a diagram showing an example of the leading mode benefit function in the X-axis direction. The horizontal axis Δxu_b in FIG. 24 represents the X coordinate of the target position and the X coordinate X of the predicted position on the trajectory generated by the generation unit 260. ｍｊ The first determination unit 220 defines the leading mode benefit function Pl_b_x in the X-axis direction so that it has a gradient in the second sign direction (negative direction).

[0160] As shown in FIG. 24, the leading mode benefit function Pl_b_x in the X-axis direction is a function whose benefit function value decreases as Δxu_b approaches 0.

[0161] 25 is a diagram showing an example of the leading mode benefit function in the Y-axis direction. The horizontal axis Δyu_b in FIG. 25 represents the Y coordinate of the target position and the Y coordinate Y of the predicted position on the trajectory generated by the generation unit 260. ｍｊ The first determination unit 220 defines the leading mode benefit function Pl_b_y in the Y-axis direction so that it has a gradient in the second sign direction (negative direction).

[0162] As shown in FIG. 25, the leading mode benefit function Pl_b_y in the Y-axis direction is a function whose benefit function value decreases as Δyu_b approaches 0.

[0163] Next, a method for calculating the leading mode benefit function value will be described. First, a plurality of target positions Vt_l′ (l′=0, 1, ..., m) at the current time k are calculated. ｕ ) are defined as in equation (25).

[0164] Vt_l'(k)=[xt_l'(k) yt_l'(k)]...(25)

[0165] Next, the coordinates Z of the predicted positions on the trajectory generated by the generation unit 260 are calculated based on the multiple target positions Vt_l′(k). ｍｊ (X ｍｊ , Y ｍｊ ) (j=1, 2,..., n ｍ ) to calculate the benefit function value.

[0166] Specifically, the X coordinate X of the predicted position on the trajectory generated by the generation unit 260 ｍｊ The difference Δxu_b_l′(k) between the X coordinate xt_l′(k) of the target position and the X coordinate xt_l′(k) of the target position is expressed as in equation (26).

[0167] Δxu_b_l'(k)=X ｍｊ (k)-xt_l'(k)...(26)

[0168] The first determination unit 220 calculates Δxu_b_l′(k) based on equation (26). Furthermore, the first determination unit 220 obtains a benefit function value Pl_b_x_j_l′(k) corresponding to the calculated Δxu_b_l′(k) from the leading mode benefit function in the X-axis direction shown in FIG.

[0169] Also, the Y coordinate Y of the predicted position on the trajectory generated by the generation unit 260 ｍｊ The difference Δyu_b_l′(k) between the Y coordinate yt_l′(k) of the target position and the Y coordinate yt_l′(k) of the target position is expressed as in equation (27).

[0170] Δyu_b_l'(k)=Y ｍｊ (k)-yt_l'(k)...(27)

[0171] The first determination unit 220 calculates Δyu_b_l′(k) based on equation (27). Furthermore, the first determination unit 220 obtains a benefit function value Pl_b_y_j_l′(k) corresponding to the calculated Δyu_b_l′(k) from the leading mode benefit function in the Y-axis direction shown in FIG.

[0172] Furthermore, the benefit function value Pl_b_j_l'(k) for each of the multiple target positions can be expressed as in equation (28). That is, Pl_b_j_l'(k) can be obtained by multiplying Pl_b_x_j_l'(k) and Pl_b_y_j_l'(k). Note that the benefit function value Pl_b_j_l'(k) may be signed so that it becomes a negative value.

[0173] Pl_b_j_l'(k)=Pl_b_x_j_l'(k)×Pl_b_y_j_l'(k)...(28)

[0174] The total benefit function value Pl_b_j(k) for the coordinates of the predicted position on the predicted trajectory can be expressed as in equation (29). That is, the benefit function values ​​Pl_b_j_l'(k) (l'=0, 1, ..., m) for all target positions can be expressed as ｕ ) to obtain Pl_b_j(k).

[0175] Pl_b_j(k) = Σ ｌ’＝０ ｍｕ (Pl_b_j_l'(k)) ...(29)

[0176] The leading mode benefit function value Pl_b(k) can be expressed as in equation (30). That is, the total benefit function value Pl_b_j(k) (j=1, 2, ..., n) for the coordinates of all predicted positions on the predicted trajectory is expressed as ｍ ) to obtain the leading mode benefit function value Pl_b(k).

[0177] Pl_b(k) = Σ ｊ＝１ ｎｍ (Pl_b_j(k)) …(30)

[0178] [Remote Operation Benefit Function] Next, the remote operation benefit function will be described. The remote operation benefit function is a function that indicates the degree to which it is recommended to run the mobile body 100, and is determined based on the operation of the mobile body 100 by the user U2. The remote operation benefit function is also a function that is set in the remote operation mode.

[0179] 26 is a diagram illustrating the state in which the mobile object 100 is automatically traveling in the remote control mode. As shown in FIG. 26, the direction of travel of the mobile object 100 is the Y axis, and the direction perpendicular to the Y axis is the X axis. Since there are multiple environmental landmarks R8 to R10 and a traffic participant U8 (hereinafter referred to as pedestrian U8) around the mobile object 100, the mobile object 100 must travel in a manner that does not interfere with these multiple environmental landmarks R8 to R10 and pedestrian U8.

[0180] In the remote control mode, the user U2 can remotely control the mobile object 100 using the information processing device 400. Specifically, the information processing device 400 transmits operation information based on the operation of the user U2 to the mobile object 100, and the mobile object 100 travels in accordance with the operation information received from the information processing device 400. Note that if a destination has been set in advance, the mobile object 100 will automatically travel toward the destination while accepting remote control by the user U2.

[0181] When the operation mode of the mobile object 100 is switched to the remote control mode, the first determination unit 220 determines whether a destination has been set in advance. For example, the user U1 may set the destination using the user terminal device 300, or the user U2 may set the destination using the information processing device 400. When the first determination unit 220 determines that the destination has been set in advance, it determines a rough route to the destination based on the destination and map information, and sets an environmental landmark-based benefit function along the route. On the other hand, the first determination unit 220 does not set an environmental landmark-based benefit function for areas other than the route.

[0182] 26 , the first determination unit 220 does not set an environmental landmark-based benefit function for the space S between environmental landmarks R8 and R9. If an environmental landmark-based benefit function were set for the space S, the mobile object 100 would be recommended to travel through the location where the benefit function is set, which could result in the mobile object 100 traveling straight ahead instead of heading toward the destination. In this way, the first determination unit 220 determines a rough route to the destination based on the destination information and map information, and sets an environmental landmark-based benefit function along that route, thereby more reliably guiding the mobile object 100 to the destination.

[0183] In the example shown in Figure 26, the mobile body 100 travels along a route A1 corresponding to the position where the environmental target benefit function is set, while avoiding interfering objects (e.g., pedestrian U8 and environmental targets R8 to R10) without any operation by user U2.

[0184] In the remote operation mode, the control device 200 controls the direction of travel of the mobile object 100 in response to an operation instruction from the user U2 using a joystick or the like of the information processing device 400. However, if the user U2 were to operate the mobile object 100 by completely manual remote operation, this would be stressful for the user U2 due to communication delays between the mobile object 100 and the information processing device 400. Therefore, in this embodiment, the mobile object 100 basically performs automatic travel, and when an operation instruction is received from the user U2, it changes its course left or right in response to the operation instruction. This reduces the stress of the user U2 when operating the mobile object 100.

[0185] FIG. 27 is a diagram illustrating a situation where user U2 instructs the moving body 100 to move leftward. In the example illustrated in FIG. 27 , when the moving body 100 travels along route A1, the distance between the moving body 100 and pedestrian U8 is short. Therefore, moving the moving body 100 to the left more reliably avoids the pedestrian U8. Therefore, when user U2 instructs the moving body 100 to move leftward using the information processing device 400, the first determination unit 220 sets the remote control benefit function in an area to the left of the moving body 100's direction of travel (the negative side of the X-axis). Since it is recommended that the moving body 100 travel near the location where the benefit function is set, the course of the moving body 100 is changed from route A1 to route A2. This allows the moving body 100 to travel to its destination while more reliably avoiding the pedestrian U8.

[0186] Next, a method for calculating the teleoperation benefit function value will be described. As described above, the coordinates of the predicted position on the predicted trajectory in the XY coordinate system are Z ｍｊ (X ｍｊ , Y ｍｊ ) and the parameter of the operation information based on the operation of the user U2 on the information processing device 400 is expressed as xop_j (j=1, 2, . . . , n ｍ For example, the parameter xop_j may be calculated based on the amount of operation (stroke) of a joystick or the like by the user U2, the number of operations in the operation direction, the operation time in the operation direction, etc. Note that the right direction with respect to the traveling direction of the moving body 100 is set to the positive direction of the parameter xop_j, and the left direction with respect to the traveling direction of the moving body 100 is set to the negative direction of the parameter xop_j.

[0187] The first detection unit 210 calculates a parameter xop_j based on operation information based on an operation of the user U2 on a joystick or the like, and outputs the parameter xop_j to the first determination unit 220. This allows the first detection unit 210 to detect an operation of the moving object 100 by the user U2. The first determination unit 220 also calculates a parameter xop_j based on the parameter xop_j and the X coordinate X ｍｊ The benefit function value is calculated based on the above. This point will be explained below.

[0188] 28 is a diagram illustrating an example of a remote control benefit function. As illustrated in FIG. 28, the first determination unit 220 determines the remote control benefit function Pe_b_op_j (j=1, 2, ..., n) so that the function has a gradient in the second sign direction (negative direction). ｍ The remote operation benefit function Pe_b_op_j is a function whose benefit function value decreases as Δxop_j approaches 0.

[0189] Here, Δxop_j is the sum of the parameter xop_j and the X coordinate X of the predicted position on the trajectory generated by the generation unit 260. ｍｊ Therefore, the difference Δxop_j(k) at the current time k is expressed as in equation (31).

[0190] Δxop_j(k)=X ｍｊ (k)-xop_j(k)...(31)

[0191] The first determination unit 220 calculates Δxop_j(k) based on equation (31). Furthermore, the first determination unit 220 obtains a benefit function value Pe_b_op_j(k) corresponding to the calculated Δxop_j(k) from the remote control benefit function shown in FIG.

[0192] The remote control benefit function value Pe_b_op(k) can be expressed as in equation (32). That is, the benefit function values ​​Pe_b_op_j(k) (j=1, 2, ..., n) for the coordinates of all predicted positions on the predicted trajectory are expressed as ｍ ) can be summed to obtain the teleoperation benefit function value Pe_b_op(k).

[0193] Pe_b_op(k)=Σ ｊ＝１ ｎｍ (Pe_b_op_j(k)) …(32)

[0194] 29 is a diagram illustrating how the traveling direction of the moving body 100 is changed when a remote control benefit function is set. As shown in Fig. 29, for example, when a user U2 uses the information processing device 400 to instruct the moving body 100 to move rightward, the remote control benefit function is set to the right of the traveling direction of the moving body 100 (the positive direction on the X axis).

[0195] Furthermore, since it is recommended that the moving body 100 travel to the position where the remote-control benefit function value is smallest (the position where Δxop_j = 0), the moving body 100 changes its direction of travel to the right. In this way, the first determination unit 220 determines the remote-control benefit function, so that the moving body 100 can change its direction of travel in response to the operation of the user U2.

[0196] [Example of Mode Change] Next, as an example of mode change, an example will be described in which user U1 and user U2 take a walk on the beach together. Note that while user U1 actually takes a walk on the beach, user U2 logs in to the mobile object 100 and takes a walk on the beach.

[0197] First, the pairing unit 290 of the mobile object 100 pairs the user U1 with the mobile object 100 in response to an operation by the user U1. For example, the pairing unit 290 may pair the user U1, whose biometric authentication has been performed using a biometric authentication function, with the mobile object 100. Specifically, the pairing unit 290 may identify the user U1 by performing face authentication based on a facial image of the user captured by the camera 80, and pair the identified user U1 with the mobile object 100. Note that the biometric authentication method is not limited to this. For example, the mobile object 100 may be equipped with a vein sensor, and the pairing unit 290 may perform vein authentication based on output from the vein sensor. Furthermore, the pairing unit 290 may identify the user U1 through communication with the user terminal device 300 and pair the identified user U1 with the mobile object 100. The user U1 paired with the mobile object 100 becomes a follower in the follow mode and a leader in the lead mode.

[0198] Next, the mode switching unit 285 of the moving body 100 sets the following mode or the leading mode in response to the operation of the user U1. When the following mode is set, the moving body 100 follows the user U1. When the leading mode is set, the moving body 100 leads the user U1. Thereafter, the moving body 100 travels with the user U1 to the beach where the user U2 plans to take a walk.

[0199] Next, the acquisition unit 280 of the mobile object 100 acquires a remote control request to remotely operate the mobile object 100 from the information processing device 400. For example, the acquisition unit 280 may acquire a login request transmitted from the information processing device 400 as the remote control request. When the acquisition unit 280 acquires the login request from the information processing device 400, a login process for user U2 is performed. For example, the login request may include the ID and password of user U2, and authentication of user U2 is performed based on this information. When authentication of user U2 is complete, the mode switching unit 285 switches the operation mode of the mobile object 100 to the remote control mode.

[0200] In this way, the mode switching unit 285 switches the operation mode of the mobile object 100 to the remote operation mode in response to receiving a remote operation request (e.g., a login request) while the mobile object 100 is operating in the following mode or the leading mode, thereby enabling the user U2 to remotely operate the mobile object 100.

[0201] When the operation mode of the mobile object 100 is switched to the remote control mode, an image of the user U2 is displayed on the display unit of the HMI 110, and the voice of the user U2 is output from the speaker of the HMI 110. For example, the information processing device 400 may photograph the user U2 using a camera provided in the information processing device 400 and transmit the photographed image of the user U2 to the mobile object 100. The display unit of the HMI 110 may display the image of the user U2 transmitted from the information processing device 400. The information processing device 400 may acquire voice data of the user U2 using a microphone provided in the information processing device 400 and transmit the acquired voice data of the user U2 to the mobile object 100. The speaker of the HMI 110 may output the voice of the user U2 based on the voice data transmitted from the information processing device 400. Similarly, the mobile object 100 may transmit an image and voice data of the user U1 to the information processing device 400. The information processing device 400 may display the image of the user U1 transmitted from the mobile object 100 on a display unit, and may output the voice of the user U1 from a speaker based on the voice data transmitted from the mobile object 100. This allows the user U1 to converse with the user U2.

[0202] In the remote control mode, a destination such as a seaside view spot may be set in advance. When a destination is set, the mobile object 100 basically travels automatically to the destination, but the user U2 can operate the mobile object 100 using a joystick or the like. However, the limiting unit 295 limits the range in which the user U2 can operate the mobile object 100 to a predetermined range from the position of the paired user U1. This prevents the user U2 from operating the mobile object 100 to follow a pedestrian other than the user U1.

[0203] In this way, by switching the operation mode of the mobile body 100 to the remote control mode in response to a remote control request (login request) from the information processing device 400, even if the user U2 is in a remote location, the user U1 and the user U2 can experience the feeling of walking on the beach together.

[0204] The mode switching unit 285 may switch the operation mode of the moving object 100 to either the follow mode or the lead mode depending on the behavior of the user U1. For example, when the mode switching unit 285 detects that the user U1 is moving in a direction deviating from a predetermined route (e.g., a direction deviating from the route to the destination), the mode switching unit 285 may switch the operation mode of the moving object 100 to the lead mode. This can eliminate the need to manually switch the operation mode of the moving object 100.

[0205] Furthermore, if the mode switching unit 285 detects that the user U1 is lost, it may switch the operation mode of the moving body 100 to the leading mode. For example, the mode switching unit 285 may detect that the user U1 is lost based on an image of the user U1 captured by the camera 80. This can save the user from having to manually switch the operation mode of the moving body 100.

[0206] Furthermore, the mode switching unit 285 may switch the operation mode of the moving body 100 to the follow mode when detecting that the user U1 is about to overtake the moving body 100. For example, the mode switching unit 285 may detect that the user U1 is about to overtake the moving body 100 based on an image of the user U1 captured by the camera 80. This can save the effort of manually switching the operation mode of the moving body 100.

[0207] The mode switching unit 285 may suggest by voice or the like to switch the operation mode of the moving object 100 to either the following mode or the leading mode depending on the behavior of the user U1, thereby preventing the operation mode of the moving object 100 from being switched against the will of the user U1.

[0208] Furthermore, when switching the operation mode of the moving object 100 to either the follow mode or the lead mode, the mode switching unit 285 may change the orientation of the display unit displaying the image of the user U2 to the direction of the user U1. This allows the user U1 to walk while looking at the face of the user U2 even when the operation mode of the moving object 100 is switched.

[0209] [Calculation Process of Evaluation Function, etc.] Next, the calculation process of the evaluation function, etc. will be described. The calculation unit 250 calculates the evaluation function J(k) based on the benefit function determined by the first determination unit 220 and the risk function determined by the second determination unit 240. Specifically, in the case of the following mode, the calculation unit 250 calculates the evaluation function J(k) according to the following equation (33). That is, the calculation unit 250 calculates the evaluation function J(k) by adding together Pc_b(k), Pe_b(k), Ptp_r(k), and Pe_r(k).

[0210] J(k)=Pc_b(k)+Pe_b(k)+Ptp_r(k)+Pe_r(k)...(33)

[0211] On the other hand, in the leading mode, the calculation unit 250 calculates the evaluation function J(k) according to the following equation (34): That is, the calculation unit 250 calculates the evaluation function J(k) by adding together Pl_b(k), Pe_b(k), Ptp_r(k), and Pe_r(k).

[0212] J(k)=Pl_b(k)+Pe_b(k)+Ptp_r(k)+Pe_r(k)...(34)

[0213] On the other hand, in the remote operation mode, the calculation unit 250 calculates the evaluation function J(k) according to the following equation (35): That is, the calculation unit 250 calculates the evaluation function J(k) by adding together Pe_b_op(k), Pe_b(k), Ptp_r(k), and Pe_r(k).

[0214] J(k)=Pe_b_op(k)+Pe_b(k)+Ptp_r(k)+Pe_r(k)...(35)

[0215] The benefit functions Pc_b(k), Pl_b(k), Pe_b_op(k), and Pe_b(k) are negative values, and the smaller the benefit function value, the higher the degree of recommendation for moving the moving body 100. The risk functions Ptp_r(k) and Pe_r(k) are positive values, and the larger the risk function value, the higher the degree of risk of interference between the moving body 100 and the interference target. For this reason, the generation unit 260 needs to generate a target trajectory for the moving body 100 so that the evaluation value calculated based on the evaluation function J(k) is small.

[0216] The generation unit 260 generates a target trajectory by using an arc model to model the trajectory of the moving body 100 and calculating the curvature or radius of curvature of the arc so as to change the evaluation function J(k) in the second sign direction (negative direction), as shown in FIG. 5 above. As described above, the generation unit 260 calculates the rotation angle θ ｍ１ ~θ ｍ３ and the radius of curvature R ｍ１ ~R ｍ３ The generation unit 260 then calculates an evaluation value for each of the generated trajectories using the evaluation function J(k) calculated by the calculation unit 250, and generates the trajectory with the smallest evaluation value as the target trajectory.

[0217] The control unit 270 controls the moving body 100 based on the target trajectory generated by the generation unit 260. Specifically, the control unit 270 controls the movement mechanism 140 (drive motor, steering device, etc.) so that the moving body 100 travels along the target trajectory generated by the generation unit 260. In this way, the control device 200 can control the moving body 100 so that the moving body 100 travels along a target trajectory that is suitable for the set mode (following mode, leading mode, or remote control mode) and has a low risk of interference between the moving body 100 and an interfering object (pedestrian, obstacle, etc.).

[0218] The control unit 270 controls the speed of the moving object 100 and the target distance between the moving object 100 and the user U1. For example, when the user U1 and the moving object 100 are paired, the control unit 270 may control the moving object 100 so that the distance between the user U1 and the moving object 100 is shorter than when the pairing is not performed. This allows the moving object 100 to operate so as to be closer to the paired user U1. The control unit 270 may also change the target distance between the moving object 100 and the user U1 depending on whether or not a conversation is occurring between the user U1 and the moving object 100. Specifically, the moving object 100 is provided with a microphone, and when the control unit 270 determines using the microphone that a conversation is occurring between the user U1 and the moving object 100, the control unit 270 may increase the target distance compared to when it is determined that no conversation is occurring. The control unit 270 may calculate the speed of the user U1 based on the trajectory of the user U1 from the past to the present, and may calculate the speed of the moving body 100 based on the calculated speed of the user U1, or may control the speed so as to maintain the target distance from the user U1. Specifically, the control unit 270 may use the calculated speed of the user U1 as the target speed of the moving body 100, or may use a speed obtained by subtracting a predetermined speed from the speed of the user U1 as the target speed of the moving body 100.

[0219] According to the control device 200 of this embodiment, the trajectory of the moving body 100 is modeled using an arc model to generate a target trajectory of the moving body 100, thereby smoothing the movement of the moving body 100. This makes it easier for traffic participants (pedestrians, etc.) to predict the behavior of the moving body 100, and makes it possible to effectively prevent contact between the moving body 100 and other traffic participants.

[0220] 30 is a flowchart showing a first example of processing executed by the control device 200. The processing according to this flowchart is executed in response to pairing between the user U1 and the moving object 100.

[0221] First, the mode switching unit 285 sets the follow mode or the lead mode based on an instruction from the user U1 (first user) (step S101). For example, the mode switching unit 285 may accept an instruction from the user U1 in response to the user U1's operation on the HMI 110. Alternatively, the mode switching unit 285 may accept an instruction from the user U1 by detecting a gesture by the user U1 using the camera 80. Alternatively, the mode switching unit 285 may accept an instruction from the user U1 based on information input by the user U1 to the user terminal device 300. Furthermore, the mode switching unit 285 may recognize a voice uttered by the user using a voice recognition function and accept an instruction from the user U1 based on the recognized voice.

[0222] Next, the acquisition unit 280 waits until it acquires a remote operation request to remotely operate the mobile object 100 (step S102). For example, the remote operation request may be a login request transmitted from the information processing device 400 to the mobile object 100.

[0223] Next, the mode switching unit 285 displays an image of the user U2 (second user) on the display unit of the HMI 110 (step S103). For example, the mode switching unit 285 may display an image of the user U2 received from the information processing device 400 on the display unit of the HMI 110.

[0224] Next, the mode switching unit 285 waits until a predetermined condition is satisfied (step S104). If the predetermined condition is satisfied, the mode switching unit 285 switches the operation mode of the mobile object 100 to the remote control mode (step S105). The predetermined condition may be, for example, a predetermined time having elapsed since receiving a login request, a mode switching instruction from user U1 or user U2, or the mobile object 100 entering an area within a predetermined distance from the destination. This allows the operation mode of the mobile object 100 to be switched to the remote control mode at an appropriate time.

[0225] The process of step S104 may be omitted. In this case, the mode switching unit 285 can quickly switch the operation mode of the moving object 100 to the remote control mode in response to the acquisition of the remote control request.

[0226] 31 is a flowchart showing a second example of the process executed by the control device 200. The process according to this flowchart is repeatedly executed at regular time intervals after the user U1 (first user) starts up the moving object 100.

[0227] First, the control device 200 determines whether pairing between the user U1 and the moving object 100 has been performed (step S201). As described above, the pairing unit 290 pairs the user U1, who has undergone biometric authentication (vein authentication, facial authentication, etc.), with the moving object 100. The control device 200 determines whether pairing has been performed by the pairing unit 290. If it is determined in step S201 that pairing between the user U1 and the moving object 100 has not been performed, the control device 200 ends the processing according to this flowchart.

[0228] On the other hand, if it is determined in step S201 that pairing between the user U1 and the moving object 100 has been established, the limiting unit 295 limits the range in which the user U2 (second user) can operate the moving object 100 to a predetermined range from the position of the paired user U1 (step S202). This prevents the user U2 from operating the moving object 100 in a way that makes it follow a pedestrian other than the user U1.

[0229] Next, the control unit 270 controls the moving body 100 so that the distance between the user U1 and the moving body 100 becomes shorter than when pairing is not performed (step S203). This allows the moving body 100 to operate so as to move closer to the paired user U1.

[0230] 32 is a flowchart showing a third example of the process executed by the control device 200. The process according to this flowchart is repeatedly executed at regular time intervals after the user U1 (first user) starts up the moving object 100.

[0231] First, the mode switching unit 285 of the control device 200 determines the operation mode of the moving object 100 (step S301). Specifically, the mode switching unit 285 determines whether the operation mode of the moving object 100 is a following mode, a leading mode, or another mode (a mode other than the following mode or the leading mode).

[0232] If it is determined in step S301 that the operation mode of the moving object 100 is the follow mode, the mode switching unit 285 determines whether the user U1 is moving in a direction that deviates from the predetermined route (e.g., a direction that deviates from the route to the destination) (step S302). If it is determined that the user U1 is moving in a direction that deviates from the predetermined route, the mode switching unit 285 switches the operation mode of the moving object 100 to the lead mode (step S304). This eliminates the need to manually switch the operation mode of the moving object 100.

[0233] If it is determined in step S302 that the user U1 is not moving in a direction deviating from the predetermined route, the mode switching unit 285 determines whether the user U1 is lost (step S303). For example, the mode switching unit 285 may detect that the user U1 is lost based on an image of the user U1 captured by the camera 80. If it is determined that the user U1 is lost, the mode switching unit 285 switches the operation mode of the moving object 100 to the leading mode (step S304). This eliminates the need to manually switch the operation mode of the moving object 100.

[0234] On the other hand, if it is determined in step S303 that the user U1 is not lost, the mode switching unit 285 ends the processing according to this flowchart.

[0235] Furthermore, if it is determined in step S301 that the operation mode of the moving body 100 is the leading mode, the mode switching unit 285 determines whether the user U1 is about to overtake the moving body 100 (step S305). For example, the mode switching unit 285 may detect that the user U1 is about to overtake the moving body 100 based on an image of the user U1 captured by the camera 80. If it is determined that the user U1 is about to overtake the moving body 100, the mode switching unit 285 switches the operation mode of the moving body 100 to the following mode. This can eliminate the need to manually switch the operation mode of the moving body 100.

[0236] On the other hand, if it is determined in step S305 that the user U1 is not about to overtake the moving body 100, the mode switching unit 285 ends the processing according to this flowchart.

[0237] Also, if it is determined in step S301 that the operation mode of the moving object 100 is another mode (a mode other than the following mode or the leading mode), the mode switching unit 285 ends the processing according to this flowchart.

[0238] As described above, the control device 200 of this embodiment is a device that operates the moving body 100 by switching between a following mode in which the moving body 100 follows a user U1 (first user), a leading mode in which the moving body 100 leads the user U1, and a remote control mode in which the moving body 100 is remotely controlled by a user U2 (second user), and includes an acquisition unit 280, a mode switching unit 285, a generation unit 260, and a control unit 270. The acquisition unit 280 acquires a remote control request to remotely control the moving body 100. The mode switching unit 285 switches the operation mode of the moving body 100 to the remote control mode in response to acquiring a remote control request while the moving body 100 is operating in the following mode or the leading mode. The generation unit 260 generates a target trajectory for the moving body 100 based on the switched operation mode. The control unit 270 controls the moving body 100 based on the generated target trajectory. As a result, the control device 200 of this embodiment can cause the moving body 100 to travel by switching between a following mode or a leading mode, and a remote control mode.

[0239] The above-described embodiment can be expressed as follows: A control device comprising: a storage device storing a program; and a hardware processor that operates a moving object by switching between a following mode in which the moving object follows a first user, a leading mode in which the moving object leads the first user, and a remote control mode in which the moving object is remotely operated by a second user, wherein the hardware processor executes the program stored in the storage device to perform the following operations: acquiring a remote control request to remotely operate the moving object, switching the operation mode of the moving object to the remote control mode in response to the remote control request being acquired while the moving object is operating in the following mode or the leading mode, generating a target trajectory of the moving object based on the switched operation mode, and controlling the moving object based on the target trajectory.

[0240] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0241] REFERENCE SIGNS LIST 100 Mobile object 200 Control device 210 First detection unit 220 First determination unit 225 Estimation unit 230 Second detection unit 240 Second determination unit 250 Calculation unit 260 Generation unit 270 Control unit 280 Acquisition unit 285 Mode switching unit 290 Pairing unit 295 Restriction unit 300 User terminal device 400 Information processing device

Claims

1. A control device that operates a moving body by switching between a following mode in which the moving body follows a first user, a leading mode in which the moving body leads the first user, and a remote control mode in which a second user remotely controls the moving body, comprising: an acquisition unit that acquires a remote control request to remotely operate the moving body; a mode switching unit that switches the operation mode of the moving body to the remote control mode in response to the remote control request being acquired while the moving body is operating in the following mode or the leading mode; a generation unit that generates a target trajectory of the moving body based on the switched operation mode; and a control unit that controls the moving body based on the target trajectory.

2. The control device according to claim 1, further comprising: a pairing unit that pairs the first user with the mobile object; and a restriction unit that restricts the range in which the second user can operate the mobile object to within a predetermined range from the position of the first user with whom the pairing has been performed.

3. The control device according to claim 2, wherein the control unit controls the mobile body so that, when the pairing is performed, the distance between the first user and the mobile body is shorter than when the pairing is not performed.

4. The control device described in claim 1, wherein the mode switching unit displays an image of the second user on a display unit provided on the mobile object in response to the remote control request being acquired, and then switches the operating mode of the mobile object to the remote control mode in response to a predetermined condition being satisfied.

5. The control device according to claim 1, wherein the mode switching unit switches the operation mode of the moving object to either the following mode or the leading mode depending on the behavior of the first user.

6. The control device according to claim 5, wherein the mode switching unit switches the operation mode of the moving object to the leading mode when it detects that the first user is moving in a direction deviating from a predetermined route.

7. The control device according to claim 5, wherein the mode switching unit switches the operation mode of the moving object to the leading mode when it detects that the first user has lost his / her way.

8. The control device according to claim 5, wherein the mode switching unit switches the operation mode of the moving object to the following mode when it detects that the first user is about to overtake the moving object.

9. A mobile body system comprising a mobile body and an information processing device capable of communicating with the mobile body via a network, wherein the mobile body comprises a control device that switches between a following mode in which the mobile body follows a first user, a leading mode in which the mobile body leads the first user, and a remote control mode in which a second user remotely controls the mobile body, and the control device comprises: an acquisition unit that acquires a remote control request to remotely control the mobile body from the information processing device; a mode switching unit that switches the operation mode of the mobile body to the remote control mode in response to the remote control request being acquired while the mobile body is operating in the following mode or the leading mode; a generation unit that generates a target trajectory of the mobile body based on the switched operation mode; and a control unit that controls the mobile body based on the target trajectory.

10. A control method in which a control device that operates a moving body by switching between a following mode in which the moving body follows a first user and a remote control mode in which the moving body is remotely controlled by a second user executes the following control method: a process of acquiring a remote control request to remotely control the moving body; a process of switching the operation mode of the moving body from the following mode to the remote control mode in response to the remote control request being acquired while the moving body is operating in the following mode; a process of generating a target trajectory of the moving body based on the switched operation mode; and a process of controlling the moving body based on the target trajectory.

11. A program for causing a processor of a control device that operates a moving body by switching between a following mode in which the moving body follows a first user and a remote control mode in which a second user remotely controls the moving body, to execute the following processes: acquiring a remote control request to remotely control the moving body; switching the operating mode of the moving body from the following mode to the remote control mode in response to the remote control request being acquired while the moving body is operating in the following mode; generating a target trajectory for the moving body based on the switched operating mode; and controlling the moving body based on the target trajectory.

Images