Control device, control method, and program

Abstract

A control device that switches between a follow mode in which a mobile body follows a first user, a route mode in which the mobile body routes the first user, and a remote operation mode in which a second user remotely operates the mobile body, and causes the mobile body to act, wherein the control device includes: an acquisition unit that acquires a remote operation request that indicates an intention to remotely operate the mobile body; a mode switching unit that switches an action mode of the mobile body to the remote operation mode based on a situation in which the remote operation request is acquired during the mobile body is acting in the follow mode or the route mode; a generation unit that generates a target track of the mobile body based on the switched action mode; and a control unit that controls the mobile body based on the target track.

Description

Technical Field

[0001] This invention relates to control devices, control methods, and procedures. Background Technology

[0002] Previously, guide robots, which are autonomous mobile entities that guide users in supermarkets, shopping malls, and other similar locations, have been proposed (Patent Document 1). These guide robots generate a path based on the surrounding congestion.

[0003] Prior technology literature

[0004] [Patent Documents]

[0005] Patent Document 1: International Publication No. 2003 / 189105 Summary of the Invention

[0006] [The problem the invention aims to solve]

[0007] However, while conventional mobile devices such as navigation robots have a navigation mode for guiding users, they lack a follow mode for following users and a remote operation mode for users to remotely control the mobile device. Therefore, conventional mobile devices sometimes cannot switch between follow mode and remote operation mode, or between navigation mode and remote operation mode.

[0008] The present invention was made in consideration of such circumstances, and one of its objectives is to provide a control device, control method, and program that enables a mobile body to travel between a following mode and a remote operation mode or a guiding mode and a remote operation mode.

[0009] [Methods used to solve problems]

[0010] The control device, control method and program involved in this invention adopt the following structure.

[0011] (1): A control device according to one aspect of the present invention causes the mobile body to move by switching between a following mode that follows a first user, a guiding mode that guides the first user, and a remote operation mode that remotely operates the mobile body by a second user. The control device includes: an acquisition unit that acquires a remote operation request that indicates the intention to remotely operate the mobile body; a mode switching unit that switches the operation mode of the mobile body to the remote operation mode based on the fact that the remote operation request was acquired while the mobile body was operating in the following mode or the guiding mode; a generation unit that generates a target track for the mobile body based on the switched operation mode; and a control unit that controls the mobile body based on the target track.

[0012] (2): In the above-mentioned (1) solution, the control device further includes: a pairing unit that pairs the first user with the mobile body; and a limiting unit that limits the range within which the second user can operate the mobile body to a predetermined range from the position of the first user who has been paired.

[0013] (3): In the above (2) scheme, when the pairing is performed, the control unit controls the mobile body in such a way that the distance between the first user and the mobile body is shorter than when the pairing is not performed.

[0014] (4): In the above (1) scheme, the mode switching unit displays the image of the second user on the display unit provided on the mobile body according to the remote operation request, and then switches the operation mode of the mobile body to the remote operation mode according to the conditions specified.

[0015] (5): In the above (1) scheme, the mode switching unit switches the action mode of the mobile body to any mode among the following mode and the guiding mode according to the behavior of the first user.

[0016] (6): In the above (5) scheme, when the mode switching unit detects that the first user is moving in a direction that deviates from the prescribed path, it switches the action mode of the moving body to the guiding mode.

[0017] (7): In the above (5) scheme, when the mode switching unit detects that the first user is lost, it switches the action mode of the mobile body to the navigation mode.

[0018] (8): In the above (5) scheme, when the mode switching unit detects that the first user is about to overtake the mobile body, it switches the action mode of the mobile body to the following mode.

[0019] (9): Another aspect of the present invention relates to a mobile body system comprising a mobile body and an information processing device capable of communicating with the mobile body via a network, wherein the mobile body comprises a control device that switches between a following mode for following a first user, a guiding mode for guiding the first user, and a remote operation mode for remotely operating the mobile body by a second user to move the mobile body, the control device comprising: an acquisition unit that acquires a remote operation request from the information processing device indicating an intention to remotely operate the mobile body; a mode switching unit that switches the operation mode of the mobile body to the remote operation mode based on the case that the remote operation request was acquired while the mobile body is operating in the following mode or the guiding mode; a generation unit that generates a target trajectory for the mobile body based on the switched operation mode; and a control unit that controls the mobile body based on the target trajectory.

[0020] (10): In other embodiments of the present invention, the control method is performed by a control device, wherein the control device switches between a following mode that follows a first user and a remote operation mode that remotely operates the mobile body by a second user to cause the mobile body to move, and the control method causes the control device to perform the following processes: obtaining a remote operation request that indicates the intention to remotely operate the mobile body; switching the operation mode of the mobile body from the following mode to the remote operation mode based on the fact that the remote operation request was obtained while the mobile body was operating in the following mode; generating a target trajectory for the mobile body based on the switched operation mode; and controlling the mobile body based on the target trajectory.

[0021] (11): In other embodiments of the present invention, the program is executed by a processor of a control device, wherein the control device causes the mobile body to move by switching between a following mode that follows a first user and a remote operation mode that remotely operates the mobile body by a second user, and the program causes the processor of the control device to perform the following processes: obtaining a remote operation request that indicates the intention to remotely operate the mobile body; switching the operation mode of the mobile body from the following mode to the remote operation mode based on the fact that the remote operation request was obtained while the mobile body was operating in the following mode; generating a target trajectory for the mobile body based on the switched operation mode; and controlling the mobile body based on the target trajectory.

[0022] [Invention Effects]

[0023] According to the schemes (1) to (11), the mobile body can switch between following mode and remote operation mode or between leading mode and remote operation mode to travel. Attached Figure Description

[0024] Figure 1 This is a diagram illustrating an example of the structure of a mobile body system 1, which includes a mobile body 100.

[0025] Figure 2 This is a three-dimensional diagram representing the moving body 100.

[0026] Figure 3 This is a diagram showing the structure of the moving body 100 equipped with the control device 200.

[0027] Figure 4 This is a diagram showing an example of the structure of the control device 200.

[0028] Figure 5 This is a diagram showing an example of a target trajectory generated by the generation unit 260.

[0029] Figure 6 It is a graph used to illustrate the risk function of environmental object targets and the benefit function based on environmental object targets.

[0030] Figure 7 It represents X e_l A diagram showing an example of the risk function for an environmental object target along the axis and the benefit function based on the environmental object target.

[0031] Figure 8 It represents Y e_l A diagram of an example of the potential field function along the axis.

[0032] Figure 9 This is a schematic diagram of the revenue function based on environmental object targets when environmental object targets are detected at multiple detection locations.

[0033] Figure 10 This is a graph used to illustrate the traffic participant-obstacle risk function.

[0034] Figure 11 This is a graph used to illustrate the traffic participant-obstacle risk function obtained based on traffic participant U3.

[0035] Figure 12 It represents X tp_l A diagram of an example of the traffic participant-obstacle risk function along the axis.

[0036] Figure 13 It represents Y tp_l A diagram of an example of the traffic participant-obstacle risk function along the axis.

[0037] Figure 14 This is a graph used to illustrate the payoff function of the follower pattern.

[0038] Figure 15 This is a diagram illustrating an example of the amount of movement of the moving body 100.

[0039] Figure 16 It is a diagram used to illustrate the user's (follower's) field of vision.

[0040] Figure 17 This is a diagram representing an example of multiple target locations.

[0041] Figure 18 This is a diagram illustrating an example of a state where a follower pattern payoff function is set at multiple target locations.

[0042] Figure 19 This is a graph representing an example of the follower pattern payoff function along the X-axis.

[0043] Figure 20 This is a graph representing an example of the follower pattern payoff function along the Y-axis.

[0044] Figure 21 This is a graph used to illustrate the revenue function of the belt mode.

[0045] Figure 22 This is a diagram used to illustrate the estimation process for the predicted trajectory of user U1.

[0046] Figure 23 This is a diagram illustrating an example of a route mode revenue function set at multiple target locations.

[0047] Figure 24 This is a diagram representing an example of the band-mode revenue function along the X-axis.

[0048] Figure 25 This is a diagram representing an example of the band-mode revenue function along the Y-axis.

[0049] Figure 26 This diagram illustrates the situation where the mobile body 100 is automatically moving in remote operation mode.

[0050] Figure 27 This diagram illustrates a scenario where user U2 instructs the moving body 100 to move to the left.

[0051] Figure 28 This is a graph representing an example of a remote operation revenue function.

[0052] Figure 29 This diagram illustrates the change in the direction of travel of the moving body 100 when a remote operation benefit function is set.

[0053] Figure 30 This is a flowchart illustrating an example of a process performed by the control device 200.

[0054] Figure 31This is a flowchart illustrating a second example of the processing performed by the control device 200.

[0055] Figure 32 This is a flowchart representing the third example of the processing performed by the control device 200. Detailed Implementation

[0056] Hereinafter, embodiments of the control device, control method, and program of the present invention will be described with reference to the accompanying drawings. The control device of the present invention controls the movement mechanism of the mobile body to move the mobile body. The mobile body in the present invention moves autonomously within a pedestrian walking area. A pedestrian walking area refers to sidewalks, open spaces, building floors, etc., and may also include driveways. In the following description, it is assumed that a person is not riding on the mobile body, but it is also possible that a person is riding on the mobile body. The mobile body operates in a following mode that follows the user, a guiding mode that leads the user, and a remote operation mode (virtual avatar mode) where the user remotely operates the mobile body. The user who becomes the object of following or guiding can be, for example, a pedestrian, a robot, or an animal.

[0057] [Mobile System]

[0058] Figure 1 This diagram illustrates an example of the structure of a mobile system 1, including a mobile body 100. The mobile system 1 includes, for example, a mobile body 100, a user terminal device 300, and an information processing device 400. They communicate, for example, via a network NW. The network NW can be any network such as a LAN, WAN, or Internet connection. It should be noted that the mobile body 100 and the user terminal device 300 may also communicate directly via short-range wireless communication without using a network NW.

[0059] [User terminal device]

[0060] User terminal device 300 is, for example, a mobile terminal device such as a smartphone or tablet operated by user U1. User terminal device 300 receives instructions from user U1 and sends the received instructions to mobile device 100. For example, user terminal device 300 may also have a touch panel to accept instructions input from user U1 using the touch panel. Additionally, user terminal device 300 may also have voice recognition functionality to accept instructions based on the user U1's voice recognized using the voice recognition function. Furthermore, user terminal device 300 may also have a display unit that displays information received from mobile device 100.

[0061] [Information processing device]

[0062] The information processing device 400 is, for example, a computer that remotely operates the mobile body 100 based on the operation of the user U2. The information processing device 400 may include an input device such as a joystick. For example, the user U2 can also operate the direction of travel of the mobile body 100 by operating the joystick. Specifically, the user U2 can instruct the mobile body 100 to move to the left relative to the direction of travel by tilting the joystick to the left. Additionally, the user U2 can instruct the mobile body 100 to move to the right relative to the direction of travel by tilting the joystick to the right.

[0063] The information processing device 400 sends the operation information obtained based on the user U2's operation to the mobile body 100. It should be noted that the joystick may also be equipped with a start button for indicative of starting the movement of the mobile body 100 and a stop button for indicative of stopping the movement of the mobile body 100.

[0064] In addition, the information processing device 400 receives image information transmitted from the mobile body 100. The image information is an image captured by a camera installed on the mobile body 100. The information processing device 400 may include a display unit such as a liquid crystal display or an organic EL display. The information processing device 400 can display the image information received from the mobile body 100 on the display unit. The user U2 operates an input device such as a joystick based on the image of the periphery of the mobile body 100 displayed on the display unit 330.

[0065] User U2 can also control the camera's viewing angle on the mobile body 100 via the information processing device 400. For example, user U2 can also control the pan, tilt, and zoom of the camera mounted on the mobile body 100 by operating the information processing device 400. Alternatively, the information processing device 400 can be a wearable device such as VR glasses. In this case, the user can also control the camera in conjunction with the movements of the VR glasses.

[0066] [Moving entity]

[0067] Figure 2 This is a perspective view of the mobile body 100. The mobile body 100 includes, for example, a base 10, a door 60 provided on the base 10, and wheels (first wheel 20, second wheel 30, and third wheel 40) assembled on the base 10. For example, user U1 can open the door 60 to insert or remove luggage from the storage compartment provided on the base 10. The first wheel 20 and second wheel 30 are drive wheels, and the third wheel 40 is an auxiliary wheel (driven wheel).

[0068] A cylindrical support 50 extending upward is provided on the upper surface of the base 10. A camera 80 for taking pictures around the moving body 100 is provided at the end of the support 50. The position of the camera 80 can also be any position different from the one described above.

[0069] Camera 80 is, for example, a camera capable of capturing the periphery of the moving object 100 with a wide angle (e.g., 360 degrees). Camera 80 may also include multiple cameras. For example, camera 80 may be implemented by combining multiple 120-degree cameras or multiple 60-degree cameras.

[0070] Figure 3 This diagram illustrates the structure of a mobile body 100 equipped with a control device 200. The mobile body 100 includes, for example: a base 160 on which an HMI 110, a detection device 120, a position determination device 130, a communication device 170, and the control device 200 are mounted; a movement mechanism 140 mounted on the base 160; and a sensor 150 mounted on the movement mechanism 140, etc. The base 160 can be connected to... Figure 2 The matrix 10 is the same.

[0071] The HMI110 displays various information to the user U1 and accepts input from the user U1. The HMI110 includes various display devices (display unit), speakers, microphones, buzzers, touch panels, switches, buttons, etc.

[0072] The detection device 120 is a device that generates data used to identify objects and the user U1 present in the vicinity of the moving body 100. The detection device 120 may include, for example, an object recognition device that identifies objects based on the output of the camera 80. It should be noted that the detection device 120 may also identify objects by performing sensor fusion processing based on sensors other than the camera 80, such as radar devices, LIDAR (Light Detection and Ranging), ultrasonic sensors, etc., and the outputs of these sensors.

[0073] The position determination device 130 is a device for determining the position of the moving body 100. The position determination device 130 may include, for example, a GNSS receiver that determines the position of the moving body 100 based on signals received from GNSS (Global Navigation Satellite System) satellites. The position determination device 130 may also determine or supplement the position of the moving body 100 using INS (Inertial Navigation System) outputs from the sensor 150 described later. Additionally, the position determination device 130 may also have an electromagnetic wave receiving function and determine or supplement the position of the moving body 100 based on the intensity of electromagnetic waves arriving from nearby electromagnetic wave transmitting sources (whose locations are known).

[0074] The movement mechanism 140 is a mechanism for moving the moving body 100 in any direction. The movement mechanism 140 may include, for example, multiple wheels (first wheel 20, second wheel 30, and third wheel 40), drive motors mounted on one or more wheels, and steering devices mounted on one or more wheels. There are no particular restrictions on the structure of the movement mechanism 140; it may also include structures other than wheels, such as feet for bipedal walking simulation, tracks, etc.

[0075] Sensor 150 is a sensor used to detect the behavior of the moving body 100. Sensor 150 includes, for example, a wheel speed sensor for detecting the speed of the wheels, an acceleration sensor for detecting the acceleration acting on the moving body 100, a yaw rate sensor mounted near the center of gravity in the horizontal direction of the base 160, a steering angle sensor for detecting the steering angle of the steering wheel (steering wheel), and an orientation sensor for detecting the orientation of the moving body 100 in the horizontal direction.

[0076] Communication device 170 is a wireless communication module that communicates wirelessly with a wireless base station connected to a network NW. Communication device 170 communicates with user terminal device 300 and information processing device 400 via the network NW. Alternatively, communication device 170 can also communicate directly with user terminal device 300 via short-range wireless communication.

[0077] Figure 4This diagram illustrates an example of the structure of the control device 200. The control device 200 includes, for example, a first detection unit 210, a first decision unit 220, an estimation unit 225, a second detection unit 230, a second decision unit 240, a calculation unit 250, a generation unit 260, a control unit 270, an acquisition unit 280, a mode switching unit 285, a pairing unit 290, and a limiting unit 295. These components are implemented, for example, by executing a program (software) using a hardware processor such as a CPU (Central Processing Unit). Some or all of these components can be implemented using hardware (including the circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit), or through a combination of software and hardware. The program can be pre-saved on storage devices such as HDDs (Hard Disk Drives) and flash memory (storage devices with non-transitory storage media), or it can be saved on removable storage media such as DVDs and CD-ROMs (non-transitory storage media), and installed on the storage device by mounting the storage media onto the drive device. It should be noted that the control device 200 can also store map information, including at least a local map of the location where the mobile body 100 is operating, on the storage device.

[0078] The first detection unit 210 detects the position of user U1 based on information input from the detection device 120. When the mobile body 100 is operating in either a following mode or a guiding mode, the first detection unit 210 repeatedly detects the position of user U1 at predetermined time intervals. It should be noted that the following mode is the mode in which the mobile body 100 follows a follower. The guiding mode is the mode in which the mobile body 100 guides a follower. Furthermore, the first detection unit 210 detects user U2's operations on the mobile body 100 based on operation information received by the communication device 170 from the information processing device 400.

[0079] The second detection unit 230 detects the positions of interference objects that the moving body 100 should avoid around the moving body 100 based on information input from the detection device 120. An interference object refers to an object that interferes with the movement of the moving body 100. For example, interference objects are objects that the moving body 100 should avoid, such as environmental objects, traffic participants, and obstacles. Environmental objects include, for example, walls or lawns, objects within the boundaries of the driving road that the moving body 100 cannot enter. Traffic participants include, for example, pedestrians and vehicles. Obstacles include, for example, stationary objects existing on the driving road that impede the movement of the moving body 100. The moving body 100 repeatedly detects the position of the interference object at predetermined time intervals to avoid contact with it during movement.

[0080] The estimation unit 225 estimates the predicted trajectory of user U1 based on the position of user U1 from the present to the past detected by the first detection unit 210. Details of the estimation process for the predicted trajectory of user U1 will be described later.

[0081] In follow mode, the first decision unit 220 determines a follow mode benefit function representing the recommended degree of travel of the mobile body 100 based on the position of user U1 from the current to the past detected by the first detection unit 210. In guide mode, the first decision unit 220 determines a guide mode benefit function representing the recommended degree of travel of the mobile body 100 based on the predicted trajectory estimated by the estimation unit 225. In remote operation mode, the first decision unit 220 determines a remote operation benefit function representing the recommended degree of travel of the mobile body 100 based on the operation of user U2 on the mobile body 100 detected by the first detection unit 210. Furthermore, the first decision unit 220 determines an environmental object target-based benefit function based on the position of an environmental object target detected by the second detection unit 230. It should be noted that details regarding the follow mode benefit function, guide mode benefit function, remote operation benefit function, and environmental object target-based benefit function will be described later.

[0082] The second decision unit 240 determines a risk function representing the degree of risk of interference between the moving body 100 and the interfering object based on the position of the interfering object detected by the second detection unit 230. Specifically, the second decision unit 240 determines an environmental object target risk function based on the position of an environmental object target detected by the second detection unit 230. Furthermore, the second decision unit 240 determines a traffic participant / obstacle risk function based on the positions of traffic participants and obstacles detected by the second detection unit 230. It should be noted that details regarding the environmental object target risk function and the traffic participant / obstacle risk function will be described later.

[0083] In follow mode, the calculation unit 250 calculates an evaluation function for evaluating the path traveled by the mobile body 100, based on the follow mode benefit function determined by the first decision unit 220 and the benefit function based on environmental object targets, and the environmental object target risk function and traffic participant / obstacle risk function determined by the second decision unit 240. In guide mode, the calculation unit 250 calculates an evaluation function for evaluating the path traveled by the mobile body 100, based on the guide mode benefit function determined by the first decision unit 220 and the benefit function based on environmental object targets, and the environmental object target risk function and traffic participant / obstacle risk function determined by the second decision unit 240. In remote operation mode, the calculation unit 250 calculates an evaluation function for evaluating the path traveled by the mobile body 100, based on the remote operation benefit function determined by the first decision unit 220 and the benefit function based on environmental object targets, and the environmental object target risk function and traffic participant / obstacle risk function determined by the second decision unit 240. It should be noted that details regarding the evaluation function will be provided later.

[0084] The generation unit 260 generates the target trajectory of the moving body 100 based on the evaluation function calculated by the calculation unit 250. For example, the generation unit 260 uses a model representing a portion of the outer periphery of a geometric shape to generate the target trajectory of the moving body 100. The model can be an arc model representing a circular arc. The generation unit 260 generates the target trajectory of the moving body 100 by modeling the trajectory of the moving body 100 using the arc model. The generation process of generating the target trajectory by the generation unit 260 will be described below.

[0085] [Target orbit generation and processing]

[0086] Figure 5 This is a diagram showing an example of a target trajectory generated by the generation unit 260. (See diagram for example.) Figure 5 As shown, the generation unit 260 generates the target trajectory of the moving body 100 by combining the first to third predicted trajectories in an arc shape.

[0087] First, let's explain the first predicted trajectory. In Figure 5 In the definition, an XY coordinate system is defined with the center of the current position of the moving body 100 as the origin, the front of the moving body 100 as the X-axis, and the left side of the moving body 100 as the Y-axis.

[0088] Furthermore, the position Z, starting from the origin of the XY coordinate system after the first prediction time, is set as the first prediction position. m1 It will be from the origin to the first predicted position Z. m1 The orbit is defined as the first predicted orbit in the shape of a circular arc.

[0089] Additionally, the radius of curvature of the first predicted trajectory is set to R.m1 And set the rotation angle of the first predicted trajectory to θ. m1 Rotation angle θ m1 This is equivalent to the angle between the Y-axis and the Y'-axis, which will be discussed later.

[0090] Additionally, regarding the distance from the origin to the first predicted position Z... m1 The predicted position when the orbit is divided into three equal parts, from the origin towards the first predicted position Z. m1 The three prediction positions Z are set sequentially. m11 Z m12 Z m13 (=Z) m1 ).

[0091] Next, the second predicted orbit will be explained. Figure 5 The first predicted position Z is defined in the middle. m1 (=Predicted position Z) m13 Set the first predicted position Z of the first predicted trajectory as the origin. m1 The direction of the tangent at the point is set as the X' axis, and the direction orthogonal to the tangent is set as the Y' axis in the X'-Y' coordinate system.

[0092] Furthermore, starting from the origin of the X'-Y' coordinate system (i.e., the first predicted position Z), m1 The position after the second prediction time is set as the second prediction position Z. m2 It will start from the first predicted position Z m1 To the second predicted position Z m2 The orbit is defined as the second predicted orbit in the shape of a circular arc.

[0093] Additionally, the radius of curvature of the second predicted trajectory is set to R. m2 The rotation angle of the second predicted orbit is set to θ. m2 Rotation angle θ m2 This is equivalent to the angle between the Y' axis and the Y” axis, which will be discussed later.

[0094] Next, we will explain the third predicted orbit. Figure 5 In the middle, the second predicted position Z is defined. m2 Let Z be the origin, and set the second predicted position of the second predicted trajectory. m2 The X”-Y” coordinate system is formed by setting the direction of the tangent at a certain point as the X” axis and setting the direction orthogonal to the tangent as the Y” axis.

[0095] Furthermore, the origin of the X”-Y” coordinate system (i.e., the second predicted position Z) will be used... m2 The position after the third prediction time is set as the third prediction position Z. m3 It will start from the second predicted position Z m2 To the third predicted position Z m3The orbit is defined as a third predicted orbit in the shape of a circular arc.

[0096] As described above, the first predicted trajectory is determined by combining three predicted trajectories that are shorter than the second and third predicted trajectories. Specifically, this is the trajectory obtained by combining the current position to the predicted position Z. m11 The orbit from the predicted position Z m11 To the predicted position Z m12 The orbit, and from the predicted position Z m12 To the predicted position Z m13 The orbit obtained from the orbit determines the first predicted orbit.

[0097] This is because the first predicted trajectory is closer to the moving body 100 than the second and third predicted trajectories, thus requiring the generation of a predicted trajectory in a way that more reliably avoids interference with the object being interfered with. It should be noted that the generation unit 260 may also determine the first predicted trajectory as a combination of two or more predicted trajectories to obtain a trajectory.

[0098] The generation unit 260 determines the rotation angle θ m1 ~θ m3 and radius of curvature R m1 ~R m3 Multiple tracks are generated by combining multiple patterns. Then, the generation unit 260 evaluates each of the generated tracks using an evaluation function calculated by the calculation unit 250. Afterward, based on the evaluation result using the evaluation function, the generation unit 260 determines one of the generated tracks as the target track for the moving body 100. By evaluating the generated tracks based on the evaluation function, the generation unit 260 can generate a target track for the moving body 100 that has a low risk of interference with the interference object and is suitable for follow-up or lead-ahead modes. It should be noted that details of the evaluation process using the evaluation function will be described later.

[0099] return Figure 4 As explained, the control unit 270 controls the moving mechanism 140 to move the moving body 100 along the target track generated by the generating unit 260. The control unit 270 controls the drive motor and the steering device to bring the position and behavior of the moving body 100, obtained from the output of the sensor 150, closer to the target track.

[0100] The acquisition unit 280 acquires a remote operation request indicating the intention to remotely operate the mobile body 100. For example, the acquisition unit 280 may acquire the remote operation request from the information processing device 400. Specifically, the information processing device 400 may send a login request for logging into the mobile body 100 based on the operation of the user U2, and the acquisition unit 280 acquires the login request received from the information processing device 400 as a remote operation request.

[0101] The mode switching unit 285 selects one of several modes (e.g., follow mode, routing mode, and remote operation mode) based on input from the user U1. For example, the mode switching unit 285 can select one of several modes based on an instruction from the user U1 input via the HMI 110. Alternatively, the mode switching unit 285 can receive a mode change instruction from the user terminal device 300 using the communication device 170 and select a mode based on the received instruction. Furthermore, the mode switching unit 285 switches the operation mode of the mobile body 100 to the remote operation mode if a remote operation request is received while the mobile body 100 is operating in follow mode or routing mode. Details regarding this mode switching will be described later.

[0102] The pairing unit 290 pairs user U1 with mobile body 100. Pairing is a process used to pair user U1 with mobile body 100 by authenticating user U1. It should be noted that user U1, who has been paired with mobile body 100, becomes a follower in follow mode and a guide in guide mode. Details of the processing by the pairing unit 290 will be described later.

[0103] In remote operation mode, the limiting unit 295 restricts the range within which user U2 can operate the mobile body 100 to a predetermined range from the location of the paired user U1. Details regarding the processing of the limiting unit 295 will be described later.

[0104] [Risk function for environmental objects and benefit function based on environmental object objectives]

[0105] Next, we will explain the environmental object target risk function and the benefit function based on the environmental object target. The environmental object target risk function represents the degree of risk of interference between the moving body 100 and the environmental object target (e.g., a wall, a lawn, etc.). Conversely, the environmental object target benefit function represents the degree of non-interference between the moving body 100 and the environmental object target.

[0106] Figure 6 This is a graph used to illustrate the risk function of environmental object targets and the benefit function based on environmental object targets. For example... Figure 6 As shown, the direction of travel of the moving body 100 is set as the Y-axis, and the direction orthogonal to the Y-axis is set as the X-axis. There are multiple environmental object targets R1 to R3 around the moving body 100, so the moving body 100 needs to travel in a way that does not interfere with these multiple environmental object targets R1 to R3.

[0107] exist Figure 6In the example shown, the second detection unit 230 detects the position of the environmental object target R2 based on information input from the detection device 120. Here, the detection position of the environmental object target R2 is set as E_l. Figure 6 In the diagram, the detection location E_l only shows one location, but in reality, environmental object target R2 was detected at multiple locations.

[0108] It should be noted that in the XY coordinate system defined by the X and Y axes, the coordinates of the detection position E_l are defined as (xe_l, ye_l). The suffix l represents the number of the detected environmental object target, denoted as l = 1, 2, ..., n. e n e This represents the total number of detected locations of environmental objects.

[0109] Additionally, the direction from the center of the moving body 100 toward the detection position E_l is defined as X. e_l axis, and will be with X e_l Let the direction of the orthogonal axis be Y. e_l Axis. Additionally, when the X-axis is compared with the X... e_l Let the angle between the axes be ψ. e_l At that time, ψ e_l As shown in the following formula (1).

[0110] ψ e_l =tan -1 (ye_l / xe_l) …(1)

[0111] Furthermore, the coordinates of the predicted position on the trajectory generated by the generation unit 260 in the XY coordinate system are defined as Z. mj (X) mj Y mj The suffix j represents the number of the predicted position on the track generated by the generation unit 260, denoted as j=1, 2, ..., n. m n m This represents the total number of predicted positions on the orbit generated by the generation unit 260. Figure 5 In the example shown, five predicted locations (Z) are obtained. m11 Z m12 Z m13 (=Z) m1 Z m2 Z m3 Therefore, it becomes n. m =5.

[0112] In addition, the coordinates Z of the predicted location are determined based on the following equations (2) and (3). mj (X) mj Y mj Transformed into X e_l axis and Y e_lThe X-axis is defined by the axis. e_l -Y e_l Coordinate Z in the coordinate system mj_el (X) mj_el Y mj_el ).

[0113] X mj_el = (X mj -xe_l)cosψ e_l +(Y mj -ye_l)sinψ e_l …(2)

[0115] Y mj_el =-(X mj -xe_l)sinψ e_l +(Y mj -ye_l)cosψ e_l … (3)

[0116] Figure 7 It represents X e_l A diagram illustrating an example of the risk function for an environmental object target along the axis and the payoff function based on the environmental object target. (See diagram below.) Figure 7 As shown, the first decision unit 220 defines a reward function Pe_x_b based on the environmental object target R2, with the detection position E_l of the environmental object target R2 set as the origin. Similarly, the second decision unit 240 defines X... e_l The environmental object target risk function Pe_x_r along the axis.

[0117] It should be noted that the second decision unit 240 defines X in a manner that has a gradient in the first sign direction (positive direction). e_l The risk function for the environmental object target along the axis is Pe_x_r. Additionally, the first decision unit 220 defines a reward function Pe_x_b based on the environmental object target in a manner that has a gradient in a second sign direction (negative direction) opposite to the first sign direction (positive direction).

[0118] like Figure 7 As shown, X e_l The environmental object target risk function Pe_x_r along the axis is X e_l A function whose risk function value increases with larger coordinates on the X-axis is only defined on the X-axis. e_l The region where the axis values ​​are positive. Additionally, X... e_l The reward function Pe_x_b based on the environment object target along the axis is X e_l A function whose payoff value is smaller for coordinates on the X-axis; this function is only defined on the X-axis. e_lThe values ​​of the axes are in the negative region. It should be noted that this indicates the following: the larger the risk function value, the higher the risk of interference between the moving body 100 and the object being interfered with; the smaller the benefit function value, the lower the risk of interference between the moving body 100 and the object being interfered with.

[0119] Figure 8 It represents Y e_l A diagram of an example of the potential field function along the axial direction. (See diagram) Figure 8 As shown, the potential field function Pe_y is a function with the detection position E_l of the environmental object target R2 set as the origin. The second decision unit 240 defines the potential field function Pe_y as Y e_l Risk function of environmental object targets along the axis.

[0120] Y e_l The environmental object target risk function (potential field function Pe_y) along the axis is Y e_l A function whose risk function value increases the closer the coordinates of the axes are to 0. For example... Figure 7 and Figure 8 As shown, X e_l The environmental object target risk function Pe_x_r and Y along the axis e_l The environmental object target risk function Pe_y along the axis becomes a function of mutually different shapes. By X e_l The environmental object target risk function Pe_x_r along the axis is defined as X e_l A larger value for the axis coordinate results in a larger risk function value, thus preventing the generation of a target trajectory for the moving body 100 towards a position beyond the front of the target object in the environment. On the other hand, regarding Y... e_l The environmental object target risk function Pe_y along the axis is also defined as X. e_l When the environmental object target risk function Pe_x_r in the axial direction is the same, the computational load on the control device 200 increases. Therefore, regarding Y... e_l The environmental object target risk function Pe_y along the axis is defined such that the position of the origin (i.e. the position where the environmental object target R2 is detected) becomes the peak value.

[0121] Next, we will explain how to calculate the risk function value of the environmental object target. As mentioned above, the coordinates Z of the predicted position on the predicted trajectory in the XY coordinate system are... mj (X) mj Y mj Transform into X e_l -Y e_l The coordinates Z of the predicted position in the coordinate system mj_el (X) mj_el Y mj_el At the current time k, the X coordinates X of the predicted position will be used. mj_elThe obtained risk function value is set as Pe_x_r_j_l(k), and the Y coordinate Y based on the predicted location is used. mj_el The risk function value is obtained and set as Pe_y_j_l(k). Pe_x_r_j_l(k) can be determined based on... Figure 7 Pe_x_r is given. Pe_y_j_l(k) can be obtained from Pe_x_r. Figure 8 Pe_y is obtained from the given information.

[0122] In this case, the risk function value Pe_r_j_l(k) for each detection location on the environmental object target can be expressed as shown in equation (4). That is, Pe_r_j_l(k) can be obtained by multiplying Pe_x_r_j_l(k) by Pe_y_j_l(k).

[0123] Pe_r_j_l(k)=Pe_x_r_j_l(k)×Pe_y_j_l(k)…(4)

[0125] Furthermore, the total risk function value Pe_r_j(k) for the coordinates of the predicted positions on the predicted trajectory can be expressed as shown in equation (5). That is, by taking the risk function values ​​Pe_r_j_l(k) (l=1, 2, ..., n) for all detection positions on the environmental object target. e In total, Pe_r_j(k) can be calculated.

[0126] Pe_r_j(k)=Σ l=1 ne (Pe_r_j_l(k))…(5)

[0127] Furthermore, the environmental object target risk function value Pe_r(k) can be expressed as shown in equation (6). That is, by using the total risk function value Pe_r_j(k) (j=1, 2, ..., n) for the coordinates of all predicted positions on the predicted trajectory. m In total, the environmental object target risk function value Pe_r(k) can be obtained.

[0128] Pe_r(k) = Σ j=1 nm (Pe_r_j(k))…(6)

[0129] Next, we will explain how to calculate the benefit function value based on the environmental object target. For example... Figure 7 As explained, the first decision unit 220 defines a reward function Pe_x_b based on the environmental object target R2, with the detection position E_l of the environmental object target R2 set as the origin. Furthermore, the first decision unit 220... Figure 8 The potential field function Pe_y shown is defined as Y e_lA axial benefit function based on environmental object targets.

[0130] As mentioned above, the Z coordinates of the predicted position on the predicted trajectory in the XY coordinate system are... mj (X) mj Y mj Transform into X e_l -Y e_l The coordinates Z of the predicted position in the coordinate system mj_el (X) mj_el Y mj_el At the current time k, the X coordinates X of the predicted position will be used. mj_el The obtained profit function value is set as Pe_x_b_j_l(k), and the Y coordinate Y based on the predicted location is used. mj_el The resulting profit function value is denoted as Pe_y_j_l(k). Pe_x_b_j_l(k) can be determined based on... Figure 7 Pe_x_b is shown. Pe_y_j_l(k) can be obtained from Pe_x_b. Figure 8 Pe_y is obtained from the given information.

[0131] In this case, the benefit function value Pe_b_j_l(k) for each detection location on the environmental object target can be expressed as shown in equation (7). That is, Pe_b_j_l(k) can be obtained by multiplying Pe_x_b_j_l(k) by Pe_y_j_l(k).

[0132] Pe_b_j_l(k)=Pe_x_b_j_l(k)×Pe_y_j_l(k)…(7)

[0134] Furthermore, the total reward function value Pe_b_j(k) for the coordinates of the predicted positions on the predicted trajectory can be expressed as shown in equation (8). That is, the reward function value Pe_b_j_l(k) (l=1, 2, ..., n) for all detection positions on the environmental object target can be obtained. e The maximum value of ) is Pe_b_j(k).

[0135] Pe_b_j(k)=max Pe_b_j_l(k)

[0136] =max Pe_x_b_j_l(k)×Pe_y_j_l(k)…(8)

[0138] Figure 9 This is a schematic diagram illustrating the reward function based on environmental objects when environmental objects are detected at multiple detection locations. For example... Figure 9As shown, for example, assuming there are walls on both the left and right sides, environmental object targets are detected on the right and left sides relative to the travel direction of the moving body 100. Furthermore, the reward function based on the detection position on the left side relative to the travel direction of the moving body 100 is set as Pe_x_b_1, and the reward function based on the detection position on the right side relative to the travel direction of the moving body 100 is set as Pe_x_b_2.

[0139] In this case, the first decision unit 220 sets the maximum value among the multiple benefit function values ​​(Pe_x_b_1 and Pe_x_b_2) as X. e_l The benefit function value based on the environmental object target along the axis. The benefit function should be set considering the balance of all environmental object targets, so the maximum value is chosen. Therefore, in the aforementioned equation (8), Pe_b_j(k) is obtained by considering the maximum value.

[0140] Furthermore, the benefit function value Pe_b(k) based on the environmental object target can be expressed as shown in equation (9). That is, it is expressed as the total benefit function value Pe_b_j(k) (j=1, 2, ..., n) for the coordinates of all predicted positions on the predicted trajectory. m In total, the benefit function value Pe_b(k) based on the environmental object target can be obtained.

[0141] Pe_b(k) = Σ j=1 nm (Pe_b_j(k))…(9)

[0142] [Traffic Participant / Obstacle Risk Function]

[0143] Next, we will explain the traffic participant / obstacle risk function. The traffic participant / obstacle risk function is a function that represents the degree of risk of interference between the moving body 100 and traffic participants and obstacles.

[0144] Figure 10 This is a graph used to illustrate the traffic participant / obstacle risk function. For example... Figure 10 As shown, the direction of travel of the moving body 100 is set as the Y-axis, and the direction orthogonal to the Y-axis is set as the X-axis. There are multiple traffic participants U3 and U4 around the moving body 100, so the moving body 100 needs to travel in a way that does not interfere with these multiple traffic participants U3 and U4.

[0145] exist Figure 10In the example shown, the second detection unit 230 detects the positions of traffic participants U3 and U4 based on information input from the detection device 120. Furthermore, the second decision unit 240 sets a traffic participant / obstacle risk function based on the positions of traffic participants U3 and U4 detected by the second detection unit 230.

[0146] Figure 11 This is a diagram used to illustrate the traffic participant / obstacle risk function obtained based on traffic participant U3. The following describes the decision process by which the second decision unit 240 determines the traffic participant / obstacle risk function obtained based on traffic participant U3. Figure 11 In this context, the detection location for traffic participant U3 is set to TP_l. Figure 11 The diagram only shows the detection location TP_l of traffic participant U3, but traffic participant U4 is also detected. It should be noted that if there are obstacles on the road besides traffic participants, the second detection unit 230 also detects the obstacles.

[0147] It should be noted that in the XY coordinate system defined by the X and Y axes, the coordinates of the detection position TP_l are defined as (xtp_l, ytp_l). The suffix l represents the number of the detected traffic participant / obstacle location, denoted as l = 1, 2, ..., n. t n t This represents the total number of detected locations of traffic participants and obstacles. Additionally, the direction of travel of traffic participant U3 is defined as the Ytp_l axis, and the direction orthogonal to the Ytp_l axis is defined as the Xtp_l axis.

[0148] Furthermore, the coordinates of the predicted position on the trajectory generated by the generation unit 260 in the XY coordinate system are defined as Z. mj (X) mj Y mj Using the same method as in equations (2) and (3) above, the coordinates Z of the predicted position are... mj (X) mj Y mj Transformed into X tp_l axis and Y tp_l The X-axis is defined by the axis. tp_l -Y tp_l Coordinate Z in the coordinate system mj_tpl (X) mj_tpl Y mj_tpl ).

[0149] Figure 12 It represents X tp_l A graph illustrating an example of the traffic participant / obstacle risk function along the axial direction. (See figure.) Figure 12As shown, the second decision unit 240 defines X by setting the detection position TP_l of traffic participant U3 as the origin. tp_l The traffic participant / obstacle risk function Ptp_x along the axial direction. It should be noted that the second decision unit 240 defines X in a manner that has a gradient in the first sign direction (positive direction). tp_l The traffic participant / obstacle risk function Ptp_x along the axis.

[0150] like Figure 12 As shown, X tp_l The traffic participant-obstacle risk function Ptp_x along the axis is X tp_l A function whose risk function value increases the closer the coordinates of the axis are to 0.

[0151] Figure 13 It represents Y tp_l A graph illustrating an example of the traffic participant / obstacle risk function along the axial direction. (See figure.) Figure 13 As shown, the second decision unit 240 defines Y by setting the detection position TP_l of traffic participant U3 as the origin. tp_l The traffic participant / obstacle risk function Ptp_y along the axial direction. It should be noted that the second decision unit 240 defines Y in a manner that has a gradient in the first sign direction (positive direction). tp_l The traffic participant / obstacle risk function Ptp_y along the axis.

[0152] like Figure 13 As shown, Y tp_l The traffic participant-obstacle risk function Ptp_y along the axis is Y tp_l A function whose risk function value increases the closer the coordinates of the axis are to 0.

[0153] Next, we will explain how to calculate the traffic participant / obstacle risk function value. As mentioned earlier, the coordinates Z of the predicted position on the predicted track in the XY coordinate system are... mj (X) mj Y mj Transform into X tp_l -Y tp_l The coordinates Z of the predicted position in the coordinate system mj_tpl (X) mj_tpl Y mj_tpl At the current time k, the X coordinates X of the predicted position will be used. mj_tpl The obtained risk function value is set as Ptp_x_j_l(k), and the Y-coordinate based on the predicted location is used. mj_tpl The obtained risk function value is set as Ptp_y_j_l(k). Ptp_x_j_l(k) can be determined based on... Figure 12 Ptp_x is obtained from the given Ptp_x. Ptp_y_j_l(k) can be obtained from... Figure 13 The value of Ptp_y is obtained from the given information.

[0154] In this case, the risk function value Ptp_r_j_l(k) for each detection location of traffic participants and obstacles can be expressed as shown in equation (10). That is, Ptp_r_j_l(k) can be obtained by multiplying Ptp_x_j_l(k) by Ptp_y_j_l(k).

[0155] Ptp_r_j_l(k)=Ptp_x_j_l(k)×Ptp_y_j_l(k)…(10)

[0157] Furthermore, the total risk function value Ptp_r_j(k) for the coordinates of the predicted positions on the predicted track can be expressed as shown in Equation (11). That is, by using the risk function values ​​Ptp_r_j_l(k) (l=1, 2, ..., n) for all detected positions of traffic participants and obstacles. t In total, Ptp_r_j(k) can be calculated.

[0158] Ptp_r_j(k)=Σ l=1 nt (Ptp_r_j_l(k))…(11)

[0159] Furthermore, the traffic participant-obstacle risk function value Ptp_r(k) can be expressed as shown in equation (12). That is, it is expressed as the total risk function value Ptp_r_j(k) (j=1, 2, ..., n) for the coordinates of all predicted positions on the predicted track. m In total, the traffic participant-obstacle risk function value Ptp_r(k) can be obtained.

[0160] Ptp_r(k) = Σ j=1 nm (Ptp_r_j(k))…(12)

[0161] [Follower Pattern Reward Function]

[0162] Next, we will explain the follower mode reward function. The follower mode reward function is a function that represents the degree of recommended movement of the mobile body 100, determined based on the user U1 (the follower)'s position from the current to the past in follower mode.

[0163] Figure 14 This is a graph used to illustrate the payoff function of the follower pattern. For example... Figure 14As shown, the direction of travel of the moving body 100 is set as the Y-axis, and the direction orthogonal to the Y-axis is set as the X-axis. There are multiple environmental object targets R4 and R5 around the moving body 100, so the moving body 100 needs to follow the user U1 in a way that does not interfere with these multiple environmental object targets R4 and R5.

[0164] The first detection unit 210 detects the position Vu_0(k) of user U1 at the current time k based on the detection result of the detection device 120. The first detection unit 210 repeatedly detects the position Vu_0(k) of user U1 at preset control intervals. It should be noted that the position Vu_0(k) of user U1 is detected as the relative position (position in the XY coordinate system) of user U1 relative to the moving body 100. Figure 14 In this context, Vu_1(k) represents the position of user U1 relative to Vu_0(k) before a control time of 1, and Vu_2(k) represents the position of user U1 relative to Vu_1(k) before a control time of 1. It should be noted that Vu_n... u (k) represents the position of the most recent user U1.

[0165] The first detection department 210 detects user U1's positions from the current position to the past positions Vu_0(k), Vu_1(k), Vu_2(k), ..., Vu_n. u (k). At this time, the moving body 100 is also actually moving, so the detection position of the user U1 needs to be corrected according to the amount of movement of the moving body 100. This point will be explained below.

[0166] Figure 15 This is a diagram illustrating an example of the amount of movement of the moving body 100. Figure 15 In the diagram, Z0 represents the position of the moving body 100 before control time 1. Δx mv This represents the distance moved in the X direction by the moving body 100 when it moves from position Z0 to its current position (the origin in the XY coordinate system). Δy mv This represents the distance traveled by the moving body 100 in the Y direction when it moves from position Z0 to its current position. The control device 200 calculates Δx in the control time based on the detection results of the sensor 150. mv and Δy mv .

[0167] Here, the position Vu_0(k) of user U1 at the current time k is defined as follows, as in equation (13).

[0168] Vu_0(k)=[xu_0(k) yu_0(k)]…(13)

[0169] In this case, the control device 200 is based on the movement amount Δx of the moving body 100. mv and Δy mv The movement trajectories Vu_1(k), Vu_2(k), ..., Vu_m of user U1 are calculated according to the following formulas (14) to (16). u (k). Here, m u is n u The following integers.

[0170] Vu_1(k)

[0171] =[xu_1(k) yu_1(k)]

[0172] =[xu_0(k-1)-Δx mv (k) yu_0(k-1)-Δy mv (k)] …(14)

[0174] Vu_2(k)

[0175] =[xu_2(k) yu_2(k)]

[0176] =[xu_1(k-1)-Δx mv (k) yu_1(k-1)-Δy mv (k)] …(15)

[0178] Vu_m u (k)

[0179] =[xu_m u (k) yu_m u (k)]

[0180] =[xu_m u -1(k-1)-Δx mv (k) yu_m u -1(k-1)-Δy mv (k)] …(16)

[0182] Figure 16 This is a diagram illustrating the user's (the follower's) field of vision. Preferably, when the mobile body 100 follows the user U1, the mobile body 100 enters the user U1's field of vision while moving. This is so that the user U1 can easily confirm the position of the mobile body 100 without turning around. Therefore, the first decision unit 220 determines the following mode benefit function based on the user U1's position and information related to the user U1's field of vision. Information related to the user U1's field of vision may, for example, be information related to the user U1's field of vision angle.

[0183] For example, the first decision unit 220 determines the relative position that the moving body 100 should follow relative to the user U1 based on the user U1's position and information related to the user U1's field of vision, and sets a following mode benefit function. Specifically, the first decision unit 220 calculates the offset Δxt in the X-axis direction and the offset Δyt in the Y-axis direction based on information related to the user U1's field of vision. Moreover, the first decision unit 220 calculates the position obtained by offsetting the user U1's position Vu_0(k) from the current time k along the X-axis direction by Δxt and along the Y-axis direction by Δyt as the target position Vt_0(k).

[0184] Figure 17 This is a diagram representing an example of multiple target locations. For example... Figure 17 As shown, the first decision unit 220 determines the position of user U1 from the current position to the past position Vu_0(k), ..., Vu_n. u (k) is used to calculate multiple target positions Vt_0(k), ..., Vt_n u (k). Multiple target locations Vt_0(k), ..., Vt_n u (k) represents the position Vu_0(k), ..., Vu_n relative to user U1 from the current position to the past position. u (k) The position obtained by offsetting Δxt along the X-axis and Δyt along the Y-axis.

[0185] Figure 18 This is a diagram illustrating an example of a state where the follower pattern's reward function is set at multiple target locations. For example... Figure 18 As shown, the first decision unit 220 calculates multiple target positions Vt_0(k), ..., Vt_n u (k) Each position sets the follow-mode benefit function. Thus, the moving body 100 is recommended to be at multiple target positions Vt_0(k), ..., Vt_n. u (k) travels around.

[0186] As mentioned above, the coordinates of the predicted position on the trajectory generated by the generation unit 260 are represented as Z. mj (X) mj Y mj Additionally, based on the coordinates of multiple target locations and the coordinates Z of the predicted location... mj (X) mj Y mj The return function value is calculated using this method.

[0187] Figure 19 This is a graph representing an example of the follower pattern payoff function along the X-axis. Figure 19 The horizontal axis Δxu_b represents the X-coordinate of the target position and the X-coordinate of the predicted position on the trajectory generated by the generator 260.mj The difference. It should be noted that the first decision unit 220 defines the following mode reward function Pc_b_x in the X-axis direction in a manner that has a gradient in the second sign direction (negative direction).

[0188] like Figure 19 As shown, the follower mode profit function Pc_b_x in the X-axis direction is a function whose profit value decreases as Δxu_b approaches 0.

[0189] Figure 20 This is a graph representing an example of the follower pattern payoff function along the Y-axis. Figure 20 The horizontal axis Δyu_b represents the Y-coordinate of the target position and the Y-coordinate of the predicted position on the trajectory generated by the generation unit 260. mj The difference. It should be noted that the first decision unit 220 defines the following mode reward function Pc_b_y in the Y-axis direction in a manner that has a gradient in the second sign direction (negative direction).

[0190] like Figure 20 As shown, the follower mode reward function Pc_b_y in the Y-axis direction is a function whose reward function value decreases as Δyu_b approaches 0.

[0191] Next, we will explain how to calculate the reward function value of the following pattern. First, we will consider the multiple target positions Vt_l' (l'=0, 1, ..., n) at the current time k. u The coordinates of ) are calculated based on equations (17) to (19).

[0192] Vt_0(k)=[xt_0(k) yt_0(k)]

[0193] =[xu_0(k)+Δxt yu_0(k)+Δyt]…(17)

[0195] Vt_1(k)=[xt_1(k) yt_1(k)]

[0196] =[xu_1(k)+Δxt yu_1(k)+Δyt]…(18)

[0198] Vt_n u (k) = [xt_n] u (k) yt_n u (k)]

[0199] =[xu_n u (k) + Δxt yu_n u (k)+Δyt] …(19)

[0201] Next, based on multiple target positions Vt_l', the coordinates Z of the predicted position on the trajectory generated by the generator 260 are calculated. mj (X) mj Y mj (j=1, 2, ..., n) m The return function value of ).

[0202] Specifically, when the X coordinate of the predicted position on the track generated by the generation unit 260 is set to X... mj (k) and when the X coordinate of the target position is set as xt_l'(k), their difference Δxu_b_l'(k) is expressed as in equation (20).

[0203] Δxu_b_l'(k)=X mj (k)-xt_l'(k)...(20)

[0204] The first decision unit 220 calculates Δxu_b_l'(k) based on equation (20). Furthermore, the first decision unit 220... Figure 19 The following model profit function shown in the X-axis direction is used to calculate the profit function value Pc_b_x_j_l'(k) corresponding to the calculated Δxu_b_l'(k).

[0205] Additionally, when the Y-coordinate of the predicted position on the track generated by the generation unit 260 is set to Y... mj (k) and when the Y coordinate of the target position is set as yt_l'(k), their difference Δyu_b_l'(k) is expressed as in equation (21).

[0206] Δyu_b_l'(k)=Y mj (k)-yt_l'(k)...(21)

[0207] The control device 200 calculates Δyu_b_l'(k) based on equation (21). Additionally, the control device 200 calculates Δyu_b_l'(k) based on... Figure 20 The following model profit function shown in the Y-axis direction is used to calculate the profit function value Pc_b_y_j_l'(k) corresponding to the calculated Δyu_b_l'(k).

[0208] Furthermore, the individual payoff function values ​​Pc_b_j_l'(k) for multiple target locations can be expressed as shown in equation (22). That is, Pc_b_j_l'(k) can be obtained by multiplying Pc_b_x_j_l'(k) by Pc_b_y_j_l'(k). It should be noted that the payoff function value Pc_b_j_l'(k) can be labeled with a negative value.

[0209] Pc_b_j_l'(k)=Pc_b_x_j_l'(k)×Pc_b_y_j_l'(k)...(22)

[0211] Furthermore, the total reward function value Pc_b_j(k) for the coordinates of the predicted positions on the predicted trajectory can be expressed as in equation (23). That is, by using the reward function values ​​Pc_b_j_l'(k) for all target positions (l'=0, 1, ..., n) u In total, Pc_b_j(k) can be calculated.

[0212] Pc_b_j(k)=Σ l’=0 nu (Pc_b_j_l'(k)) ...(23)

[0213] Furthermore, the follower pattern reward function value Pc_b(k) can be expressed as shown in equation (24). That is, it is expressed as the total reward function value Pc_b_j(k) (j=1, 2, ..., n) for the coordinates of all predicted positions on the predicted trajectory. m In total, the follower pattern profit function value Pc_b(k) can be obtained.

[0214] Pc_b(k) = Σ j=1 nm (Pc_b_j(k))…(24)

[0215] [Belt and Road Initiative Revenue Function]

[0216] Next, we will explain the routing mode revenue function. The routing mode revenue function is a function that represents the degree of recommended travel of the mobile body 100, determined based on the user U1 (the guide object) from the current to the past position in the routing mode.

[0217] Figure 21 This is a graph used to illustrate the revenue function of the belt-mode. For example... Figure 21 As shown, the direction of travel of the moving body 100 is set as the Y-axis, and the direction orthogonal to the Y-axis is set as the X-axis. There are multiple environmental object targets R6 and R7 around the moving body 100, so the moving body 100 needs to guide the user U1 in a way that does not interfere with these multiple environmental object targets R6 and R7.

[0218] In the wayfinding mode, mobile body 100 guides elderly people, people with leg injuries, and other individuals with mobility impairments as users U1. These individuals often have difficulty moving nimbly to the left or right and sometimes struggle to navigate through crowds. To ensure these individuals can walk safely in crowds, mobile body 100 guides them as users U1, thus providing a smooth walking space with minimal lateral movement. Furthermore, mobile body 100 infers the direction the individual intends to travel based on their movement trajectory and guides them in that direction. It should be noted that the guidance provided by mobile body 100 is not limited to individuals with mobility impairments; it can also be used for pedestrians unfamiliar with their destination in specific facilities (stations, airports, shopping malls, etc.).

[0219] The first detection unit 210 detects the position Vu_0(k) of user U1 at the current time k based on the detection result of the detection device 120. The first detection unit 210 repeatedly detects the position Vu_0(k) of user U1 at preset control intervals. It should be noted that the position Vu_0(k) of user U1 is detected as the relative position (position in the XY coordinate system) of user U1 relative to the moving body 100. Figure 21 In this context, Vu_1(k) represents the position of user U1 relative to the control time before Vu_0(k)1, and Vu_2(k) represents the position of user U1 relative to the control time before Vu_1(k)1. It should be noted that Vu_n... u (k) represents the position of the most recent user U1.

[0220] The first detection department 210 detects user U1's positions from the current position to the past positions Vu_0(k), Vu_1(k), Vu_2(k), ..., Vu_n. u (k). At this time, the moving body 100 is also actually moving, so it is necessary to correct the previous detection position of user U1 based on the amount of movement of the moving body 100. The method for correcting the previous detection position of user U1 is the same as that used... Figure 15 The same method is used to explain equations (13) to (16), so the explanation is omitted.

[0221] Figure 22 This is a diagram illustrating the estimation process for the predicted trajectory of user U1. The estimation unit 225 is based on user U1's positions Vu_0(k), Vu_1(k), ..., Vu_n from the current position to the past. u(k) is used to estimate the predicted trajectory of user U1. The path that the moving body 100 travels along the guide is ideally a curved path with minimal left and right movement. Therefore, the estimation unit 225 estimates the predicted trajectory of user U1 based on a model representing a portion of the outer perimeter of the geometric shape from the current position to the past position. Specifically, the estimation unit 225 can use a circular arc model to estimate the circularized predicted trajectory. For example, the estimation unit 225 can, as described above... Figure 5 As illustrated, the predicted trajectory of user U1 in the bandpass mode is estimated by combining three arcs.

[0222] exist Figure 22 In the example shown, the estimation unit 225 estimates the predicted trajectory of user U1 by combining the first to third predicted trajectories, which are in the shape of arcs. The estimation unit 225 determines the first to third predicted trajectories by appropriately setting the radius of curvature ρm1 of the first predicted trajectory, the radius of curvature ρm2 of the second predicted trajectory, and the radius of curvature ρm3 of the third predicted trajectory.

[0223] Specifically, the estimation unit 225 determines the first predicted trajectory in a way that reduces the deviation from the past movement trajectory of user U1 and reduces the risk of interference between the moving body 100 and surrounding traffic participants U5-U7. Furthermore, the estimation unit 225 determines the second and third predicted trajectories in a way that reduces the risk of interference between the moving body 100 and surrounding traffic participants U5-U7. For example, the estimation unit 225 can determine the curvature or radius of curvature of the predicted trajectory by varying the degree of risk obtained from a risk function set by the positions of traffic participants U5-U7 in the second sign direction (negative direction), thereby determining the predicted trajectory. This risk function is set in a way that has a gradient in the first sign direction (positive direction).

[0224] exist Figure 22 In the process, the estimation unit 225 calculates the leading edge of the third predicted trajectory as the target position Vt_0(k) at the current time k. Furthermore, the estimation unit 225 considers the moving speed and control cycle of traffic participants U5~U7 and calculates the target position Vt_j(k) (j=1, 2, ..., m) along the first, second, and third predicted trajectories. u ).

[0225] Figure 23 This is a diagram illustrating an example of a route mode revenue function set at multiple target locations. For example... Figure 23 As shown, the first decision unit 220 determines multiple target positions Vt_0(k), ..., Vt_m on the predicted trajectory estimated by the estimation unit 225. u (k) The respective positions define the routing mode benefit function. Therefore, the moving body 100 is recommended to be at multiple target positions Vt_0(k), ..., Vt_m.u (k) travels around.

[0226] As mentioned above, the coordinates of the predicted position on the trajectory generated by the generation unit 260 are represented as Z. mj (X) mj Y mj Additionally, based on the coordinates of multiple target locations and the coordinates Z of the predicted location... mj (X) mj Y mj The return function value is calculated using this method.

[0227] Figure 24 This is a diagram representing an example of the band-mode revenue function along the X-axis. Figure 24 The horizontal axis Δxu_b represents the X-coordinate of the target position and the X-coordinate of the predicted position on the trajectory generated by the generator 260. mj The difference. It should be noted that the first decision unit 220 defines the bandway mode benefit function Pl_b_x in the X-axis direction in a manner that has a gradient in the second sign direction (negative direction).

[0228] like Figure 24 As shown, the band mode profit function Pl_b_x in the X-axis direction is a function whose profit value decreases as Δxu_b approaches 0.

[0229] Figure 25 This is a diagram representing an example of the band-mode revenue function along the Y-axis. Figure 25 The horizontal axis Δyu_b represents the Y-coordinate of the target position and the Y-coordinate of the predicted position on the trajectory generated by the generation unit 260. mj The difference. It should be noted that the first decision unit 220 defines the band mode profit function Pl_b_y in the Y-axis direction in a manner that has a gradient in the second sign direction (negative direction).

[0230] like Figure 25 As shown, the band mode profit function Pl_b_y in the Y-axis direction is a function whose profit value decreases as Δyu_b approaches 0.

[0231] Next, we will explain how to calculate the profit function value of the route mode. First, we define multiple target positions Vt_l' (l'=0, 1, ..., m) at the current time k as shown in equation (25). u The coordinates of ).

[0232] Vt_l'(k)=[xt_l'(k) yt_l'(k)]...(25)

[0233] Next, based on multiple target positions Vt_l'(k), the coordinates Z of the predicted position on the trajectory generated by the generator 260 are calculated.mj (X) mj Y mj (j=1, 2, ..., n) m The return function value of ).

[0234] Specifically, the X coordinates of the predicted position on the orbit generated by the generation unit 260 are... mj The difference Δxu_b_l'(k) between (k) and the X coordinate xt_l'(k) of the target position is expressed as in equation (26).

[0235] Δxu_b_l'(k)=X mj (k)-xt_l'(k)...(26)

[0236] The first decision unit 220 calculates Δxu_b_l'(k) based on equation (26). Furthermore, the first decision unit 220... Figure 24 The benefit function of the band mode shown in the X-axis direction is used to calculate the benefit function value Pl_b_x_j_l'(k) corresponding to the calculated Δxu_b_l'(k).

[0237] Additionally, the Y-coordinate of the predicted position on the track generated by the generation unit 260 mj The difference Δyu_b_l'(k) between (k) and the Y coordinate yt_l'(k) of the target position is expressed as in equation (27).

[0238] Δyu_b_l'(k)=Y mj (k)-yt_l'(k)...(27)

[0239] The first decision unit 220 calculates Δyu_b_l'(k) based on equation (27). Furthermore, the first decision unit 220... Figure 25 The benefit function of the band mode in the Y-axis direction is shown to obtain the benefit function value Pl_b_y_j_l'(k) corresponding to the calculated Δyu_b_l'(k).

[0240] Furthermore, the profit function value Pl_b_j_l'(k) for each of the multiple target locations can be expressed as shown in equation (28). That is, Pl_b_j_l'(k) can be obtained by multiplying Pl_b_x_j_l'(k) by Pl_b_y_j_l'(k). It should be noted that the profit function value Pl_b_j_l'(k) can be labeled with a negative value.

[0241] Pl_b_j_l'(k)=Pl_b_x_j_l'(k)×Pl_b_y_j_l'(k)...(28)

[0243] Furthermore, the total reward function value Pl_b_j(k) for the coordinates of the predicted positions on the predicted trajectory can be expressed as shown in equation (29). That is, it can be expressed by the reward function values ​​Pl_b_j_l'(k) for all target positions (l'=0, 1, ..., m). u The total is used to calculate Pl_b_j(k).

[0244] Pl_b_j(k) = Σ l’=0 mu (Pl_b_j_l'(k))…(29)

[0245] Furthermore, the route mode revenue function value Pl_b(k) can be expressed as shown in equation (30). That is, it can be expressed by the total revenue function value Pl_b_j(k) (j=1, 2, ..., n) for the coordinates of all predicted positions on the predicted trajectory. m The total value of the revenue function Pl_b(k) for the route mode is calculated.

[0246] Pl_b(k) = Σ j=1 nm (Pl_b_j(k))…(30)

[0247] [Remote Operation Revenue Function]

[0248] Next, the remote operation benefit function will be explained. The remote operation benefit function is a function that represents the recommended degree of movement of the mobile body 100, determined based on the user U2's operations on the mobile body 100. Furthermore, the remote operation benefit function is a function set during remote operation mode.

[0249] Figure 26 This diagram illustrates the situation where the mobile body 100 is automatically moving in remote operation mode. For example... Figure 26 As shown, the direction of travel of the moving body 100 is set as the Y-axis, and the direction orthogonal to the Y-axis is set as the X-axis. There are multiple environmental objects R8~R10 and traffic participants U8 (hereinafter referred to as pedestrians U8) around the moving body 100, so the moving body 100 needs to travel in a way that does not interfere with these multiple environmental objects R8~R10 and pedestrians U8.

[0250] In remote operation mode, user U2 can remotely operate mobile body 100 using information processing device 400. Specifically, information processing device 400 sends operation information obtained based on user U2's operation to mobile body 100, and mobile body 100 moves according to the operation information received from information processing device 400. It should be noted that, if a destination is preset, mobile body 100 automatically moves towards the destination while receiving remote operation from user U2.

[0251] When the operation mode of the mobile body 100 is switched to remote operation mode, the first decision unit 220 determines whether a destination has been preset. For example, user U1 may set the destination using user terminal device 300, or user U2 may set the destination using information processing device 400. If it is determined that a destination has been preset, the first decision unit 220 determines an approximate path to the destination based on the destination and map information, and sets a reward function based on environmental object targets along the path. On the other hand, the first decision unit 220 does not set a reward function based on environmental object targets for areas outside the path.

[0252] Therefore, in Figure 26 In the example shown, the first decision unit 220 does not set a benefit function based on the environmental object targets in the space S between the environmental object targets R8 and R9. If a benefit function based on the environmental object targets were set in space S, the mobile body 100 would be recommended to travel at the location where the benefit function is set, and therefore might not go to the destination but travel straight. Thus, the first decision unit 220 determines a general path to the destination based on destination information and map information, and sets a benefit function based on the environmental object targets along that path, thereby enabling the mobile body 100 to be guided to the destination more reliably.

[0253] exist Figure 26 In the example shown, the mobile body 100, without the operation of user U2, travels along path A1 corresponding to the position where the environmental object target benefit function is set, while avoiding interference objects (such as pedestrian U8, environmental object targets R8~R10).

[0254] In remote operation mode, the control device 200 controls the direction of travel of the mobile body 100 based on the operation instructions from the joystick or the like used by the user U2 using the information processing device 400. However, when the mobile body 100 is operated remotely in a purely manual manner based on the user U2, the communication delay between the mobile body 100 and the information processing device 400 puts pressure on the user U2. Therefore, in this embodiment, the mobile body 100 basically travels automatically, changing its path left or right according to the operation instructions received from the user U2. This reduces the pressure on the user U2 in operating the mobile body 100.

[0255] Figure 27 This diagram illustrates a scenario where user U2 instructs the moving body 100 to move to the left. Figure 27In the example shown, when the mobile body 100 is traveling along path A1, the distance between the mobile body 100 and the pedestrian U8 is short, so moving the mobile body 100 to the left allows for more reliable avoidance of the pedestrian U8. Therefore, when the user U2 instructs the information processing device 400 to move the mobile body 100 to the left, the first decision unit 220 sets a remote operation benefit function in the region to the left (the negative direction on the X-axis) relative to the direction of travel of the mobile body 100. The mobile body 100 is recommended to travel near the location where the benefit function is set, thus changing the path of travel from path A1 to path A2. As a result, the mobile body 100 can travel towards its destination while more reliably avoiding the pedestrian U8.

[0256] Next, we will explain how to calculate the remote operation benefit function value. As mentioned earlier, the coordinates of the predicted position on the predicted trajectory in the XY coordinate system are represented as Z. mj (X) mj Y mj Additionally, the parameters of the operation information obtained based on the user U2's operation on the information processing device 400 are set as xop_j (j=1, 2, ..., n). m For example, the parameter xop_j can be calculated based on the amount of operation (travel) of the joystick, etc., by the user U2, the number of operations in the operation direction, the operation time in the operation direction, etc. It should be noted that the right direction is set as the positive direction of parameter xop_j relative to the travel direction of the moving body 100, and the left direction is set as the negative direction of parameter xop_j relative to the travel direction of the moving body 100.

[0257] The first detection unit 210 calculates the parameter xop_j based on operation information obtained from user U2's operation of the joystick, etc., and outputs the parameter xop_j to the first decision unit 220. Thus, the first detection unit 210 can detect user U2's operation on the moving body 100. Furthermore, the first decision unit 220 determines the position based on the parameter xop_j and the predicted X-coordinate. mj To calculate the return function value. This point will be explained below.

[0258] Figure 28 This is a graph representing an example of a remote operation payoff function. For example... Figure 28 As shown, the first decision unit 220 defines the remote operation benefit function Pe_b_op_j (j=1, 2, ..., n) in such a way that it has a gradient in the second sign direction (negative direction). m The remote operation benefit function Pe_b_op_j is a function whose value decreases as Δxop_j approaches 0.

[0259] Here, Δxop_j is the parameter xop_j and the X-coordinate of the predicted position on the trajectory generated by the generator 260.mj The difference. Therefore, the difference Δxop_j(k) at the current time k is expressed as in equation (31).

[0260] Δxop_j(k) = X mj (k)-xop_j(k)…(31)

[0261] The first decision unit 220 calculates Δxop_j(k) based on equation (31). Furthermore, the first decision unit 220 calculates Δxop_j(k) based on equation (31). Figure 28 The remote operation revenue function shown is used to calculate the revenue function value Pe_b_op_j(k) corresponding to the calculated Δxop_j(k).

[0262] Furthermore, the remote operation benefit function value Pe_b_op(k) can be expressed as shown in equation (32). That is, by using the benefit function value Pe_b_op_j(k) (j=1, 2, ..., n) for the coordinates of all predicted positions on the predicted orbit. m In total, the remote operation benefit function value Pe_b_op(k) can be obtained.

[0263] Pe_b_op(k) = Σ j=1 nm (Pe_b_op_j(k))…(32)

[0264] Figure 29 This diagram illustrates the change in the direction of travel of the moving body 100 when a remote operation benefit function is set. (Example) Figure 29 As shown, for example, when user U2 uses information processing device 400 to instruct the mobile body 100 to move in the right direction, a remote operation benefit function is set on the right side (positive direction side on the X-axis) relative to the direction of travel of the mobile body 100.

[0265] Furthermore, the mobile body 100 is recommended to travel at the position where the remote operation benefit function value is minimized (becoming the position where Δxop_j=0), therefore the direction of travel is changed to the right. In this way, the first decision unit 220 determines the remote operation benefit function, thereby enabling the mobile body 100 to change its direction of travel according to the operation of the user U2.

[0266] [Specific examples of model change]

[0267] Next, as a specific example of a mode change, we will illustrate the example of user U1 and user U2 taking a walk together on the beach. It should be noted that, suppose user U1 is actually taking a walk on the beach, but user U2 is logging into mobile device 100 and taking a walk on the beach.

[0268] First, the pairing unit 290 of the mobile body 100 pairs the user U1 with the mobile body 100 based on the user U1's operation. For example, the pairing unit 290 may also pair the user U1, who has undergone biometric authentication based on a biometric authentication function, with the mobile body 100. Specifically, the pairing unit 290 may also perform facial authentication based on a facial image of the user captured by the camera 80, thereby identifying the user U1, and pairing the identified user U1 with the mobile body 100. It should be noted that the biometric authentication method is not limited to this. For example, the mobile body 100 may also be equipped with a vein sensor, and the pairing unit 290 may also perform vein authentication based on the output from the vein sensor. In addition, the pairing unit 290 may also identify the user U1 through communication with the user terminal device 300 and pair the identified user U1 with the mobile body 100. The user U1 paired with the mobile body 100 becomes a follower in follow mode and a guide in guide mode.

[0269] Next, the mode switching unit 285 of the mobile body 100 sets either a follow mode or a guide mode according to the operation of the user U1. In follow mode, the mobile body 100 follows the user U1. In guide mode, the mobile body 100 guides the user U1. Afterwards, the mobile body 100 and the user U1 move together to the beach where they are scheduled to take a walk with the user U2.

[0270] Next, the acquisition unit 280 of the mobile unit 100 acquires a remote operation request from the information processing device 400, indicating the intention to remotely operate the mobile unit 100. For example, the acquisition unit 280 may acquire a login request sent from the information processing device 400 as a remote operation request. When the acquisition unit 280 acquires a login request from the information processing device 400, it performs login processing for the user U2. For example, the login request may include the user U2's ID and password, and authentication of the user U2 is performed based on this information. When the authentication of the user U2 is completed, the mode switching unit 285 switches the operation mode of the mobile unit 100 to the remote operation mode.

[0271] Thus, the mode switching unit 285 switches the operation mode of the mobile body 100 to the remote operation mode when a remote operation request (e.g., a login request) is received during the operation of the mobile body 100 in follow mode or guide mode. As a result, the user U2 can remotely operate the mobile body 100.

[0272] When the operation mode of the mobile unit 100 is switched to remote operation mode, the image of user U2 is displayed on the display unit of HMI 110, and the sound of user U2 is output from the speaker of HMI 110. For example, information processing device 400 can use a camera installed on information processing device 400 to photograph user U2 and send the photographed image of user U2 to mobile unit 100. In addition, the display unit of HMI 110 can display the image of user U2 sent from information processing device 400. In addition, information processing device 400 can use a microphone installed on information processing device 400 to acquire the sound data of user U2 and send the acquired sound data of user U2 to mobile unit 100. In addition, the speaker of HMI 110 can output the sound of user U2 based on the sound data sent from information processing device 400. Similarly, mobile unit 100 can send the image and sound data of user U1 to information processing device 400. The information processing device 400 can display the image of user U1 transmitted from mobile body 100 on a display unit, and output the voice of user U1 from a speaker based on the sound data transmitted from mobile body 100. Thus, user U1 can have a conversation with user U2.

[0273] In remote operation mode, destinations such as seaside attractions can be preset. When a destination is set, the mobile unit 100 basically travels to the destination automatically, but the user U2 can operate the mobile unit 100 using a joystick or the like. However, the limiting unit 295 restricts the range within which the user U2 can operate the mobile unit 100 to a predetermined range from the location of the paired user U1. This prevents the user U2 from operating the mobile unit 100 in a manner similar to following pedestrians other than the user U1.

[0274] In this way, the operation mode of the mobile body 100 is switched to remote operation mode according to the remote operation request (login request) from the information processing device 400, so that even if user U2 is in a remote location, user U1 and user U2 can have the feeling of walking together on the beach.

[0275] It should be noted that the mode switching unit 285 can also switch the operation mode of the mobile body 100 to either the following mode or the guiding mode based on the actions of the user U1. For example, the mode switching unit 285 can also switch the operation mode of the mobile body 100 to the guiding mode when it detects that the user U1 is moving in a direction that deviates from the prescribed path (e.g., deviating from the route to the destination). This eliminates the need to manually switch the operation mode of the mobile body 100.

[0276] Furthermore, the mode switching unit 285 can also switch the operation mode of the mobile body 100 to the navigation mode if it detects that the user U1 is lost. For example, the mode switching unit 285 can detect that the user U1 is lost based on an image of the user U1 captured by the camera 80. This eliminates the need to manually switch the operation mode of the mobile body 100.

[0277] Furthermore, the mode switching unit 285 can also switch the motion mode of the mobile body 100 to a following mode when it detects that the user U1 is about to overtake the mobile body 100. For example, the mode switching unit 285 can detect that the user U1 is about to overtake the mobile body 100 based on an image of the user U1 captured by the camera 80. As a result, the effort of manually switching the motion mode of the mobile body 100 can be eliminated.

[0278] It should be noted that the mode switching unit 285 can also propose to switch the operation mode of the mobile body 100 to either the follow mode or the guide mode based on the user U1's behavior, using sound or other means. This prevents the mobile body 100's operation mode from being switched against the user U1's wishes.

[0279] Furthermore, the mode switching unit 285 can also change the orientation of the display unit showing the image of user U2 to the direction of user U1 when the operation mode of the mobile body 100 is switched to either the follow mode or the guide mode. Thus, even when the operation mode of the mobile body 100 is switched, user U1 can walk while looking at user U2's face.

[0280] [Calculation and processing of the evaluation function, etc.]

[0281] Next, the calculation process of the evaluation function will be explained. The calculation unit 250 calculates the evaluation function J(k) based on the profit function determined by the first decision unit 220 and the risk function determined by the second decision unit 240. Specifically, in the case of the follower mode, the calculation unit 250 calculates the evaluation function J(k) according to the following equation (33). That is, the calculation unit 250 calculates the evaluation function J(k) by adding Pc_b(k), Pe_b(k), Ptp_r(k), and Pe_r(k) together.

[0282] J(k)=Pc_b(k)+Pe_b(k)+Ptp_r(k)+Pe_r(k)…(33)

[0284] On the other hand, in the case of the bandpass mode, the calculation unit 250 calculates the evaluation function J(k) according to the following equation (34). That is, the calculation unit 250 calculates the evaluation function J(k) by adding Pl_b(k), Pe_b(k), Ptp_r(k) and Pe_r(k) together.

[0285] J(k)=Pl_b(k)+Pe_b(k)+Ptp_r(k)+Pe_r(k)…(34)

[0287] On the other hand, in the remote operation mode, the calculation unit 250 calculates the evaluation function J(k) according to the following formula (35). That is, the calculation unit 250 calculates the evaluation function J(k) by adding Pe_b_op(k), Pe_b(k), Ptp_r(k) and Pe_r(k).

[0288] J(k)=Pe_b_op(k)+Pe_b(k)+Ptp_r(k)+Pe_r(k)…(35)

[0289] The benefit functions Pc_b(k), Pl_b(k), Pe_b_op(k), and Pe_b(k) are negative; the smaller the benefit function value, the higher the recommended travel degree of the moving body 100. The risk functions Ptp_r(k) and Pe_r(k) are positive; the larger the risk function value, the higher the risk of interference between the moving body 100 and the object being interfered with. Therefore, the generation unit 260 needs to generate the target trajectory of the moving body 100 in a way that reduces the evaluation value calculated based on the evaluation function J(k).

[0290] As described above, the generation unit 260 Figure 5 As shown, the trajectory of the moving body 100 is modeled using an arc model, and the curvature or radius of curvature of the arc is calculated by varying the evaluation function J(k) in the second sign direction (negative direction), thereby generating the target trajectory. As previously described, the generation unit 260 determines... Figure 5 rotation angle θ m1 ~θ m3 and radius of curvature R m1 ~R m3 Multiple patterns are combined to generate multiple tracks. Furthermore, the generation unit 260 uses the evaluation function J(k) calculated by the calculation unit 250 to calculate the evaluation value for each of the multiple generated tracks, and generates the track with the smallest evaluation value as the target track.

[0291] The control unit 270 controls the mobile body 100 based on the target track generated by the generation unit 260. Specifically, the control unit 270 controls the movement mechanism 140 (drive motor, steering device, etc.) in such a way that the mobile body 100 travels on the target track generated by the generation unit 260. Thus, the control device 200 can control the mobile body 100 to travel on the target track where the risk of interference between the mobile body 100 and the object of interference (pedestrian, obstacle, etc.) is low and suitable for the set mode (following mode, guiding mode, or remote operation mode).

[0292] It should be noted that the control unit 270 controls the speed of the mobile body 100 and the target distance between the mobile body 100 and the user U1. For example, when the user U1 and the mobile body 100 are paired, the control unit 270 can control the mobile body 100 to shorten the distance between the user U1 and the mobile body 100 compared to the case where they are not paired. This allows the mobile body 100 to move in close following of the paired user U1. Furthermore, the control unit 270 can also change the target distance between the mobile body 100 and the user U1 based on whether there is a conversation between them. Specifically, if the mobile body 100 is equipped with a microphone, and the control unit 270 determines that the user U1 and the mobile body 100 are having a conversation using the microphone, it can increase the target distance compared to the case where no conversation is determined. The control unit 270 can also calculate the speed of user U1 based on the trajectory of user U1 from the past to the present, and calculate the speed of the moving body 100 based on the calculated speed of user U1, and control the speed in a way that ensures the target distance between the moving body and user U1. Specifically, the control unit 270 can either directly set the calculated speed of user U1 as the target speed of the moving body 100, or it can set the speed obtained by subtracting a predetermined speed from the speed of user U1 as the target speed of the moving body 100.

[0293] According to the control device 200 of this embodiment, the target trajectory of the moving body 100 is generated by modeling the trajectory of the moving body 100 using an arc model, thus enabling the movement of the moving body 100 to be smooth. Therefore, traffic participants (pedestrians, etc.) can easily predict the movement of the moving body 100, and contact between the moving body 100 and traffic participants can be effectively prevented.

[0294] [flow chart]

[0295] Figure 30 This is a flowchart illustrating a first example of a process performed by the control device 200. The process based on this flowchart is performed according to the pairing of user U1 with the mobile body 100.

[0296] First, the mode switching unit 285 sets either the follow mode or the lead mode based on instructions from user U1 (the first user) (step S101). For example, the mode switching unit 285 can accept instructions from user U1 based on user U1's operation of HMI 110. Alternatively, the mode switching unit 285 can also accept instructions from user U1 by detecting user U1's gestures using camera 80. Furthermore, the mode switching unit 285 can also accept instructions from user U1 based on information input by user U1 into user terminal device 300. Moreover, the mode switching unit 285 can also use voice recognition to recognize the voice emitted by the user and accept instructions from user U1 based on the recognized voice.

[0297] Next, the acquisition unit 280 waits until it receives a remote operation request indicating its intention to remotely operate the mobile body 100 (step S102). For example, the remote operation request may be a login request sent from the information processing device 400 to the mobile body 100.

[0298] Next, the mode switching unit 285 displays the image of user U2 (second user) on the display unit of HMI 110 (step S103). For example, the mode switching unit 285 can display the image of user U2 received from information processing device 400 on the display unit of HMI 110.

[0299] Next, the mode switching unit 285 waits until the predetermined conditions are met (step S104). If the predetermined conditions are met, the mode switching unit 285 switches the operation mode of the mobile body 100 to the remote operation mode (step S105). The predetermined conditions may include, for example, a predetermined time elapsed since the login request was received, an instruction for mode switching by user U1 or user U2, or the mobile body 100 entering an area within a predetermined distance of the destination. Therefore, the operation mode of the mobile body 100 can be switched to the remote operation mode at an appropriate time.

[0300] It should be noted that the processing of step S104 can also be omitted. In this case, the mode switching unit 285 can quickly switch the operation mode of the mobile body 100 to the remote operation mode based on the fact that a remote operation request has been obtained.

[0301] Figure 31 This is a flowchart illustrating a second example of the processing performed by the control device 200. The processing based on this flowchart is repeatedly executed at regular intervals starting from the user U1 (first user) initiating the movement 100.

[0302] First, the control device 200 determines whether pairing of user U1 and mobile body 100 has been performed (step S201). As described above, the pairing unit 290 pairs user U1, who has undergone biometric authentication (vein authentication, facial authentication, etc.), with mobile body 100. The control device 200 determines whether pairing has been performed by the pairing unit 290. If it is determined in step S201 that pairing of user U1 and mobile body 100 has not been performed, the control device 200 terminates the processing based on this flowchart.

[0303] On the other hand, if it is determined in step S201 that user U1 and mobile body 100 have been paired, the limiting unit 295 restricts the range in which user U2 (the second user) can operate mobile body 100 to a predetermined range from the position of the paired user U1 (step S202). As a result, it is possible to prevent user U2 from operating mobile body 100 in a manner that follows pedestrians other than user U1.

[0304] Next, the control unit 270 controls the mobile body 100 in such a way that the distance between the user U1 and the mobile body 100 is shorter than when they are not paired (step S203). As a result, the mobile body 100 can move in a way that closely follows the paired user U1.

[0305] Figure 32 This is a flowchart illustrating a third example of the processing performed by the control device 200. The processing based on this flowchart is repeatedly executed at regular intervals starting from the user U1 (first user) initiating the movement 100.

[0306] First, the mode switching unit 285 of the control device 200 determines what the operating mode of the mobile body 100 is (step S301). Specifically, the mode switching unit 285 determines which mode the operating mode of the mobile body 100 is among following mode, guiding mode, and other modes (modes other than following mode and guiding mode).

[0307] If, in step S301, it is determined that the operating mode of the mobile body 100 is the following mode, the mode switching unit 285 determines whether the user U1 is moving in a direction that deviates from the prescribed path (e.g., deviating from the route to the destination) (step S302). If it is determined that the user U1 is moving in a direction that deviates from the prescribed path, the mode switching unit 285 switches the operating mode of the mobile body 100 to the guiding mode (step S304). This eliminates the need for manually switching the operating mode of the mobile body 100.

[0308] If, in step S302, it is determined that user U1 has not moved in a direction deviating from the prescribed path, the mode switching unit 285 determines whether user U1 is lost (step S303). For example, the mode switching unit 285 can detect whether user U1 is lost based on an image of user U1 captured by camera 80. If it is determined that user U1 is lost, the mode switching unit 285 switches the operation mode of the mobile body 100 to the navigation mode (step S304). This eliminates the need to manually switch the operation mode of the mobile body 100.

[0309] On the other hand, if it is determined in step S303 that user U1 is not lost, the mode switching unit 285 ends the processing based on this flowchart.

[0310] Furthermore, if it is determined in step S301 that the operating mode of the mobile body 100 is the following mode, the mode switching unit 285 determines whether the user U1 is about to overtake the mobile body 100 (step S305). For example, the mode switching unit 285 can detect that the user U1 is about to overtake the mobile body 100 based on the image of the user U1 captured by the camera 80. If it is determined that the user U1 is about to overtake the mobile body 100, the mode switching unit 285 switches the operating mode of the mobile body 100 to the following mode. This eliminates the need to manually switch the operating mode of the mobile body 100.

[0311] On the other hand, if it is determined in step S305 that user U1 does not intend to overtake moving body 100, the mode switching unit 285 ends the processing based on this flowchart.

[0312] In addition, if it is determined in step S301 that the movement mode of the moving body 100 is another mode (a mode other than the following mode and the guiding mode), the mode switching unit 285 ends the processing based on this flowchart.

[0313] As explained above, the control device 200 of this embodiment is a device that switches between a following mode (following user U1, the first user), a guiding mode (leading user U1), and a remote operation mode (user U2, the second user) to operate the mobile body 100, thereby enabling the mobile body 100 to move. It includes an acquisition unit 280, a mode switching unit 285, a generation unit 260, and a control unit 270. The acquisition unit 280 acquires a remote operation request indicating the intention to remotely operate the mobile body 100. The mode switching unit 285 switches the operation mode of the mobile body 100 to the remote operation mode based on the fact that a remote operation request was acquired while the mobile body 100 is operating in either the following or guiding mode. The generation unit 260 generates a target track for the mobile body 100 based on the switched operation mode. The control unit 270 controls the mobile body 100 based on the generated target track. Therefore, the control device 200 of this embodiment enables the mobile body 100 to move by switching between the following mode and the remote operation mode, or between the guiding mode and the remote operation mode.

[0314] The implementation methods described above can be performed as follows.

[0315] A control device comprising:

[0316] Storage device, which stores a program; and

[0317] The hardware processor causes the mobile body to move by switching between a follow mode that follows a first user, a guide mode that leads the first user, and a remote operation mode that allows a second user to remotely operate the mobile body.

[0318] The hardware processor performs the following processing by executing a program stored in the storage device:

[0319] Remote operation requests to obtain the intent to remotely operate the mobile body;

[0320] Based on the fact that the remote operation request was obtained while the mobile body was operating in the following mode or the guiding mode, the operation mode of the mobile body is switched to the remote operation mode.

[0321] The target trajectory of the moving body is generated based on the switched action mode; and

[0322] The moving body is controlled based on the target trajectory.

[0323] The above description uses the embodiments described in relation to the specific implementation, but the present invention is not limited to such embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

[0324] [Explanation of Labels in the Attached Image]

[0325] 100 moving bodies

[0326] 200 control device

[0327] 210 First Testing Department

[0328] 220 First Decision Department

[0329] 225 Presumption Section

[0330] 230 Second Inspection Department

[0331] 240 Second Decision Department

[0332] 250 Calculation Department

[0333] 260 Generation Department

[0334] 270 Control Department

[0335] 280 Acquisition Department

[0336] 285 Mode Switching Unit

[0337] 290 Pairing Section

[0338] 295 Restriction Section

[0339] 300 User Terminal Devices

[0340] 400 Information processing device.

Claims

1. A control device that causes the mobile body to move by switching between a following mode that follows a first user, a guiding mode that guides the first user, and a remote operation mode that allows a second user to remotely operate the mobile body, wherein... The control device includes: The acquisition unit acquires remote operation requests that express the intention to remotely operate the mobile body; The mode switching unit switches the operation mode of the mobile body to the remote operation mode based on the fact that the remote operation request was obtained while the mobile body was operating in the following mode or the guiding mode. The generation unit generates the target trajectory of the moving body based on the switched action mode; as well as The control unit controls the moving body based on the target trajectory.

2. The control device according to claim 1, wherein, The control device also includes: The pairing unit performs pairing between the first user and the mobile body; and The limiting unit restricts the range within which the second user can operate the mobile body to a predetermined range from the position of the first user who has been paired.

3. The control device according to claim 2, wherein, When the pairing is performed, the control unit controls the mobile body in such a way that the distance between the first user and the mobile body is shortened compared to when the pairing is not performed.

4. The control device according to claim 1, wherein, The mode switching unit displays the image of the second user on the display unit provided on the mobile body according to the remote operation request, and then switches the operation mode of the mobile body to the remote operation mode according to the conditions being met.

5. The control device according to claim 1, wherein, The mode switching unit switches the movement mode of the mobile body to any one of the following mode and the guiding mode based on the behavior of the first user.

6. The control device according to claim 5, wherein, When the mode switching unit detects that the first user is moving in a direction that deviates from the prescribed path, it switches the movement mode of the moving body to the guiding mode.

7. The control device according to claim 5, wherein, When the mode switching unit detects that the first user is lost, it switches the movement mode of the mobile body to the navigation mode.

8. The control device according to claim 5, wherein, When the mode switching unit detects that the first user is about to overtake the moving body, it switches the movement mode of the moving body to the following mode.

9. A mobile system comprising a mobile body and an information processing device capable of communicating with the mobile body via a network, wherein, The mobile body is equipped with a control device that switches between a following mode that follows a first user, a guiding mode that leads the first user, and a remote operation mode that allows a second user to remotely operate the mobile body, thereby causing the mobile body to move. The control device includes: The acquisition unit obtains from the information processing device a remote operation request indicating the intention to remotely operate the mobile body; The mode switching unit switches the operation mode of the mobile body to the remote operation mode based on the fact that the remote operation request was obtained while the mobile body was operating in the following mode or the guiding mode. The generation unit generates the target trajectory of the moving body based on the switched action mode; as well as The control unit controls the moving body based on the target trajectory.

10. A control method, wherein processing is performed by a control device, wherein, The control device switches between a following mode that follows the first user and a remote operation mode that allows the second user to remotely operate the mobile body, thereby causing the mobile body to move. The control method causes the control device to perform the following processing: Remote operation requests to obtain the intent to remotely operate the mobile body; Based on the fact that the remote operation request was obtained during the movement of the mobile body in the following mode, the movement mode of the mobile body is switched from the following mode to the remote operation mode. The target trajectory of the moving body is generated based on the switched action mode; as well as The moving body is controlled based on the target trajectory.

11. A program that performs processing via a processor of a control device, wherein, The control device switches between a following mode that follows the first user and a remote operation mode that allows the second user to remotely operate the mobile body, thereby causing the mobile body to move. The program is used to cause the processor of the control device to perform the following processing: Remote operation requests to obtain the intent to remotely operate the mobile body; Based on the fact that the remote operation request was obtained during the movement of the mobile body in the following mode, the movement mode of the mobile body is switched from the following mode to the remote operation mode. The target trajectory of the moving body is generated based on the switched action mode; as well as The moving body is controlled based on the target trajectory.

Images