MOBILE BODY CONTROL DEVICE, MOBILE BODY, MOBILE BODY CONTROL METHOD, PROGRAM, AND LEARNING DEVICE

The mobile object control device and method address the challenge of object interference by determining routes that minimize movement vector changes and consider personal space, enabling compatible actions with surrounding objects through a reward-based learning approach.

JP7584385B2Active Publication Date: 2024-11-15HONDA MOTOR CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2021161960
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-09-30
Publication Date
2024-11-15
Estimated Expiration
2041-09-30

AI Technical Summary

Technical Problem

Existing technologies fail to account for the influence of a moving object on the behavior of surrounding objects, leading to potential interference and difficulty in accurately predicting their motion, which hinders compatible actions in crowded traffic environments.

Method used

A mobile object control device and method that determines a route to minimize changes in movement vectors of surrounding objects by employing a route determination unit and a control section to adjust the mobile object's path, considering the movement vectors of other objects and personal space, using a reward function to evaluate and learn optimal actions.

Benefits of technology

The solution enables the mobile object to take actions that are highly compatible with surrounding objects without predicting their future movements, reducing interference and considering personal space, thereby enhancing compatibility with other moving objects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007584385000003
    Figure 0007584385000003
  • Figure 0007584385000004
    Figure 0007584385000004
  • Figure 0007584385000005
    Figure 0007584385000005
Patent Text Reader

Abstract

To provide a mobile body control device capable of making a mobile body take high-affinity actions with respect to other mobile bodies in the surroundings, and to provide the mobile body, mobile body control method, program, and learning device.SOLUTION: A mobile body control device is provided, comprising a route determination unit configured to determine a route for a host mobile body in such a way that changes in movement vectors of other mobile bodies present in the surroundings are reduced, and a control unit configured to move the host mobile body along the route determined by the route determination unit.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a mobile object control device, a mobile object, a mobile object control method, a program, and a learning device. [Background technology]

[0002] In recent years, with the development of AI (artificial intelligence), research has been conducted on determining routes by using reinforcement learning in environments where autonomous mobile objects coexist with humans. However, in crowded traffic environments, interference between robots and pedestrians occurs frequently.

[0003] In relation to this, a route determination device has been disclosed that determines the route an autonomous mobile robot will take to a destination under conditions where traffic participants, including pedestrians, are present in the traffic environment to the destination in order to take safe and secure avoidance actions against the movement of people (see Patent Document 1). This route determination device includes a predicted route determination unit that determines a predicted route, which is a predicted value of the robot's route, using a predetermined prediction algorithm so as to avoid interference between the robot and traffic participants, and a route determination unit that determines the robot's route using a predetermined control algorithm so that an objective function including the distance to the nearest traffic participant and the robot's speed as independent variables is maximized when it is assumed that the robot moves along the predicted route from its current position.

[0004] Furthermore, Non-Patent Document 1 describes how, in order to improve cooperation with people around the robot, a reward function is created taking into consideration three patterns, namely, crossing, coming oncoming, and overtaking, and the robot is made to learn the reward function using a specified algorithm.

[0005] Furthermore, Non-Patent Document 2 describes a reward function that is created based on the number of people moving in an overlapping area along the direction of travel of the robot and people, and that the robot learns the reward function using a specified algorithm. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2020 / 136977 [Non-patent literature]

[0007] [Non-Patent Document 1] “Socially Aware Motion Planning with Deep Reinforcement Learning”, Yu Fan Chen, Michael Everett, Miao Liu, Jonathan P.How, 2017.3.26, <<https: / / arxiv.org / pdf / 1703.08862.pdf> > [Non-Patent Document 2] “Mapless Navigation among Dynamics with Social-safety-awareness:a reinforcement learning approach from 2D laser scans”, Jun Jin, Nhat M. Nguyen, Nazmus Sakib, Daniel Graves, Hengshuai Yao, and Martin Jagersand, 2020.3.5., <<https: / / arxiv.org / pdf / 1911.03074.pdf> > Summary of the Invention [Problem to be solved by the invention]

[0008] In the above conventional technology, since the influence of a moving object on the behavior of other moving objects in the vicinity is not taken into consideration, there are cases where the moving object cannot take an action with high affinity with other moving objects in the vicinity. In addition, although the technology described in Patent Document 1 predicts the behavior of other moving objects (robot paths), it is difficult even with current technology to predict the behavior of other moving objects with high accuracy.

[0009] The present invention has been made taking such circumstances into consideration, and one of its objectives is to provide a mobile body control device, a mobile body, a mobile body control method, a program, and a learning device that can cause a mobile body to take actions that are highly compatible with other surrounding moving bodies without predicting the future actions of the other surrounding moving bodies. [Means for solving the problem]

[0010] A mobile object control device, a mobile object, a mobile object control method, a program, or a learning device according to the present invention employs the following configuration. (1): A mobile body control device according to one embodiment of the present invention is a mobile body control device that includes a route determination unit that determines a route for the mobile body so as to reduce changes in the movement vectors of other moving bodies present in the vicinity of the mobile body, and a control unit that moves the mobile body along the route determined by the route determination unit.

[0011] (2) In the aspect of (1) above, the route determination unit determines the route of the own moving body so as to reduce a sum of changes in the movement vectors of the other moving bodies.

[0012] (3): In the above aspect (1) or (2), the route determination unit determines the route of the own moving body so that the value of a reward function having a change in the movement vector of the other moving body as an independent variable becomes a good value.

[0013] (4): In any of the above aspects (1) to (3), the route determination unit determines the route of the moving body so as to avoid entering an area that is large in the direction of the movement vector of the other moving body and small to the side or opposite to the direction of the movement vector of the other moving body.

[0014] (5): A moving body according to another aspect of the present invention is a moving body comprising a moving body control device described in any one of (1) to (4) above, a surrounding detection device that detects the surrounding environment, a working unit for providing a specified service to a user, and a drive unit controlled by the moving body control device that moves the moving body, wherein the moving body control device inputs the state of other moving bodies based on the surrounding environment and outputs control parameters for moving the moving body.

[0015] (6): A moving body control method according to another aspect of the present invention is a moving body control method in which a computer determines a route for a moving body so as to reduce changes in the movement vectors of other moving bodies present around the moving body, and moves the moving body along the route determined by the route determination unit.

[0016] (7): A program according to another aspect of the present invention is a program that causes a computer to determine a route for a moving body so as to reduce changes in the movement vectors of other moving bodies present in the vicinity of the moving body, and to move the moving body along the route determined by the route determination unit.

[0017] (8): A learning device according to another aspect of the present invention includes a simulation unit that simulates the movement motion of a self-moving body and another moving body, an evaluation unit that applies a reward function to the processing result of the simulation unit and evaluates at least the movement motion of the self-moving body, and a learning unit that learns (a suitable movement motion of the self-moving body) based on the evaluation result of the evaluation unit, and the evaluation unit is a learning device that evaluates the movement motion of the self-moving body more highly the smaller the change in the movement vector of the other moving body.

[0018] (9): In the above aspect (8), the evaluation unit evaluates the movement motion of the self-moving body as low when the self-moving body enters an area that is large in the direction of the movement vector of the other moving body and small to the side and in the opposite direction to the direction of the movement vector of the other moving body. Effect of the Invention

[0019] According to the above aspects (1) to (3) and (5) to (7), it is possible to move a moving object so as not to hinder the movement of other moving objects as much as possible without predicting the future actions of other moving objects in the vicinity. As a result, it is possible to make the moving object take actions that are highly friendly to other moving objects in the vicinity. According to the above aspect (4), the route of the moving object can be determined taking into account the personal space. As a result, according to the above aspects (1) to (7), it is possible to make a moving body take actions that are more compatible with other moving bodies in the vicinity, without predicting the future actions of the other moving bodies in the vicinity. According to the above aspect (8), it is possible to learn to avoid hindering the movement of other moving objects as much as possible without predicting the future actions of other moving objects in the vicinity, and as a result, it is possible to generate a policy that allows a moving object to take actions that are highly compatible with other moving objects in the vicinity. According to the above aspect (9), learning can be performed while taking personal space into consideration. As a result, according to the above aspects (8) to (9), it is possible to learn how to make a moving body take actions that are more compatible with other moving bodies in the vicinity, without predicting the future actions of the other moving bodies in the vicinity. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a schematic diagram showing a system configuration of an embodiment. [Diagram 2] FIG. 1 is a configuration diagram of a learning device 100. [Diagram 3] FIG. 13 is a diagram to explain the reward function R3. [Figure 4] FIG. 13 is a diagram to explain reward function R4. [Diagram 5] 1 is a flowchart showing an example of processing of a learning process of reinforcement learning performed by the learning device 100. [Figure 6] FIG. 2 is a diagram illustrating the configuration of a moving object 200. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, with reference to the drawings, embodiments of a mobile object control device, a mobile object, a mobile object control method, a program, and a learning device of the present invention will be described.

[0022] [Learning device] FIG. 1 is a schematic diagram showing a system configuration of an embodiment. The mobile object control system 1 includes a learning device 100 and a mobile object 200. The learning device 100 is realized by one or more processors. The learning device 100 is a device that determines actions for a plurality of mobile objects by computer simulation, derives or acquires rewards based on environmental changes caused by the actions, and learns actions (movements) that maximize the rewards. The actions are, for example, movements within a simulation space. Although movements other than movements may be the learning target, in the following description, the actions refer to movements. A simulator (a simulation unit described later) that determines the movements may be executed in a device different from the learning device 100, but in the following description, the simulator is executed by the learning device 100. The learning device 100 stores in advance environmental information that is the premise of the simulation, such as map information. The learning result of the learning device 100 is mounted on the mobile object 200 as a behavior determination model MD.

[0023] 2 is a configuration diagram of the learning device 100. The learning device 100 includes, for example, a learning unit 110, a simulation unit 120, and an evaluation unit 130. The learning device 100 is a device that inputs an action goal generated for a self-agent (a moving body in the moving body 200) to arrive at a destination, and the positions, moving directions, moving speeds, and the like of other agents (other moving bodies) into a policy, performs reinforcement learning to update the policy based on the result of evaluating a state change (change in the environment) that occurs as a result, and outputs the learned policy.

[0024] The self-agent is a virtual agent that is assumed to be a moving object such as a robot or a vehicle. Similarly, other agents are also virtual agents that are assumed to be moving objects such as robots or vehicles. Policies are also used to determine the actions of other agents, but the policies of other agents may or may not be updated.

[0025] The learning unit 110, the simulation unit 120, and the evaluation unit 130 are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). The program may be stored in advance in a storage device (non-transient storage medium) such as a HDD (Hard Disk Drive) or a flash memory, or may be stored in a removable storage medium (non-transient storage medium) such as a DVD (Digital Versatile Disc) or a CD-ROM (Read Only Memory), and may be installed by mounting the storage medium in a drive device. Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware.

[0026] The learning unit 110 updates the policy according to various reinforcement learning algorithms based on the evaluation result of the evaluation unit 130 of the state change generated by the simulation unit 120 and the collision determination result. The learning unit 110 repeatedly outputs the updated policy to the simulation unit 120 until learning is completed.

[0027] The simulation unit 120 inputs the action goal and the previous state (initial state if it is immediately after the start of the simulation) into the policy, and derives a state change that is a result of the actions of the agent itself and the other agents. The policy is, for example, a deep neural network (DNN), but may be a rule-based policy or other form of policy. The policy derives the occurrence probability for each of a plurality of types of action that are assumed. For example, in a simple example, assuming that the assumed plane extends up, down, left, and right, the simulation unit 120 outputs results such as 80% right movement, 10% left movement, 10% up movement, and 0% down movement. The simulation unit 120 applies a random number to this result, and derives the agent's state change as follows: right movement if the random number value is 0% or more and less than 80%, left movement if the random number value is 80% or more and less than 90%, and upward movement if the random number value is 90% or more.

[0028] The evaluation unit 130 calculates the value of the reward function R (reward function value) that evaluates the state change of the own agent output by the simulation unit 120, and evaluates the behavior of the own agent.

[0029] The reward function R is given when the agent reaches the destination as shown in equation (1). 1 and the reward function R given when the agent achieves a smooth movement. 2 and the reward function R that becomes smaller when the agent changes the movement vector of the other agent. 3 and a reward function R that changes the distance that the agent should maintain when approaching another agent depending on the direction the other agent is facing. 4 The reward function R 3 is an example of the first reward function, and the reward function R 4 is an example of a second reward function.

[0030]

number

[0031] Reward function R1 is a function that takes a fixed positive value when the destination is reached, and takes a value proportional to the change in distance to the destination when the destination has not been reached (positive if the change in distance is decreasing, negative if the change in distance is increasing).

[0032] Reward function R 2 is a function that is the third derivative of the agent's position on a two-dimensional plane, i.e., the smaller the jerk, the larger its value becomes.

[0033] Figure 3 shows the reward function R 3 FIG. 1 is a diagram for explaining the reward function R calculated at time (control cycle) t. 3 is the movement vector a´ of other agents from the state of other agents at time t-1 to time t i , t (the movement vector of the other agent when the self agent does not exist) and the movement vector of the other agent from the state of the other agent at time t-1 to time t (the movement vector of the other agent when the self agent exists) a i , t The larger the difference between them, the smaller the resulting evaluation value becomes. In other words, the reward function R 3 The reward function R evaluates the behavior of the agent more highly if the agent does not affect the movement vectors of other agents in the vicinity. 3 is an objective function that has the change in the movement vector of the other agent as an independent variable, and for example, the larger the value, the better the value. The evaluation unit 130 calculates the movement vector a' of the other agent from the state of the other agent at time t-1 to time t that is assumed if the own agent does not exist. i , t may be derived by itself, or may request the simulation unit 120 to derive it.

[0034]

number

[0035] In formula (2), W is a function that returns a lower evaluation value as the negative coefficient or the value after Σ becomes larger. i , t is the movement vector of each other agent from time t-1 to time t (assuming the existence of the agent), and a´ i , t is the movement vector of each other agent from time t-1 to time t (assuming that the own agent does not exist). i is the identification number of the other agent, and N is the total number of other agents present.

[0036] In FIG. 3, agent H is the agent itself, and agents A1 to A5 are other agents. For example, at time t, other agent A1 is 1,t The other agent A2 moves with a vector of a 2,t The other agent A3 moves with a vector of a 3,t The other agent A4 moves with a vector of 4,t The other agent A5 moves with a vector of 5,t In contrast, if we return to the state at time t-1 and assume that the agent H does not exist, the movement vector for the other agent A1 is a´ 1,t , and for the other agent A2, a´ 2,t , and for the other agent A3, a´ 3,t , and for other agent A4, a´ 4,t , and for the other agent A5, a´ 5,t It is expressed as:

[0037] Figure 4 shows the reward function R 4 FIG. 1 is a diagram for explaining the reward function R 4is a function that returns a low evaluation value when the agent enters a specified area. The area around another agent A is considered to be divided into the following four areas (spaces). For example, it is assumed to be divided into an intimate space surrounded by boundary line D1, a personal space surrounded by boundary lines D1 and D2, a social space surrounded by boundary lines D2 and D3, and a public space surrounded by boundary lines D3 and D4.

[0038] In this embodiment, for example, the reward function R 4 returns a low evaluation value when an agent enters D2, which is the outer boundary of the personal space. Like social space and public space, personal space is wide in the direction (F) in which other agent A is facing (or moving) and narrow in other directions, with other agent A as the base. This means that a low evaluation is given to actions that pass in front of other agent A, and a not-so-low evaluation is given to actions that pass to the side or behind other agent A.

[0039] The evaluation unit 130 may determine that the own agent and the other agent have collided when their coordinates match, or may determine that the own agent and the other agent have collided when the own agent enters the personal space of the other agent. If it determines that a collision has occurred, the evaluation unit 130 ends the episode, initializes the state of each agent, and starts the next episode. The evaluation unit 130 outputs the collision determination result and the action evaluation result to the learning unit 110. Details will be explained using a flowchart.

[0040] FIG. 5 is a flowchart showing an example of the processing of the learning process of reinforcement learning performed by the learning device 100.

[0041] First, simulation unit 120 receives an action goal of its own agent from learning device 100 (step S200). Next, learning device 100 uses the action goal as one of the inputs and simulates the action of each agent for one cycle (step S202).

[0042] Next, the evaluation unit 130 judges whether or not the own agent has collided with another agent (step S204). If it is judged that the own agent has not collided with another agent in the vicinity, the evaluation unit 130 evaluates the operation of the own agent using the reward function R (step S206), and outputs the evaluation result to the learning unit 110.

[0043] Next, the learning unit 110 updates the policy according to a reinforcement learning algorithm based on the evaluation result of the evaluation unit 130 (step S208). The learning unit 110 outputs the updated policy to the simulation unit 120, and the simulation unit 120 uses the accepted policy to simulate the behavior of each agent in the next cycle.

[0044] Next, the learning device 100 judges whether the amount of update of the policy parameter each time is equal to or less than a threshold value based on the state change resulting from the actions of the own agent and the other agents (step S210). The amount of update of the parameter here is, for example, the amount of change in the parameter such as the movement vector of the own agent in the nth time compared with the parameter such as the movement vector of the own agent in the n-1th time, and is the sum of the absolute values ​​of the amount of change of the parameter. If the amount of update of the policy parameter is equal to or less than a certain threshold value m, that is, if the parameter of the policy has not changed much, the learning device 100 ends the processing of the learning process. If the amount of update of the policy parameter is not equal to or less than a certain threshold value m, the learning device 100 returns to step S202. Alternatively, the processing of the learning process may be ended when a predetermined number of cycles of processing have been completed.

[0045] If it is determined in step S204 that the agent itself has collided with another nearby agent, the evaluation unit 130 outputs the determination result to the learning unit 110 and lowers the evaluation value of the reward function (step S212). Then, the evaluation unit 130 outputs the evaluation result to the learning unit 110, and the learning unit 110 updates the policy based on the evaluation result of the evaluation unit 130 (step S214). Furthermore, the learning device 100 initializes the state of each agent and returns to step S202.

[0046] According to the learning device 100 described above, a behavior decision model (policy) can be generated by reinforcement learning so as not to hinder the behavior of other surrounding moving bodies as much as possible. As a result, the moving body control device 250 that employs the behavior decision model can cause the moving body 200 to take a behavior that has high affinity with the behavior of other surrounding moving bodies.

[0047] [Mobile object] 6 is a configuration diagram of a mobile body 200. The mobile body 200 includes, for example, a mobile body control device 250, a periphery detection device 210, a mobile body sensor 220, a working unit 230, and a driving device 240. The mobile body 200 may be a vehicle or a device such as a robot. The mobile body control device 250, the periphery detection device 210, the mobile body sensor 220, the working unit 230, and the driving device 240 are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like.

[0048] The surrounding detection device 210 is a device for detecting the environment around the moving body 200 and the operation of other moving bodies in the vicinity. The surrounding detection device 210 includes, for example, a positioning device including a GPS receiver and map information, and an object recognition device such as a radar device and a camera. The positioning device determines the position of the moving body 200 and matches the position with the map information. The radar device radiates radio waves such as millimeter waves around the moving body 200 and detects radio waves (reflected waves) reflected by the object to detect at least the position (distance and direction) of the object. The radar device may detect the position and movement vector of the object. The camera is, for example, a digital camera using a solid-state image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and is provided with an image processing device that recognizes the position of the object from the captured image. The periphery detection device 210 outputs to the mobile object control device 250 information such as the position of the mobile object 200 on a map and the positions of objects (including other mobile objects equivalent to the aforementioned other agents) present in the periphery of the mobile object 200.

[0049] The mobile body sensor 220 includes, for example, a speed sensor that detects the speed of the mobile body 200, an acceleration sensor that detects the acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, and an orientation sensor that detects the orientation of the mobile body 200. The mobile body sensor 220 outputs the detection results to the mobile body control device 250.

[0050] The working unit 230 is, for example, a device that provides a predetermined service to a user. The service here is, for example, work such as loading and unloading cargo onto transportation equipment. The working unit 230 includes, for example, a magic arm, a loading platform, and an HMI (Human machine Interface) such as a microphone and a speaker. The working unit 230 operates according to the contents of an instruction from the mobile object control device 250.

[0051] The driving device 240 is a device for moving the moving body 200 in a desired direction. If the moving body 200 is a robot, the driving device 240 includes, for example, two or more legs and an actuator. If the moving body 200 is a vehicle, a micromobile, or a robot that moves on wheels, the driving device 240 includes wheels (steering wheels, driving wheels) and a motor, engine, etc. for rotating the wheels.

[0052] The mobile object control device 250 includes, for example, a route determination unit 252, a control unit 254, and a storage unit 256. Each of the route determination unit 252 and the control unit 254 is realized by, for example, a hardware processor such as a CPU executing a program (software). The program may be stored in advance in a storage device (non-transient storage medium) such as a HDD or flash memory, or may be stored in a removable storage medium (non-transient storage medium) such as a DVD or CD-ROM, and may be installed by mounting the storage medium in a drive device. Some or all of these components may be realized by hardware (including circuitry) such as an LSI, ASIC, FPGA, or GPU, or may be realized by cooperation between software and hardware.

[0053] The storage unit 256 is, for example, a HDD, a flash memory, a RAM, a ROM, etc. The storage unit 256 stores information such as a behavioral decision model MD256A. The behavioral decision model MD256A is generated by the learning device 100 and is based on a policy at the final point of the processing in the learning stage.

[0054] The route determination unit 252 determines the next position to which the moving body 200 should advance by inputting information such as the position of the moving body 200 on a map and the positions of objects present around the moving body 200 (object states) detected by the periphery detection device 210, as well as destination information input by the user, into the behavior decision model MD256A. The route determination unit 252 repeats this process to sequentially determine the route of the moving body 200.

[0055] The control unit 254 controls the driving device 240 so that the moving body 200 moves along the path determined by the path determination unit 252 .

[0056] According to the mobile body control device 250 described above, a route for the moving body 200 is generated based on a behavior decision model (policy) generated by reinforcement learning so as not to hinder the actions of other surrounding moving bodies as much as possible, and the moving body 200 is moved along the route, so that the moving body 200 can be made to take actions that are highly compatible with the actions of other surrounding moving bodies.

[0057] In this embodiment, it is assumed that policy updates are performed only during the learning phase and not after the system is installed in a mobile body, although learning may continue even after the system is installed in a mobile body.

[0058] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0059] The above embodiment can be expressed as follows. A storage device storing a program; a hardware processor coupled to the storage device; The hardware processor executes the program, determining a route for the moving object so as to reduce a change in a movement vector of another moving object existing around the moving object; moving the own moving body along the determined route; Mobile control device.

[0060] The above embodiment can also be expressed as follows. A storage device storing a program; a hardware processor coupled to the storage device; The hardware processor executes the program, Simulating the movement of the own moving body and other moving bodies; applying a reward function to the result of the simulation to evaluate at least the movement motion of the self moving object; learning based on the results of said evaluation; When performing the evaluation, the smaller the change in the movement vector of the other moving body, the higher the evaluation of the movement operation of the moving body itself. Learning device. [Explanation of symbols]

[0061] 100 Learning Measures 110 Learning Department 120 Simulation Department 130 Evaluation Department 200 Mobile 210 Periphery detection device 230 Working Section 240 Drive Unit 250 Mobile control device 252 Route Determination Unit 254 Control Unit 256 Memory section 256A Behavioral Decision Model MD

Claims

1. A route determination unit that inputs position information of other moving bodies around the moving body and information indicating the moving direction of the moving body into a model, and determines a route of the moving body based on an output result output by the model so as to reduce changes in the movement vectors of the other moving bodies existing around the moving body; a control unit that moves the moving object along the route determined by the route determination unit, The model applies a reward function to a processing result obtained by simulating the movement of each of the self moving body and the other moving body, and learns based on an evaluation result obtained by evaluating at least the movement of the self moving body, so that the smaller the change in the movement vector of the other moving body, the higher the movement of the self moving body is evaluated. Mobile control device.

2. The route determination unit determines a route of the moving body so as to reduce a sum of changes in the movement vectors of the other moving bodies. The mobile object control device according to claim 1 .

3. the path determination unit determines a path of the own moving body such that a value of a reward function having a change in a movement vector of the other moving body as an independent variable becomes a good value. The mobile object control device according to claim 1 or 2.

4. the route determination unit determines a route of the own moving body such that the route is large in a direction in which the movement vector of the other moving body is directed and does not enter into a small area to the side of and in an opposite direction to the direction in which the movement vector of the other moving body is directed; The mobile object control device according to any one of claims 1 to 3.

5. A mobile object control device according to any one of claims 1 to 4, A surrounding detection device that detects the surrounding environment; a working unit for providing a predetermined service to a user; A drive unit that is controlled by the moving body control device and moves the moving body; Equipped with the mobile object control device inputs a state of another mobile object based on the surrounding environment, and outputs a control parameter for moving the mobile object; Mobile body.

6. The computer By inputting at least position information of other moving bodies around the moving body and information indicating the moving direction of the moving body into a model, a route of the moving body is determined based on an output result output by the model so as to reduce changes in the movement vectors of the other moving bodies existing around the moving body; Moving the moving body along the path; The model applies a reward function to a processing result obtained by simulating the movement of each of the self moving body and the other moving body, and learns based on an evaluation result obtained by evaluating at least the movement of the self moving body, so that the smaller the change in the movement vector of the other moving body, the higher the movement of the self moving body is evaluated. A method for controlling a moving object.

7. On the computer, By inputting at least position information of other moving bodies around the moving body and information indicating the moving direction of the moving body into the model, a route of the moving body is determined based on an output result output by the model so as to reduce changes in the movement vectors of the other moving bodies existing around the moving body; moving the own moving body along the route; The model applies a reward function to a processing result obtained by simulating the movement of each of the self moving body and the other moving body, and learns based on an evaluation result obtained by evaluating at least the movement of the self moving body, so that the smaller the change in the movement vector of the other moving body, the higher the movement of the self moving body is evaluated. program.

8. a simulation unit that simulates the movement of the own moving body and other moving bodies; an evaluation unit that applies a reward function to a processing result of the simulation unit and evaluates at least the movement motion of the own moving object; A learning unit that learns based on the evaluation result of the evaluation unit, The evaluation unit evaluates the movement operation of the own moving body higher as the change in the movement vector of the other moving body becomes smaller. Learning device.

9. the evaluation unit evaluates the movement of the own moving body as low when the own moving body enters an area that is large in a direction in which the movement vector of the other moving body is directed and small to the side and in a direction opposite to the direction in which the movement vector of the other moving body is directed. The learning device according to claim 8.

Citation Information

Patent Citations

  • Autonomous mobile robot device and method of avoiding collision due to sudden appearance in the device

    JP2010055498A

  • Method and system for controlling a vehicle

    US20190361452A1

  • Moving body

    WO2016104265A1

  • Path determination device, robot, and path determination method

    WO2020136977A1