Vehicle control device, vehicle control method, and program

Abstract

To obtain an appropriate solution by setting proper constraints when executing speed control taking a future state into consideration.SOLUTION: A vehicle controller includes: a state value calculation section which calculates a state value for each of a plurality of state paths defined by selecting an action amount generated between a plurality of time points in a state space from among a plurality of candidates; and a speed determination section which determines a future speed transition of a vehicle according to the state path having a high state value. The state value calculation section calculates reward function values based on at least one evaluation object amount for respective time points in a period from a start time point to a target time point, and sums up the reward function values on a time-series basis to calculate the state value. The state value is calculated according to first regions reducing the state value and second regions reducing the reward function value which are set in some of or all of a plurality of partial spaces, each with at least two elements of a plurality of elements as axes.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a program. [Background technology]

[0002] Conventionally, the practical application of technology for automatically controlling vehicles (autonomous driving) has been progressing. Research is being conducted on various controls in autonomous driving, not only through feedback control based on the current environment, but also through modeling that takes into account future conditions. Patent Documents 1 and 2 describe the generation of an action plan for autonomous driving through modeling using a Markov decision process. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2020-510570 [Patent Document 2] International Publication No. 2019 / 167457 Summary of the Invention [Problem to be solved by the invention]

[0004] There are various factors to consider in vehicle speed control, and if appropriate constraints are not set, an incorrect solution may be derived. There is also a concern that unnecessary trial and error may occur, resulting in an excessive processing load.

[0005] The present invention has been made in consideration of these circumstances, and one of its objectives is to provide a vehicle control device, a vehicle control method, and a program that are capable of obtaining a suitable solution by setting appropriate constraints when performing speed control that takes future states into account. [Means for solving the problem]

[0006] A vehicle control device, a vehicle control method, and a program according to the present invention employ the following configuration. (1): A vehicle control device according to one embodiment of the present invention includes a state value calculation unit that calculates a state value for each of a plurality of state paths between a start point and a target point in a state space including time and having a plurality of elements related to the movement of the vehicle as its axes, the state path being defined by selecting from a plurality of candidates an amount of action occurring between a plurality of time points between the start point and the target point, and a speed determination unit that determines a future speed change of the vehicle according to a state path with a high state value. The state value calculation unit calculates a reward function value based on one or more evaluation target quantities for each time point between the start point and the target point, and calculates the state value by summing up the reward function values ​​in a time series, and calculates the state value according to a first region that reduces the state value and a second region that reduces the reward function value, which are set in some or all of a plurality of subspaces having axes of two or more elements among the plurality of elements.

[0007] (2): In the above aspect (1), the plurality of elements include some or all of time, position in the direction of travel of the vehicle, speed, and acceleration.

[0008] (3): In the above aspect (1) or (2), the amount of movement is a jerk.

[0009] (4): In any of the above aspects (1) to (3), the one or more evaluation target quantities include the speed, acceleration, and jerk of the vehicle.

[0010] (5): In any of the above aspects (1) to (4), a recognition unit is further provided that recognizes the surrounding conditions of the vehicle, and the state value calculation unit sets one or both of the first area and the second area in some or all of the multiple subspaces based on the surrounding conditions of the vehicle.

[0011] (6): In the above aspect (5), when the vehicle's surrounding conditions include a static object, the state value calculation unit sets the first region and the second region in a subspace whose axes are position and speed.

[0012] (7): In the above aspect (5) or (6), when the vehicle's surrounding conditions include a dynamic target object, the state value calculation unit sets the first region and the second region in a subspace having position and time as axes.

[0013] (8): In any one of the above aspects (5) to (7), the state value calculation unit sets the first area and the second area adjacent to each other.

[0014] (9): A vehicle control method according to another aspect of the present invention includes a vehicle control device that calculates a state value for each of a plurality of state paths between a start point and a target point in a state space including time and having a plurality of elements related to vehicle movement as its axes, the state path being defined by selecting from a plurality of candidates an amount of action occurring between a plurality of time points between the start point and the target point, determines a future speed change of the vehicle according to the state path having a high state value, calculates a reward function value based on one or more evaluation target quantities for each time point between the start point and the target point in calculating the state value, and calculates the state value by summing up the reward function values ​​in a time series, and calculates the state value according to a first region that reduces the state value and a second region that reduces the reward function value, which are set in some or all of a plurality of subspaces having axes of two or more elements among the plurality of elements.

[0015] (10): A program according to another aspect of the present invention is a program that causes a computer to calculate a state value for each of a plurality of state paths between a start point and a target point in a state space including time and having a plurality of elements related to the movement of the vehicle as its axes, the state path being defined by selecting from a plurality of candidates an amount of action occurring between the start point and the target point, and to determine a future speed transition of the vehicle according to the state path with the highest state value. When calculating the state value, a reward function value based on one or more evaluation target quantities is calculated for each time point between the start point and the target point, and the state value is calculated by summing the reward function values ​​in a time series. The state value is calculated according to a first region that reduces the state value and a second region that reduces the reward function value, which are set in some or all of a plurality of subspaces having axes of two or more of the plurality of elements. Effect of the Invention

[0016] According to the above aspects (1) to (10), when performing speed control taking into account future states, a suitable solution can be obtained by setting appropriate constraints. [Brief description of the drawings]

[0017] [Figure 1] 1 is a configuration diagram of a vehicle system 1 that uses a vehicle control device according to an embodiment. [Diagram 2] 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160. FIG. [Diagram 3] FIG. 11 is a diagram for explaining the definition of a state path. [Figure 4] FIG. 13 is a diagram showing a first example of a subspace constraint map 146. [Diagram 5] FIG. 13 is a diagram showing a second example of the subspace constraint map 146. [Figure 6] 13 is a flowchart showing an example of the flow of processes executed by a state value calculation unit 142 and a speed determination unit 144. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a vehicle control device, a vehicle control method, and a program according to the present invention will be described with reference to the drawings.

[0019] [Overall configuration] 1 is a configuration diagram of a vehicle system 1 that uses a vehicle control device according to an embodiment. The vehicle on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharged power from a secondary battery or a fuel cell.

[0020] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driving operator 80, an automatic driving control device 100, a driving force output device 200, a brake device 210, and a steering device 220. These devices and equipment are connected to each other by multiple communication lines such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, etc. Note that the configuration shown in FIG. 1 is merely an example, and some of the configuration may be omitted, or other configuration may be added.

[0021] The camera 10 is, for example, a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to an arbitrary location of a vehicle (hereinafter, the vehicle M) in which the vehicle system 1 is mounted. When capturing an image of the front, the camera 10 is attached to the top of the front windshield, the back of the rearview mirror, or the like. The camera 10, for example, periodically and repeatedly captures images of the surroundings of the vehicle M. The camera 10 may be a stereo camera.

[0022] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by an object (reflected waves) to detect at least the position (distance and direction) of the object. The radar device 12 is attached to an arbitrary location of the vehicle M. The radar device 12 may detect the position and speed of an object by an FM-CW (Frequency Modulated Continuous Wave) method.

[0023] The LIDAR 14 irradiates light (or electromagnetic waves with a wavelength close to that of light) around the vehicle M and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between light emission and light reception. The irradiated light is, for example, a pulsed laser light. The LIDAR 14 is attached to an arbitrary location of the vehicle M.

[0024] The object recognition device 16 performs sensor fusion processing on some or all of the detection results from the camera 10, the radar device 12, and the LIDAR 14 to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition result to the automatic driving control device 100. The object recognition device 16 may output the detection results from the camera 10, the radar device 12, and the LIDAR 14 directly to the automatic driving control device 100. The object recognition device 16 may be omitted from the vehicle system 1.

[0025] The communication device 20 communicates with other vehicles in the vicinity of the vehicle M, for example, using a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc., or communicates with various server devices via a wireless base station.

[0026] The HMI 30 presents various information to an occupant of the vehicle M and accepts input operations by the occupant. The HMI 30 includes various display devices, a speaker, a buzzer, a touch panel, switches, keys, and the like.

[0027] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity around a vertical axis, a direction sensor that detects the direction of the host vehicle M, and the like.

[0028] The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 holds first map information 54 in a storage device such as a HDD (Hard Disk Drive) or a flash memory. The GNSS receiver 51 identifies the position of the vehicle M based on a signal received from a GNSS satellite. The position of the vehicle M may be identified or complemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, and the like. The navigation HMI 52 may be partially or entirely shared with the above-mentioned HMI 30. The route determination unit 53, for example, determines a route (hereinafter, a route on a map) from the position of the vehicle M identified by the GNSS receiver 51 (or an arbitrary position input) to a destination input by a driver using the navigation HMI 52, with reference to the first map information 54. The first map information 54 is, for example, information in which road shapes are expressed by links indicating roads and nodes connected by the links. The first map information 54 may include road curvature and POI (Point Of Interest) information. The route on the map is output to the MPU 60. The navigation device 50 may perform route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized by, for example, a function of a terminal device such as a smartphone or tablet terminal owned by a passenger. The navigation device 50 may transmit a current position and a destination to a navigation server via the communication device 20, and obtain a route equivalent to the route on the map from the navigation server.

[0029] The MPU 60 includes, for example, a recommended lane determination unit 61, and holds second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, by dividing it into 100 m intervals in the vehicle travel direction), and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines which lane from the left the vehicle should travel in. When a branch point is present on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the host vehicle M can travel along a reasonable route to proceed to the branch point.

[0030] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of lanes or information on boundaries of lanes. The second map information 62 may also include road information, traffic regulation information, address information (address and zip code), facility information, telephone number information, and the like. The second map information 62 may be updated as needed by the communication device 20 communicating with another device.

[0031] The driving operators 80 include, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a special steering wheel, a joystick, and other operators. The driving operators 80 are fitted with sensors that detect the amount of operation or the presence or absence of operation, and the detection results are output to the automatic driving control device 100, or some or all of the driving force output device 200, the brake device 210, and the steering device 220.

[0032] The automatic driving control device 100 includes, for example, a first control unit 120 and a second control unit 160. The first control unit 120 and the second control unit 160 are each realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). In addition, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device (storage device having a non-transient storage medium) such as an HDD or flash memory of the automatic driving control device 100, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the automatic driving control device 100 by mounting the storage medium (non-transient storage medium) in a drive device. The automatic driving control device 100 is an example of a "vehicle control device", and the combination of the action plan generation unit 140 and the second control unit 160 is an example of a "driving control unit".

[0033] 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and a behavior plan generation unit 140. The behavior plan generation unit 140 includes a state value calculation unit 142 and a speed determination unit 144. The state value calculation unit 142 includes a state path definition unit 142A and a calculation unit 142B. One or more subspace constraint maps 146 are stored in a memory area that the behavior plan generation unit 140 can refer to. The contents of the state value calculation unit 142, the speed determination unit 144, and the subspace constraint map 146 will be described later.

[0034] The first control unit 120, for example, realizes a function based on AI (Artificial Intelligence) and a function based on a pre-given model in parallel. For example, the function of "recognizing an intersection" may be realized by executing in parallel the recognition of an intersection by deep learning or the like and the recognition based on pre-given conditions (such as traffic lights and road markings that can be pattern matched), and by scoring both and evaluating them comprehensively. This ensures the reliability of the autonomous driving.

[0035] The recognition unit 130 recognizes the position, speed, acceleration, and other states of objects around the vehicle M based on information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16. The position of the object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. The "state" of the object may include the acceleration or jerk of the object, or the "behavior state" (for example, whether or not the object is changing lanes or is about to change lanes).

[0036] The recognition unit 130 also recognizes, for example, the lane in which the vehicle M is traveling (driving lane). For example, the recognition unit 130 recognizes the driving lane by comparing a pattern of road dividing lines (for example, an arrangement of solid lines and dashed lines) obtained from the second map information 62 with a pattern of road dividing lines around the vehicle M recognized from an image captured by the camera 10. The recognition unit 130 may recognize the driving lane by recognizing road boundaries (road boundaries) including not only road dividing lines but also road dividing lines, shoulders, curbs, median strips, guard rails, etc. In this recognition, the position of the vehicle M obtained from the navigation device 50 and the processing results by the INS may be taken into consideration. The recognition unit 130 also recognizes stop lines, obstacles, red lights, toll booths, and other road phenomena.

[0037] When recognizing the driving lane, the recognition unit 130 recognizes the position and attitude of the host vehicle M with respect to the driving lane. For example, the recognition unit 130 may recognize the deviation of the reference point of the host vehicle M from the center of the lane and the angle with respect to a line connecting the centers of the lanes in the traveling direction of the host vehicle M as the relative position and attitude of the host vehicle M with respect to the driving lane. Alternatively, the recognition unit 130 may recognize the position of the reference point of the host vehicle M with respect to either side end of the driving lane (a road dividing line or a road boundary) as the relative position of the host vehicle M with respect to the driving lane.

[0038] The action plan generating unit 140 generates a target trajectory along which the host vehicle M will travel in the future automatically (without relying on the driver's operation) so that the host vehicle M will travel in the recommended lane determined by the recommended lane determining unit 61 in principle and can further respond to the surrounding conditions of the host vehicle M. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) to be reached by the host vehicle M. The trajectory points are points to be reached by the host vehicle M at each predetermined travel distance (for example, about several [m]) along the road, and separately, the target speed and target acceleration for each predetermined sampling time (for example, about a few tenths [sec]) are generated as part of the target trajectory. The trajectory points may also be positions to be reached by the host vehicle M at each sampling time for each predetermined sampling time. In this case, the information on the target speed and target acceleration is expressed as the interval between the trajectory points.

[0039] The behavior plan generating unit 140 may set an event of autonomous driving when generating the target trajectory. The autonomous driving events include a constant speed driving event, a low speed following driving event, a lane change event, a branching event, a merging event, a takeover event, etc. The behavior plan generating unit 140 generates a target trajectory according to the activated event.

[0040] The second control unit 160 controls the traveling driving force output device 200, the braking device 210, and the steering device 220 so that the host vehicle M passes the target trajectory generated by the action plan generating unit 140 at the scheduled time.

[0041] Returning to FIG. 2, the second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140, and stores it in a memory (not shown). The speed control unit 164 controls the traveling driving force output device 200 or the brake device 210 based on a speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the host vehicle M and feedback control based on the deviation from the target trajectory.

[0042] The driving force output device 200 outputs a driving force (torque) for the vehicle to travel to the driving wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components according to information input from the second control unit 160 or information input from the driving operation device 80.

[0043] The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the driving operation element 80, so that a brake torque corresponding to a braking operation is output to each wheel. The brake device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of a brake pedal included in the driving operation element 80 to the cylinder via a master cylinder. Note that the brake device 210 is not limited to the configuration described above, and may be an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure in the master cylinder to the cylinder.

[0044] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor changes the direction of the steered wheels by applying a force to, for example, a rack and pinion mechanism. The steering ECU drives the electric motor in accordance with information input from the second control unit 160 or information input from the driving operator 80 to change the direction of the steered wheels.

[0045] [Speed ​​control based on state value] Hereinafter, a description will be given of speed control by the state value calculation unit 142 and the speed determination unit 144. The state value calculation unit 142 and the speed determination unit 144 determine a future speed transition (hereinafter, a speed profile) of the host vehicle M using dynamic programming, more specifically, a Markov decision process.

[0046] The state path definition unit 142A of the state value calculation unit 142 defines a state path by selecting, from among multiple candidates, an amount of action occurring between a start time point and a target time point in a state space having multiple elements including time as axes, which are multiple elements related to the movement of the vehicle. The "start time point" is, for example, the current time (or a time point after an infinitesimal time considering control delay). The "multiple elements" are, for example, {time, position in the traveling direction (longitudinal direction of the road), speed, and acceleration}. Elements other than time may be excluded from the multiple elements, or another element may be added to the multiple elements. Hereinafter, an element that is determined by each element taking a specific value is referred to as a "state". The "amount of action" may be any physical quantity related to speed, such as jerk (jerk). In this process, time progresses in increments of a predetermined width (for example, 1 [sec]). FIG. 3 is a diagram for explaining the definition of a state path. The state path definition unit 142A prepares a plurality of jerk candidates that occur between a plurality of time points (every 1 [sec] elapses) between the start time point and the target time point, such as (0.5), (0.3), (0.1), (0), (-0.1), (-0.3), and (-0.5), and calculates the state at the next time point when constant jerk running is performed with one jerk selected from the jerk candidates between the time points. The state path definition unit 142A executes this in a cascading manner as the time point progresses, and performs it until the target time point (for example, several [sec] to a dozen [sec] after the start time point). In the figure, SP represents one of the state paths. A state path refers to a series of states that are determined by selecting one at each time point from the states that can be traced in order from the start time point. If k jerk candidates are prepared and the time from the start time point to the target time point is h [sec], a state path of k to the power of h is generated.

[0047] The calculation unit 142B calculates a reward function value based on one or more evaluation target quantities for each state path at each time point between the start time point and the target time point. The evaluation target quantities are physical quantities related to the movement of the vehicle, and include, for example, speed, acceleration, and jerk. The reward function f(i,t) for calculating the reward function value is expressed, for example, by Equation (1). In the equation, i is identification information of the state path, and t is a time point. The reward function f(i,t) is, for example, a function that returns a higher value as the speed is higher, returns a lower value as the absolute value of the acceleration is higher, and returns a lower value as the absolute value of the jerk is higher. A high speed and low acceleration or jerk indicate that the host vehicle M is able to travel without unnecessary acceleration or deceleration, so the reward function f(i,t) is defined to return a high value (a value indicating good) in such a case. In addition, the minimum value of the value output by the reward function f(i,t) is set to be zero.

[0048] Reward function = f(i,t) {velocity, acceleration, jerk} …(1)

[0049] Then, the calculation unit 142B calculates a state value SV(i) for each state path i by summing the reward function values ​​in a time series. The state value is expressed by, for example, Equation (2).

[0050] SV(i) = Σ t=0 h f(i,t) …(2)

[0051] In the above calculation, the calculation unit 142B sets the subspace constraint map 146 based on the recognition result of the recognition unit 130, and calculates the reward function value and the state value by reflecting the set map. The subspace constraint map 146 is set in some or all of the multiple subspaces having two or more elements of the multiple "elements" as axes. The calculation unit 142B sets, in the subspace constraint map 146, one or both of a prohibited area (first area) in which the state value SV(i) is reduced (for example, set to zero) and a non-recommended area (second area) in which the reward function value is reduced (for example, set to zero).

[0052] FIG. 4 is a diagram showing a first example of the subspace constraint map 146. This subspace constraint map 146(1) specifies constraints on static targets, and is defined on a plane with the position and speed as axes. In the figure, A1 is a prohibited area, and A2 is a non-recommended area. Also, V1 is a speed limit such as a legal speed limit, and X1 is "a position where the vehicle should stop (the speed should be set to zero)". For example, the position of a stop line in front of a traffic light is set as X1. The calculation unit 142B sets an area where the speed is equal to or greater than V1 on a plane with the position and speed as axes as the prohibited area A1. Also, the calculation unit 142B sets an area where the vehicle cannot stop at the position X1 even if the vehicle decelerates to the maximum extent possible as the prohibited area A1. Also, the calculation unit 142B sets an area of ​​the boundary close to the prohibited area A1 in an area other than the prohibited area A1 as a non-recommended area A2. In other words, the prohibited area A1 and the non-recommended area are adjacent to each other. By integrating these, the subspace constraint map 146(1) shown in FIG. 4 is obtained. The gradient of the straight line portion A1a, which is the boundary line of the prohibited area A1, is based on the maximum deceleration permitted for autonomous driving. The gradient of the straight line portion A2a, which is the boundary line of the non-recommended area A2, is based on the deceleration recommended for autonomous driving (the absolute value of which is smaller than the maximum deceleration).

[0053] FIG. 5 is a diagram showing a second example of the subspace constraint map 146. This subspace constraint map 146(2) specifies constraints on dynamic objects, and is defined on a plane with time and position as axes. The subspace constraint map 146(2) is generated in a scene where the host vehicle M is traveling following a vehicle ahead. The vehicle ahead is a vehicle traveling in the same lane as the host vehicle M, immediately before the host vehicle M (meaning that there is no vehicle between them), and in the same direction as the host vehicle M. The calculation unit 142B predicts the future position of the vehicle ahead using a constant speed model, a constant acceleration model, a constant jerk model, a Kalman filter, or the like, and then sets the region occupied by the future vehicle ahead plus a margin region as the prohibited region A1, and sets the region from the boundary line A1b of the prohibited region A1 to a line obtained by translating the boundary line A1b by the target inter-vehicle distance TD as the non-recommended region A2.

[0054] The calculation unit 142B holds table information for determining how to define the prohibited area A1 and the non-recommended area A2 depending on road events (stop lines, traffic participants such as vehicles, and traffic light status) for each of the scenes illustrated in Figures 4 and 5, and obtains definition rules for the prohibited area A1 and the non-recommended area A2 from the table information depending on the event activated by the action plan generation unit 140, and defines the prohibited area A1 and the non-recommended area A2.

[0055] The calculation unit 142B fixes the state value SV(i) to zero (minimum value) for a state path i that passes through the prohibited area A1 at least once. In addition, the calculation unit 142B sets the reward function f(i, t) for a state (i, t) that exists in the non-recommended area A2 to zero. This also reduces the state value SV(i), but it does not mean that the state value SV(i) becomes zero, and there is a possibility that a state path i whose state (i, t) exists in the non-recommended area A2 may be selected. Note that the subspace constraint map 146 is not limited to being defined on a plane, and may be defined in a space of three or more dimensions.

[0056] The speed determination unit 144 determines the future speed of the host vehicle M according to the jerk at each time point of the state path with a high state value SV(i) calculated by reflecting the subspace constraint map 146. This suppresses unnecessary acceleration and deceleration, and determines a speed profile that does not pass through the prohibited area A1 according to the scene and does not pass through the non-recommended area A2 as much as possible.

[0057] 6 is a flowchart showing an example of the flow of processing executed by the state value calculation unit 142 and the speed determination unit 144. First, the state value calculation unit 142 acquires the surrounding conditions of the host vehicle M from the recognition unit 130 (step S100), and generates a subspace constraint map 146 according to the surrounding conditions (step S102).

[0058] Next, the state value calculation unit 142 generates a plurality of state paths by the above-mentioned method (step S104), calculates the reward function f(i,t) for each state path according to the subspace constraint map 146, and then calculates the state value SV(i) (step S106). Then, the speed determination unit 144 determines the speed profile of the host vehicle M according to the jerk at each time point of the state path with a high state value SV(i) (step S108).

[0059] According to the embodiment described above, when performing speed control taking into account future states, a suitable solution can be obtained by setting appropriate constraints.

[0060] The above-described embodiment can be expressed as follows. A storage device storing a program; a hardware processor; The hardware processor executes the program stored in the storage device, Calculating a state value for each of a plurality of state paths between a start time point and a target time point, the state path being defined by selecting, from a plurality of candidates, an amount of action occurring between a plurality of time points between the start time point and the target time point in a state space including time and having a plurality of elements related to the movement of the vehicle as axes; determining a future speed transition of the vehicle according to the state path with a high state value; When calculating the condition value, Calculating a reward function value based on one or more evaluation target quantities for each time point between the start time point and the target time point, and calculating the state value by summing the reward function values ​​in a time series; Calculating the state value according to a first region for decreasing the state value and a second region for decreasing the reward function value, the first region being set in a part or all of a plurality of subspaces having axes of two or more elements among the plurality of elements; The vehicle control device is configured as follows.

[0061] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0062] 100 Automatic driving control device 130 Recognition part 140 Action Plan Generation Unit 142 State Value Calculation Unit 142A State Path Definition Section 142B Calculation Section 144 Speed ​​determination section 146 Subspace Constraint Map

Claims

1. a state evaluation amount calculation unit that calculates a state evaluation amount for each of a plurality of state paths between a start time point and a target time point, the state path being defined by selecting, from a plurality of candidates, an amount of action occurring between a plurality of time points between a start time point and a target time point in a state space including time and having axes of a plurality of elements related to the movement of the vehicle; a speed determination unit that determines a future speed transition of the vehicle according to a state path having a high state evaluation value; Equipped with The state evaluation amount calculation unit is For each time point between the start time point and the target time point, a reward function value based on one or more evaluation target quantities is calculated, and the state evaluation quantity is calculated by summing the reward function values ​​in a time series, Calculating the state evaluation amount according to a first region for reducing the state evaluation amount and a second region for reducing the reward function value, the first region being set in a part or all of a plurality of subspaces having axes of two or more elements among the plurality of elements; Vehicle control device.

2. The plurality of elements include some or all of time, position in the direction of travel of the vehicle, velocity, and acceleration. The vehicle control device according to claim 1.

3. The amount of movement is a jerk. The vehicle control device according to claim 1 or 2.

4. The one or more evaluation target quantities include a speed, an acceleration, and a jerk of the vehicle. The vehicle control device according to any one of claims 1 to 3.

5. A recognition unit that recognizes a surrounding situation of the vehicle is further provided, The state evaluation amount calculation unit sets one or both of the first area and the second area in a part or all of the plurality of partial spaces based on a surrounding situation of the vehicle. The vehicle control device according to any one of claims 1 to 4.

6. the state evaluation amount calculation unit sets the first area and the second area in a subspace having axes of a position and a speed when a static target is included in the surrounding situation of the vehicle; The vehicle control device according to claim 5.

7. the state evaluation amount calculation unit sets the first area and the second area in a subspace having axes of position and time when a dynamic target object is included in the surrounding situation of the vehicle; The vehicle control device according to claim 5 or 6.

8. The vehicle control device according to claim 5 , wherein the state evaluation amount calculation unit sets the first region and the second region adjacent to each other.

9. A vehicle control device Calculating a state evaluation value for each of a plurality of state paths between a start time point and a target time point, the state path being defined by selecting, from a plurality of candidates, an amount of action occurring between a plurality of time points between the start time point and the target time point in a state space including time and having a plurality of elements related to the movement of the vehicle as axes; determining a future speed transition of the vehicle according to a state path having a high state metric; When calculating the condition value, Calculating a reward function value based on one or more evaluation target quantities for each time point between the start time point and the target time point, and calculating the state evaluation quantity by summing the reward function values ​​in a time series; Calculating the state evaluation amount according to a first region for reducing the state evaluation amount and a second region for reducing the reward function value, the first region being set in a part or all of a plurality of subspaces having axes of two or more elements among the plurality of elements; A vehicle control method.

10. On the computer, calculating a state evaluation amount for each of a plurality of state paths between a start time point and a target time point, the state path being defined by selecting, from a plurality of candidates, an amount of action occurring between a plurality of time points between the start time point and the target time point in a state space including time and having a plurality of elements related to the movement of the vehicle as axes; A program for determining a future speed transition of the vehicle according to a state path having a high state evaluation amount, When calculating the state evaluation amount, Calculating a reward function value based on one or more evaluation target quantities for each time point between the start time point and the target time point, and calculating the state evaluation quantity by summing the reward function values ​​in a time series; Calculating the state evaluation amount according to a first region for reducing the state evaluation amount and a second region for reducing the reward function value, the first region being set in a part or all of a plurality of subspaces having axes of two or more elements among the plurality of elements; program.

Images