Vehicle control system

Abstract

To appropriately control the vehicle's movement even on roads with varying curvature. [Solution] The vehicle control device comprises: external information acquisition means for acquiring external information including the road as a camera image; classification means for classifying the image region into a drivable region and a non-drivable region; curvature estimation means for calculating the virtual curvature of the road based on the driving state of the vehicle; virtual trajectory calculation means for calculating the virtual future trajectory of the vehicle in a bird's-eye view coordinate system based on the virtual curvature; determination means for determining whether the virtual future trajectory is included in the drivable region; future trajectory setting means for setting the virtual future trajectory as the vehicle's future trajectory based on the determination result of the determination means; and control means for performing driving control for the vehicle based on the vehicle's future trajectory. The determination means converts the virtual future trajectory calculated by the virtual trajectory calculation means from a bird's-eye view coordinate system to a perspective coordinate system of external information, and superimposes the virtual future trajectory on the external information to determine whether the virtual future trajectory is included in the drivable region.

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device that performs vehicle control based on the estimated shape of the travel path. [Background technology]

[0002] As a technique of this kind, there is a known technique that estimates the road shape within a first distance as a curve with a constant rate of curvature change, and estimates the road shape beyond the first distance as a curve with a constant curvature (see Patent Document 1). [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Patent No. 6285321 [Overview of the project] [Problems that the invention aims to solve]

[0004] Conventional techniques estimate the shape of distant roads based on the assumption that the curvature is constant beyond a certain distance. Therefore, it has been difficult to apply them to complex road shapes with inconsistent curvature, such as those found on ordinary roads. Since road curvature is necessary to generate the trajectory of a moving vehicle, accurately estimating curvature contributes to appropriate vehicle control in terms of factors such as speed and steering angle. In other words, it becomes possible to improve traffic safety while suppressing a decline in traffic flow. [Means for solving the problem]

[0005] A vehicle control device according to one aspect of the present invention includes: external information acquisition means for acquiring external information of the surroundings of the vehicle, including the road, as an image; classification means for classifying the image region into a drivable region and a non-drivable region by predetermined segmentation processing; curvature estimation means for calculating a virtual curvature as an estimated value of the curvature of the road based on the driving state, including the steering angle of the vehicle; virtual trajectory calculation means for calculating a virtual future trajectory as an estimated value of the future trajectory of the vehicle in a bird's-eye view coordinate system based on the virtual curvature; determination means for comparing the external information and the virtual future trajectory and determining whether the virtual future trajectory is included in the drivable region; future trajectory setting means for setting the virtual future trajectory as the vehicle's future trajectory based on the determination result of the determination means; and control means for performing driving control for the vehicle based on the vehicle's future trajectory. The determination means converts the virtual future trajectory calculated by the virtual trajectory calculation means from a bird's-eye view coordinate system to a perspective coordinate system of the external information, and determines whether the virtual future trajectory is included in the drivable region by superimposing the virtual future trajectory on the external information. [Effects of the Invention]

[0006] According to the present invention, it becomes possible to appropriately control the vehicle's movement even on roads with varying curvature. [Brief explanation of the drawing]

[0007] [Figure 1] A diagram illustrating the configuration of a vehicle control device according to an embodiment of the present invention. [Figure 2] A block diagram illustrating the configuration of a speed control device. [Figure 3] A block diagram illustrating the main components of the speed control device. [Figure 4A] A schematic diagram illustrating the process for determining the curvature of a road surface. [Figure 4B] A schematic diagram illustrating the updating of curvature. [Figure 4C] A schematic diagram illustrating the relationship between virtual curvature, radius of curvature, and position before and after the update. [Figure 4D] A schematic diagram illustrating the estimated value of the virtual curvature. [Figure 5A]A flowchart for explaining an example of arithmetic processing executed by an arithmetic unit based on a program. [Figure 5B] A flowchart for explaining the details of pre-judgment processing.

Embodiments for Carrying Out the Invention

[0008] Hereinafter, embodiments of the invention will be described with reference to the drawings. As an example of a speed control device which is a vehicle control device according to an embodiment of the present invention, when a vehicle follows a target path (which may be called a target trajectory) on a traveling road, the traveling speed of the vehicle is controlled so that longitudinal and lateral accelerations not less than a predetermined value do not occur. Further, the steering angle by a steering device (for example, a power steering device) may be controlled so that the traveling position of the vehicle follows the target trajectory. The speed control device can be applied to, for example, a vehicle having an automatic driving function, that is, an autonomous vehicle. Note that the speed control device according to the embodiment is applicable to both a manual driving vehicle having a driving support function and an autonomous vehicle. For the sake of convenience of explanation, hereinafter, the case of applying it to an autonomous vehicle will be taken as an example. In addition, in the embodiment, the vehicle on which the speed control device is mounted may be referred to as a host vehicle to distinguish it from other vehicles. The host vehicle may be any of an engine vehicle having an internal combustion engine (engine) as a traveling drive source, an electric vehicle having a traveling motor as a traveling drive source, and a hybrid vehicle having an engine and a traveling motor as traveling drive sources. The host vehicle can travel not only in an automatic driving mode in which driving operation by a driver is unnecessary but also in a manual driving mode by the driver's driving operation.

[0009] <Configuration of Vehicle> First, the general configuration of the vehicle involved in autonomous driving will be described. Figure 1 is a block diagram illustrating the configuration of the vehicle control device 200 of the vehicle having a speed control device according to the embodiment. As shown in Figure 1, the vehicle control device 200 mainly comprises a controller 10, a group of external sensors 1 and 2, an input / output device 3, a positioning unit 4, a map database 5, a navigation device 6, a communication unit 7, and an actuator AC for driving.

[0010] External sensor group 1 is a collective term for multiple sensors (external sensors) that detect external conditions, which are information about the surroundings of the vehicle. External sensor group 1 includes, for example, a lidar that measures the distance from the vehicle to surrounding obstacles by measuring scattered light from the vehicle's omnidirectional illumination; a radar that detects other vehicles and obstacles around the vehicle by emitting electromagnetic waves and detecting reflected waves; and a camera mounted on the vehicle that has an image sensor such as a CCD or CMOS sensor to capture images of the area around the vehicle (front, rear, and sides).

[0011] Internal sensor group 2 is a collective term for multiple sensors (internal sensors) that detect the driving state of the vehicle. Internal sensor group 2 includes, for example, a vehicle speed sensor that detects the vehicle's speed, acceleration sensors that detect the vehicle's acceleration in the longitudinal direction (direction of travel) and lateral direction (lane width direction), a rotation speed sensor that detects the rotation speed of the driving source, and a yaw rate sensor that detects the rotational angular velocity of the vehicle's center of gravity around the vertical axis. Sensors that detect the driver's driving operations in manual driving mode, such as operation of the accelerator pedal, brake pedal, and steering wheel, are also included in internal sensor group 2.

[0012] Input / output device 3 is a general term for devices that receive commands from the driver or output information to the driver. Input / output device 3 includes, for example, various switches that the driver uses to input commands by operating control members, a microphone that the driver uses to input commands by voice, a display that provides information to the driver via displayed images, and a speaker that provides information to the driver by voice.

[0013] The positioning unit (GNSS unit) 4 has a positioning sensor that receives positioning signals transmitted from positioning satellites. Positioning satellites are artificial satellites such as GPS satellites and quasi-zenith satellites. The positioning unit 4 uses the positioning information received by the positioning sensor to measure the current position (latitude, longitude, altitude) of the vehicle.

[0014] The map database 5 is a device that stores general map information used in the navigation device 6, and is composed of, for example, a magnetic disk or semiconductor elements. The map information may include road location information, road shape information (curvature, etc.), and location information of intersections and junctions. Note that the map information stored in the map database 5 is different from the high-precision map information stored in the storage unit 12 of the controller 10.

[0015] The navigation device 6 is a device that, for example, searches for a route along the road to a destination entered by the driver and provides driving guidance along the searched route. The input of the destination and driving guidance along the searched route are performed via the input / output device 3. The route search is performed based on the current position of the vehicle measured by the positioning unit 4, the position of the entered destination, and the map information stored in the map database 5. The current position of the vehicle can also be measured using the detection values ​​of the external sensor group 1, and the route may be searched based on this current position and high-precision map information stored in the storage unit 12.

[0016] The communication unit 7 communicates with various servers (not shown) via a network including wireless communication networks such as the Internet and mobile phone networks, and obtains map information, driving history information, and traffic information from the servers periodically or at arbitrary times. In addition to obtaining driving history information, the communication unit 7 may also transmit its own vehicle's driving history information to the server. The network includes not only public wireless communication networks but also closed communication networks established for each predetermined management area, such as wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), etc. The acquired map information is output to the map database 5 and the storage unit 12, and the map information is updated.

[0017] Actuators AC are actuators used to control the movement of the vehicle. When the driving source is an engine, actuator AC includes a throttle actuator that adjusts the opening degree (throttle opening) of the engine's throttle valve. When the driving source is a motor, the motor is included in actuator AC. Brake actuators that operate the vehicle's braking system and actuators that drive the steering system are also included in actuator AC.

[0018] The controller 10 is comprised of an electronic control unit (ECU). More specifically, the controller 10 includes a computer having an arithmetic unit 11 such as a CPU (microprocessor), a storage unit 12 such as ROM and RAM, and other peripheral circuits (not shown) such as an I / O interface. Although it is possible to provide multiple ECUs with different functions, such as an ECU for engine control, an ECU for drive motor control, and an ECU for braking system, for convenience, Figure 1 shows the controller 10 as a collection of these ECUs.

[0019] The memory unit 12 stores high-precision, detailed map information for autonomous driving. The high-precision map information may include information on the location of roads, information on road shapes (radius of curvature, etc.), information on road gradients, information on the location of intersections and junctions, information on the type and location of lane markings such as white lines, information on the number of lanes (driving lanes), lane width and location information for each lane (information on the center position of the lane and the boundary lines of the lane positions), location information of landmarks as markers on the map (traffic lights, signs, buildings, etc.), and information on road surface profiles such as road surface irregularities. The high-precision map information stored in the memory unit 12 may include high-precision map information acquired from outside the vehicle via the communication unit 7, or it may include high-precision map information created by the vehicle itself using detection values ​​from the external sensor group 1 or detection values ​​from the external sensor group 1 and the internal sensor group 2. The memory unit 12 may also store information such as various control programs and thresholds used in the programs. The calculation unit 11 has a functional configuration that includes a vehicle position recognition unit 13, an external environment recognition unit 14, an action plan generation unit 15, and a driving control unit 16.

[0020] The vehicle position recognition unit 13 recognizes the vehicle's position on the map (vehicle position) based on the vehicle's position information obtained by the positioning unit 4 and the map information in the map database 5. The vehicle position may also be recognized using high-precision map information stored in the storage unit 12 and surrounding information of the vehicle detected by the external sensor group 1, thereby enabling high-precision recognition of the vehicle's position. Furthermore, the vehicle's movement information (direction of movement, distance traveled) can be calculated based on the detection values ​​of the internal sensor group 2, and the vehicle's position can be recognized accordingly. Furthermore, when the vehicle's position can be measured by sensors installed on or beside the road, the vehicle's position can also be recognized by communicating with those sensors via the communication unit 7.

[0021] The external environment recognition unit 14 recognizes the external conditions around the vehicle based on signals from the external sensor group 1, such as cameras, lidars, and radars. For example, it recognizes the position, speed, and acceleration of surrounding vehicles (vehicles in front and behind) traveling around the vehicle, the position of surrounding vehicles stopped or parked around the vehicle, and the position and state of other objects, and creates target information. In this embodiment, it is sufficient to recognize the external conditions around the vehicle based on signals from at least the camera. Other objects include signs, traffic lights, roads, buildings, guardrails, utility poles, billboards, pedestrians, and bicycles. Markings on the road surface, such as lane markings (white lines, etc.) and stop lines, are also included in other objects (roads). The state of other objects includes the color of traffic lights (red, blue, yellow), the speed and direction of pedestrians and cyclists, etc. Some of the stationary objects among the other objects constitute landmarks that serve as indicators of location on the map, and the external environment recognition unit 14 also recognizes the location and type of these landmarks.

[0022] The action plan generation unit 15 generates a driving trajectory (target trajectory) for the vehicle from the present time to a predetermined time in advance, based on, for example, the route searched by the navigation device 6, the high-precision map information stored in the memory unit 12, the vehicle's position recognized by the vehicle position recognition unit 13, and the external conditions recognized by the external environment recognition unit 14. If there are multiple possible target trajectories on the route searched by the navigation device 6, the action plan generation unit 15 selects the optimal trajectory from among them that complies with laws and regulations and satisfies criteria such as efficient and safe driving, and generates the selected trajectory as the target route. The action plan generation unit 15 then generates an action plan corresponding to the generated target route. The action plan generation unit 15 generates various action plans corresponding to driving modes such as overtaking to overtake a preceding vehicle, changing lanes, following a preceding vehicle, lane keeping to maintain the lane, decelerating, or accelerating. When generating a target route, the action plan generation unit 15 first determines the driving mode and generates the target route based on the driving mode (this may be called a trajectory plan). Then, it determines the steering angle (this may be called a steering angle plan) and the driving speed (this may be called a speed plan) so as to follow the target route and so as not to generate lateral acceleration exceeding a specified value.

[0023] In autonomous driving mode, the driving control unit 16 controls each actuator AC so that the vehicle travels along the target path generated by the action plan generation unit 15. For example, in autonomous driving mode, the driving control unit 16 considers the driving resistance determined by the road gradient, etc., and calculates the required driving force to achieve the speed plan (e.g., target acceleration per unit time) calculated by the action plan generation unit 15. Then, it provides feedback control to the actuator AC so that the actual acceleration detected by, for example, the internal sensor group 2 becomes the target acceleration. In other words, it controls the drive actuator AC so that the vehicle travels at the target speed and target acceleration. Furthermore, in the automatic driving mode, the driving control unit 16 outputs a steering angle instruction signal to realize the steering angle plan (the optimal steering angle for the vehicle to follow a target path) calculated by the action plan generation unit 15 based on vehicle state quantities observed by the internal sensor group 2, etc., and controls the steering actuator AC. In manual driving mode, the driving control unit 16 controls each actuator AC in accordance with driving commands (such as steering operations) from the driver acquired by the internal sensor group 2.

[0024] <Regarding the estimation of the radius of curvature> However, for example, when a vehicle travels for the first time on a road that is not stored as high-precision map information in the memory unit 12, or when the position of the driving lane is temporarily changed due to construction or other reasons, and the shape of the road the vehicle is traveling on differs from the high-precision map information, the vehicle cannot refer to the existing high-precision map information. In this embodiment, when driving on such a road, the actual shape of the road (radius of curvature in this embodiment) is estimated based on camera images acquired by the camera, which is part of the external sensor group 1. By appropriately estimating the radius of curvature of the road, it becomes possible to drive along a target route, maintain a lane, or drive within a predetermined lateral acceleration limit, even on roads not included in high-precision map information. Note that the reciprocal of the radius of curvature is the curvature. In the following explanation, the radius of curvature may sometimes be simply referred to as curvature. Also, the road on which a vehicle travels may sometimes be referred to as the roadway.

[0025] A characteristic of camera images (perspective views) acquired by typical cameras is that objects farther away from the camera (in this embodiment, the road surface) appear smaller (in other words, the further away from the vehicle, the fewer pixels make up the road surface in the camera image), resulting in lower resolution for distant road surfaces. This makes it difficult to accurately estimate the curvature of distant roads based on camera images. Estimating curvature is particularly difficult for roads with inconsistent curvature. Therefore, in this embodiment, the shape of the road surface is represented as a bird's-eye view of the road from above, the vehicle's trajectory (referred to as a virtual future trajectory) is calculated in the bird's-eye view corresponding to the curvature of the road estimated in the bird's-eye view, this virtual future trajectory is transformed from a bird's-eye view coordinate system to a perspective view coordinate system, and the transformed virtual future trajectory is superimposed on the camera image.

[0026] Specifically, the process involves repeatedly estimating (updating) the curvature of the road until the virtual future trajectory after the coordinate transformation falls within the road surface area (referred to as the drivable area) in the camera image, calculating the vehicle's virtual future trajectory corresponding to the updated curvature in the bird's-eye view coordinates, and then transforming the virtual future trajectory calculated in the bird's-eye view coordinates and superimposing it onto the camera image shown in perspective coordinates. In this embodiment, the process of repeatedly estimating (updating) the curvature, calculating the virtual future trajectory, and superimposing it onto the camera image is called a search. It may also be called a curvature search to distinguish it from the search performed by the navigation device 6 to find a route. The speed control device according to this embodiment appropriately estimates the curvature necessary for speed planning by performing a curvature search, thereby ensuring that the driving trajectory in the camera image matches the drivable area while assuming a realistic driving trajectory. Even for roads with inconsistent curvature, the system appropriately estimates the curvature, enabling proper vehicle control based on the estimated curvature. The configuration of this speed control device will be described in more detail below.

[0027] <Speed ​​control device> Figure 2 is a block diagram illustrating the configuration of a speed control device 50 according to an embodiment. Figure 3 is a block diagram illustrating the main parts of the speed control device. For example, the speed control device 50 is configured as part of the functions of the controller 10 in Figure 1 and performs a part of the functions of the vehicle control device 200. The controller 10 is connected to a camera 1a, a steering angle sensor 2a, a steering angular velocity sensor 2b, a steering torque sensor 2c, a vehicle speed sensor 2d, an acceleration sensor 2e, a navigation device 6, and an actuator AC.

[0028] Camera 1a is a monocular camera having an image sensor and constitutes part of the external sensor group 1 in Figure 1. Camera 1a is mounted, for example, at a predetermined position on the front of the vehicle and continuously captures images of the space in front of the vehicle at a predetermined frame rate (e.g., 10 frames / second) to acquire images of objects (camera images). Objects include lane markings on the road. In addition, objects may be detected by radar, lidar, etc., along with camera 1a.

[0029] The steering angle sensor 2a detects, for example, the rotation angle (steering angle) of the steering shaft connected to a steering wheel (not shown). The steering angular velocity sensor 2b detects the rotational angular velocity (steering angular velocity) of the steering shaft. Steering angular velocity may also be simply called steering angular velocity. The steering torque sensor 2c detects the steering operation by the driver, more specifically, the steering torque acting on the steering wheel. For example, the steering angle detected by the steering angle sensor 2a when the steering wheel is rotated counterclockwise from the neutral position is defined as a positive value, and the steering angle detected by the steering angle sensor 2a when the steering wheel is rotated clockwise from the neutral position is defined as a negative value. The steering angle sensor 2a, steering angular velocity sensor 2b, and steering torque sensor 2c described above constitute a part of the internal sensor group 2 shown in Figure 1.

[0030] Furthermore, the vehicle speed sensor 2d detects the vehicle's speed. The acceleration sensor 2e detects the vehicle's acceleration in the longitudinal and lateral directions, respectively. The vehicle speed sensor 2d and acceleration sensor 2e also constitute part of the internal sensor group 2 in Figure 1. Furthermore, one of the internal sensor group 2 may include an IMU (Inertial Measurement Unit) that detects the translational and rotational motion of the vehicle in the three axes.

[0031] The controller 10 has the following functional configurations, which are handled by the calculation unit 11 (Figure 1): an odometry calculation unit 131, a target calculation unit 141, a classification unit 142, a curvature estimation unit 143, a determination unit 144, a previous plan update unit 145, a virtual trajectory calculation unit 146, a future trajectory setting unit 151, and a velocity planning unit 152. In addition, as described above, the controller 10 has a storage unit 12. The odometry calculation unit 131 may constitute part of the vehicle position recognition unit 13. The target calculation unit 141, classification unit 142, curvature estimation unit 143, determination unit 144, previous plan update unit 145, and virtual trajectory calculation unit 146 may constitute part of the external environment recognition unit 14. The future trajectory setting unit 151 and speed planning unit 152 may constitute part of the action plan generation unit 15. The previous curvature retention unit 121 may constitute part of the memory unit 12.

[0032] <Amount of movement> The odometry calculation unit 131 calculates the amount of movement of the vehicle based on the vehicle speed information detected by the vehicle speed sensor 2d, the amount of wheel rotation, etc.

[0033] <Target> The target calculation unit 141 calculates information indicating targets present around the vehicle. Based on signals input from the external sensor group 1, including the camera 1a, lidar, and radar, the target calculation unit 141 recognizes targets including moving objects such as other vehicles, bicycles, and pedestrians, as well as stationary objects (which may also be called terrain features) such as guardrails and signs, and outputs target information indicating the recognized targets.

[0034] <Drivable area> The classification unit 142 performs a predetermined segmentation process on the camera image, which is external information about the vehicle's surroundings, to classify the image into drivable areas and non-drivable areas. The drivable area is the road surface area in the direction of travel on the road (roadway), and the non-drivable area is the area other than the drivable area.

[0035] <Virtual curvature> The curvature estimation unit 143 calculates a virtual curvature as an estimated value of the curvature of the road based on the driving state of the vehicle, including steering angle information detected by the steering angle sensor 2a and vehicle speed information detected by the vehicle speed sensor 2d. In this embodiment, the virtual curvature calculated by the curvature estimation unit 143 is called the estimated value. Furthermore, when the estimated value is updated by the curvature update unit 144C during the determination process by the determination unit 144, which will be described later, the virtual curvature is called the updated value.

[0036] <Collision detection> The determination unit 144 includes a coordinate transformation unit 144A, a collision determination unit 144B, and a curvature update unit 144C. The coordinate transformation unit 144A performs a coordinate transformation from a bird's-eye view coordinate system to a perspective view coordinate system. Specifically, it transforms the coordinate system from a bird's-eye view coordinate system, which shows the road and other structures viewed from above, to a perspective view coordinate system corresponding to the camera image. In this embodiment, a collision is defined as the virtual future trajectory deviating from a drivable area into an undrivable area. The collision determination unit 144B compares the drivable area included in the camera image as external information with the virtual future trajectory calculated by the virtual trajectory calculation unit 146 (described later) to determine whether the virtual future trajectory is included in the drivable area (in other words, whether the virtual future trajectory collides with an undrivable area). If the virtual future trajectory is included in the drivable area, it means that the estimated curvature of the track is appropriate. Conversely, if the virtual future trajectory collides with an undrivable area, it means that the virtual future trajectory deviates from the drivable area into an undrivable area, in other words, that the estimated curvature of the track is inappropriate. The curvature update unit 144C updates the estimated value (initial curvature) when the hypothetical future trajectory deviates from the drivable region to the undrivable region. This curvature update may also be called a correction.

[0037] <Previous plan update> The previous plan update unit 145 outputs information about the curvature (estimated or updated value) adopted during the previous curvature search to the curvature estimation unit 143 and the curvature update unit 144C in order to efficiently search for curvature (in other words, to reduce the number of times the curvature is updated). The information about the curvature adopted during the previous curvature search is temporarily stored in the previous curvature retention unit 121 in the memory unit 12. As a result, the curvature estimation unit 143 can calculate a virtual curvature based on the steering angle information detected by the steering angle sensor 2a and the curvature used in the previous curvature search. Furthermore, the curvature update unit 144C can update the virtual curvature using the curvature adopted in the previous curvature search.

[0038] <Virtual Trajectory> The virtual trajectory calculation unit 146 calculates a virtual future trajectory as an estimated value of the trajectory the vehicle will travel in the future (referred to as the future trajectory) based on the most recent virtual curvature. The virtual trajectory calculation unit 146 calculates a new virtual future trajectory each time an estimated value of virtual curvature is calculated by the curvature estimation unit 143 or the virtual curvature is updated by the curvature update unit 144C, based on the estimated or updated value of virtual curvature and the virtual future trajectory calculated in the past.

[0039] <Future trajectory> The future track setting unit 151 sets the virtual future track as the future track of its own vehicle (which may also be called the vehicle's future track).

[0040] <Speed ​​Plan> The speed planning unit 152 determines the travel speed in parallel with the determination of the steering angle by the steering angle planning unit (not shown), so as to follow the vehicle's future trajectory as the target path and so as not to generate lateral acceleration exceeding a specified value.

[0041] <Driving control> The driving control unit 16 outputs instruction information to each actuator AC so that the vehicle travels along the future trajectory of the vehicle at the determined planned speed and planned steering angle.

[0042] <Flow of curvature search> The process of searching for the curvature of the track so that the virtual future trajectory after coordinate transformation falls within the traversable area will be explained with reference to Figure 4A. In this embodiment, the curvature search process is divided into three phases: (a) In the first phase, the curvature of the road is estimated or updated. (b) In the second phase, a virtual future trajectory of the vehicle corresponding to the curvature is calculated in bird's-eye view coordinates. (c) In the third phase, the virtual future trajectory calculated in bird's-eye view coordinates is transformed and superimposed onto the camera image shown in perspective view coordinates.

[0043] Furthermore, in this embodiment, the above-mentioned (a) first phase to (c) third phase are treated as a single set process, and the set process is repeated multiple times until the virtual future trajectory after the coordinate transformation falls within the drivable area in the camera image. Figure 4A illustrates the case where the set process is repeated four times from (1) to (4).

[0044] (a) Phase 1 In Figure 4A, when a new frame of camera image is acquired by camera 1a, the calculation unit 11 uses the classification unit 142 to classify the area of ​​the camera image into the drivable area and the non-drivable area as described above. In parallel with the classification of camera image regions by the classification unit 142, the calculation unit 11 performs the first phase of the (1) set processing. In the first phase of processing, the calculation unit 11 determines the search start position 101 at a position that has advanced a distance s0 in the direction of travel of the vehicle (to the right in the figure). For example, the search start position 101 is the position corresponding to point p0, which corresponds to the bottom of the screen of the camera image F when it is later converted to perspective coordinates. The calculation unit 11 further estimates the virtual curvature 100 for the camera image of the current frame using the curvature estimation unit 143, based on the steering angle information detected by the steering angle sensor 2a and the curvature updated during the previous curvature search for the camera image of the previous frame.

[0045] (b) Phase 2 Once the calculation unit 11 estimates the virtual curvature 100, it performs the second phase of the (1) set processing in bird's-eye view coordinates. In the second phase of processing, the virtual trajectory calculation unit 146 of the calculation unit 11 calculates a virtual future trajectory 102 as an estimated value of the future trajectory that the vehicle will travel in the future, based on the virtual curvature 100. The virtual future trajectory 102 may also be called the predicted travel position.

[0046] The calculation unit 11 further generates a first detection line 103 with the same shape as the virtual future trajectory 102 on the left side in the direction of travel of the vehicle, relative to the virtual future trajectory 102, at a predetermined detection line interval d, and generates a second detection line 104 with the same shape as the virtual future trajectory 102 on the right side in the direction of travel of the vehicle, relative to the virtual future trajectory 102, at the same detection line interval d. The detection line interval d may be, for example, the width of the vehicle body, or a value obtained by adding a margin to the width of the vehicle body.

[0047] (c) Phase 3 When the calculation unit 11 generates the virtual future trajectory 102, the first detection line 103, and the second detection line 104, it performs the processing of the third phase in the (1)th set processing by converting from the bird's-eye view coordinate system to the perspective view coordinate system.

[0048] The calculation unit 11 uses the coordinate transformation unit 144A of the determination unit 144 to perform coordinate transformations on the virtual future trajectory 102, the first detection line 103, and the second detection line 104, respectively, from the bird's-eye view coordinate system to the perspective view coordinate system, thereby generating the virtual future trajectory, the first detection line, and the second detection line in perspective view coordinates.

[0049] In the (1)th set processing in Figure 4A, the camera image F of one frame, shown in perspective coordinates, is classified by the classification unit 142 into a drivable area 107 and a non-drivable area 108. The calculation unit 11 superimposes the virtual future trajectory after coordinate transformation (thick line), the first detection line 105, and the second detection line 106 onto the camera image F.

[0050] The collision determination unit 144B compares the drivable area 107 included in the camera image F with the virtual future trajectory (thick line) in the perspective coordinates and determines whether the virtual future trajectory (thick line) is included in the drivable area 107 (in other words, whether the virtual future trajectory (thick line) collides with the non-drivable area 108).

[0051] As an example, the collision determination unit 144B checks whether the first detection line 105 and the second detection line 106 are included in the drivable area 107, following the trajectories of the first detection line 105 and the second detection line 106 in the direction of travel (upward) starting from the position of point p0 corresponding to the search start position 101. At this time, if at least one of the first detection line 105 and the second detection line 106 changes from the drivable area 107 to the non-drivable area 108 (in other words, if at least one of the first detection line 105 and the second detection line 106 collides with the non-drivable area 108), it can be considered that the curvature of the travel path has not been properly estimated. Conversely, if both the first detection line 105 and the second detection line 106 are included within the drivable area 107 (in other words, if the first detection line 105 and the second detection line 106 do not collide with the non-drivable area 108), it can be considered that the curvature of the travel path has been properly estimated. Therefore, the determination unit 144 performs a curvature update using the curvature update unit 144C when at least one of the first detection line 105 and the second detection line 106 collides with the impassable area 108.

[0052] Specifically, in the camera image F, if the first detection line 105 collides with the impassable area 108 at point c0, the curvature update unit 144C calculates the distance e0 corresponding to point c0 in bird's-eye view coordinates. Then, the curvature update unit 144C updates the curvature of the section from point s0 to point e0 in the negative direction so that the curvature curves further to the right (to the right in the direction of travel). The amount of the update will be described later.

[0053] As another example (not shown), in camera image F, if the second detection line 106 collides with the impassable area 108, the curvature update unit 144C calculates the distance e0 corresponding to the collision point in bird's-eye view coordinates. Then, the curvature update unit 144C updates the curvature of the section from point s0 to point e0 in the positive direction so that the curvature curves further to the left (to the left in the direction of travel). The amount of the update will be described later.

[0054] When the curvature update unit 144C performs the curvature update, the calculation unit 11 performs the (2)th set process. (a) Phase 1 The calculation unit 11 performs the first phase of the (2)th set processing. In the first phase of processing, the calculation unit 11, using the curvature update unit 144C, updates the virtual curvature 100 estimated in the (1)th set processing, specifically the section from point s0 to point e0, to virtual curvature 100a. The amount of the update will be described later.

[0055] (b) Phase 2 When the calculation unit 11 updates the virtual curvature 100a, it performs the second phase of the (2)th set process in bird's-eye view coordinates. The calculation unit 11 starts the search from a point s1 that is a predetermined distance in the direction of travel of its own vehicle (to the right in the diagram) from the point s0 where the search was started during the (1)th set process. The reason for starting the search in the (2)th set process from point s0 where the search was started during the (1)th set process is that sections where the first detection line 105 and the second detection line 106 do not collide with the impassable area 108 do not need to be included as the search target during the (2)th set process. Furthermore, the amount by which the starting position of the search is moved from point s0 to point s1 is sufficient if, for example, the amount of movement from point p0 to point p1 in the camera image F when converted to perspective coordinates corresponds to at least 1 pixel. In other words, the amount of movement from point p0 to point p1 may be greater than the amount corresponding to 1 pixel. In the second phase of processing, the virtual trajectory calculation unit 146 of the calculation unit 11 calculates a virtual future trajectory 102a as an estimated value of the future trajectory that the vehicle will travel in the future, based on the updated virtual curvature 100a.

[0056] The calculation unit 11 further generates a first detection line 103a with the same shape as the virtual future trajectory 102a on the left side in the direction of travel of the vehicle relative to the virtual future trajectory 102a at a predetermined detection line interval d, and generates a second detection line 104a with the same shape as the virtual future trajectory 102a on the right side in the direction of travel of the virtual future trajectory 102a at a predetermined detection line interval d.

[0057] (c) Phase 3 When the calculation unit 11 generates the virtual future trajectory 102a, the first detection line 103a, and the second detection line 104a, it performs the processing of the third phase in the (2)th set processing by converting from the bird's-eye view coordinate system to the perspective view coordinate system.

[0058] The calculation unit 11 uses the coordinate transformation unit 144A of the determination unit 144 to perform coordinate transformations on the virtual future trajectory 102a, the first detection line 103a, and the second detection line 104a from the bird's-eye view coordinate system to the perspective view coordinate system, thereby generating the virtual future trajectory, the first detection line, and the second detection line in perspective view coordinates.

[0059] In the (2nd) setting process in Figure 4A, similar to the (1st) setting process, the camera image F of one frame, shown in perspective coordinates, is classified into a drivable area 107 and a non-drivable area 108. The calculation unit 11 superimposes the virtual future trajectory after coordinate transformation (thick line), the first detection line 105a, and the second detection line 106a onto the camera image F.

[0060] The collision determination unit 144B, similar to the (1) setting process, compares the drivable area 107 included in the camera image F with the virtual future trajectory (thick line) in the perspective coordinates, and determines whether the virtual future trajectory (thick line) is included in the drivable area 107 (in other words, whether the virtual future trajectory (thick line) collides with the non-drivable area 108).

[0061] As an example, the collision determination unit 144B checks whether the first detection line 105a and the second detection line 106a are included in the drivable area 107, following the trajectories of the first detection line 105a and the second detection line 106a in the direction of travel (upward) from the position of point p1. The procedure is the same as the (1) setting process. If at least one of the first detection line 105a and the second detection line 106a collides with the non-drivable area 108, the determination unit 144C performs a curvature update again.

[0062] Specifically, in the camera image F, if the first detection line 105a collides with the impassable area 108 at the position of point c1, the curvature update unit 144C calculates the distance e1 corresponding to point c1 in bird's-eye view coordinates. Then, the curvature update unit 144C updates the curvature of the section from point s1 to point e1 in the negative direction so that the curvature curves further to the right (to the right in the direction of travel). The amount of the update will be described later.

[0063] As another example (not shown), in camera image F, if the second detection line 106a collides with the impassable area 108, the curvature update unit 144C calculates the distance e1 corresponding to the collision point in bird's-eye view coordinates. Then, the curvature update unit 144C updates the curvature of the section from point s1 to point e1 in the positive direction so that the curvature curves further to the left (to the left in the direction of travel). The amount of the update will be described later.

[0064] The (3rd) set process in Figure 4A is the same as the (2nd) set process described above, so we will omit the explanation. (3) When the curvature update unit 144C updates the curvature during the set processing, the calculation unit 11 performs the (4) set processing. (a) Phase 1 The calculation unit 11 performs the first phase of the (4th) set processing. In the first phase of processing, the calculation unit 11, using the curvature update unit 144C, updates the virtual curvature 100b estimated in the (3)th set processing, specifically the section from point s2 to point e2, to virtual curvature 100c. The amount of the update will be described later.

[0065] (b) Phase 2 When the calculation unit 11 updates to the virtual curvature 100c, it performs the second phase of the (4th) set process in bird's-eye view coordinates. The calculation unit 11 starts the search from a point s3 that is a predetermined distance in the direction of travel of the vehicle (to the right in the figure) from point s2, where the search was started during the (3rd) set process. The reason for starting the search in the (4th) set process from point s2, where the search was started during the (3rd) set process, is that sections where the first detection line 105b and the second detection line 106b do not collide with the impassable area 108 do not need to be included as the search target during the (4th) set process. The amount by which the starting position of the search is moved from point s2 to point s3 is sufficient if, for example, the amount of movement from point p2 to point p3 in the camera image F when converted to perspective coordinates corresponds to at least 1 pixel. In other words, the amount of movement from point p2 to point p3 may be greater than the amount corresponding to 1 pixel. In the second phase of processing, the virtual trajectory calculation unit 146 of the calculation unit 11 calculates a virtual future trajectory 102c as an estimated value of the future trajectory that the vehicle will travel in the future, based on the updated virtual curvature 100c.

[0066] The calculation unit 11 further generates a first detection line 103c with the same shape as the virtual future trajectory 102c on the left side in the direction of travel of the vehicle relative to the virtual future trajectory 102c at a predetermined detection line interval d, and generates a second detection line 104c with the same shape as the virtual future trajectory 102c on the right side in the direction of travel of the virtual future trajectory 102c at a predetermined detection line interval d.

[0067] (c) Phase 3 When the calculation unit 11 generates the virtual future trajectory 102c, the first detection line 103c, and the second detection line 104c, it performs the processing of the third phase in the (4th) set processing by converting from the bird's-eye view coordinate system to the perspective view coordinate system.

[0068] The calculation unit 11 uses the coordinate transformation unit 144A of the determination unit 144 to perform coordinate transformations on the virtual future trajectory 102c, the first detection line 103c, and the second detection line 104c from the bird's-eye view coordinate system to the perspective view coordinate system, thereby generating the virtual future trajectory, the first detection line, and the second detection line in perspective view coordinates.

[0069] In the (4th) set processing in Figure 4A, the camera image F of one frame, shown in perspective coordinates, is classified into a drivable area 107 and a non-drivable area 108, which is the same as in the set processing described earlier. The calculation unit 11 superimposes the virtual future trajectory after coordinate transformation (thick line), the first detection line 105c, and the second detection line 106c onto the camera image F.

[0070] The collision determination unit 144B compares the drivable area 107 included in the camera image F with the virtual future trajectory (thick line) and determines whether the virtual future trajectory (thick line) is included in the drivable area 107 (in other words, whether the virtual future trajectory (thick line) collides with the non-drivable area 108).

[0071] For example, the determination unit 144 terminates the setting process if both the first detection line 105c and the second detection line 106c do not collide with the impassable area 108. Once the setting process is completed, the future trajectory setting unit 151 sets the virtual future trajectory as the vehicle's future trajectory.

[0072] The calculation unit 11 performs the set processing described above on the camera image of the same frame, in accordance with the frame rate at which camera 1a acquires camera images. More specifically, once camera 1a acquires one frame of camera image, the above set processing is repeated multiple times until the next frame of camera image is acquired. An upper limit (for example, 10 times) may be set for the number of times the set processing is repeated. As with the (4th) set process described above, if the first detection line 105x and the second detection line 106x after the coordinate transformation fall within the drivable area in the camera image before the number of set process repetitions reaches the upper limit (10 times) (in other words, if neither the first detection line 105x nor the second detection line 106x collides with the non-drivable area 108), the set process may be terminated at that point. Furthermore, if the first detection line 105 and the second detection line 106 after the coordinate transformation are within the drivable area in the camera image after the (1) setting process, the setting process may be terminated without repeating it.

[0073] Furthermore, if the road is at the end of a road, such as a T-junction, or if there is a vehicle ahead of the vehicle on the road, the first detection line 105 and the second detection line 106 may not fall within the drivable area during the curvature search process. In this case, the curvature search may be stopped, and the planned speed may be set to 0 relative to the search position. Doing so makes it possible to prevent the vehicle from going into an area where it cannot be driven.

[0074] <Update amount> Figure 4B is a schematic diagram illustrating the curvature update. Figure 4B is an enlarged view of the portion corresponding to the (a) first phase of the (2)th set process in Figure 4A. The lower part of Figure 4B is a further enlarged view of the upper part of Figure 4B. As described above, the curvature update unit 144C of the calculation unit 11 updates the section from point s0 to point e0 (shown by the dashed line) of the virtual curvature 100 estimated in the (1) set processing to virtual curvature 100a (shown by the solid line). Figure 4B illustrates the case in which the curvature update unit 144C updates the curvature of the section from point s0 to point e0 in the negative direction so that it curves to the right (to the right in the direction of travel) more than the virtual curvature 100.

[0075] <Example of calculating update amount> Figure 4C is a schematic diagram illustrating the relationship between virtual curvature, radius of curvature, and position before and after the update. It is assumed that point si was already at the center of the road in the section from point si to point ei, which is the section where the virtual curvature is to be updated. In this case, the virtual curvature of that section before the update is κ i Assume that at this point, the vehicle collides with an impassable area at position l in the direction of travel (in other words, it collides with the left edge of the drivable area). In this case, the updated virtual curvature κ i+1 We need to determine how to do this. Assuming that the traversable region always continues at a distance of the above detection line interval d from the virtual future trajectory after updating the virtual curvature, the virtual curvature κ before the update i And, radius of curvature R i The relationship between the points and position l is as shown in Figure 4C. From the relationship in Figure 4C, the following system of equations can be obtained.

number

[0076] <Calculation of virtual curvature> The virtual curvature (estimated value) calculated by the curvature estimation unit 143 will be explained with reference to Figure 4D. As described above, based on the steering angle information detected by the steering angle sensor 2a and the driving state of the host vehicle (for example, the current vehicle speed), the curvature estimation unit 143 determines the virtual curvature κ sj to calculate. Since the first calculated value is the initial search value when obtaining the virtual curvature, it may be called the initial value of the virtual curvature. Also, the curvature estimation unit 143 uses the information from the previous plan update unit 145 to obtain the above virtual curvature κ sj and the virtual curvature κ pi obtained by the previous curvature search, and the current vehicle speed v detected by the vehicle speed sensor 2d, and inputs them to obtain the virtual curvature κ j by the following equation (5). κ j =W sj ×κ sj +W pj ×κ´ pj …………(5) However, the weight W sj is a weight value that changes according to the distance s from the host vehicle with respect to the estimated curvature obtained from the steering angle. It approaches 1 as it gets closer to the host vehicle and approaches 0 as it gets farther from the host vehicle. Also, the weight W pj is a weight value that changes according to the distance s from the host vehicle with respect to the estimated curvature obtained from the steering angle, similar to the weight W sj . It approaches 0 as it gets closer to the host vehicle and approaches 1 as it gets farther from the host vehicle. The sum of the weights W sj and the weight W pj is always 1 regardless of the distance s. Furthermore, the virtual curvature κ´ pj is the virtual curvature obtained by shifting the virtual curvature κ pj obtained in the previous search by the distance Δs traveled by the host vehicle during the processing cycle ΔT from the previous search to the current search (in the embodiment, corresponding to the frame interval at which the camera 1a acquires the camera image). According to the above equation (5), since the current search is performed using the virtual curvature κ pj obtained in the previous search, the previous virtual curvature κ pjCompared to not using this method, the number of iterations (in other words, the number of repetitions of the set process) can be reduced, and the computational load for the search can be reduced. The above distance Δs can be calculated using the following equation (6). Δs = v × ΔT ………(6)

[0077] <Explanation of the flowchart> Figure 5A is a flowchart showing an example of a calculation process executed by the calculation unit 11 of the controller 10 in Figure 2 according to a predetermined program. The process shown in this flowchart is repeatedly executed, for example, when the vehicle is driving in automatic driving mode. Alternatively, it may be executed when the vehicle is driving in manual driving mode and, for example, when the lane keeping function, which is one of the driver assistance functions, is enabled, i.e., when the vehicle is driving in lane keeping mode.

[0078] In step S10, the calculation unit 11 acquires camera images from camera 1a on a frame-by-frame basis and proceeds to step S20. In step S20, the calculation unit 11 uses the classification unit 142 to classify the area of ​​the camera image into a drivable area 107 and a non-drivable area 108, and then proceeds to step S30.

[0079] In step S30, the calculation unit 11 calculates a virtual curvature as an estimated value of the curvature of the road using the curvature estimation unit 143, and proceeds to step S40. In step S40, the calculation unit 11 uses the virtual trajectory calculation unit 146 to calculate a virtual future trajectory as an estimated value of the future trajectory that the vehicle will travel in the future, and proceeds to step S50. The processing in step S40 corresponds to the processing of the second phase (b) in the set processing described above.

[0080] In step S50, the calculation unit 11 performs pre-determination processing and proceeds to step S60. Details of the pre-determination processing will be described later with reference to the flowchart shown in Figure 5B. The processing in step S50 corresponds to the (b) second phase processing and the (c) first part of the third phase processing in the set processing described above.

[0081] In step S60, the calculation unit 11 uses the collision determination unit 144B to determine whether the virtual future trajectory will collide with the impassable area 108. For example, if the calculation unit 11 determines that at least one of the first detection line 105 and the second detection line 106 will collide with the impassable area 108, it affirms step S60 and proceeds to step S70. If it determines that neither the first detection line 105 nor the second detection line 106 will collide with the impassable area 108, it denies step S60 and proceeds to step S90. The processing in step S60 corresponds to the latter part of the third phase processing (c) in the set processing described above.

[0082] In step S70, the calculation unit 11 determines whether the number of repetitions of the set process is less than the limit. If the number of repetitions is less than the limit, the calculation unit 11 affirms step S70 and proceeds to step S80. If the number of repetitions is not less than the limit (in other words, the upper limit has been reached), it negates step S70 and proceeds to step S120.

[0083] In step S80, the calculation unit 11 updates the virtual curvature using the curvature update unit 144C and returns to step S40. The reason for returning to step S40 is to repeat the setting process. The processing in step S80 corresponds to the processing of the first phase (a) in the set processing described above for the second and subsequent times.

[0084] In step S90, which proceeds after determining that step S60 was negative, the calculation unit 11 sets the virtual future trajectory as the vehicle's future trajectory using the future trajectory setting unit 151 and proceeds to step S100.

[0085] In step S100, the calculation unit 11 sends a command to the driving control unit 16 to use the planned speed and planned steering angle based on the vehicle's future trajectory for driving control, and proceeds to step S110. In step S110, the calculation unit 11 determines whether or not to terminate the process. If, for example, the automatic driving mode is deactivated, the calculation unit 11 determines step S110 to be positive and terminates the process shown in Figure 5A. If, for example, the automatic driving mode is to be continued, the calculation unit 11 determines step S110 to be negative and returns to step S10, and repeats the process described above.

[0086] In step S120, which proceeds after determining that step S70 was negative, the calculation unit 11 uses the future trajectory setting unit 151 to set the virtual future trajectory at that point in time as the vehicle's future trajectory and proceeds to step S130.

[0087] In step S130, the calculation unit 11 performs a predetermined cancellation process to terminate the process shown in Figure 5A. One example of this cancellation process is to stop the curvature search and set the planned speed to 0 relative to the search position. This prevents the vehicle from overshooting areas where it cannot travel.

[0088] <Pre-judgment processing> Details of the pre-determination processing will be explained with reference to the flowchart shown in Figure 5B. Figure 5B is a flowchart showing an example of the pre-determination processing in step S50, which is performed in the calculation unit 11. In step S501, the calculation unit 11 generates a first detection line 103 and a second detection line 104 with the same shape as the virtual future trajectory using the determination unit 144, and proceeds to step S503.

[0089] In step S503, the calculation unit 11 uses the coordinate transformation unit 144A of the determination unit 144 to perform coordinate transformations on the virtual future trajectory, the first detection line, and the second detection line from the bird's-eye view coordinate system to the perspective view coordinate system, thereby generating the virtual future trajectory, the first detection line, and the second detection line in perspective view coordinates, and proceeding to step S505.

[0090] In step S505, the calculation unit 11 superimposes the virtual future trajectory after coordinate transformation, the first detection line, and the second detection line onto the camera image F, then completes the processing shown in Figure 5B and proceeds to step S60 in Figure 5A.

[0091] According to the embodiments described above, the following effects and advantages are achieved. (1) The vehicle control device 200 includes a camera 1a as an external information acquisition means that acquires external information of the surrounding area of ​​the vehicle, including the road, as a camera image F; a classification unit 142 that classifies the area of ​​the camera image F into a drivable area 107 and an undrivable area 108 by predetermined segmentation processing; a curvature estimation unit 143 that calculates a virtual curvature as an estimated value of the curvature of the road based on the driving state including the steering angle of the vehicle; a virtual trajectory calculation unit 146 that calculates a virtual future trajectory 102 as an estimated value of the future trajectory of the vehicle in a bird's-eye view coordinate system based on the virtual curvature; and a virtual trajectory calculation unit 146 that compares the camera image F and the virtual future trajectory 102. The system includes a determination unit 144 that determines whether the future trajectory 102 is included in the drivable area 107, a future trajectory setting unit 151 that sets the virtual future trajectory 102 as the vehicle's future trajectory based on the determination result of the determination unit 144, and a driving control unit 16 that acts as a control means for performing driving control for the vehicle based on the vehicle's future trajectory. The determination unit 144 converts the virtual future trajectory 102 calculated by the virtual trajectory calculation unit 146 from the bird's-eye view coordinate system to the perspective coordinate system of the camera image F, and superimposes the virtual future trajectory 102 onto the camera image F to determine whether the virtual future trajectory 102 is included in the drivable area 107. With this configuration, by superimposing the virtual future trajectory 102, calculated in a bird's-eye view coordinate system, onto the perspective coordinate system of the camera image F captured by camera 1a, after performing a coordinate transformation from the bird's-eye view coordinate system to the perspective coordinate system, it is possible to appropriately determine whether or not the virtual future trajectory 102 is included in the drivable area 107. In particular, it becomes possible to make a more accurate determination for roads with non-constant curvature or for roads far away from the vehicle. Furthermore, by performing driving control based on the virtual future trajectory 102 calculated with such high accuracy, it becomes possible to appropriately control the vehicle's driving even when, for example, it is driving on a road that is not stored as high-precision map information in the memory unit 12 for the first time.

[0092] (2) In the vehicle control device 200 described in (1) above, if the determination unit 144 determines that a part of the virtual future trajectory 102 is not included in the drivable area 107, it updates the virtual curvature based on the position where the virtual future trajectory 102 deviates from the drivable area 107 to the non-drivable area 108, and then determines again whether the virtual future trajectory 102a, which has been recalculated by the virtual trajectory calculation unit 146 based on the updated virtual curvature, is included in the drivable area 107. With this configuration, it becomes possible to reliably generate a virtual future trajectory 102a on which the vehicle can travel.

[0093] (3) In the vehicle control device 200 described in (2) above, the determination unit 144 generates a first detection line 103 with the same shape as the virtual future trajectory 102 to the left in the direction of travel of the vehicle with respect to the virtual future trajectory 102 at a predetermined detection line interval d, and generates a second detection line 104 with the same shape as the virtual future trajectory 102 to the right in the direction of travel of the virtual future trajectory 102 at a predetermined detection line interval d, and performs collision determination with the impassable area 108 from a position corresponding to a predetermined reference point p0 in the direction of travel for the first detection line 103 (105) and the second detection line 104 (106), respectively, and updates the virtual curvature to increase to the right if the first detection line 103 (105) collides with the impassable area 108, and updates the virtual curvature to increase to the left if the second detection line 104 (106) collides with the impassable area 108. With this configuration, when a collision with the impassable area 108 is detected, it becomes easy to determine the direction in which to update the virtual future trajectory 102 (whether to increase the virtual curvature to the right or to the left).

[0094] (4) In the vehicle control device 200 described in (3) above, if either the first detection line 103(105) or the second detection line 104(106) collides with the impassable area 108, the determination unit 144 regenerates the first detection line 103a and the second detection line 104a on the left and right sides of the virtual future trajectory 102a based on the updated virtual curvature, and moves the position corresponding to the respective reference point p0 in the direction of travel. For the regenerated first detection line 103a(105a) and the second detection line 104a(106a), the determination unit 144 performs collision determination with the impassable area 108 from the position corresponding to the moved reference point p1 in the direction of travel. With this configuration, by gradually offsetting the position where collision detection begins while searching for curvature, it becomes possible to suppress excessive changes in curvature for the next updated virtual future trajectory 102a.

[0095] (5) In the vehicle control device 200 described in (4) above, when the number of updates to the virtual curvature reaches a predetermined upper limit, the future track setting unit 151 sets the virtual future track 102 etc. as the vehicle's future track, and the driving control unit 16 recognizes the road as a dead end and performs predetermined driving control. With this configuration, in situations where a virtual future trajectory cannot be generated no matter how many times the virtual curvature is updated, such as a dead end, it becomes possible to terminate the curvature search.

[0096] (6) In the vehicle control device 200 described in (5) above, the determination unit 144 returns the reference points to the position corresponding to point p0 as the initial position when the reference points such as the first detection line 105 and the second detection line 106 reach the depth distance (corresponding to the vanishing point) corresponding to the end of the virtual future track 102 and the number of updates is less than the upper limit, and repeats the determination of whether the virtual future track 102 based on the updated virtual curvature is included in the drivable area 107. With this configuration, for example, by changing the calculation conditions and redoing the curvature search, it becomes possible to calculate a more appropriate hypothetical future trajectory.

[0097] (7) In the vehicle control device 200 described in (1) above, the virtual trajectory calculation unit 146 calculates the vehicle's future trajectory at predetermined time intervals, and calculates the virtual future trajectory based on the virtual curvature and the vehicle's future trajectory calculated in the past. With this configuration, by incorporating the previously calculated information on the virtual future trajectory, it becomes possible to reduce the number of times the above set process—which involves updating the virtual curvature, calculating the virtual future trajectory based on the updated virtual curvature, and performing collision detection for areas where travel is impossible—is repeated.

[0098] (8) The vehicle control device 200 described in (1) above further includes a speed planning unit 152 as a speed planning means that sets an upper limit of the vehicle's speed based on the vehicle's future trajectory and the vehicle's allowable acceleration, and generates a speed plan for the vehicle based on the upper limit of the vehicle's speed, and the driving control unit 16 performs driving control based on the speed plan. With this configuration, it becomes possible to correctly recognize the curvature of curves included in the vehicle's future trajectory and to plan an appropriate speed so as not to cause uncomfortable acceleration for the occupants.

[0099] The above embodiment can be modified into various forms. Modifications will be described below. (Variation 1) In the above embodiment, an example was described in which the detection line spacing d between the virtual future trajectory 102 and the first detection line 103, and the detection line spacing d between the virtual future trajectory 102 and the second detection line 104, are set to the width of the vehicle body. Alternatively, they may be set to a value different from the vehicle body width. For example, in the vehicle control device 200 described above, the determination unit 144 may set the detection line interval d based on at least one of the following: the motion characteristics of the vehicle, the width of the drivable area 107, first information regarding driving characteristics learned by the vehicle, and second information regarding driving characteristics set by the occupants of the vehicle. By configuring it in this way, the spacing between the first detection line 103 and the second detection line 104 can be changed according to the situation to update the virtual curvature to one that is more suitable for the crew, and a virtual future trajectory 102a can be generated based on the updated virtual curvature.

[0100] (Modification 2) Instead of determining a collision between the first detection line 103 and the second detection line 104 and the impassable area 108, the collision with the impassable area 108 may be determined as follows. For example, in the vehicle control device 200 described above, the determination unit 144 may set a detection area that includes a virtual future trajectory 102 and has a predetermined width in the left-right direction of the vehicle in the direction of travel of the vehicle, and perform a collision determination with the impassable area 108 from a predetermined reference point in the direction of travel of this detection area, and update the virtual curvature to increase to the right if the left end of the left-right edge of the detection area collides with the impassable area 108, and update the virtual curvature to increase to the left if the right end of the left-right edge of the detection area collides with the impassable area 108. Even with this configuration, similar to the embodiment described above, it becomes possible to easily determine the direction in which to update the virtual future trajectory 102 when a collision with the impassable area 108 is determined.

[0101] The above description is merely an example, and the present invention is not limited by the embodiments and modifications described above, as long as they do not impair the features of the present invention. [Explanation of symbols]

[0102] 1 External sensor group, 1a Camera, 2 Internal sensor group, 2d Vehicle speed sensor, 2e Acceleration sensor, 6 Navigation device, 10 Controller, 11 Calculation unit, 12 Memory unit, 13 Vehicle position recognition unit, 14 External environment recognition unit, 15 Action plan generation unit, 16 Driving control unit, 50 Speed ​​control device, 100, 100a, 100b, 100c Virtual curvature, 102, 102a, 102c Virtual future trajectory, 103, 103a, 103c, 105, 105a, 105b, 105c, 105x First detection line, 104, 104a, 104c, 106, 106a, 106b, 106c, 106x Second detection line, 107 Driving area, 108 Driving area, 121 Previous curvature holding unit, 131 Odometry calculation unit, 141 Target calculation unit, 142 Classification unit, 143 Curvature estimation unit, 144 Judgment unit, 144A Coordinate transformation unit, 144B Collision judgment unit, 144C Curvature update unit, 145 Previous plan update unit, 146 Virtual trajectory calculation unit, 151 Future trajectory setting unit, 152 Speed ​​planning unit, 200 Vehicle control device, AC Actuator, F Camera image

Claims

1. External information acquisition means that acquires external information about the surroundings of the vehicle, including the road, as an image, A classification means that classifies the region of the image into a drivable region and a non-drivable region by a predetermined segmentation process, A curvature estimation means calculates a virtual curvature as an estimated value of the curvature of the road based on the driving conditions including the steering angle of the vehicle, A virtual trajectory calculation means that calculates a virtual future trajectory as an estimated value of the future trajectory of the vehicle based on the virtual curvature in a bird's-eye view coordinate system, A determination means for comparing the external information with the virtual future trajectory and determining whether or not the virtual future trajectory is included in the drivable area, A future trajectory setting means sets the virtual future trajectory as the vehicle's future trajectory based on the determination result of the determination means, The system includes control means for performing driving control on the vehicle based on the vehicle's future trajectory, The determination means converts the virtual future trajectory calculated by the virtual trajectory calculation means from the bird's-eye view coordinate system to the perspective coordinate system of the external information, and superimposes the virtual future trajectory onto the external information to determine whether or not the virtual future trajectory is included in the drivable area. A vehicle control device characterized by the following features.

2. In the vehicle control device according to claim 1, If the determination means determines that a portion of the virtual future trajectory is not included in the drivable area, it updates the virtual curvature based on the position where the virtual future trajectory deviates from the drivable area into the non-drivable area, and then determines again whether the virtual future trajectory, recalculated by the virtual trajectory calculation means based on the updated virtual curvature, is included in the drivable area. A vehicle control device characterized by the following features.

3. In the vehicle control device according to claim 2, The determination means is, A first detection line having the same shape as the virtual future trajectory is generated on the left side of the direction of travel of the vehicle with respect to the virtual future trajectory at a predetermined detection line interval, and a second detection line having the same shape as the virtual future trajectory is generated on the right side of the direction of travel with respect to the virtual future trajectory at a predetermined detection line interval. For the first detection line and the second detection line, a collision determination with the non-travelable area is performed from a predetermined reference point in the direction of travel. If the first detection line collides with the impassable area, the virtual curvature is updated to increase to the right, and if the second detection line collides with the impassable area, the virtual curvature is updated to increase to the left. A vehicle control device characterized by the following features.

4. In the vehicle control device according to claim 3, The determination means is, If either the first detection line or the second detection line collides with the area where travel is impossible, the first detection line and the second detection line are regenerated on the left and right sides of the virtual future trajectory based on the updated virtual curvature, respectively, and the respective reference points are moved in the direction of travel. For the first and second detection lines that have been regenerated, a collision determination is performed with the non-travelable area in the direction of travel from the reference point after the movement. A vehicle control device characterized by the following features.

5. In the vehicle control device according to claim 4, The future trajectory setting means sets the virtual future trajectory as the vehicle's future trajectory when the number of updates to the virtual curvature reaches a predetermined upper limit. The control means recognizes the travel path as a dead end and performs predetermined travel control. A vehicle control device characterized by the following features.

6. In the vehicle control device according to claim 5, The determination means is, When the reference points of the first detection line and the second detection line each reach a depth distance corresponding to the end of the virtual future trajectory, and the number of updates is less than the upper limit, the reference points are returned to their initial positions, and the determination of whether the virtual future trajectory based on the virtual curvature after the update is included in the drivable region is repeated. A vehicle control device characterized by the following features.

7. In the vehicle control device according to claim 3, The determination means is, The predetermined detection line interval is set based on at least one of the following: the motion characteristics of the vehicle, the width of the drivable area, first information relating to the driving characteristics learned by the vehicle, and second information relating to the driving characteristics set by the occupants of the vehicle. A vehicle control device characterized by the following features.

8. In the vehicle control device according to claim 1, The virtual trajectory calculation means is The system calculates the vehicle's future trajectory at predetermined time intervals, and calculates the virtual future trajectory based on the virtual curvature and the vehicle's future trajectory calculated in the past. A vehicle control device characterized by the following features.

9. In the vehicle control device according to claim 1, The system further includes a speed planning means that sets an upper limit on the vehicle's speed based on the vehicle's future trajectory and the vehicle's allowable acceleration, and generates a speed plan for the vehicle based on the upper limit on the vehicle's speed. The control means performs the driving control based on the speed plan. A vehicle control device characterized by the following features.

10. In the vehicle control device according to claim 2, The determination means is, A detection area is set that includes the aforementioned virtual future trajectory and has a predetermined width in the left-right direction of the vehicle in the direction of travel of the vehicle, and a collision determination is made with respect to the detection area from a predetermined reference point in the direction of travel with respect to the area where travel is impossible. If the left end of the detection area in the left-right direction collides with the area where travel is impossible, the virtual curvature is updated to increase to the right. If the right-hand end of the detection region in the left-right direction collides with the non-travelable region, the virtual curvature is updated to increase on the left side. A vehicle control device characterized by the following features.

Images