Vehicle control device

Abstract

A vehicle control device is provided with: an external information acquisition unit (1a) that acquires, as a camera image, external information including a travel road; a classification unit (142) that classifies a region of the image into a travelable region and a non-travelable region; a curvature estimation unit (143) that calculates a virtual curvature of the travel road on the basis of the travel state of the host vehicle; a virtual trajectory calculation unit (146) that calculates a virtual future trajectory of the host vehicle in the bird's-eye view coordinate system on the basis of the virtual curvature; a determination unit (144) that determines whether or not the virtual future trajectory is included in the travelable region; a future trajectory setting unit (151) that sets the virtual future trajectory as a future travel trajectory on the basis of the determination result of the determination unit (144); and a travel control unit (16) that performs travel control of the host vehicle on the basis of the set future travel trajectory, and the determination unit (144) converts the virtual future trajectory calculated by the virtual trajectory calculation unit (146) from the bird's-eye view coordinate system to a perspective coordinate system of external information, and performs travel control of the host vehicle by superimposing the virtual future trajectory on the external information. Whether a virtual future trajectory is included in the drivable area is determined.

Description

Technical Field

[0001] This invention relates to a vehicle control device that controls the vehicle based on the estimated shape of the driving road. Background Technology

[0002] As such a technology, there is a known technique for estimating the road shape within a first distance as a curve with a constant rate of curvature change, and for estimating the road shape beyond the first distance as a curve with a constant curvature (see Patent Document 1).

[0003] In the existing technology, the road shape at a distance is estimated based on the premise that the curvature is constant beyond the first distance, which makes it difficult to apply to complex road shapes with non-constant curvature, such as ordinary roads.

[0004] Since the curvature of a road is necessary for generating the trajectory of a moving vehicle, appropriately estimating the curvature helps in making appropriate vehicle controls, such as regarding speed and steering angle. That is, it can improve traffic safety while suppressing a decrease in traffic flow.

[0005] Existing technical documents Patent documents Patent Document 1: Japanese Patent No. 6285321 (JP6285321B2). Summary of the Invention

[0006] A vehicle control device according to one technical solution of the present invention comprises: an external information acquisition unit that acquires external information including the vehicle's surroundings and the driving road as an image; a classification unit that classifies the image area into a drivable area and a non-drivable area through a predetermined segmentation process; a curvature estimation unit that calculates a virtual curvature based on the driving state including the vehicle's steering angle as an estimated value of the curvature of the driving road; a virtual trajectory calculation unit that calculates a virtual future trajectory in a bird's-eye view coordinate system based on the virtual curvature as an estimated value of the vehicle's future driving trajectory; a determination unit that compares the external information and the virtual future trajectory to determine whether the virtual future trajectory is included in the drivable area; a future trajectory setting unit that sets the virtual future trajectory as the future driving trajectory based on the determination result of the determination unit; and a control unit that performs driving control on the vehicle based on the future driving trajectory set by the future trajectory setting unit. The determination unit transforms the virtual future trajectory calculated by the virtual trajectory calculation unit from the bird's-eye view coordinate system to the perspective coordinate system of the external information, and determines whether the virtual future trajectory is included in the drivable area by superimposing the virtual future trajectory on the external information. Attached Figure Description

[0007] The objectives, features, and advantages of the present invention are further illustrated by the following description of embodiments in conjunction with the accompanying drawings; Figure 1 This is a diagram illustrating the structure of a vehicle control device according to an embodiment of the present invention; Figure 2 This is a block diagram illustrating the structure of a speed control device; Figure 3 This is a block diagram illustrating the main parts of the speed control device; Figure 4A This is a schematic diagram illustrating the process of searching for the curvature of the driving road; Figure 4B This is a schematic diagram illustrating curvature updates; Figure 4C This is a schematic diagram illustrating the relationship between the virtual curvature, radius of curvature, and position before and after the update; Figure 4D This is a schematic diagram illustrating virtual curvature; Figure 5A This is a flowchart illustrating an example of the computational processing performed by the arithmetic unit according to the program; Figure 5B This is a flowchart detailing the pre-processing steps for the determination. Detailed Implementation

[0008] The embodiments of the invention will now be described with reference to the accompanying drawings.

[0009] As an example of a vehicle control device according to one embodiment of the present invention, a speed control device controls the vehicle's speed while the vehicle is traveling along a target path (also referred to as a target trajectory) on a road, so as to prevent acceleration in the longitudinal and lateral directions exceeding a predetermined value. Alternatively, the steering angle of a steering device (e.g., a power steering system) can be controlled such that the vehicle's position follows the target trajectory. The speed control device can be applied, for example, to vehicles with autonomous driving capabilities, i.e., autonomous vehicles.

[0010] It should be noted that the speed control device of the implementation method can also be applied to either manually driven vehicles or autonomous vehicles with driving assistance functions, but for the sake of explanation, the following example is taken as applied to autonomous vehicles.

[0011] Furthermore, in the implementation, the vehicle equipped with the speed control device is sometimes referred to as "this vehicle" to distinguish it from other vehicles. This vehicle can be any of the following: an engine vehicle with an internal combustion engine as the driving source, an electric vehicle with a drive motor as the driving source, or a hybrid vehicle with both an engine and a drive motor as driving sources. This vehicle can operate not only in an automated driving mode that requires no driver intervention, but also in a manual driving mode based on driver input.

[0012] <Vehicle Structure> First, a general description of the vehicle's structure related to autonomous driving will be given. Figure 1 This is a block diagram illustrating the structure of a vehicle control device 200 of this vehicle having an embodiment of a speed control device. For example... Figure 1 As shown, the vehicle control device 200 mainly includes a controller 10, an external sensor group 1 and an internal sensor group 2 that are communicatively connected to the controller 10, an input / output device 3, a positioning unit 4, a map database 5, a navigation device 6, a communication unit 7, and a driving actuator AC.

[0013] External sensor group 1 is a collective term for multiple sensors (external sensors) that detect external conditions as information about the vehicle's surroundings. External sensor group 1 may include, for example, a lidar that measures the distance from the vehicle to surrounding obstacles by measuring the scattered light from the vehicle's omnidirectional illumination; a radar that detects other vehicles or obstacles around the vehicle by illuminating electromagnetic waves and detecting the reflected waves; and a camera mounted on the vehicle and equipped with imaging elements (image sensors) such as CCD (charge-coupled device) or CMOS (complementary metal-oxide-semiconductor) sensors to capture images of the vehicle's surroundings (front, rear, and sides).

[0014] Internal sensor group 2 is a collective term for multiple sensors (internal sensors) that detect the driving status of the vehicle. Internal sensor group 2 includes, for example, a vehicle speed sensor to detect the vehicle's speed, an acceleration sensor to detect the vehicle's acceleration in the longitudinal (travel direction) and lateral (lane width) directions, a speed sensor to detect the rotational speed of the driving source, and a yaw rate sensor to detect the angular velocity of the vehicle's center of gravity about its vertical axis. Sensors that detect driver operations in manual driving mode, such as operations on the accelerator pedal, brake pedal, and steering wheel, are also included in internal sensor group 2.

[0015] Input / output devices 3 are a general term for devices that input commands to the driver or output information to the driver. Input / output devices 3 include, for example, various switches that allow the driver to input various commands by operating the control components, microphones that allow the driver to input commands by voice, displays that provide information to the driver via images, and speakers that provide information to the driver by sound.

[0016] The positioning unit (GNSS unit) 4 has a positioning sensor that receives positioning signals transmitted from positioning satellites. Positioning satellites are artificial satellites such as GPS satellites and quasi-zenith satellites. The positioning unit 4 uses the positioning information received by the positioning sensor to determine the vehicle's current position (latitude, longitude, and altitude).

[0017] Map database 5 is a device that stores general map information used in navigation device 6, and may be composed of, for example, a disk or semiconductor element. The map information may also include road location information, road shape (curvature, etc.) information, and the location information of intersections and forks in the road. It should be noted that the map information stored in map database 5 is different from the high-precision map information stored in storage unit 12 of controller 10.

[0018] The navigation device 6 is, for example, a device that searches for a route along a road leading to a destination input by the driver and provides driving guidance along the searched route. The input of the destination and the driving guidance along the searched route are performed via the input / output device 3. The route search is based on the vehicle's current position determined by the positioning unit 4, the location of the input destination, and map information stored in the map database 5. Alternatively, the vehicle's current position can be determined using the detection values ​​from the external sensor group 1, and the route can be searched based on this current position and high-precision map information stored in the storage unit 12.

[0019] Communication unit 7 communicates with various servers (not shown) via a network including wireless communication networks such as the Internet and mobile phone networks, and periodically or at arbitrary intervals retrieves map information, driving history information, and traffic information from the servers. In addition to retrieving driving history information, communication unit 7 can also send the vehicle's driving history information to the servers. The network includes not only public wireless communication networks but also closed communication networks set up for each designated management area, such as wireless LAN, Wi-Fi (registered trademark), and Bluetooth (registered trademark). The retrieved map information is output to map database 5 and storage unit 12 to update the map information.

[0020] An actuator (AC) is a driving actuator used to control the movement of the vehicle. When the driving source is an engine, the actuator AC includes a throttle actuator for adjusting the opening of the engine's throttle valve (throttle opening). When the driving source is a drive motor, the drive motor is included in the actuator AC. Braking actuators that operate the vehicle's braking system and actuators that drive the steering mechanism are also included in the actuator AC.

[0021] The controller 10 is composed of an electronic control unit (ECU). More specifically, the controller 10 is composed of a computer having an arithmetic unit 11 such as a CPU (microprocessor), a storage unit 12 such as a ROM (read-only memory) and RAM (random access memory), and other peripheral circuits not shown, such as I / O interfaces.

[0022] It should be noted that although multiple ECUs with different functions, such as an engine control ECU, a drive motor control ECU, and a braking system ECU, can be set up separately, in... Figure 1 For convenience, controller 10 is shown as a collection of these ECUs.

[0023] Storage unit 12 stores high-precision, detailed map information for autonomous driving. This high-precision map information may include road location information, road shape (radius of curvature, etc.), road slope information, location information of intersections or forks in the road, types of road markings such as white lines and their location information, information on the number of lanes (driving lanes), lane width and location information of each lane (center position of the lane, information on the lane's boundary lines), location information of landmarks (traffic lights, signs, buildings, etc.) used as markers on the map, and information on road surface contours such as road surface undulations. The high-precision map information stored in storage unit 12 may include high-precision map information obtained from outside the vehicle via communication unit 7, or high-precision map information generated by the vehicle itself using detection values ​​from external sensor group 1 or detection values ​​from external sensor group 1 and internal sensor group 2. Storage unit 12 may also store various control programs, thresholds used in the programs, and other information.

[0024] The computing unit 11 includes a vehicle position recognition unit 13, an external recognition unit 14, an action plan generation unit 15, and a driving control unit 16 as its functional structure.

[0025] The vehicle position recognition unit 13 identifies the vehicle's position on the map (vehicle position) based on the vehicle's position information obtained from the positioning unit 4 and the map information from the map database 5. Alternatively, it can use high-precision map information stored in the storage unit 12 and surrounding information detected by the external sensor group 1 to identify the vehicle's position, thereby enabling high-precision identification. Furthermore, it can calculate the vehicle's movement information (movement direction, movement distance) based on the detection values ​​from the internal sensor group 2, thereby identifying the vehicle's position.

[0026] It should be noted that when the vehicle's position can be determined using external sensors installed on or beside the road, the vehicle's position can also be identified by communicating with the sensor via the communication unit 7.

[0027] The external environment identification unit 14 identifies the external conditions around the vehicle based on signals from the external sensor group 1, such as a camera, lidar, or radar. For example, it identifies the position, speed, and acceleration of surrounding vehicles (vehicles in front and behind) traveling around the vehicle, the position of surrounding vehicles parked or stationary around the vehicle, and the position and state of other objects, thus creating target information. It should be noted that, in this embodiment, it is sufficient to identify the external conditions around the vehicle based at least on signals from the camera.

[0028] Other objects include signs, traffic lights, roads, buildings, guardrails, utility poles, billboards, pedestrians, bicycles, etc. Road markings (white lines, etc.) and stop lines on the road surface are also included in other objects (roads). The state of other objects includes the color of traffic lights (red, green, yellow), the speed or direction of movement of pedestrians or bicycles, etc. A portion of stationary objects among other objects constitutes a landmark that becomes a marker of location on the map, and the external identification unit 14 also identifies the location and category of the landmark.

[0029] The action plan generation unit 15 generates, for example, the vehicle's driving trajectory (target trajectory) from the current moment to a predetermined time elapsed on the path searched by the navigation device 6, the high-precision map information stored in the storage unit 12, the vehicle's position identified by the vehicle position recognition unit 13, and the external conditions identified by the external environment recognition unit 14. When multiple trajectories may exist as candidate trajectories on the path searched by the navigation device 6, the action plan generation unit 15 selects the best trajectory from among them, which meets the criteria of complying with laws and regulations and driving efficiently and safely, and generates the selected trajectory as the target path.

[0030] Then, the action plan generation unit 15 generates an action plan corresponding to the generated target path. The action plan generation unit 15 generates various action plans corresponding to driving modes such as overtaking (passing ahead of a vehicle), lane changing (changing lanes), following (following ahead of a vehicle), lane keeping (maintaining lane position without deviating from the lane), deceleration, and acceleration. When generating the target path, the action plan generation unit 15 first determines the driving mode and generates the target path (also called the trajectory plan) based on the driving mode. Furthermore, it determines the steering angle (also called the steering angle plan) and the driving speed (also called the speed plan) in a manner that ensures lateral acceleration along the target path does not exceed a predetermined value.

[0031] In autonomous driving mode, the driving control unit 16 controls each actuator AC to make the vehicle travel along the target path generated by the action plan generation unit 15. For example, the driving control unit 16 considers the driving resistance determined by road gradient and other factors in autonomous driving mode, and calculates the required driving force to achieve the speed plan (e.g., target acceleration per unit time) calculated by the action plan generation unit 15. Furthermore, it performs feedback control on the actuator ACs, for example, using the actual acceleration detected by the internal sensor group 2 as the target acceleration. That is, it controls the drive actuator ACs to make the vehicle travel at the target speed and target acceleration.

[0032] In addition, the driving control unit 16 outputs a rudder angle indication signal to control the rudder actuator AC based on vehicle state quantities observed by the internal sensor group 2 and the like in the automatic driving mode, in order to realize the rudder angle plan (the optimal steering angle for the vehicle to follow the target path) calculated by the action plan generation unit 15.

[0033] It should be noted that in manual driving mode, the driving control unit 16 controls each actuator AC according to the driving commands (steering operations, etc.) from the driver obtained by the internal sensor group 2.

[0034] <Estimation of the radius of curvature> However, for example, when the vehicle is driving on a road that is not stored as high-precision map information in the storage unit 12 for the first time, or when the driving lane position is temporarily changed due to construction or other reasons, and the shape of the road that the vehicle is driving on is different from the high-precision map information, the vehicle cannot refer to the existing high-precision map information.

[0035] In this implementation, when driving on such a road, the actual shape of the road is estimated based on camera images acquired by the camera, which is the external sensor group 1. Specifically, the radius of curvature of the road is estimated. By appropriately estimating the radius of curvature of the road, it is possible to drive along a target path, perform lane keeping (lane maintenance), or drive within a predetermined lateral acceleration, even on roads not included in the high-precision map information.

[0036] It should be noted that the reciprocal of the radius of curvature is the curvature. Therefore, the curvature of a road can be inferred as the actual shape of the road. In the following explanation, the case of inferred road curvature as the actual shape of the road will be used as an example. Furthermore, in the following explanation, the road on which this vehicle travels will sometimes be referred to as the driving road.

[0037] As is typical of camera images (perspective views) acquired by a camera, distant subjects (in this embodiment, the road surface) appear smaller in the image (in other words, the further away from the vehicle, the fewer pixels constitute the road surface in the camera image), resulting in lower resolution for distant road surfaces. Consequently, it is difficult to accurately estimate the curvature of distant road surfaces from the camera image. This estimation is particularly challenging for roads with non-constant curvature.

[0038] Therefore, in the implementation, as a bird's-eye view of the driving road from above, the shape of the road surface is shown, which corresponds to the curvature of the driving road estimated in the bird's-eye view. The driving trajectory of the vehicle (called the virtual future trajectory) is calculated in the bird's-eye view. After the virtual future trajectory is transformed from the bird's-eye view coordinate system to the perspective view coordinate system, the virtual future trajectory after the coordinate transformation is superimposed on the camera image.

[0039] Specifically, until the virtual future trajectory after the coordinate transformation converges within the area of ​​the road surface in the camera image (referred to as the drivable area), the process of estimating (updating) the curvature of the driving road, calculating the virtual future trajectory of the vehicle corresponding to the updated curvature in the bird's-eye view coordinates, and performing coordinate transformation on the virtual future trajectory calculated in the bird's-eye view coordinates and overlaying it onto the camera image represented in perspective coordinates is repeated.

[0040] In this embodiment, the process of repeatedly performing the above-mentioned curvature estimation (update), the calculation of the virtual future trajectory, and the overlay process on the camera image is called a search. It should be noted that, to distinguish it from the path-finding search of the navigation device 6, it can also be called a curvature search. The speed control device of this embodiment, by performing a curvature search, envisions the actual driving trajectory while seeking consistency between the driving trajectory in the camera image and the drivable area, thereby appropriately estimating the curvature required for the speed plan. By appropriately estimating the curvature even for driving roads with non-constant curvature, appropriate driving control of the vehicle based on the estimated curvature can be performed. The structure of such a speed control device will be described in more detail.

[0041] <Speed ​​Control Device> Figure 2 This is a block diagram illustrating the structure of the speed control device 50 according to an embodiment. Figure 3 This is a block diagram illustrating the main components of the speed control device. As an example, the speed control device 50 is configured as follows: Figure 1 The controller 10 is part of the function of the vehicle control device 200. The controller 10 is connected to a camera 1a, a steering angle sensor 2a, a steering angular velocity sensor 2b, a steering torque sensor 2c, a vehicle speed sensor 2d, an acceleration sensor 2e, a navigation device 6, and an actuator AC.

[0042] Camera 1a is a single-lens reflex camera with an image sensor, consisting of... Figure 1 It is part of the external sensor group 1. Camera 1a is mounted, for example, at a predetermined position at the front of the vehicle, and continuously captures images of the space in front of the vehicle at a predetermined frame rate (e.g., 10 frames / second) to obtain images of objects (camera images). Objects include road markings on lanes on a designated road. It should be noted that objects can also be detected together with camera 1a using radar or lidar, etc.

[0043] The steering angle sensor 2a detects, for example, the rotation angle (steering angle) of the steering shaft connected to a steering wheel (not shown). The steering angular velocity sensor 2b detects the rotational angular velocity (steering angular velocity) of the steering shaft. The steering angular velocity can also be simply referred to as the steering angle velocity. The steering torque sensor 2c detects the driver's steering input, and more specifically, detects the steering torque acting on the steering wheel. For example, the steering angle detected by the steering angle sensor 2a when the steering wheel is rotated to the left (counterclockwise) from the neutral position is set to a positive value, and the steering angle detected by the steering angle sensor 2a when the steering wheel is rotated to the right (clockwise) from the neutral position is set to a negative value.

[0044] The aforementioned steering angle sensor 2a, steering angular velocity sensor 2b, and steering torque sensor 2c constitute... Figure 1 It is part of the internal sensor group 2.

[0045] In addition, vehicle speed sensor 2d detects the vehicle's speed. Accelerometer sensor 2e detects the vehicle's longitudinal and lateral accelerations, respectively. The vehicle speed sensor 2d and acceleration sensor 2e together constitute... Figure 1 It is part of the internal sensor group 2.

[0046] It should be noted that, as one of the internal sensor groups 2, it can also be equipped with an IMU (Inertial Measurement Unit) to detect translational and rotational motion in the three axial directions of the vehicle.

[0047] Controller 10 serves as the arithmetic unit 11 ( Figure 1 The functional structure it undertakes includes: a range calculation unit 131, a target calculation unit 141, a classification unit 142, a curvature estimation unit 143, a judgment unit 144, a previous plan update unit 145, a virtual trajectory calculation unit 146, a future trajectory setting unit 151, and a speed planning unit 152. In addition, as described above, the controller 10 has a storage unit 12.

[0048] It should be noted that the range calculation unit 131 can also be part of the vehicle position recognition unit 13. The target calculation unit 141, classification unit 142, curvature estimation unit 143, judgment unit 144, previous plan update unit 145, and virtual trajectory calculation unit 146 can also be part of the external recognition unit 14. The future trajectory setting unit 151 and speed planning unit 152 can also be part of the action plan generation unit 15. The previous curvature retention unit 121 can also be part of the storage unit 12.

[0049] <Movement Quantity> The range calculation unit 131 calculates the movement of the vehicle based on the vehicle speed information detected by the vehicle speed sensor 2d, the amount of wheel rotation, etc.

[0050] <Target> The target object calculation unit 141 calculates information representing target objects existing around the vehicle. Based on signals input from the external sensor group 1 such as camera 1a, lidar, and radar, the target object calculation unit 141 identifies target objects including moving objects such as other vehicles, bicycles, and pedestrians, as well as stationary objects (also referred to as ground features) such as guardrails and signs, and outputs target object information representing the identified target objects.

[0051] <Drivable Area> The classification unit 142 performs prescribed segmentation processing on camera images that serve as information about the surrounding environment of the vehicle, classifying the image regions into drivable and non-drivable areas. The drivable area is the road surface area in the direction of travel of the road (driving road), and the non-drivable area is the area outside the drivable area.

[0052] <Virtual Curvature> The curvature estimation unit 143 calculates a virtual curvature, which is an estimated value of the curvature of the driving road, based on the driving state of the vehicle, including the steering angle information detected by the steering angle sensor 2a and the vehicle speed information detected by the vehicle speed sensor 2d.

[0053] In this embodiment, the virtual curvature calculated by the curvature estimation unit 143 is called the estimated value. Furthermore, during the determination process performed by the determination unit 144 (described later), the virtual curvature where the estimated value has been updated by the curvature update unit 144C is called the updated value.

[0054] <Conflict Determination> The determination unit 144 includes a coordinate transformation unit 144A, a conflict determination unit 144B, and a curvature update unit 144C.

[0055] The coordinate transformation unit 144A performs a coordinate transformation from the bird's-eye view coordinate system to the perspective view coordinate system. Specifically, the bird's-eye view coordinate system of a bird's-eye view of a road or similar object viewed from above is transformed into a perspective view coordinate system corresponding to the camera image.

[0056] In this implementation, a deviation of the virtual future trajectory from a drivable area to a non-drivable area is referred to as a conflict. The conflict determination unit 144B compares the drivable area contained in the camera image (which serves as external information) with the virtual future trajectory calculated by the virtual trajectory calculation unit 146 (described later) to determine whether the virtual future trajectory is contained within the drivable area (in other words, whether the virtual future trajectory conflicts with a non-drivable area). If the virtual future trajectory is contained within the drivable area, it means that the estimated curvature of the driving road is appropriate. Conversely, if the virtual future trajectory conflicts with a non-drivable area, it means that the virtual future trajectory deviates from the drivable area to a non-drivable area; in other words, the estimated curvature of the driving road is inappropriate.

[0057] When the virtual future trajectory deviates from the drivable region to the indrivable region, the curvature update unit 144C updates the estimated value (initial curvature). The curvature update can also be called a correction.

[0058] <Previous Plan Update> In order to efficiently perform curvature search (in other words, to suppress the number of times curvature is updated), the previous planning update unit 145 outputs information about the curvature (estimated value or updated value) used in the previous curvature search to the curvature estimation unit 143 and the curvature update unit 144C. The curvature information used in the previous curvature search is temporarily stored in the previous curvature holding unit 121 in the storage unit 12.

[0059] Therefore, the curvature estimation unit 143 can calculate the virtual curvature based on the steering angle information detected by the steering angle sensor 2a and the curvature used in the previous curvature search. In addition, the curvature updating unit 144C can update the virtual curvature using the curvature used in the previous curvature search.

[0060] <Virtual Track> The virtual trajectory calculation unit 146 calculates a virtual future trajectory as an estimate of the trajectory (referred to as the future trajectory) of the vehicle in the future, based on the most recent virtual curvature. Each time the curvature estimation unit 143 calculates an estimate of the virtual curvature or the curvature update unit 144C updates the virtual curvature, the virtual trajectory calculation unit 146 recalculates the virtual future trajectory based on the estimated or updated virtual curvature value and previously calculated virtual future trajectories.

[0061] <Future Trajectory> The future trajectory setting unit 151 sets the virtual future trajectory as the future trajectory of the vehicle (also referred to as the future trajectory of the vehicle).

[0062] Project Speed The speed planning unit 152 determines the driving speed in parallel with the determination of the rudder angle by the rudder angle planning unit (not shown) in a manner that follows the future trajectory of the vehicle as the target path and does not generate a lateral acceleration exceeding a specified value.

[0063] <Driving Control> The driving control unit 16 outputs instruction information to each actuator AC so that the vehicle travels along its future trajectory at the speed (hereinafter referred to as the planned speed) and rudder angle (hereinafter referred to as the planned rudder angle) determined above.

[0064] <Curvature Search Process> Reference Figure 4A The process of processing the curvature of the search driving road is explained, which makes the virtual future trajectory after coordinate transformation converge within the driving area.

[0065] In this implementation, the curvature search process is divided into three stages. In (a) the first stage, the curvature of the driving road is estimated or updated. Then, in (b) the second stage, a virtual future trajectory of the vehicle corresponding to the curvature is calculated in bird's-eye view coordinates. Then, in (c) the third stage, the virtual future trajectory calculated in bird's-eye view coordinates is transformed to overlap the camera image represented in perspective coordinates.

[0066] Furthermore, in the implementation, the process from the first stage (a) to the third stage (c) is repeated multiple times as a single setting process until the virtual future trajectory after the coordinate transformation converges within the drivable area of ​​the camera image. Figure 4A The diagram illustrates a scenario where the setup process is repeated four times. The numbers in parentheses in the diagram indicate the nth time the setup process was executed.

[0067] (a) Phase 1 exist Figure 4A In the process, when camera 1a acquires a new frame of camera image, the arithmetic unit 11 classifies the area of ​​the camera image into the aforementioned drivable area and non-drivable area through the classification unit 142.

[0068] In parallel with the classification of camera image regions performed by classification unit 142, calculation unit 11 performs the first stage of processing in the first setting process. In the first stage of processing, calculation unit 11 determines the search start position 101 as the position where the vehicle has traveled a distance s0 in the direction of travel (right direction in the figure). For example, the position corresponding to the point p0 at the bottom of the frame corresponding to camera image F when it is later transformed into perspective coordinates is set as the search start position 101.

[0069] The calculation unit 11 further estimates the virtual curvature 100 of the camera image of the current frame by the curvature estimation unit 143 based on the steering angle information detected by the steering angle sensor 2a and the curvature updated when searching for the previous curvature of the camera image of the previous frame.

[0070] (b) Second phase When the arithmetic unit 11 estimates the virtual curvature 100, it performs the second stage of the first setting process in the bird's-eye view coordinates. In the second stage of processing, the virtual trajectory calculation unit 146 of the arithmetic unit 11 calculates the virtual future trajectory 102, which is the estimated value of the future trajectory of the vehicle, based on the virtual curvature 100. The virtual future trajectory 102 can also be referred to as the predicted driving position.

[0071] The calculation unit 11 further generates, via the determination unit 144, a first detection line 103 of the same shape as the virtual future trajectory 102 on the left side relative to the virtual future trajectory 102 and toward the direction of travel of the vehicle, at a predetermined detection line interval d, and generates a second detection line 104 of the same shape as the virtual future trajectory 102 on the right side relative to the virtual future trajectory 102 and toward the direction of travel, at the same detection line interval d. The detection line interval d can be, for example, the vehicle width, or a value obtained by adding a margin to the vehicle width.

[0072] (c) Third stage When the computing unit 11 generates the virtual future trajectory 102, the first detection line 103, and the second detection line 104, it transforms the third stage of the first setup process from the bird's-eye view coordinate system to the perspective view coordinate system.

[0073] The calculation unit 11 performs coordinate transformation on the virtual future trajectory 102, the first detection line 103 and the second detection line 104 from the bird's-eye view coordinate system to the perspective view coordinate system through the coordinate transformation unit 144A of the determination unit 144, and generates the virtual future trajectory, the first detection line and the second detection line in the perspective view coordinate system.

[0074] exist Figure 4A In the first setup process, the classification unit 142 classifies a camera image F of a frame shown in the perspective coordinates into a drivable area 107 and a non-drivable area 108.

[0075] The arithmetic unit 11 makes the virtual future trajectory (thick line) after coordinate transformation, the first detection line 105 and the second detection line 106 overlap with the camera image F.

[0076] The conflict determination unit 144B compares the drivable area 107 contained in the camera image F and the virtual future trajectory (thick line) in the perspective view coordinates to determine whether the virtual future trajectory (thick line) is contained in the drivable area 107 (in other words, whether the virtual future trajectory (thick line) conflicts with the non-drivable area 108).

[0077] As an example, the conflict determination unit 144B starts from point p0 corresponding to the search start position 101 and sequentially moves in the direction of travel (upward) along the trajectory of the first detection line 105 and the second detection line 106, checking whether the first detection line 105 and the second detection line 106 are contained within the drivable area 107. In this case, if at least one of the first detection line 105 and the second detection line 106 changes from the drivable area 107 to the non-drivable area 108 (in other words, if at least one of the first detection line 105 and the second detection line 106 conflicts with the non-drivable area 108), it can be considered that the curvature of the driving road cannot be properly estimated. Conversely, if both the first detection line 105 and the second detection line 106 are contained within the drivable area 107 (in other words, if the first detection line 105 and the second detection line 106 do not conflict with the non-drivable area 108), it can be considered that the curvature of the driving road can be properly estimated.

[0078] Therefore, if at least one of the first detection line 105 and the second detection line 106 conflicts with the non-drivable area 108, the determination unit 144 performs curvature update by the curvature update unit 144C.

[0079] Specifically, in the camera image F, when the first detection line 105 conflicts with the non-drivable area 108 at point c0, the curvature update unit 144C calculates the distance e0 corresponding to point c0 in the bird's-eye view coordinates. Then, the curvature update unit 144C updates the curvature of the interval from point s0 to point e0 in the negative direction to a curvature that bends further to the right (to the right of the direction of travel). The amount of the update will be explained below.

[0080] As another example (illustrations omitted), in camera image F, when the second detection line 106 conflicts with the non-drivable area 108, the curvature update unit 144C calculates the distance e0 corresponding to the conflict point in the bird's-eye view coordinates. Then, the curvature update unit 144C updates the curvature of the interval from point s0 to point e0 in the positive direction to a curvature that bends further to the left (to the left of the direction of travel). The amount of the update will be explained below.

[0081] When the curvature update unit 144C performs curvature update, the calculation unit 11 performs a second setting process.

[0082] (a) Phase 1 The arithmetic unit 11 performs the first stage of the second setup process.

[0083] In the first stage of processing, the calculation unit 11 updates the interval from point s0 to point e0 in the virtual curvature 100 estimated in the first setup process to virtual curvature 100a through the curvature update unit 144C. The amount of the update will be explained below.

[0084] (b) Second phase When the calculation unit 11 is updated to virtual curvature 100a, it performs the second stage of the second setting process in the bird's-eye view coordinates. The calculation unit 11 starts searching from a point s1 that has advanced a predetermined distance from the point s0 where the search began in the first setting process towards the direction of travel of the vehicle (to the right in the figure). The reason for advancing the search position in the second setting process from the point s0 where the search began in the first setting process is that the intervals where the first detection line 105 and the second detection line 106 do not conflict with the non-drivable area 108 can be excluded from the search objects in the second setting process.

[0085] It should be noted that the amount by which the search starts and moves from point s0 to point s1, for example, in the camera image F transformed into perspective coordinates, the movement from the corresponding point p0 to point p1, only needs to be at least equivalent to one pixel. In other words, the movement from point p0 to point p1 can be greater than equivalent to one pixel.

[0086] In the second stage of processing, the virtual trajectory calculation unit 146 of the calculation unit 11 calculates the virtual future trajectory 102a as the estimated value of the future trajectory of the vehicle based on the updated virtual curvature 100a.

[0087] The calculation unit 11 also generates a first detection line 103a with the same shape as the virtual future trajectory 102a on the left side relative to the virtual future trajectory 102a and the direction of travel of the vehicle by the determination unit 144 at a predetermined detection line interval d, and generates a second detection line 104a with the same shape as the virtual future trajectory 102a on the right side relative to the virtual future trajectory 102a and the direction of travel by the predetermined detection line interval d.

[0088] (c) Third stage When the computing unit 11 generates the virtual future trajectory 102a, the first detection line 103a, and the second detection line 104a, it transforms the third stage of the second setting process from the bird's-eye view coordinate system to the perspective view coordinate system.

[0089] The calculation unit 11 performs coordinate transformation on the virtual future trajectory 102a, the first detection line 103a, and the second detection line 104a from the bird's-eye view coordinate system to the perspective view coordinate system through the coordinate transformation unit 144A of the determination unit 144, and generates the virtual future trajectory, the first detection line, and the second detection line in the perspective view coordinate system.

[0090] exist Figure 4A In the second setup process, similar to the first setup process, the camera image F of a frame shown in the perspective coordinates is classified into a drivable area 107 and a non-drivable area 108.

[0091] The arithmetic unit 11 makes the virtual future trajectory (thick line) after coordinate transformation, the first detection line 105a and the second detection line 106a overlap with the camera image F.

[0092] Similar to the first setup process, the conflict determination unit 144B compares the drivable area 107 contained in the camera image F and the virtual future trajectory (thick line) in the perspective coordinates to determine whether the virtual future trajectory (thick line) is contained in the drivable area 107 (in other words, whether the virtual future trajectory (thick line) conflicts with the non-drivable area 108).

[0093] As an example, the conflict determination unit 144B checks whether the first detection line 105a and the second detection line 106a are included in the drivable area 107, starting from the position of point p1 and moving sequentially towards the direction of travel (upward). The steps are the same as the initial setup process. When at least one of the first detection line 105a and the second detection line 106a conflicts with the non-drivable area 108, the determination unit 144 updates the curvature again through the curvature update unit 144C.

[0094] Specifically, in camera image F, if the first detection line 105a at point c1 conflicts with the non-drivable area 108, the curvature update unit 144C calculates the distance e1 corresponding to point c1 in the bird's-eye view coordinates. Then, the curvature update unit 144C updates the curvature of the interval from point s1 to point e1 in the negative direction to a curvature that bends further to the right (to the right of the direction of travel). The amount of the update will be explained below.

[0095] As another example (illustration omitted), in camera image F, when the second detection line 106a conflicts with the non-drivable area 108, the curvature update unit 144C calculates the distance e1 corresponding to the conflict point in the bird's-eye view coordinates. Then, the curvature update unit 144C updates the curvature of the interval from point s1 to point e1 in the positive direction to a curvature that bends further to the left (to the left of the direction of travel). The amount of the update will be explained below.

[0096] about Figure 4A The third setting process is the same as the second setting process described above, so the explanation is omitted.

[0097] When the curvature update unit 144C performs curvature update in the third setting process, the calculation unit 11 performs the fourth setting process.

[0098] (a) Phase 1 The arithmetic unit 11 performs the first stage of the fourth setup process.

[0099] In the first stage of processing, the calculation unit 11 updates the interval from point s2 to point e2 in the virtual curvature 100b estimated in the third setting process to virtual curvature 100c via the curvature update unit 144C. The amount of the update will be explained below.

[0100] (b) Second phase When the calculation unit 11 updates to the virtual curvature 100c, it performs the second stage of the fourth setup process in the bird's-eye view coordinates. The calculation unit 11 starts searching from a point s3 that has advanced a predetermined distance from the point s2 that was searched at the beginning of the third setup process in the vehicle's direction of travel (to the right in the figure). The reason for advancing the search position at the beginning of the fourth setup process from the point s2 that was searched at the beginning of the third setup process is that the intervals where the first detection line 105b and the second detection line 106b do not conflict with the non-drivable area 108 can be excluded from the search objects during the fourth setup process.

[0101] It should be noted that the amount by which the search starts and moves from point s2 to point s3, for example, in the camera image F transformed into perspective coordinates, the movement from the corresponding point p2 to point p3 only needs to be at least equivalent to one pixel. In other words, the movement from point p2 to point p3 can also be greater than equivalent to one pixel.

[0102] In the second stage of processing, the virtual trajectory calculation unit 146 of the calculation unit 11 calculates the virtual future trajectory 102c, which is the estimated value of the future trajectory of the vehicle, based on the updated virtual curvature 100c.

[0103] The calculation unit 11 also generates a first detection line 103c with the same shape as the virtual future trajectory 102c on the left side relative to the virtual future trajectory 102c and the direction of travel of the vehicle by the determination unit 144 at a predetermined detection line interval d, and generates a second detection line 104c with the same shape as the virtual future trajectory 102c on the right side relative to the virtual future trajectory 102c and the direction of travel by the predetermined detection line interval d.

[0104] (c) Third stage When the computing unit 11 generates the virtual future trajectory 102c, the first detection line 103c, and the second detection line 104c, it transforms the third stage of the fourth setting process from the bird's-eye view coordinate system to the perspective view coordinate system.

[0105] The calculation unit 11 performs coordinate transformation on the virtual future trajectory 102c, the first detection line 103c, and the second detection line 104c from the bird's-eye view coordinate system to the perspective view coordinate system through the coordinate transformation unit 144A of the determination unit 144, and generates the virtual future trajectory, the first detection line, and the second detection line in the perspective view coordinate system.

[0106] exist Figure 4A In the fourth setup process, a camera image F of a frame shown in the perspective coordinates is classified into a drivable area 107 and a non-drivable area 108, which is the same as the setup process described above.

[0107] The arithmetic unit 11 makes the virtual future trajectory (thick line) after coordinate transformation, the first detection line 105c and the second detection line 106c overlap with the camera image F.

[0108] Similarly to the above-described setting process, the conflict determination unit 144B compares the drivable area 107 contained in the camera image F with the virtual future trajectory (thick line) and determines whether the virtual future trajectory (thick line) is contained in the drivable area 107 (in other words, whether the virtual future trajectory (thick line) conflicts with the non-drivable area 108).

[0109] As an example, if neither the first detection line 105c nor the second detection line 106c conflicts with the non-drivable area 108, the determination unit 144 ends the setting process. After the setting process is completed, the future trajectory setting unit 151 sets the virtual future trajectory as the future trajectory of the vehicle.

[0110] The arithmetic unit 11 performs the setting process described above on camera images of the same frame based on the frame rate at which camera 1a acquires the camera image. More specifically, when camera 1a acquires a camera image of one frame, the above setting process is repeated multiple times until the next camera image is acquired. The number of times the setting process is repeated can also be set to an upper limit (e.g., 10 times).

[0111] As with the fourth setting process described above, if the first detection line 105x and the second detection line 106x after the coordinate transformation converge within the drivable area in the camera image before the number of repetitions of the setting process reaches the upper limit (10 times), the setting process can be terminated at that moment (in other words, if neither the first detection line 105x nor the second detection line 106x conflicts with the non-drivable area 108).

[0112] In addition, if the first detection line 105 and the second detection line 106 after the coordinate transformation converge within the drivable area in the camera image after the first setting process, the process can be terminated without repeating the setting process.

[0113] It should be noted that when the driving route is at the end of a road, such as a T-junction, or when there is a vehicle ahead of the vehicle on the driving route, the first detection line 105 and the second detection line 106 may not converge within the drivable area during the curvature search. In this case, the curvature search can be terminated, and the planned speed can be set to 0 relative to the search position. This prevents over-exploration of drivable areas.

[0114] <Update Count> Figure 4B This is a schematic diagram illustrating the update of curvature. Figure 4B Is with Figure 4A Enlarged view of the part corresponding to the first stage of the second setup process (a). Figure 4B The next paragraph is... Figure 4B The image above is a further enlarged version.

[0115] As described above, the curvature update unit 144C of the calculation unit 11 updates the interval from point s0 to point e0 (represented by a dashed line) in the virtual curvature 100 estimated in the first setting process to virtual curvature 100a (represented by a solid line). Figure 4B For example, the curvature update unit 144C updates the curvature of the interval from point s0 to point e0 in the negative direction to a curvature that bends to the right (to the right of the direction of travel) compared to the virtual curvature 100.

[0116] <Example of Calculating Update Quantity> Figure 4C This is a schematic diagram illustrating the relationship between the virtual curvature, radius of curvature, and position before and after the update. Assume that point si to point ei is in the interval where the virtual curvature to be updated, and point si is already at the center of the road. At this time, when the virtual curvature of this interval before the update is κi, assume that the location at position l in the direction of travel conflicts with a non-drivable area (in other words, conflicts with the left end of the drivable area). Then, determine how to set the updated virtual curvature κi+1. Assuming that the drivable area is always separated from the virtual future trajectory after the update of the virtual curvature by the above detection line interval d, the relationship between the virtual curvature κi before the update, the radius of curvature Ri, and the position l is as follows: Figure 4C As shown. According to Figure 4C Based on the relationship, the following simultaneous equations are obtained.

[0117]

[0118] The computation unit 11 uses equations (1) to (4) above, and according to Newton's method, to obtain the radius of curvature Ri+1 (the updated radius of curvature) when the radius of curvature Ri, position l, and detection line interval d are given. The updated virtual curvature κi+1 is obtained by taking the reciprocal of the above radius of curvature Ri+1.

[0119] <Calculation of Virtual Curvature> Reference Figure 4D Explain the virtual curvature (estimated value) calculated by the curvature estimation unit 143.

[0120] As described above, the curvature estimation unit 143 calculates the virtual curvature κsj based on the steering angle information detected by the steering angle sensor 2a and the vehicle's driving state (for example, the current vehicle speed). The initial calculated value is the initial search value when the virtual curvature is determined, and therefore can also be called the initial value of the virtual curvature.

[0121] In addition, the curvature estimation unit 143 uses information from the previous planning update unit 145, inputs the above-mentioned virtual curvature κsj, the virtual curvature κpi obtained by searching the previous curvature, and the current vehicle speed v detected by the vehicle speed sensor 2d, and calculates the virtual curvature κj by the following formula (5).

[0122] κj=Wsj×κsj+Wpj×κ´pj(5) The weight Wsj is a weight value that varies with the distance s from the vehicle relative to the estimated curvature calculated from the rudder angle. The closer to the vehicle, the closer it is to 1, and the farther away from the vehicle, the closer it is to 0.

[0123] Furthermore, like weight Wsj, weight Wpj is a weight value that varies with the distance s from the vehicle, relative to the estimated curvature calculated from the rudder angle. The closer to the vehicle, the closer it is to 0; the farther away from the vehicle, the closer it is to 1. The sum of weight Wsj and weight Wpj is always 1, regardless of the distance s.

[0124] Furthermore, the virtual curvature κ´pj is a virtual curvature obtained by offsetting the virtual curvature κpj obtained in the previous search by the distance Δs traveled by the vehicle during the processing cycle ΔT from the previous search to the current search (in the implementation, it corresponds to the frame interval at which the camera 1a acquires the camera image).

[0125] According to equation (5), since the virtual curvature κpj obtained from the previous search is used for this search, the number of iterations can be reduced (in other words, the number of repetitions of the processing is reduced) compared to the case where the virtual curvature κpj is not used, thus reducing the computational load for the search. The distance Δs is obtained by equation (6).

[0126] Δs=v×ΔT (6) <Explanation of the flowchart> Figure 5A It shows that according to a predetermined procedure, by Figure 2 The flowchart illustrates an example of the computational processing performed by the arithmetic unit 11 of the controller 10. The processing shown in this flowchart is repeated, for example, while the vehicle is driving in autonomous driving mode. Alternatively, it can be performed while the vehicle is driving in manual driving mode, for example, when lane keeping assist, as one of the driving assistance functions, is active, i.e., during lane keeping operation.

[0127] In step S10, the arithmetic unit 11 acquires camera images from camera 1a in frames and proceeds to step S20.

[0128] In step S20, the calculation unit 11 classifies the area of ​​the camera image into a drivable area 107 and a non-drivable area 108 by the classification unit 142, and then proceeds to step S30.

[0129] In step S30, the calculation unit 11 calculates the virtual curvature, which is the estimated value of the curvature of the driving road, through the curvature estimation unit 143, and proceeds to step S40.

[0130] In step S40, the calculation unit 11 calculates a virtual future trajectory, which is an estimated value of the future trajectory of the vehicle, through the virtual trajectory calculation unit 146, and proceeds to step S50. The processing in step S40 corresponds to the processing in the second stage (b) of the above-described setting process.

[0131] In step S50, the arithmetic unit 11 performs pre-determination processing and proceeds to step S60. For details regarding the pre-determination processing, please refer to the following text. Figure 5B The flowchart shown is used for illustration. The processing of step S50 corresponds to the first part of the second stage processing (b) and the third stage processing (c) in the above-mentioned setup processing.

[0132] In step S60, the calculation unit 11 determines whether the virtual future trajectory conflicts with the non-drivable area 108 through the conflict determination unit 144B. For example, when it is determined that at least one of the first detection line 105 and the second detection line 106 conflicts with the non-drivable area 108, the calculation unit 11 determines step S60 as affirmative (S60: Yes) and proceeds to step S70. When it is determined that neither the first detection line 105 nor the second detection line 106 conflicts with the non-drivable area 108, the calculation unit 11 determines step S60 as negative (S60: No) and proceeds to step S90.

[0133] The processing in step S60 corresponds to the latter part of the third stage of the setting process described above.

[0134] In step S70, the arithmetic unit 11 determines whether the number of repetitions in the setting process is less than the limit number. When the number of repetitions is less than the limit number, the arithmetic unit 11 determines that step S70 is affirmative (S70: Yes) and proceeds to step S80. When the number of repetitions is not less than the limit number (in other words, the upper limit is reached), the arithmetic unit 11 determines that step S70 is negative (S70: No) and proceeds to step S120.

[0135] In step S80, the calculation unit 11 updates the virtual curvature through the curvature update unit 144C and returns to step S40. Returning to step S40 is to repeat the setting process.

[0136] The processing in step S80 corresponds to the first stage of the setting process (a) in the second and subsequent setting processes described above.

[0137] In step S90, which proceeds after determining that step S60 is negative (S60: No), the calculation unit 11 sets the virtual future trajectory as the future trajectory of the vehicle by the future trajectory setting unit 151, and proceeds to step S100.

[0138] In step S100, the arithmetic unit 11 sends an instruction to the driving control unit 16 to use the planned speed and planned rudder angle based on the future trajectory of the vehicle for driving control, and then proceeds to step S110.

[0139] In step S110, the arithmetic unit 11 determines whether to end the process. For example, when the automatic driving mode is deactivated, the arithmetic unit 11 determines that step S110 is affirmative (S110: Yes) and ends the process. Figure 5A The processing is as follows. For example, if the autonomous driving mode continues, the arithmetic unit 11 determines that step S110 is negative (S110: No), returns to step S10, and repeats the above processing.

[0140] In step S120, which is entered after determining that step S70 is negative (S70: No), the calculation unit 11 sets the virtual future trajectory of the current moment as the future trajectory of the vehicle by the future trajectory setting unit 151, and then enters step S130.

[0141] In step S130, the arithmetic unit 11 performs the prescribed termination process and ends. Figure 5A The processing involves halting the curvature search and setting the planned speed to 0 relative to the search position. This prevents reckless exploration of traversable areas.

[0142] <Pre-judgment processing> Reference Figure 5B The flowchart shown illustrates the details of the pre-decision processing. Figure 5BThis is a flowchart illustrating an example of the pre-determination processing of step S50 executed by the arithmetic unit 11.

[0143] In step S501, the calculation unit 11 generates a first detection line 103 and a second detection line 104 with the same shape as the virtual future trajectory through the determination unit 144, and proceeds to step S503.

[0144] In step S503, the calculation unit 11 performs coordinate transformation on the virtual future trajectory, the first detection line and the second detection line from the bird's-eye view coordinate system to the perspective view coordinate system through the coordinate transformation unit 144A of the determination unit 144, generates the virtual future trajectory, the first detection line and the second detection line in the perspective view coordinate system, and then proceeds to step S505.

[0145] In step S505, the computation unit 11 terminates when the virtual future trajectory after coordinate transformation, the first detection line, and the second detection line overlap with the camera image F. Figure 5B The processing, enter Figure 5A Step S60.

[0146] The implementation method described above achieves the following effects.

[0147] (1) The vehicle control device 200 includes: a camera 1a as an external information acquisition unit, which acquires external information including the surrounding area of ​​the vehicle and the driving road as a camera image F; a classification unit 142, which classifies the area of ​​the camera image F into a drivable area 107 and a non-drivable area 108 through a predetermined segmentation process; a curvature estimation unit 143, which calculates a virtual curvature as an estimated value of the curvature of the driving road based on the driving state including the steering angle of the vehicle; a virtual trajectory calculation unit 146, which calculates a virtual future trajectory 102 as an estimated value of the future trajectory of the vehicle in a bird's-eye view coordinate system based on the virtual curvature; and a determination unit 144, which determines the camera image... The system compares the virtual future trajectory 102 with the image F and determines whether the virtual future trajectory 102 is included in the drivable area 107. The future trajectory setting unit 151 sets the virtual future trajectory 102 as the future trajectory of the vehicle based on the determination result of the determination unit 144. The driving control unit 16 controls the driving of the vehicle based on the future trajectory of the vehicle. The determination unit 144 transforms the virtual future trajectory 102 calculated by the virtual trajectory calculation unit 146 from the bird's-eye view coordinate system to the perspective view coordinate system of the camera image F. By superimposing the virtual future trajectory 102 on the camera image F, it determines whether the virtual future trajectory 102 is included in the drivable area 107.

[0148] Because of this configuration, by transforming the virtual future trajectory 102 calculated in the bird's-eye view coordinate system to the perspective view coordinate system and aligning it with the perspective view coordinate system of the camera image F captured by camera 1a, it is possible to appropriately determine whether the virtual future trajectory 102 is included in the drivable area 107. This is especially true for driving roads with non-constant curvature or driving roads far away from the vehicle, allowing for more accurate determination. Furthermore, by performing driving control based on this appropriately calculated virtual future trajectory 102, for example, even when driving for the first time on a road not stored as high-precision map information in the storage unit 12, appropriate driving control of the vehicle can be achieved.

[0149] (2) In the vehicle control device 200, when the determination unit 144 determines that a part of the virtual future trajectory 102 is not included in the drivable area 107, it updates the virtual curvature based on the position of the virtual future trajectory 102 deviating from the drivable area 107 to the non-drivable area 108, and based on the updated virtual curvature, it determines again whether the virtual future trajectory 102a recalculated by the virtual trajectory calculation unit 146 is included in the drivable area 107.

[0150] Because of this configuration, it is possible to reliably generate a virtual future trajectory 102a that the vehicle can travel on.

[0151] (3) In the vehicle control device 200, the determination unit 144 generates a first detection line 103 with the same shape as the virtual future trajectory 102 on the left side relative to the virtual future trajectory 102 and the direction of travel of the vehicle at a predetermined detection line interval d, and generates a second detection line 104 with the same shape as the virtual future trajectory 102 on the right side relative to the virtual future trajectory 102 and the direction of travel at a predetermined detection line interval d. For the first detection line 103 (105) and the second detection line 104 (106), a conflict determination with the non-drivable area 108 is performed in the direction of travel from the position corresponding to the predetermined reference point p0. If the first detection line 103 (105) conflicts with the non-drivable area 108, the virtual curvature is updated to increase to the right. If the second detection line 104 (106) conflicts with the non-drivable area 108, the virtual curvature is updated to increase to the left.

[0152] Because of this configuration, in the event that the non-drivable area 108 is determined to be in conflict, it is easy to determine the direction of updating the virtual future trajectory 102 (whether the virtual curvature increases to the right or to the left).

[0153] (4) In the vehicle control device 200, if either the first detection line 103 (105) or the second detection line 104 (106) conflicts with the non-drivable area 108, the determination unit 144 regenerates the first detection line 103a and the second detection line 104a to the left and right of the virtual future trajectory 102a based on the updated virtual curvature, respectively, and moves the position corresponding to each reference point p0 in the travel direction. For the regenerated first detection line 103a (105a) and the second detection line 104a (106a), a conflict determination with the non-drivable area 108 is performed in the travel direction from the position corresponding to the moved reference point P1. The amount of movement from reference point p0 to reference point p1 in the camera image F is at least equivalent to one pixel.

[0154] Because of this configuration, by gradually shifting the position where the conflict determination begins while searching for curvature, it is possible to suppress excessive curvature changes for the subsequently updated virtual future trajectory 102a.

[0155] (5) In the vehicle control device 200, when the number of times the virtual curvature is updated reaches a predetermined upper limit, the future trajectory setting unit 151 sets the virtual future trajectory 102, etc., as the future trajectory of the vehicle, and the driving control unit 16 identifies the driving road as untravelable and performs the prescribed driving control.

[0156] Because of this configuration, in situations where progress is impossible or even multiple updates to the virtual curvature fail to generate a virtual future trajectory, the curvature search can be terminated.

[0157] (6) In the vehicle control device 200, if the determination unit 144 reaches the depth distance (corresponding to the vanishing point) corresponding to the end of the virtual future trajectory 102, etc., at the reference points of the first detection line 105, etc. and the second detection line 106, etc., respectively, and the number of updates is less than the upper limit number, the reference points are returned to the positions corresponding to the point p0, which is the initial position, and the determination of whether the virtual future trajectory 102, etc., based on the updated virtual curvature is included in the drivable area 107 is repeated.

[0158] Because of this configuration, for example, by changing the calculation conditions and re-searching for curvature, it is possible to calculate a more appropriate virtual future trajectory.

[0159] (7) In the vehicle control device 200, the virtual trajectory calculation unit 146 calculates the future trajectory of the vehicle at a predetermined time interval, and calculates the virtual future trajectory based on the virtual curvature and the future trajectory of the vehicle calculated in the past.

[0160] Because of this configuration, by reflecting the information of the previously calculated virtual future trajectory, it is possible to suppress the number of times the above-mentioned setup process of updating the virtual curvature → calculating the virtual future trajectory based on the updated virtual curvature → determining conflicts in drivable areas is repeated.

[0161] (8) The vehicle control device 200 also includes a speed planning unit 152, which sets the upper limit of the vehicle speed based on the future trajectory of the vehicle and the allowable acceleration of the vehicle, and generates a speed plan for the vehicle based on the upper limit of the vehicle speed. The driving control unit 16 performs driving control based on the speed plan.

[0162] Because of this configuration, based on the correct identification of the curvature of curves included in the future trajectory of the vehicle, an appropriate speed plan can be formulated in a way that does not produce acceleration that would be unpleasant to the occupants.

[0163] The above-described embodiments can be modified in various ways. The following describes some modifications.

[0164] In the above embodiment, an example was described in which the detection line interval d between the virtual future trajectory 102 and the first detection line 103, and the detection line interval d between the virtual future trajectory 102 and the second detection line 104, were set to the vehicle body width. Alternatively, they can be set to values ​​different from the vehicle body width.

[0165] For example, in the vehicle control device 200 described above, the determination unit 144 may also set the detection line interval d based on at least one of the following: the motion characteristics of the vehicle, the width of the drivable area 107, first information related to the driving characteristics learned by the vehicle, and second information related to the driving characteristics set by the occupants of the vehicle.

[0166] With this configuration, by changing the interval between the first detection line 103 and the second detection line 104 according to the situation, the virtual curvature can be updated to be more suitable for the occupants, and a virtual future trajectory 102a can be generated based on the updated virtual curvature.

[0167] Instead of performing conflict determination between the first detection line 103 and the second detection line 104 and the non-drivable area 108, the conflict with the non-drivable area 108 can also be determined as follows.

[0168] For example, in the vehicle control device 200 described above, the determination unit 144 may also set a detection area including a virtual future trajectory 102 and having a predetermined area width in the left and right directions of the vehicle in the direction of travel of the vehicle. The detection area is used to determine whether it conflicts with the non-drivable area 108 in the direction of travel from a predetermined reference point. If the left side of the left and right ends of the detection area conflicts with the non-drivable area 108, the virtual curvature is updated to increase to the right. If the right side of the left and right ends of the detection area conflicts with the non-drivable area 108, the virtual curvature is updated to increase to the left.

[0169] In this configuration, similar to the above-described implementation, when making a conflict determination with the non-drivable area 108, the direction of updating the virtual future trajectory 102 can be easily determined.

[0170] The above description is only one example. As long as it does not impair the characteristics of the present invention, the present invention is not limited to the above embodiments and variations.

[0171] Using this invention, driving control can be appropriately performed even on roads with non-constant curvature.

Claims

1. A vehicle control device, characterized in that, have: The external information acquisition unit (1a) acquires external information, including the road on which the vehicle is traveling, as an image. The classification unit (142) classifies the regions of the image into drivable regions and non-drivable regions through a prescribed segmentation process; The curvature estimation unit (143) calculates a virtual curvature as an estimated value of the curvature of the driving road based on the driving state including the steering angle of the vehicle. The virtual trajectory calculation unit (146) calculates a virtual future trajectory in the bird's-eye view coordinate system based on the virtual curvature, which is an estimated value of the future driving trajectory of the vehicle. The determination unit (144) compares the external information and the virtual future trajectory to determine whether the virtual future trajectory is included in the drivable area; The future trajectory setting unit (151) sets the virtual future trajectory as the future driving trajectory based on the determination result of the determination unit (144). as well as The driving control unit (16) performs driving control of the vehicle based on the future driving trajectory set by the future trajectory setting unit (151). The determination unit (144) transforms the virtual future trajectory calculated by the virtual trajectory calculation unit (146) from the bird's-eye view coordinate system to the perspective view coordinate system of the external information, and determines whether the virtual future trajectory is included in the drivable area by superimposing the virtual future trajectory on the external information.

2. The vehicle control device according to claim 1, Its characteristics are as follows. When the determination unit (144) determines that a part of the virtual future trajectory is not included in the drivable area, it updates the virtual curvature based on the position where the virtual future trajectory deviates from the drivable area to the non-drivable area, and based on the updated virtual curvature, it determines again whether the virtual future trajectory recalculated by the virtual trajectory calculation unit (146) is included in the drivable area.

3. The vehicle control device according to claim 2, characterized in that, The determination unit (144). Relative to the virtual future trajectory, a first detection line of the same shape as the virtual future trajectory is generated on the left side, towards the direction of travel of the vehicle, at a predetermined detection line interval. Similarly, a second detection line of the same shape as the virtual future trajectory is generated on the right side, towards the direction of travel, at a predetermined detection line interval. The first detection line and the second detection line are respectively used to determine the conflict with the non-drivable area in the direction of travel from a predetermined reference point. If the first detection line conflicts with the non-drivable area, the update is performed by increasing the virtual curvature to the right; if the second detection line conflicts with the non-drivable area, the update is performed by increasing the virtual curvature to the left.

4. The vehicle control device according to claim 3, characterized in that, The determination unit (146). If either the first detection line or the second detection line conflicts with the drivable area, the first detection line and the second detection line are regenerated to the left and right of the virtual future trajectory based on the updated virtual curvature, respectively, and each of the reference points is moved in the direction of travel. For the regenerated first detection line and second detection line, a conflict determination is made with the non-drivable area from the moved reference point in the direction of travel.

5. The vehicle control device according to claim 4, characterized in that, When the number of updates to the virtual curvature reaches a predetermined upper limit, the future trajectory setting unit (151) sets the virtual future trajectory as the future driving trajectory. The driving control unit (16) identifies the driving road as impassable and performs the prescribed driving control.

6. The vehicle control device according to claim 5, characterized in that, The determination unit (144). If the reference points of the first detection line and the second detection line reach the depth distance corresponding to the terminal of the virtual future trajectory, and the number of updates is less than the upper limit number, the reference points are returned to their initial positions, and the determination of whether the virtual future trajectory based on the updated virtual curvature is included in the drivable area is repeated.

7. The vehicle control device according to claim 3, characterized in that, The determination unit (144). The prescribed detection line interval is set based on at least one of the following: the motion characteristics of the vehicle, the width of the drivable area, first information related to the driving characteristics learned by the vehicle, and second information related to the driving characteristics set by the occupants of the vehicle.

8. The vehicle control device according to claim 1, characterized in that, The virtual trajectory calculation unit (146). The future driving trajectory is calculated at specified time intervals, and the virtual future trajectory is calculated based on the virtual curvature and the future driving trajectory calculated in the past.

9. The vehicle control device according to claim 1, characterized in that, It also includes a speed planning unit (152), which sets an upper limit value for the vehicle's speed based on the future driving trajectory set by the future trajectory setting unit (151) and the permissible acceleration of the vehicle, and generates a speed plan for the vehicle based on the upper limit value. The driving control unit (16) performs driving control according to the speed plan.

10. The vehicle control device according to claim 2, characterized in that, The determination unit (144). A detection area is defined, containing the virtual future trajectory and having a specified width in the left-right direction of the vehicle's travel direction. Conflict determination is then performed between the detection area and the non-drivable area from a specified reference point in the travel direction. If the left side of the left-right end of the detection area conflicts with the non-drivable area, the update is performed by increasing the virtual curvature to the right. If the right side of the left-right end of the detection area conflicts with the non-drivable area, the update is performed by increasing the virtual curvature to the left.

Images