Vehicle control device, vehicle control method, and computer program for vehicle control

The vehicle control system stabilizes vehicle behavior by estimating future road shape and averaging curvature over multiple images, addressing inaccuracies in lane marking detection to prevent unstable driving.

JP2026182153APending Publication Date: 2026-11-19TOYOTA JIDOSHA KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2025078119
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2025-05-08
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Existing vehicle control systems based on lane markings and obstacle detection can lead to unstable vehicle behavior due to inaccuracies in road shape estimation, especially in environments where lane markings are obscured or not well-represented in high-precision maps.

Method used

A vehicle control system that estimates future road shape using onboard imaging, sets a target driving trajectory, calculates curvature by averaging over multiple images, and adjusts speed accordingly to stabilize vehicle control without relying on high-precision maps.

Benefits of technology

Stabilizes vehicle behavior by smoothing curvature and speed changes, preventing deviations from the intended path and reducing abrupt accelerations or decelerations, even in environments with poor lane marking visibility.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2026182153000001_ABST
    Figure 2026182153000001_ABST
Patent Text Reader

Abstract

To provide a vehicle control device capable of preventing improper vehicle control. [Solution] The vehicle control device includes a road shape estimation unit 21 that estimates the shape of the road section that the vehicle 1 will travel in the future based on an image representing the surroundings of the vehicle 1 generated by an imaging unit 2 mounted on the vehicle 1 at predetermined intervals; a trajectory setting unit 22 that sets a target driving trajectory for the vehicle 1 based on the estimated shape each time the shape of the road section is estimated; a curvature calculation unit 23 that calculates the curvature of each point in the road section by averaging the curvature of that point obtained based on the shape of the road section estimated from each of a plurality of images obtained in the most recent predetermined period for each point in the road section; a speed setting unit 24 that sets a target speed for when the vehicle 1 travels in the road section according to the curvature of each point in the road section; and a vehicle control unit 25 that controls the vehicle to travel in the road section at a target speed along the target driving trajectory.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a vehicle control computer program. BACKGROUND ART

[0002] A technology has been proposed that generates a target trajectory based on lane marking information represented in a space obtained by converting an image captured by a camera mounted on a mobile object to a bird's-eye view coordinate system, and generates a speed plan indicating a target speed based on the curvature of the target trajectory (see Patent Document 1). PRIOR ART DOCUMENTS PATENT DOCUMENTS

[0003] PATENT DOCUMENT 1 Japanese Unexamined Patent Publication No. 2023-82451 SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] By generating the target trajectory based on the lane markings detected from the image as described above, it is possible to set the target speed in a curved section without using a high-precision map in which the road shape is represented with high accuracy. However, due to the detection accuracy of features such as lane markings and the influence of obstacles present on the road, the shape of the road on which the vehicle is traveling may not be accurately estimated from the image in some cases. In such a case, the behavior of the vehicle controlled based on the estimated road shape may become unstable.

[0005] Accordingly, an object of the present invention is to provide a vehicle control device capable of preventing inappropriate vehicle control from being executed. MEANS FOR SOLVING THE PROBLEMS

[0006] According to one embodiment, a vehicle control device is provided. This vehicle control device includes: a road shape estimation unit that estimates the shape of a road section on which the vehicle will travel in the future based on an image representing the area around the vehicle generated by an imaging unit mounted on the vehicle at predetermined intervals; a trajectory setting unit that sets a target driving trajectory for the vehicle based on the estimated shape each time the shape of the road section is estimated; a curvature calculation unit that calculates the curvature of each point in the road section by averaging the curvature of that point obtained based on the shape of the road section estimated from each of a plurality of images obtained in the most recent predetermined period for each point in the road section; a speed setting unit that sets a target speed for when the vehicle travels through the road section according to the curvature of each point in the road section; and a vehicle control unit that controls the vehicle to travel through the road section at a target speed along the target driving trajectory.

[0007] In one embodiment, the road shape estimation unit detects lane markings from an image, estimates the shape of the road section based on the detected lane markings, and the curvature calculation unit makes a predetermined period longer at a point in the road section where no lane markings are detected in the latest image than a predetermined period at a point where lane markings are detected in the latest image.

[0008] In one embodiment, the road shape estimation unit detects lane markings by inputting an image into a classifier that has been pre-trained to detect lane markings, and the curvature calculation unit lengthens a predetermined period as the number of training images that represent the environment around a vehicle when it is traveling on that road section decreases among the multiple training images used to train the classifier.

[0009] Another embodiment provides a vehicle control method. This vehicle control method includes estimating the shape of a road section that the vehicle will travel in the future based on an image of the area around the vehicle generated by an imaging unit mounted on the vehicle at predetermined intervals; setting a target driving trajectory for the vehicle based on the estimated shape each time the shape of the road section is estimated; determining the curvature of each individual point in the road section by averaging the curvature of that point obtained based on the shape of the road section estimated from each of a plurality of images obtained in the most recent predetermined period; setting a target speed for the vehicle when it travels along the road section according to the curvature of each individual point in the road section; and controlling the vehicle to travel along the target driving trajectory at the target speed in the road section.

[0010] In yet another embodiment, a computer program for vehicle control is provided. This computer program for vehicle control includes instructions to cause a processor mounted on the vehicle to perform the following actions: estimate the shape of a road section that the vehicle will travel on in the future based on an image of the area around the vehicle generated by an imaging unit mounted on the vehicle at predetermined intervals; set a target driving trajectory for the vehicle based on the estimated shape each time the shape of the road section is estimated; determine the curvature of each individual point in the road section by averaging the curvature of that point obtained based on the shape of the road section estimated from each of a plurality of images obtained in the most recent predetermined period; set a target speed for the vehicle when it travels on the road section according to the curvature of each individual point in the road section; and control the vehicle to travel on the road section at a target speed along the target driving trajectory. [Effects of the Invention]

[0011] The vehicle control device described herein has the effect of preventing inappropriate vehicle control. [Brief explanation of the drawing]

[0012] [Figure 1] This is a schematic diagram of a vehicle in which a vehicle control system is implemented. [Figure 2]This is a functional block diagram of the processor of an electronic control unit, which is one embodiment of a vehicle control system. [Figure 3] This is a timing chart showing the time changes of curvature and target speed according to this embodiment, and the time changes of curvature and target speed according to a comparative example. [Figure 4] This is an operation flowchart of the vehicle control process. [Modes for carrying out the invention]

[0013] The following describes the vehicle control device, the vehicle control method executed on the vehicle control device, and the vehicle control computer program, with reference to the figures. This vehicle control device estimates the shape of the road section that the vehicle will travel in the future based on images representing the surrounding conditions of the vehicle generated by an onboard camera, and sets a target driving trajectory for the vehicle based on the estimated shape each time the shape of the road section is estimated. Furthermore, for each point in the road section, this vehicle control device determines the curvature of each point in the road section by averaging the curvature of that point obtained based on the shape of the road section estimated from each of a plurality of images obtained in the most recent predetermined period, and sets a target speed for the vehicle when it travels through that road section according to the curvature of each point in the road section. Then, this vehicle control device controls the vehicle to travel along the target driving trajectory at the target speed in that road section. With this configuration, this vehicle control device aims to stabilize the vehicle's behavior by setting a target driving trajectory that does not deviate in curved sections without using a high-precision map containing information about the shape of each road section, and by suppressing abrupt fluctuations in the target speed in curved sections.

[0014] Figure 1 is a schematic diagram of a vehicle on which a vehicle control device is implemented. In this embodiment, vehicle 1 has a camera 2 and an electronic control unit (ECU) 3, which is an example of a vehicle control device. Camera 2 and ECU 3 are connected to each other so as to be able to communicate with each other. Vehicle 1 may further have a distance measuring sensor (not shown), such as LiDAR or radar, for measuring the distance to objects present around vehicle 1. Vehicle 1 may also further have a wireless communication terminal (not shown) for wirelessly communicating with external equipment. Furthermore, vehicle 1 may further have a positioning device (not shown), such as a GPS receiver, for determining the position of vehicle 1 in accordance with a satellite positioning system.

[0015] Camera 2 is an example of an imaging unit and is mounted on Vehicle 1 facing a predetermined area (for example, the area in front of Vehicle 1) so that the area is included in the shooting range of Camera 2. Camera 2 then photographs the predetermined area at predetermined shooting intervals (for example, 1 / 30 second to 1 / 10 second) and generates an image representing the predetermined area. Note that Vehicle 1 may be equipped with multiple cameras with different shooting directions or focal lengths.

[0016] Camera 2 outputs the generated image and the time of image generation to ECU 3 each time it generates an image.

[0017] The ECU3 performs vehicle control processing for the vehicle 1. To this end, the ECU3 has a communication interface 11, a memory 12, and a processor 13. The communication interface 11, the memory 12, and the processor 13 may each be configured as separate circuits, or they may be integrated as a single integrated circuit.

[0018] The communication interface 11 has an interface circuit for connecting the ECU 3 to other equipment in the vehicle 1. The communication interface 11 then passes the image received from the camera 2 and its generation time to the processor 13.

[0019] The memory 12 is an example of a storage unit, and includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 12 stores various types of data used in vehicle control processing executed by the processor 13. For example, the memory 12 stores parameters of the camera 2 such as a focal length, an imaging direction, and a mounting position, as well as various parameters for specifying a classifier used for detection of features and the like. The memory 12 also temporarily stores the image received from the camera 2 and the generation time thereof, as well as various types of data generated during the vehicle control processing. Furthermore, the memory 12 may store map information representing the position and range of each individual road section and the maximum speed for that road section. Note that, unlike a high-precision map, this map information does not need to include information related to the shape of each road section.

[0020] The processor 13 includes one or a plurality of CPUs (Central Processing Units) and peripheral circuits thereof. The processor 13 may further include other arithmetic circuits such as a logic operation unit, a numerical operation unit, or a graphics processing unit. The processor 13 executes vehicle control processing for the vehicle 1. The processor 13 also stores the image received from the camera 2 and the generation time thereof in the memory 12.

[0021] Figure 2 is a functional block diagram of the processor 13 related to vehicle control processing. The processor 13 includes a road shape estimation unit 21, a trajectory setting unit 22, a curvature calculation unit 23, a speed setting unit 24, and a vehicle control unit 25. These units included in the processor 13 are, for example, functional modules implemented by a computer program operating on the processor 13. Alternatively, these units included in the processor 13 may be dedicated arithmetic circuits provided in the processor 13.

[0022] The road shape estimation unit 21 estimates the shape of the road segment that the vehicle 1 will travel in the future at predetermined cycles, based on an image representing the surroundings of the vehicle 1 generated by the camera 2 whose imaging range is a predetermined area in the traveling direction of the vehicle 1. To this end, the road shape estimation unit 1 inputs the latest image to a discriminator that has been pre-trained to detect predetermined features provided on or around the road, so as to detect the predetermined features represented in the image. The predetermined features are features provided along the shape of the road, for example, road markings drawn on the road surface along the extending direction of the road, such as guardrails, curbs, or lane division lines. The discriminator is configured by a convolutional neural network (CNN) for object detection such as Single Shot MultiBox Detector or YOLO, or a CNN for semantic segmentation such as U-Net. Alternatively, the discriminator may be configured by a deep neural network (DNN) having an attention mechanism, such as a Vision Transformer. Alternatively, the discriminator may be a discriminator based on a machine learning method other than DNN, such as a support vector machine. The discriminator is pre-trained according to a predetermined learning method such as an error backpropagation method, using a plurality of training images including images representing any type of feature to be detected.

[0023] When a feature is detected, the road shape estimation unit 21 estimates the road shape based on that feature. To do this, the road shape estimation unit 21 refers to parameters of the camera 2, such as the camera's mounting position, shooting direction, and field of view, and for each feature, it determines the position in a real-space coordinate system based on the position of the camera 2, corresponding to the pixel in which the feature is represented, thereby estimating the position of the feature in that coordinate system. The road shape estimation unit 21 then considers the set of positions of individual features represented in that coordinate system to represent the shape of the road that vehicle 1 will travel on in the future. In particular, if multiple lane markings are detected as features, the road shape estimation unit 21 considers the two lane markings closest to vehicle 1 as the lane markings that define the lane in which vehicle 1 is currently traveling (hereinafter referred to as "the vehicle's lane"). The road shape estimation unit 21 then estimates the shape represented by the two lane markings that define the vehicle's lane as the shape of the road that vehicle 1 will travel on in the future.

[0024] Each time the road shape estimation unit 21 estimates the road shape, it passes the set of positions of individual features representing the estimated road shape to the trajectory setting unit 22 and the curvature calculation unit 23.

[0025] Each time the shape of a road section is estimated, the trajectory setting unit 22 sets a target driving trajectory, which is the trajectory that vehicle 1 is expected to take when traveling through that road section, based on the estimated road shape. For example, if two lane markings are detected that demarcate the vehicle's own lane, and the road shape is represented by these two lane markings, the trajectory setting unit 22 sets the target driving trajectory to pass through the center of these two lane markings. Also, if the road section that vehicle 1 is traveling through is a left-hand traffic section, and a curb or guardrail is detected located to the left of vehicle 1 in the direction of travel, and the road shape is represented by this curb or guardrail, the trajectory setting unit 22 sets the target driving trajectory to pass through a position that is a predetermined offset distance from the curb or guardrail towards the center of the road. The predetermined offset distance is set so that there is a gap of about 1m to severalm between the curb or guardrail and vehicle 1.

[0026] Furthermore, in undetected sections where features used for road shape estimation have not been detected, the trajectory setting unit 22 can set the target driving trajectory for the undetected section by interpolating according to a predetermined interpolation method so as to smoothly connect the target driving trajectories set for the sections before and after the undetected section. The predetermined interpolation method can be, for example, spline interpolation.

[0027] The trajectory setting unit 22 notifies the vehicle control unit 25 of the set target driving trajectory.

[0028] The curvature calculation unit 23 calculates the curvature of each individual point in the road section where the road shape has been estimated, by averaging the curvature of that point, which is obtained based on the shape of the road section estimated from each of a plurality of images obtained in the most recent predetermined period.

[0029] The curvature calculation unit 23 calculates the curvature at each point in the road section based on the set of locations of features representing the road shape each time the road shape is estimated. In doing so, the curvature calculation unit 23 finds an approximate curve that passes through the location of the feature corresponding to the point of interest and the locations of the features before and after it, and uses the curvature of this approximate curve as the curvature at the point of interest. The curvature calculation unit 23 calculates the curvature at each point by repeating the above process while changing the point of interest. If two lane markings that demarcate the current lane are detected as representing the estimated road shape, the curvature calculation unit 23 only needs to calculate the curvature for each point on either lane marking. Alternatively, the curvature calculation unit 23 may calculate the curvature for each of the two lane markings, for each point on that lane marking.

[0030] Furthermore, the curvature calculation unit 23 aligns multiple road shapes estimated from multiple images generated at different times. To this end, the curvature calculation unit 23 estimates the position of vehicle 1 at the generation time of each image used to estimate the road shape. In doing so, the curvature calculation unit 23 uses dead reckoning to determine the difference between the position and direction of travel of vehicle 1 at the generation time of each past image obtained within a predetermined period and the position and direction of travel of vehicle 1 at the generation time of the most recent image. Alternatively, the curvature calculation unit 23 may determine the position of vehicle 1 at the generation time of each image within a predetermined period using a positioning device (not shown). Similarly, the curvature calculation unit 23 may measure the direction of travel of vehicle 1 at the generation time of each image within a predetermined period using a compass sensor (not shown) mounted on vehicle 1. The curvature calculation unit 23 may then determine, based on the position and direction of travel of the vehicle 1 at the time each image is generated, the difference between the position and direction of travel of the vehicle 1 at the time the most recent image is generated and

[0031] The curvature calculation unit 23 aligns multiple road shapes by changing the position and orientation of the road shape estimated from past images by the difference between the position and direction of travel of vehicle 1 at the generation time of each image and the position and direction of travel of vehicle 1 at the generation time of the latest image. The curvature calculation unit 23 then calculates the curvature of a point by averaging the curvature of the same point in each of the aligned multiple road shapes. In this case, the curvature calculation unit 23 may consider a point of interest on the road shape estimated from the latest image as the same point if the point closest to that point in the road shape estimated from past images after alignment is within a predetermined distance (for example, several tens of centimeters to 1 meter) from that point of interest. Alternatively, the curvature calculation unit 23 may weight the curvature of the same point in each of the multiple road shapes using a weighting coefficient that increases as the image generation time approaches the present time. Alternatively, the curvature calculation unit 23 may average the curvature of individual points during the most recent road shape estimation with the curvature of corresponding points in the road shape estimated from the latest image, using a forgetting coefficient. This improves the accuracy of curvature estimation, as curvatures that are relatively close to the vehicle 1 and appear larger in the image are given greater weight for each point. Alternatively, the curvature calculation unit 23 may weight-average the curvature of the same point in each of multiple road shapes using a weighting coefficient that increases as the certainty of feature detection at the point of interest increases. The certainty of feature detection can be the confidence level regarding the detected feature at that point, output by the classifier.

[0032] Furthermore, when lane markings are detected to represent the road shape, the curvature calculation unit 23 may extend the predetermined period for locations where lane markings are not detected, even though they are expected to exist in the latest image in which lane markings were detected, to a longer period than the predetermined period for locations where lane markings were detected. In this way, by averaging the curvature over a longer period for locations where lane marking detection failed, even if the location is obscured by other vehicles in the latest image, the possibility of referring to an image obtained at a time when the lane markings at that location were not obscured increases. Therefore, the curvature calculation unit 23 can improve the accuracy of curvature estimation even for such locations. Furthermore, locations where lane markings are not detected, even though they are expected to exist, may include, for example, locations where lane markings are not detected at the location itself, but are determined to exist based on interpolation using lane markings detected in the preceding and succeeding sections, and locations within a predetermined distance from such locations.

[0033] Furthermore, regarding the surrounding environment of vehicle 1, the curvature calculation unit 23 may set a longer predetermined period in environments where the number of training images used to train the classifier used for detecting lane markings is small. In this case, a period length table representing the length of the predetermined period corresponding to the number of training images for each environment is pre-stored in the memory 12. The curvature calculation unit 23 then identifies the environment around vehicle 1 based on images obtained by camera 2 or sensor signals obtained by sensors (not shown) mounted on vehicle 1 for detecting the environment around vehicle 1. The curvature calculation unit 23 then determines the length of the predetermined period corresponding to the identified environment by referring to the period length table.

[0034] The environment surrounding vehicle 1 is represented, for example, by the weather around vehicle 1, such as sunny or rainy weather, or by the brightness around vehicle 1. For example, if the number of training images in rainy weather is less than the number of training images in sunny weather, the length of the predetermined period corresponding to rainy weather will be set to be longer than the length of the predetermined period corresponding to sunny weather in the period length table. Also, if the number of training images in a dark environment, such as nighttime, is less than the number of training images in a bright environment, such as daytime, the length of the predetermined period corresponding to the dark environment will be set to be longer than the length of the predetermined period corresponding to the bright environment in the period length table.

[0035] If the conditions around vehicle 1 are represented by weather, the curvature calculation unit 23 determines that the weather around vehicle 1 is rainy if the amount of rain detected by the rain sensor (an example of the sensors mentioned above) is above a predetermined threshold, and determines that the weather around vehicle 1 is sunny if the detected amount of rain is below the predetermined threshold. The curvature calculation unit 23 then determines the length of a predetermined period corresponding to the weather around vehicle 1 by referring to a period length table. If the conditions around vehicle 1 are represented by brightness, the curvature calculation unit 23 can determine the length of a predetermined period corresponding to the illuminance detected by the illuminance sensor (another example of the sensors mentioned above) by referring to a period length table. The curvature calculation unit 23 may also identify the environment around vehicle 1 by inputting the image generated by camera 2 into a classifier for identifying the environment around vehicle 1. Such a classifier may be composed of a CNN or a DNN with an attention mechanism.

[0036] Similarly, for road sections on which vehicle 1 is traveling, the curvature calculation unit 23 may set a longer predetermined period for road sections with fewer training images used to train the classifier used for detecting lane markings. In this case, a period length table representing the length of a predetermined period corresponding to the number of training images representing the surrounding environment of each road section is pre-stored in memory 12 for each road section. The curvature calculation unit 23 then identifies the road section on which vehicle 1 is traveling by referring to the current position of vehicle 1, which is determined by a positioning device (not shown) mounted on vehicle 1, and the map information stored in memory 12. The curvature calculation unit 23 then determines the length of a predetermined period corresponding to the identified road section by referring to the period length table. In this way, the curvature calculation unit 23 can improve the accuracy of estimating the curvature of road sections even in environments where the classifier has not been sufficiently trained, by averaging the curvature over a longer period.

[0037] The curvature calculation unit 23 notifies the speed setting unit 24 of the curvature of each point in the road section where the road shape has been estimated.

[0038] The speed setting unit 24 sets a target speed for when vehicle 1 travels through a road section, according to the curvature of each point in the road section whose road shape has been estimated. To do this, the speed setting unit 24 determines the maximum value among the curvatures of each point and determines the upper limit speed corresponding to the maximum value of curvature by referring to a reference table that represents the upper limit speed for each curvature. The speed setting unit 24 then sets that upper limit speed as the target speed. The reference table only needs to be pre-stored in the memory 12. If the speed set by the driver via the control device installed in the vehicle's cabin is lower than the upper limit speed, the speed setting unit 24 may set the speed set by the driver as the target speed. Furthermore, if map information representing the location and range of each road section and the maximum speed for that road section is stored in the memory 12, the speed setting unit 24 identifies the maximum speed set for the road section including the vehicle's current position. If the identified maximum speed is lower than the upper limit speed corresponding to the maximum curvature, the speed setting unit 24 may set that maximum speed as the target speed. The current position of vehicle 1 can be the most recent position determined by the positioning device.

[0039] The speed setting unit 24 may also modify the newly set target speed so that the difference between the previously set target speed and the newly set target speed falls within a predetermined allowable range of change, in order to prevent the vehicle 1 from accelerating or decelerating suddenly.

[0040] The speed setting unit 24 notifies the vehicle control unit 25 of the set target speed.

[0041] The vehicle control unit 25 controls vehicle 1 to travel along a target trajectory at a target speed in a road section where the road shape has been estimated. For example, the vehicle control unit 25 projects the target trajectory onto the image by referring to parameters of camera 2, such as the mounting position, shooting direction, and focal length of camera 2. The vehicle control unit 25 then determines the difference between the position of the projected target trajectory and the position of the target trajectory on the image when vehicle 1 is on the target trajectory (hereinafter referred to as the ideal position), and controls the steering so that the difference becomes zero. If the position of the projected target trajectory is to the left of the ideal position, vehicle 1 is traveling to the right of the target trajectory, so the vehicle control unit 25 controls the steering so that the direction of travel of vehicle 1 is to the left of the current direction of travel. Also, if the position of the projected target trajectory is to the right of the ideal position, the vehicle control unit 25 controls the steering so that the direction of travel of vehicle 1 is to the right of the current direction of travel. Note that the larger the difference, the larger the amount of steering angle rotation the vehicle control unit 25 may increase.

[0042] Furthermore, the vehicle control unit 25 determines the difference between the speed of vehicle 1, measured by a vehicle speed sensor (not shown) mounted on vehicle 1, and the target speed, and controls the powertrain and brakes so that the difference becomes zero.

[0043] If there is another vehicle (hereinafter referred to as the preceding vehicle) traveling ahead of vehicle 1 in its own lane, the vehicle control unit 25 may control the powertrain and brakes so that the distance between vehicle 1 and the preceding vehicle is greater than or equal to a predetermined distance. In this case, the vehicle control unit 25 detects the other vehicle shown in the image by inputting the image generated by camera 2 into a classifier that has been pre-trained to detect other vehicles. The vehicle control unit 25 then identifies the vehicle shown within the area of ​​the own lane, enclosed by the two lane markings that demarcate the own lane in the image, as the preceding vehicle. Here, there is a one-to-one correspondence between the position in the image and the direction as seen from camera 2. Furthermore, the position of the lower edge of the area in the image where the preceding vehicle is shown is presumed to represent the position where the preceding vehicle is in contact with the road surface. Therefore, the vehicle control unit 25 can estimate the distance from vehicle 1 to the preceding vehicle by referring to the position of the lower edge of the area in the image where the preceding vehicle is shown, and the parameters of camera 2, such as the mounting position, shooting direction, and field of view of camera 2. If the estimated distance to the preceding vehicle is less than a predetermined distance, the vehicle control unit 25 should control the powertrain and brakes so that vehicle 1 decelerates. On the other hand, if the estimated distance to the preceding vehicle is greater than or equal to the predetermined distance, the vehicle control unit 25 should control the powertrain and brakes so that the speed of vehicle 1 approaches the target speed.

[0044] Figure 3 is a timing chart showing the time changes of curvature and target speed according to this embodiment and according to the comparative example. In Figure 3, in the comparative example, each time the road shape is estimated, the curvature itself calculated from the estimated road shape is used to set the target speed. In Figure 3, the horizontal axis represents elapsed time. In the upper chart, the vertical axis represents curvature. Graph 301 shows the time change of curvature used to set the target speed according to this embodiment, and graph 311 shows the time change of curvature used to set the target speed according to the comparative example. In the lower chart, the vertical axis represents speed. Graph 302 shows the time change of target speed according to this embodiment, and graph 312 shows the time change of target speed according to the comparative example.

[0045] In the comparative example, since curvature averaging is not performed, the curvature may fluctuate rapidly at the same point each time the road shape is estimated in a curved section due to the influence of calculation errors in curvature depending on the detection accuracy of the features used to estimate the road shape. Also, since the curvature is larger in curved sections, the target speed is set to be lower than the speed Sm set by the driver. In the comparative example, since the curvature changes rapidly each time the road shape is estimated, the set target speed also changes rapidly. Because the target speed changes rapidly, in order to make vehicle 1 follow the target speed, vehicle 1 will have to be accelerated or decelerated rapidly, which may destabilize the behavior of vehicle 1 and cause anxiety to the occupants of vehicle 1. In contrast, in this embodiment, since the curvature is averaged for each point over the most recent predetermined period, the time change of the curvature used to set the target speed (i.e., the maximum value of curvature in the road section where the road shape is estimated) becomes smooth. Therefore, the set target speed also changes smoothly, and the destabilization of vehicle 1's behavior is prevented.

[0046] Figure 4 is an operation flowchart of the vehicle control process. The processor 13 should execute the vehicle control process according to the following operation flowchart.

[0047] The road shape estimation unit 21 estimates the shape of the road section that the vehicle 1 will travel on in the future based on the image generated by the camera 2 (step S101). The trajectory setting unit 22 sets a target travel trajectory based on the estimated road shape (step S102).

[0048] The curvature calculation unit 23 calculates the curvature of each point in the road section where the road shape has been estimated by averaging the curvature of that point, which is obtained based on the shape of the road section estimated from each of a plurality of images obtained in the most recent predetermined period (step S103). The speed setting unit 24 sets the target speed according to the curvature of each point in the road section where the road shape has been estimated (step S104).

[0049] The vehicle control unit 25 controls vehicle 1 to travel along the target trajectory at the target speed (step S105). Then the processor 13 terminates the vehicle control process.

[0050] As explained above, this vehicle control system estimates the shape of the road section the vehicle will travel in the future based on images representing the surrounding environment of the vehicle generated by an onboard camera, and sets the vehicle's target driving trajectory based on the estimated shape each time the shape of the road section is estimated. In this way, when setting the target driving trajectory, the road shape estimated from each individual image is used, rather than the road shape averaged across multiple images or preceding and succeeding sections. Therefore, at points where the curvature changes significantly, such as the entry point into a curve, the estimated change in curvature may not be able to keep up with the actual change in road curvature, preventing the target driving trajectory from shifting to the outside of the curve. As a result, even when the vehicle is driven along the target driving trajectory, it is prevented from deviating from the road it is traveling on. Furthermore, for each point in the road section, this vehicle control system determines the curvature of that point by averaging the curvature of that point obtained based on the shape of the road section estimated from each of multiple images obtained in the most recent predetermined period, and sets the target speed for the vehicle when traveling on that road section according to the curvature of each point in the road section. This suppresses abrupt changes in curvature caused by fluctuations in the estimated road shape due to factors such as the accuracy of feature detection, thereby also suppressing abrupt changes in the target speed. As a result, this vehicle control device can set a target driving trajectory that does not deviate in curved sections and stabilize the vehicle's behavior without using a high-precision map containing information about the shape of individual road sections.

[0051] In a modified version, the road shape estimation unit 21 may estimate the traversable paths for each lane (hereinafter referred to as traversable paths) for the road section that the vehicle 1 will travel in the future, as the shape of that road section. In this case, the road shape estimation unit 21 obtains the traversable paths for each lane by inputting the image generated by the camera 2 into a classifier that has been pre-trained to output the traversable paths for each lane. Such a classifier is configured as a CNN or a DNN with an attention mechanism. Furthermore, the traversable paths for each lane are represented in a coordinate system based on the position of the camera 2. In this case, the trajectory setting unit 22 can identify the traversable path closest to the vehicle 1 from among the traversable paths for each lane as the traversable path for its own lane, and select the identified trajectory as the target driving trajectory. Furthermore, the curvature calculation unit 23 can calculate the curvature of each point on the traversable path for its own lane in the same manner as in the above embodiment, and average the curvature of that point over a predetermined period of time.

[0052] The computer program that realizes the functions of the processor 13 of the ECU3 according to the above embodiment or modification may be provided in the form of being recorded on a computer-readable portable recording medium such as semiconductor memory, magnetic recording medium, or optical recording medium.

[0053] As described above, those skilled in the art can make various modifications within the scope of the present invention to suit the implemented form. [Explanation of symbols]

[0054] 1 Vehicle, 2 Camera, 3 Electronic Control Unit (ECU), 11 Communication Interface, 12 Memory, 13 Processor, 21 Road Shape Estimation Unit, 22 Trajectory Setting Unit, 23 Curvature Calculation Unit, 24 Speed ​​Setting Unit, 25 Vehicle Control Unit

Claims

1. A road shape estimation unit estimates the shape of the road section that the vehicle will travel on in the future, based on an image representing the area around the vehicle generated by an imaging unit mounted on the vehicle at predetermined intervals. Each time the shape of the road section is estimated, a trajectory setting unit sets the target driving trajectory of the vehicle based on the estimated shape, A curvature calculation unit that determines the curvature of each point in the road section by averaging the curvature of each point in the road section, which is obtained based on the shape of the road section estimated from each of the multiple images obtained in the most recent predetermined period, A speed setting unit that sets a target speed when the vehicle travels along the road section according to the curvature of each point in the road section, A vehicle control unit controls the vehicle to travel along the target travel trajectory at the target speed in the aforementioned road section, A vehicle control device having

2. The road shape estimation unit detects lane markings from the image and estimates the shape of the road section based on the detected lane markings. The vehicle control device according to claim 1, wherein the curvature calculation unit makes the predetermined period at a point in the road section where no lane markings are detected in the latest image longer than the predetermined period at a point where lane markings are detected in the latest image.

3. The road shape estimation unit detects the lane markings by inputting the image into a classifier that has been pre-trained to detect lane markings. The vehicle control device according to claim 1, wherein the curvature calculation unit lengthens the predetermined period as the number of training images representing the environment around the vehicle when the vehicle is traveling on the road section decreases among the plurality of training images used to train the classifier.

4. At predetermined intervals, the shape of the road section that the vehicle will travel on in the future is estimated based on an image representing the area around the vehicle generated by an imaging unit mounted on the vehicle. Each time the shape of the road section is estimated, the target driving trajectory of the vehicle is set based on the estimated shape. For each point in the aforementioned road section, the curvature of that point is determined by averaging the curvature of that point, which is obtained based on the shape of the road section estimated from each of the multiple images obtained in the most recent predetermined period, thereby determining the curvature of each point in the aforementioned road section. A target speed is set for the vehicle when traveling along the road section, according to the curvature of each point in the road section. Controlling the vehicle to travel along the target trajectory at the target speed in the aforementioned road section. A vehicle control method that includes the following.

5. At predetermined intervals, the shape of the road section that the vehicle will travel on in the future is estimated based on an image representing the area around the vehicle generated by an imaging unit mounted on the vehicle. Each time the shape of the road section is estimated, the target driving trajectory of the vehicle is set based on the estimated shape. For each point in the aforementioned road section, the curvature of that point is determined by averaging the curvature of that point, which is obtained based on the shape of the road section estimated from each of the multiple images obtained in the most recent predetermined period, thereby determining the curvature of each point in the aforementioned road section. A target speed is set for the vehicle when traveling along the road section, according to the curvature of each point in the road section. Controlling the vehicle to travel along the target trajectory at the target speed in the aforementioned road section. A vehicle control computer program that causes the processor mounted on the vehicle to perform the following action.

Citation Information

Patent Citations

  • Mobile object control device, mobile object control method, and storage medium

    JP2023082451A