Vehicle control device, vehicle control method, and vehicle control computer program

Abstract

To provide a vehicle control device that is able to safely control a vehicle even when an object that is difficult to detect exists on a traveling path of the vehicle.SOLUTION: A vehicle control device includes: a detection unit that detects a candidate of an object 401 existing on a road surface ahead of a vehicle 10, from an image showing a periphery of the vehicle 10 generated by an imaging unit provided in the vehicle 10; a determination unit that determines whether a direction of the vehicle 10 is deflected by a predetermined angle or more when the vehicle 10 reaches a position of a candidate of the detected object 401; and a vehicle control unit that performs control to decelerate the vehicle 10 when the direction of the vehicle 10 is deflected by the predetermined angle or more.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a computer program for vehicle control. [Background technology]

[0002] A technology has been proposed in which obstacles present around a vehicle are detected from sensor signals obtained by sensors mounted on the vehicle, and the detection results are used for automatic driving control of the vehicle (see Patent Document 1).

[0003] The vehicle control system described in Patent Document 1 recognizes the distribution of obstacles in the vehicle's traveling direction, determines a target trajectory for each wheel of the vehicle based on the recognized distribution of obstacles, and then automatically drives the vehicle along the target trajectory. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2018 / 179359 Summary of the Invention [Problem to be solved by the invention]

[0005] An obstacle on the path of a vehicle may be an object whose shape, color, and size cannot be specified in advance, such as a fallen object or a pothole. Such an object may not be accurately detected from the sensor signals obtained by the sensors mounted on the vehicle. As a result, appropriate vehicle control may not be executed.

[0006] Therefore, an object of the present invention is to provide a vehicle control device that can safely control a vehicle even if an object that is difficult to detect is present on the path of the vehicle. [Means for solving the problem]

[0007] According to one embodiment, there is provided a vehicle control device including: a detection unit that detects candidate objects present on a road surface ahead of the vehicle from an image showing the surroundings of the vehicle generated by an imaging unit provided in the vehicle; a determination unit that determines whether the orientation of the vehicle has deviated by a predetermined angle or more when the vehicle reaches the position of the detected candidate object; and a vehicle control unit that controls the vehicle to decelerate when the orientation of the vehicle has deviated by the predetermined angle or more.

[0008] In this vehicle control device, the detection unit determines whether or not further object candidates can be detected from the ranging signal generated by the ranging sensor mounted on the vehicle, and the vehicle control unit preferably controls the vehicle to decelerate if the vehicle's direction deviates by more than a predetermined angle when no object candidate can be detected from the ranging signal.

[0009] In this case, when a candidate object is also detected from the ranging signal, the vehicle control unit preferably controls the vehicle so that the vehicle avoids the position of the candidate object.

[0010] According to another embodiment, there is provided a vehicle control method including: detecting a candidate object present on a road surface ahead of the vehicle from an image showing the surroundings of the vehicle generated by an imaging unit provided in the vehicle; determining whether the orientation of the vehicle has deviated by a predetermined angle or more when the vehicle reaches the position of the detected candidate object; and controlling the vehicle to decelerate if the orientation of the vehicle has deviated by the predetermined angle or more.

[0011] According to yet another embodiment, there is provided a computer program for controlling a vehicle, the computer program including instructions for causing a processor mounted on the vehicle to execute the following: detect a candidate object present on a road surface ahead of the vehicle from an image representing the surroundings of the vehicle generated by an imaging unit provided in the vehicle, determine whether the orientation of the vehicle has deviated by a predetermined angle or more when the orientation of the vehicle has deviated by the predetermined angle or more, and control the vehicle to decelerate if the orientation of the vehicle has deviated by the predetermined angle or more. [Effects of the Invention]

[0012] The vehicle control device according to the present disclosure has the effect of being able to safely control the vehicle even if an object that is difficult to detect is present on the path of the vehicle. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a schematic configuration diagram of a vehicle control system in which a vehicle control device is implemented. [Figure 2] 1 is a hardware configuration diagram of an electronic control device that is one embodiment of a vehicle control device. [Figure 3] FIG. 2 is a functional block diagram of a processor of an electronic control unit related to vehicle control processing. [Figure 4] FIG. 3 is a diagram illustrating an example of a vehicle control process according to the present embodiment. [Figure 5] FIG. 10 is a diagram illustrating another example of the vehicle control process according to the present embodiment. [Figure 6] FIG. 10 is a diagram illustrating yet another example of the vehicle control process according to the present embodiment. [Figure 7] 4 is an operational flowchart of a vehicle control process. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, a vehicle control device, a vehicle control method, and a vehicle control computer program executed by the vehicle control device will be described with reference to the drawings. The vehicle control device detects candidate objects present on the road surface ahead of the vehicle from an image showing the vehicle's surroundings generated by an imaging unit provided in the vehicle. Furthermore, when the vehicle reaches the position of the detected candidate object, the vehicle control device determines whether the vehicle's orientation has deviated by more than a predetermined angle based on a sensor signal or image obtained by a behavior sensor that detects the vehicle's behavior. If the vehicle's orientation has deviated by more than the predetermined angle, the vehicle control device controls the vehicle to decelerate.

[0015] FIG. 1 is a schematic diagram of a vehicle control system in which a vehicle control device is implemented. The vehicle control system 1 is mounted on a vehicle 10 and controls the vehicle 10. To this end, the vehicle control system 1 includes a camera 2, a distance measurement sensor 3, a behavior sensor 4, and an electronic control unit (ECU) 5, which is an example of a vehicle control device. The camera 2, the distance measurement sensor 3, and the behavior sensor 4 are communicatively connected to the ECU 5. The vehicle control system 1 may also include a navigation device (not shown) for searching for a planned driving route to a destination. The vehicle control system 1 may also include a GPS receiver (not shown) for determining the position of the vehicle 10. The vehicle control system 1 may also include a storage device (not shown) for storing map information. The vehicle control system 1 may also include a wireless communication terminal (not shown) for wirelessly communicating with devices external to the vehicle 10.

[0016] Camera 2 is an example of an imaging unit that generates an image showing the surroundings of vehicle 10. Camera 2 has a two-dimensional detector configured with an array of photoelectric conversion elements, such as a CCD or C-MOS, that are sensitive to visible light, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. Camera 2 is attached, for example, inside the passenger compartment of vehicle 10 so as to face forward of vehicle 10. Camera 2 photographs the area ahead of vehicle 10 at predetermined photographing intervals (for example, 1 / 30 to 1 / 10 seconds) and generates an image showing the area ahead. The image obtained by camera 2 may be a color image or a grayscale image. Note that vehicle 10 may be provided with two or more cameras with different photographing directions or focal lengths.

[0017] Every time the camera 2 generates an image, it outputs the generated image to the ECU 5 via the in-vehicle network.

[0018] The ranging sensor 3 is an example of a ranging unit that generates ranging signals that indicate the distance to objects around the vehicle 10. The ranging sensor 3 can be, for example, a ranging sensor using LiDAR, radar, or sonar. The ranging sensor 3 generates ranging signals that indicate the distance to objects in each direction included in a predetermined ranging range around the vehicle 10 at predetermined intervals. The ranging sensor 3 is preferably attached to the vehicle 10 so that the ranging range of the ranging sensor at least partially overlaps with the shooting area of ​​the camera 2. Note that the vehicle 10 may be provided with multiple ranging sensors with different ranging ranges.

[0019] Every time the distance measurement sensor 3 generates a distance measurement signal, the distance measurement sensor 3 outputs the generated distance measurement signal to the ECU 5 via the in-vehicle network.

[0020] The behavior sensor 4 is a sensor for detecting the behavior of the vehicle 10, and generates a behavior signal that represents a predetermined behavior of the vehicle 10 at predetermined intervals. In this embodiment, the behavior sensor 4 may be a yaw rate sensor for detecting the yaw rate of the vehicle 10. The behavior sensor 4 may be a sensor that can measure not only the yaw rate of the vehicle 10 but also the pitch rate of the vehicle 10, such as a gyro sensor having two or more axes.

[0021] The ECU 5 is configured to perform automatic driving control of the vehicle 10 under predetermined circumstances.

[0022] Fig. 2 is a hardware configuration diagram of an ECU 5, which is an example of a vehicle control device. As shown in Fig. 2, the ECU 5 has a communication interface 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 may each be configured as separate circuits, or may be integrated into a single integrated circuit.

[0023] The communication interface 21 has an interface circuit for connecting the ECU 5 with the camera 2, the distance measurement sensor 3, and the behavior sensor 4. Each time the communication interface 21 receives an image from the camera 2, it passes the received image to the processor 23. Furthermore, each time the communication interface 21 receives a distance measurement signal from the distance measurement sensor 3, it passes the received distance measurement signal to the processor 23. Furthermore, each time the communication interface 21 receives a behavior signal from the behavior sensor 4, it passes the received behavior signal to the processor 23.

[0024] The memory 22 is an example of a storage unit and includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 22 stores various data used in the vehicle control process executed by the processor 23 of the ECU 5. For example, the memory 22 stores parameters representing the focal length, angle of view, shooting direction, mounting position, and shooting range of the camera 2, as well as the ranging range of the ranging sensor 3. The memory 22 also stores a parameter set for identifying an object detection classifier used to detect objects such as obstacles present around the vehicle 10. The memory 22 also temporarily stores sensor signals such as images, ranging signals, and behavior signals. The memory 22 also temporarily stores various data generated during the vehicle control process.

[0025] The processor 23 includes one or more central processing units (CPUs) and their peripheral circuits. The processor 23 may further include other arithmetic circuits such as a logic unit, a numerical calculation unit, or a graphics processing unit. The processor 23 executes vehicle control processing for the vehicle 10.

[0026] 3 is a functional block diagram of the processor 23 related to vehicle control processing. The processor 23 has a detection unit 31, a determination unit 32, and a vehicle control unit 33. Each of these units in the processor 23 is a functional module realized by, for example, a computer program running on the processor 23. Alternatively, each of these units in the processor 23 may be a dedicated arithmetic circuit provided in the processor 23.

[0027] The detection unit 31 detects candidates for objects present on the road surface ahead of the vehicle 10 based on the latest image received by the ECU 5 from the camera 2 at predetermined intervals.

[0028] For example, the detection unit 31 detects candidate objects on the road surface by inputting an image acquired from the camera 2 into a first classifier that has been trained in advance to detect objects on the road surface. In this embodiment, the object on the road surface to be detected is, for example, a three-dimensional structure that should not be present on the road surface, such as a box that has fallen onto the road surface, or a pothole formed in the road surface. Furthermore, the detection unit 31 may use a deep neural network (DNN) with a convolutional neural network (CNN) architecture as the first classifier. More specifically, the first classifier may use a DNN for semantic segmentation that identifies the object represented by each pixel, such as a fully convolutional network (FCN) or U-net. Alternatively, the detection unit 31 may use a classifier based on a machine learning method other than a neural network, such as a random forest, as the first classifier. The first classifier is trained in advance according to a predetermined learning method, such as backpropagation, using a large number of training images representing the object to be detected.

[0029] The detection unit 31 determines a set of pixels output by the first classifier as representing an object on the road surface as an object candidate region representing a candidate object present on the road surface.

[0030] Furthermore, based on the ranging signal, the detection unit 31 determines whether or not a candidate object present on the road surface can be detected at a position in real space corresponding to the object candidate region. In this case, the detection unit 31 may detect the candidate object present on the road surface by inputting the ranging signal to a second classifier that has been trained in advance to detect an object present on the road surface from the ranging signal. The detection unit 31 may use a DNN with a CNN-type or self-attention network-type architecture as the second classifier. Alternatively, the detection unit 31 may detect the candidate object present on the road surface according to another method for detecting an object from a ranging signal.

[0031] Here, each pixel on the image has a one-to-one correspondence with the direction from the camera 2 to the object represented by that pixel. Furthermore, the object represented in the object candidate area is assumed to be located on the road surface. Therefore, the detection unit 31 can estimate the position of the object represented in the object candidate area in real space, based on the position of the camera 2, using parameters such as the installation height, shooting direction, and focal length of the camera 2. Furthermore, the detection unit 31 can estimate the direction of the position of the object represented in the object candidate area in real space as seen from the ranging sensor 3, based on the installation positions of the camera 2 and the ranging sensor 3. Therefore, if the second classifier detects an object on the road surface from the ranging signal in the estimated direction, the detection unit 31 determines that it has detected a candidate object present on the road surface at a position in real space corresponding to the object candidate area. In this case, the detection unit 31 detects the candidate object as a three-dimensional object actually present on the road surface.

[0032] Every time the detection unit 31 detects an object candidate region from the image, it notifies the determination unit 32 of information indicating the position and range of that object candidate region on the image. Furthermore, when the detection unit 31 detects a candidate object present on the road surface from both the image and the ranging signal, it notifies the vehicle control unit 33 that it has detected a three-dimensional object present on the road surface and the position of the object in real space.

[0033] When the vehicle 10 reaches the position of a candidate object on the road surface detected from the image, the determination unit 32 determines whether the direction of the vehicle 10 has deflected by more than a predetermined angle based on the behavior signal obtained by the behavior sensor 4 or the image obtained by the camera 2.

[0034] For example, the determination unit 32 estimates the distance between the vehicle 10 and an object candidate existing on the road surface based on the position of the object candidate area on the image when the object candidate existing on the road surface was last detected from the image. As described in the detection unit 31, each pixel on the image has a one-to-one correspondence with the orientation from the camera 2 to the object represented in that pixel. Furthermore, the object represented in the object candidate area is assumed to be located on the road surface. Therefore, the determination unit 32 can estimate the position of the object represented in the object candidate area in real space based on the position of the camera 2 using parameters such as the installation height, shooting direction, and focal length of the camera 2. Therefore, the determination unit 32 may set the distance to the position of the object candidate existing on the road surface based on the position of the camera 2, which is estimated from the position of the object candidate area on the image when the object candidate existing on the road surface was last detected, as the distance between the vehicle 10 and the object candidate.

[0035] The determination unit 32 estimates the timing at which the vehicle 10 will reach the position of the candidate object by dividing the distance between the candidate object on the road surface and the vehicle 10 by the speed of the vehicle 10 measured by a vehicle speed sensor (not shown) mounted on the vehicle 10. Hereinafter, the estimated timing at which the vehicle 10 will reach the position of the candidate object may be simply referred to as the estimated arrival timing. The determination unit 32 then determines whether the direction of the vehicle 10 has deflected by a predetermined angle or more during a predetermined period (e.g., 1 to 2 seconds) before and after the estimated arrival timing.

[0036] For example, the determination unit 32 determines the amount of change in the orientation of the vehicle 10 in the yaw direction within a predetermined period before and after the estimated arrival timing, based on a plurality of time-series behavior signals received by the ECU 5 from the behavior sensor 4. If the amount of change in the orientation of the vehicle 10 in the yaw direction within the predetermined period is equal to or greater than a predetermined angle, the determination unit 32 determines that the orientation of the vehicle 10 has deflected by equal to or greater than the predetermined angle when the vehicle 10 reaches the candidate position of an object on the road surface detected from the image.

[0037] Furthermore, if the behavior sensor 4 is a sensor capable of detecting pitch rate, the determination unit 32 may determine the amount of change in the orientation of the vehicle 10 in the pitch direction within a predetermined period before and after the estimated arrival timing, based on a plurality of time-series behavior signals. If the amount of change in the orientation of the vehicle 10 in the pitch direction within the predetermined period is equal to or greater than a predetermined angle, the determination unit 32 may determine that the orientation of the vehicle 10 has deflected by equal to or greater than the predetermined angle when the vehicle 10 arrives at the candidate position of an object present on the road surface detected from the image.

[0038] Alternatively, the determination unit 32 may determine the amount of change in the orientation of the vehicle 10 in the pitch direction within a predetermined period before and after the estimated arrival time based on multiple images in time series received by the ECU 5 from the camera 2. In this case, the determination unit 32 determines the vanishing point in each of the multiple images. Furthermore, the determination unit 32 compares the amount of change in the position of the vanishing point in the vertical direction of the image within the predetermined period before and after the estimated arrival time with the number of pixels corresponding to a predetermined angle. If the amount of change in the position of the vanishing point is equal to or greater than the number of pixels corresponding to the predetermined angle, the determination unit 32 determines that the orientation of the vehicle 10 has deflected by more than the predetermined angle when the vehicle 10 arrived at the candidate position of an object on the road surface detected from the image.

[0039] The determination unit 32 detects multiple lane markings displayed in the image to determine the vanishing point in the image. To do so, the determination unit 32 detects multiple lane markings by inputting the image into a third classifier that has been trained in advance to detect lane markings. In this case, a classifier similar to the first classifier is used as the third classifier. The determination unit 32 performs a labeling process on the sets of pixels representing lane markings output by the third classifier, thereby determining each set of adjacent pixels as an object region representing a single lane marking. The determination unit 32 determines lines that approximate the lane markings for each object region. In this case, the determination unit 32 may determine lines that approximate the lane markings for each object region so as to minimize the sum of squares of the distances to each pixel included in that object region. The determination unit 32 then determines the intersections of the lines approximating each of the multiple lane markings as vanishing points. In addition, if three or more lane markings are detected and the lines approximating each of them do not intersect at a single point, the determination unit 32 may determine the vanishing point to be the position where the sum of the distances to each approximation line is the smallest.

[0040] The first classifier used by the detection unit 31 may be trained in advance to identify not only object candidates present on the road surface but also lane markings. In this case, the determination unit 32 simply receives, for each image, information from the detection unit 31 indicating a set of pixels representing lane markings.

[0041] Alternatively, the determination unit 32 may determine the amount of change in the orientation of the vehicle 10 in the yaw or pitch direction within a predetermined period before and after the estimated arrival time based on multiple time-series distance measurement signals received by the ECU 5 from the distance measurement sensor 3.

[0042] In this case, the determination unit 32 calculates the cross-correlation value between two consecutive ranging signals while shifting them in the yaw or pitch direction. To avoid being affected by other vehicles traveling around the vehicle 10, the determination unit 32 may use only measurement points in the ranging signals whose distances are equal to or greater than a predetermined distance to calculate the cross-correlation value. The determination unit 32 then determines the angle in the yaw or pitch direction at which the cross-correlation value between two consecutive ranging signals is greatest as the amount of change in the orientation of the vehicle 10 in the yaw or pitch direction between the times at which the two ranging signals were generated. The determination unit 32 may determine the sum of the amounts of change in the orientation of the vehicle 10 calculated between two consecutive ranging signals over a predetermined period as the amount of change in the orientation of the vehicle 10 when the vehicle 10 reaches the candidate position of an object on the road surface.

[0043] If the orientation of the vehicle 10 is deflected by a predetermined angle or more when the vehicle 10 reaches the position of a candidate object on the road surface, there is a high possibility that the candidate object is an object that actually exists on the road surface and is tall. It is then estimated that the orientation of the vehicle 10 has changed due to the vehicle 10 coming into contact with the object. Therefore, when the determination unit 32 determines that the orientation of the vehicle 10 has deflected by a predetermined angle or more when the vehicle 10 reaches the position of a candidate object on the road surface detected from the image, it notifies the vehicle control unit 33 of the determination result.

[0044] When the vehicle control unit 33 is notified by the determination unit 32 of the determination result that the orientation of the vehicle 10 has deflected by a predetermined angle or more when the vehicle 10 reaches the position of a candidate object on the road surface detected from the image, the vehicle control unit 33 controls each unit of the vehicle 10 to decelerate the vehicle 10 at a predetermined deceleration. In other words, if a candidate object on the road surface is detected from the image but the candidate object is not detected from the ranging signal, and the orientation of the vehicle 10 has deflected by a predetermined angle or more when the vehicle 10 reaches the position of the candidate object, the vehicle control unit 33 decelerates the vehicle 10.

[0045] The vehicle control unit 33 sets the accelerator opening or braking amount so as to achieve the set deceleration. The vehicle control unit 33 then calculates the fuel injection amount according to the set accelerator opening and outputs a control signal corresponding to the fuel injection amount to a fuel injection device of the engine of the vehicle 10. Alternatively, the vehicle control unit 33 controls a power supply device to a motor for driving the vehicle 10 so as to supply power corresponding to the set accelerator opening to the motor. Alternatively, the vehicle control unit 33 outputs a control signal corresponding to the set braking amount to the brake of the vehicle 10.

[0046] Furthermore, when the vehicle control unit 33 is notified by the detection unit 31 that a three-dimensional object present on the road surface has been detected, the vehicle control unit 33 may control each unit of the vehicle 10 to avoid the position of the object. This prevents the vehicle 10 from coming into contact with the object. In this case, the vehicle control unit 33 sets a planned driving route for the vehicle 10 so that the vehicle 10 is at least a predetermined distance away from the position of the object in real space notified by the detection unit 31. The vehicle control unit 33 then controls each unit of the vehicle 10 so that the vehicle 10 drives along the planned driving route. For example, the vehicle control unit 33 calculates a steering angle of the vehicle 10 for driving the vehicle 10 along the planned driving route based on the planned driving route and the current position of the vehicle 10, and outputs a control signal corresponding to the steering angle to an actuator (not shown) that controls the steering wheels of the vehicle 10. The vehicle control unit 33 determines the latest position of the vehicle 10 measured by a GPS receiver (not shown) mounted on the vehicle 10 as the current position of the vehicle 10. Alternatively, the vehicle control unit 33 may determine the current position of the vehicle 10 by using the position of the vehicle 10 when the planned travel route was set as a reference and determining the amount of movement and change in the traveling direction of the vehicle 10 based on the acceleration and angular velocity of the vehicle 10 thereafter. The acceleration and angular velocity of the vehicle 10 are measured by an acceleration sensor and a gyro sensor mounted on the vehicle 10, respectively.

[0047] Furthermore, the vehicle control unit 33 may detect other objects that may obstruct the travel of the vehicle 10, such as other vehicles, pedestrians, or guardrails, that are present around the vehicle 10, based on the images received from the camera 2 or the distance measurement signals received from the distance measurement sensor 3. The vehicle control unit 33 may detect other objects by inputting the images or distance measurement signals to a classifier that has been trained in advance to detect such objects. In this case, the vehicle control unit 33 sets the planned travel trajectory so as to keep the vehicle 10 at a predetermined distance or more from the detected other objects.

[0048] Furthermore, if it is not possible to set a planned driving route that keeps the vehicle 10 at a predetermined distance or more from the detected object, the vehicle control unit 33 may control the vehicle 10 to stop in front of a three-dimensional object on the road surface. The vehicle control unit 33 may then notify the driver that the vehicle will stop to avoid a collision via a notification device such as a display device or speaker provided in the vehicle cabin.

[0049] 4 is a diagram illustrating an example of vehicle control processing according to this embodiment. In this example, a candidate object 401 present on the road surface at position P2 ahead of the vehicle 10 is detected from an image 400 generated by the camera 2 when the vehicle 10 is located at position P1. However, no candidate object present on the road surface at position P2 is detected from the ranging signal. When the vehicle 10 reaches position P2 where the candidate object 401 is present, the deflection angle α of the direction of the vehicle 10, indicated by arrow 410, is equal to or greater than a predetermined angle. Therefore, the candidate object 401 present on the road surface is estimated to be a three-dimensional object actually present on the road surface, and the vehicle 10 is controlled to decelerate.

[0050] FIG. 5 is a diagram illustrating another example of vehicle control processing according to this embodiment. In this example, similar to the example shown in FIG. 4, a candidate object 501 present on the road surface at position P2 ahead of the vehicle 10 is detected from an image 500 generated by the camera 2 when the vehicle 10 is located at position P1. On the other hand, no candidate object present on the road surface at position P2 is detected from the ranging signal. In this example, even when the vehicle 10 reaches position P2 where the candidate object 501 is present, the orientation of the vehicle 10 indicated by the arrow 510 does not change. Therefore, it is estimated that the candidate object 501 present on the road surface is actually dirt on the road surface or a marking drawn on the road surface. Therefore, in this example, the vehicle 10 is not subjected to deceleration control, and the speed of the vehicle 10 is maintained.

[0051] 6 is a diagram illustrating yet another example of the vehicle control process according to this embodiment. In this example, as in the example shown in FIG. 4, an object candidate 601 present on the road surface at position P2 ahead of the vehicle 10 is detected from an image 600 generated by the camera 2 when the vehicle 10 is located at position P1. Furthermore, an object candidate present on the road surface at position P2 is also detected from the ranging signal. Therefore, the vehicle 10 is controlled so that the vehicle 10 travels along a route indicated by an arrow 602 that avoids position P2 of the object candidate 601.

[0052] 7 is an operational flowchart of the vehicle control process executed by the processor 23. The processor 23 executes the vehicle control process at predetermined intervals in accordance with the following operational flowchart.

[0053] The detection unit 31 of the processor 23 detects a candidate object present on the road surface ahead of the vehicle 10 based on the image obtained by the camera 2 (step S101). When the candidate object is detected, the detection unit 31 determines, based on the ranging signal, whether or not the candidate object present on the road surface can be detected at a position in real space corresponding to the candidate object area in which the candidate object is displayed (step S102). When the candidate object present on the road surface is also detected from the ranging signal (step S102-Yes), the candidate object is estimated to be a three-dimensional object actually present on the road surface. Therefore, the vehicle control unit 33 of the processor 23 controls the vehicle 10 to avoid the estimated position of the object in real space (step S103).

[0054] On the other hand, if no candidate object on the road surface is detected from the ranging signal (step S102—No), the determination unit 32 of the processor 23 determines whether the orientation of the vehicle 10 has deviated by a predetermined angle or more when the vehicle 10 reaches the position of the candidate object (step S104). If the orientation of the vehicle 10 has deviated by a predetermined angle or more (step S104—Yes), it is highly likely that the vehicle 10 has come into contact with an actual object on the road surface that corresponds to the candidate object. Therefore, the vehicle control unit 33 decelerates the vehicle 10 (step S105). On the other hand, if the change in the orientation of the vehicle 10 is less than the predetermined angle (step S104—No), it is highly likely that the candidate object is dirt on the road surface or a sign drawn on the road surface. Therefore, the vehicle control unit 33 maintains the speed of the vehicle 10 (step S106). Note that instead of maintaining the speed of the vehicle 10, the vehicle control unit 33 may continue the control of the vehicle 10 that was being performed immediately before the vehicle 10 reached the position of the candidate object. For example, if the vehicle 10 is accelerating immediately before reaching the position of the candidate object, the vehicle control unit 33 may continue accelerating the vehicle 10 in step S106.

[0055] After step S103, S105, or S106, processor 23 ends the vehicle control process.

[0056] As described above, the vehicle control device detects candidate objects on the road surface ahead of the vehicle from an image representing the vehicle's surroundings generated by an imaging unit installed in the vehicle. Furthermore, when the vehicle reaches the position of the detected candidate object, the vehicle control device determines whether the vehicle's orientation has deviated by a predetermined angle or more based on a sensor signal or image obtained by a behavior sensor that detects the vehicle's behavior. If the vehicle's orientation has deviated by the predetermined angle or more, the vehicle control device controls the vehicle to decelerate. This prevents the vehicle from getting into a dangerous situation and safely controls the vehicle, even when a low obstacle that is difficult to detect using a distance measuring sensor installed in the vehicle is present in the vehicle's path. Furthermore, even if dirt on the road surface displayed in the image representing the vehicle's surroundings is erroneously detected as a low obstacle, the vehicle control device can prevent the vehicle from performing unnecessary avoidance maneuvers. Therefore, the vehicle control device can prevent vehicle control that causes the driver to feel uncomfortable.

[0057] According to a modified example, when a candidate object on the road surface is detected, the vehicle control unit 33 may increase the hydraulic pressure of the brakes of the vehicle 10 before the vehicle 10 reaches the position of the candidate object. In this way, even if the candidate object is a three-dimensional object that actually exists on the road surface, the vehicle control unit 33 can apply the brakes immediately when the vehicle 10 comes into contact with the object.

[0058] In addition, the vehicle control unit 33 may increase the hydraulic pressure of the brakes of the vehicle 10 before the vehicle 10 reaches the candidate position of an object on the road surface only if either of the following two conditions is met: (i) The width of the shoulder of the road on which the vehicle 10 is traveling is less than a predetermined width. (ii) Detecting other vehicles traveling in adjacent lanes adjacent to the lane in which the vehicle 10 is traveling. If the shoulder of the road on which the vehicle 10 is traveling is narrow, or if another vehicle is traveling in an adjacent lane to the lane on which the vehicle 10 is traveling, contact between the vehicle 10 and an object on the road surface may cause the vehicle 10 to deflect, thereby compromising the safety of the vehicle 10. Therefore, by increasing the brake hydraulic pressure in advance, the vehicle control unit 33 can prevent the safety of the vehicle 10 from being compromised.

[0059] The vehicle control unit 33 may detect other vehicles traveling in adjacent lanes by inputting the image obtained by the camera 2 or the distance measurement signal obtained by the distance measurement sensor 3 into a classifier that has been trained in advance to detect other vehicles. In this case, the vehicle control unit 33 may determine whether the other vehicle is traveling in an adjacent lane based on the direction of the detected other vehicle and, if detected from the distance measurement signal, the distance to the other vehicle. The vehicle control unit 33 may also identify the width of the shoulder of the road on which the vehicle 10 is traveling by referring to the position of the vehicle 10 measured by a GPS receiver mounted on the vehicle 10 and the map information stored in the memory 22.

[0060] According to yet another modification, the vehicle control device according to the present disclosure may be applied to a vehicle that is not equipped with a distance measurement sensor. In this case, the processes of steps S102 and S103 in the flowchart of Fig. 7 are omitted. That is, if a candidate object present on the road surface ahead of the vehicle 10 is detected based on the image from the camera 2, and if it is determined that the orientation of the vehicle 10 has deflected by a predetermined angle or more when the vehicle 10 reaches the position of the candidate object, the vehicle control unit 33 decelerates the vehicle 10.

[0061] A computer program that realizes the functions of the processor 23 of the ECU 5 according to the above embodiment or variant may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium or an optical recording medium.

[0062] As described above, those skilled in the art can make various modifications to the embodiments within the scope of the present invention. [Explanation of symbols]

[0063] 1. Vehicle control system 10 vehicles 2 Cameras 3. Distance sensor 4. Behavior Sensor 5 Electronic Control Unit (ECU) 21 Communication Interface 22 Memory 23 processors 31 Detector 32 Judgment section 33 Vehicle control unit

Claims

1. a detection unit that detects candidate objects present on a road surface ahead of the vehicle from an image representing the surroundings of the vehicle generated by an imaging unit provided in the vehicle; a determination unit that determines whether or not the orientation of the vehicle has deflected by a predetermined angle or more when the vehicle reaches the position of the detected candidate object; a vehicle control unit that controls the vehicle to decelerate when the orientation of the vehicle is deflected by more than the predetermined angle; and the detection unit determines whether the object candidate can be further detected from a ranging signal generated by a ranging sensor mounted on the vehicle; The vehicle control unit controls the vehicle to decelerate if the direction of the vehicle is deflected by more than the predetermined angle when the object candidate cannot be detected from the ranging signal. Vehicle control device.

2. The vehicle control device according to claim 1 , wherein, when the object candidate is also detected from the ranging signal, the vehicle control unit controls the vehicle so that the vehicle avoids a position of the object candidate.

3. detecting candidate objects present on a road surface ahead of the vehicle from an image representing the surroundings of the vehicle generated by an imaging unit provided in the vehicle; When the vehicle reaches the position of the detected candidate object, it is determined whether the orientation of the vehicle has deflected by a predetermined angle or more; When the direction of the vehicle is deflected by the predetermined angle or more, the vehicle is controlled to decelerate. This includes: Detecting the object candidate includes determining whether the object candidate can be further detected from a ranging signal generated by a ranging sensor mounted on the vehicle; controlling the vehicle to decelerate includes controlling the vehicle to decelerate when the direction of the vehicle has deviated by the predetermined angle or more in a case where the object candidate cannot be detected from the ranging signal. Vehicle control method.

4. detecting candidate objects present on a road surface ahead of the vehicle from an image representing the surroundings of the vehicle generated by an imaging unit provided in the vehicle; When the vehicle reaches the position of the detected candidate object, it is determined whether the orientation of the vehicle has deflected by a predetermined angle or more; When the direction of the vehicle is deflected by the predetermined angle or more, the vehicle is controlled to decelerate. causing a processor mounted on the vehicle to execute the above steps; Detecting the object candidate includes determining whether the object candidate can be further detected from a ranging signal generated by a ranging sensor mounted on the vehicle; controlling the vehicle to decelerate includes controlling the vehicle to decelerate when the direction of the vehicle has deviated by the predetermined angle or more in a case where the object candidate cannot be detected from the ranging signal. A computer program for vehicle control.

Images