Vehicle control device, vehicle control method, and vehicle control computer program

Abstract

To provide a vehicle control device capable of controlling traveling of a vehicle so as to appropriately avoid an obstacle present on a route of the vehicle.SOLUTION: A vehicle control device comprises: a preceding vehicle detection unit 31 for detecting a preceding vehicle traveling ahead of a vehicle 10 from a series of time-series sensor signals obtained by a sensor 2 provided on the vehicle 10 for sensing objects around the vehicle 10; a tracking unit 32 for tracking the preceding vehicle detected from the series of sensor signals; a determination unit 34 for determining whether the preceding vehicle has performed an avoidance action, on the basis of a result of the tracking; a trajectory detection unit 35 for detecting, when the preceding vehicle has performed the avoidance action, a trajectory of an end of the preceding vehicle opposite to an avoidance direction during the avoidance action; and a control unit 36 for controlling traveling of the vehicle 10 so that an end of the vehicle 10 opposite to the avoidance direction moves along the detected trajectory.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device for controlling the running of a vehicle, a vehicle control method, and a computer program for vehicle control. [Background technology]

[0002] When an autonomously driven vehicle encounters a fallen object in its path or a road surface damage such as a pothole in its path, it is necessary to control the vehicle to avoid the object to prevent an accident. However, obstacles such as fallen objects and road damage, whose shape, color, and size are not specified, may not be accurately detected from sensor signals representing the vehicle's surroundings, such as images of the vehicle's surroundings taken by an onboard camera. As a result, it may be difficult to control the vehicle to avoid the obstacle. Therefore, technologies have been proposed that control the vehicle to avoid the obstacle based on the behavior of other vehicles traveling ahead of the vehicle (see, for example, Patent Documents 1 and 2).

[0003] The driving support device disclosed in Patent Document 1 calculates the amount of lateral movement of a preceding vehicle based on the behavior of the preceding vehicle traveling on the target traveling path of the host vehicle. If the amount of lateral movement exceeds a predetermined threshold, the driving support device determines that the preceding vehicle has taken obstacle avoidance action, and sets the traveling path of the preceding vehicle as the target traveling path of the host vehicle.

[0004] The driving assistance device disclosed in Patent Document 2 calculates the driving route of a preceding vehicle and stores the calculated driving route. The driving assistance device also stores, as an initial lateral displacement amount, the lateral displacement amount of the preceding vehicle from the center position of the lane when the lateral acceleration of the preceding vehicle exceeds a lateral acceleration threshold. Furthermore, the driving assistance device determines that the preceding vehicle has performed obstacle avoidance if the difference between the initial lateral displacement amount and the lateral displacement amount after a predetermined time required to perform the avoidance is within a predetermined displacement difference that allows the initial lateral displacement amount and the lateral displacement amount after the predetermined time to be determined to be the same. When the driving assistance device determines that the preceding vehicle has performed obstacle avoidance, it retrieves the driving route of the preceding vehicle during the predetermined time from a memory unit and performs driving control to follow the retrieved driving route. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2018-171959 [Patent Document 2] Japanese Patent Application Publication No. 2017-13678 Summary of the Invention [Problem to be solved by the invention]

[0006] If the size of the vehicle itself differs from the size of the preceding vehicle, even if the vehicle control device controls the vehicle itself so that it travels along the traveling path of the preceding vehicle when avoiding an obstacle, there is a risk that the vehicle itself will not be able to completely avoid the obstacle.

[0007] Therefore, an object of the present invention is to provide a vehicle control device that can control the traveling of a vehicle so as to appropriately avoid obstacles that exist on the path of the vehicle. [Means for solving the problem]

[0008] According to one embodiment, there is provided a vehicle control device that includes: a leading vehicle detection unit that detects a leading vehicle traveling ahead of the vehicle from a series of time-series sensor signals obtained by a sensor provided on the vehicle for detecting objects around the vehicle, a tracking unit that tracks the leading vehicle detected from the series of sensor signals, a determination unit that determines whether the leading vehicle has taken evasive action based on the results of tracking the leading vehicle, a trajectory detection unit that, when the leading vehicle has taken evasive action, detects a trajectory of an end of the leading vehicle on the side opposite to the avoidance direction during the evasive action, and a control unit that controls driving of the vehicle so that the end of the vehicle on the side opposite to the avoidance direction moves along the detected trajectory.

[0009] Preferably, the vehicle control device further includes a lighting state detection unit that detects the lighting state of the turn signal of the leading vehicle based on a series of time-series images obtained by an imaging unit that captures images of the surroundings of the vehicle. The control unit then determines, based on the lighting state of the turn signal of the leading vehicle, whether or not the leading vehicle has turned on the turn signal on the side of the evasive direction before taking evasive action, and preferably causes the vehicle to travel along the lane in which the vehicle is traveling if the turn signal on the side of the evasive direction has been turned on before taking evasive action.

[0010] In addition, in this vehicle control device, it is preferable that the preceding vehicle detection unit detects multiple preceding vehicles, the judgment unit judges whether or not each of the multiple preceding vehicles has taken evasive action, and identifies the point at which the preceding vehicle immediately preceding the vehicle among the multiple preceding vehicles started evasive action, and the control unit transfers control of the vehicle to the driver of the vehicle if each of the multiple preceding vehicles has taken evasive action and the vehicle can be stopped by the point at which the preceding vehicle immediately preceding the vehicle started evasive action.

[0011] According to another embodiment, a vehicle control method is applied, which includes detecting a leading vehicle traveling ahead of the vehicle from a series of time-series sensor signals obtained by a sensor provided on the vehicle for detecting objects around the vehicle, tracking the leading vehicle detected from the series of sensor signals, determining whether the leading vehicle has taken evasive action based on the tracking result, detecting a trajectory of an end of the leading vehicle on a side opposite to the avoidance direction during the evasive action, and controlling travel of the vehicle so that the end of the vehicle on the side opposite to the avoidance direction moves along the detected trajectory.

[0012] According to yet another embodiment, a vehicle control computer program is applied, the vehicle control computer program including instructions to cause a processor mounted on the vehicle to execute the following steps: detect a leading vehicle traveling ahead of the vehicle from a series of time-series sensor signals obtained by a sensor provided on the vehicle for detecting objects around the vehicle, track the leading vehicle detected from the series of sensor signals, determine whether the leading vehicle has taken evasive action based on the tracking result, and, if the leading vehicle has taken evasive action, detect a trajectory of an end of the leading vehicle on the opposite side to the evasive direction during the evasive action, and control traveling of the vehicle so that the end of the vehicle on the opposite side to the evasive direction moves along the detected trajectory. [Effects of the Invention]

[0013] The vehicle control device according to the present invention has the effect of being able to control the traveling of a vehicle so as to appropriately avoid obstacles that exist on the path of the vehicle. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a schematic configuration diagram of a vehicle control system in which a vehicle control device is implemented. [Figure 2] 1 is a hardware configuration diagram of an electronic control device that is one embodiment of a vehicle control device. [Figure 3] FIG. 2 is a functional block diagram of a processor of an electronic control unit related to vehicle control processing. [Figure 4] 10 is a diagram showing an example of the relationship between the trajectory of the end of the preceding vehicle immediately ahead of the vehicle on the opposite side to the avoidance direction and the planned travel route of the vehicle; FIG. [Figure 5] 4 is an operational flowchart of a vehicle control process. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, with reference to the drawings, a vehicle control device, and a vehicle control process and a vehicle control computer program executed on the vehicle control device will be described. The vehicle control device detects another vehicle (hereinafter referred to as a leading vehicle) traveling ahead of the host vehicle from a series of time-series sensor signals obtained by a sensor provided on the host vehicle for detecting objects around the host vehicle. Furthermore, the vehicle control device tracks the detected leading vehicle to determine whether the leading vehicle has taken action to avoid an obstacle (hereinafter simply referred to as an avoidance action). When the leading vehicle has taken an avoidance action, the vehicle control device detects the trajectory of the edge of the leading vehicle on the opposite side to the direction to which the leading vehicle will move (hereinafter referred to as the avoidance direction) during the evasive action. Then, the vehicle control device controls the traveling of the host vehicle so that the edge of the host vehicle on the opposite side to the avoidance direction moves along the detected trajectory.

[0016] FIG. 1 is a schematic configuration diagram of a vehicle control system in which a vehicle control device is implemented. FIG. 2 is a hardware configuration diagram of an electronic control device, which is one embodiment of the vehicle control device. In this embodiment, the vehicle control system 1 is mounted on a vehicle 10 and controls the vehicle 10. The vehicle control system 1 includes a camera 2 and an electronic control unit (ECU) 3, which is an example of a vehicle control device. The camera 2 and the ECU 3 are communicatively connected via an in-vehicle network conforming to a standard such as a controller area network. The vehicle control system 1 may further include a storage device (not shown) that stores a map used for autonomous driving control of the vehicle 10. The vehicle control system 1 may further include a ranging sensor (not shown) such as a LiDAR or radar, and a receiver (not shown) such as a GPS receiver for determining the self-position of the vehicle 10 in accordance with a satellite positioning system. The vehicle control system 1 may further include a wireless terminal (not shown) for wireless communication with other devices, and a navigation device (not shown) for searching for a planned driving route of the vehicle 10.

[0017] Camera 2 is an example of a sensor for detecting objects around vehicle 10, and includes a two-dimensional detector configured with an array of photoelectric conversion elements, such as a CCD or C-MOS, that are sensitive to visible light, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. Camera 2 is mounted, for example, inside the passenger compartment of vehicle 10 so as to face forward of vehicle 10. Camera 2 photographs the area ahead of vehicle 10 at predetermined photographing intervals (for example, 1 / 30 to 1 / 10 seconds) and generates an image of the area ahead. The image obtained by camera 2 may be a color image or a gray image. The image generated by camera 2 is an example of a sensor signal. Vehicle 10 may also be provided with multiple cameras with different photographing directions or focal lengths.

[0018] Every time the camera 2 generates an image, it outputs the generated image to the ECU 3 via the in-vehicle network.

[0019] The ECU 3 controls the vehicle 10. In this embodiment, the ECU 3 controls the vehicle 10 so that the vehicle 10 automatically drives based on the avoidance behavior of a preceding vehicle detected from a series of time-series images acquired by the camera 2. To this end, the ECU 3 has a communication interface 21, a memory 22, and a processor 23.

[0020] The communication interface 21 is an example of a communication unit, and has an interface circuit for connecting the ECU 3 to an in-vehicle network. That is, the communication interface 21 is connected to the camera 2 via the in-vehicle network. Every time the communication interface 21 receives an image from the camera 2, it passes the received image to the processor 23.

[0021] The memory 22 is an example of a storage unit and includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 22 stores various data used in the vehicle control process executed by the processor 23 of the ECU 3, such as images received from the camera 2 and various parameters for identifying classifiers used in the vehicle control process. The memory 22 may also store map information representing features related to vehicle travel, such as road markings or road signs, such as lane markings.

[0022] The processor 23 is an example of a control unit and includes one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 23 may further include other arithmetic circuits such as a logic operation unit, a numerical operation unit, or a graphics processing unit. While the vehicle 10 is traveling, the processor 23 executes vehicle control processing on the received image every time it receives an image from the camera 2. The processor 23 controls the vehicle 10 to drive automatically based on the evasive action of a leading vehicle detected from the image.

[0023] 3 is a functional block diagram of the processor 23 of the ECU 3, which is related to vehicle control processing. The processor 23 includes a preceding vehicle detection unit 31, a tracking unit 32, a lighting state detection unit 33, a determination unit 34, a trajectory detection unit 35, and a vehicle control unit 36. Each of these units included in the processor 23 is a functional module implemented by a computer program running on the processor 23. Alternatively, each of these units included in the processor 23 may be a dedicated arithmetic circuit provided in the processor 23.

[0024] Every time an image is received from the camera 2, the leading vehicle detection unit 31 detects one or more leading vehicles from the received image. In this embodiment, the leading vehicle detection unit 31 inputs the latest image to a first classifier, thereby detecting other vehicles traveling around the vehicle 10 shown in the image, and identifying the leading vehicle traveling ahead of the vehicle 10 from among the other detected vehicles.

[0025] The leading vehicle detection unit 31 uses, as a first classifier, a DNN that has been trained in advance to detect an object region including a detection target object (in this embodiment, other vehicles traveling around the vehicle 10) shown in an image and to identify the type of the detection target object. The DNN used as the first classifier can be, for example, a DNN having a convolutional neural network (hereinafter simply referred to as CNN) type architecture, such as a Single Shot MultiBox Detector (SSD) or Faster R-CNN.

[0026] The first classifier is trained in advance to detect the target object according to a predetermined learning method such as backpropagation using training data including a plurality of images showing the target object. By using the trained first classifier in this way, the leading vehicle detection unit 31 can accurately detect the target object from the image.

[0027] The detection target object may include an object that affects the driving control of the vehicle 10 other than other vehicles around the vehicle 10. Such objects include, for example, people, road signs, traffic lights, road markings such as lane markings, and other objects on the road. In this case, the first classifier may be trained in advance to detect these objects as well. The leading vehicle detection unit 31 can then detect these objects by inputting an image to the first classifier.

[0028] The preceding vehicle detection unit 31 identifies, among the other detected vehicles, one or more vehicles that are traveling ahead of the vehicle 10 and in the same lane as the lane in which the vehicle 10 is traveling (hereinafter referred to as the own lane) as preceding vehicles. In this embodiment, the camera 2 is mounted to capture an image of the area ahead of the vehicle 10, so a predetermined range including the horizontal center of the image is identified as the range corresponding to the own lane. Alternatively, the preceding vehicle detection unit 31 may identify, as the range corresponding to the own lane, an area on the image sandwiched between two lane markings that are detected on the image and that separate the own lane from adjacent lanes on the left and right. The preceding vehicle detection unit 31 may then identify, as preceding vehicles, other vehicles located within the range corresponding to the own lane on the image.

[0029] The preceding vehicle detection unit 31 notifies the tracking unit 32 of the position and range on the image of the object region in which the detected preceding vehicle is represented.

[0030] The tracking unit 32 tracks each preceding vehicle detected from the latest image by associating it with a preceding vehicle detected from a past image.

[0031] The tracking unit 32 tracks the leading vehicle of interest by applying a tracking process based on optical flow, such as the Lucas-Kanade algorithm, to an object region representing the leading vehicle of interest in the latest image and an object region representing the leading vehicle in the past image. To this end, the tracking unit 32 extracts multiple feature points from the object region representing the leading vehicle of interest by applying a feature point extraction filter, such as SIFT or a Harris operator, to the object region representing the leading vehicle of interest. The tracking unit 32 then calculates the optical flow by identifying corresponding points in the object region representing the leading vehicle of interest in the past image according to the applied tracking method. Alternatively, the tracking unit 32 may track the leading vehicle of interest by applying another tracking method applied to tracking moving objects detected from images to the object region representing the leading vehicle of interest in the latest image and the object region representing the leading vehicle in the past image.

[0032] The tracking unit 32 notifies the lighting state detection unit 33, the determination unit 34, and the trajectory detection unit 35 of the tracking results of each preceding vehicle.

[0033] The lighting state detection unit 33 detects the lighting state of the turn signal of each preceding vehicle being tracked. Since the lighting state detection unit 33 only needs to perform the same process for each preceding vehicle, the following describes the process for one preceding vehicle.

[0034] The lighting state detection unit 33 inputs features determined from pixel values ​​in an object region including the preceding vehicle being tracked to a second classifier that performs a convolution operation in the time axis direction. As a result, the lighting state detection unit 33 calculates, for each of the left and right turn signals of the preceding vehicle, a certainty factor that the turn signal is in a flashing state, and a certainty factor that neither the left nor the right turn signal is flashing. The lighting state detection unit 33 then detects the state with the highest certainty factor as the current lighting state of the turn signal of the preceding vehicle.

[0035] The lighting state detection unit 33 can use, for example, features included in the object region of the feature map calculated by the convolutional layer of the first classifier as features determined from pixel values ​​in the object region representing the preceding vehicle. For example, if the resolution of the feature map is the same as the resolution of the image input to the first classifier, each feature included in the region on the feature map corresponding to the object region on the image becomes a feature determined from pixel values ​​in the object region. Also, if the resolution of the feature map is lower than the resolution of the image input to the first classifier, the position and range obtained by correcting the coordinates of the object region according to the ratio of the resolution of the feature map to the resolution of the input image becomes the region on the feature map corresponding to the object region. Then, each feature included in the region on the feature map becomes a feature determined from pixel values ​​in the object region. For example, suppose the upper left and upper right corner positions of the object region on an input image are (tlX, tlY) and (brX, brY), respectively, and the feature map is calculated by downsizing the input image by 1 / N (N is an integer greater than or equal to 2). In this case, the upper left and lower right corner positions of the region on the feature map corresponding to the object region on the image are (tlX / N, tlY / N) and (brX / N, brY / N), respectively.

[0036] According to a modified example, the lighting state detection unit 33 may be characterized in that it inputs the values ​​of each pixel in the object area representing the preceding vehicle on the image input to the first classifier to the second classifier.

[0037] The lighting state detection unit 33 resizes the extracted features of the object region representing the preceding vehicle to a predetermined size (for example, 32 × 32) by downsampling or upsampling. As a result, even if the relative distance between vehicle 10 and the preceding vehicle changes during tracking of the preceding vehicle and the size of the preceding vehicle on the image changes, the second classifier can treat the input features as having a constant size, thereby simplifying the configuration of the second classifier.

[0038] The lighting state detection unit 33 inputs features determined from pixel values ​​in an object region including a preceding vehicle for a series of time-series images included in a recent predetermined period into a second classifier that performs a convolution operation in the time axis direction in chronological order, thereby calculating a certainty factor for each possible lighting state of the turn signal of the preceding vehicle.

[0039] The length of the predetermined period is preferably approximately the same as the blinking cycle of the turn signal. This is because, generally, when a turn signal is turned on, the turn signal repeatedly turns on and off at a predetermined blinking cycle. Therefore, by inputting features obtained from each image during a period having a length equal to or longer than the blinking cycle into the second classifier, the second classifier can accurately calculate the confidence level regarding the turn signal's on state. On the other hand, as the predetermined period becomes longer, the number of features input to the second classifier increases, and the amount of calculation by the second classifier also increases. Therefore, the amount of hardware resources required to perform calculations by the second classifier also increases. Thus, from the perspective of the required hardware resources, it is preferable that the predetermined period be short. For this reason, as described above, the length of the predetermined period is preferably approximately the same as the blinking cycle of the turn signal. However, if the ECU 3 has sufficient hardware resources, the length of the predetermined period may be longer than the blinking cycle of the turn signal. In this case, for example, when designing and training the second classifier, the length of the predetermined period may be set so as to achieve the best trade-off between the required hardware resources and the accuracy of classifying the illumination state of the turn signal.

[0040] The lighting state detection unit 33 may use a neural network with a CNN-type architecture as a second classifier that performs a convolution operation in the time axis direction. In this case, the second classifier may have, for example, one or more convolution layers (hereinafter referred to as "temporal feature convolution layers") that perform a convolution operation in the time axis direction on the feature map output from the previous layer. The kernel size in the time axis direction of each temporal feature convolution layer is set, for example, so that the convolution operation is performed over the entire predetermined period that includes multiple features input at one time by passing through all temporal feature convolution layers. The temporal feature convolution layer may perform a convolution operation in both the time axis direction and the spatial direction (hereinafter referred to as "three-dimensional convolution operation"), or may perform a convolution operation only in the time axis direction (hereinafter referred to as "temporal dimension convolution operation"). Furthermore, the temporal feature convolution layer may also perform a convolution operation or a fully connected operation in the channel direction. Furthermore, if the second classifier has multiple temporal feature convolution layers, one of the multiple temporal feature convolution layers may perform a 3D convolution operation, and another of the multiple temporal feature convolution layers may perform a temporal dimension convolution operation. Furthermore, the second classifier may have one or more convolution layers (hereinafter referred to as spatial feature convolution layers) that perform a convolution operation in the spatial direction without performing a convolution operation in the time axis direction. This spatial feature convolution layer may also perform a convolution operation or a fully connected operation in the channel direction. If the second classifier has one or more spatial feature convolution layers, the spatial feature convolution layer and the temporal feature convolution layer may be arranged in any order. For example, from the input side to the output side, the spatial feature convolution layer and the temporal feature convolution layer may be arranged in this order, or vice versa. Furthermore, the spatial feature convolution layer and the temporal feature convolution layer may be arranged alternately. Furthermore, the second classifier may have one or more pooling layers. Furthermore, the second classifier may have one or more activation layers and one or more fully connected layers. The output layer of the second classifier uses, for example, a sigmoid function or a softmax function as an activation function.The output layer of the second classifier then outputs the confidence level for each possible lighting state of the turn signal.

[0041] The training data used to train the second classifier includes, for example, a combination of a set of features obtained from an object region including a vehicle for each of the time-series series of images over the predetermined period, and a label indicating the lighting state of the turn signal corresponding to the set of features. By using a large amount of such training data to train the second classifier according to the backpropagation method, the second classifier can accurately calculate the confidence level for each possible lighting state of the turn signal.

[0042] Since the second classifier only needs to process features included in the object region, the input layer and hidden layer can be smaller in size than the first classifier, and the number of parameters defining the second classifier, such as weighting coefficients, can be reduced. Therefore, the second classifier requires less computational effort than the first classifier, and the computational load on the processor 23 can be reduced. Furthermore, the computational effort required for training the second classifier is also reduced. Note that when the first classifier and the second classifier are each configured as neural networks, these neural networks may be trained together using common training data by backpropagation.

[0043] Furthermore, the lighting state detection unit 33 may use a neural network with a recursive structure, such as a recurrent neural network (RNN), a long short-term memory (LSTM), or a gated recurrent unit (GRU), as the second classifier. In such a neural network with a recursive structure, the calculation results calculated from previously input data are stored as internal states, and the calculation results stored as internal states are referenced each time new data is input. Therefore, changes in the appearance of the turn signals of the preceding vehicle along the time axis are used to calculate the confidence levels of the individual turn signals. Therefore, by using a neural network with a recursive structure as the second classifier, the lighting state detection unit 33 can accurately calculate the confidence levels of the turn signals of the preceding vehicle. Even when a neural network with a recursive structure is used as the second classifier, the second classifier may be trained using training data similar to the training data described above.

[0044] In this case, the lighting state detection unit 33 may input features determined from pixel values ​​in an object region including the preceding vehicle to the second classifier every time an image is obtained from the camera 2. This allows the lighting state detection unit 33 to calculate the degree of certainty for each possible lighting state of the turn signal of the preceding vehicle.

[0045] The lighting state detection unit 33 identifies the state with the highest degree of certainty among the possible lighting states of the turn signals of the preceding vehicle.The lighting state detection unit 33 then detects the state with the highest degree of certainty among the possible lighting states of the turn signals as the current lighting state of the turn signals.For example, if the certainty is highest regarding the flashing state of the left turn signal of the preceding vehicle, the lighting state detection unit 33 detects the flashing state of the left turn signal of the preceding vehicle as the current lighting state of the turn signals of the preceding vehicle.Also, if the certainty is highest regarding the state in which both the left and right turn signals of the preceding vehicle are off, the lighting state detection unit 33 detects a state in which both turn signals of the preceding vehicle are off as the current lighting state of the turn signals of the preceding vehicle.

[0046] The lighting state detection unit 33 notifies the vehicle control unit 36 ​​of the detection result of the lighting state of the turn signal for each preceding vehicle being tracked.

[0047] The determination unit 34 determines whether or not a preceding vehicle traveling just before vehicle 10 has taken evasive action. Note that it is assumed that the closer a preceding vehicle is to vehicle 10, the larger the size of the preceding vehicle in the image generated by camera 2 will be, and the closer the position of the bottom edge of the preceding vehicle in the image will be to the bottom edge of the image. Therefore, the determination unit 34 identifies, among the preceding vehicles, the preceding vehicle that is represented in the largest object area in the latest image as the preceding vehicle traveling just before vehicle 10. Alternatively, the determination unit 34 may identify, among the preceding vehicles, the preceding vehicle that is represented in the object area closest to the bottom edge of the latest image as the preceding vehicle traveling just before vehicle 10.

[0048] Based on the tracking results of the preceding vehicle immediately preceding vehicle 10, if the moving speed of the preceding vehicle in a direction perpendicular to the traveling direction of vehicle 10 (hereinafter referred to as the lateral direction for convenience) is equal to or greater than a predetermined speed threshold, the determination unit 34 determines that the preceding vehicle has taken evasive action. In this case, the determination unit 34 can estimate the lateral speed based on the change in the position of the preceding vehicle in each image acquired during tracking of the preceding vehicle. For example, it is estimated that the bottom edge of the object area depicting the preceding vehicle represents the position where the preceding vehicle contacts the road surface. Therefore, the determination unit 34 can estimate the distance from vehicle 10 to the preceding vehicle based on the parameters of camera 2, such as the focal length, shooting direction, and installation height of camera 2, and the position of the bottom edge of the object area depicting the preceding vehicle. Furthermore, there is a one-to-one correspondence between the position of the preceding vehicle on the image and the orientation from camera 2 to the preceding vehicle. Therefore, if the distance from the vehicle 10 to the preceding vehicle can be estimated, the lateral movement amount of the preceding vehicle between the times when each image is generated can be estimated based on the change in the lateral position of the preceding vehicle between each image obtained during tracking and the lateral movement amount of the vehicle 10 itself between the times when each image is generated. Therefore, the determination unit 34 can estimate the lateral movement speed of the preceding vehicle by dividing the lateral movement amount by the image generation interval. Note that the determination unit 34 can estimate the lateral movement amount of the vehicle 10 itself between the times when each image is generated based on the yaw rate obtained by a gyro sensor or the like mounted on the vehicle 10. Alternatively, the determination unit 34 can estimate the position and direction of the vehicle 10 at the time each image is generated by projecting features detected from each image onto a map and determining the position and direction of the vehicle 10 when the detected features most closely match the corresponding features on the map. Therefore, the determination unit 34 can estimate the lateral movement amount of the vehicle 10 between the times when each image is generated based on the position of the vehicle 10 at the time each image is generated. Furthermore, if the vehicle 10 is equipped with a distance measurement sensor, the determination unit 34 can estimate, as the distance from the vehicle 10 to the preceding vehicle, the distance measured by the distance measurement sensor in a direction corresponding to the position of the preceding vehicle on the image captured by the camera 2. Therefore, the determination unit 34 may estimate the lateral speed of the preceding vehicle based on changes in the distance and direction from the vehicle 10 to the preceding vehicle during tracking, and the amount of lateral movement of the vehicle 10 itself between the times when each image was generated.

[0049] Alternatively, the determination unit 34 may determine that the leading vehicle has taken evasive action when the lateral movement amount of the leading vehicle is equal to or greater than a predetermined movement amount threshold. Alternatively, the determination unit 34 may determine that the leading vehicle has taken evasive action when the lateral acceleration of the leading vehicle is equal to or greater than a predetermined acceleration threshold. In this case, the determination unit 34 can estimate the lateral acceleration of the leading vehicle at the time each image is generated by dividing the lateral movement speed of the leading vehicle at the time each image is generated by the image generation interval. Alternatively, the determination unit 34 may determine that the leading vehicle has taken evasive action when two or more of the above-mentioned conditions for determining evasive action are satisfied.

[0050] The determination unit 34 notifies the trajectory detection unit 35 and the vehicle control unit 36 ​​of the determination result as to whether or not the preceding vehicle immediately preceding the vehicle 10 has taken evasive action.

[0051] The trajectory detection unit 35 refers to the determination result notified by the determination unit 34 as to whether the preceding vehicle immediately preceding the vehicle 10 has taken evasive action. If the trajectory detection unit 35 determines that the preceding vehicle has taken evasive action, the trajectory detection unit 35 detects the trajectory of the end of the preceding vehicle opposite the avoidance direction (hereinafter, for convenience of explanation, may simply be referred to as the opposite end) during the evasive action. For example, the trajectory detection unit 35 estimates a predetermined period before and after the determination that the preceding vehicle has taken evasive action as the period during which the evasive action was performed (hereinafter, referred to as the evasive action period). In this case, the trajectory detection unit 35 may determine that the lateral distance of the preceding vehicle from the center of the lane in which the vehicle 10 is traveling becomes equal to or greater than a predetermined distance as the time when the evasive action is started, i.e., the start of the evasive action period. Alternatively, the trajectory detection unit 35 may determine that the lateral speed or acceleration of the preceding vehicle becomes equal to or greater than a predetermined evasive action start threshold as the start of the evasive action period. Note that the evasive action start threshold is preferably set equal to or less than the movement amount threshold and the acceleration threshold. The trajectory detection unit 35 then determines the position of the preceding vehicle at the start of the evasive maneuver as the start point of the evasive maneuver. The trajectory detection unit 35 then estimates that one end of the object region representing the preceding vehicle in each image during the evasive maneuver, on the side opposite the avoidance direction, corresponds to the opposite end of the preceding vehicle. The trajectory detection unit 35 then estimates the position of the opposite end of the preceding vehicle on a bird's-eye image with camera 2 as the origin by applying a viewpoint conversion process using camera 2 parameters to the one end of the object region corresponding to the end of the preceding vehicle in each image during the evasive maneuver. Furthermore, the trajectory detection unit 35 converts the position of the opposite end of the preceding vehicle on the bird's-eye image at the time of generating each image during the evasive maneuver into a position on the world coordinate system based on the position and direction of vehicle 10 at the time of generating each image. The trajectory detection unit 35 then chronologically arranges the converted positions to detect the trajectory of the opposite end of the preceding vehicle during the evasive maneuver. As explained in relation to the determination unit 34, the trajectory detection unit 35 may estimate the position and direction of the vehicle 10 at the time of generating each image by comparing features detected from the image with corresponding features shown on the map. Alternatively, the trajectory detection unit 35 may acquire the position and direction of the vehicle 10 at the time of generating each image from the determination unit 34.

[0052] The trajectory detection unit 35 notifies the vehicle control unit 36 ​​of the trajectory of the opposite end of the preceding vehicle.

[0053] The vehicle control unit 36 ​​refers to the determination result notified by the determination unit 34 as to whether or not the preceding vehicle immediately preceding the vehicle 10 has taken evasive action. If the vehicle control unit 36 ​​determines that the preceding vehicle has taken evasive action, it sets a planned driving route for the vehicle 10 so that the opposite end of the vehicle 10 moves along the trajectory of the opposite end of the preceding vehicle notified by the trajectory detection unit 35. On the other hand, if it determines that the preceding vehicle immediately preceding the vehicle has not taken evasive action, the vehicle control unit 36 ​​sets a planned driving route so that the vehicle 10 continues to travel along the lane in which the vehicle 10 is currently traveling.

[0054] The vehicle control unit 36 ​​controls each unit of the vehicle 10 so that the vehicle 10 travels along the set planned travel route. For example, the vehicle control unit 36 ​​calculates a target acceleration of the vehicle 10 based on the planned travel route and the current vehicle speed of the vehicle 10 measured by a vehicle speed sensor (not shown), and sets an accelerator opening or a braking amount to achieve the target acceleration. The vehicle control unit 36 ​​then calculates a fuel injection amount based on the set accelerator opening and outputs a control signal corresponding to the fuel injection amount to a fuel injection device of the engine of the vehicle 10. Alternatively, the vehicle control unit 36 ​​controls a power supply device to a motor for driving the vehicle 10 so that the motor supplies power corresponding to the set accelerator opening to the motor. Alternatively, the vehicle control unit 36 ​​outputs a control signal corresponding to the set braking amount to the brakes of the vehicle 10. Furthermore, the vehicle control unit 36 ​​calculates a steering angle of the vehicle 10 for traveling along the planned travel route based on the planned travel route and the current position of the vehicle 10, and outputs a control signal corresponding to the steering angle to an actuator (not shown) that controls the steered wheels of the vehicle 10.

[0055] 4 is a diagram showing an example of the relationship between the trajectory of the opposite end of a preceding vehicle immediately before vehicle 10 and the planned driving route of vehicle 10. In the example shown in FIG. 4, a fallen object 401 is present ahead of vehicle 10 on road 400 on which vehicle 10 is traveling. Therefore, preceding vehicle 410 traveling immediately before vehicle 10 is taking evasive action to avoid fallen object 401, and is therefore moving to the right with respect to the traveling direction of preceding vehicle 410 (i.e., the avoidance direction is a direction toward the right with respect to the traveling direction of vehicle 10). Therefore, trajectory 411 of the left end of preceding vehicle 410 is detected, and planned driving route 412 of vehicle 10 is set so that the left end of vehicle 10 moves along trajectory 411.

[0056] By setting the planned driving route in this manner, even if the size (particularly the width) of the vehicle 10 differs from the size of the preceding vehicle, the planned driving route is set so that the vehicle 10 will not collide with a fallen object. Therefore, the vehicle control unit 36 ​​can control the driving of the vehicle 10 so that the vehicle 10 will not collide with a fallen object.

[0057] Note that even if it is determined that the immediately preceding vehicle has taken evasive action, if the immediately preceding vehicle was flashing its turn signal in the direction of avoidance for a predetermined period (e.g., several seconds) immediately before starting the evasive action, it is possible that the preceding vehicle simply changed lanes for the purpose of overtaking or the like. Therefore, the vehicle control unit 36 ​​determines, based on the detection result of the turn signal illumination state, whether the immediately preceding vehicle was flashing its turn signal in the direction of avoidance for a predetermined period immediately before starting the evasive action of the immediately preceding vehicle. If the immediately preceding vehicle was flashing its turn signal in the direction of avoidance for that predetermined period, the vehicle control unit 36 ​​may set a planned travel route so that the vehicle 10 continues traveling along the lane in which the vehicle 10 is currently traveling.

[0058] Furthermore, not only the vehicle immediately preceding vehicle 10 but also other preceding vehicles traveling further ahead of the immediately preceding vehicle may be flashing their turn signals in the avoidance direction. In this case, the immediately preceding vehicle may have taken evasive action after observing the behavior of the other preceding vehicles further ahead. Therefore, the vehicle control unit 36 ​​may set the planned driving route of vehicle 10 in the same manner as above if each of multiple preceding vehicles, including the immediately preceding vehicle, was flashing its turn signal in the avoidance direction during a predetermined period immediately before it was determined that the immediately preceding vehicle had taken evasive action. That is, in this case, the vehicle control unit 36 ​​may set the planned driving route of vehicle 10 so that the opposite end of vehicle 10 moves along the trajectory of the opposite end of the immediately preceding vehicle.

[0059] 5 is an operational flowchart of the vehicle control process executed by the processor 23. The processor 23 executes the vehicle control process in accordance with the operational flowchart shown in FIG.

[0060] The preceding vehicle detection unit 31 of the processor 23 detects one or more preceding vehicles from the image received from the camera 2 (step S101).

[0061] Furthermore, the tracking unit 32 of the processor 23 tracks each preceding vehicle detected from the latest image by associating it with a preceding vehicle detected from a past image (step S102).

[0062] The lighting state detection unit 33 of the processor 23 detects the lighting state of the turn signal of each preceding vehicle being tracked (step S103).

[0063] Furthermore, the determination unit 34 of the processor 23 determines, based on the tracking result, whether or not the preceding vehicle traveling just before the vehicle 10 among the preceding vehicles has taken evasive action (step S104).

[0064] If it is determined that the preceding vehicle immediately before has taken evasive action (step S104-Yes), the trajectory detection unit 35 of the processor 23 detects the trajectory of the end of the preceding vehicle on the opposite side to the evasive direction during the evasive action (step S105).

[0065] Thereafter, the vehicle control unit 36 ​​of the processor 23 determines, based on the illumination state of the turn signals of each preceding vehicle, whether or not only the preceding vehicle immediately preceding the vehicle 10 has flashed the turn signal in the avoidance direction during a predetermined period before taking evasive action (step S106). If multiple preceding vehicles have flashed the turn signal in the avoidance direction, or if the immediately preceding preceding vehicle has not flashed the turn signal in the avoidance direction (step S106-No), it is assumed that the immediately preceding preceding vehicle has taken evasive action to avoid some kind of obstacle. Therefore, the vehicle control unit 36 ​​sets a planned travel route for the vehicle 10 so that the end of the vehicle 10 on the same side moves along the trajectory of the opposite end of the preceding vehicle (step S107).

[0066] On the other hand, if it is determined in step S104 that the immediately preceding vehicle has not taken evasive action (step S104-No), the vehicle control unit 36 ​​sets a planned driving route so that the vehicle 10 continues traveling along the own lane (step S108). Also, if in step S106 only the immediately preceding vehicle flashed the turn signal in the evasive direction during the predetermined period before taking evasive action (step S106-Yes), the immediately preceding vehicle may have taken an action such as changing lanes simply to overtake. Therefore, the vehicle control unit 36 ​​sets a planned driving route so that the vehicle 10 continues traveling along the own lane (step S108).

[0067] After setting the planned driving route, the vehicle control unit 36 ​​controls each part of the vehicle 10 so that the vehicle 10 travels along the planned driving route (step S109). Then, the processor 23 ends the vehicle control process.

[0068] As described above, the vehicle control device tracks a detected leading vehicle and determines whether the leading vehicle immediately preceding the host vehicle has taken evasive action. If the leading vehicle has taken evasive action, the vehicle control device sets a planned driving route so that the opposite end of the host vehicle moves along the trajectory of the opposite end of the leading vehicle. Therefore, even if an obstacle that is difficult to detect accurately, such as a fallen object or a damage to the road surface, exists on the path of the host vehicle, the vehicle control device controls the vehicle to avoid the obstacle, thereby preventing an accident. In particular, by controlling the driving of the host vehicle so that the end of the host vehicle on the same side moves along the trajectory of the opposite end of the leading vehicle, the vehicle control device can control the host vehicle to avoid the obstacle even if the size of the leading vehicle and the size of the vehicle itself are different.

[0069] According to a modified example, the determination unit 34 may determine whether or not evasive action has been taken not only for the vehicle immediately preceding the vehicle 10 but also for other preceding vehicles. If multiple preceding vehicles are taking evasive action and the vehicle 10 can be stopped by the point where the immediately preceding vehicle began its evasive action, the vehicle control unit 36 ​​may transfer control of the vehicle 10 to the driver. For example, if the vehicle 10 decelerates from its current speed at a predetermined deceleration rate, the vehicle speed of the vehicle 10 will be zero by the time the immediately preceding vehicle begins its evasive action, the determination unit 34 determines that the vehicle 10 can be stopped by that point. In this case, the vehicle control unit 36 ​​notifies the driver via a user interface, such as a display device, speaker, or vibration device, provided in the vehicle cabin of the vehicle 10 that control will be transferred and that there is a possibility of an obstacle ahead of the vehicle 10. After a predetermined period of time has elapsed since the notification and the touch sensor on the steering wheel detects that the driver has held the steering wheel, the vehicle control unit 36 ​​transfers control to the driver. If it is not detected that the driver has held the steering wheel even after a predetermined period of time has elapsed, the vehicle control unit 36 ​​may control each unit of the vehicle 10 to stop the vehicle 10 at the point where the immediately preceding vehicle has started to take evasive action. This allows the vehicle control unit 36 ​​to more reliably prevent the vehicle 10 from colliding with an obstacle when there is a possibility that the obstacle is present ahead of the vehicle 10, and also prevents the vehicle 10 from taking unnecessary evasive action when no obstacle actually exists.

[0070] According to another modification, the leading vehicle detection unit 31 may detect a leading vehicle based on a sensor signal acquired by a sensor other than the camera 2 for detecting objects present around the vehicle 10, such as a distance measurement signal from a distance measurement sensor. In this case, the first classifier used by the leading vehicle detection unit 31 may be trained in advance to output a confidence level for other vehicles for each of multiple areas set within the detection range of the sensor based on the sensor signal acquired by the sensor. In this case, the first classifier may also be configured using a DNN, as in the above embodiment or modification. Alternatively, the first classifier may be a classifier based on a machine learning method other than a DNN, such as a support vector machine.

[0071] According to yet another modification, the vehicle control unit 36 ​​may set the planned driving route without referring to the lighting state of the turn signal of the immediately preceding vehicle. That is, the vehicle control unit 36 ​​may set the planned driving route so that, when the immediately preceding vehicle takes evasive action, the end of the vehicle 10 moves along the trajectory of the end of the vehicle 10 on the opposite side of the evasive direction of the immediately preceding vehicle. This may result in the vehicle 10 taking unnecessary evasive action in some cases, but the vehicle control unit 36 ​​can more reliably prevent the vehicle 10 from colliding with an obstacle. According to this modification, the processing by the lighting state detection unit 33 may be omitted.

[0072] In some cases, such as when a large vehicle is traveling in the adjacent lane on the side of the avoidance direction, it may not be desirable to set a travel trajectory such that the end of the vehicle 10 on the opposite side of the avoidance direction moves along the trajectory of the end of the preceding vehicle on the opposite side of the avoidance direction. Therefore, according to another modified example, the vehicle control unit 36 ​​may set a planned travel route such that the end of the vehicle 10 on the same side as the opposite end passes through a position offset by a predetermined distance from the trajectory of the opposite end of the preceding vehicle.

[0073] In addition, a computer program that realizes the functions of each part of the processor 23 of the vehicle control device according to the above embodiment or variant may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium or an optical recording medium.

[0074] As described above, those skilled in the art can make various modifications to the embodiments within the scope of the present invention. [Explanation of symbols]

[0075] 1. Vehicle control system 2 Cameras 3 Electronic control unit (vehicle control unit) 21 Communication Interface 22 Memory 23 processors 31 Leading vehicle detection unit 32 Tracking Department 33 Lighting status detection unit 34 Judgment section 35 Trajectory detection unit 36 Vehicle control unit

Claims

1. a preceding vehicle detection unit that detects a preceding vehicle traveling ahead of the vehicle from a series of time-series sensor signals obtained by a sensor provided in the vehicle for detecting objects around the vehicle; a tracking unit that tracks the preceding vehicle detected from the series of sensor signals; a determination unit that determines whether the preceding vehicle has taken evasive action based on the tracking result; a trajectory detection unit that detects, when the preceding vehicle takes an avoidance action, a trajectory of an end of the preceding vehicle on the opposite side to the avoidance direction during the avoidance action; a control unit that controls the traveling of the vehicle so that an end of the vehicle on the opposite side to the avoidance direction moves along the trajectory; A vehicle control device having the above.

2. a lighting state detection unit that detects a lighting state of a turn signal of the preceding vehicle based on the series of sensor signals; 2. The vehicle control device according to claim 1, wherein the control unit determines whether the turn signal on the side of the avoidance direction is turned on before the preceding vehicle takes evasive action based on the lighting state, and when the turn signal on the side of the avoidance direction is turned on before the preceding vehicle takes evasive action, the control unit causes the vehicle to travel along the lane in which the vehicle is traveling.

3. the preceding vehicle detection unit detects a plurality of preceding vehicles, the determination unit determines whether or not the avoidance behavior has been taken for each of the plurality of preceding vehicles, and identifies a point at which a preceding vehicle immediately preceding the vehicle among the plurality of preceding vehicles has started the avoidance behavior; 3. The vehicle control device according to claim 1, wherein the control unit transfers control of the vehicle to a driver of the vehicle when each of the plurality of preceding vehicles takes evasive action and the vehicle can be stopped by a point where a preceding vehicle immediately before the vehicle has started the evasive action.

4. Detecting a preceding vehicle traveling ahead of the vehicle from a series of time-series sensor signals obtained by a sensor provided on the vehicle for detecting objects around the vehicle; tracking the preceding vehicle detected from the series of sensor signals; determining whether the preceding vehicle has taken evasive action based on the tracking result; When the preceding vehicle takes an avoidance action, a trajectory of an end of the preceding vehicle on the opposite side to the avoidance direction during the avoidance action is detected; controlling the travel of the vehicle so that an end of the vehicle on the opposite side to the avoidance direction moves along the trajectory; A vehicle control method comprising:

5. Detecting a preceding vehicle traveling ahead of the vehicle from a series of time-series sensor signals obtained by a sensor provided on the vehicle for detecting objects around the vehicle; tracking the preceding vehicle detected from the series of sensor signals; determining whether the preceding vehicle has taken evasive action based on the tracking result; When the preceding vehicle takes an avoidance action, a trajectory of an end of the preceding vehicle on the opposite side to the avoidance direction during the avoidance action is detected; controlling the travel of the vehicle so that an end of the vehicle on the opposite side to the avoidance direction moves along the trajectory; A computer program for vehicle control that causes a processor mounted on the vehicle to execute the above.

Images