Vehicle control device and computer program

Abstract

The present invention addresses the problem of quickening driving support control when another vehicle overpasses. A vehicle control device is provided with: a control unit that executes driving support control, the control unit performing a process for identifying another vehicle present in first imaging data in which an environment behind the vehicle has been imaged; when the other vehicle cannot be recognized from the first camera data, determining the possibility that the other vehicle surpasses the vehicle; a recognition unit that recognizes the moving object as the other vehicle if the moving object exceeding the vehicle includes a feature quantity indicating the other vehicle in second imaging data in which the possibility is greater than or equal to a predetermined value and in which the environment in front of the vehicle is imaged; and executing prescribed driving assistance control for causing the vehicle to avoid the other vehicle.

Description

Technical Field

[0001] This invention relates to a vehicle control device and computer program for performing driving assistance control. Background Technology

[0002] Patent document 1 describes a technology that uses a camera to monitor other vehicles traveling behind a following vehicle. When other vehicles disappear from the camera's field of view, the system determines that the other vehicle has overtaken the vehicle and performs driving assistance.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2008-262401 Summary of the Invention

[0004] According to the technology described in Patent Document 1, it is not determined whether other vehicles have actually overtaken the vehicle. According to the technology described in Patent Document 1, there is a possibility that the timing of driver assistance may be delayed when other vehicles squeeze in front of the vehicle after it has overtaken it.

[0005] The purpose of this invention is to provide a vehicle control device and computer program that enables rapid driver assistance control when other vehicles are overtaking.

[0006] One aspect of the present invention is a vehicle control device comprising: a control unit that performs driving assistance control, the control unit performing the following processing: identifying other vehicles present in first camera data that has captured images of the environment behind the vehicle; if the other vehicle can no longer be identified from the first camera data, determining the possibility that the other vehicle may overtake the vehicle; if the possibility is greater than or equal to a certain value and if, in second camera data that has captured images of the environment in front of the vehicle, a moving object that overtakes the vehicle contains a feature quantity representing the other vehicle, identifying the moving object as the other vehicle; and performing a predetermined driving assistance control for causing the vehicle to avoid the other vehicle.

[0007] Invention Effects

[0008] According to the present invention, driving assistance control can be rapidly achieved when other vehicles are overtaking. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating the structure of a vehicle according to an embodiment of the present invention.

[0010] Figure 2 This is a diagram showing the camera range of the first and second cameras.

[0011] Figure 3 This is a diagram representing an example of the data from the first camera.

[0012] Figure 4This is a graph showing the other vehicles that moved in the first camera's data.

[0013] Figure 5 This is a diagram representing the state of other vehicles overtaking.

[0014] Figure 6 It is a diagram representing the defined area set in the second camera data.

[0015] Figure 7 This is a diagram illustrating an example of driver assistance control.

[0016] Figure 8 This is a diagram illustrating an example of driver assistance control.

[0017] Figure 9 This is a diagram illustrating an example of the threshold used when driver assistance control is initiated.

[0018] Figure 10 It is a flowchart representing the processing flow of a driver assistance control method. Detailed Implementation

[0019] like Figure 1 and Figure 2 As shown, vehicle 1 is configured to perform driving assistance. Vehicle 1 consists of a detection unit 2 and a vehicle control unit 10, enabling it to perform driving assistance functions such as Advanced Driver-Assistance Systems (ADAS). The detection unit 2 detects the environment surrounding vehicle 1. The detection unit 2 consists of multiple devices configured according to its purpose. The detection values ​​detected by the detection unit 2 are used by driving assistance or navigation devices, etc.

[0020] The detection unit 2 includes a first camera 3 that captures images of the environment behind the vehicle 1. The first camera 3 captures images of the environment behind the vehicle 1 and outputs first image data. In this embodiment, the first camera 3 captures a first imaging range R1 behind the vehicle 1. The detection unit 2 also includes a second camera 4 that captures images of the environment in front of the vehicle 1. The second camera 4 captures images of the environment in front of the vehicle 1 and outputs second image data. In this embodiment, the second camera 4 captures a second imaging range R2 in front of the vehicle 1.

[0021] The first image data from camera 3 and the second image data from camera 4 are used, for example, in driver assistance or a dashcam for vehicle 1. The first camera range R1 of camera 3 and the second camera range R2 of camera 4 may vary depending on vehicle 1. Areas outside the first camera range R1 and the second camera range R2 may contain, for example, first blind spots N1 and second blind spots N2 that are not captured by camera 3 and camera 4.

[0022] The detection unit 2 may be equipped with a position sensor (not shown) that detects the current position of vehicle 1. The position sensor may be, for example, a Global Positioning System (GPS) sensor or a Global Navigation Satellite System (GNSS) sensor. The position sensor may supplement the position of vehicle 1 with autonomous sensors (not shown) used in autonomous navigation, such as gyroscope sensors and accelerometers.

[0023] Vehicle 1 includes a drive unit 5 that generates power for movement. Drive unit 5 is, for example, composed of an internal combustion engine that uses fuel. If vehicle 1 is an electric vehicle, drive unit 5 is composed of an electric motor. If vehicle 1 is a hybrid vehicle, drive unit 5 may be composed of a combination of an internal combustion engine and an electric motor. When driving assistance is activated, drive unit 5 is controlled by vehicle control device 10, which adjusts its speed.

[0024] Vehicle 1 is equipped with a braking unit 6 for decelerating the vehicle speed and controlling it to a stop. The braking unit 6 is, for example, a braking device that generates braking force. In the case that vehicle 1 is an electric vehicle, the braking unit 6 may be integrated with the drive unit 5. When driving assistance is performed, the braking unit 6 is controlled by the vehicle control unit 10.

[0025] The vehicle 1 is equipped with a steering unit 7 for operating the driving direction. The steering unit 7 consists of a power steering device that imparts a steering angle to the steering wheels based on the operation of the handlebars. When the vehicle 1 is an electric vehicle, the steering unit 7 can be integrated with a drive unit 5 that can variablely control the left and right driving forces of the drive wheels. When driving assistance is performed, the steering unit 7 is controlled by the vehicle control device 10 to adjust the steering angle.

[0026] The vehicle control device 10 includes a control unit 11 that performs controls related to the driving of the vehicle 1. Based on detection values ​​detected by the detection unit 2, the control unit 11 comprehensively executes controls for driving, driving assistance, navigation, and other functions of the vehicle 1. The control unit 11 is composed of at least one hardware processor, such as a central processing unit (CPU). The control unit 11 can be implemented using hardware (including a circuitry) such as large-scale integration (LSI), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or graphics processing unit (GPU), or it can be implemented through a combination of software and hardware.

[0027] The vehicle control unit 10 includes a storage unit 12 for storing data or programs. The storage unit 12 is composed of a non-transitory storage medium such as a hard disk drive (HDD) or a solid-state drive (SSD). The storage unit 12 stores computer programs and data required for the control of the vehicle 1. The computer program can be pre-stored in the storage unit 12, or it can be stored on an externally connectable storage medium such as a DVD or CD-ROM, and installed in the storage unit 12 by installing the storage medium in the drive unit. The control unit 11 controls the drive unit 5, the braking unit 6, and the steering unit 7 based on the detection values ​​detected by the detection unit 2, and performs driver assistance controls such as Obstacle Anticipation Assist (OAA).

[0028] The control unit 11 acquires first video data captured by the first camera 3 at a predetermined frame rate. The control unit 11 generates multiple consecutive first video data sequences as first motion image data and stores them in the storage unit 12 at predetermined intervals. The control unit 11 acquires second video data captured by the second camera 4 at a predetermined frame rate. The control unit 11 generates multiple consecutive second video data sequences as second motion image data and stores them in the storage unit 12 at predetermined intervals. After a predetermined period, the control unit 11 deletes both the first and second motion image data. In the event of events such as sudden braking, sharp steering, or approaching other vehicles or objects in the vehicle 1, the control unit 11 restricts the deletion of the first and / or second motion image data.

[0029] The control unit 11 is configured to comprehensively determine the first recognition result of the first camera data and the second recognition result of the second camera data, and then perform driver assistance control. The control unit 11 pre-executes machine learning such as deep learning, which uses camera data of the driving road environment as training data, and is configured to be able to identify the environment around the vehicle 1, objects existing in the environment, and other vehicles driving around the vehicle 1.

[0030] The control unit 11 analyzes the first and second camera data to identify roads, road structures, road signs, road surface displays, buildings, and other traffic participants such as vehicles and pedestrians from the environment surrounding the vehicle 1.

[0031] The control unit 11 consists of at least one hardware processor, such as a central processing unit (CPU). The storage unit 12 consists of a non-transitory storage medium, such as a hard disk drive (HDD) or a solid-state drive (SSD). The computer programs and data required for control are stored in the storage unit 12.

[0032] The control unit 11 can be implemented using hardware such as Large Scale Integration (LSI), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or Graphics Processing Unit (GPU), including the circuitry section. It can also be implemented through a combination of software and hardware. The program can be pre-stored in a storage device such as an HDD or flash memory (a storage device with a non-transitory storage medium) within the storage unit 12, or it can be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the storage unit 12 by mounting the storage medium (non-transitory storage medium) in a drive device.

[0033] Storage unit 12 may be, for example, a hard disk drive, flash memory, or other storage device. Storage unit 12 stores, for example, programs and other various information required for the operation of control unit 11. Vehicle 1 is equipped with a display unit 8 capable of displaying information based on images and characters. Display unit 8 may be, for example, a display device composed of a liquid crystal display (LCD) or an organic electroluminescent (EL) display. Display unit 8 may include a speaker for outputting sound. Display unit 8 may be composed of a touch panel capable of inputting information. Display unit 8 may include a microphone capable of inputting sound. Display unit 8 can input information based on sound.

[0034] like Figure 3 As shown, the control unit 11 acquires first camera data M1, which captures images of the environment behind vehicle 1. The control unit 11 analyzes the first camera data M1 to identify the road S, other vehicles Vp (p: a natural number), road structures, etc., within the environment behind vehicle 1. For example, the control unit 11 identifies the road boundary Lm (m: a natural number) based on boundary lines, road paving, and other structural boundaries. The control unit 11 identifies the lane Sn (n: a natural number) based on the road boundary Lm in road S. The control unit 11 extracts feature points from the characteristic parts of other vehicles Vp and calculates feature quantities based on the configuration relationships of these feature points. For example, the control unit 11 calculates feature quantities based on other features such as the body shape and partial shapes of other vehicles Vp, as well as other features such as color and the illumination status of lights. The control unit 11 identifies individual other vehicles Vp based on their feature quantities.

[0035] like Figure 4As shown, the control unit 11 compares the first camera data M1 in a time sequence to analyze the position and movement of other vehicles Vp. The control unit 11 determines the lane Sn in which the other vehicles Vp are traveling. The control unit 11 determines the speed of the other vehicles Vp. The control unit 11 determines the direction Q of movement of the other vehicles Vp. Based on the determination results of the speed and direction of movement of the other vehicles Vp, the control unit 11 identifies whether the other vehicles Vp are moving away from vehicle 1 or moving closer to vehicle 1. The control unit 11 also identifies whether the other vehicles Vp have changed lanes.

[0036] The control unit 11 determines the likelihood of another vehicle Vp overtaking vehicle 1 based on the actions of other vehicles Vp. The control unit 11 calculates the likelihood of another vehicle Vp overtaking vehicle 1 based on a preset threshold. In the control unit 11, if another vehicle Vp attempts to overtake vehicle 1, the likelihood of the other vehicle Vp overtaking vehicle 1 is calculated as a value higher than the preset threshold (probability: high). In the control unit 11, if another vehicle Vp does not attempt to overtake vehicle 1, the likelihood of the other vehicle Vp overtaking vehicle 1 is calculated as a value lower than the preset threshold (probability: low).

[0037] The control unit 11 determines the direction of movement of other vehicles Vp from the first camera data M1. In the control unit 11, if other vehicles Vp are moving away from vehicle 1 at a speed, the probability of other vehicles Vp overtaking vehicle 1 is set to be less than a preset threshold. In the control unit 11, if other vehicles Vp and vehicle 1 are traveling in the same lane, the probability of other vehicles Vp overtaking vehicle 1 is set to be less than a preset threshold. In the control unit 11, if other vehicles Vp are moving in an adjacent lane at a speed approaching vehicle 1, the probability of other vehicles Vp overtaking vehicle 1 is set to be greater than or equal to a preset threshold.

[0038] In the control unit 11, if another vehicle Vp approaches vehicle 1 and moves in a direction that would allow it to overtake vehicle 1, and the other vehicle Vp can no longer be detected from the first camera data M1, the probability that the other vehicle Vp will overtake vehicle 1 is set to a preset threshold or higher. When the other vehicle Vp can no longer be detected from the first camera data M1, the other vehicle Vp moves to the area of ​​the first blind spot N1 or the second blind spot N2 generated by the first camera 3 and the second camera 4 (see reference). Figure 2 ).

[0039] The control unit 11 determines the moving speed of other vehicles Vp from the first camera data M1. In the control unit 11, if other vehicles Vp can no longer be identified from the first camera data M1 at a relative moving speed approaching vehicle 1, the probability that other vehicles Vp will overtake vehicle 1 can be set to a preset threshold or higher. Through this process, the control unit 11 can enable vehicle 1 to initiate driver assistance control of other vehicles Vp earlier than normally.

[0040] like Figure 5 As shown, in the control unit 11, if other vehicles Vp can no longer be identified from the first camera data M1 and the probability of other vehicles Vp passing over vehicle 1 exceeds a threshold, the processing of determining the action of other vehicles Vp is transferred to the second camera 4. The control unit 11 determines the action of other vehicles Vp based on the second camera data M2 captured by the second camera 4. In the control unit 11, if other vehicles Vp move to one side of vehicle 1 (e.g., lane S3 side), it is determined whether other vehicles Vp can no longer be identified from the first camera data M1.

[0041] In the control unit 11, when other vehicles Vp can no longer be identified from the first camera data M1, a predetermined area W is set on one side of the second camera data M2. The control unit 11 can set the predetermined area W within a fixed range within the second camera data M2, or it can adjust the predetermined area W according to the road width or speed. Compared to the normal state, the recognition threshold for moving objects in the predetermined area W is lowered. The control unit 11 can determine that if not only has other vehicles Vp disappeared from the first camera data M1, but a portion of other vehicles Vp has disappeared from the first camera data M1 by a predetermined amount or more, then other vehicles Vp can no longer be identified from the first camera data M1.

[0042] The control unit 11 can determine whether other vehicles Vp are moving in the direction that disappears from the first camera data M1, or whether other vehicles Vp have overtaken it, based on the position and direction of movement of other vehicles Vp. Based on the above processing, the control unit 11 increases the sensitivity of moving object recognition within the specified area W compared to the normal state, and makes the time of recognizing other vehicles Vp in the second camera data M2 faster than the normal state. The recognition sensitivity is set, for example, based on the probability (reliability %) of recognizing an object as another vehicle Vp.

[0043] In the control unit 11, under normal conditions, if the reliability is above a threshold, the identified object is identified as another vehicle Vp. However, within the designated area W, the control unit 11 determines whether a moving object is another vehicle Vp based on a reliability that is below the threshold. For example, within the designated area W, if the object contains a number of features of another vehicle Vp that are above a predetermined number below the normal threshold, the control unit 11 identifies the object as another vehicle Vp.

[0044] Based on the above processing, if the control unit 11 can no longer identify other vehicles (Vp) from the first camera data, and the probability of other vehicles overtaking the vehicle is more than a certain amount, the following processing is performed. In the control unit 11, if the second camera data M2, which captures the environment in front of vehicle 1, indicates that the moving object overtaking vehicle 1 contains more than a certain number of features representing other vehicles (Vp), the moving object is identified as another vehicle (Vp).

[0045] like Figure 6 As shown, in the second camera data M2, the control unit 11 calculates the probability of another vehicle Vp overtaking vehicle 1 to be above a certain threshold, and sets a predetermined area W in the second camera data M2. The control unit 11 calculates whether the moving object representing overtaking vehicle 1 in the predetermined area W of the second camera data M2, which captures the environment in front of vehicle 1, contains a feature quantity representing another vehicle. If, in the predetermined area W, the moving object representing overtaking vehicle 1 contains a feature quantity of another vehicle Vp that is below a certain threshold, the control unit 1 identifies the moving object as another vehicle Vp. If the control unit 11 identifies another vehicle Vp, it executes a predetermined driving assistance control to make vehicle 1 avoid the other vehicle Vp.

[0046] like Figure 7 As shown, when the control unit 11 identifies another vehicle Vp in the second camera data M2, it causes vehicle 1 to deviate from the direction of separation from the other vehicle Vp and move it. The control unit 11 controls the drive unit 5, the braking unit 6, and the steering unit 7 to adjust the deviation amount F of vehicle 1 without exceeding the lane S2 in which vehicle 1 is traveling, based on acceleration / deceleration and steering amount that will not cause discomfort to the occupants. Through the above processing, when another vehicle Vp changes lanes from lane S3 adjacent to lane S2 to lane S2, the risk of contact between vehicle 1 and other vehicle Vp can be reduced. If the control unit 11 determines that there is an object such as a road structure in the direction of deviation of vehicle 1 and ensures a safety margin, it is not necessarily necessary to adjust the deviation amount.

[0047] like Figure 8As shown, when other vehicles Vp change lanes, the control unit 11, in addition to adjusting the offset F, also calculates the Time to Collision (TTC) of vehicle 1 compared to other vehicles Vp over time. The control unit 11 compares the TTC with a threshold (e.g., 1.5s) to determine the risk of contact between vehicle 1 and other vehicles Vp. The control unit 11 starts counting the frames of the second camera data M2 at the moment other vehicles Vp are identified. The control unit 11 monitors the second camera data M2 frame by frame and calculates the TTC.

[0048] In the control unit 11, if the state of TTC being below the threshold continues for a predetermined number of frames after TTC falls below the threshold, driving assistance control to decelerate vehicle 1 is executed. If, in the control unit 11, the state of TTC being below the threshold does not continue for a predetermined number of frames after TTC falls below the threshold and other vehicles Vp move away from vehicle 1 by a predetermined distance, driving assistance control to decelerate vehicle 1 is not executed, and the adjusted offset F is restored to its original state.

[0049] Based on the above processing, by accelerating the time of identifying other vehicles Vp, the control unit 11 can accelerate the time of initiating driver assistance control to avoid other vehicles Vp, compared to the case where no specified area W is set in the second camera data M2. By accelerating the time of identifying other vehicles Vp, the control unit 11 can also accelerate the time of identifying the state where driver assistance control does not need to avoid other vehicles Vp. The control unit 11 can suppress unnecessary deceleration and avoidance actions.

[0050] The control unit 11 can set a predetermined region W within the time interval during which other vehicles Vp appear in the second camera data M2. For example, the control unit 11 calculates the relative speed of other vehicles Vp in the first camera data M1, and calculates the time when other vehicles Vp appear in the second camera data M2 based on the calculation result. The control unit 11 can set the predetermined region W within a time period near the calculated time.

[0051] The control unit 11 can adjust the timing of the execution of driver assistance control for vehicle Vp based on the relative distance between other vehicles Vp and vehicle 1 relative to the direction of travel. The control unit 11 can adjust the timing of driver assistance control by counting the number of frames of the second camera data M2 corresponding to the relative distance.

[0052] exist Figure 9The diagram illustrates the relationship between the relative distance between other vehicles Vp and vehicle 1 and the number of frames in the second camera data M2 at which driver assistance control begins. The control unit 11 can adjust the timing of driver assistance control by progressively decreasing the frame count as the relative distance decreases. The control unit 11 can execute driver assistance control independently of the relative distance, at a point when a certain number of frames have been counted since the other vehicle Vp was identified in the second camera data M2.

[0053] exist Figure 10 The diagram illustrates the processing flow of a driving assistance control method executed in the vehicle control unit 10. The driving assistance control method is executed based on a computer program installed in the computer mounted on the vehicle control unit 10. The computer program causes the control unit 11 (processor) of the computer to perform the following processes.

[0054] The control unit 11 identifies other vehicles present in the first camera data M1, which captures the environment behind vehicle 1 (S100). The control unit 11 determines whether other vehicles Vp can no longer be identified from the first camera data M1 (S102). If, in the control unit 11, other vehicles Vp move into the blind spot of the first camera 3 and can no longer be identified from the first camera data M1, the possibility of other vehicles Vp overtaking vehicle 1 is determined (S104). If, in the control unit 11, the possibility of other vehicles Vp overtaking vehicle 1 is greater than or equal to a certain value, the control unit 11 monitors the second camera data M2, which captures the area in front of vehicle 1 (S106).

[0055] The control unit 11 determines whether the moving object that is overtaking the vehicle 1 in the second camera data M2, which captures the environment in front of the vehicle 1, contains a feature quantity indicating another vehicle (S108). If the moving object contains a feature quantity, the control unit 11 identifies the moving object as another vehicle and executes the prescribed driving assistance control to make the vehicle 1 avoid the other vehicle Vp (S110).

[0056] As described above, according to the vehicle control device 10, if other vehicles Vp cannot be identified in the first camera data M1 captured behind vehicle 1, the recognition sensitivity of other vehicles Vp in the second camera data captured in front of vehicle 1 can be increased compared to the normal state, thereby accelerating the initiation of driver assistance control. According to the vehicle control device 10, by setting a predetermined area W in the second camera data M2, other vehicles Vp can be identified earlier than the normal state. According to the vehicle control device 10, the timing of driver assistance control when other vehicles Vp overtake vehicle 1 can be rapidly accelerated.

[0057] In the above embodiments, the computer programs executed in each component of the vehicle control device 10 can be provided in a computer-readable portable recording medium such as a semiconductor memory, magnetic recording medium, or optical recording medium. The computer programs can be provided as a program product.

[0058] Symbol Explanation

[0059] 1-Vehicle, 2-Detection unit, 3-First camera, 4-Second camera, 5-Drive unit, 6-Brake unit, 7-Steering unit, 8-Display unit, 10-Vehicle control device, 11-Control unit, 12-Storage unit, F-Offset, Lm-Road boundary, M1-First camera data, M2-Second camera data, N1-First blind spot, N2-Second blind spot, P-Road display, Q-Direction of movement, R1-First camera range, R2-Second camera range, S-Road, Sn-Lane, Vp-Other vehicles, W-Designated area.

Claims

1. A vehicle control device, characterized in that, have: The control unit performs driver assistance controls. The control unit performs the following processing: Identify other vehicles present in the environment behind the vehicle captured by the first camera; If the other vehicle can no longer be identified from the data of the first camera, determine the likelihood that the other vehicle is overtaking the first vehicle; If the probability is above a certain level and the second camera data that captured the environment in front of the vehicle contains a feature quantity representing the other vehicle, then the moving object is identified as the other vehicle. and Execute the prescribed driver assistance controls to enable the vehicle to avoid other vehicles.

2. The vehicle control device according to claim 1, characterized in that, The control unit performs the following processing: From the first camera data, the direction of movement of the other vehicles is determined; and If other vehicles move in a direction that surpasses the vehicle and can no longer be identified from the first camera data, the probability is set to a level above a pre-set threshold.

3. The vehicle control device according to claim 2, characterized in that, The control unit performs the following processing: The speed of the other vehicles is determined from the first camera footage; and If another vehicle can no longer be identified from the first camera data at a relative speed of movement when it approaches the vehicle, the probability is set to be above a pre-set threshold.

4. The vehicle control device according to claim 2, characterized in that, The control unit performs the following processing: If, in the first camera data, another vehicle moves to one side of the vehicle and can no longer be identified from the first camera data, the recognition threshold for the moving object in a specified area on that side in the second camera data is lowered.

5. A computer program installed on a computer mounted on a vehicle control device that performs driving assistance control, characterized in that, The computer is instructed to perform the following processing: Identify other vehicles present in the environment behind the vehicle captured by the first camera; If the other vehicle can no longer be identified from the data of the first camera, determine the likelihood that the other vehicle is overtaking the first vehicle; If the probability is above a certain level and the second camera data that captured the environment in front of the vehicle contains a feature quantity representing the other vehicle, then the moving object is identified as the other vehicle. and Execute the prescribed driver assistance controls to enable the vehicle to avoid other vehicles.

Images