Information recording device for vehicle

Abstract

To provide an information recording device for vehicle that can record a situation in which a driver of the vehicle senses any danger excluding a situation intended by the driver.SOLUTION: A processor 152 of an ECU 150 comprises: a distance determination unit 152a for determining a distance between a vehicle and a target which exists around the vehicle; an operation determination unit 152b for determining whether an operation appliance for driving the vehicle has been operated by a driver of the vehicle; and a recording unit 152c for recording information relating to the driving of the vehicle when it is determined that an operation has not been performed for a first time duration before a distance between the vehicle and a target becomes equal to or less than a predetermined value and when it is determined that an operation has been performed within a second time duration after that point in time.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information recording device for a vehicle.

Background Art

[0002] Conventionally, it has been known to acquire the situation around a vehicle, detect dangerous driving of other vehicles, and record the video around the vehicle (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Recording vehicle data at the time of an accident is useful for analyzing vehicle control and driver operations at the time of the accident. In addition, with the spread of advanced safety systems for vehicles in the future, even if an accident has not actually occurred, it is assumed that information related to vehicle operation will be recorded and analyzed in situations where the driver has a scare (situations where the driver feels some kind of danger).

[0005] In the technology described in Patent Document 1 above, when it is determined that the driving is not aggressive based on the situation around the own vehicle acquired by the drive recorder of the own vehicle, event recording is performed. However, when assuming recording of situations where the driver has a scare, if recording is performed based on the approach of the vehicle to an object such as another vehicle, recording will also be performed when the driver approaches the object in an intended situation, resulting in a problem that unnecessary data is recorded and accumulated.

[0006] Therefore, an object of the present invention is to provide a vehicle information recording device capable of recording a situation when a driver feels some danger except for a situation intended by the driver of the vehicle.

Means for Solving the Problems

[0007] The gist of the present disclosure is as follows.

[0008] (1) A distance determination unit that determines the distance between the vehicle and an object existing around the vehicle, An operation determination unit that determines whether an operation by the driver of the vehicle has been performed on the operating device for driving the vehicle, When it is determined that the operation has not been continuously performed during a first time period before a time point when the distance between the vehicle and the object becomes equal to or less than a predetermined value, and it is determined that the operation has been performed during a second time period after that time point, a recording unit that records information related to the driving of the vehicle, A vehicle information recording device comprising:

[0009] (2) The vehicle information recording device according to (1) above, wherein the information related to the driving of the vehicle is image information obtained by photographing the periphery of the vehicle or the position information of the vehicle.

[0010] (3) The vehicle information recording device according to (1) above, wherein the information related to the driving of the vehicle is information indicating the driving state of the vehicle.

Advantages of the Invention

[0011] According to the present invention, there is provided a vehicle information recording device capable of recording a situation when a driver feels some danger except for a situation intended by the driver of the vehicle.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0013] Hereinafter, several embodiments according to the present invention will be described with reference to the drawings. However, these descriptions are intended merely as examples of preferred embodiments of the present invention and are not intended to limit the present invention to such specific embodiments.

[0014] FIG. 1 is a schematic configuration diagram of a vehicle control system 1000 according to one embodiment. The control system 1000 is mounted on a vehicle such as an automobile, and includes an in-vehicle camera 110, an operation information acquisition sensor 120, a vehicle control device 130, a peripheral monitoring sensor 140, an electronic control unit (ECU: Electronic Control Unit, hereinafter referred to as ECU) 150, a warning device 160, and a positioning information receiver 170. Each of the in-vehicle camera 110, the operation information acquisition sensor 120, the vehicle control device 130, the peripheral monitoring sensor 140, the ECU 150, the warning device 160, and the positioning information receiver 170 is communicably connected via an in-vehicle network conforming to a standard such as a Controller Area Network (CAN).

[0015] The in-vehicle camera 110 includes a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or a C-MOS, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. The in-vehicle camera 110 is provided near the dashboard inside the vehicle or near the front glass, photographs the surroundings of the vehicle (for example, the front of the vehicle), and generates an image representing the environment around the vehicle. The in-vehicle camera 110 performs photographing at a predetermined photographing cycle (for example, 1 / 30 second to 1 / 10 second). The in-vehicle camera 110 may be composed of a stereo camera and may be configured to obtain the distance from each structure on the image based on the parallax between the left and right images. Each time the in-vehicle camera 110 generates an image, it outputs the generated image to the ECU 150 via the in-vehicle network.

[0016] The operation information acquisition sensor 120 is a sensor that acquires operation information of the driver of the vehicle on the operation devices for driving the vehicle. Examples of the operation devices for driving the vehicle include an accelerator pedal, a brake pedal, a steering wheel, and a shift lever. The operation information acquisition sensor 120 includes an accelerator sensor that acquires the operation information of the accelerator pedal, a brake sensor that acquires the operation information of the brake pedal, a steering sensor that acquires the operation information of the steering wheel, and a shift position sensor that acquires the operation information (shift position) of the shift lever. The steering sensor is, for example, a torque sensor of an electric power steering system (EPS), and detects the operation torque applied by the driver as the operation information of the steering wheel. The operation on the operation device is not limited to the operation during manual driving, and may be an operation during autonomous driving. In the case of an operation during autonomous driving, a switch from autonomous driving to manual driving may be performed.

[0017] The vehicle control device 130 is various devices related to vehicle control, and includes a driving device such as an internal combustion engine or an electric motor as a driving source for driving the vehicle, a transmission, a braking device for braking the vehicle, a steering device for turning the vehicle, and the like. When the driving source for driving the vehicle is an electric motor, the vehicle control device 130 may include a battery for storing electric power, a fuel cell for supplying electric power to the electric motor, and the like. Note that when the vehicle is an electric vehicle (EV), the internal combustion engine may not be included in the vehicle control device 130. Further, when the vehicle is an engine vehicle, the electric motor may not be included in the vehicle control device 130.

[0018] The peripheral monitoring sensor 140 is a sensor for monitoring the periphery of the vehicle. The peripheral monitoring sensor 140 includes sensors such as a lidar (Light Detection and Ranging) and a radar, for example. The radar includes a front side radar sensor inside the front bumper and a rear side radar sensor inside the rear bumper.

[0019] The ECU 150 is an aspect of the information recording device of the vehicle according to the present disclosure. The ECU 150 has a processor 152, a memory 154, and a communication interface 156. The processor 152 has one or a plurality of CPUs (Central Processing Unit) and its peripheral circuits. The processor 152 may further have other arithmetic circuits such as a logical arithmetic unit, a numerical arithmetic unit, or a graphic processing unit. The memory 154 has, for example, a volatile semiconductor memory and a non-volatile semiconductor memory, and stores data related to the processing according to the present embodiment. The communication interface 156 has an interface circuit for connecting the ECU 150 to the in-vehicle network.

[0020] The warning device 160 includes a display device and a speaker. The display device is composed of, for example, a liquid crystal display (LCD), is provided near the meter panel or the dashboard, and displays and outputs a warning in response to an instruction from the ECU 150. The speaker outputs a warning in voice in response to an instruction from the ECU 150.

[0021] The positioning information receiver 170 acquires positioning information representing the current position and attitude of the vehicle. For example, the positioning information receiver 170 can be a GPS (Global Positioning System) receiver.

[0022] FIG. 2 is a plan view of the vehicle 10 as seen from above, and shows the distance threshold for determining the object 20 existing around the vehicle 10. For example, as shown in FIG. 2, when the object 20 (such as another vehicle or a person) in front of the vehicle 10 approaches the vehicle 10 to a distance of the threshold TH0, the advanced safety system is activated, and warnings and automatic brakes by the collision damage reduction brake are activated. In order for the driver to record a near miss situation, it is conceivable to record data when the advanced safety system is activated.

[0023] On the other hand, even when the object 20 around the vehicle 10 approaches the threshold TH0, the advanced safety system may not be activated. For example, when the vehicle speed exceeds a predetermined value, it may be preset so that the collision damage reduction brake does not operate from a safety perspective. In this case, even if the driver of the vehicle 10 has a near miss due to the approach of the object 20, the data may not be recorded because the advanced safety system is not activated.

[0024] Also, even when the advanced safety system is not activated or an accident has not actually occurred, it is beneficial to record and analyze information in a situation where the driver has a near miss. In addition, there is a situation where regulations are being considered in the United Nations regulations to require reporting when monitoring the occurrence of an incident.

[0025] As described above, when recording is performed based on the approach of the vehicle 10 and the object 20, recording is carried out even in a situation intended by the driver, that is, a situation known to the driver, and unnecessary data is recorded. Examples of situations intended by the driver include, for example, approaching an adjacent parked vehicle or a wall when parking in a parking lot, and approaching a barrier when passing through a highway gate. Also, examples of situations intended by the driver include cases where the vehicle 10 approaches a person or an object when a person is getting on the vehicle or when loading an object, and when the vehicle 10 is traveling at a low speed and stopped.

[0026] In the present embodiment, in a situation where the advanced safety system is not operating, except for cases where it is a situation intended by the driver, situations where the driver has a near miss are recorded. As an example, the range of the distance to the object 20 that is the recording target in the present embodiment is, for example, the range of the threshold TH1 before and after the vehicle with hatching in FIG. 2 and the range of the threshold TH2 on the side of the vehicle, and is a range narrower than the range in which the advanced safety system operates. In the present embodiment, when the object 20 approaches within the hatched range in FIG. 2, information on situations where the driver has a near miss is recorded, except for cases where it is a situation intended by the driver. The information to be recorded is information related to the operation of the vehicle in a situation where the driver has a near miss. For example, the information to be recorded may be an image representing the environment around the vehicle generated by the in-vehicle camera 110 in a situation where the driver has a near miss. Also, the information to be recorded may be information indicating the driving state of the vehicle (vehicle speed, vehicle acceleration and deceleration, accelerator opening, brake pedal depression amount, steering position, shift position, rotational speed of the internal combustion engine or motor, coolant water temperature, etc., information related to the operation of the advanced safety system) in a situation where the driver has a near miss. Also, the information to be recorded may be the position information of the vehicle in a situation where the driver has a near miss. Note that even in a situation where the advanced safety system is operating, information related to the operation of the vehicle is recorded.

[0027] FIG. 3 is a schematic diagram showing a functional block of the processor 152 of the ECU 150 for realizing the above-described processing. The processor 152 of the ECU 150 includes a distance determination unit 152a, an operation determination unit 152b, a recording unit 152c, and a driving support unit 152d. Each of these units included in the processor 152 is a functional module realized by, for example, a computer program operating on the processor 152. That is, the functional block of the processor 152 is composed of the processor 152 and a program (software) for operating the same. Further, the program may be recorded in the memory 154 provided in the ECU 150 or a recording medium connected externally. Alternatively, each of these units included in the processor 152 may be a dedicated arithmetic circuit provided in the processor 152.

[0028] The distance determination unit 152a of the processor 152 determines the distance between the vehicle and an object existing around the vehicle based on an image representing the environment around the vehicle generated by the in-vehicle camera 110 and information detected by the peripheral monitoring sensor 140. At this time, for example, by template matching between a template image and the image generated by the in-vehicle camera 110, or by inputting the image generated by the in-vehicle camera 110 to an identifier learned for object detection, an object is detected from the image, and the distance from the disparity between the left and right images to the object in the image is obtained.

[0029] Note that as the above-described identifier, the distance determination unit 152a can use, for example, a segmentation identifier that is pre-learned to output, for each type of object that may be represented by each pixel of the input image, the probability that the object is represented by the pixel, and identify that the object with the maximum probability is represented. As such an identifier, the distance determination unit 152a can use, for example, a deep neural network (DNN) having an architecture of a convolutional neural network type (CNN) for segmentation, such as a Fully Convolutional Network (FCN).

[0030] Based on the information detected by the operation information acquisition sensor 120, the operation determination unit 152b of the processor 152 determines whether an operation by the driver of the vehicle has been performed on the operation device for driving the vehicle.

[0031] When the recording unit 152c of the processor 152 determines, based on the operation determination unit 152b, that no operation has been continuously performed during the first hour before the time when the distance between the vehicle and the target becomes equal to or less than a predetermined value, and that an operation has been performed during the second hour after that time, the recording unit 152c records information related to the driving of the vehicle in the memory 154. The recorded information related to the driving of the vehicle can be used for analyzing the driving state of the vehicle, the driver's operation, etc. The recorded information related to the driving of the vehicle may be transmitted outside the vehicle.

[0032] The driving support unit 152d of the processor 152 supports the driver's driving by the advanced safety system. Based on the image representing the environment around the vehicle generated by the in-vehicle camera 110 and the information detected by the peripheral monitoring sensor 140, the driving support unit 152d controls the vehicle control device 130 and also controls the warning device 160 to support the driver's driving.

[0033] When the distance between the vehicle and a target in the vehicle's surroundings reaches a predetermined threshold value, the driving support unit 152d performs driving support by the collision mitigation brake. For example, when the driving support unit 152d detects a target such as another vehicle or a person in front of the vehicle based on the image generated by the in-vehicle camera 110 or the information detected by the peripheral monitoring sensor 140 and determines that there is a possibility of collision with the target, the driving support unit 152d causes the warning device 160 to output a warning and also activates the automatic brake in the braking device. In this case, when the distance from the vehicle to the target becomes equal to or less than the threshold value TH0 shown in FIG. 2, the driving support unit 152d outputs a warning and also activates the automatic brake.

[0034] FIG. 4 and FIG. 5 are flowcharts showing the processing performed by the processor 152 of the ECU 150 at each predetermined control cycle. FIG. 4 is a flowchart for setting a driver operation monitoring flag by target detection, and FIG. 5 is a flowchart for determining driver operation after target detection.

[0035] First, in step S10 of FIG. 4, the distance to a target around the vehicle is acquired. Next, it is determined whether the collision damage mitigation brake has been activated (step S12). If the collision damage mitigation brake has not been activated, the distance determination unit 152a determines whether the condition that the distance between the vehicle and the target ahead is less than the threshold value TH1 has changed from non - established to established (step S14). The threshold value TH1 is, for example, 1 [m]. On the other hand, if it is determined in step S12 that the collision damage mitigation brake has been activated, the processing in this control cycle ends.

[0036] If, in step S14, the condition that the distance to the target ahead is less than the threshold value TH1 has not changed from non - established to established, the distance determination unit 152a determines whether the condition that the distance between the vehicle and the target on the side is less than the threshold value TH2 has changed from non - established to established (step S16). The threshold value TH2 is, for example, 0.5 [m]. If, in step S16, the condition that the distance to the target on the side is less than the threshold value TH2 has not changed from non - established to established, the processing in this control cycle ends.

[0037] If, in step S14, the condition that the distance to the target ahead is less than the threshold value TH1 has changed from non - established to established, or if, in step S16, the condition that the distance to the target on the side is less than the threshold value TH2 has changed from non - established to established, the operation determination unit 152b determines whether there has been a driver operation during the most recent T1 time period (step S18). If the condition in step S18 is not satisfied, that is, if there has been no continuous driver operation during the T1 time period, the driver operation monitoring flag is set to ON (step S20). On the other hand, if there has been a driver operation during the T1 time period, the processing in this control cycle ends without setting the driver operation monitoring flag to ON.

[0038] In the flowchart of FIG. 5, first, it is determined whether the driver operation monitoring flag is off (step S30). If the driver operation monitoring flag is not off, that is, if the driver operation monitoring flag is on, it is determined whether T2 hours have elapsed since the driver operation monitoring flag became on (step S32). If T2 hours have not elapsed since the driver operation monitoring flag became on, the operation determination unit 152b determines whether there has been a driver operation during the T2 hours (step S34).

[0039] If there has been a driver operation during the T2 hours in step S34, the recording trigger is established, and the recording unit 152c records information related to the vehicle operation in the memory 154 (step S36). After step S36, the driver operation monitoring flag is set to off (step S38), and after step S38, the processing in this control cycle ends. Also, if the driver operation monitoring flag is off in step S30, or if T2 hours have elapsed since the driver operation monitoring flag became on in step S32, the processing in this control cycle ends.

[0040] FIG. 6 is a diagram for explaining whether the recording trigger is established according to the driver's operation pattern by the processes of FIGS. 4 and 5. In FIG. 6, at the uppermost stage, it shows how the distance between the vehicle and the object in front of the vehicle changes with the passage of time, and the state where the object approaches the vehicle (characteristic C). Also, patterns I to V shown in FIG. 6 are the driver's operation patterns when the object approaches the vehicle according to characteristic C. Further, in the right column of FIG. 6, for each of patterns I to V, it shows whether the recording trigger is established in step S34 of FIG. 5.

[0041] As shown in characteristic C of FIG. 6, as time passes, an object in front of the vehicle approaches, and at time t1, the distance between the vehicle and the object reaches the threshold TH1. In pattern I, there is no continuous driver operation within the T1 hours before time t1, and there is driver operation within the T2 hours after time t1. In this case, since the object approaches the vehicle, there is a possibility that the driver made an operation after time t1 due to a sense of unease. In the case of pattern I, the determination in step S18 of FIG. 4 is NO, and the determination in step S34 of FIG. 5 is YES, so the recording trigger is established in step S36.

[0042] In pattern II, driver operation is performed before time t1, but the driver operation is not continuously performed during the T1 hours. And there is driver operation within the T2 hours after time t1. In this case, since the object approaches the vehicle, there is a possibility that the driver made an operation due to a sense of unease before time t1. Also in the case of pattern II, the determination in step S18 of FIG. 4 is NO, and the determination in step S34 of FIG. 5 is YES, so the recording trigger is established in step S36.

[0043] In pattern III, since there is no driver operation before and after time t1, the driver is aware that an object in front of the vehicle is approaching, and it is highly likely to be a situation intended by the driver, such as during autonomous driving. In the case of pattern III, the determination in step S18 of FIG. 4 is NO, and the determination in step S34 of FIG. 5 is NO, so the recording trigger in step S36 is not established.

[0044] In pattern IV, the driver operation is continuously performed during the T1 hours before time t1, and the driver operation is also performed within the T2 hours after time t1. In this case, since the operation is performed before time t1, the driver is aware that an object in front of the vehicle is approaching and has made an intentional operation, and it is highly likely to be a situation intended by the driver. In the case of pattern IV, the determination in step S18 of FIG. 4 is YES, and since the driver operation monitoring flag is not turned on (ON), the recording trigger in step S36 of FIG. 5 is not established.

[0045] In Pattern V, the driver's operation is continuously performed within T1 hours before time t1, and the driver's operation continues even after time t1. In this case, from before time t1 to after time t1, the driver is performing an intentional operation, and it is highly likely that it is the situation intended by the driver. Also in the case of Pattern V, the determination in step S18 of FIG. 4 becomes YES, and since the driver operation monitoring flag does not turn on (ON), the recording trigger in step S36 of FIG. 5 does not hold.

[0046] As described above, according to the present embodiment, when the advanced safety system does not operate, information related to the driving of the vehicle in a situation where the driver has a near miss is recorded, excluding the situation intended by the driver. Therefore, unnecessary information in the situation intended by the driver is not recorded, and it becomes possible to reduce the amount of information to be recorded. As a result, the capacity of a recording medium such as the memory 154 for recording information can be reduced, and the communication volume when transmitting and receiving information can also be reduced. In addition, since the hardware of the existing system 1000 can be used without adding new components, the introduction cost is suppressed.

Explanation of Signs

[0047] 10 Vehicle 20 Target 110 In-vehicle camera 120 Operation information acquisition sensor 130 Vehicle control device 140 Peripheral monitoring sensor 150 Electronic control unit (ECU) 152 Processor 152a Distance determination unit 152b Operation determination unit 152c Recording unit 152d Driving support unit 154 Memory 156 Communication interface 160 Warning device 170 Positioning information receiver 1000 Vehicle control system

Claims

1. a distance determination unit that determines the distance between the vehicle and an object existing around the vehicle; an operation determination unit that determines whether an operation by the driver of the vehicle has been performed on an operating device for driving the vehicle; a recording unit that records information related to the operation of the vehicle when it is determined that the operation has not been continuously performed during a first time period before the time when the distance between the vehicle and the object becomes equal to or less than a predetermined value, and it is determined that the operation has been performed during a second time period after that time point; An information recording device for a vehicle, comprising the above components.

2. The information recording device for a vehicle according to claim 1, wherein the information related to the operation of the vehicle is image information obtained by photographing the periphery of the vehicle or position information of the vehicle.

3. The information recording device for a vehicle according to claim 1, wherein the information related to the operation of the vehicle is information indicating the driving state of the vehicle.

Images