Drive recorder system

Abstract

To record images around a vehicle while reducing installation cost, when a dangerous state of the vehicle occurs.SOLUTION: A drive recorder system 1 comprises a front camera 2 for capturing an image ahead of a vehicle VH and a video recorder 5 for recording the image captured by the front camera 2. The front camera 2 comprises, as a functional configuration of an ECU 25: a determination part 251 for determining generation of a dangerous situation of the vehicle VH based on the captured image; and a signal transmission part 252 for transmitting a control signal S1 to the front camera 2 based on a determination result DR of the determination part 251. The signal transmission part 252 transmits the control signal S1 for starting recording of the image in the video recorder 5 when the determination part 251 determines the generation of the dangerous situation. The signal transmission part 252 transmits the control signal S1 for stopping recording of the image in the video recorder 5 when the determination part 251 determines the end of the dangerous situation.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a drive recorder system.

Background Art

[0002] A drive recorder system records the video around a vehicle captured by a photographing device in a recording device (see, for example, Patent Document 1). The video around the vehicle can be used as evidence, for example, when a dangerous situation such as an accident occurs in the vehicle, or for improving the safety measures of the vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to reliably record the dangerous situation of a vehicle, it is conceivable to always perform recording during the driving of the vehicle in a drive recorder system. However, when always performing recording, a large recording capacity is required for the recording device, and the installation cost increases. In a drive recorder system, it is required to record the video around the vehicle when a dangerous situation of the vehicle occurs while reducing the installation cost. Note that, not limited to such a requirement, the actions and effects derived from each configuration disclosed in the "Mode for Carrying Out the Invention" described later, and which are actions and effects not obtained by the conventional technology, can also be positioned as other objects of the present case.

Means for Solving the Problems

[0005] The present invention is a photographing device that photographs the surroundings of a vehicle, a recording device that records the video photographed by the photographing device, and includes The imaging device is equipped with a determination unit that determines the occurrence of a dangerous situation of the vehicle based on the captured video, and a signal transmission unit that transmits a control signal to the imaging device based on the determination result of the determination unit. The signal transmission unit is configured as a drive recorder system that, when the determination unit determines the occurrence of the dangerous situation, transmits a control signal to start recording the video to the recording device, and when the determination unit determines the end of the dangerous situation, transmits a control signal to stop recording the video to the recording device.

Advantages of the Invention

[0006] According to the present invention, it is possible to record the video around the vehicle when a dangerous situation of the vehicle occurs while reducing the installation cost.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Embodiments for Carrying Out the Invention

[0008] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the embodiment, an example of applying the drive recorder system to a vehicle equipped with an automatic driving system based on a smart camera will be described. Note that the automatic driving includes both fully automatic driving and partial automatic driving such as ADAS (Advanced Driving Assistance System). FIG. 1 is a schematic diagram of a vehicle VH equipped with a drive recorder system 1. FIG. 2 is a block diagram showing the configuration of the drive recorder system 1. Hereinafter, the arrangement relationship of each element constituting the drive recorder system 1 will be described with reference to the front-rear direction, left-right direction (vehicle width direction), and up-down direction (the direction of the front side and the back side of the paper surface) shown in FIG. 1. As shown in FIGS. 1 and 2, the drive recorder system 1 includes a front camera 2 (imaging device) that is a smart camera. The front camera 2 captures the front (surrounding) of the vehicle VH. The drive recorder system 1 includes a recording device 5 that records the video captured by the front camera 2. The drive recorder system 1 can include at least one sensor 3 that detects an object around the vehicle VH. The drive recorder system 1 applied to the vehicle VH equipped with an automatic driving system can include a display device 4 that displays the video of the front camera 2 in real time, as shown in FIG. 2.

[0009] The front camera 2 and the sensor 3 are connected via an in-vehicle network such as CAN / Flexray or Ethernet, for example, so that data can be transmitted and received. The front camera 2 is connected to the display device 4 and the recording device 5 via, for example, an LVDS cable or a coaxial cable for video output. The front camera 2 can output the captured video as a video signal VS to the display device 4 and the recording device 5. Further, the front camera 2 and the recording device 5 are connected via an in-vehicle network, and data can be transmitted and received.

[0010] The front camera 2 can be installed, for example, at the upper part of the front glass of the vehicle VH. The front camera 2 captures the front of the vehicle VH and acquires sensor data SD from the sensor 3. In the automatic driving system, the front camera 2 can execute various functions for controlling actuators related to automatic driving by using the captured video and the sensor data SD acquired from the sensor 3. The actuators are, for example, brakes, electric power steering, engines, vehicle driving motors, etc. The front camera 2 can use the sensor data SD, for example, to confirm objects within the ROI (Region of Interest). Here, the ROI means the area in front of the vehicle, which is the shootable range of the front camera 2. Alternatively, the front camera 2 can support functions outside the ROI by using the sensor data SD.

[0011] The sensor 3 is for detecting objects around the vehicle VH and can be, for example, a camera, radar, ultrasonic sensor, etc. The sensor 3 can be distributed and arranged at various locations on the vehicle VH so as to cover 360° around the vehicle VH. In FIG. 1, cameras (front camera 2, rear camera 31, and corner cameras 32) are shown as white squares, radars (front radar 33 and corner radars 35) are shown as hatched squares, and the ultrasonic sensor 36 is shown as a circle. Also, in FIG. 1, the shootable ranges of the cameras (front camera 2, rear camera 31, and corner cameras 32) are shown by solid lines, the detectable ranges of the radars (front radar 33 and corner radars 35) are shown by dashed-dotted lines, and the detectable range of the ultrasonic sensor 36 is shown by dashed lines. Note that the detectable range of the ultrasonic sensor 36 shows the integrated detectable ranges of a plurality of ultrasonic sensors 36 provided on the front side and the rear side of the vehicle VH, respectively. The types, numbers, and positions of the sensors 3 provided in the drive recorder system 1 are not limited to the illustrated examples and can be changed as appropriate.

[0012] The camera can be installed, for example, at positions where it can shoot areas such as the rear, side, and interior of the vehicle VH. Objects such as the vehicle VH, pedestrians, obstacles, signals, signs, lights, weather, brightness, etc. can be detected from the video captured by the camera.

[0013] Radar and ultrasonic sensors can detect objects such as other vehicles, pedestrians, and obstacles, for example. The radar transmits millimeter-wave radio waves around the vehicle VH, for example. The radar detects an object by receiving the radio waves reflected by the object and can acquire data on the distance or direction to the object. The radar is particularly useful for detecting objects at a long distance from the vehicle VH in the front-rear direction of the vehicle VH. As shown in FIG. 1, the photographable range (solid line) of a general camera is about 100°, and the radar has a wider detectable range (dashed-dotted line) than a general camera. Also, the radar is less expensive than the camera. As shown in FIG. 1, by providing a plurality of radars on the vehicle VH, a wide range and distance can be covered.

[0014] The ultrasonic sensor transmits ultrasonic waves around the vehicle VH. The ultrasonic sensor detects an object by receiving the ultrasonic waves reflected by the object and can acquire data on the distance or direction to the object. The ultrasonic sensor is suitable for detecting objects close to the vehicle VH and can detect objects in a range that is a blind spot for the camera and radar. Also, the ultrasonic sensor is less expensive than the camera. As shown in FIG. 1, by providing a plurality of ultrasonic sensors 36, a wide range can be covered. In this way, by installing a plurality of types of sensors 3 at various locations on the vehicle VH, it becomes possible to cover 360° around the vehicle VH.

[0015] In FIGS. 1 and 2, the following examples are illustrated as the sensor 3. · Rear camera 31... Installed at the upper part of the rear glass of the vehicle VH or the like, and photographs the rear of the vehicle VH. · Corner camera 32... Installed on the left and right doors of the vehicle VH or the like, and photographs the side of the vehicle VH. · Front radar 33... It is arranged on the front side of the vehicle VH. It is used as a redundant sensor for the front camera 2 that monitors the front of the vehicle. · Corner radar 35... It is arranged at each of the four corners of the vehicle VH. It is used for monitoring traffic across the vehicle VH and supporting the lane change function. · Rear radar 34... It is arranged on the rear side of the vehicle VH. It is used for monitoring pedestrians behind the vehicle VH and the vehicle VH (in Fig. 1, the illustration of the rear radar 34 is omitted). · Ultrasonic sensor 36... A plurality of ultrasonic sensors 36 are arranged at intervals on each of the front and rear sides of the vehicle VH. It is used for supporting the low-speed operation and parking function of the vehicle VH. The drive recorder system 1 can be equipped with, as other sensors 3, for example, a driver monitoring camera that monitors the occupants inside the vehicle VH, a LiDAR (Light Detection And Ranging) that irradiates a laser that emits light in pulses to detect an object, and the like.

[0016] The display device 4 (see Fig. 2) can be composed of, for example, a display installed at a position visible to the occupants, such as the dashboard of the vehicle VH. The display device 4 can display the video captured by the front camera 2 in real time. Although detailed description is omitted, the display device 4 can display various information in addition to the video captured by the front camera 2, for example, as an infotainment display of the autonomous driving system.

[0017] The recording device 5 includes a storage 51 capable of recording video in HD (High Definition) quality and at a high fps (frames per second) rate. The storage 51 can be, for example, an SSD (Solid State Drive) or an HD (Hard Disk). The recording capacity of the recording device 5 can be expanded, for example, up to several hundred GB according to the user's requirements. Although details will be described later, in this embodiment, the recording device 5 starts and stops recording video using the control signal S1 from the front camera 2 as a trigger. As shown in FIG. 1, the recording device 5 can be installed, for example, in the trunk room or the like at the rear side of the vehicle VH, away from the front camera 2. In this case, for example, even if the front camera 2 installed on the windshield is damaged in a collision accident or the like, the possibility that the recording device 5 is spared from damage is increased.

[0018] As shown in FIG. 2, the front camera 2 includes a lens 21, an imaging element 22, a serializer 23, a SoC 24 (System on a chip), an ECU 25 (Electronic Control Unit), and the like. The front camera 2 also includes a video output connector 26 and a vehicle-mounted connector 27 for connecting to the vehicle network. The vehicle network can be, for example, CAN / Flexray or Ethernet.

[0019] The lens 21, which is an optical element, is installed facing the front of the vehicle. The lens 21 can use, for example, a wide-angle lens. The imaging element 22 is composed of a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, or the like. The imaging element 22 converts the incident light from the lens 21 into a video signal VS, which is an electrical signal.

[0020] As shown in FIG. 2, the imaging device 22 is connected to the serializer 23 and the SoC 24 via an interface such as MIPI (Mobile Industry Processor Interface). The imaging device 22 divides and outputs the video signal VS to the serializer 23 and the SoC 24 respectively.

[0021] The serializer 23 serializes the video signal VS input from the imaging device 22, and also performs signal conditioning and format conversion to LVDS (Low voltage differential signaling) or coaxial format, Ethernet, MPEG4, etc. The serializer 23 outputs the converted video signal VS to the display device 4 and the recording device 5 via an interface that supports LVDS or coaxial format, a video output connector 26, an LVDS cable or a coaxial cable, etc.

[0022] The SoC 24 (analysis unit) can use one in which an algorithm for video analysis is incorporated by prior machine learning. The SoC 24 is connected to the ECU 25 via an I2C / SPI-based interface or the like. The SoC 24 analyzes the video signal VS input from the imaging device 22 and extracts elements for determining the dangerous situation of the vehicle VH in the ECU 25. The elements are not limited to specific ones, but examples include the following. · Pedestrians (number, direction, distance from the vehicle VH) · Other vehicles (number, direction, distance from the vehicle VH) · Obstacles such as curbs (type, number, direction, distance from the vehicle VH) · Signals (color of the light) · Signs (types of signs such as no entry, slow driving) · Weather (sunny, rainy, snowy) · Brightness (dawn, darkness) The SoC 24 outputs the data of the analysis result AR to the ECU 25.

[0023] The ECU 25 controls the operation of the front camera 2. The ECU 25 can also operate as an ADAS-ECU that constitutes an ADAS (Advanced Driver Assistance System, Advanced Driving Assistance System) used in the autonomous driving system. The ADAS includes various functions for controlling autonomous driving, and the ECU 25 can be assumed to incorporate algorithms capable of executing those functions. Further, in the drive recorder system 1 of the present embodiment, the ECU 25 uses the algorithm for executing the functions of the ADAS to determine the dangerous situation of the vehicle VH, which serves as a trigger for starting and stopping recording in the recording device 5.

[0024] Figure 3 is a block diagram showing the functional configuration of the ECU 25 in the drive recorder system 1. The ECU 25 includes a determination unit 251 and a signal transmission unit 252. The determination unit 251 determines the dangerous situation of the vehicle VH based on the analysis result AR input from the SoC 24. The signal transmission unit 252 transmits a control signal S1 to the recording device 5 based on the determination result DR of the determination unit 251. The ECU 25 also accesses the sensor 3 via the in-vehicle connector 27 and the in-vehicle network to obtain sensor data SD. When the sensor 3 is a radar or ultrasonic sensor, the sensor data SD can be numerical data. When the sensor 3 is a camera, the sensor data SD can be a video signal.

[0025] The determination unit 251 can determine the dangerous situation of the vehicle VH using the sensor data SD together with the analysis result AR of the SoC 24. The signal transmission unit 252 generates a control signal S1 for controlling the recording device 5 according to the determination result DR of the dangerous situation by the determination unit 251. The control signal S1 can include, for example, the following contents. · Trigger for starting recording · Trigger for stopping recording · Tag ID / Synchronization index (hereinafter simply referred to as "tag") · Functional data related to the determination of the dangerous situation · Sensor data SD

[0026] The above content does not need to be included in one control signal S1 and may be transmitted separately or in parallel. The signal transmission unit 252 can transmit, for example, the control signal S1 that serves as a trigger for recording start and stop to the recording device 5 as a binary signal of a High signal and a Low signal. When the determination unit 251 determines the occurrence of a dangerous situation, the signal transmission unit 252 outputs a High signal to the recording device 5. When the determination unit 251 determines the end of the dangerous situation, the signal transmission unit 252 switches from the High signal to a Low signal and outputs it. When the binary signal input from the determination unit 251 switches from the Low signal to the High signal, the recording device 5 starts recording the video. When the binary signal input from the determination unit 251 switches from the High signal to the Low signal, the recording device 5 ends the recording of the video.

[0027] The signal transmission unit 252 may, for example, store and transmit tags, function data, and sensor data SD in the data frame of the same control signal S1. While the signal transmission unit 252 is outputting a High signal that serves as a trigger for recording start, it may transmit a data frame including tags, function data, and sensor data SD in parallel. Alternatively, after the signal transmission unit 252 switches from the High signal to a Low signal that serves as a trigger for recording stop, it may transmit a data frame including tags, function data, and sensor data SD.

[0028] The tag is for identifying the type of dangerous situation determined by the determination unit 251 for the video recorded in the recording device 5 and synchronizing the function data and sensor data SD with the video. The function data related to the determination of the dangerous situation is data for the determination unit 251 to grasp the content of the function used for determining the dangerous situation. Below, the types of dangerous situations determined in the ECU 25 are exemplified together with the ADAS functions for determining the dangerous situations. (a) When the Collision Avoidance system operates The collision avoidance system is a function for avoiding collisions of the vehicle VH with objects (other vehicles, obstacles, pedestrians, etc.). The collision avoidance system includes, for example, the following functions. · RCTA (Rear Cross Traffic Alert) RCTA is a function for detecting other vehicles approaching from the left and right of the vehicle VH when the vehicle VH starts to move backward from a stop position. In particular, when there are many obstacles around the stop position of the vehicle VH, it may be difficult to detect approaching other vehicles from the camera image. In such a case, by using the sensor data SD of the radar, it becomes easier to detect approaching other vehicles. · FCTA (Front Cross Traffic Alert) FCTA is a function for detecting other vehicles approaching from the left and right of the vehicle VH when the vehicle VH enters an intersection or the like. FCTA can use, for example, the image captured by the front camera 2 and the sensor data SD of the front radar 33 and the corner radar 35. In particular, at intersections or the like where there are many obstacles and the visibility is poor, it may be difficult to detect approaching other vehicles from the camera image. Even in such a case, by using the sensor data SD of the radar, it becomes easier to detect approaching other vehicles. · ACC (Adaptive Cruise Control System) ACC is a function for detecting a vehicle traveling ahead and causing the vehicle VH to travel while maintaining a certain inter-vehicle distance. ACC can use, for example, the image captured by the front camera 2 and the sensor data SD of the front radar 33. · AEB (Advanced Emergency Braking) AEB is a function for urgently braking the vehicle VH in order to reduce damage caused by collisions with other vehicles or obstacles. AEB can use, for example, the image captured by the front camera 2 or the rear camera 31, etc., and the sensor data SD of the front radar 33, the corner radar 35, the rear radar 34, etc. ·PMP (Pedal Misapplication Prevention) PMP is a function that suppresses acceleration caused by misapplying the pedal. PMP operates, for example, when the vehicle VH is stopped or traveling at a low speed (including reverse travel) and an object approaching the vehicle VH is detected. Under these conditions, if the driver of the vehicle VH accidentally depresses the accelerator pedal instead of the brake pedal, PMP automatically suppresses the acceleration of the vehicle VH, reducing the possibility of the vehicle VH colliding with the object or reducing the damage in case of a collision. Since PMP needs to detect an approaching object within the range of 0.4 to 3.0 m of the vehicle VH, in particular, the sensor data SD of the ultrasonic sensor 36 is useful. The ultrasonic sensor 36 can also cover the proximity range of the vehicle VH that is a blind spot for the camera. The ultrasonic sensor 36 can also detect objects that are difficult to detect from the image of the camera on a glass surface or the like. Also, since the ultrasonic sensor 36 is relatively inexpensive, a wide range can be covered by arranging a plurality of ultrasonic sensors 36. ·BSW (Blind Spot Warning) BSW is a function that notifies the driver when there is another vehicle in the driver's blind spot when changing lanes. BSW uses, for example, sensor data SD such as the corner radar 35 and the rear radar 34 to detect other vehicles approaching the vehicle VH from the adjacent lane while traveling in the adjacent lane and warns the driver. In the situation (a) described above, the signal transmission unit 252 transmits function data related to, for example, RCTA, FCTA, AEB, PMP, etc. to the recording device 5.

[0029] (b) Aggressive cut-in and cut-out An aggressive cut-in, for example, means a situation where another vehicle suddenly overtakes and cuts in front of the vehicle VH on the road. An aggressive lane change, for example, means a situation where another vehicle running in front of the vehicle VH suddenly changes lanes without operating the turn signal. These situations can be determined by the functions of the ACC described above. In the situation of (b), the signal transmission unit 252 transmits, for example, function data related to ACC or the like to the recording device 5.

[0030] (c) When the host vehicle VH changes lanes without operating the turn signal, when the road departure monitoring system operates, when changing lanes in a situation where there is another vehicle running at a slower speed than the host vehicle VH in the target lane These situations can be determined by functions that assist the steering operation of the vehicle VH to maintain the lane in which the vehicle VH is traveling, such as LKAS (Lane keeping assist system) or LCC (lane centering control). These functions can detect, for example, the lane in which the vehicle VH is traveling and adjacent lanes using images from the front camera 2 or the corner camera 32. In particular, when detecting the speed of another vehicle traveling in the target lane, the sensor data SD of the corner radar 35 is useful. In the situation of (c), the signal transmission unit 252 transmits, for example, function data related to LKAS, LCC, etc.

[0031] (d) Reverse driving of the host vehicle VH or another vehicle Reverse driving can be determined by functions such as TSR (Traffic Sign Recognition), ISA (Intelligent Speed Assistance), LKAS, LCC described above. TSR is, for example, a function of recognizing traffic signs indicating speed limits, entry prohibitions, etc. from images captured by a camera, and ISA is a function of assisting in adjusting the speed of the vehicle VH in accordance with traffic signs in conjunction with TSR. For example, when the vehicle VH enters a road where a no-entry sign is detected from the video captured by the camera, the ECU 25 can determine that the vehicle VH is reversing. Also, when the ECU 25 detects another vehicle approaching the vehicle VH, it can determine that the other vehicle is reversing. In order to determine the reverse movement of the other vehicle before it approaches the vehicle VH, it is particularly useful to use the sensor data SD of a radar capable of long-distance detection. In the situation of (d), the signal transmission unit 252 transmits, for example, function data related to TSR / ISA, LKAS, LCC, etc. to the recording device 5.

[0032] (e) Sudden stop of the preceding vehicle The sudden stop of the preceding vehicle can be determined, for example, by the function of object tracking. Object tracking is a function of tracking many objects (other vehicles, pedestrians, buildings, etc.) around the vehicle VH using, for example, camera image processing or LiDAR. It is possible to measure the distance between the vehicle VH and the preceding vehicle and calculate the speed of the preceding vehicle by means of machine learning or LiDAR. In the situation of (e), the signal transmission unit 252 transmits, for example, function data related to object tracking, etc. to the recording device 5.

[0033] When the determination unit 251 determines a dangerous situation, the signal transmission unit 252 includes the sensor data SD acquired from the sensor 3 in the control signal S1. In addition to the sensor data SD used for determining the dangerous situation, the signal transmission unit 252 can also include in the control signal S1 the sensor data SD that has not been used for the determination. The sensor data SD can be, for example, images or videos captured by the rear camera 31 or the corner camera 32. Alternatively, the sensor data SD can be numerical data that is the detection result of the front radar 33, rear radar 34, corner radar 35, ultrasonic sensor 36, etc.

[0034] When the determination unit 251 determines a dangerous situation, the signal transmission unit 252 acquires function data from the determination unit 251 and acquires sensor data SD from the sensor 3. The signal transmission unit 252 creates a tag corresponding to the type of dangerous situation determined by the determination unit 251. The signal transmission unit 252 creates a control signal S1 including the tag, function data, and sensor data SD, and transmits it to the recording device 5.

[0035] When the recording device 5 receives the control signal S1, it records the function data and sensor data SD together with the video in the storage 51. Specifically, the recording device 5 records the video, function data, and sensor data SD linked by a tag in the storage 51. As a result, in the recording device 5, the video of the front camera 2 when a dangerous situation occurs, the function data for determining the dangerous situation, and the sensor data SD are recorded as integrated data. Thus, when verifying a dangerous situation or submitting evidence of a dangerous situation, for example, the user does not need to extract and collate data from various devices, and can quickly access the integrated data. In addition, since the data recorded in the recording device 5 is classified by the type of dangerous situation by tags, the user can easily access the data recorded for a specific dangerous situation by performing a search using the tags.

[0036] FIG. 4 is a flowchart showing the processing flow of the ECU 25 of the front camera 2 during the running of the vehicle VH. Here, the processing of the ECU 25 in the drive recorder system 1 will be described. When the vehicle VH starts running, the front camera 2 (see FIG. 2) starts shooting, and a video signal VS is input from the image sensor 22 to each of the SoC 24 and the serializer 23. The serializer 23 performs processing such as serialization and format conversion on the video signal VS, and outputs it to the display device 4 and the recording device 5. The display device 4 displays the input video in real time. The recording device 5 does not record the video until a control signal S1 serving as a recording start trigger is input from the ECU 25. The SoC24 analyzes the video input from the imaging device 22 and inputs the analysis result AR to the ECU25 as needed.

[0037] As shown in FIG. 4, the determination unit 251 of the ECU25 determines whether a dangerous situation has occurred in the vehicle VH based on the analysis result AR input from the SoC24 (step S01). Note that the determination unit 251 may perform the determination using the sensor data SD acquired from the sensor 3 in addition to the analysis result AR of the SoC24. When the determination unit 251 determines the occurrence of a dangerous situation (step S01: Yes), the signal transmission unit 252 transmits a control signal S1 serving as a recording start trigger to the recording device 5 (step S02). The determination unit 251 continues to determine the dangerous situation of the vehicle VH based on the analysis result AR input from the SoC24 (step S03). When the determination unit 251 determines the end of the dangerous situation of the vehicle VH (step S03: Yes), the signal transmission unit 252 transmits a control signal S1 serving as a recording stop trigger to the recording device 5 (step S04). The signal transmission unit 252 transmits a control signal S1 including tags, function data, and sensor data SD to the recording device 5 (step S05). Note that the process of step S05 does not necessarily have to be performed after the process of step S04, and may be performed in parallel with the processes of steps S03 to S04 after step S02. The ECU25 repeatedly performs the processes of steps S01 to S05, for example, while the vehicle VH is running.

[0038] As described above, the drive recorder system 1 according to the present embodiment has, for example, the following configuration. (1) The drive recorder system 1 includes a front camera 2 (imaging device) that captures the front (surroundings) of the vehicle VH, and a recording device 5 that records the video captured by the front camera 2. The front camera 2 Based on the captured video, a determination unit 251 that determines the occurrence of a dangerous situation of the vehicle VH, and a signal transmission unit 252 that transmits a control signal S1 to the front camera 2 based on the determination result DR of the determination unit 251. When the determination unit 251 determines the occurrence of a dangerous situation, the signal transmission unit 252 transmits a control signal S1 to start recording the video by the recording device 5. When the determination unit 251 determines the end of the dangerous situation, the signal transmission unit 252 transmits a control signal S1 to stop recording the video by the recording device 5.

[0039] In this way, while reducing the installation cost of the drive recorder system 1, when a dangerous situation of the vehicle VH occurs, the video around the vehicle VH can be recorded by the recording device 5. For example, when the vehicle VH travels for 100 hours, the time during which a dangerous situation of the vehicle VH occurs is usually considered to be about 10 to 20 minutes. Since the drive recorder system 1 of the present embodiment records the video only when a dangerous situation occurs, the recording time is greatly reduced compared to the case where recording is constantly performed during the travel of the vehicle VH, so the recording capacity required for the recording device 5 can be reduced.

[0040] Also, in the present embodiment, the video can be recorded by a recording device 5 that is separate from the imaging device. The ECU 25 of the front camera 2 does not need to perform the recording process, and only needs to send a control signal S1 for recording start and recording stop to the recording device 5, so the processing load can be reduced. In addition, since the recording device 5 is separate from the front camera 2, it can be arranged on the rear side of the vehicle VH away from the front camera 2. As a result, for example, even when the front camera 2 installed on the windshield is damaged in a collision accident of the vehicle VH, the possibility that the recording device 5 is spared from damage is increased. Furthermore, although it may be difficult to expand the recording capacity of the front camera 2 that requires miniaturization, in the case of the recording device 5 that is separate from the front camera 2, it is relatively easy to expand the recording capacity. In the embodiment, an example in which the "imaging device" is the front camera 2 that images the front of the vehicle VH has been described. However, for example, the "imaging device" may be the rear camera 31 or the corner camera 32, and the front camera 2 may be one of the plurality of sensors 3.

[0041] In the above (1), (2) The front camera 2 includes a lens 21 and an imaging element 22. The lens 21 is installed facing the surroundings of the vehicle VH. The imaging element 22 converts the incident light of the lens 21 into a video signal VS which is an electrical signal, and divides and outputs the video signal VS into a video signal VS recorded by the recording device 5 and a video signal VS used for determining the occurrence of a dangerous situation.

[0042] In this way, by the front camera 2 dividing and outputting the video signal VS, the video captured by one camera (front camera 2) can be used for both recording by the recording device 5 and determination of a dangerous situation by the determination unit 251. Thereby, the installation cost of the camera in the vehicle VH can be reduced. In the embodiment, an example in which the imaging element 22 divides the video signal VS has been shown, but the division process may be performed by a chip or the like different from the imaging element 22.

[0043] In the above (1) or (2), (i) The front camera 2 includes a SoC 24 (analysis unit) that analyzes the video output from the imaging element 22 using an algorithm set by machine learning and extracts elements for determining the dangerous situation of the vehicle VH. The determination unit 251 determines the dangerous situation of the vehicle VH using the analysis result AR of the SoC 24.

[0044] The video signal VS has a large data size. If the video is directly input to the ECU25 for analysis and processing, the processing load on the ECU25 will increase. When the ECU25 also functions as the ADAS-ECU of the autonomous driving system, the processing load will be even greater. In this embodiment, upstream of the ECU25, the machine learning-trained SoC24 analyzes the video. As a result, only the analysis result AR, which is numerical data, is input to the ECU25, so the processing load on the ECU25 can be reduced.

[0045] In the above (1), (2) or (i), (3) The drive recorder system 1 is provided in the vehicle VH and includes at least one sensor 3 capable of detecting objects around the vehicle VH. When the determination unit 251 of the signal transmission unit 252 of the ECU25 determines the occurrence of a dangerous situation, it transmits a control signal S1 to the recording device 5 to cause the sensor data SD (data of the sensor 3) to be recorded together with the video captured by the front camera 2.

[0046] With this configuration, the recording device 5 comprehensively records the sensor data SD for detecting objects around the vehicle VH together with the video in which the dangerous situation of the vehicle VH is determined. As a result, the user does not need to collect the data, and it becomes possible to analyze the dangerous situation that occurred in the vehicle VH from various data in a multi-faceted manner. In the embodiment, an example in which the signal transmission unit 252 includes the sensor data SD in the control signal S1 and transmits it has been described. However, in the drive recorder system 1, the sensor data SD may be directly input to the recording device 5. In this case, the signal transmission unit 252 can transmit a control signal S1 that serves as a trigger for starting and stopping the recording of the sensor data SD to the recording device 5.

[0047] In the above (3), (4) The sensor 3 can be a radar or an ultrasonic sensor 36. The determination unit 251 determines the occurrence of a dangerous situation using the data of the radar or ultrasonic sensor 36. The radar can be, for example, a front radar 33, a rear radar 34, a corner radar 35, etc.

[0048] The shooting range of a general camera is about 120°. For example, if a wide-angle fish-eye lens is used, the shooting range becomes about 150°, but the processing of the captured video may become complicated. The radar has a wider detectable range than a general camera and is less expensive than a camera. The radar is further suitable for detecting objects at a long distance from the vehicle VH in the front-rear direction of the vehicle VH. The ultrasonic sensor is suitable for detecting objects close to the vehicle VH and can detect objects in a range that is a blind spot for the camera and radar. Also, the ultrasonic sensor is less expensive than a camera, and by providing a plurality of ultrasonic sensors 36, a wide range can be covered. In the present embodiment, by using a radar (such as a front radar 33, a rear radar 34, a corner radar 35, etc.) or an ultrasonic sensor 36 as the sensor 3, it becomes easier to determine a dangerous situation that is difficult to determine only from the video of the front camera 2, or a dangerous situation that occurs outside the shooting range of the front camera 2. For example, the radar is suitable for detecting a vehicle VH approaching from the left and right of the vehicle VH when entering an intersection with many blind spots. Also, for example, since the ultrasonic sensor is suitable for detecting an object close to the vehicle VH, it is suitable for determining a misstep of the pedal. In this way, in the drive recorder system 1, by using sensor data SD such as a radar or an ultrasonic sensor to determine a dangerous situation, various dangerous situations can be recorded in the recording device 5.

[0049] In the above (3) or (4), (5) The sensor 3 can be a camera capable of shooting a range outside the shooting range of the front camera 2. The camera can be, for example, a rear camera 31, a corner camera 32, etc. When the determination unit 251 determines the occurrence of a dangerous situation, the signal transmission unit 252 of the ECU 25 transmits a control signal S1 to the recording device 5 to cause the recording device 5 to record the video or image captured by the camera together with the video captured by the front camera 2.

[0050] With such a configuration, the drive recorder system 1 can record, in the recording device 5, the video captured outside the shootable range of the front camera 2 together with the video captured by the front camera 2. As a result, it becomes possible to analyze the dangerous situation that has occurred in the vehicle VH from various angles based on the videos captured in various ranges, and the reliability as evidence can be enhanced. In the embodiment, an example has been described in which the signal transmission unit 252 includes the videos or images captured by the rear camera 31 and the corner cameras 32 in the control signal S1 and transmits them. However, in the drive recorder system 1, the videos or images captured by these cameras may also be directly input to the recording device 5. In this case, the signal transmission unit 252 can transmit a control signal S1 that serves as a trigger for starting and stopping the recording of the camera video or image to the recording device 5.

[0051] In any of the above (3) to (5), (6) When the determination unit 251 determines the occurrence of a dangerous situation, the signal transmission unit 252 transmits a control signal S1 including function data related to the determination of the dangerous situation to the recording device 5, and causes the recording device 5 to record the function data together with the video captured by the front camera 2.

[0052] The drive recorder system 1 can determine various types of dangerous situations, for example, by utilizing the functions of an ADAS-ECU used in an automatic driving system. In this case, by recording the function data related to the determination of the dangerous situation together with the video in which the dangerous situation is determined in the recording device 5, when the user verifies the video, it is possible to smoothly grasp what kind of dangerous situation has been determined. In addition, when the drive recorder system 1 utilizes the functions of the autonomous driving system, data related to the autonomous driving function can be recorded in the recording device 5 and utilized for application development and data training.

[0053] In any of the above (3) to (6), (7) When the determination unit 251 determines the occurrence of a dangerous situation, the signal transmission unit 252 transmits a control signal S1 including a tag for identifying the type of dangerous situation to the recording device 5, and causes the recording device 5 to record the video captured by the front camera 2, the data of the sensor 3, and the function data in association with each other using the tag.

[0054] With this configuration, the recording device 5 can record the video of the front camera 2 when a dangerous situation occurs, the function data related to the determination of the dangerous situation, and the sensor data SD as integrated data. When the user utilizes the data of various devices that have recorded dangerous situations, there is no need to collect data from each device, so the convenience can be improved. Furthermore, the data recorded in the recording device 5 is classified according to the type of dangerous situation by tags. Therefore, the user can easily access the data recorded for a specific dangerous situation by performing a search using the tags.

[0055] In any of the above (2) to (7), (i), (8) The drive recorder system 1 includes a display device 4 that displays the video captured by the front camera 2 in real time. The video signal VS output from the imaging element 22 of the front camera 2 to the recording device 5 is also input to the display device 4.

[0056] The autonomous driving system of the vehicle VH may include an in-vehicle infotainment display that displays the image captured by the camera in real time. By the imaging device 22 also outputting the video signal VS to the display device 4, the video captured by one camera (front camera 2) can be used not only for recording by the recording device 5 and determination of a dangerous situation by the determination unit 251, but also for live streaming on the display device 4. As a result, a dashboard camera for an infotainment display becomes unnecessary, and the installation cost of the camera can be reduced.

[0057] (9) In any of the above (1) to (8), (i), The front camera 2 can output a control signal to an actuator related to the automatic driving of the vehicle VH.

[0058] With such a configuration, the front camera 2 can function as an imaging device of the drive recorder system and also as a control device for an actuator related to the automatic driving of the vehicle VH. As a result, there is no need to separately provide a camera for the drive recorder system, and the installation cost of the camera can be reduced.

[0059] As described above, the embodiments and modified examples of the present invention have been explained, but the present invention is not limited to these, and can be appropriately changed within the scope of the technical idea of the invention.

Explanation of Reference Numerals

[0060] 1: Drive recorder system 2: Front camera (imaging device) 3: Sensor 4: Display device 5: Recording device 21: Lens 22: Imaging device 23: Serializer 24: SoC (analysis unit) 25: ECU 251: Determination unit 252: Signal transmission unit 26: Video output connector 27: Vehicle-mounted connector 31: Rear camera 33: Front radar 32: Corner camera 34: Rear radar 35: Corner radar 36: Ultrasonic sensor VH: Vehicle VS: Video signal AR: Analysis result S1: Control signal SD: Sensor data

Claims

1. An imaging device that captures the surroundings of a vehicle, and a recording device that records the video captured by the imaging device, The imaging device includes a determination unit that determines the occurrence of a dangerous situation of the vehicle based on the captured video, and a signal transmission unit that transmits a control signal to the imaging device based on the determination result of the determination unit, The signal transmission unit transmits a control signal to start recording the video to the recording device when the determination unit determines the occurrence of the dangerous situation, and transmits a control signal to stop recording the video to the recording device when the determination unit determines the end of the dangerous situation. A drive recorder system characterized by this.

2. In Claim 1, The imaging device includes a lens installed facing the surroundings of the vehicle, and an imaging element that converts the incident light of the lens into a video signal that is an electrical signal, The video signal is divided and output into a video signal recorded by the recording device and a video signal used for determining the occurrence of the dangerous situation. A drive recorder system characterized by this.

3. In Claim 1 or Claim 2, It is provided on the vehicle and includes at least one sensor capable of detecting an object around the vehicle, When the determination unit determines the occurrence of the dangerous situation, the signal transmission unit transmits a control signal to the recording device to record the data of the sensor together with the video captured by the imaging device. A drive recorder system characterized by this.

4. In Claim 3, The sensor is a radar or an ultrasonic sensor, The determination unit determines the occurrence of the dangerous situation using the data of the radar or the ultrasonic sensor. A drive recorder system characterized by this.

5. In claim 3, the sensor is a camera capable of photographing a range outside the photographable range of the photographing device, and when the determination unit determines the occurrence of the dangerous situation, the signal transmission unit transmits a control signal to the recording device to cause the recording device to record the video or image captured by the camera together with the video captured by the photographing device. A drive recorder system characterized by the above.

6. In claim 3, when the determination unit determines the occurrence of the dangerous situation, the signal transmission unit transmits a control signal including function data related to the determination of the dangerous situation to the recording device, and causes the recording device to record the function data together with the video captured by the photographing device. A drive recorder system characterized by the above.

7. In claim 6, when the determination unit determines the occurrence of the dangerous situation, the signal transmission unit transmits a control signal including a tag for identifying the type of the dangerous situation to the recording device, and causes the recording device to record the video captured by the photographing device, the data of the sensor, and the function data in association with the tag. A drive recorder system characterized by the above.

8. In claim 2, a display device for displaying the video captured by the photographing device in real time is provided, and the video signal output from the imaging element to the recording device is also input to the display device. A drive recorder system characterized by the above.

9. In claim 1 or claim 2, the photographing device outputs a control signal to an actuator related to the automatic driving of the vehicle. A drive recorder system characterized by the above.

Images