Recording control device for vehicle, driving support system, and recording control method

Abstract

To provide a recording control device for a vehicle, a driving support system, and a recording control method for enabling a driver to appropriately recognize that an unintended deviation of a route has occurred.SOLUTION: A recording control device for a vehicle comprises: a route information acquisition section 123 for acquiring re-route information obtained by re-searching a new route from a deviation point to a destination when a vehicle has deviated from a route to be guided to the destination during traveling; a deviation determination section 122 for determining whether or not the deviation of the route is the one intended by a driver based on behavior information of the driver and the re-route information; a recording control section 114 for preserving peripheral video information including the deviation point when the unintended deviation is determined; and a reproduction control section 115 for reproducing the preserved peripheral video information before the vehicle passes the deviation point.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a vehicle recording control device, a driving support system, and a recording control method.

Background Art

[0002] Patent Document 1 discloses a technique for acquiring a deviation point at which a vehicle has deviated from a guidance route, and determining whether or not the above deviation is caused intentionally by a driver based on at least one of behavior information regarding the behavior of the vehicle before and after the deviation point, biometric information of the driver of the vehicle, or information regarding the traveling route of the vehicle after the deviation point.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when the deviation from the route is not intentional by the driver, i.e., unintentional, there is a loss in the driving distance and time to return to the normal route. Therefore, when the driver unintentionally deviates from the route, it is desirable to appropriately make the driver recognize so that this unintentional deviation is not repeated.

[0005] The present invention has been made in view of the above, and an object thereof is to provide a vehicle recording control device, a driving support system, and a recording control method that can appropriately make a driver recognize that there has been an unintentional deviation from a route.

Means for Solving the Problems

[0006] In order to solve the above-described problems and achieve the object, a vehicle recording control device according to the present invention includes an imaging information acquisition unit that acquires, as imaging information, peripheral video information that is video around the vehicle and in-vehicle video information that is video inside the vehicle; a biometric information acquisition unit that acquires biometric information of a driver of the vehicle; a driver behavior information acquisition unit that acquires driver behavior information based on at least one of the biometric information and the in-vehicle video information; a route information acquisition unit that, when the vehicle deviates from a route while traveling on a route guided to a destination, acquires reroute information obtained by re-searching a new route from the deviation point to the destination; a deviation determination unit that determines whether the deviation of the route is intentional by the driver based on the driver behavior information and the reroute information; a recording control unit that stores peripheral video information including the deviation point when it is determined that the deviation is unintentional; and a reproduction control unit that reproduces the stored peripheral video information before the vehicle passes through the deviation point.

[0007] Further, a recording control method according to the present invention includes a step of acquiring, as imaging information, peripheral video information that is video around the vehicle and in-vehicle video information that is video inside the vehicle; a step of acquiring biometric information of a driver of the vehicle; a step of acquiring driver behavior information based on at least one of the biometric information and the in-vehicle video information; a step of acquiring reroute information obtained by re-searching a new route from a deviation point to a destination when the vehicle deviates from a route while traveling on a route guided to the destination; a step of determining whether the deviation of the route is intentional by the driver based on the driver behavior information and the reroute information; a step of storing peripheral video information including the deviation point when it is determined that the deviation is unintentional; and a step of reproducing the stored peripheral video information before the vehicle passes through the deviation point.

Advantages of the Invention

[0008] According to the present invention, there is an effect that the driver can be appropriately made aware of an unintentional deviation of the route.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

[0010] Embodiments of a driving support system, a vehicle recording control device, and a recording control method according to the present invention will be described in detail below with reference to the accompanying drawings. Note that the present invention is not limited by the following embodiments.

[0011] [First Embodiment] FIG. 1 is a block diagram showing a configuration example of a vehicle recording device having a vehicle recording control device according to the first embodiment. FIG. 2 is a table in which the degree of deviation of a route is scored based on each item of driver behavior information and reroute information. FIG. 3 is a diagram showing an example of an intersection where a route deviation has occurred. FIG. 4 is a diagram showing an example of a surrounding video when traveling through the intersection of FIG. 3. The vehicle recording device 10 is a so-called drive recorder that records events occurring in the vehicle. When the driver inadvertently (mistakenly) deviates from the route while driving the vehicle according to the route guidance to the destination, the vehicle recording device 10 records the surrounding video information of the vehicle including the deviation point. In addition to being mounted on the vehicle, the vehicle recording device 10 may be a portable device that can be used in the vehicle. Further, the vehicle recording device 10 may be realized including functions or configurations of a device or a navigation device that is previously installed in the vehicle.

[0012] The vehicle recording device 10 includes a first camera (surrounding imaging unit) 21, a second camera (in-vehicle imaging unit) 22, a recording unit 23, an operation unit 24, a biosensor 25, a GPS (Global Positioning System) receiving unit 26, a CAN (Controller Area Network) interface unit (hereinafter referred to as "CANIF unit") 27, a navigation interface unit (hereinafter referred to as "navi IF unit") 28, a beacon receiving unit 29, a display unit 30, and a control unit (vehicle recording control device) 100.

[0013] The first camera 21 is a camera that captures the surroundings (e.g., the front) of the vehicle. In this embodiment, the first camera 21 is described using a single camera facing the front of the vehicle, but it may also be a plurality of camera groups. The first camera 21 is disposed, for example, at a position where it can capture the front of the vehicle in the front of the vehicle interior. In this embodiment, the first camera 21 constantly captures images while the accessory power supply (ACC power supply) of the vehicle is on. The second camera 22 described later operates in the same manner. The first camera 21 outputs the captured surrounding video information to the imaging information acquisition unit 111 of the control unit 100. The video data is, for example, a moving image composed of images at 30 frames per second.

[0014] The second camera 22 is disposed facing the rear inside the vehicle and captures the inside of the vehicle, particularly the driver. The second camera 22 captures a wide-angle view, for example, with a shooting range of about 180° in the horizontal direction. In this embodiment, the second camera 22 is described using a single camera facing the rear of the vehicle, but it may also be a plurality of camera groups. The second camera 22 outputs the captured in-vehicle video information to the imaging information acquisition unit 111 and the driver information acquisition unit 118 of the control unit 100. This in-vehicle video information may include driver information that can identify the driver, for example, by image recognition. Also, the in-vehicle video information is assumed to include audio information inside the vehicle. For this reason, it is preferable that the second camera 22 has a configuration including a microphone that picks up the audio information of the in-vehicle audio together with the in-vehicle video. This microphone may be provided separately from the second camera 22 inside the vehicle.

[0015] When there is no particular need to distinguish between the surrounding video information and the in-vehicle video information, it is described as video information. Also, in this embodiment, the first camera 21 acquires the surrounding video and the second camera 22 acquires the in-vehicle video, but it is also possible to acquire the surrounding video and the in-vehicle video using one camera, for example, a camera that can capture the entire surrounding area (360°).

[0016] The recording unit 23 is used for temporarily storing data in the vehicle recording device 10. The recording unit 23 is, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a recording unit such as a memory card. Further, the recording unit 23 may be an external recording unit wirelessly connected via a communication device (not shown). The recording unit 23 includes a loop recording unit 23A and a deviation point recording unit 23B. The recording unit 23 records the peripheral video information in the loop recording unit 23A or the deviation point recording unit 23B based on a control signal output from the recording control unit 114 of the control unit 100.

[0017] The loop recording unit 23A loop - records the peripheral video information. The loop recording unit 23A continuously records the peripheral video information. When the capacity of the loop recording unit 23A is full, new peripheral video information overwrites and records the oldest peripheral video information. The deviation point recording unit 23B records the peripheral video information including the deviation point when, during vehicle driving, the driver unintentionally deviates from the route according to the pre - set route guidance to the destination. Here, "unintentionally deviating from the route" means that although the driver intends to drive according to the route guidance, the driver turns (or does not turn) at a point different from the route guidance, thereby deviating from the route. Whether the deviation is intentional or not is determined from the driver's behavior information when deviating from the route (for example, in - vehicle video information recording the driver's expression and voice, or the driver's biological information detected by the biological sensor 25).

[0018] The operation unit 24 can receive various operations on the vehicle recording device 10. For example, the operation unit 24 can receive an operation to manually save the captured peripheral video information in the deviation point recording unit 23B. Also, for example, the operation unit 24 can receive an operation to play back the peripheral video information recorded in the deviation point recording unit 23B. For example, the operation unit 24 can receive an operation to erase the peripheral video information recorded in the deviation point recording unit 23B. The operation unit 24 outputs the operation information to the operation control unit 116 of the control unit 100.

[0019] The biological sensor 25 detects various biological information of the driver of the vehicle equipped with the vehicle recording device 10. The biological sensor 25 may include various sensors such as a heartbeat sensor, a pulse sensor, a blood pressure sensor, a body temperature sensor, a respiration sensor, and a sweating sensor. The biological sensor 25 detects, as biological information, for example, the heartbeat value, pulse value, blood pressure value, body temperature value, respiration rate, and sweating amount of the driver of the vehicle. Further, the biological sensor 25 may include a fingerprint sensor that detects, as biological information (driver information), for example, fingerprint information provided on the steering wheel of the vehicle to identify the driver. The biological sensor 25 outputs the detected biological information to the biological information acquisition unit 117 and the driver information acquisition unit 118.

[0020] The GPS receiver 26 receives radio waves from a GPS satellite (not shown). The GPS receiver 26 outputs the signal of the received radio waves to the position information acquisition unit 119 of the control unit 100.

[0021] The CANIF unit 27 is an interface for acquiring various vehicle information via CAN. The vehicle information includes at least one of driving operation information indicating the driving operation of the vehicle and state information indicating the state of the vehicle. In the present embodiment, it is configured to be able to measure the force applied to the steering wheel and the force of gripping the steering wheel, and when deviating from the intended route, for example, the force of the driver suddenly pressing or gripping the steering wheel is acquired as vehicle information (driving operation information). The driving operation information includes, for example, operation information such as steering operation, brake operation, and accelerator operation. The state information includes, for example, state information such as the vehicle speed and acceleration of the vehicle.

[0022] The navigation IF unit 28 is an interface connected to a navigation device (not shown) mounted on the vehicle, for transmitting and receiving various information to and from this navigation device. The navigation IF unit 28 acquires route information indicating a guidance route to a set destination from the navigation device. This route information includes re-route information obtained by re-searching for a new route to the destination during route guidance. The navigation IF unit 28 outputs the received route information (re-route information) to the route information acquisition unit 123 of the control unit 100. Also, the navigation IF unit 28 may transmit the peripheral video information recorded in the deviation point recording unit 23B to the navigation device and cause the reproduced peripheral video information to be displayed on the display unit of the navigation device.

[0023] The beacon reception unit 29 receives a beacon including road traffic information provided by a VICS (registered trademark) (Vehicle Information and Communication System) center. This beacon is, for example, an optical beacon including signal information indicating the stop time (waiting time) until the red signal of this traffic signal changes to a green signal when stopped at a traffic signal. The beacon reception unit 29 outputs the signal information included in the received beacon to the signal information acquisition unit 124 of the control unit 100.

[0024] The display unit 30 is, as an example, a display device unique to the vehicle recording device 10, but may be a display device shared with other systems including the navigation device. The display unit 30 may be integrally formed with the first camera 21 or the second camera 22. The display unit 30 is, for example, a display including a liquid crystal display (LCD) or an organic EL (Electro-Luminescence) display. In the present embodiment, the display unit 30 is disposed on the dashboard, instrument panel, center console, etc. in front of the driver of the vehicle. The display unit 30 displays an image based on the video signal output from the display control unit 125 of the control unit 100. The display unit 30 displays the image captured by the first camera 21 or the second camera 22, or the image (peripheral video information) recorded in the deviation point recording unit 23B.

[0025] The control unit 100 is an arithmetic processing unit (control device) composed of, for example, a CPU (Central Processing Unit). The control unit 100 loads the stored program into the memory and executes the instructions included in the program. The control unit 100 includes an internal memory (not shown), and the internal memory is used for temporarily storing data in the control unit 100. The control unit 100 is connected to the bus BS and includes an imaging information acquisition unit 111, a buffer memory 112, a video information processing unit 113, a recording control unit 114, a playback control unit 115, an operation control unit 116, a biological information acquisition unit 117, a driver information acquisition unit 118, a position information acquisition unit 119, a vehicle information acquisition unit 120, a driver behavior information acquisition unit 121, a deviation determination unit 122, a route information acquisition unit 123, a signal information acquisition unit 124, and a display control unit 125.

[0026] The imaging information acquisition unit 111 acquires, as imaging information, the peripheral video information from the first camera 21 that captures the periphery of the vehicle and the in-vehicle video information from the second camera 22 that captures the inside of the vehicle. This imaging information shall include at least the in-vehicle audio information together with the in-vehicle video information. The imaging information acquisition unit 111 outputs the acquired imaging information to the buffer memory 112. Further, the imaging information acquisition unit 111 outputs the acquired in-vehicle video information to the driver behavior information acquisition unit 121.

[0027] The buffer memory 112 is an internal memory provided in the control unit 100, and is a memory that temporarily records the video information (imaging information) for a certain period of time acquired by the imaging information acquisition unit 111 while updating it.

[0028] The video information processing unit 113 converts the video information temporarily stored in the buffer memory 112 into an arbitrary file format such as the MP4 format, encoded with an arbitrary codec such as H.264 or MPEG-4 (Moving Picture Experts Group). The video information processing unit 113 generates video information in the form of a file for a certain period of time from the video information temporarily stored in the buffer memory 112. As a specific example, the video information processing unit 113 generates, as a file, video information for 60 seconds in the recording order from the video information temporarily stored in the buffer memory 112. The video information processing unit 113 outputs the generated video information to the recording control unit 114. Also, the video information processing unit 113 outputs the generated video information to the display control unit 125 after decoding the image through the playback control unit 115. Although the period of the video information generated as a file is, as an example, 60 seconds, it is not limited to this. The video information referred to here may be peripheral video information including audio in addition to the video captured by the second camera 22.

[0029] The recording control unit 114 performs control to cause the recording unit 23 to record the video information filed by the video information processing unit 113. During the period when the loop recording process is executed, such as when the accessory power supply of the vehicle is ON, the recording control unit 114 records the video information filed by the video information processing unit 113 as overwritable video information in the loop recording unit 23A. More specifically, during the period when the loop recording process is executed, the recording control unit 114 continuously records the peripheral video information generated by the video information processing unit 113 in the loop recording unit 23A, and when the capacity of the loop recording unit 23A is full, overwrites and records the new peripheral video information on the oldest peripheral video information.

[0030] Furthermore, when it is determined that the vehicle has deviated from an unintended route during vehicle driving following the route guidance, the recording control unit 114 saves the peripheral video information including the deviation point. The recording control unit 114 also saves the peripheral video information including the deviation point in the deviation point recording unit 23B as information where overwriting is prohibited.

[0031] The peripheral video information that the recording control unit 114 stores in the deviation point recording unit 23B copies the peripheral video information for a predetermined period of about 30 seconds before and after the time when the deviation from an unintended route is determined from the buffer memory 112, and stores it as peripheral video information including the deviation point.

[0032] The playback control unit 115 controls to play back the peripheral video information recorded in the recording unit 23 based on the playback operation control signal output from the operation control unit 116. The playback control unit 115 includes a decoder (not shown) and decodes the supplied compressed data to play back various data. The playback control unit 115 plays back the peripheral video information including the deviation point stored in the deviation point recording unit 23B. More specifically, when the playback control unit 115 is guided on the same route to the same destination as when deviating from the guidance route, it plays back the peripheral video information including the deviation point until reaching the deviation point. For example, when the vehicle enters within a predetermined distance range (for example, 1 km) from the deviation point, the playback control unit 115 plays back the peripheral video information including the deviation point each time the vehicle stops at a traffic signal in front of the deviation point. Here, in order to effectively play back the peripheral video information, if the playback control unit 115 receives the waiting time (signal information) from when the traffic signal changes from red (stop) to green (go), it may set the playback time according to this waiting time. Specifically, assuming that the maximum time of the peripheral video information stored in the deviation point recording unit 23B is 60 seconds, it includes the deviation point and sets the peripheral video information to be shortened according to the waiting time.

[0033] In addition, when the deviation of the route is scored, the playback control unit 115 may set the playback time according to this score. Specifically, in this embodiment, based on the driver's behavior and the driving distance (or driving time) from the deviation point until returning to the regular route again, the more corresponding items there are for these behaviors and driving distances, the higher the score for the deviation of the route is set. And when the maximum time of the surrounding video information stored in the deviation point recording unit 23B is 60 seconds, the playback control unit 115 sets the surrounding video information to be shortened including the deviation point and as the score decreases. In addition, when the driver is identified, the playback control unit 115 plays back the surrounding video information corresponding to the driver.

[0034] The operation control unit 116 acquires the operation information of the operation received by the operation unit 24. For example, the operation control unit 116 acquires save operation information indicating a manual save operation of the surrounding video information, playback operation information indicating a playback operation, or erase operation information indicating an erase operation of the surrounding video information, and outputs a control signal.

[0035] The biological information acquisition unit 117 acquires the biological information of the driver detected by the biological sensor 25. The biological information is, for example, the heartbeat value, pulse value, blood pressure value, body temperature value, respiratory rate, sweating amount, and fingerprint information of the driver of the vehicle. The biological information may be, for example, a combination of fingerprint information for identifying the driver and a heart rate for determining the psychological state (presence or absence of tension or excitement state) of the driver. The biological information acquisition unit 117 outputs the acquired biological information to the driver behavior information acquisition unit 121.

[0036] The driver information acquisition unit 118 acquires the driver information detected by the biological sensor 25. This driver information is biological information for identifying the driver, and for example, fingerprint information can be exemplified. In addition, the driver information acquisition unit 118 may acquire the in-vehicle video information captured by the second camera 22 as driver information to identify the driver.

[0037] The position information acquisition unit 119 acquires position information indicating the current position of the vehicle. The position information acquisition unit 119 calculates the position information of the current position of the vehicle by a known method based on the radio wave received by the GPS reception unit 26. The position information acquisition unit 119 may calculate the position information of the current position of the vehicle using another positioning system including quasi-zenith satellites.

[0038] The vehicle information acquisition unit 120 acquires the vehicle information acquired by the CANIF unit 27. More specifically, the vehicle information acquisition unit 120 acquires vehicle information indicating the behavior of the vehicle, including at least one of driving operation information indicating a driving operation on the vehicle and state information indicating the state of the vehicle.

[0039] The driver behavior information acquisition unit 121 acquires driver behavior information based on at least one of the acquired biometric information and in-vehicle video information. Here, the driver behavior information is information indicating the behavior executed by the driver when the driver is in an excited or tense state. Specifically, when the driver becomes aware that he / she has deviated from the guided route unintentionally while driving the vehicle according to the route guidance to the destination, the mental state of the driver changes from calm to excited or tense. Therefore, the driver behavior information is information indicating the behavior unconsciously executed by the driver when deviating from the guided route unintentionally. For example, it is information indicating that the driver's heart rate, pulse rate, and sweating amount as biometric information increase above a predetermined value, or information indicating behaviors such as the driver's surprised or excited expression or shouting voice (loud voice) extracted from the in-vehicle video information. The driver behavior information acquisition unit 121 outputs the acquired driver behavior information to the deviation determination unit 122.

[0040] The route information acquisition unit 123 acquires, via the navigation IF unit 28, position information regarding the set destination and route information indicating the guidance route to the destination from the navigation device. In the present embodiment, when the vehicle deviates from the route during the route guidance to the destination, the route information acquisition unit 123 acquires re-route information obtained by re-searching a new route from the deviation point to the destination. In this case, the navigation device monitors the position information of the vehicle, and when this position information deviates from the guidance route, it re-searches for a new route and outputs the re-route information to the navigation IF unit 28. It is desirable that this re-route information includes time information when the route deviation occurred, position information of the deviation point, and the travel distance of the new route. The route information acquisition unit 123 outputs the acquired re-route information to the deviation determination unit 122.

[0041] The deviation determination unit 122 determines whether the route deviation is intentional by the driver based on the acquired driver behavior information and the re-route information. More specifically, the deviation determination unit 122 determines that the vehicle has deviated from the guidance route by the acquisition of the re-route information. In addition to this, for example, when the driver's heart rate, pulse rate, or sweating amount has increased above a predetermined value, or an expression of surprise or excitement or a shouting voice (sound) of the driver is extracted as the driver behavior information, the deviation determination unit 122 determines that the route deviation is not intentional by the driver. Further, the deviation determination unit 122 may determine that the route deviation is not intentional by the driver not only based on the driver behavior information but also, for example, when the force or grip force with which the driver presses the steering wheel has increased above a predetermined value as the vehicle information. When the deviation determination unit 122 determines that the route deviation is not intentional by the driver, it outputs the time information and position information of the deviation included in the re-route information to the recording control unit 114.

[0042] Further, the deviation determination unit 122 quantifies the degree of deviation based on the driver behavior information acquired at the deviation point of the route and the re-route information. Specifically, as shown in FIG. 2, when there is content that satisfies the items corresponding to the preset driver behavior information (for example, the driver's heart rate, expression, driver's voice), and the items corresponding to the re-route information (for example, the driving distance), points (for example, 1 point) are added to the corresponding items respectively to calculate the total points. Here, for the item of "driving distance" corresponding to the re-route information, depending on the re-route associated with the deviation, for example, if the driving distance increases by a predetermined distance or more, this item is satisfied, and if the increase is less than the predetermined distance, it is considered that this item is not satisfied. In the example of FIG. 2, the score at the deviation point 2 is 4 points because it corresponds to the four items of the driver's heart rate, expression, voice, and driving distance. Usually, when deviating from the route, there is a loss in driving distance and time to return to the regular route. Also, if this deviation is unintentional, it is conceivable that the driver will receive a mental shock due to misinterpreting the route. In this embodiment, by quantifying each item of the driver behavior information and the re-route information, the magnitude of the impact (degree of deviation) when the driver unintentionally deviates from the route can be defined.

[0043] Also, in the example of FIG. 2, the number of items corresponding to the driver behavior information and the re-route information is set to four, but it is not limited to this, and the number of items may be increased further. Also, in the example of FIG. 2, the scores of each item are made uniform. However, for important items such as the driving distance, thresholds may be set in multiple stages, and weighting may be performed so that the score is increased by adding points each time the driving distance exceeds each threshold.

[0044] The signal information acquisition unit 124 acquires signal information indicating the stop time (waiting time until the red signal changes to the green signal) at the traffic signal where the vehicle has stopped via the beacon reception unit 29. The signal information acquisition unit 124 acquires the signal information indicating the above stop time, for example, each time the vehicle stops at a traffic signal. The signal information acquisition unit 124 outputs the acquired signal information to the reproduction control unit 115.

[0045] The display control unit 125 controls the display of video information on the display unit 30. The display control unit 125 outputs a video signal for outputting peripheral video information to the display unit 30. More specifically, the display control unit 125 displays the video captured by the first camera 21 or the second camera 22, or the peripheral video information recorded in the deviation point recording unit 23B and reproduced by the reproduction control unit 115. Further, the display control unit 125 superimposes and displays at least one of the image information indicating the normal route and the image information indicating the deviated route on the peripheral video information reproduced by the reproduction control unit 115.

[0046] Here, it will be described that the deviation of the route of the guided vehicle 50 occurs at the five-way intersection 60 shown in FIG. 3. The five-way intersection 60 shown in FIG. 3 includes an approach road 61 into which the vehicle 50 enters, a straight road 62 located on the extension line of the traveling direction of the approach road 61, a left-turn road 63 extending to the left with respect to the traveling direction, a right-turn road 64 extending to the right with respect to the traveling direction, and a right diagonal road 65 that exists between the straight road 62 and the right-turn road 64 and extends in the diagonally right direction. In this case, the vehicle 50 is guided to travel from the approach road 61 to the right-turn road 64, but has deviated from the route by entering from the approach road 61 into the right diagonal road 65.

[0047] As shown in FIG. 4, the display control unit 125 superimposes and displays, on the peripheral video information 70 reproduced by the reproduction control unit 115, image information such as a circle symbol image 71 for indicating the right-turn road 64, which is the normal route, and image information such as a cross symbol image 72 for indicating the right diagonal road 65, which is the deviated route. With this configuration, a driver who views the peripheral video information when deviating from the route can intuitively understand that the route has been misjudged, and it is possible to prevent this misjudgment (deviation) from occurring repeatedly.

[0048] Next, with reference to FIGS. 5 and 6, the processing flow in the control unit 100 will be described. FIG. 5 is a flowchart showing an example of the procedure for storing peripheral video information including a deviation point. FIG. 6 is a flowchart showing an example of the procedure for playing back the stored peripheral video information including the deviation point. In the present embodiment, for example, when the vehicle is being driven according to the route guidance to the destination by a navigation device and the driver deviates from the guided route unintentionally, the peripheral video information including this deviation point is stored and the peripheral video information is played back at an appropriate timing. While the vehicle is being route-guided to the destination, the peripheral video information including the deviation point where the vehicle has deviated from the route is stored by the processing of the flowchart shown in FIG. 5. Also, while the vehicle is being route-guided to the same destination and along the same route as when the above deviation occurred and the vehicle is stopped at a traffic signal, the peripheral video information including the stored deviation location is played back by the processing of the flowchart shown in FIG. 6.

[0049] First, the procedure for the process of storing the peripheral video information including the deviation point will be described. As shown in FIG. 5, the control unit 100 determines whether the vehicle is being driven according to the guided route to the destination (step S11). More specifically, when the control unit 100 has acquired the route information for guiding to the destination by the route information acquisition unit 123, it determines that the vehicle is being driven according to the guided route to the destination, and when it has not acquired the above route information, it determines that the vehicle is not being driven according to the guided route to the destination. In this determination, if the vehicle is not being driven according to the guided route to the destination (step S11; No), the control unit 100 ends the process. On the other hand, if the vehicle is being driven according to the guided route to the destination (step S11; Yes), the control unit 100 proceeds to step S12.

[0050] Next, the control unit 100 acquires the peripheral video information and the in-vehicle video information (step S12). More specifically, the control unit 100 acquires the peripheral video information from the first camera 21 and the in-vehicle video information from the second camera 22 by the imaging information acquisition unit 111. Here, the in-vehicle video information preferably includes video information capturing the driver's expression.

[0051] Next, the control unit 100 acquires the biometric information of the driver (step S13). More specifically, the control unit 100 acquires the biometric information of the driver detected by the biometric sensor 25 through the biometric information acquisition unit 117. This biometric information is preferably acquired by combining, for example, fingerprint information for identifying the driver of the vehicle and information such as a heart rate value, a pulse rate value, and a sweating amount for determining the mental state of the driver. Alternatively, the in-vehicle video information acquired in step S12 may be used to identify the driver of the vehicle and combined with the biometric information.

[0052] Next, the control unit 100 acquires driver behavior information indicating the behavior of the driver based on the in-vehicle video information and the biometric information (step S14). More specifically, the control unit 100 acquires the driver behavior information through the driver behavior information acquisition unit 121 based on at least one of the acquired biometric information and the in-vehicle video information. This driver behavior information is information indicating the behavior of the driver when the driver unintentionally deviates from (makes a mistake in) the route during vehicle operation according to the route guidance or when the driver is in an excited or tense state. The driver behavior information is, for example, information indicating a behavior in which the driver's heart rate value, pulse rate value, or sweating amount (biometric information) rises above a predetermined value, or information indicating a surprised or excited expression of the driver, or an exclamation such as "Mistake!", "Ah!", "Bad!", "Danger!" (voice; in-vehicle video information).

[0053] Next, the control unit 100 determines whether it has acquired reroute information (step S15). More specifically, when the vehicle deviates from the guidance route, the navigation device re-searches for a new route from the deviation point to the destination and outputs the reroute information to the vehicle recording device 10. The control unit 100 determines whether it has acquired the reroute information from the navigation device by the route information acquisition unit 123. This reroute information includes the time information when the route deviation occurred, the position information of the deviation point, and the travel distance of the new route. In this determination, if the control unit 100 has not acquired the reroute information (step S15; No), since no route deviation has occurred, the control unit 100 stores (records) the surrounding video information in the loop recording unit 23A by the recording control unit 114 (step S19). Specifically, the recording control unit 114 continuously records the surrounding video information in the loop recording unit 23A as loop recording processing. When the capacity of the loop recording unit 23A is full, the oldest surrounding video information is overwritten and recorded with the new surrounding video information. Then, the process proceeds to step S20.

[0054] On the other hand, if the control unit 100 has acquired the reroute information (step S15; Yes), since a route deviation has occurred, the process proceeds to step S16. Next, the control unit 100 determines whether the route deviation is an unintended one (step S16). More specifically, the control unit 100 determines whether the route deviation is unintended based on the driver behavior information and the reroute information by the deviation determination unit 122. For example, when the deviation determination unit 122 extracts, as driver behavior information, that the driver's heart rate, pulse rate, or sweating amount has increased above a predetermined value, or the driver's surprised or excited expression, or a shouting sound (voice), it determines that the driver's mental state has changed rapidly during driving. Then, if the acquisition time of the driver behavior information and the occurrence time of the route deviation in the reroute information are the same time or within a predetermined time (for example, 10 seconds), the deviation determination unit 122 determines that the route deviation is not intended by the driver.

[0055] In this determination, if it is determined that it is not a deviation from an unintended route (step S16; No), the control unit 100 proceeds to step S19. On the other hand, if it is determined that it is a deviation from an unintended route (step S16; Yes), the control unit 100 scores the acquired driver behavior information and reroute information respectively by the deviation determination unit 122 (step S17). Specifically, when there is content that satisfies the items corresponding to the driver behavior information (for example, the driver's heart rate, the driver's expression, the driver's voice) and the items corresponding to the reroute information (for example, the driving distance), the deviation determination unit 122 adds a score (for example, 1 point) to the corresponding item to calculate the total score.

[0056] Next, the control unit 100 stores (records) the peripheral video information including the deviation point in the deviation point recording unit 23B together with the score, the destination, and the route (the previous route) by the recording control unit 114 (step S18). Specifically, the recording control unit 114 also records the peripheral video information including the deviation point in the deviation point recording unit 23B as information with write prohibition, together with the score information, the position information of the destination, and the route information. Also, when the acquired biometric information or in-vehicle video information includes driver information for identifying the driver, the peripheral video information may be stored in association with this driver information.

[0057] Next, the control unit 100 determines whether the vehicle has arrived at the destination (step S20). Specifically, it is determined whether information that the vehicle has arrived at the destination is obtained from the navigation device, or whether information that the position information of the vehicle matches (or exists within a predetermined distance range) the position information of the destination is obtained. In this determination, if the vehicle has not arrived at the destination (step S20; No), the control unit 100 returns the process to step S12 and repeatedly executes steps S12 to S20. Also, if the vehicle has arrived at the destination (step S20; Yes), the control unit 100 ends the process.

[0058] Next, the procedure of the process for playing back the surrounding video information including the saved deviation point will be described. As shown in FIG. 6, when the route information acquisition unit 123 acquires the route information (route information) in which the same destination and the same route as before are set (step S21), the control unit 100 identifies the driver from the driver's biometric information or in-vehicle video information (step S22). More specifically, the control unit 100 identifies the driver using the biometric information (for example, fingerprint information) or in-vehicle video information that can identify the driver acquired by the driver information acquisition unit 118.

[0059] Next, the control unit 100 determines whether the surrounding video information including the deviation point is stored in the deviation point recording unit 23B (step S23). The surrounding video information including the deviation point recorded in the deviation point recording unit 23B is the surrounding video information when the driver deviated from the route unintentionally as described above. In this case, the control unit 100 may simply determine whether the surrounding video information is stored in the deviation point recording unit 23B, but it is preferable to determine whether there is surrounding video information stored within a predetermined period (for example, the most recent one year). Further, the control unit 100 may determine whether the surrounding video information associated with the identified driver is stored in the deviation point recording unit 23B.

[0060] In this determination, if the surrounding video information including the deviation point is not stored in the deviation point recording unit 23B (step S23; No), the control unit 100 ends the process. Also, if the surrounding video information including the deviation point is stored in the deviation point recording unit 23B (step S23; Yes), the control unit 100 determines whether there is a deviation point on the route to the destination (step S24). More specifically, the control unit 100 determines whether the position information of the deviation point where the driver deviated unintentionally as determined by the deviation determination unit 122 is on the set route to the destination. In this determination, if there is no deviation point on the route to the destination (step S24; No), the control unit 100 ends the process.

[0061] On the other hand, when there is a deviation point where the driver has deviated unintentionally on the route to the destination (step S24; Yes), the control unit 100 sets, by the playback control unit 115, the maximum playback time of the peripheral video information including the deviation point existing on the route according to the points associated therewith (step S25). When there is peripheral video information associated with the identified driver, the playback control unit 115 sets the maximum playback time of the peripheral video information including the deviation point existing on the route among the peripheral video information associated with this driver according to the points associated therewith.

[0062] When the recording time of the peripheral video information stored in the deviation point recording unit 23B is set to a maximum of 60 seconds, the playback control unit 115 shortens and sets the maximum playback time of the peripheral video information as the number of points decreases. Specifically, the maximum playback time of the peripheral video information with the highest number of points (4 points in the example of FIG. 2) is set to 60 seconds, the playback time of the peripheral video information with the lowest number of points (for example, 1 point) is set to 30 seconds, and the playback times of the peripheral video information with the intermediate number of points (from 2 points to 3 points) are shortened and set according to the number of points respectively. Further, shortening and setting the playback time means that, for example, the peripheral video information may be fast-forwarded to shorten the playback time, or the time before and after the time of deviating from the route may be cut centered on the time of deviating from the route to shorten the playback time.

[0063] Next, the control unit 100 determines whether the vehicle is in front of the deviation point and within a predetermined distance (e.g., 1 km) from the deviation point (step S26). More specifically, the control unit 100 monitors the current position information of the vehicle and determines whether the vehicle is in front of the deviation point and within a predetermined distance from the deviation point. In this determination, if the vehicle is not in front of the deviation point and within a predetermined distance from the deviation point (step S26; No), the control unit 100 repeatedly executes the process until the vehicle enters in front of the deviation point and within a predetermined distance from the deviation point. On the other hand, if the vehicle is in front of the deviation point and within a predetermined distance from the deviation point (step S26; Yes), the control unit 100 acquires, from the beacon, the signal information of the traffic signal at which the vehicle has stopped by the signal information acquisition unit 124. This signal information includes information regarding the stop time (waiting time until the red signal changes to the green signal) at the traffic signal at which the vehicle has stopped. Therefore, the control unit 100 can determine how much stop time remains at this traffic signal when the vehicle stops at the traffic signal.

[0064] Next, the control unit 100 causes the playback control unit 115 to play back the surrounding video information including the deviation point according to the stop time until the traffic signal changes to the green signal (step S28). More specifically, the playback control unit 115 plays back the surrounding video information including the deviation point according to the stop time until the traffic signal changes to the green signal within the maximum playback time set in step S25 described above. That is, if the stop time of the vehicle at the traffic signal is equal to or longer than the maximum playback time described above, the surrounding video information including the deviation point is played back for this maximum playback time. Further, when the stop time of the vehicle at the traffic signal is shorter than the maximum playback time described above, the playback time is shortened so as to include at least the deviation point in accordance with this stop time, and the surrounding video information is played back.

[0065] Furthermore, in this case, the control unit 100 causes the display control unit 125 to superimpose and display, on the peripheral video information 70 of FIG. 4 processed by the playback control unit 115, image information such as a circle symbol image 71 for indicating a right turn road 64 which is a normal route, and image information such as a cross symbol image 72 for indicating a right diagonal road 65 which is a deviated route. According to this configuration, when driving to the destination, the driver can visually review the driving at the previous deviation point, so that when driving through the target deviation point this time, the driver can drive with attention and prevent deviating from the route.

[0066] Next, the control unit 100 determines whether the vehicle has passed through the deviation point (step S29). More specifically, it monitors the current position information of the vehicle and determines whether the vehicle has approached the destination more than the deviation point. In this determination, if the vehicle has not passed through the deviation point (step S29; No), the control unit 100 returns the process to step S27 and repeatedly executes the processes from step S27 to step S29. On the other hand, if the vehicle has passed through the deviation point (step S29; Yes), the control unit 100 ends the process.

[0067] [Second Embodiment] FIG. 7 is a block diagram showing a schematic configuration example of a driving support system according to the second embodiment. FIG. 8 is a diagram showing an example of a guidance screen when driving through the intersection of FIG. 3. The driving support system 200 shown in FIG. 7 is a navigation system in which the vehicle recording device 10A and the navigation device 80 cooperate. For the vehicle recording device 10A, the same components as those of the vehicle recording device 10 described in the first embodiment are denoted by the same reference numerals and the description thereof is omitted. When the driver has unintentionally deviated from the route during driving according to the route guidance to the destination and drives the same destination and the same route again, the driving support system 200 also displays the deviation point on the navigation device.

[0068] The control unit 100A (vehicle recording control device) of the vehicle recording device 10A further includes a deviation information output unit 126. This deviation information output unit 126 outputs deviation information, which is determined by the deviation determination unit 122 and indicates that the deviation of the route is not intended by the driver, to the navigation device 80.

[0069] The navigation device 80 includes a drive recorder IF unit 81, a map information storage unit 82, a display unit 83, and a control unit 90 (navigation control device). Note that the navigation device 80 may have an existing configuration in addition to the above-described configuration.

[0070] The drive recorder IF unit 81 is an interface connected to the vehicle recording device 10A mounted on the vehicle for outputting information from this navigation device 80. The drive recorder IF unit 81 acquires deviation information indicating that the deviation of the route is not intended by the driver from the vehicle recording device 10A. The drive recorder IF unit 81 outputs the acquired deviation information to the deviation information acquisition unit 91 of the control unit 90. Further, the drive recorder IF unit 81 may display the reproduced peripheral video information on the display unit 83 of the navigation device 80.

[0071] The map information storage unit 82 is a storage device that stores map information. The map information includes information regarding the number of lanes and speed limits of various roads, and information regarding facilities (stores) around the road. The display unit 83 is a display device specific to the navigation device 80 and is provided, for example, on the center console of the vehicle. Map information around the vehicle is displayed on the display unit 83.

[0072] The control unit 90 is an arithmetic processing unit (control device) composed of, for example, a CPU (Central Processing Unit), and includes a deviation information acquisition unit 91, a route setting guidance unit 92, a display control unit 93, and a route information output unit 94. The deviation information acquisition unit 91 acquires, via the drive recorder IF unit 81, deviation information indicating that the deviation of the route from the vehicle recording device 10A is not intended by the driver. That is, if the above-mentioned deviation is intended by the driver or is intentional, the deviation information is not acquired.

[0073] When a destination is set via an operation unit (not shown), the route setting guidance unit 92 sets a route to this destination and guides the vehicle to travel according to this route. Specifically, it displays the map information read from the map information storage unit 82 on the display unit 83, and also displays the route between the own vehicle and the destination on this map information, and displaces the displayed map information as the own vehicle moves.

[0074] The route information output unit 94 outputs the route information to the destination set to the vehicle recording device 10A. The display control unit 93 controls the display of the map information on the display unit 83. The display control unit 93 outputs an image signal for causing the display unit 83 to output the map information. More specifically, the display control unit 93 superimposes and displays at least one of the image information indicating the normal route and the image information indicating the deviated route on the map information for route guidance based on the deviation information. Specifically, as shown in FIG. 8, the display control unit 93 superimposes and displays on the map information 140 for route guidance the image information such as the ○ symbol image 141 indicating the right turn road 64 which is the normal route and the image information such as the × symbol image 142 indicating the right diagonal road 65 which is the deviated route. With this configuration, a driver driving according to the route guidance can intuitively understand that the route was misjudged at this point before by looking at the symbol image superimposed and displayed on the map information 140 of the display unit 83, and it is possible to prevent this misjudgment (deviation) from occurring repeatedly.

[0075] As described above, the control unit 100 of the present embodiment includes an imaging information acquisition unit 111 that acquires, as imaging information, peripheral video information that is video around the vehicle and in-vehicle video information that is video inside the vehicle, a biometric information acquisition unit 117 that acquires biometric information of the driver of the vehicle, a driver behavior information acquisition unit 121 that acquires driver behavior information based on at least one of the biometric information and the in-vehicle video information, a route information acquisition unit 123 that acquires re-route information obtained by re-searching a new route from the deviation point to the destination when the vehicle deviates from the route while traveling on the route guided to the destination, a deviation determination unit 122 that determines whether or not the deviation of the route is intentional based on the driver behavior information and the re-route information, a recording control unit 114 that stores peripheral video information including the deviation point when it is determined that the deviation is unintentional, and a reproduction control unit 115 that reproduces the stored peripheral video information before the vehicle passes through the deviation point. According to this configuration, by appropriately reproducing the peripheral video information including the deviation point of the unintentional route, the driver can be appropriately made aware that there has been an unintentional deviation of the route.

[0076] Further, in the control unit 100 of the present embodiment, when a deviation point exists on the same route to the destination, when it is determined that the vehicle has stopped at a traffic signal in front of the deviation point within a predetermined distance range from the deviation point and in the traveling direction of the deviation point, the reproduction control unit reproduces the peripheral video information including the deviation point according to the vehicle stop time until the traffic signal turns green. According to this configuration, since the peripheral video information including the deviation point is reproduced using the time when the vehicle is stopped at the traffic signal within a predetermined distance range from the deviation point, the driver can be effectively made aware that the route has been deviated from in the past.

[0077] Also, in the control unit 100 of the present embodiment, the deviation determination unit 122 quantifies the degree of deviation based on the behavior information of the driver and the re-route information, and the playback control unit 115 sets the playback time of the surrounding video information based on the magnitude of the score. According to this configuration, the higher the degree of deviation, that is, the more corresponding items there are, the longer the playback time of the surrounding video information is set for the deviation point. Therefore, it is possible to focus on checking the deviation points with a high degree of deviation and effectively prevent the recurrence of route deviation.

[0078] Also, in the control unit 100 of the present embodiment, a driver information acquisition unit 118 for acquiring driver information is provided, and the playback control unit 115 plays back the surrounding video information associated with the driver. According to this configuration, since the driver can review the unintended deviations experienced in the past, when driving through the deviation point next time, the driver can drive with attention and prevent route deviation.

[0079] Also, in the control unit 100 of the present embodiment, a display control unit 125 is provided that superimposes and displays at least one of image information such as a circle symbol image 71 indicating a regular route and image information such as a cross symbol image 72 indicating a deviated route on the surrounding video information 70 played back by the playback control unit 115. According to this configuration, when driving to the destination, the driver can visually review the driving at the previous deviation point. Therefore, when driving through the target deviation point this time, the driver can drive with attention and prevent deviating from the route.

[0080] In addition, the driving support system 200 of the present embodiment includes a vehicle recording device 10A and a navigation device 80 that is connected to the vehicle recording device 10A so as to be capable of transmitting and receiving information and guides the route of the vehicle to the destination. The vehicle recording device 10A includes a deviation information output unit 126 that outputs deviation information indicating that the deviation of the route is an unintended deviation by the driver. The navigation device 80 includes a deviation information acquisition unit 91 that acquires the deviation information, and a display control unit 93 that superimposes and displays at least one of image information such as a circle symbol image 141 indicating a normal route and image information such as a cross symbol image 142 indicating a deviated route on the map information 140 being route-guided. According to this configuration, when the driver drives to the destination, the driver can visually review the driving at the previous deviation point. Therefore, when driving through the target deviation point this time, the driver can drive with attention and prevent deviating from the route.

[0081] As described above, the embodiments of the present invention have been described, but the embodiments are not limited by the contents of this embodiment. In addition, the above-described components include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or changes of the components can be made without departing from the gist of the above-described embodiments.

[0082] Also, in this embodiment, when stopping at least once by a signal within a predetermined distance range from the target deviation point, the configuration is such that the peripheral video information including the deviation point is reproduced. However, if stopping multiple times, the peripheral video information may be reproduced each time. However, when arriving at the target deviation point without stopping at a traffic signal even once within the above-described predetermined distance range, it may be indicated by voice that "You are approaching the intersection where you made a mistake last time. At this intersection, please proceed in the right direction (or the ○○ direction)".

[0083] In this embodiment, as an example of the biological sensor 25, a configuration in which a fingerprint sensor capable of identifying a driver is provided on the vehicle's steering wheel has been described. However, it may be provided at a location that the driver always touches, for example, on the engine start switch. Alternatively, a steering wheel cover attached to the steering wheel may be provided, and a fingerprint sensor may be provided on this steering wheel cover.

[0084] Furthermore, each component of the control unit (vehicle recording control device) 100 according to this embodiment is conceptually functional and does not necessarily have to be physically configured as shown in the drawings. That is, the specific form of each device is not limited to that shown in the drawings, and all or part of it may be functionally or physically distributed or integrated in any unit according to the processing load of each device, usage situation, etc. The configuration of the control unit 100 is realized, for example, by a program loaded into the memory as software. In the above embodiment, it has been described as a functional block realized by the cooperation of this hardware or software. That is, these functional blocks can be realized in various forms by only hardware, only software, or a combination thereof.

Explanation of Reference Numerals

[0085] 10, 10A Vehicle recording device 23A Loop recording unit 23B Deviation point recording unit 25 Biological sensor 29 Beacon receiving unit 30 Display unit 70 Peripheral video information 71, 141 〇 symbol image (image information indicating a normal route) 72, 142 × symbol image (image information indicating a deviated route) 80 Navigation device 83 Display unit 90 Control unit 91 Deviation information acquisition unit 92 Route setting proposal unit 93 Display control unit 94 Route information output unit 100, 100A Control Unit (Vehicle Recording Control Device) 111 Imaging Information Acquisition Unit 114 Recording Control Unit 115 Playback Control Unit 117 Biometric Information Acquisition Unit 121 Driver Behavior Information Acquisition Unit 122 Deviation Determination Unit 123 Route Information Acquisition Unit 124 Signal Information Acquisition Unit 125 Display Control Unit 126 Deviation Information Output Unit 140 Map Information 200 Driving Support System

Claims

1. An imaging information acquisition unit that acquires, as imaging information, peripheral video information that is video of the periphery of the vehicle and in-vehicle video information that is video inside the vehicle of the vehicle; A biometric information acquisition unit that acquires biometric information of the driver of the vehicle; A driver behavior information acquisition unit that acquires behavior information of the driver based on at least one of the biometric information and the in-vehicle video information; A route information acquisition unit that, when the vehicle deviates from a route while traveling on a route guided to a destination, acquires reroute information obtained by re-searching for a new route from the deviation point to the destination; A deviation determination unit that determines whether or not the deviation from the route is intended by the driver based on the behavior information of the driver and the reroute information; A recording control unit that saves the peripheral video information including the deviation point when determined to be an unintended deviation; A vehicle recording control device comprising a reproduction control unit that reproduces the saved peripheral video information before the vehicle passes through the deviation point.

2. When the deviation point exists on the same route to the destination, When it is determined that the vehicle has stopped at a traffic signal in front of the deviation point within a predetermined distance range from the deviation point and in the traveling direction of the deviation point, The reproduction control unit reproduces the peripheral video information including the deviation point according to the vehicle stop time until the traffic signal changes to a green signal, according to the vehicle recording control device according to claim 1.

3. The vehicle recording control device according to claim 1 or 2, further comprising a display control unit that superimposes and displays on the peripheral video information reproduced by the reproduction control unit at least one of image information indicating a regular route and image information indicating a deviated route.

4. A driving support system comprising the vehicle recording control device according to claim 1 or 2, and a navigation device that is connected so as to be able to transmit and receive information to and from the vehicle recording control device and guides the route of the vehicle to the destination, The vehicle recording control device includes a deviation information output unit that outputs deviation information indicating that the deviation from the route is an unintended deviation by the driver, The navigation device includes a deviation information acquisition unit that acquires the deviation information, and a display control unit that superimposes and displays on the map information being route-guided at least one of image information indicating a regular route and image information indicating a deviated route based on the deviation information.

5. A step of acquiring, as imaging information, peripheral video information which is video of the periphery of the vehicle and in-vehicle video information which is video inside the vehicle of the vehicle; A step of acquiring biometric information of a driver of the vehicle; A step of acquiring behavior information of the driver based on at least one of the biometric information and the in-vehicle video information; A step of acquiring reroute information obtained by re-searching for a new route from a deviation point to the destination when the vehicle deviates from a route while traveling on a route guided to the destination; A step of determining whether or not the deviation from the route is intended by the driver based on the behavior information of the driver and the reroute information; A step of saving the peripheral video information including the deviation point when determined to be an unintended deviation; A recording control method comprising a step of playing back the saved peripheral video information before the vehicle passes through the deviation point.

Images