On-vehicle system, and guidance program

Abstract

To provide accurate guidance to a driver on how to use a blinker.SOLUTION: An acquisition unit (21) acquires detection data for indicating a peripheral situation of a vehicle from a sensor (10) installed in the vehicle. A recognition unit (22) recognizes at least one of a road sign or a road surface sign from the detection data. A notification control unit (23) specifies a sort of a connection configuration at a road connection point where a traffic lane confluences, branches, or intersects based on the at least one of recognized road sign or road surface sign, and notifies a driver of guidance on how to turn on a blinker in accordance with a traffic rule that corresponds to the specified sort of the connection configuration.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an in-vehicle system and a guidance program for guiding how to operate a turn signal.

Background Art

[0002] While driving on the road, a driver may sometimes be confused about which direction to operate the turn signal at intersections or merging points. For example, at a rotary or roundabout (circular intersection), the driver may be confused about when to operate the turn signal.

[0003] In response, a method of notifying the driver of guidance on how to operate the turn signal can be considered. For example, a method has been proposed in which, in cooperation with a car navigation system, when approaching an intersection, the turn signal and brake lamp are lit on a virtual vehicle to guide the driver on the direction to turn and the need to perform a deceleration operation (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the above method, cooperation with a car navigation system is essential. In addition, according to the Enforcement Regulations of the Road Traffic Law, the timing of operating the turn signal is specified as "30 m before an intersection" and "3 seconds before changing the driving route". Due to the positioning error of GPS (Global Positioning System), there is a risk that the timing of the guidance indicating the direction to operate the turn signal may be delayed. Also, when the type of intersection changes, if the update of the map information is delayed, there is a risk of guiding the wrong way to operate the turn signal according to the wrong traffic rules.

[0006] This embodiment has been made in view of such circumstances, and an object thereof is to provide a technique for accurately guiding a driver on how to operate a turn signal.

Means for Solving the Problems

[0007] In order to solve the above problems, an in-vehicle system according to an aspect of this embodiment includes an acquisition unit that acquires detection data indicating the surrounding situation of the vehicle from a sensor installed in the vehicle, a recognition unit that recognizes at least one of a road sign or a road surface marking from the detection data, and based on at least one of the recognized road sign or road surface marking, specifies the type of connection form of a road connection point where lanes merge, branch, or intersect, and notifies the driver of guidance on how to operate a turn signal according to the traffic rules corresponding to the specified type of connection form.

[0008] Note that any combination of the above components, and those obtained by converting the expressions of this embodiment among a method, an apparatus, a system, a recording medium, a computer program, etc., are also effective as aspects of this embodiment.

Advantages of the Invention

[0009] According to this embodiment, it is possible to accurately guide a driver on how to operate a turn signal.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

MODE FOR CARRYING OUT THE INVENTION

[0011] FIG. 1 is a diagram showing a configuration example of the in-vehicle system 1 according to Embodiment 1. The in-vehicle system 1 according to Embodiment 1 is configured by linking a drive recorder 1a and a car navigation system 1b. The drive recorder 1a and the car navigation system 1b are connected by, for example, a dedicated communication cable. Wireless communication may also be used.

[0012] The car navigation system 1b includes an operation unit 50, a GPS sensor 51, an acceleration sensor 52, a processing unit 60, a map data storage unit 70, a display unit 81, and an audio output unit 82. In this embodiment, the display unit 81 and the audio output unit 82 also function as a notification unit 80.

[0013] The operation unit 50 includes user interfaces such as a touch panel and physical buttons. The operation unit 50 receives the user's operations and outputs operation signals corresponding to the user's operation contents to the processing unit 60. The display unit 81 includes a touch panel display such as a liquid crystal display, an organic EL display, and a mini LED display. The audio output unit 82 includes a speaker.

[0014] The GPS sensor 51 is an example of a GNSS (Global Navigation Satellite System) receiver, detects the current position information of the vehicle in terms of latitude, longitude, and altitude, and outputs the detected position information to the processing unit 60. Specifically, the GPS sensor 51 receives radio waves including their respective transmission times from a plurality of GPS satellites, and calculates the latitude, longitude, and altitude of the reception point based on the plurality of transmission times respectively included in the plurality of received radio waves.

[0015] For example, a three-axis acceleration sensor is used as the acceleration sensor 52, which detects the acceleration values in the XYZ directions applied to the acceleration sensor 52 and outputs them to the processing unit 60.

[0016] The processing unit 60 includes an acquisition unit 61, a route guidance unit 62, and a user interface control unit 63 (hereinafter referred to as the UI control unit 63). The processing unit 60 is realized by the cooperation of hardware resources and software resources, or by hardware resources only. As hardware resources, a CPU, a ROM, a RAM, a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), and other LSIs can be used. As software resources, programs such as firmware can be used.

[0017] The acquisition unit 61 acquires the departure point and the destination input by the user from the operation unit 50 on the route search screen displayed on the display unit 81. The acquisition unit 61 acquires the current position information of the vehicle from the GPS sensor 51. The acquisition unit 61 acquires the acceleration value from the acceleration sensor 52. The acquisition unit 61 acquires map information including the range from the departure point to the destination from the map data holding unit 70.

[0018] The route guidance unit 62 searches for a route from the departure point to the destination with reference to the acquired map information. When one or more routes are searched by the route guidance unit 62, the UI control unit 63 causes the display unit 81 to display the one or more routes. The route guidance unit 62 sets the route selected by the user from the operation unit 50 as the navigation route.

[0019] The UI control unit 63 superimposes an arrow indicating the current position of the vehicle on the map image near the current position of the vehicle acquired from the GPS sensor 51 and causes the display unit 81 to display it. Based on the route guidance information corresponding to the set navigation route and the current position information of the vehicle, the UI control unit 63 causes the display unit 81 to display a guide of the traveling direction and outputs an audio guide from the audio output unit 82.

[0020] In the present embodiment, the route guidance unit 62 has a function of transmitting the route guidance information and the current position information of the vehicle to the drive recorder 1a in real time via a communication cable.

[0021] The drive recorder 1a includes an imaging unit 10, a GPS sensor 11, an acceleration sensor 12, a processing unit 20, a display unit 31, an audio output unit 32, and a recording medium 40. In the present embodiment, the display unit 31 and the audio output unit 32 function as a notification unit 30.

[0022] The drive recorder 1a is installed at a specific position inside the vehicle (e.g., the upper part of the windshield) in a direction where the imaging unit 10 images the front of the vehicle. The drive recorder 1a may further include an external rear imaging unit that images the rear of the vehicle. The drive recorder 1a may be a digital rearview mirror type drive recorder that also has the function of a digital rearview mirror. The drive recorder 1a may be a smartphone or an information terminal device installed with predetermined application software, and may be installed and used on the dashboard or the like inside the vehicle. Note that the camera (corresponding to the imaging unit 10) and various sensors (corresponding to the GPS sensor 11 and the acceleration sensor 12) provided in the smartphone or the information terminal device can be used.

[0023] The imaging unit 10 includes a lens and a solid-state imaging device. For the solid-state imaging device, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor can be used. The solid-state imaging device converts the light incident through the lens into an electrical video signal and outputs it to the processing unit 20. The imaging unit 10 may image a visible light image, an infrared image, or both.

[0024] The GPS sensor 11 is an example of a GNSS receiver, detects the current position information of the vehicle in terms of latitude, longitude, and altitude, and outputs the detected position information to the processing unit 20. For example, a three-axis acceleration sensor is used for the acceleration sensor 12, and the acceleration values in the XYZ directions applied to the acceleration sensor 12 are detected and output to the processing unit 20.

[0025] The display unit 31 includes a touch panel display such as a liquid crystal display, an organic EL display, or a mini LED display. The audio output unit 32 includes a speaker.

[0026] The recording medium 40 is a large-capacity non-volatile recording medium and can record the video around the vehicle imaged by the imaging unit 10. For the recording medium 40, for example, a built-in flash memory (e.g., NAND-type flash memory) or an external semiconductor memory card is used.

[0027] The processing unit 20 includes an acquisition unit 21, an image recognition unit 22, a notification control unit 23, and a recording control unit 24. The processing unit 20 is realized by the cooperation of hardware resources and software resources, or only by hardware resources. As hardware resources, a CPU, ROM, RAM, GPU, ASIC, FPGA, and other LSIs can be used. As software resources, programs such as firmware can be used.

[0028] The acquisition unit 21 acquires the video in front of the vehicle from the imaging unit 10. The acquisition unit 21 acquires the current position information of the vehicle from the GPS sensor 11. The acquisition unit 21 acquires the acceleration value from the acceleration sensor 12. The acquisition unit 21 acquires the route guidance information and the current position information of the vehicle from the associated car navigation system 1b.

[0029] When a drive recorder 1a and an ECU (Electronic Control Unit) that manages the turn signal switch are connected via the in-vehicle network, the acquisition unit 21 can acquire the operation information of the turn signal switch via the in-vehicle network. For the in-vehicle network, for example, CAN (Controller Area Network) or LIN (Local Interconnect Network) can be used.

[0030] The image recognition unit 22 analyzes the video acquired from the imaging unit 10 to detect at least one of road signs or road markings. For example, a learning model of traffic signs is generated by machine learning a large number of sample images of all types of traffic signs in advance. For example, a learning model of traffic signs is generated using a CNN (Convolution Neural Network) model. The image recognition unit 22 extracts the contour of the video acquired from the imaging unit 10 and identifies the road signs in the video with reference to the learning model of traffic signs. Note that, simply, the road signs may be identified from the captured video by pattern matching.

[0031] Similarly, a learning model of road markings is generated by machine learning a large number of sample images of road markings such as road marking lines, characters, numbers, marks, etc. in advance. The image recognition unit 22 extracts the contour of the video acquired from the imaging unit 10 and identifies the road markings in the video with reference to the learning model of road markings.

[0032] Note that the road marking lines may be identified as follows. The image recognition unit 22 generates an edge image of the video acquired from the imaging unit 10 using, for example, the Canny method. The image recognition unit 22 performs Hough transform on the generated edge image to detect a straight line. In detecting the road marking lines, the color of the straight line to be detected may be limited to white and orange.

[0033] When the image recognition unit 22 detects a stop line extending horizontally in the image, the distance from the imaging unit 10 to the stop line can be estimated based on the y - coordinate of the stop line detected in the image and a previously derived table or function. The table or function describes the relationship between the y - coordinate of the stop line in the image and the distance from the imaging unit 10 to the stop line, which is determined according to the installation position (height from the road surface) of the imaging unit 10 and the viewing angle of the imaging unit 10.

[0034] When the imaging unit 10 is configured as a binocular stereo camera, the image recognition unit 22 can estimate the distance from the imaging unit 10 to the stop line based on the parallax of the images captured by the two eyes. Also, when the imaging unit 10 is equipped with a TOF (Time Of Flight) sensor such as LiDAR (Light Detection and Ranging), the image recognition unit 22 can estimate the distance from the imaging unit 10 to the stop line based on the detection value of the TOF sensor.

[0035] In addition, the image recognition unit 22 may analyze a distance image (depth map) in which the values detected by the TOF sensor are arranged in a bitmap pattern to detect at least one of a road sign or a road surface marking. Also, the distance to the stop line may be estimated using a millimeter-wave radar or an ultrasonic sensor. In this way, by acquiring detection data such as an image or a reflected wave indicating the surrounding situation of the vehicle from at least one sensor installed in the vehicle, at least one of a road sign or a road surface marking is recognized.

[0036] The notification control unit 23 identifies the type of connection form of a point where lanes merge, branch, or intersect (hereinafter referred to as a road connection point in this specification) based on at least one of the image-recognized road sign or road surface marking. Specific examples of the connection form of the road connection point will be described later. When the road information included in the route guidance information acquired from the car navigation system 1b includes information on the connection form of the road connection point, the notification control unit 23 may identify the connection form of the road connection point based on the information included in the route guidance information.

[0037] The notification control unit 23 notifies the driver, by at least one of video or audio, of how to operate the turn signal in accordance with the traffic rules corresponding to the type of connection form of the specified road connection point. During route guidance, the notification control unit 23 can notify the driver, by at least one of video or audio, of guidance on the direction in which the turn signal should be operated (guidance on whether to operate the left turn signal or the right turn signal) based on the traffic rules corresponding to the type of connection form of the specified road connection point and the direction in which the vehicle should proceed at the road connection point based on the route guidance information. The notification control unit 23 may also specify the direction in which the turn signal should be operated with reference to the direction indicated by the turn signal of the vehicle ahead in the forward video recognized by image recognition.

[0038] When the acceleration sensor 12 detects an acceleration value equal to or greater than a predetermined value in the always-recording mode, the recording control unit 24 determines that a large impact has been applied to the drive recorder 1a due to an accident collision, sudden braking, etc., and stores the video data at the time of detection in the recording medium 40 with overwrite prohibited (automatic deletion prohibited). When the acceleration sensor 12 detects an acceleration value equal to or greater than a predetermined value in the event recording mode, the recording control unit 24 starts recording the video data captured by the imaging unit 10 in the recording medium 40.

[0039] In the present embodiment, the recording control unit 24 discards, without storing in the recording medium 40, the video when the vehicle captured by the imaging unit 10 passes through the road connection point if a turn signal operation in accordance with the above-described guidance on how to operate the turn signal is performed, and can control to store in the recording medium 40 if a turn signal operation contrary to the guidance on how to operate the turn signal is performed. The video when passing through the road connection point stored in the recording medium 40 is stored with overwrite prohibited. At that time, the position information of the road connection point is also stored as metadata.

[0040] The video when passing through the road connection point stored in the recording medium 40 and a message indicating an error in how to operate the turn signal may be transmitted via a network to a pre-registered address (e.g., email address, SNS address) at a later date. When the vehicle is a commercial vehicle, it may be transmitted to the address of the operation manager of the company to which the driver belongs.

[0041] As an example of a road connection point where it is easy to make a mistake in the way of turning on the blinker, there is a merging point where a side road merges into the main line. The types of merging points where a side road merges into the main line include intersections and merging into an additional lane. In the case of an intersection, since the vehicle can enter in either the left or right direction, it is necessary to turn on the right blinker when turning right and the left blinker when turning left. On the other hand, since an additional lane can only merge, similar to when changing lanes, in countries where traffic moves on the left side, it is necessary to turn on the right blinker when merging, and in countries where traffic moves on the right side, it is necessary to turn on the left blinker when merging.

[0042] The notification control unit 23 determines the presence or absence of a stop line by image recognition, and determines that it is an intersection if there is a stop line, and determines that it is a merge into an additional lane if there is no stop line. Further, the notification control unit 23 may detect a warning sign of "there is an intersection", a warning sign of "there is merging traffic", or a regulation sign of "prohibited from proceeding outside the specified direction" by image recognition, and discriminate whether it is an intersection or a merge into an additional lane. Further, the notification control unit 23 may discriminate whether it is an intersection or a merge into an additional lane based on the road information included in the route guidance information. In principle, when the road connection point is an intersection, the notification control unit 23 notifies the driver that both right and left turns are possible, and when it is an additional lane, notifies the driver that it is necessary to turn on the blinker for lane change.

[0043] Figs. 2(a) and (b) are diagrams showing an example of guidance on how to turn on the blinker, which is displayed when the type of merging point where a side road merges into the main line is an intersection. The example shown in Fig. 2(a) is an example where the direction in which the vehicle V1 should travel at an intersection based on the route guidance information is the left direction (the direction toward the upper part of the screen). The notification control unit 23 transmits an instruction signal for notifying a guidance to induce turning on the left blinker to the car navigation system 1b. The UI control unit 63 of the car navigation system 1b superimposes an OSD (On-Screen Display) on the route guidance screen 81a in which the pictogram Bl of the left blinker is lit and the pictogram Br of the right blinker is not lit. The UI control unit 63 may cause the pictogram Bl of the left blinker to blink.

[0044] Note that the notification control unit 23 may cause the display unit 31 of the drive recorder 1a to display an OSD in which the pictogram Bl of the left turn signal is lit and the pictogram Br of the right turn signal is not lit. The notification control unit 23 causes an audio guidance such as "Temporarily stop at the stop line, turn on the left turn signal, and turn left." to be output from the audio output unit 82 of the car navigation system 1b or the audio output unit 32 of the drive recorder 1a.

[0045] The example shown in FIG. 2(b) is an example in which the direction in which the vehicle V1 should travel at the intersection based on the route guidance information is the right direction (the direction toward the lower part of the screen). The notification control unit 23 transmits an instruction signal for notifying a guidance for guiding the vehicle to turn on the right turn signal to the car navigation system 1b. The UI control unit 63 of the car navigation system 1b superimposes an OSD in which the pictogram Bl of the left turn signal is not lit and the pictogram Br of the right turn signal is lit on the route guidance screen 81b. The notification control unit 23 causes an audio guidance such as "Temporarily stop at the stop line, turn on the right turn signal, and turn right." to be output from the audio output unit 82 of the car navigation system 1b or the audio output unit 32 of the drive recorder 1a.

[0046] When approaching an intersection where the vehicle merges from the side road into the main line when not in the middle of route guidance, the notification control unit 23 causes an audio guidance such as "Temporarily stop at the stop line. If you want to turn left, turn on the left turn signal and turn. If you want to turn right, turn on the right turn signal and turn." to be output from the audio output unit 32 of the drive recorder 1a.

[0047] Figure 3 is a diagram showing an example of guidance on how to turn on the turn signal, which is displayed when the type of the merging point where the side road merges into the main line is an acceleration lane. The notification control unit 23 transmits an instruction signal for notifying guidance to induce turning on the right turn signal to the car navigation system 1b. The UI control unit 63 of the car navigation system 1b superimposes an OSD on the route guidance screen 81c where the pictogram Bl of the left turn signal is off and the pictogram Br of the right turn signal is on. The notification control unit 23 outputs voice guidance such as "You will merge soon. Please turn on the right turn signal and be careful of the vehicles in the merging lane when merging." from the voice output unit 82 of the car navigation system 1b or the voice output unit 32 of the drive recorder 1a.

[0048] The notification control unit 23 executes the notification to the driver by the pictorial display of the turn signal or the voice guidance 30 m or more before or 3 seconds or more before the intersection or the merging point of the acceleration lane. Note that the timing for issuing the guidance on how to turn on the turn signal may be a specification that can be changed by the driver.

[0049] As another example of a road connection point where it is easy to make a mistake in how to turn on the turn signal, there are a rotary and a roundabout (circular intersection). A rotary is a type of circular plane intersection where vehicles entering the circular road are given priority, and the traffic on the circular road is one-way clockwise (right turn). At a rotary, it is necessary to follow the surrounding signals and traffic signs. It is necessary to turn on the left turn signal when entering, and also when exiting the rotary, it is necessary to turn on the left turn signal when passing by the side just before the exit.

[0050] A roundabout is a type of circular plane intersection where the traffic on the circular road is given priority, and the traffic on the circular road is one-way clockwise. It is not subject to signal or stop regulations. Vehicles enter the circular road slowly, and a stop is not necessary if there are no passing vehicles on the circular road. It is not necessary to turn on the turn signal when entering, and when exiting the circular intersection, it is necessary to turn on the left turn signal when passing by the side just before the exit.

[0051] The notification control unit 23 detects a warning sign of "with rotary" or a regulatory sign of "right-turn traffic in a circular intersection" by image recognition, and determines whether it is a rotary or a roundabout. Further, the notification control unit 23 may determine whether it is a rotary or a roundabout based on the road information included in the route guidance information.

[0052] FIG. 4(a) shows a regulatory sign of "right-turn traffic in a circular intersection", and FIG. 4(b) shows a warning sign of "with rotary". The regulatory sign of "right-turn traffic in a circular intersection" has a circular shape, and a white symbol is drawn on a blue background. The warning sign of "with rotary" has a square shape rotated by 90 degrees, and a black symbol is drawn on a yellow background.

[0053] FIGS. 5(a) and 5(b) are diagrams showing an example of guidance on how to operate the turn signal when the type of the road connection point is a roundabout. The examples shown in FIGS. 5(a) and 5(b) are examples when the vehicle V1 should turn right in the roundabout based on the route guidance information.

[0054] FIG. 5(a) is a diagram showing an example of guidance on how to operate the turn signal immediately before entering the roundabout. The notification control unit 23 transmits an instruction signal for notifying guidance that no turn signal is required to the car navigation system 1b. The UI control unit 63 of the car navigation system 1b superimposes an OSD on the route guidance screen 81d in which the pictogram Bl of the left turn signal is not lit and the pictogram Br of the right turn signal is also not lit. The notification control unit 23 outputs voice guidance such as "You are approaching a circular intersection soon. The road where the circulating vehicles have priority. Please slow down and turn according to the direction of the sign. Merge without using the turn signal." from the voice output unit 82 of the car navigation system 1b or the voice output unit 32 of the drive recorder 1a.

[0055] Figure 5(b) is a diagram showing an example of guidance on how to use the turn signal, which is displayed immediately before the exit of the roundabout. The notification control unit 23 transmits an instruction signal for notifying guidance to induce the use of the left turn signal to the car navigation system 1b. The UI control unit 63 of the car navigation system 1b superimposes an OSD on the route guidance screen 81e, in which the pictogram Bl of the left turn signal is lit and the pictogram Br of the right turn signal is not lit. The notification control unit 23 outputs voice guidance such as "You are approaching the exit. Please turn on the left turn signal and turn left." from the voice output unit 82 of the car navigation system 1b or the voice output unit 32 of the drive recorder 1a.

[0056] When approaching the roundabout when not in the middle of route guidance, the notification control unit 23 outputs voice guidance such as "You are approaching a roundabout. The road on which the circulating vehicles have priority. Please turn slowly in accordance with the direction of the sign. Please merge without using the turn signal. When exiting the roundabout, please turn on the left turn signal and turn left." from the voice output unit 32 of the drive recorder 1a.

[0057] Figures 6(a) and (b) are diagrams showing an example of guidance on how to use the turn signal, which is displayed when the type of road connection point is a rotary. The examples shown in Figures 6(a) and (b) are examples where the vehicle V1 should turn right at the rotary based on the route guidance information.

[0058] FIG. 6(a) is a diagram showing an example of guidance on how to turn on the blinker, which is displayed immediately before entering the rotary. The notification control unit 23 transmits an instruction signal for notifying guidance to induce turning on the left blinker to the car navigation system 1b. The UI control unit 63 of the car navigation system 1b superimposes an OSD on the route guidance screen 81f in which the pictogram Bl of the left blinker is lit and the pictogram Br of the right blinker is not lit. The notification control unit 23 outputs voice guidance such as "You will soon reach the rotary. Vehicles entering have priority, so be careful of merging vehicles while circulating. " from the voice output unit 82 of the car navigation system 1b or the voice output unit 32 of the drive recorder 1a.

[0059] FIG. 6(b) is a diagram showing an example of guidance on how to turn on the blinker, which is displayed immediately before exiting the rotary. The notification control unit 23 transmits an instruction signal for notifying guidance to induce turning on the left blinker to the car navigation system 1b. The UI control unit 63 of the car navigation system 1b superimposes an OSD on the route guidance screen 81g in which the pictogram Bl of the left blinker is lit and the pictogram Br of the right blinker is not lit. The notification control unit 23 outputs voice guidance such as "You will soon reach the exit. Turn on the left blinker and turn left. " from the voice output unit 82 of the car navigation system 1b or the voice output unit 32 of the drive recorder 1a.

[0060] When approaching the rotary when not in route guidance, the notification control unit 23 outputs voice guidance such as "You will soon reach the rotary. Vehicles entering have priority, so be careful of merging vehicles while circulating. When exiting the rotary, turn on the left blinker and turn left. " from the voice output unit 32 of the drive recorder 1a.

[0061] As yet another example of a road connection point where it is easy to make a mistake in how to operate the turn signal, there is a special intersection. As an example of a special intersection, there is a T-junction where the priority road of a wide road makes an L-shaped curve, and the non-priority road of a narrow road is connected to the bent part of the L-shaped curve. When entering straight from the non-priority road of the narrow road onto the priority road of the wide road that makes an L-shaped curve to the right, it is necessary to turn on the left turn signal. Conversely, when entering straight from the non-priority road of the narrow road onto the priority road of the wide road that makes an L-shaped curve to the left, it is necessary to turn on the right turn signal.

[0062] The notification control unit 23 detects at least one of the road information included in the route guidance information and the stop line, the warning sign of "road intersection", and the regulatory sign of "stop temporarily" by image recognition, and discriminates a special T-junction.

[0063] Figs. 7(a) and 7(b) are diagrams showing an example of guidance on how to operate the turn signal, which is displayed when approaching a T-junction where the priority road of a wide road makes an L-shaped curve to the right and the non-priority road of a narrow road is connected to the bent part of the L-shaped curve. The example shown in Fig. 7(a) is an example where the direction in which the vehicle V1 should travel at the T-junction based on the route guidance information is straight ahead. The notification control unit 23 transmits an instruction signal for notifying a guidance for inducing to turn on the left turn signal to the car navigation system 1b. The UI control unit 63 of the car navigation system 1b superimposes an OSD on the route guidance screen 81h in which the pictogram Bl of the left turn signal is lit and the pictogram Br of the right turn signal is not lit. The notification control unit 23 outputs voice guidance such as "Stop temporarily at the stop line, turn on the left turn signal and go straight ahead." from the voice output unit 82 of the car navigation system 1b or the voice output unit 32 of the drive recorder 1a.

[0064] The example shown in Fig. 7(b) is an example where the direction in which the vehicle V1 should proceed at a T-junction based on route guidance information is a right turn. The notification control unit 23 transmits an instruction signal for notifying guidance to induce the vehicle to turn on the right turn signal to the car navigation system 1b. The UI control unit 63 of the car navigation system 1b superimposes an OSD on the route guidance screen 81i where the pictogram Bl of the left turn signal is off and the pictogram Br of the right turn signal is on. The notification control unit 23 outputs voice guidance such as "Temporarily stop at the stop line, turn on the right turn signal and turn right." from the voice output unit 82 of the car navigation system 1b or the voice output unit 32 of the drive recorder 1a.

[0065] When not in the middle of route guidance and approaching the above special T-junction, the notification control unit 23 outputs voice guidance such as "Temporarily stop at the stop line. If you want to go straight, turn on the left turn signal and go straight. If you want to turn right, turn on the right turn signal and turn right." from the voice output unit 32 of the drive recorder 1a.

[0066] Fig. 8 is a flowchart showing the flow of the guidance process 1 for how to turn on the turn signal by the in-vehicle system 1 according to Embodiment 1. The acquisition unit 21 of the drive recorder 1a acquires an image of the front of the vehicle from the imaging unit 10 (step S10). The acquisition unit 21 acquires route guidance information from the car navigation system 1b and acquires the current position information of the vehicle from the car navigation system 1b or the GPS sensor 11 (step S11). The image recognition unit 22 analyzes the image acquired from the imaging unit 10 and detects at least one of road signs or road markings (step S12).

[0067] The notification control unit 23 detects the presence or absence of approach to a road connection point based on at least one of the image-recognized road signs or road markings, the acquired route guidance information, and the current position information of the vehicle (step S13). When detecting the approach to a road connection point (step S13: Yes), the notification control unit 23 identifies the type of connection form of the road connection point based on at least one of the road signs, road markings, and road information included in the route guidance information recognized by the image (step S14). When the approach to the road connection point is not detected (step S13: No), the process returns to step S10.

[0068] Based on the traffic rules corresponding to the type of connection form of the identified road connection point and the direction in which the vehicle should proceed at the road connection point based on the route guidance information, the notification control unit 23 identifies the direction in which the turn signal should be turned on (step S15). The notification control unit 23 controls so that the guidance on the direction in which the turn signal should be turned on is notified to the driver by at least one of video or audio (step S16). The processes of step S10 - step S16 described above are repeatedly executed until the driving ends (step S110: Yes) (step S110: No).

[0069] FIG. 9 is a flowchart showing the flow of the guidance process 2 for the way of turning on the turn signal by the in-vehicle system 1 according to Embodiment 1. Since the processes of step S10 - step S16 are the same as the guidance process 1 shown in FIG. 8, the description thereof is omitted. After the guidance on the direction in which the turn signal should be turned on is notified to the driver, the acquisition unit 21 of the drive recorder 1a acquires the operation information of the turn signal switch via the in-vehicle network (step S17).

[0070] The recording control unit 24 compares the direction in which the driver should turn on the turn signal with the operation information of the turn signal switch (step S18). When both match (step S18: Yes), the recording control unit 24 discards the video taken by the imaging unit 10 when the vehicle passes through the road connection point without saving it to the recording medium 40 (skips step S19). When both do not match (step S18: No), the recording control unit 24 saves the video when the vehicle passes through the road connection point to the recording medium 40 (step S19). The processes of steps S10 - S19 described above are repeatedly executed until the driving ends (step S110: Yes) (step S110: No).

[0071] As described above, the in - vehicle system 1 according to Embodiment 1 has been described in terms of the configuration of the drive recorder 1a and the car navigation system 1b. However, these functions may be realized by a smartphone or an information terminal device on which application software (program) is installed. Either both or either one of the configurations of the drive recorder 1a and the car navigation system 1b may be realized by a single smartphone or information terminal device.

[0072] FIG. 10 is a diagram showing a configuration example of the in - vehicle system 1 according to Embodiment 2. The in - vehicle system 1 according to Embodiment 2 is composed of the drive recorder 1a alone. Therefore, it is not possible to obtain route guidance information from the car navigation system 1b.

[0073] FIG. 11 is a flowchart showing the flow of the guidance process for how to turn on the turn signal by the in - vehicle system 1 according to Embodiment 2. The acquisition unit 21 of the drive recorder 1a acquires the video in front of the vehicle from the imaging unit 10 (step S20). The acquisition unit 21 acquires the current position information of the vehicle from the GPS sensor 11 (step S21). The image recognition unit 22 analyzes the video acquired from the imaging unit 10 and detects at least one of road signs or road surface markings (step S22).

[0074] The notification control unit 23 detects the presence or absence of approach to a road connection point based on at least one of the image-recognized road signs and road surface markings, and the current position information of the vehicle (step S23). When detecting the approach to a road connection point (step S23: Yes), the notification control unit 23 identifies the type of the road connection point based on at least one of the image-recognized road signs and road surface marking information (step S24). If the type of the road connection point cannot be identified, the guidance on how to operate the turn signal described later is not notified. When the approach to the road connection point is not detected (step S23: No), the process returns to step S20.

[0075] The notification control unit 23 identifies how to operate the turn signal based on the traffic rules corresponding to the type of the connection form of the identified road connection point (step S25). The notification control unit 23 controls so that the guidance on how to operate the turn signal is notified to the driver by at least one of video and audio (step S26). The processes of step S20 - step S26 described above are repeatedly executed until the driving ends (step S27: Yes) (step S27: No).

[0076] As described above, according to the embodiment, based on at least one of the road signs and road surface markings detected by image recognition, by guiding the driver on how to operate the turn signal, it is possible to accurately guide how to operate the turn signal. Since the way of operating the turn signal at a road connection point where the driver has misrecognized and is prone to mistakes is corrected, it contributes to traffic safety. In addition, by using the detection data of the sensors installed in the vehicle to detect the approach to the road connection point, the guidance on how to operate the turn signal can be notified at the right timing. Also, even when not cooperating with the car navigation system, the rules on how to operate the turn signal at the road connection point can be guided to the driver. Further, even when the type of the intersection changes and the update of the map information of the car navigation system 1b is delayed, it is possible to guide the correct way of operating the turn signal according to the correct traffic rules.

[0077] The above has described the present invention based on embodiments. These embodiments are illustrative, and it is understood by those skilled in the art that various modifications are possible for each of these components and combinations of each processing process, and such modifications are also within the scope of the present invention.

[0078] In Embodiment 2, an example in which the in-vehicle system 1 is configured by the drive recorder 1a alone has been described. In this regard, the in-vehicle system 1 may be configured by a combination of a single external camera and the car navigation system 1b. Further, the in-vehicle system 1 may be created by a dedicated in-vehicle device, a smartphone on which application software (program) is installed, or an information terminal device.

[0079] The guidance on the rules for operating the turn signal when not in route guidance may not be performed after it is estimated that the driver has learned. The notification control unit 23 records the direction in which the driver should turn on the turn signal and the position information of the road connection point where the operation information of the turn signal switch matches in the non-volatile memory of the processing unit 20 or the recording medium 40. When the position information of the approaching road connection point matches the position information of the road connection point where the turn signal operation was correct, which is recorded in the non-volatile memory of the processing unit 20 or the recording medium 40, the notification control unit 23 skips the guidance on the rules for operating the turn signal.

Explanation of Reference Numerals

[0080] 1 In-vehicle system, 1a Drive recorder, 1b Car navigation system, 10 Imaging unit, 11 GPS sensor, 12 Acceleration sensor, 20 Processing unit, 21 Acquisition unit, 22 Image recognition unit, 23 Notification control unit, 24 Recording control unit, 30 Notification unit, 31 Display unit, 32 Voice output unit, 40 Recording medium, 50 Operation unit, 51 GPS sensor, 52 Acceleration sensor, 60 Processing unit, 61 Acquisition unit, 62 Route guidance unit, 63 UI control unit, 70 Map data holding unit, 80 Notification unit, 81 Display unit, 82 Voice output unit.

Claims

1. An acquisition unit that acquires detection data indicating the surrounding situation of the vehicle from a sensor installed in the vehicle; A recognition unit that recognizes at least one of a road sign or a road marking from the detection data; Based on at least one of the recognized road sign or road marking, identify the type of connection form of a road connection point where lanes merge, branch, or intersect, and notify the driver of guidance on how to operate the turn signal according to the traffic rules corresponding to the identified type of connection form. A notification control unit; An in-vehicle system comprising:

2. The acquisition unit further acquires route guidance information of the vehicle and current position information of the vehicle, The notification control unit notifies guidance on whether to turn on the left turn signal or the right turn signal based on the traffic rules corresponding to the type of connection form and the direction in which the vehicle should travel at the road connection point based on the route guidance information. The in-vehicle system according to claim 1.

3. The sensor is an imaging unit installed in the vehicle, When the vehicle passes through the road connection point imaged by the imaging unit, if a turn signal operation is performed according to the guidance on how to operate the turn signal, it is discarded without being saved, and if a turn signal operation is performed contrary to the guidance on how to operate the turn signal, it is saved. A recording control unit for controlling as follows. The in-vehicle system according to claim 1 or 2. The in-vehicle system according to claim 1 or 2.

4. When the road connection point is an intersection, the notification control unit notifies that both right and left turns are possible, and when it is a merge of an additional lane, it notifies that it is necessary to turn on the turn signal for lane change. The in-vehicle system according to claim 1 or 2.

5. A process of acquiring detection data indicating the surrounding situation of the vehicle from a sensor installed in the vehicle; A process of recognizing at least one of a road sign or a road marking from the detection data; Based on at least one of the recognized road sign or road marking, identify the type of connection form of a road connection point where lanes merge, branch, or intersect, and a process of notifying the driver of guidance on how to operate the turn signal according to the traffic rules corresponding to the identified type of connection form. A guidance program for causing a computer to execute.

Images