Traffic sign information display system and traffic sign information display method

Abstract

To effectively display sign information in accordance with a behavior state of a driver.SOLUTION: A sign information display system comprises: an imaging device 20 that captures a surrounding image around a vehicle V; a sign acquisition unit 100 that acquires sign information for a traffic sign present in a surrounding region on the basis of the surrounding image; a driver specific behavior information acquisition unit 120 that acquires driver specific behavior information including driving operation information for a driver; display devices 40 to 43 that are located at positions visible to the driver and can display the sign information acquired by the sign acquisition unit 100; and a display control unit 130 that controls screen display of the display devices 40 to 43. The display control unit 130 selects one of the plurality of display devices 40 to 43 to display the sign information acquired by the sign acquisition unit 100 on the basis of a type of the sign information acquired by the sign acquisition unit 100 and the driver specific behavior information acquired by the driver specific behavior information acquisition unit 120.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a sign information display system and a sign information display method. [Background technology]

[0002] Patent Document 1 discloses a device that captures a forward image with a camera mounted on a vehicle, recognizes traffic signs from the captured forward image, and displays sign icons indicating the recognized traffic signs on an in-vehicle monitor, thereby presenting sign information to the driver. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-095756 A Summary of the Invention

[0004] When the driver turns the vehicle left, the driver's line of sight is directed to the left front of the vehicle (diagonally forward to the left). When the driver turns the vehicle right or rotates, or when the driver overtakes a preceding vehicle, the driver's line of sight is directed to the right front of the vehicle (diagonally forward to the right).

[0005] The conventional device displays traffic sign icons on a central monitor, such as a general-purpose monitor or a multi-information display, located approximately in the center of the instrument panel. Therefore, depending on the driver's behavior, such as when the driver's gaze is directed to the left or right front of the vehicle, the driver may overlook the display on the central monitor, and there is room for improvement.

[0006] The present disclosure has been made to solve the above-mentioned problems. That is, one of the objects of the present disclosure is to effectively display sign information in accordance with the driver's behavior state.

[0007] The device of the present disclosure comprises: an image capturing device (20) for capturing an image of a surrounding area including at least a forward area of ​​a vehicle (V); a sign acquisition unit (100) that acquires sign information of traffic signs present in the surrounding area based on the surrounding image; A driver specific behavior information acquisition unit (120) that acquires driver specific behavior information including driving operation information of a driver who drives the vehicle (V); A plurality of display devices (40 to 43) are arranged in at least two or more different positions within the vehicle interior of the vehicle (V) that are visible to the driver and are capable of displaying the sign information acquired by the sign acquisition unit (100); A sign information display system (1) for a vehicle (V) including a display control unit (130) that controls screen display of the plurality of display devices (40 to 43), The display control unit (130) A selection is made as to which of the plurality of display devices (40 to 43) to display the sign information acquired by the sign acquisition unit (100) based on the type of the sign information acquired by the sign acquisition unit and the driver specific behavior information acquired by the driver specific behavior information acquisition unit (120).

[0008] The method of the present disclosure comprises: Obtaining a surrounding image by photographing a surrounding area including at least a front area of ​​the vehicle (V); acquiring sign information of traffic signs present in the surrounding area based on the surrounding image; Acquire driver specific behavior information including driving operation information of a driver who drives the vehicle (V), A sign information display method for a vehicle (V) that executes display control to display the acquired sign information on a plurality of display devices (40 to 43) arranged at at least two or more different positions within a cabin of the vehicle (V) that are visible to the driver, In the display control, A selection is made as to which of the plurality of display devices (40-43) the acquired sign information is to be displayed on, based on the type of the acquired sign information and the acquired driver specific behavior information.

[0009] According to the above configuration, the sign information display system (1) displays the sign information acquired from the surrounding image on the display device (40-43) disposed at a suitable position according to the type of the sign information and the behavior state of the driver. This makes it possible to effectively present the sign information to the driver and to call his / her attention.

[0010] In another aspect of the present disclosure, The display devices (40 to 43) include The vehicle includes a left display device (41) disposed on the left front side of a driver's seat in the vehicle interior, and a right display device (42) disposed on the right front side of the driver's seat, The driver specific behavior information acquisition unit (120) As the driver specific behavior information, a counterclockwise operation and a clockwise operation of a turn signal lever of the vehicle (V) by the driver are acquired, The display control unit (130) When the type of the sign information acquired by the sign acquisition unit (100) is specific sign information, if the driver specific behavior information acquired by the driver specific behavior information acquisition unit (120) is an operation in the counterclockwise direction, the specific sign information is displayed on the left display device (41), and if the driver specific behavior information acquired by the driver specific behavior information acquisition unit (120) is an operation in the clockwise direction, the specific sign information is displayed on the right display device (42).

[0011] According to this aspect, the sign information display system (1) selects a display device (41, 42) in a direction in which the driver's line of sight is likely to be directed, based on the driver's driving operation, and displays specific sign information on the selected display device (41, 42). This makes it possible to appropriately display the specific sign information on the display device (41, 42) that is suitable for the driver's behavior state, and effectively prevents the driver from overlooking the information.

[0012] In another aspect of the present disclosure, The display devices (40 to 43) include The vehicle includes a left display device (41) disposed on the left front side of a driver's seat in the vehicle interior, and a right display device (42) disposed on the right front side of the driver's seat, The driver specific behavior information acquisition unit (120A) a camera (50) for photographing a face of the driver seated in the driver's seat, and acquiring, as the driver specific behavior information, a line of sight direction of the driver based on a face image photographed by the camera (50); The display control unit (130A) When the type of the sign information acquired by the sign acquisition unit (100) is specific sign information, if the line of sight direction acquired by the driver specific behavior information acquisition unit (120A) is to the left front, the specific sign information is displayed on the left display device (41), and when the line of sight direction acquired by the driver specific behavior information acquisition unit (120A) is to the right front, the specific sign information is displayed on the right display device (42).

[0013] According to this aspect, the sign information display system (1) directly detects the line of sight of the driver based on the face image of the driver, and displays specific sign information on the display device (41, 42) located in the detected line of sight. This makes it possible to accurately display the specific sign information on the display device (41, 42) located in the line of sight of the driver, and effectively prevents the driver from overlooking the information.

[0014] In another aspect of the present disclosure, The display control unit (130) When the specific sign information acquired by the sign acquisition unit (100) is a sign information of a left turn prohibition that prohibits a driving operation of turning the traveling direction of the vehicle (V) to the left, and when the driver specific behavior information acquired by the driver specific behavior information acquisition unit (120) is an operation in the leftward direction, the sign information of a left turn prohibition is displayed on the left display device (41); In the case where the specific sign information acquired by the sign acquisition unit (120) is right turn no sign information that prohibits a driving operation of turning the direction of travel of the vehicle (V) to the right, and the driver specific behavior information acquired by the driver specific behavior information acquisition unit (120) is an operation in the rightward direction, the right turn no sign information is displayed on the right side display device (42). The display control unit (130A) When the specific sign information acquired by the sign acquisition unit (100) is a sign information of a left turn prohibition that prohibits a driving operation of turning the traveling direction of the vehicle to the left, and when the line of sight direction acquired by the driver specific behavior information acquisition unit (120A) is a left front direction, the sign information of a left turn prohibition is displayed on the left display device (41); In the case where the specific sign information acquired by the sign acquisition unit (100) is right turn no sign information that prohibits a driving operation of turning the vehicle's direction of travel to the right, when the line of sight acquired by the driver specific behavior information acquisition unit (120A) is to the right front, the right turn no sign information is displayed on the right display device (42).

[0015] According to this aspect, the sign information display system (1) effectively displays prohibition sign information corresponding to the driving operation of the driver on the display devices (41, 42) in the direction in which the driver's gaze is likely to be directed, thereby making it possible to effectively prevent the driver from violating the traffic laws.

[0016] In the above description, in order to facilitate understanding of the invention, the symbols used in the embodiments are added in parentheses to the constituent elements of the invention corresponding to the embodiments, but each constituent element of the invention is not limited to the embodiment defined by the said symbols. [Brief description of the drawings]

[0017] [Figure 1] 1 is a schematic diagram showing a hardware configuration of a sign information display system according to an embodiment of the present invention. [Diagram 2]FIG. 2 is a schematic diagram illustrating the arrangement of monitors according to the embodiment. [Diagram 3] FIG. 2 is a schematic diagram showing a software configuration of the sign information display system according to the present embodiment. [Figure 4] FIG. 4 is a schematic diagram illustrating an example of a sign image according to the embodiment. [Diagram 5] FIG. 4 is a schematic diagram illustrating an example of a sign image according to the embodiment. [Figure 6] 11 is a diagram for explaining the correspondence between a driver's line of sight direction, sign information, and a sign image. FIG. [Figure 7] 5 is a flowchart illustrating a routine for recognition processing of sign information by an ECU. [Figure 8] 4 is a flowchart illustrating a routine of a line of sight direction estimation process performed by an ECU. [Figure 9] 5 is a flowchart illustrating a routine for display processing of the left monitor by the ECU. [Figure 10] 5 is a flowchart illustrating a routine for display processing of the right monitor by the ECU. [Figure 11] 6 is a flowchart illustrating a routine of a display process for the upper monitor by the ECU. [Figure 12] FIG. 13 is a schematic diagram showing a hardware configuration of a modified sign information display system. [Figure 13] FIG. 13 is a schematic diagram showing a software configuration of a modified sign information display system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, a sign information display system and a sign information display method for a vehicle according to the present embodiment will be described with reference to the drawings.

[0019] [Hardware configuration] 1 is a schematic diagram showing a hardware configuration of a sign information display system 1 according to this embodiment. The sign information display system 1 is mounted on a vehicle V and has an ECU 10. The ECU 10 includes a microcomputer as a main part. ECU is an abbreviation for Electronic Control Unit.

[0020] The ECU 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, and an interface device 14. The CPU 11 executes various programs stored in the ROM 12. The ROM 12 is a non-volatile memory that stores data and the like required for the CPU 11 to execute the various programs. The RAM 13 is a volatile memory that provides a working area into which the various programs are expanded when the CPU 11 executes them. The interface device 14 is a communication device for communicating with external devices.

[0021] In this embodiment, the ECU 10 executes a sign information display process that displays sign information indicated by road signs and road markings (hereinafter collectively referred to as traffic signs) present in the area ahead of the vehicle V to present to the driver. The sign information display process is performed by recognizing sign information of traffic signs present in the area ahead of the vehicle V and appropriately displaying a sign image representing the recognized sign information on a monitor in a direction in which the driver's line of sight is presumed to be directed. To this end, the ECU 10 is communicatively connected to a camera sensor 20, a turn signal switch 21, a shift sensor 22, a plurality of monitors 40 to 43, and the like.

[0022] The camera sensor 20 photographs the surrounding area of ​​the vehicle V. Specifically, the camera sensor 20 includes a front camera that photographs the front area of ​​the vehicle V, a rear camera that photographs the rear area of ​​the vehicle V, a left side camera that photographs the left side area of ​​the vehicle V, and a right side camera that photographs the right side area of ​​the vehicle V. In this embodiment, the "forward area" includes not only the front direction of the vehicle V but also the left front, right front, diagonally upward and downward forward. In addition, the "rear area" includes not only the rear direction of the vehicle V but also the left rear, right rear, diagonally upward and downward rear. The camera sensor 20 sequentially photographs the surrounding area of ​​the vehicle V and generates data of the photographed images (hereinafter, image data). In addition, the camera sensor 20 sequentially transmits the generated image data to the ECU 10.

[0023] The turn indicator switch 21 detects the direction of operation of a turn indicator lever (not shown) by the driver. When the driver operates the turn indicator lever clockwise, the turn indicator switch 21 transmits a signal indicating that the turn indicator lever is being operated clockwise (hereinafter referred to as a right operation signal) to the ECU 10. When the driver operates the turn indicator lever counterclockwise, the turn indicator switch 21 transmits a signal indicating that the turn indicator lever is being operated counterclockwise (hereinafter referred to as a left operation signal) to the ECU 10.

[0024] The shift sensor 22 detects the operating position of a shift lever for switching the shift range of a transmission (not shown). The shift sensor 22 detects the operating position of the shift lever, such as parking position P, reverse position R, neutral position N, drive position D, etc. The shift sensor 22 transmits the detected operating position of the shift lever to the ECU 10.

[0025] The multiple monitors 40-43 are disposed in positions within the vehicle cabin that are visible to the driver. The monitors 40-43 are display devices for displaying various screens including sign images, which will be described later. In this embodiment, the multiple monitors 40-43 are configured to include a total of four monitors: a central monitor 40, a left monitor 41, a right monitor 42, and an upper monitor 43. The detailed locations of the monitors 40-43 will be described below.

[0026] [Monitor Arrangement] FIG. 2 is a schematic diagram illustrating the arrangement of monitors 40 to 43 according to this embodiment.

[0027] As shown in FIG. 2, the central monitor 40 is a monitor that the driver can easily view when the driver's line of sight is directed toward the front of the vehicle V. For the central monitor 40, for example, a center display 40A provided at approximately the center in the width direction of the instrument panel 2 can be used. Also, for the central monitor 40, a multi-information display 40B arranged at a position in front of the driver's seat (on the back side of the steering wheel SW as seen from the driver) can be used. Alternatively, for the central monitor 40, a head-up display 40C that displays various information by projecting a virtual image on the windshield FG in front of the driver's seat can be used. Note that the central monitor 40 may be a combination of at least two of the center display 40A, the multi-information display 40B, and the head-up display 40C.

[0028] The left side monitor 41 is a monitor whose screen can be easily viewed by the driver when the driver's line of sight is directed to the left front of the vehicle V. Specifically, the left side monitor 41 is disposed to the left front of the driver's seat (as viewed from the driver seated in the driver's seat), and more specifically, disposed on the left side in the width direction of the instrument panel 2. For example, in the case where the vehicle V is equipped with an electronic side mirror, the left side monitor 41 can be a left side monitor for displaying an image captured by a left side camera (not shown) of the electronic side mirror.

[0029] The right side monitor 42 is a monitor whose screen can be easily viewed by the driver when the driver's line of sight is directed to the right front of the vehicle V. Specifically, the right side monitor 42 is disposed on the right front of the driver's seat (as viewed from the driver seated in the driver's seat), and more specifically, on the right side in the width direction of the instrument panel 2. For example, if the vehicle V is equipped with an electronic side mirror, the right side monitor 42 can be a right side monitor for displaying an image captured by a right side camera (not shown) of the electronic side mirror.

[0030] The upper monitor 43 is a monitor whose screen can be easily viewed by the driver when the driver directs his / her line of sight toward the inner mirror 43A (diagonally upward and forward) provided at the upper portion of the center of the width direction of the windshield FG. If the inner mirror 43A is an electronic inner mirror for displaying a rear image of the vehicle V, the upper monitor 43 can use the electronic inner mirror. In addition, if the vehicle V is not equipped with an electronic inner mirror, the upper monitor 43 can be configured by a monitor attached to the inner mirror 43A.

[0031] On each of the monitors 40 to 43, a sign image showing a traffic sign (road sign or road marking) existing in the area ahead of the vehicle V is displayed as an image for calling attention to prevent the driver's driving operation from becoming a traffic violation. A road sign is a signboard installed near a road that displays traffic regulations on the road. A road marking is a sign that displays traffic regulations on the road by lines or symbols drawn on the road surface. Examples of the road sign and road marking include traffic signs such as "no left turn, no right turn, no U-turn, no overtaking, no lane change, no crossing, no parking, no parking and no stopping". In this embodiment, the sign image is appropriately displayed on the monitor 40 to 43 in the direction in which the driver's line of sight is estimated to be directed. Hereinafter, a specific sign information display process will be described in detail.

[0032] [Software configuration] FIG. 3 is a schematic diagram showing the software configuration of the sign information display system 1 according to this embodiment.

[0033] As shown in FIG. 3, the ECU 10 includes a sign information recognition unit 100, a sign image storage unit 110, a gaze direction estimation unit 120, and a display control unit 130 as functional elements. The sign information recognition unit 100, the gaze direction estimation unit 120, and the display control unit 130 are realized by the CPU 11 of the ECU 10 reading a program stored in the ROM 12 into the RAM 13 and executing the program. The sign image storage unit 110 is realized as a part of a storage area provided by the ROM 12 of the ECU 10. These functional elements are described as being included in the ECU 10, which is an integrated piece of hardware, but some of these elements may be provided in another ECU separate from the ECU 10. In addition, all or some of the functional elements of the ECU 10 may be provided in an information processing device in a facility (e.g., a management center) capable of communicating with the vehicle V.

[0034] The sign information recognition unit 100 performs image recognition processing on image data sequentially transmitted from the camera sensor 20 to determine whether or not a traffic sign preset as a detection target exists in the area ahead of the vehicle V. In other words, the sign information recognition unit 100 detects traffic signs preset as detection targets among traffic signs present in the area ahead of the vehicle V based on the image recognition processing. A known method such as pattern matching can be applied to the image recognition processing. Specifically, the sign information recognition unit 100 performs known image processing such as edge detection on the image data acquired from the camera sensor 20 to extract the contours of all objects included in the image. Then, the sign information recognition unit 100 performs pattern matching processing on the image data that has been subjected to the image processing to determine whether or not a traffic sign set as a detection target exists in the image.

[0035] In this embodiment, the detection targets of the sign information recognition unit 100 include specific types of road signs and road markings that are likely to be illegal when the driver tries to change the traveling direction of the vehicle V to the left or right (turning right or left, rotating, changing lanes, etc.) or back up the vehicle V to stop it. Specifically, the road signs that are set to be detected by the sign information recognition unit 100 include "no left turn", "no right turn", "no turning", "no overtaking", "no crossing", "no parking", and "no parking and stopping". In addition, the road markings that are set to be detected by the sign information recognition unit 100 include "no turning", "no overtaking", "no lane changing", "no parking", and "no parking and stopping".

[0036] When the sign information recognition unit 100 determines that a road sign prohibiting left turns exists in an image acquired by the camera sensor 20 (for example, the front camera), that is, when a road sign prohibiting left turns is detected, the sign information recognition unit 100 recognizes the left turn prohibition sign as sign information of the road on which the vehicle is traveling. When the sign information recognition unit 100 determines that a road sign prohibiting right turns exists in an image acquired by the camera sensor 20 (for example, the front camera), that is, when a road sign prohibiting right turns is detected, the sign information recognition unit 100 recognizes the right turn prohibition sign as sign information of the road on which the vehicle is traveling.

[0037] When the sign information recognition unit 100 determines that a no turn road sign is present in an image acquired by the camera sensor 20 (for example, the front camera), that is, when a no turn road sign is detected, the sign information recognition unit 100 recognizes the no turn sign as sign information of the road on which the vehicle is traveling. When the sign information recognition unit 100 determines that a no turn road marking is present in an image acquired by the camera sensor 20 (for example, the front camera), that is, when a no turn road marking is detected, the sign information recognition unit 100 recognizes the no turn sign as sign information of the road on which the vehicle is traveling.

[0038] When the sign information recognition unit 100 determines that a no-overtaking road sign is present in an image acquired by the camera sensor 20 (for example, the front camera), that is, when the sign information recognition unit 100 detects a no-overtaking road sign, the sign information recognition unit 100 recognizes the no-overtaking sign as sign information of the road on which the vehicle is traveling. When the sign information recognition unit 100 determines that a no-overtaking road marking is present in an image acquired by the camera sensor 20 (for example, the front camera), that is, when the sign information recognition unit 100 detects a no-overtaking road marking, the sign information recognition unit 100 recognizes the no-overtaking sign as sign information of the road on which the vehicle is traveling.

[0039] When the sign information recognition unit 100 determines that a road marking prohibiting lane change is present in an image acquired by the camera sensor 20 (for example, the front camera), that is, when the sign information recognition unit 100 detects a road marking prohibiting lane change, the sign information recognition unit 100 recognizes the lane change prohibition marking as sign information of the road on which the vehicle is traveling. When the sign information recognition unit 100 determines that a road sign prohibiting crossing is present in an image acquired by the camera sensor 20 (for example, the front camera), that is, when the sign information recognition unit 100 detects a road sign prohibiting crossing, the sign information recognition unit 100 recognizes the crossing prohibition sign as sign information of the road on which the vehicle is traveling.

[0040] When the sign information recognition unit 100 determines that a no parking road sign is present in an image acquired by the camera sensor 20 (for example, the rear camera), that is, when a no parking road sign is detected, the sign information recognition unit 100 recognizes the no parking sign as sign information of the road on which the vehicle is traveling. When the sign information recognition unit 100 determines that a no parking road marking is present in an image acquired by the camera sensor 20 (for example, the rear camera), that is, when a no parking road marking is detected, the sign information recognition unit 100 recognizes the no parking sign as sign information of the road on which the vehicle is traveling.

[0041] When the sign information recognition unit 100 determines that a road sign prohibiting parking and stopping is present in an image acquired by the camera sensor 20 (for example, the rear camera), that is, when a road sign prohibiting parking and stopping is detected, the sign information recognition unit 100 recognizes the no parking and stopping sign as sign information of the road on which the vehicle is traveling. When the sign information recognition unit 100 determines that a road marking prohibiting parking and stopping is present in an image acquired by the camera sensor 20 (for example, the rear camera), that is, when a road marking prohibiting parking and stopping is detected, the sign information recognition unit 100 recognizes the no parking and stopping sign as sign information of the road on which the vehicle is traveling.

[0042] The sign information recognition unit 100 sequentially transmits the recognized sign information to the display control unit 130.

[0043] The sign image storage unit 110 stores a plurality of sign images (see Figs. 4 and 5) to be displayed on each of the monitors 40 to 43. Here, a sign image is an image (icon) that typically represents a road sign or road marking to be detected by the sign information recognition unit 100 using figures, characters, etc. Details of the sign image of this embodiment will be described below with reference to Figs. 4 and 5. Note that, although the sign image is configured by combining figures and characters in the examples shown in Figs. 4 and 5, it may be configured by only either figures or characters.

[0044] 4 and 5 are schematic diagrams showing specific examples of sign images stored in the sign image storage unit 110. FIG. 4(A) is a sign image showing a road sign prohibiting left turns. Therefore, hereinafter, the sign image shown in FIG. 4(A) is referred to as a "no left turn sign image G1." FIG. 4(B) is a sign image showing a road sign prohibiting right turns. Therefore, hereinafter, the sign information image shown in FIG. 4(B) is referred to as a "no right turn sign image G2." FIG. 4(C) is a sign image showing a road sign prohibiting turns. Therefore, hereinafter, the sign image shown in FIG. 4(C) is referred to as a "no turn sign image G3."

[0045] Fig. 4(D) is a sign image showing a road sign prohibiting overtaking. Therefore, hereinafter, the sign image shown in Fig. 4(D) is referred to as a "no overtaking sign image G4". Fig. 4(E) is a sign image showing a road sign prohibiting crossing. Therefore, hereinafter, the sign image shown in Fig. 4(E) is referred to as a "no crossing sign image G5".

[0046] Fig. 4(F) is a sign image showing a road sign prohibiting parking. Therefore, hereinafter, the sign image shown in Fig. 4(F) is referred to as a "no parking sign image G6". Fig. 4(G) is a sign image showing a road sign prohibiting parking and stopping. Therefore, hereinafter, the sign image shown in Fig. 4(G) is referred to as a "no parking and stopping sign image G7".

[0047] FIG. 5(A) is a sign image showing a road marking forbidding overtaking (a center line drawn in a solid yellow or white line). Therefore, hereinafter, the sign image shown in FIG. 5(A) is referred to as a "no overtaking sign image G8." FIG. 5(B) is a sign image showing a road marking forbidding lane changing (a boundary line drawn in a solid yellow line). Therefore, hereinafter, the sign image shown in FIG. 5(B) is referred to as a "no lane changing sign image G9." FIG. 5(C) is a sign image showing a road marking forbidding turns. Therefore, hereinafter, the sign image shown in FIG. 5(C) is referred to as a "no turn sign image G10."

[0048] FIG. 5(D) is a sign image showing a road marking forbidding parking (a shoulder line drawn in yellow dashed lines). Therefore, hereinafter, the sign image shown in FIG. 5(D) is referred to as a "no parking sign image G11." FIG. 5(E) is a sign image showing a road marking forbidding parking and stopping (a shoulder line drawn in yellow solid lines). Therefore, hereinafter, the sign image shown in FIG. 5(E) is referred to as a "no parking and stopping sign image G12." Note that, hereinafter, when there is no need to particularly distinguish between these images G1 to G12, they may be collectively referred to as sign images G1 to G12.

[0049] In this embodiment, the sign image storage unit 110 stores sign information and sign images G1 to G12 in association with the driver's line of sight. Specifically, as shown in FIG. 6, the left turn prohibition sign image G1 and the lane change prohibition sign image G9 are associated with the driver's line of sight toward the "left front" of the vehicle V. The right turn prohibition sign image G2, the turn prohibition sign image G3, the turn prohibition sign image G10, the overtaking prohibition sign image G4, the overtaking prohibition sign image G8, the crossing prohibition sign image G5, and the lane change prohibition sign image G9 are associated with the driver's line of sight toward the "right side" of the vehicle V. The parking prohibition sign image G6, the parking prohibition sign image G11, the parking and stopping prohibition sign image G7, and the parking and stopping prohibition sign image G12 are associated with the driver's line of sight toward the "upward diagonal front" of the vehicle V.

[0050] Note that the correspondence shown in FIG. 6 indicates a case where traffic drives on the left side of the road, but in a case where traffic drives on the right side of the road, the no turn sign image G3, the no turn marking image G10, the no overtaking sign image G4, the no overtaking marking image G8, and the no crossing sign image G5 can be associated with each other when the driver's line of sight is directed to the "left side."

[0051] 3 , the gaze direction estimation unit 120 is an example of a driver specific behavior information acquisition unit of the present disclosure, and estimates the gaze direction of the driver based on the driver specific behavior information including driving operation information of the driver of the vehicle V. In this embodiment, the gaze direction estimation unit 120 estimates the gaze direction of the driver based on input signals from the turn signal switch 21 and the shift sensor 22.

[0052] When the direction indicator switch 21 detects a left operation signal while the shift sensor 22 detects the drive position D, it is presumed that the driver will turn left at the intersection or change lanes to the adjacent left lane. In this case, the gaze direction estimation unit 120 presumes that the driver's gaze direction is directed to the left front. When the direction indicator switch 21 detects a right operation signal while the shift sensor 22 detects the drive position D, it is presumed that the driver will turn the vehicle V right at the intersection, turn around, overtake a preceding vehicle, cross an oncoming lane, or change lanes to the adjacent right lane. In this case, the gaze direction estimation unit 120 presumes that the driver's gaze direction is directed to the right front.

[0053] When the shift sensor 22 detects the reverse position R, it is presumed that the driver is attempting to park or stop the vehicle V while looking at the inner mirror 43A. In this case, the line of sight direction estimating unit 120 estimates that the line of sight of the driver is directed diagonally upward and forward (toward the inner mirror 43A).

[0054] The gaze direction estimation unit 120 successively transmits the estimated gaze direction of the driver to the display control unit 130.

[0055] When the sign information recognition unit 100 recognizes sign information of road signs and road markings that are set as detection targets in advance, and the gaze direction estimation unit 120 estimates the gaze direction based on the driver's driving operation to be either the left front, the right front, or the obliquely upward front, the display control unit 130 searches (extracts) an image associated with the driver's gaze direction estimated by the gaze direction estimation unit 120 from the multiple sign images G1 to G12 stored in the sign image storage unit 110. In addition, the display control unit 130 selects the monitors 41 to 43 corresponding to the driver's gaze direction, and executes display control to display the extracted sign images G1 to G12 on the selected monitors 41 to 43.

[0056] Specifically, when the sign information recognition unit 100 recognizes a no left turn sign as sign information and the line of sight direction estimation unit 120 estimates that the driver's line of sight is to the left front (i.e., when the driver operates the turn signal operation lever in a counterclockwise direction), the display control unit 130 displays a no left turn sign image G1 on the left monitor 41. This allows the driver to easily understand that a left turn is prohibited on the road on which the driver is traveling, by looking at the left monitor 41 which is in the line of sight. After displaying the no left turn sign image G1 on the left monitor 41, when the driver returns the turn signal operation lever to the neutral position, the display control unit 130 ends the display on the left monitor 41 and returns to a normal display mode in which various displays are performed on the central monitor 40.

[0057] When the sign information recognition unit 100 recognizes a lane change prohibition sign as sign information and the line of sight direction estimation unit 120 estimates that the driver's line of sight is to the left front (i.e., when the driver operates the turn signal operation lever in a counterclockwise direction), the display control unit 130 displays a lane change prohibition sign image G9 on the left monitor 41. This allows the driver to easily understand that lane changes are prohibited on the road on which the driver is traveling, by looking at the left monitor 41 in the line of sight. After displaying the lane change prohibition sign image G9 on the left monitor 41, when the driver returns the turn signal operation lever to the neutral position, the display control unit 130 ends the display on the left monitor 41 and returns to a normal display mode in which various displays are performed on the center monitor 40.

[0058] When the sign information recognition unit 100 recognizes a lane change prohibition sign as sign information and the line of sight direction estimation unit 120 estimates that the driver's line of sight is to the right front (i.e., when the driver operates the turn signal operation lever in a clockwise direction), the display control unit 130 displays a lane change prohibition sign image G9 on the right monitor 42. This allows the driver to easily understand that lane changes are prohibited on the road on which the driver is traveling, by looking at the right monitor 42 in the line of sight. After displaying the lane change prohibition sign image G9 on the right monitor 42, when the driver returns the turn signal operation lever to the neutral position, the display control unit 130 ends the display on the right monitor 42 and returns to a normal display mode in which various displays are performed on the center monitor 40.

[0059] When the sign information recognition unit 100 recognizes a right turn prohibition sign as sign information and the line of sight direction estimation unit 120 estimates that the driver's line of sight is to the right front (i.e., when the driver operates the turn signal operation lever in a clockwise direction), the display control unit 130 displays a right turn prohibition sign image G2 on the right monitor 42. This allows the driver to easily understand that a right turn is prohibited on the road on which the driver is traveling, from the right monitor 42 in the line of sight. After displaying the right turn prohibition sign image G2 on the right monitor 42, when the driver returns the turn signal operation lever to the neutral position, the display control unit 130 ends the display on the right monitor 42 and returns to a normal display mode in which various displays are performed on the central monitor 40.

[0060] When the sign information recognition unit 100 recognizes a no turn sign and / or a no turn sign as sign information and the line of sight direction estimation unit 120 estimates that the driver's line of sight is to the right front (i.e., when the driver operates the turn signal operation lever in a clockwise direction), the display control unit 130 displays a no turn sign image G3 and / or a no turn sign image G10 on the right monitor 42. This allows the driver to easily understand that the road on which the driver is traveling is no turnable, based on the right monitor 42 in the line of sight. After displaying the no turn sign image G3 and / or the no turn sign image G10 on the right monitor 42, when the driver returns the turn signal operation lever to the neutral position, the display control unit 130 ends the display on the right monitor 42 and returns to a normal display mode in which various displays are performed on the center monitor 40.

[0061] When the sign information recognition unit 100 recognizes a no-overtaking sign and / or a no-overtaking sign as sign information and the line-of-sight direction estimation unit 120 estimates that the driver's line-of-sight direction is to the right front (i.e., when the driver operates the turn signal operation lever in a clockwise direction), the display control unit 130 displays a no-overtaking sign image G4 and / or a no-overtaking sign image G8 on the right monitor 42. This allows the driver to easily understand that the road on which the driver is traveling is no-overtaking by looking at the right monitor 42 in the line-of-sight direction. After displaying the no-overtaking sign image G4 and / or the no-overtaking sign image G8 on the right monitor 42, when the driver returns the turn signal operation lever to the neutral position, the display control unit 130 ends the display on the right monitor 42 and returns to a normal display mode in which various displays are performed on the central monitor 40.

[0062] When the sign information recognition unit 100 recognizes a no-crossing sign as sign information and the line-of-sight direction estimation unit 120 estimates that the driver's line-of-sight direction is to the right front (i.e., when the driver operates the turn signal operation lever in a clockwise direction), the display control unit 130 displays a no-crossing sign image G5 on the right monitor 42. This allows the driver to easily understand that crossing is prohibited on the road on which the driver is traveling, from the right monitor 42 in the line-of-sight direction. After displaying the no-crossing sign image G5 on the right monitor 42, when the driver returns the turn signal operation lever to the neutral position, the display control unit 130 ends the display on the right monitor 42 and returns to a normal display mode in which various displays are performed on the central monitor 40.

[0063] When the sign information recognition unit 100 recognizes a no parking sign and / or a no parking sign as sign information and the line of sight direction estimation unit 120 estimates the driver's line of sight to be diagonally upward and forward (i.e., when the driver operates the shift lever to the reverse position R), the display control unit 130 displays the no parking sign image G6 and / or the no parking sign image G11 on the upper monitor 43. This allows the driver to easily understand that parking is prohibited on the road on which the driver is traveling, by looking at the upper monitor 43 located in the line of sight. After displaying the no parking sign image G6 and / or the no parking sign image G11 on the upper monitor 43, when the driver switches the shift lever from the reverse position R to another shift position, the display control unit 130 ends the display on the upper monitor 43 and returns to a normal display mode in which various displays are performed on the central monitor 40.

[0064] When the sign information recognition unit 100 recognizes a parking prohibition sign and / or a parking prohibition sign as sign information and the line of sight direction estimation unit 120 estimates the driver's line of sight to be diagonally upward in front (i.e., when the driver operates the shift lever to the reverse position R), the display control unit 130 displays the parking prohibition sign image G7 and / or the parking prohibition sign image G12 on the upper monitor 43. This allows the driver to easily understand that parking and stopping is prohibited on the road on which the driver is traveling, by the upper monitor 43 in the line of sight. After displaying the parking prohibition sign image G7 and / or the parking prohibition sign image G12 on the upper monitor 43, when the driver switches the shift lever from the reverse position R to another shift position, the display control unit 130 ends the display on the upper monitor 43 and returns to a normal display mode in which various displays are performed on the central monitor 40.

[0065] Next, a routine for recognition processing of sign information by the ECU 10 will be described with reference to the flowchart shown in Fig. 7. This routine is started by turning on the accessory of the vehicle V. At the start of this routine, i.e., when the accessory is turned on, the display mode is set to the normal display mode in which various information is displayed on the central monitor 40.

[0066] In step S100, the ECU 10 acquires image data of the area ahead of the vehicle V captured by the camera sensor 20. Next, in step S110, the ECU 10 recognizes sign information of traffic signs (road signs, road markings) present in the area ahead of the vehicle V by performing image recognition processing on the image data acquired in step S100.

[0067] Next, in step S120, the ECU 10 determines whether or not the sign information recognized in step S110 is the sign information that has been preset as a detection target. If the determination result is positive (Yes), the ECU 10 proceeds to the line of sight direction estimation process shown in Fig. 8. On the other hand, if the determination result is negative (No), the ECU 10 ends (returns) this routine once. That is, the normal display mode is maintained.

[0068] Next, a gaze direction estimation process routine by the ECU 10 will be described with reference to the flow shown in Fig. 8. This routine is started when the determination result of step S120 in the flow shown in Fig. 7 is affirmative (Yes).

[0069] In step S200, it is determined whether or not the shift sensor 22 detects the drive position D. If the shift sensor 22 detects the drive position D (Yes), the ECU 10 proceeds to the process of step S210. On the other hand, if the shift sensor 22 does not detect the drive position D (No), the ECU 10 proceeds to the process of step S260.

[0070] In step S210, the ECU 10 judges whether or not the turn indicator switch 21 has detected a left operation signal. If the turn indicator switch 21 has detected a left operation signal (Yes), the ECU 10 proceeds to processing in step S220, estimates that the driver's line of sight is to the left front, and proceeds to display processing on the left monitor 41 shown in Fig. 9. On the other hand, if the turn indicator switch 21 has not detected a left operation signal (No), the ECU 10 proceeds to processing in step S240.

[0071] In step S240, the ECU 10 judges whether the direction indicator switch 21 has detected a right operation signal. If the direction indicator switch 21 has detected a right operation signal (Yes), the ECU 10 proceeds to processing in step S250, estimates that the driver's line of sight is to the right front, and proceeds to display processing of the right monitor 42 shown in Fig. 10. On the other hand, if the direction indicator switch 21 has not detected a right operation signal (No), the ECU 10 once ends (returns) this routine without estimating the driver's line of sight. That is, the normal display mode is maintained.

[0072] When the process proceeds from step S200 to step S260, the ECU 10 determines whether or not the shift sensor 22 detects the reverse position R. If the shift sensor 22 detects the reverse position R (Yes), the ECU 10 proceeds to step S270, where it estimates that the driver's line of sight is diagonally upward and forward, and proceeds to display processing of the upper monitor 43 shown in Fig. 11. On the other hand, if the shift sensor 22 does not detect the reverse position R (No), the ECU 10 once ends (returns) this routine without estimating the driver's line of sight. That is, the normal display mode is maintained.

[0073] Next, a display process routine for the left monitor 41 by the ECU 10 will be described with reference to the flow shown in Fig. 9. This routine is started when the ECU 10 estimates that the driver's line of sight is to the left front in the line of sight direction estimation process shown in Fig. 8.

[0074] In step S300, the ECU 10 judges whether or not the sign information recognized as the detection target in the sign information recognition process shown in Fig. 7 is included in the sign information (see Fig. 6) associated with the left front which is the line of sight of the driver. If the judgment result is negative (No), the ECU 10 temporarily ends (returns) this routine. In other words, if the sign information recognized as the detection target is not associated with the left front which is the line of sight of the driver, the normal display mode is maintained. This makes it possible to effectively prevent unnecessary sign information from being presented to the driver who is directing his / her line of sight to the left front of the vehicle V.

[0075] If the determination result of step S300 is positive (Yes), the ECU 10 proceeds to the process of step S310 and selects the sign image of the recognized sign information as the display image. Specifically, if a no left turn sign is recognized as the sign information, the ECU 10 selects the no left turn sign image G1. Also, if a no lane change sign is recognized as the sign information, the ECU 10 selects the no lane change sign image G9.

[0076] In step S320, the ECU 10 displays the sign image selected in step S310 on the left monitor 41. If multiple sign images are selected, the ECU 10 displays the sign images on the left monitor 41 side by side, either vertically or horizontally.

[0077] In step S330, the ECU 10 determines whether or not the driver has returned the turn signal lever to the neutral position. If the determination result is negative (No), the ECU 10 returns to the process of step S320 and continues to display the sign image. On the other hand, if the determination result is positive (Yes), the ECU 10 proceeds to the process of step S340, switches the display mode to the normal display mode, and temporarily ends (returns) this routine. Note that the display of the sign image on the left monitor 41 may end when a predetermined upper limit time has elapsed.

[0078] Next, a display process routine for the right monitor 42 by the ECU 10 will be described with reference to the flow shown in Fig. 10. This routine is started when the ECU 10 estimates that the driver's line of sight is to the right front in the line of sight direction estimation process shown in Fig. 8.

[0079] In step S400, the ECU 10 judges whether or not the sign information recognized as the detection target in the sign information recognition process shown in Fig. 7 is included in the sign information (see Fig. 6) associated with the right front which is the line of sight of the driver. If the judgment result is negative (No), the ECU 10 temporarily ends (returns) this routine. In other words, if the sign information recognized as the detection target is not associated with the right front which is the line of sight of the driver, the normal display mode is maintained. This makes it possible to effectively prevent unnecessary sign information from being presented to the driver who is directing his / her line of sight to the right front of the vehicle V.

[0080] If the determination result of step S400 is positive (Yes), the ECU 10 proceeds to the process of step S410 and selects the sign image of the recognized sign information as the display image. Specifically, if a right turn prohibition sign is recognized as the sign information, the ECU 10 selects the right turn prohibition sign image G2. If a turn prohibition sign is recognized as the sign information, the ECU 10 selects the turn prohibition sign image G3. If a turn prohibition sign is recognized as the sign information, the ECU 10 selects the turn prohibition sign image G10. If an overtaking prohibition sign is recognized as the sign information, the ECU 10 selects the overtaking prohibition sign image G4. If a overtaking prohibition sign is recognized as the sign information, the ECU 10 selects the overtaking prohibition sign image G8. If a crossing prohibition sign is recognized as the sign information, the ECU 10 selects the crossing prohibition sign image G5. Moreover, when a lane change prohibition sign is recognized as the sign information, the ECU 10 selects the lane change prohibition sign image G9.

[0081] In step S420, the ECU 10 displays the sign image selected in step S410 on the right monitor 42. If multiple sign images are selected, the ECU 10 displays the sign images on the right monitor 42 side by side, one above the other and / or one to the left and right.

[0082] In step S430, the ECU 10 determines whether or not the driver has returned the turn signal lever to the neutral position. If the determination result is negative (No), the ECU 10 returns to the process of step S420 and continues to display the sign image. On the other hand, if the determination result is positive (Yes), the ECU 10 proceeds to the process of step S440, switches the display mode to the normal display mode, and temporarily ends (returns) this routine. Note that the display of the sign image on the right monitor 42 may end when a predetermined upper limit time has elapsed.

[0083] Next, a routine for display processing on the upper monitor 43 by the ECU 10 will be described with reference to the flow shown in Fig. 11. This routine is started when the ECU 10 estimates that the driver's line of sight is diagonally upward and forward in the line of sight direction estimation processing shown in Fig. 8.

[0084] In step S500, the ECU 10 judges whether or not the sign information recognized as the detection target in the sign information recognition process shown in FIG. 7 is included in the sign information (see FIG. 6) associated with the driver's line of sight diagonally upward toward the front. If the judgment result is negative (No), the ECU 10 temporarily ends (returns) this routine. In other words, if the sign information recognized as the detection target is not associated with the driver's line of sight diagonally upward toward the front, the normal display mode is maintained. This makes it possible to effectively prevent unnecessary sign information from being presented to the driver who is directing his / her line of sight diagonally upward toward the front of the vehicle V (to the inner mirror 43).

[0085] If the determination result of step S500 is positive (Yes), the ECU 10 proceeds to the process of step S510 and selects the sign image of the recognized sign information as the display image. Specifically, if a no parking sign is recognized as the sign information, the ECU 10 selects the no parking sign image G6. If a no parking sign is recognized as the sign information, the ECU 10 selects the no parking sign image G11. If a no parking sign is recognized as the sign information, the ECU 10 selects the no parking sign image G7. If a no parking sign is recognized as the sign information, the ECU 10 selects the no parking sign image G12.

[0086] In step S520, the ECU 10 displays the sign image selected in step S510 on the upper monitor 43. If a plurality of sign images are selected, the ECU 10 displays the sign images side-by-side on the upper monitor 43.

[0087] In step S530, the ECU 10 determines whether or not the driver has operated the shift lever from the reverse position R to another shift position. If the determination result is negative (No), the ECU 10 returns to the process of step S520 and continues to display the sign image. On the other hand, if the determination result is positive (Yes), the ECU 10 proceeds to the process of step S540, switches the display mode to the normal display mode, and temporarily ends (returns) this routine. Note that the display of the sign image on the upper monitor 43 may end when a predetermined upper limit time has elapsed.

[0088] According to the present embodiment described above in detail, the sign information recognition unit 100 recognizes specific sign information that is preset as a detection target based on image data acquired by the camera sensor 20. The gaze direction estimation unit 120 estimates the gaze direction of the driver based on the operation direction of the turn signal lever by the driver and the operation position of the shift lever. When the sign information recognition unit 100 recognizes specific sign information and the gaze direction estimation unit 120 estimates the gaze direction based on the driving operation of the driver to be either the left front, the right front, or the diagonally upward front, the display control unit 130 selects an image associated with the gaze direction of the driver from the multiple sign images G1 to G12 stored in the sign image storage unit 110. Then, the display control unit 130 displays the selected sign image on the monitors 41 to 43 corresponding to the gaze direction of the driver.

[0089] This allows specific sign images to be appropriately displayed on the monitors 41, 42, and 43 in the line of sight of the driver, making it possible to effectively call the driver's attention. Also, since the sign images displayed on the monitors 41, 42, and 43 are limited to specific sign images associated with the line of sight of the driver, it is possible to effectively prevent unnecessary sign information from being presented to the driver, and it is also possible to effectively prevent the driver from feeling annoyed or uncomfortable.

[0090] The above describes the sign information display system and display method according to the present embodiment, but the present disclosure is not limited to the above embodiment, and various modifications are possible without departing from the purpose of the present invention.

[0091] [Variations] As shown in Fig. 12, the sign information display system 1 of the modified example further includes a driver monitor camera 50. The driver monitor camera 50 generates face image data by sequentially capturing an area including the face of the driver sitting in the driver's seat of the vehicle V, and sequentially transmits the generated face image data to the ECU 10. The driver monitor camera 50 is disposed in the vehicle cabin so as to capture an image of the driver's face. Note that other configurations are similar to those of the above embodiment, and therefore description thereof will be omitted.

[0092] 13, the ECU 10 of the modified example includes a sign information recognition unit 100, a sign image storage unit 110, a gaze direction detection unit 120A, and a display control unit 130A. The sign information recognition unit 100 and the sign image storage unit 110 are configured similarly to the above embodiment, and therefore description thereof will be omitted.

[0093] The gaze direction detection unit 120A is an example of a driver specific behavior information acquisition unit of the present disclosure, and sequentially acquires face image data generated by the driver monitor camera 50, detects feature points from the acquired face image data using a known image processing technique, and detects the face area, the eye area, the iris, etc. in the face area. The driver monitor camera 50 is fixed at a predetermined position in the cabin of the vehicle V, and the imaging direction is also fixed, so that the position of the driver's face in the vehicle V can be specified according to the position and size of the face area in the image data. The gaze direction detection unit 120A detects the driver's gaze direction by sequentially calculating the driver's gaze direction from the size of the face area and the position of the eye area and the position of the iris in the face area. The gaze direction detection unit 120A sequentially transmits the detected driver's gaze direction to the display control unit 130A.

[0094] When the sign information recognition unit 100 recognizes sign information that is set as a detection target in advance and the gaze direction detection unit 120A detects any one of the left front, right front, and front diagonally upward as the gaze direction of the driver, the display control unit 130A selects an image associated with the gaze direction of the driver detected by the gaze direction detection unit 120A from the multiple sign images G1 to G12 stored in the sign image storage unit 110. The display control unit 130A also executes display control to display the selected sign images G1 to G12 on the monitors 41 to 43 corresponding to the gaze direction of the driver. After the display control unit 130A displays the selected sign images G1 to G12 on the monitors 41 to 43, the display control unit 130A switches the display mode to the normal display mode when the gaze direction detection unit 120A detects a change in the gaze direction of the driver.

[0095] According to the modified example configured as above, the sign information recognition unit 100 recognizes specific sign information that is preset as a detection target based on image data acquired by the camera sensor 20. The gaze direction detection unit 120A directly detects the gaze direction of the driver based on the face image data of the driver. When the sign information recognition unit 100 recognizes sign information and the gaze direction detection unit 120A detects the driver's gaze direction as either the left front, the right front, or the obliquely upward front, the display control unit 130A selects an image associated with the driver's gaze direction from the multiple sign images G1 to G12 stored in the sign image storage unit 110. Then, the display control unit 130 displays the selected sign image on the monitors 41 to 43 corresponding to the driver's gaze direction.

[0096] This makes it possible to directly detect the driver's line of sight based on the facial image data, and to appropriately display specific sign images on the monitors 41, 42, and 43 in the driver's line of sight, thereby making it possible to reliably call the driver's attention. In addition, since the sign images displayed on the monitors 41, 42, and 43 are limited to the specific sign images associated with the driver's line of sight, it is possible to effectively prevent unnecessary sign information from being presented to the driver, and it is also possible to effectively prevent the driver from feeling annoyed or uncomfortable.

[0097] [others] In the above embodiment, the gaze direction estimation unit 120 has been described as estimating the gaze direction of the driver, but it may be configured to simply acquire the operation direction (left or right) of the turn indicator lever by the driver and the operation position of the shift lever (reverse position R) as the driver specific behavior information without estimating the gaze direction. In this case, the display control unit 130 may select the left monitor 41 when the turn indicator lever is operated in the left direction, select the right monitor 42 when the turn indicator lever is operated in the right direction, and select the upper monitor 43 when the shift lever is operated to the reverse position R. [Explanation of symbols]

[0098] 1...signal information display system, 10...ECU, 20...camera sensor, 21...turn signal switch, 22...shift sensor, 40...center monitor, 41...left monitor, 42...right monitor, 43...upper monitor, 100...signal information recognition unit, 110...signal image storage unit, 120...gaze direction estimation unit, 130...display control unit, V...vehicle

Claims

1. an image capturing device for capturing an image of a surrounding area including at least a front area of ​​a vehicle; a sign acquisition unit that acquires sign information of traffic signs present in the surrounding area based on the surrounding image; a driver specific behavior information acquisition unit that acquires driver specific behavior information including driving operation information of a driver who drives the vehicle; a plurality of display devices arranged at at least two or more different positions within a cabin of the vehicle that are visible to the driver and capable of displaying the sign information acquired by the sign acquisition unit; A display control unit that controls screen display of the plurality of display devices, The display control unit is Selecting on which of the plurality of display devices to display the sign information acquired by the sign acquisition unit based on a type of the sign information acquired by the sign acquisition unit and the driver specific behavior information acquired by the driver specific behavior information acquisition unit. Sign information display system.

2. The sign information display system according to claim 1, The plurality of display devices include a left display device disposed on the left front side of a driver's seat in the vehicle interior, and a right display device disposed on the right front side of the driver's seat, The driver specific behavior information acquisition unit As the driver specific behavior information, a counterclockwise operation and a clockwise operation of a turn signal lever of the vehicle by the driver are acquired, The display control unit is When the type of the sign information acquired by the sign acquisition unit is specific sign information, if the driver specific behavior information acquired by the driver specific behavior information acquisition unit is an operation in the counterclockwise direction, the specific sign information is displayed on the left display device, and if the driver specific behavior information acquired by the driver specific behavior information acquisition unit is an operation in the clockwise direction, the specific sign information is displayed on the right display device. Sign information display system.

3. The sign information display system according to claim 1, The plurality of display devices include a left display device disposed on the left front side of a driver's seat in the vehicle interior, and a right display device disposed on the right front side of the driver's seat, The driver specific behavior information acquisition unit a camera for photographing a face of the driver seated in the driver's seat, and acquiring, as the driver specific behavior information, a line of sight direction of the driver based on a face image photographed by the camera; The display control unit is When the type of the sign information acquired by the sign acquisition unit is specific sign information, if the line of sight direction acquired by the driver specific behavior information acquisition unit is a left front direction, the specific sign information is displayed on the left display device, and when the line of sight direction acquired by the driver specific behavior information acquisition unit is a right front direction, the specific sign information is displayed on the right display device. Sign information display system.

4. The sign information display system according to claim 2, The display control unit is When the specific sign information acquired by the sign acquisition unit is a left turn prohibited sign information that prohibits a driving operation of turning the traveling direction of the vehicle to the left, and when the driver specific behavior information acquired by the driver specific behavior information acquisition unit is an operation in the leftward direction, the left turn prohibited sign information is displayed on the left display device, When the specific sign information acquired by the sign acquisition unit is right turn prohibited sign information that prohibits a driving operation of turning the traveling direction of the vehicle to the right, and when the driver specific behavior information acquired by the driver specific behavior information acquisition unit is an operation in the rightward direction, the right turn prohibited sign information is displayed on the right display device. Sign information display system.

5. The sign information display system according to claim 3, The display control unit is When the specific sign information acquired by the sign acquisition unit is a sign information of a left turn prohibition that prohibits a driving operation of turning the traveling direction of the vehicle to the left, and when the line of sight direction acquired by the driver specific behavior information acquisition unit is a left front direction, the sign information of the left turn prohibition is displayed on the left display device, When the specific sign information acquired by the sign acquisition unit is right turn prohibited sign information that prohibits a driving operation of turning the traveling direction of the vehicle to the right, and when the line of sight direction acquired by the driver specific behavior information acquisition unit is a right front direction, the right turn prohibited sign information is displayed on the right display device. Sign information display system.

6. Acquire a surrounding image by photographing a surrounding area including at least a front area of ​​the vehicle; acquiring sign information of traffic signs present in the surrounding area based on the surrounding image; Acquire driver specific behavior information including driving operation information of a driver who drives the vehicle; A sign information display method for a vehicle that performs display control to display the acquired sign information on a plurality of display devices arranged at at least two or more different positions within a passenger compartment of the vehicle that are visible to the driver, comprising: In the display control, Selecting which of the plurality of display devices to display the acquired sign information on the basis of a type of the acquired sign information and the acquired driver specific behavior information. How to display sign information.

Images