Recognition support system

Abstract

To provide a recognition support system capable of providing a situation useful for a driving operation of a vehicle to a driver, when performing the driving operation in accordance with road signs in addition to enhancing visibility of the road signs.SOLUTION: The recognition support system comprises a first recognition unit 31 capable of recognizing a road sign based on an image ahead of a vehicle, which is acquired by a camera 2; a second recognition unit 32 capable of recognizing a road-surface sign and a peripheral vehicle as recognition objects based on the image; an object determination unit 33 for determining a recognition object related to the road sign recognized by the first recognition unit 31 out of the recognition objects recognized by the second recognition unit 32, as an irradiation object of illumination light from a head lamp 5; and a light distribution control unit 34 for distributing the illumination light from the head lamp 5 to each of the road sign and the irradiation object.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a recognition assistance system applied to a vehicle, and in particular to a measure for improving the visibility of road signs and the like for a driver who drives the vehicle. [Background technology]

[0002] Conventionally, as disclosed in Patent Documents 1 and 2, systems have been proposed for improving the visibility of road signs at night (the visibility of road signs to a driver driving a vehicle).

[0003] Specifically, these patent documents disclose a method for determining whether or not a road sign is present in a forward image captured by an onboard camera, and if a road sign is present, recognizing the position of the road sign and directing the headlamp illumination direction toward the road sign, thereby improving the visibility of the road sign. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2006-327380 A [Patent Document 2] JP 2020-121615 A Summary of the Invention [Problem to be solved by the invention]

[0005] When a driver recognizes a road sign, the driver checks the road markings and surrounding vehicles, and operates the vehicle according to the contents of the road sign. For example, when a driver recognizes the existence of a "No straying" road sign, the driver will visually recognize the lane markers (yellow lane markers) painted on the road and operate the vehicle so that the vehicle does not stray from the lane markers. Also, when a driver recognizes the existence of a "Stop" road sign, the driver will visually recognize the stop line painted on the road and operate the vehicle so that the vehicle stops in front of the stop line.

[0006] The inventors of the present invention focused on the fact that such driving operations are performed, and came to the new insight that, in assisting the driver in driving, it is preferable not only to increase the visibility of road signs, but also to provide the driver with a situation that can be useful for subsequent driving operations (driving operations associated with checking the next road markings, etc. that need to be recognized after recognizing a road sign).

[0007] The present invention has been made in consideration of the above points, and its purpose is to propose a recognition assistance system that can not only increase the visibility of road signs, but also provide the driver with situations that are useful for driving a vehicle in accordance with the road signs. [Means for solving the problem]

[0008] The solution of the present invention for achieving the above-mentioned object is characterized in that it comprises a first recognition unit capable of recognizing a road sign based on an image acquired by an imaging means, a second recognition unit capable of recognizing at least one of a road marking and a surrounding vehicle as a recognition target based on the image, a target determination unit that determines, among the recognition targets recognized by the second recognition unit, a recognition target related to the road sign recognized by the first recognition unit as a target to be illuminated with illumination light from an on-board lighting device, and a light distribution control unit that distributes illumination light from the on-board lighting device to each of the road sign and the illumination target.

[0009] With this specification, when a road sign is recognized (recognized by the first recognition unit) based on an image acquired by the photographing means while the vehicle is traveling, and an illumination target such as a road marking or a surrounding vehicle that is a recognition target related to the road sign is recognized (recognized by the second recognition unit), illumination light from the vehicle-mounted lighting device is distributed to the road sign and the illumination target. By distributing illumination light toward the road sign, the visibility of the road sign can be improved. In addition, by distributing illumination light toward the illumination target (a recognition target related to the road sign), the driver can easily recognize the illumination target that is a recognition target to be recognized in the driving operation when driving the vehicle according to the contents of the road sign. In other words, a situation that is useful for driving operation after recognizing a road sign (a situation that is useful for driving operation when driving the vehicle according to the road sign) can be provided to the driver. Effect of the Invention

[0010] In the present invention, road signs are recognized based on images acquired by a photographing means, and illumination targets related to road signs are determined from among the recognized recognition targets, and illumination light from an on-board lighting device is distributed to each of these road signs and illumination targets. This not only increases the visibility of road signs, but also increases the visibility of recognition targets, and provides the driver with a situation that is useful for driving operations when driving a vehicle according to road signs (a situation that is useful for driving operations after recognizing road signs). [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a schematic configuration of a recognition assistance system according to an embodiment. [Diagram 2] FIG. 1 is a diagram showing examples of road signs in various countries and related recognition objects to be recognized. [Diagram 3] FIG. 11 is a flowchart illustrating a procedure of a recognition assistance process according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, the present invention will be described as being applied to a recognition support system that determines an object to be illuminated with illumination light from a headlamp in accordance with each of a plurality of types of road signs.

[0013] -Outline of the recognition support system- Fig. 1 is a block diagram showing a schematic configuration of a recognition support system 1 according to this embodiment. As shown in Fig. 1, the recognition support system 1 according to this embodiment includes a camera (imaging means) 2 that captures images of the periphery of a vehicle (hereinafter, sometimes referred to as a host vehicle), a recognition support ECU 3, a headlamp ECU 4, and a headlamp 5.

[0014] The camera 2 may be a well-known camera equipped with an image pickup element such as a complementary metal-oxide semiconductor (CMOS), a charge coupled device (CCD), etc. The camera 2 captures images at a predetermined time interval from when the vehicle ignition switch is turned ON to when it is turned OFF, and transmits a signal corresponding to the captured image to the recognition assistance ECU 3.

[0015] The recognition assistance ECU 3 includes, for example, a processor such as a CPU (Central Processing Unit), a ROM (Read-Only Memory) for storing a control program, a RAM (Random-Access Memory) for temporarily storing data, and an input / output port. The recognition assistance ECU 3 includes, as functional units realized by the control program, a first recognition unit 31, a second recognition unit 32, a target determination unit 33, and a light distribution control unit 34. Below, an outline of the functions of each of these units will be described.

[0016] The first recognition unit 31 is capable of recognizing road signs ahead of the vehicle based on the image acquired by the camera 2. Specifically, the first recognition unit 31 recognizes the presence or absence of road signs in the image, the type of the road sign when a road sign is present, and the position of the road sign (relative position to the vehicle) by performing image processing such as edge processing and pattern matching on the image information transmitted from the camera 2. For example, images of various road signs such as "No overrunning", "No turns", "Stop", and "Yield" are stored in the ROM of the recognition assistance system 1, and image processing such as pattern matching using the images of these road signs is performed on the image information transmitted from the camera 2 to recognize the presence or absence of road signs in the image, the type of road signs, and the position of the road signs. Note that the recognition of road signs may be performed by deep learning using a neural network.

[0017] The second recognition unit 32 is capable of recognizing road markings (also called road markings) and surrounding vehicles based on the images acquired by the camera 2. These road markings and surrounding vehicles correspond to the recognition targets referred to in the present invention. Specifically, the second recognition unit 32 recognizes the presence or absence of road markings in the image, the type of the road markings when the road markings exist, and the position of the road markings (relative position to the vehicle) by performing image processing such as edge processing and pattern matching on the image information transmitted from the camera 2. For example, the ROM of the recognition assistance system 1 stores images of various road markings such as "lane markers," "no turn road markings," and "stop line or stop road markings," and performs image processing such as pattern matching using the images of these road markings on the image information transmitted from the camera 2 to recognize the presence or absence of road markings in the image, the type of road markings, and the position of the road markings. Note that the recognition of road markings may be performed by deep learning using a neural network. The second recognition unit 32 can also recognize the presence or absence of a vehicle (other vehicle) traveling in the vicinity, and the relative position of the other vehicle if the other vehicle is present. The other vehicles that can be recognized here include not only other vehicles traveling in front of the host vehicle, but also other vehicles traveling to the side (including diagonally to the side) of the host vehicle.

[0018] The target determination unit 33 determines, among the road markings and surrounding vehicles (recognition targets) recognized by the second recognition unit 32, the recognition targets related to the road sign recognized by the first recognition unit 31 as targets to be illuminated with illumination light from the vehicle's headlamp (vehicle lighting device) 5. Fig. 2 shows examples of road signs in various countries and road markings (road markings) and vehicles that are related recognition targets to be recognized. That is, it shows examples of various road signs such as "No overtaking", "No turn", "Stop", and "Yield" and road markings and vehicles that are related recognition targets.

[0019] Specifically, when a driver recognizes the presence of a road sign indicating "no crossing the lane line", the driver will visually recognize the lane markers painted on the road and perform driving operations so that the vehicle does not cross the lane marker. For this reason, the lane marker is associated with the road sign indicating "no crossing the lane line" as a recognition target. In other words, when the first recognition unit 31 recognizes the road sign indicating "no crossing the lane line" and recognizes the position of the road sign, the second recognition unit 32 recognizes the presence or absence of a lane marker, the type (color, etc.) of the lane marker if a lane marker exists, and the position of the lane marker. In this case, the target determination unit 33 determines the lane marker as an illumination target (a recognition target related to the road sign and an illumination target of the illumination light from the headlamp 5 of the vehicle).

[0020] Also, when a driver recognizes the presence of a "no turn" road sign, the driver will drive the vehicle (without turning) while visually recognizing the no turn road marking painted on the road. For this reason, the no turn road marking is associated with the no turn road sign as a recognition target. In other words, when the first recognition unit 31 recognizes the no turn road sign and the position of the road sign, the second recognition unit 32 recognizes the presence or absence of the no turn road marking and, if the no turn road marking exists, the position of the no turn road marking. In this case, the target determination unit 33 determines the no turn road marking as the target to be illuminated.

[0021] Also, when a driver recognizes the presence of a "stop" road sign, the driver will perform a driving operation such that the vehicle stops in front of the stop line while visually recognizing the stop line or stop road markings painted on the road. For this reason, a stop line (including a stop road marking) is associated with the "stop" road sign as a recognition target. In other words, when the first recognition unit 31 recognizes a "stop" road sign and recognizes the position of this road sign, the second recognition unit 32 will recognize the presence or absence of a stop line and, if a stop line exists, the position of the stop line. In this case, the target determination unit 33 determines the stop line as an illumination target.

[0022] Furthermore, when the driver recognizes the presence of a "yield" road sign, the driver operates the vehicle so as not to impede the travel of other vehicles (vehicles traveling in the priority lane) while visually recognizing other vehicles traveling around the vehicle. For this reason, the "yield" road sign is associated with other vehicles (approaching vehicles on the priority side) as a recognition target. In other words, when the first recognition unit 31 recognizes the "yield" road sign and recognizes the position of this road sign, the second recognition unit 32 recognizes the presence or absence of other vehicles and the positions of the other vehicles (relative positions to the vehicle). In this case, the target determination unit 33 determines the other vehicles as the illumination target.

[0023] The light distribution control unit 34 generates a light distribution control signal for distributing illumination light from the headlamp 5 of the vehicle to each of the road signs recognized by the first recognition unit 31 and the illumination targets determined by the target determination unit 33, and transmits the signal to the headlamp ECU 4. The light distribution control signal generated here is configured to simultaneously irradiate the road signs and illumination targets with illumination light (start of irradiation of illumination light toward the road signs and illumination targets) and stop of irradiation (end of irradiation of illumination light toward the road signs and illumination targets). For example, irradiation of illumination light toward the road signs and illumination targets is started when the distance to the road signs reaches a predetermined distance or less, and irradiation of illumination light toward the road signs and illumination targets is stopped when the road signs are no longer recognized due to passing the installation position of the road signs.

[0024] Incidentally, this light distribution control signal may be one that intermittently irradiates the road signs and the illumination target with light at a predetermined time interval (by blinking the headlamps 5), or one that alternates between irradiating the road signs with light and irradiating the target with light at a predetermined time interval.

[0025] The headlamp ECU 4 controls the lighting of the headlamp 5 according to the light distribution control signal received from the light distribution control unit 34. As described above, since the direction of illumination light from the headlamp 5 needs to be directed toward the road sign and the illumination target, the headlamp 5 is configured to be able to change the illumination direction of the illumination light in various directions in addition to switching between so-called low beam and high beam. For example, the headlamp 5 is provided with a plurality of light sources (e.g., LED: Light Emitting Diode) and configured to be able to automatically adjust the direction of each light source as desired, so that when it becomes necessary to illuminate the road sign and the illumination target, the direction of some of the light sources is changed so that the illumination light can be illuminated on the road sign and the illumination target. Note that, when the vehicle is traveling, the relative positions of the road sign and the illumination target with respect to the vehicle (own vehicle) change, so the light distribution control signal is generated to adjust the illumination direction of the illumination light according to the position of the road sign recognized by the first recognition unit 31 and the position of the illumination target recognized by the second recognition unit 32. In addition, the headlamp 5 may be provided with a first light source for illuminating road signs and a second light source for illuminating an object to be illuminated, and the first light source may be turned on only when it is necessary to illuminate a road sign with illumination light, and the second light source may be turned on only when it is necessary to illuminate an object to be illuminated with illumination light.

[0026] -Recognition support processing- Next, the procedure of the recognition support process by the recognition support system 1 configured as described above will be described with reference to the flowchart of Fig. 3. This flowchart is repeatedly executed at predetermined time intervals from when the ignition switch of the vehicle is turned ON until it is turned OFF.

[0027] First, in step ST1, image information is acquired from the camera 2. The acquired image information is transmitted to the first recognition unit 31 and the second recognition unit 32, respectively.

[0028] In step ST2, it is determined whether or not a road sign is present in the acquired image. As described above, this determination is made by the first recognition unit 31. If no road sign is present in the acquired image and the determination is NO in step ST2, the process returns directly.

[0029] On the other hand, if a road sign is present in the acquired image and the result of the determination in step ST2 is YES, each recognition operation is performed in step ST3. The recognition operations performed here include the type of road sign, the position of the road sign, the type of road marking, the position of the road marking, and the position of other vehicles. The type of road sign and the position of the road sign are recognized by the first recognition unit 31. The type of road marking, the position of the road marking, and the position of other vehicles are recognized by the second recognition unit 32.

[0030] In step ST4, it is determined whether the type of the recognized road sign is "No overrunning". If the type of the recognized road sign is "No overrunning" and the determination in step ST4 is YES, the process proceeds to step ST5, where the lane marker is determined as the target to be illuminated.

[0031] Thereafter, in step ST6, illumination light from the headlamps 5 is distributed to the road signs and illumination objects. That is, illumination light from the headlamps 5 is distributed toward the "No Journey Beyond" road signs and lane markers.

[0032] In the judgment of step ST4, if the type of the recognized road sign is not "No turning", and the judgment is NO, the process proceeds to step ST7, where it is judged whether the type of the recognized road sign is "No turning". If the type of the recognized road sign is "No turning" and the judgment is YES in step ST7, the process proceeds to step ST8, where the no turning pavement marking is determined as the target to be illuminated.

[0033] In this case, in step ST6, illumination light from the headlamp 5 is distributed toward the "no turn" road sign and the no turn pavement markings.

[0034] In the judgment of step ST7, if the type of the recognized road sign is not "no turn" and the judgment is NO, the process proceeds to step ST9, where it is judged whether the type of the recognized road sign is "stop." If the type of the recognized road sign is "stop" and the judgment is YES in step ST9, the process proceeds to step ST10, where the stop line (including the stop road marking) is determined as the illumination target.

[0035] In this case, in step ST6, illumination light from the headlamp 5 is distributed toward the "stop" road sign and the stop line.

[0036] In the judgment of step ST9, if the type of the recognized road sign is not "stop" and the judgment is NO, the process proceeds to step ST11, where it is judged whether the type of the recognized road sign is "yield." If the type of the recognized road sign is "yield" and the judgment is YES in step ST11, the process proceeds to step ST12, where another vehicle (an approaching vehicle on the priority side) is determined to be the illumination target.

[0037] In this case, in step ST6, illumination light from the headlamp 5 is distributed toward the "yield" road sign and other vehicles.

[0038] Also, if the type of the recognized road sign is not "yield" and the result of the judgment in step ST11 is NO, the road sign recognized in step ST2 is determined to be not a road sign targeted by the recognition assistance system 1, and the process returns. The above operations are repeated.

[0039] -Effects of the embodiment- As described above, in this embodiment, road signs are recognized based on images acquired by the camera 2, and illumination targets that are related to road signs among the recognized recognition targets are determined, and illumination light from the headlamp 5 is distributed to each of these road signs and illumination targets. This not only increases the visibility of road signs, but also increases the visibility of recognition targets, and provides the driver with a situation that is useful for driving the vehicle in accordance with the road signs (a situation that is useful for driving after recognizing road signs).

[0040] -Other embodiments- The present invention is not limited to the above-described embodiment, and all modifications and applications within the scope of the claims and equivalents thereto are possible.

[0041] For example, in the above embodiment, the road signs to be targeted are “No overtaking”, “No turns”, “Stop”, and “Yield”, but the present invention is not limited to these and may target other road signs. [Industrial Applicability]

[0042] INDUSTRIAL APPLICABILITY The present invention is applicable to a recognition assistance system that can provide a situation that is helpful for driving a vehicle in accordance with road signs. [Explanation of symbols]

[0043] 1... Recognition support system 2... Camera (photographing means) 31... First recognition unit 32: Second recognition unit 33: Object determination unit 34: Light distribution control unit

Claims

[Claim 1] A first recognition unit capable of recognizing road signs based on images acquired by an imaging means; a second recognition unit capable of recognizing at least one of a road marking and a surrounding vehicle as a recognition target based on the image; an object determining unit that determines, among the objects recognized by the second recognition unit, an object related to the road sign recognized by the first recognition unit as an object to be illuminated with illumination light from an in-vehicle lighting device; a light distribution control unit that distributes illumination light from the vehicle-mounted lighting device to each of the road sign and the illumination target.

Images