Entry Prohibition Detection Device

Abstract

To properly determine whether a vehicle is entering a road where entry is prohibited when turning right or left. [Solution] The entry prohibition determination device 100 includes an entry restriction display recognition unit 11 that recognizes an entry restriction display M indicating that vehicle entry to the road is prohibited, which is provided on the road, based on an image captured by the front camera 1 of the vehicle 20, and an entry prohibition determination unit 12 that determines whether the vehicle 20 has entered a road where entry is prohibited when the vehicle 20 is in a right or left turning state and at least one entry restriction display M is recognized near the entrance of the road R into which the vehicle 20 is entering.

Description

Technical Field

[0001] The present invention relates to an access prohibition determination device.

Background Art

[0002] For example, Patent Document 1 describes an access prohibition determination device (access determination device) that determines whether a vehicle enters a prohibited access road. This access prohibition determination device determines whether a vehicle enters a prohibited access road based on the recognition that two access prohibition signs exist on the left and right of the vehicle's driving route in the captured image of the vehicle's front camera.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the access prohibition determination device as described above, there may be an incorrect determination of whether a vehicle enters a prohibited access road depending on the installation environment of the access prohibition sign and the way the access prohibition sign appears in the captured image. In particular, during a right or left turn, even if two access prohibition signs are installed near the entrance of the access destination road that the vehicle is entering, one of the access prohibition signs may not be within the visual range of the imaging device, and it may not be determined that the vehicle enters the prohibited access road.

Means for Solving the Problems

[0005] The access prohibition determination device according to one aspect of the present invention includes an access regulation display recognition unit that recognizes an access regulation display provided on a road and indicating prohibition of vehicle entry into the road based on a captured image of a front camera of a vehicle, and an access prohibition determination unit that determines that the vehicle enters a prohibited access road when at least one access regulation display is recognized near the entrance of the access destination road that the vehicle is entering when the vehicle is in a right or left turn state. [Effects of the Invention]

[0006] According to one aspect of the present invention, it is possible to appropriately determine whether a vehicle is entering a road where entry is prohibited when turning left or right. [Brief explanation of the drawing]

[0007] [Figure 1] This is a block diagram showing an entry prohibition determination device according to one embodiment. [Figure 2] (a) A plan view showing an example of the situation before a vehicle turns left at an intersection. (b) A plan view showing an example of the situation when a vehicle turns left and enters a road where entry is prohibited. (c) A plan view showing an example of the situation when a vehicle has entered a road where entry is prohibited. [Figure 3] This is a flowchart showing an example of a process for determining whether entry is prohibited. [Figure 4] This is a plan view illustrating an example of how entry prohibition is determined when a vehicle is proceeding straight through an intersection. [Modes for carrying out the invention]

[0008] The following describes exemplary embodiments with reference to the drawings. In each drawing, identical or equivalent elements are denoted by the same reference numerals, and redundant explanations are omitted.

[0009] Figure 1 is a block diagram showing an embodiment of the no-entry detection device 100. The no-entry detection device 100 is mounted on a vehicle 20 (see Figure 2(a)) and is a device for preventing the vehicle 20 from mistakenly entering a no-entry road. The no-entry detection device 100 recognizes no-entry signs using images captured by the vehicle 20's front camera 1, for example, and alerts the driver if the vehicle 20 is about to enter a no-entry road. No-entry signs and no-entry roads will be described in more detail later.

[0010] The entry prohibition determination device 100 is equipped with an ECU (Electronic Control Unit) 10 that comprehensively manages the device. The ECU 10 is an electronic control unit having a CPU (Central Processing Unit) and a memory unit. The memory unit consists of, for example, ROM (Read Only Memory), RAM (Random Access Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), etc. The ECU 10 realizes various functions by, for example, executing a program stored in the memory unit with the CPU. The ECU 10 may be composed of multiple electronic control units.

[0011] The ECU10 is connected to a front camera 1, a turn signal sensor 2, a steering angle sensor 3, a yaw rate sensor 4, a display 5, and a speaker 6. These devices connected to the ECU10 are mounted on the vehicle 20.

[0012] The front camera 1 is installed, for example, behind the windshield of the vehicle 20 and captures images of the area in front of the vehicle 20. The front camera 1 captures images of the area in front of the vehicle 20 that falls within the imaging range A (see Figure 2(a)) and transmits the captured images to the ECU 10. The installation location and number of front cameras 1 are not limited.

[0013] The turn signal sensor 2 detects the operating state and direction of the vehicle's turn signal (e.g., a turn signal indicator) and transmits data regarding the operating state and direction to the ECU 10. The turn signal sensor 2 detects whether the vehicle's turn signal is operating or not, and also detects whether the turn signal is operating to the right or left.

[0014] The steering angle sensor 3 is installed, for example, on the steering column and detects the steering angle (rotation angle) of the steering wheel of the vehicle 20. The steering angle sensor 3 transmits data related to the detected steering angle to the ECU 10. The yaw rate sensor 4 is installed, for example, near the center of gravity of the vehicle and detects the rotational speed (yaw rate) around the yaw axis (vertical axis) of the vehicle 20. The yaw rate sensor 4 detects the rotational speed using, for example, a gyroscope. The yaw rate sensor 4 transmits data related to the detected rotational speed to the ECU 10.

[0015] Display 5 is a display device that is installed, for example, inside the cabin of a vehicle 20 and presents information to the driver of the vehicle 20 visually. Display 5 may be, for example, a center display, a MID (Multi Information Display), or a HUD (Head Up Display). Display 5 displays various information in accordance with the control signals of the ECU 10. Speaker 6 is a device that is installed, for example, inside the cabin of a vehicle 20 and presents information to the driver of the vehicle 20 by sound. Speaker 6 outputs sound and the like based on the control signals of the ECU 10.

[0016] Next, the functional configuration of the ECU10 will be described. Functionally, the ECU10 has an entry restriction display recognition unit 11 and an entry prohibition determination unit 12.

[0017] The entry restriction sign recognition unit 11 recognizes entry restriction signs that are installed on the road and indicate that vehicle entry to the road is prohibited, based on the image captured by the front camera 1. An entry restriction sign is a sign that prohibits the entry of a vehicle 20. Entry restriction signs include, for example, entry prohibition signs installed as road signs near the entrance of a road where entry is prohibited. Entry restriction signs may also include one-way signs that show an arrow indicating the direction in which a vehicle 20 would be traveling in the wrong direction if it enters. Entry restriction signs may also include road markings that prohibit the entry of a vehicle 20. A road where entry is prohibited is a road where entry of a vehicle 20 is prohibited. A road where entry is prohibited includes, for example, a road where entry of a vehicle 20 is prohibited to ensure the safety of pedestrians and cyclists. A road where entry is prohibited may also include one-way roads viewed from the exit side (one-way roads where a vehicle 20 would be traveling in the wrong direction if it entered).

[0018] Here, Figure 2(a) is a plan view showing an example of the situation before vehicle 20 turns left at an intersection. Figure 2(a) shows the road R into which vehicle 20 is turning left, and the entry restriction sign M installed on road R. In Figure 2(a), vehicle 20 is about to turn left at a crossroads intersection and enter road R. Road R is a one-way street, and since vehicle 20 is located on the exit side of the one-way street, entry by vehicle 20 is prohibited.

[0019] As shown in Figure 2(a), the entry restriction sign M includes an entry prohibition sign M1 and a one-way sign M2. The entry prohibition sign M1 in Figure 2(a) is placed on the left and right sides of the entry road R near the entrance to the entry road R. The one-way sign M2 is placed on one of the left or right sides of the entry road R near the entrance to the entry road R (for example, the left side when viewed from the intersection). Note that the entrance to the entry road R means the entrance as viewed from the vehicle 20, and is the exit for a one-way road. The area near the entrance to the entry road R can be defined as within a certain distance from the entrance to the entry road R.

[0020] The entry prohibition sign M1 is a road sign that indicates the prohibition of vehicle entry, for example, a white horizontal line is drawn in a red-filled circle. On the other hand, the one-way sign M2 is a road sign that indicates one-way traffic in a specific direction, for example, a white arrow is drawn in a blue-filled square. The entry restriction display recognition unit 11 recognizes the one-way sign M2 as an entry restriction display M when the direction indicated by the arrow of the one-way sign M2 is opposite to the entry direction to the entry destination road R of the vehicle 20. The entry prohibition sign M1 and the one-way sign M2 are not limited to the above-described shapes and display contents, and may be road signs in a form defined by a country or region so as to restrict the entry of the vehicle 20.

[0021] In the situation shown in Fig. 2(a), the entry restriction display recognition unit 11 recognizes the entry restriction display M provided near the entrance of the entry destination road R based on the captured image of the front camera 1. The entry restriction display recognition unit 11 recognizes the entry prohibition sign M1 and the one-way sign M2 as the entry restriction display M.

[0022] The entry restriction display recognition unit 11 may analyze the captured image obtained from the front camera 1 and recognize these entry restriction displays M. The entry restriction display recognition unit 11 may recognize the entry restriction display M, for example, by comparing the template of the pre-registered entry restriction display M with the pattern in the captured image data. Also, the recognition accuracy may be improved by constructing a recognition model of the entry restriction display M using a machine learning algorithm.

[0023] The entry restriction sign recognition unit 11 may not recognize entry restriction signs M whose distance from the vehicle 20 is greater than or equal to the recognition distance threshold, but may recognize entry restriction signs M whose distance from the vehicle 20 is less than the recognition distance threshold. In this way, by excluding entry restriction signs M whose distance from the vehicle 20 is greater than or equal to the recognition distance threshold from being recognized, it is possible to suppress the erroneous recognition of entry restriction signs M installed at a location far from the entrance of the road to be entered R. As a result, it is possible to suppress the erroneous determination that the vehicle 20 is entering a prohibited road when it is actually entering a permitted road. As another example, when the vehicle 20 enters a permitted road where entry is permitted, and the road adjacent to the permitted road on the opposite side is a prohibited road, the entry restriction sign recognition unit 11 can suppress the erroneous recognition of entry restriction signs M installed on the prohibited road on the opposite side.

[0024] The recognition distance threshold is, for example, 4m, but can be set to any value. The distance from the vehicle 20 to the entry restriction sign M may be calculated, for example, based on the image captured by the front camera 1, or it may be acquired by a radar sensor installed on the vehicle 20.

[0025] The entry prohibition determination unit 12 determines that vehicle 20 is entering a prohibited road when vehicle 20 is in a right or left turning state and at least one entry restriction sign M is recognized near the entrance of the road R into which vehicle 20 is entering. The functions of the entry prohibition determination unit 12 will be described below with reference to Figures 2(a) to 2(c).

[0026] Figure 2(b) is a plan view showing an example of a vehicle entering a no-entry road by turning left. Figure 2(c) is a plan view showing an example of a vehicle having entered a no-entry road.

[0027] First, the no-entry determination unit 12 determines whether the vehicle 20 is in a right or left turning state. This determination may be made based on the operating state of the vehicle's turn signals, steering angle, and yaw rate. Specifically, the no-entry determination unit 12 may determine whether the turn signals are activated (whether there is an intention to turn right or left) based on data acquired from the turn signal sensor 2. If the turn signals are activated (there is an intention to turn right or left), the no-entry determination unit 12 may use data acquired from the steering angle sensor 3 and the yaw rate sensor 4 to determine whether the steering angle or yaw rate in the direction of the turn signal operation is above a threshold in that direction. For example, if the turn signal is activated to the right, it determines whether the steering angle or yaw rate is above a threshold in the right direction. This threshold is, for example, a preset value and can be arbitrarily set according to the configuration of the vehicle 20.

[0028] The entry prohibition determination unit 12 may determine that the vehicle 20 is in a right-turn or left-turn state if the vehicle is in a right-turn or left-turn state and the steering angle or yaw rate in the direction in which the turn signal is activated is greater than or equal to a threshold (see, for example, Figure 2(b)). On the other hand, the entry prohibition determination unit 12 may determine that the vehicle is not in a right-turn or left-turn state (is going straight) if the turn signal is not activated (there is no intention to turn right or left), or if the turn signal is activated but the steering angle and yaw rate in the direction in which the turn signal is activated are not greater than or equal to a threshold.

[0029] Next, the entry prohibition determination unit 12 determines that the vehicle 20 has entered a prohibited road when the vehicle 20 is in a right or left turning state and the entry restriction sign recognition unit 11 recognizes at least one entry restriction sign M near the entrance of the road R to be entered. For example, as shown in Figure 2(b), while the vehicle 20 is turning left, one entry prohibition sign M1 and one-way sign M2 are each located inside the imaging range A of the front camera 1, and these signs may be recognized. In this case, the entry prohibition determination unit 12 may determine that the vehicle 20 has entered a prohibited road.

[0030] As shown in Figure 3, when a vehicle 20 enters the destination road R in a straight-ahead state, the entry prohibition determination unit 12 determines whether or not entry restriction signs M exist on both sides of the destination road R. In this case, when the vehicle 20 is in a straight-ahead state, the entry prohibition determination unit 12 does not need to determine that the vehicle has entered a prohibited road based solely on the recognition of one entry restriction sign M on one side.

[0031] If the entry prohibition determination unit 12 determines that the vehicle 20 has entered a road where entry is prohibited, it may alert the driver through the display 5 and speaker 6. The entry prohibition determination unit 12 may alert the driver by, for example, activating an alert device connected to the display 5 and speaker 6. The display 5 may display a warning message indicating entry into a road where entry is prohibited, or the speaker 6 may emit a warning sound, for example, to notify the driver of entry into a road where entry is prohibited.

[0032] The warning issued by the no-entry determination unit 12 may be given before turning right or left, as shown in Figure 2(a), or after entering the no-entry road, as shown in Figure 2(c). Furthermore, the warning may be given after entering the no-entry road, after the vehicle 20 has passed the no-entry restriction sign M, and after it has traveled a predetermined distance from the no-entry restriction sign M.

[0033] Next, the process of the entry prohibition determination device 100 will be explained with reference to Figure 4. Note that the process shown in Figure 4 is executed when the vehicle 20 starts moving.

[0034] As shown in Figure 4, the ECU 10 determines whether the vehicle 20's turn signal is activated (step S1). If the turn signal is activated (step S1: YES), the ECU 10 determines whether the steering angle or yaw rate in the direction of the turn signal's activation is greater than or equal to a threshold (step S2). If the turn signal is not activated (step S1: NO), the ECU 10 determines that the vehicle 20 is moving straight and executes the process of step S4 described below.

[0035] If the steering angle or yaw rate in the direction in which the turn signal is activated is greater than or equal to a threshold (Step S2: YES), the ECU 10 determines that the vehicle 20 is in a right or left turning state and determines whether the entry restriction sign M is recognized at a distance less than the recognition distance threshold from the vehicle near the entrance of the road R to be entered (Step S3). If the steering angle or yaw rate in the direction in which the turn signal is activated is not greater than or equal to a threshold (Step S2: NO), the ECU 10 determines that the vehicle 20 is in a straight-ahead state and executes the process of Step S4 described below.

[0036] If the entry restriction sign M is detected at a distance less than the recognition distance threshold from the vehicle 20 near the entrance to the road R to be entered (Step S3: YES), the ECU 10 issues a warning to the vehicle 20 about entering the road where entry is prohibited (Step S5). After that, the ECU 10 terminates processing. If the entry restriction sign M is not detected at a distance less than the recognition distance threshold from the vehicle near the entrance to the road R to be entered (Step S3: NO), the ECU 10 restarts processing from the beginning.

[0037] As described above, if the turn signal is not activated (Step S1: NO), and if the steering angle or yaw rate in the direction the turn signal is activated is not above a threshold (Step S2: NO), the ECU 10 determines that the vehicle 20 is in a straight-ahead state and executes Step S4. In Step S4, the ECU 10 determines whether or not entry restriction signs M are recognized on both sides of the road R to be entered. If entry restriction signs are recognized on both sides of the road R to be entered (Step S4: YES), the warning in Step S5 is issued. If entry restriction signs are not recognized on both sides of the road R to be entered (Step S4: NO), the ECU 10 executes the process again from the start.

[0038] Furthermore, if the turn signal is activated and the steering angle or yaw rate in the direction the turn signal is activated is below a threshold, the ECU 10 may, after executing step S4, further determine whether the vehicle 20 has passed the entry restriction sign M. In this case, if the vehicle 20 has passed the entry restriction sign M, the ECU 10 may issue a warning in step S5, but if it has not passed the entry restriction sign M, it may not issue a warning and restart the process from the beginning. This can prevent the system from mistakenly determining that the vehicle is entering a no-entry road by going straight, even though it intends to turn right or left, and can prevent unnecessary warnings from being issued. Also, if the turn signal is activated and the steering angle or yaw rate in the direction the turn signal is activated is below a threshold, the ECU 10 may skip step S4 and restart the process from the beginning. Even in this case, unnecessary warnings can be prevented.

[0039] As described above, in the entry prohibition determination device 100 according to this embodiment, when the vehicle 20 is in a right or left turning state, the ECU 10 determines that the vehicle 20 has entered a prohibited road when at least one entry restriction indicator M is recognized near the entrance of the road R into which the vehicle 20 is entering. This allows for more appropriate determination of whether a vehicle has entered a prohibited road when turning right or left, compared to, for example, when the ECU 10 determines entry into a prohibited road only when entry restriction indicators are recognized on both sides of the road R into which the vehicle is entering.

[0040] In other words, as shown in Figure 3, when vehicle 20 enters the destination road R in a straight line, the entry restriction signs M on both sides of the no-entry road are often recognized. On the other hand, as shown in Figure 2(b), when entering the destination road R in a right or left turn, even if entry restriction signs M are installed on both sides of the destination road R, one of the entry restriction signs M may not be within the imaging range A of the front camera 1, and only one of the entry restriction signs M may be recognized. In this case, entry into the no-entry road may not be detected, and a warning may not be issued.

[0041] On the other hand, in the entry prohibition determination device 100 according to this embodiment, when the vehicle 20 is in a right or left turning state, if at least one entry restriction display M is recognized near the entrance of the road R to be entered, it is determined that the vehicle 20 has entered a prohibited road. Therefore, even if, for example, only one entry restriction display M is recognized when turning right or left, it is possible to appropriately determine whether the vehicle has entered a prohibited road. As a result, appropriate warnings can be issued when a vehicle mistakenly enters a prohibited road.

[0042] Furthermore, the entry prohibition determination device 100 may determine whether the vehicle 20 is in a turning state based on at least one of the following: whether the vehicle 20's turn signal is activated, the steering angle of the vehicle 20, and the yaw rate of the vehicle 20. This allows for an appropriate determination of whether the vehicle 20 is in a turning state.

[0043] Furthermore, the entry prohibition determination device 100 may determine whether the vehicle 20 is in a turning state based on the steering angle or yaw rate of the vehicle 20 when the vehicle 20's turn signal is activated. This allows for a more accurate determination of whether the vehicle 20 is in a turning state.

[0044] Furthermore, in the entry prohibition determination device 100, the entry restriction display recognition unit 11 may not recognize entry restriction displays M whose distance from the vehicle 20 is greater than or equal to the recognition distance threshold, but may recognize entry restriction displays M whose distance from the vehicle 20 is less than the recognition distance threshold. This prevents, for example, the vehicle 20 from mistakenly determining that it has entered a prohibited road based on an entry restriction display M installed at a distance, even though it has entered an permitted road where entry is permitted.

[0045] Furthermore, in the entry prohibition determination device 100, if the vehicle 20 is in a straight-ahead state, and it is determined that there are entry restriction signs M on both sides of the road R to be entered, the ECU 10 may determine that the vehicle 20 has entered a prohibited road. In other words, in a straight-ahead state, even if one entry restriction sign M is incorrectly recognized, the ECU 10 does not need to determine that the vehicle has entered a prohibited road. This prevents unnecessary warnings from being issued, for example, when the vehicle 20 is entering a permitted road. Therefore, according to the entry prohibition determination device 100 of the above embodiment, unnecessary warnings during straight-ahead driving can be suppressed while appropriately determining whether the vehicle has entered a prohibited road when turning right or left.

[0046] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. The present invention can be implemented in various forms, starting with the embodiments described above, by making various changes and improvements based on the knowledge of those skilled in the art.

[0047] In the above embodiment, the entry prohibition determination unit 12 determined that the vehicle 20 was in a right-turn or left-turn state when the steering angle or yaw rate in the direction of operation of the turn signal was above a threshold. However, the entry prohibition determination unit 12 may also determine that the vehicle 20 is in a right-turn or left-turn state when both the steering angle and yaw rate are above a threshold, without considering the direction of operation of the turn signal.

[0048] Furthermore, the no-entry determination unit 12 may determine whether or not the vehicle is in a right or left turning state based solely on whether or not the turn signals are activated. Alternatively, the no-entry determination unit 12 may determine whether or not the vehicle is in a right or left turning state based solely on the steering angle, or based solely on the yaw rate. In other words, the no-entry determination unit 12 may determine whether or not the vehicle 20 is in a right or left turning state based on at least one of the following: whether or not the turn signals are activated, the steering angle, and the yaw rate. Alternatively, the no-entry determination unit 12 may determine whether or not the vehicle is in a right or left turning state by means other than the activation of the turn signals, the steering angle, and the yaw rate. The no-entry determination unit 12 may also determine whether or not the vehicle 20 is in a right or left turning state based on the image captured by the front camera 1.

[0049] Furthermore, in the above embodiment, the entry prohibition determination unit 12 issued a warning to the driver when it determined that the vehicle 20 was entering a road where entry is prohibited. However, the entry prohibition determination unit 12 does not have to issue a warning, and may, for example, control the vehicle 20 to decelerate. The entry prohibition determination device 100 does not necessarily need to require that both entry restriction signs M on both sides of the road be recognized when the vehicle 20 is entering the road R in a straight-ahead state to determine that it has entered a road where entry is prohibited. For example, if the vehicle 20 is in a straight-ahead state, the entry prohibition determination device 100 may determine that it has entered a road where entry is prohibited based on the vehicle 20's position information and map information. The entry prohibition determination device 100 may also determine that it has entered a road where entry is prohibited by other methods. [Explanation of Symbols]

[0050] 1...Front camera, 2...Direction indicator sensor, 3...Steering angle sensor, 4...Yaw rate sensor, 10...ECU, 11...Entry restriction display recognition unit, 12...Entry prohibition determination unit, 20...Vehicle, 100...Entry prohibition determination device, M...Entry restriction display, M1...Entry prohibition sign, M2...One-way sign, R...Enter destination road, A...Image range.

Claims

1. An entry restriction sign recognition unit recognizes an entry restriction sign installed on a road that indicates a prohibition of vehicle entry onto the road, based on an image captured by a front-facing camera of the vehicle. An entry prohibition determination device comprising: an entry prohibition determination unit that determines whether the vehicle is entering a prohibited road when the vehicle is in a right or left turning state and at least one of the entry restriction signs is recognized near the entrance of the road into which the vehicle is entering.

2. The entry prohibition determination device according to claim 1, which determines whether the vehicle is in a right or left turning state based on whether the vehicle's turn signal is activated, the steering angle of the vehicle, and the yaw rate of the vehicle.

3. The entry prohibition determination device according to claim 1, which determines whether the vehicle is in a right or left turning state based on the steering angle or yaw rate of the vehicle when the vehicle's turn signal is activated.

4. The entry restriction display recognition unit does not recognize entry restriction displays whose distance from the vehicle is greater than or equal to a recognition distance threshold, and recognizes entry restriction displays whose distance from the vehicle is less than a recognition distance threshold, as described in any one of claims 1 to 3.

Images