Passage prohibition determination device

Abstract

The present invention addresses the problem of appropriately determining that a vehicle enters a passage prohibited road when turning left and right. A passage prohibition determination device is provided with: an entry restriction sign recognition unit that recognizes an entry restriction sign that is provided on a road and indicates prohibition of the vehicle from entering the road, on the basis of a captured image of a camera in front of the vehicle; and a passage prohibition determination unit that determines that the vehicle enters the passage prohibition road when at least one entry restriction sign is recognized in the vicinity of the entrance of the entry destination road into which the vehicle enters, when the vehicle is in the left-right turning state.

Description

Technical Field

[0001] This invention relates to a device for determining whether passage is prohibited. Background Technology

[0002] For example, Patent Document 1 describes a vehicle entry determination device (entry determination device) for determining whether a vehicle has entered a prohibited road. This entry determination device determines that the vehicle has entered a prohibited road based on the image captured by a camera in front of the vehicle, which identifies two prohibition signs on the left and right sides of the vehicle's travel path.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2016-224714 Summary of the Invention

[0004] In the prohibited passage determination device described above, depending on the placement environment of the prohibited passage sign and the way the prohibited passage sign is presented in the camera image, it may sometimes incorrectly determine that a vehicle has entered a prohibited road. In particular, when making left or right turns, even if there are two prohibited passage signs near the entrance to the road leading to the destination that the vehicle is entering, sometimes one of the prohibited passage signs may not be within the visual recognition range of the camera device, thus failing to determine that the vehicle has entered a prohibited road.

[0005] One aspect of the present invention relates to a no-entry determination device comprising: an entry restriction sign recognition unit that identifies an entry restriction sign placed on the road and indicating that a vehicle is prohibited from entering the road based on an image captured by a camera in front of the vehicle; and a no-entry determination unit that, when the vehicle is in a left or right turning state, determines that the vehicle has entered a no-entry road when it identifies at least one entry restriction sign near the entrance of the road to the destination that the vehicle is entering.

[0006] Invention Effects

[0007] According to one aspect of the present invention, it is possible to appropriately determine whether a vehicle entering a prohibited road is making a left or right turn. Attached Figure Description

[0008] Figure 1 This is a block diagram illustrating a method for determining a prohibition of passage according to one embodiment.

[0009] Figure 2 (a) is a top view showing an example of the situation before a vehicle makes a left turn at an intersection. Figure 2 (b) is a top view showing an example of a vehicle turning left onto a road where traffic is prohibited. Figure 2 (c) is a top view showing an example of a vehicle entering a road where passage is prohibited.

[0010] Figure 3This is a flowchart illustrating an example of a process for determining whether passage is prohibited.

[0011] Figure 4 This is a top view showing an example of a situation where a vehicle is prohibited from proceeding straight through an intersection. Detailed Implementation

[0012] Hereinafter, exemplary embodiments will be described with reference to the accompanying drawings. Furthermore, in each drawing, the same or equivalent elements are labeled with the same symbols, and repeated descriptions are omitted.

[0013] Figure 1 This is a block diagram illustrating the passage restriction determination device 100 according to the embodiment. The passage restriction determination device 100 is, for example, mounted on a vehicle 20 (see reference). Figure 2 (a) A device for preventing vehicle 20 from accidentally entering a prohibited road. The prohibited road determination device 100 uses, for example, an image captured by the front camera 1 of vehicle 20 to identify a prohibited road sign, and provides a warning to the driver if vehicle 20 intends to enter a prohibited road. Prohibited road signs and prohibited roads will be described in detail later.

[0014] The passage restriction determination device 100 includes an Electronic Control Unit (ECU) 10 with a centralized management device. The ECU 10 is an ECU with a Central Processing Unit (CPU) and a storage unit. The storage unit may consist of, for example, a Read Only Memory (ROM), Random Access Memory (RAM), or Electrically Erasable Programmable Read-Only Memory (EEPROM). In the ECU 10, various functions are implemented by executing programs stored in the storage unit, for example, via the CPU. The ECU 10 may consist of multiple ECUs.

[0015] The ECU10 is connected to a front camera 1, a direction indicator sensor 2, a steering angle sensor 3, a yaw rate sensor 4, a display 5, and a speaker 6. These devices connected to the ECU10 are mounted on the vehicle 20.

[0016] For example, a front camera 1 is mounted behind the windshield of vehicle 20 to capture images of the front of vehicle 20. For example, the front camera 1 captures images entering the shooting range A (see reference). Figure 2 (a) The front camera 1 is positioned in front of the vehicle 20 and the captured images are sent to the ECU 10. The location or number of the front camera 1 is not limited.

[0017] The direction indicator sensor 2 detects the operating status and direction of the direction indicator (e.g., turn signal) of the vehicle 20, and sends data related to the operating status and direction to the ECU 10. The direction indicator sensor 2 detects whether the direction indicator of the vehicle 20 is working, and detects whether the direction indicator is working in the right or left direction.

[0018] Steering angle sensor 3, for example, is mounted on the steering column to detect the steering angle (rotation angle) of the steering wheel of vehicle 20. Steering angle sensor 3 sends data related to the detected steering angle to ECU 10. Yaw rate sensor 4, for example, is mounted near the vehicle's center of gravity to detect the rotational speed (yaw rate) of vehicle 20 about the yaw axis (vertical axis). Yaw rate sensor 4 uses a gyroscope, for example, to detect the rotational speed. Yaw rate sensor 4 sends data related to the detected rotational speed to ECU 10.

[0019] Display 5, for example, is installed in the interior of vehicle 20 and is a display device that provides visual information to the driver of vehicle 20. Display 5 can be, for example, a central display, a multi-information display (MID), or a head-up display (HUD). Display 5 displays various information according to control signals from ECU 10. Speaker 6, for example, is installed in the interior of vehicle 20 and is a device that provides voice information to the driver of vehicle 20. Speaker 6 outputs voice, etc., according to control signals from ECU 10.

[0020] Next, the functional structure of ECU10 will be explained. Functionally, ECU10 has an entry restriction sign recognition unit 11 and a passage prohibition determination unit 12.

[0021] The entry restriction sign recognition unit 11 identifies entry restriction signs placed on the road that indicate a prohibition on vehicle entry, based on the image captured by the forward camera 1. An entry restriction sign is a sign used to prohibit vehicle 20 from entering. Entry restriction signs may include, for example, a prohibition sign placed near the entrance of a prohibited road as a road sign. Entry restriction signs may include, for example, a one-way sign with an arrow indicating the direction of travel if vehicle 20 enters. Entry restriction signs may include road markings used to prohibit vehicle 20 from entering. A prohibited road is a road where vehicle 20 is prohibited from entering. Prohibited roads may include, for example, roads where vehicle 20 is prohibited from entering to ensure the safety of pedestrians or bicycles. Prohibited roads may include one-way roads viewed from the exit side (one-way roads that become the direction of travel if vehicle 20 enters).

[0022] Here, Figure 2(a) is a top view showing an example of the situation before a vehicle 20 makes a left turn at an intersection. Figure 2 In (a), the entry road R for the left-turning vehicle 20 to its destination and the entry restriction sign M set on the entry road R are shown. Figure 2 In (a), vehicle 20 intends to turn left at the intersection to enter destination road R. Destination road R is a one-way road, and vehicle 20 is prohibited from entering the road in order to place vehicle 20 on the exit side of the one-way traffic.

[0023] like Figure 2 As shown in (a), the entry restriction sign M includes a no-entry sign M1 and a one-way sign M2. Figure 2 (a) The no-entry sign M1 is placed on both the left and right sides of the entrance to the destination road R. The one-way sign M2 is placed on one side of the entrance to the destination road R (e.g., the left side when viewed from an intersection). The entrance to the destination road R refers to the entrance as viewed from vehicle 20, and serves as an exit for a one-way road. The vicinity of the entrance to the destination road R can be set within a certain distance from the entrance to the destination road R.

[0024] The no-entry sign M1 is, for example, a road sign with a white horizontal line drawn inside a red-filled circle, indicating that vehicles are prohibited from entering. The one-way sign M2 is, for example, a road sign with a white arrow drawn inside a blue-filled quadrilateral, indicating one-way traffic in a specific direction. When the direction indicated by the arrow of the one-way sign M2 is opposite to the direction in which vehicle 20 enters the destination road R, the entry restriction sign recognition unit 11 recognizes the one-way sign M2 as an entry restriction sign M. The no-entry sign M1 and the one-way sign M2 are not limited to the above-described shapes or display content, and may also be road signs that restrict vehicle 20 entry according to national or regional regulations.

[0025] exist Figure 2 In the situation shown in (a), the entry restriction sign recognition unit 11 identifies the entry restriction sign M located near the entrance to the destination road R based on the image captured by the camera 1 ahead. The entry restriction sign recognition unit 11 identifies the no-entry sign M1 and the one-way sign M2 as the entry restriction sign M.

[0026] The entry restriction sign recognition unit 11 can analyze the video images acquired from the front camera 1 and identify these entry restriction signs M. For example, the entry restriction sign recognition unit 11 can identify the entry restriction sign M by comparing a pre-registered template of the entry restriction sign M with a pattern within the video image data. Furthermore, the recognition accuracy can be improved by using machine learning algorithms to construct a recognition model for the entry restriction sign M.

[0027] The entry restriction sign recognition unit 11 can exclude entry restriction signs M located at a distance greater than or equal to the recognition distance threshold from the vehicle 20, and instead recognize entry restriction signs M located at a distance less than the recognition distance threshold. By excluding entry restriction signs M located at a distance greater than or equal to the recognition distance threshold from the recognition targets, for example, it can suppress the erroneous recognition of entry restriction signs M located near the entrance to the destination road R, which is far from the road. As a result, it can suppress the erroneous identification of entry as a prohibited road even when the vehicle 20 is entering a permitted road. As another example, the entry restriction sign recognition unit 11 can suppress the erroneous recognition of entry restriction signs M located on the opposite side of a prohibited road when the vehicle 20 enters a permitted road and the road on the opposite side adjacent to that permitted road is a prohibited road.

[0028] Furthermore, the recognition distance threshold is, for example, 4m, but can be set to any value. Also, the distance from vehicle 20 to the entry restriction sign M can be calculated, for example, from a camera image acquired from the front camera 1, or obtained from a radar sensor installed on vehicle 20.

[0029] When vehicle 20 is turning left or right, if the prohibition determination unit 12 identifies at least one entry restriction sign M near the entrance to the destination road R that vehicle 20 is entering, it determines that vehicle 20 may enter the prohibited road. Hereinafter, refer to... Figure 2 (a)~ Figure 2 (c) The function of the prohibition determination unit 12 is explained.

[0030] Figure 2 (b) is a top view showing an example of a vehicle turning left onto a road where traffic is prohibited. Figure 2 (c) is a top view showing an example of a vehicle entering a road where passage is prohibited.

[0031] First, the prohibition-of-passage determination unit 12 determines whether the vehicle 20 is in a left or right turning state. This determination can be made based on the operating status of the vehicle's direction indicator, steering angle, and yaw rate. Specifically, the prohibition-of-passage determination unit 12 can determine whether the direction indicator is working (whether there is an intention to turn left or right) based on data obtained from the direction indicator sensor 2. If the direction indicator is working (if there is an intention to turn left or right), the prohibition-of-passage determination unit 12 can use data obtained from the steering angle sensor 3 and the yaw rate sensor 4 to determine whether the steering angle or yaw rate in the direction of the direction indicator's operation is above a threshold. For example, if the direction indicator is working in the right direction, it determines whether the steering angle or yaw rate in the right direction is above a threshold. This threshold is, for example, a preset value that can be arbitrarily set according to the structure of the vehicle 20.

[0032] The prohibition determination unit 12 determines the passage when the vehicle 20 is turning left or right and the steering angle or yaw rate in the direction of operation of the direction indicator exceeds a threshold (for example, referring to...). Figure 2 (b)) can be determined that vehicle 20 is in a left or right turning state. On the other hand, if the direction indicator is not working (there is no intention to turn left or right) or if the direction indicator is working but the steering angle and yaw rate in the direction of the direction indicator are not above the threshold, the prohibition determination unit 12 can determine that it is not a left or right turning state (it is a straight-going state).

[0033] Next, when the vehicle 20 is in a left or right turning state, the prohibition determination unit 12 determines entry into the prohibited road based on the vehicle 20 when the entry restriction sign recognition unit 11 identifies at least one entry restriction sign M near the entrance to the destination road R. For example, as Figure 2 As shown in (b), when vehicle 20 is making a left turn, there is one no-entry sign M1 and one-way traffic sign M2 located inside the shooting range A of the camera 1 ahead, and these signs can be identified respectively. In this case, the no-entry determination unit 12 can determine the entry of vehicle 20 into the no-entry road.

[0034] like Figure 3 As shown, when vehicle 20 enters the destination road R while traveling straight, the prohibition determination unit 12 determines whether there are entry restriction signs M on both sides of the destination road R. Thus, if vehicle 20 is traveling straight and the prohibition determination unit 12 only recognizes one entry restriction sign M on one side, it may not determine whether entry into the prohibited road is allowed.

[0035] If the vehicle 20 is determined to be entering a prohibited road, the prohibition determination unit 12 can alert the driver via the display 5 and the speaker 6. The prohibition determination unit 12 can alert the driver, for example, by activating an alert device connected to the display 5 and the speaker 6. The display 5 can show a warning message indicating a prohibited road, and the speaker 6 can emit a warning sound, for example, announcing a prohibited road.

[0036] In addition, such as Figure 2 As shown in (a), the attention reminder based on the prohibition-of-passage determination unit 12 can be given before turning left or right, or as shown in (a). Figure 2 (c) is shown and is performed after entering a road where passage is prohibited. Furthermore, this warning can be given after the vehicle 20 has passed the entry restriction sign M and moved a predetermined distance from the entry restriction sign M, following its entry into the road where passage is prohibited.

[0037] Next, refer to Figure 4 The processing of the passage restriction determination device 100 will be explained. Additionally, Figure 4 The processing shown is performed simultaneously with the start of the movement of vehicle 20.

[0038] like Figure 4 As shown, ECU10 determines whether the direction indicator of vehicle 20 is working (step S1). If the direction indicator is working (step S1: Yes), ECU10 determines whether the steering angle or yaw rate in the working direction of the direction indicator is above a threshold (step S2). If the direction indicator is not working (step S1: No), ECU10 determines that vehicle 20 is in a straight-ahead state and executes the processing in step S4 described later.

[0039] If the steering angle or yaw rate in the direction of operation of the direction indicator is above the threshold (step S2: Yes), ECU10 determines that vehicle 20 is turning left or right, and determines whether an entry restriction sign M is detected near the entrance to the destination road R at a distance less than the recognition distance threshold (step S3). If the steering angle or yaw rate in the direction of operation of the direction indicator is not above the threshold (step S2: No), ECU10 determines that vehicle 20 is going straight, and executes the processing described in step S4 below.

[0040] If an entry restriction sign M is detected near the entrance to the destination road R at a distance less than the recognition distance threshold (step S3: Yes), ECU 10 issues a warning to vehicle 20 regarding entering a prohibited road (step S5). Then, ECU 10 terminates the process. If no entry restriction sign M is detected near the entrance to the destination road R at a distance less than the recognition distance threshold (step S3: No), ECU 10 restarts the process.

[0041] As described above, if the direction indicator is not working (step S1: No) and the steering angle or yaw rate in the direction the direction indicator is working is not above a threshold (step S2: No), ECU 10 determines that vehicle 20 is traveling straight and executes step S4. In step S4, ECU 10 determines whether entry restriction signs M are detected on both sides of the road leading to the destination R. If entry restriction signs are detected on both sides of the road leading to the destination R (step S4: Yes), a warning is given in step S5. If entry restriction signs are not detected on both sides of the road leading to the destination R (step S4: No), ECU 10 restarts the process.

[0042] Furthermore, if the ECU 10 is operating the turn indicator and the steering angle or yaw rate in the direction the turn indicator is operating is less than a threshold, it can further determine whether the vehicle 20 has passed the entry restriction sign M after executing step S4. In this case, the ECU 10 will issue a warning in step S5 if the vehicle 20 has passed the entry restriction sign M, and will not issue a warning if the vehicle 20 has not passed the entry restriction sign M, and will restart the process. This can suppress the mistaken identification of entry into a prohibited straight-ahead road even if there is an intention to turn left or right, and can suppress unnecessary warnings. Moreover, if the ECU 10 is operating the turn indicator and the steering angle or yaw rate in the direction the turn indicator is operating is less than a threshold, it can also skip step S4 and restart the process. Even in this case, unnecessary warnings can be suppressed.

[0043] As explained above, in the prohibited passage determination device 100 of this embodiment, when the vehicle 20 is turning left or right, the ECU 10 determines whether the vehicle 20 is allowed to enter a prohibited passage road when it detects at least one entry restriction sign M near the entrance of the destination road R that the vehicle 20 is entering. Therefore, compared to the case where the ECU 10 determines whether the vehicle is allowed to enter a prohibited passage road only when it detects entry restriction signs on both sides of the destination road R while turning left or right, it is possible to appropriately determine whether a vehicle is allowed to enter a prohibited passage road while turning left or right.

[0044] That is, such as Figure 3 As shown, when vehicle 20 enters the destination road R while traveling straight, the entry restriction signs M on both sides of the road where passage is prohibited are typically identified. On the other hand, as... Figure 2 As shown in (b), when entering the destination road R while making a left or right turn, even if there are entry restriction signs M on both sides of the destination road R, it is possible that one of the entry restriction signs M is not within the shooting range A of the camera 1 ahead, and only one entry restriction sign M is recognized. In this case, it is possible that the entry into the prohibited road will not be determined, and no attention warning will be given.

[0045] On the other hand, in the prohibited passage determination device 100 according to this embodiment, when the vehicle 20 is in a left or right turn state, if at least one entry restriction sign M is identified near the entrance to the destination road R, entry into the prohibited road is determined based on the vehicle 20. Therefore, even if only one entry restriction sign M is identified when turning left or right, entry into the prohibited road can be appropriately determined. As a result, appropriate warnings can be given when someone mistakenly enters a prohibited road.

[0046] Furthermore, in the prohibition-of-passage determination device 100, it is possible to determine whether the vehicle 20 is turning left or right based on at least one of the following: whether the vehicle 20's direction indicator is working, the vehicle 20's steering angle, and the vehicle 20's yaw rate. Thus, it is possible to appropriately determine whether the vehicle 20 is turning left or right.

[0047] Furthermore, in the prohibition-of-passage determination device 100, when the vehicle 20's direction indicator is active, it can determine whether the vehicle 20 is turning left or right based on the vehicle 20's steering angle or yaw rate. This allows for a more appropriate determination of whether the vehicle 20 is turning left or right.

[0048] Furthermore, in the prohibition determination device 100, the entry restriction sign recognition unit 11 can identify entry restriction signs M that are at a distance of more than or equal to the recognition distance threshold from the vehicle 20, but instead identify entry restriction signs M that are at a distance of less than the recognition distance threshold from the vehicle 20. Therefore, even if the vehicle 20 enters a permitted road, it is possible to suppress erroneous determinations of entry as a prohibited road based on a distantly located entry restriction sign M.

[0049] Furthermore, in the no-entry determination device 100, when the vehicle 20 is traveling straight, if it is determined that there are entry restriction signs M on both sides of the destination road R, the ECU 10 can determine entry into the no-entry road based on the vehicle 20. That is, even if an entry restriction sign M is incorrectly identified while traveling straight, the ECU 10 can choose not to determine entry into the no-entry road. Thus, for example, even if the vehicle 20 enters a permitted entry road, unnecessary attention alerts can be suppressed. Therefore, the no-entry determination device 100 according to the above embodiment can suppress unnecessary attention alerts when traveling straight and appropriately determine the entry of a vehicle into a no-entry road when turning left or right.

[0050] The embodiments of the present invention have been described above, but the present invention is not limited to the above embodiments. The present invention can be implemented in various ways, including various modifications and improvements based on the above embodiments and the knowledge of those skilled in the art.

[0051] In the above embodiment, the prohibition determination unit 12 determines that the vehicle 20 is turning left or right when the vehicle 20 is turning left or right and the steering angle or yaw rate in the working direction of the direction indicator is above the threshold. However, the prohibition determination unit 12 may also determine that the vehicle 20 is turning left or right when both the steering angle and yaw rate are above the threshold without considering the working direction of the direction indicator.

[0052] Furthermore, the prohibition-of-passage determination unit 12 can also determine whether a vehicle is turning left or right based solely on whether the direction indicator is working. It can also determine whether a vehicle is turning left or right based solely on the steering angle, or solely on the yaw rate. That is, the prohibition-of-passage determination unit 12 can determine whether the vehicle 20 is turning left or right based on at least one of the direction indicator working, steering angle, and yaw rate. Furthermore, the prohibition-of-passage determination unit 12 can also determine whether a vehicle is turning left or right using methods other than the direction indicator working, steering angle, and yaw rate. The prohibition-of-passage determination unit 12 can also determine whether the vehicle 20 is turning left or right based on the image captured by the forward camera 1.

[0053] Furthermore, in the above embodiment, when the prohibition determination unit 12 determines that the vehicle 20 is entering a prohibited road, a warning is issued to the driver. However, the prohibition determination unit 12 may also choose not to issue a warning, for example, by controlling the vehicle 20 to slow down. When the vehicle 20 enters the destination road R in a straight-ahead state, the prohibition determination device 100 does not need to necessarily identify both entry restriction signs M on both sides of the road to determine the entry into a prohibited road. For example, when the vehicle 20 is in a straight-ahead state, the prohibition determination device 100 can determine the entry into a prohibited road based on the vehicle 20's location information and map information. The prohibition determination device 100 may also determine the entry into a prohibited road using other methods.

[0054] Symbol Explanation

[0055] 1-Front-facing camera, 2-Direction indicator sensor, 3-Steering angle sensor, 4-Yaw rate sensor, 10-ECU, 11-Entry restriction sign recognition unit, 12-No-passage determination unit, 20-Vehicle, 100-No-passage determination device, M-Entry restriction sign, M1-No-passage sign, M2-One-way sign, R-Destination road, A-Shooting range.

Claims

1. A device for determining whether passage is prohibited, characterized in that, have: The entry restriction sign recognition unit identifies entry restriction signs placed on the road that indicate that vehicles are prohibited from entering the road, based on the image captured by the camera in front of the vehicle. and The no-passage determination unit determines that the vehicle has entered a no-passage road when it identifies at least one of the entry restriction signs near the entrance of the road leading to the destination that the vehicle has entered while the vehicle is turning left or right.

2. The prohibition of passage determination device according to claim 1, characterized in that, Whether the vehicle is turning left or right is determined based on at least one of the following: whether the vehicle's direction indicator is working, the vehicle's steering angle, and the vehicle's yaw rate.

3. The prohibition of passage determination device according to claim 1, characterized in that, When the vehicle's direction indicator is working, determine whether the vehicle is turning left or right based on the vehicle's steering angle or yaw rate.

4. The prohibition determination device according to any one of claims 1 to 3, characterized in that, The entry restriction sign recognition unit does not recognize entry restriction signs that are more than or equal to the recognition distance threshold from the vehicle, but recognizes entry restriction signs that are less than the recognition distance threshold from the vehicle.

Images