Traffic sign recognition device

Abstract

To determine whether a traffic sign recognized in the neighborhood of an intersection is to be applied to an own vehicle.SOLUTION: An intersection which a vehicle is to enter is identified based on map information. When a traffic sign is recognized from image information acquired in the periphery of the entering intersection, an inter-virtual surface angle is calculated, which indicates an angle formed by a first virtual surface orthogonal to an information display surface of the recognized traffic sign and a second virtual surface vertical to a first road on which the vehicle travels and including a reference line in a passage direction of the first road. An inter-road angle is calculated, which indicates an angle formed by a second road crossing with the first road at the entering intersection and a virtual road orthogonal to the first road at the entering intersection, based on information of a road crossing angle at the entering intersection. It is determined whether the recognized traffic sign is relevant to the traffic sign related to the first road based on the calculated inter-virtual surface angle and the calculated inter-road angle.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present disclosure relates to an apparatus for recognizing traffic signs provided on a road.

Background Art

[0002] Japanese Unexamined Patent Application Publication No. 2017-102665 discloses an apparatus for assisting the driving of a vehicle driver. This conventional apparatus acquires image information of a vehicle camera and determines whether a stop sign or a no-entry sign recognized from the image information applies to the host vehicle. Stop signs and no-entry signs tend to be installed on both sides of a road for the purpose of attracting attention. Utilizing this tendency, when two stop signs or no-entry signs are recognized on one side of the road, the conventional apparatus determines that the recognized signs do not apply to the host vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Determining whether various traffic signs recognized from camera image information apply to the host vehicle is not limited to some traffic signs such as stop signs and no-entry signs, and is important from the viewpoint of assisting the driving of a vehicle user. In particular, in an urban area, traffic signs tend to be installed in large numbers near intersections. Therefore, there is room for improvement to suppress a situation where traffic signs on a road intersecting with the road on which the host vehicle is traveling are erroneously determined to apply to the host vehicle.

[0005] One object of the present disclosure is to provide an apparatus capable of determining whether a traffic sign recognized near an intersection applies to the host vehicle.

Means for Solving the Problems

[0006] The present disclosure is an apparatus for recognizing traffic signs provided on a road and has the following features. The apparatus includes a camera, a storage device, and a processor. The camera is mounted on the vehicle. The camera also acquires image information in front of the vehicle. Map information including traffic signs provided on the road is stored in the storage device. The processor is configured to perform various information processing. Based on the map information, the processor identifies an intersection that the vehicle is scheduled to enter. When a traffic sign is recognized from the image information acquired around the intersection that the vehicle is scheduled to enter, the processor calculates an angle between virtual planes, which is the angle formed by a first virtual plane orthogonal to the information display surface of the recognized traffic sign and a second virtual plane perpendicular to a first road on which the vehicle travels and including a reference line in the traffic direction of the first road. The processor also calculates an angle between roads, which is the angle formed by a second road that intersects the first road at the intersection that the vehicle is scheduled to enter and a virtual road orthogonal to the first road at the intersection that the vehicle is scheduled to enter, based on the information on the road intersection angle at the intersection that the vehicle is scheduled to enter. Then, based on the calculated angle between virtual planes and the calculated angle between roads, the processor determines whether the recognized traffic sign corresponds to a traffic sign related to the first road.

Advantages of the Invention

[0007] According to the present disclosure, when a traffic sign is recognized from the image information acquired around an intersection that the vehicle is scheduled to enter, it is possible to determine whether the recognized traffic sign corresponds to a traffic sign related to the road on which the vehicle travels, based on the angle between virtual planes and the angle between roads calculated respectively.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, the structures and the like described in the following embodiments are not necessarily essential to the present disclosure, unless otherwise specifically stated or clearly specified by the principle.

[0010] 1. Example of System Configuration FIG. 1 is a conceptual diagram for explaining an overall configuration example of a driving support system including a traffic sign recognition device according to an embodiment. The driving support system 2 notifies information to the user US of the vehicle 1. The user US is, for example, an operator of the vehicle 1. The operator of the vehicle 1 may be a driver riding in the vehicle 1 or a remote operator remotely operating (remote driving, remote support) the vehicle 1. The vehicle 1 may be a manually driven vehicle or an autonomously driven vehicle. The driving support system 2 may be mounted on the vehicle 1 or may be included in a remote operator terminal operated by a remote operator. A part of the driving support system 2 may be included in a cloud server.

[0011] The driving support system 2 includes an information processing device 3, a display device 4, and a map DB (database) 5. The information processing device 3 includes at least one processor 31 and at least one storage device 32. The processor 31 executes various information processes. Examples of the processor 31 include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), ASICs (Application Specific Integrated Circuits), an FPGA (Field-Programmable Gate Array), and the like. The storage device 32 stores (stores) various information. Examples of the storage device 32 include a volatile memory, a non-volatile memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like.

[0012] The information processing device 3 acquires image information IMG in front of the traveling road DR of the vehicle 1 from a camera 11 mounted on the vehicle 1. The information processing device 3 also recognizes a traffic sign TS included in the image information IMG by analyzing the image information IMG. When the traffic sign TS is recognized, the information processing device 3 outputs a traffic rule TRR indicated by the recognized traffic sign TS (hereinafter, also referred to as "recognized sign RTS") to the display device 4.

[0013] The information processing device 3 acquires position information POS indicating the position (latitude and longitude) of the vehicle 1 from a GNSS (Global Navigation Satellite System) sensor 12 mounted on the vehicle 1. The information processing device 3 also specifies the traveling road DR based on the history of the position information POS and the map information MAP stored in the map DB 5. The information processing device 3 further specifies an intersection that the vehicle 1 is scheduled to enter based on the history of the position information POS and the map information MAP.

[0014] The display device 4 displays the traffic rule TRR received from the information processing device 3. The display device 4 may be a display mounted on the vehicle 1 or a head-up display (HUD), or may be a display of a remote operator terminal. The traffic rule TRR is information on traffic rules regarding the driving road DR. Examples of the traffic rule TRR include an image of a traffic sign TS, an icon of the traffic sign TS, or the content indicated by the traffic sign TS.

[0015] The map DB5 is formed in a predetermined storage device (for example, a non-volatile recording medium such as a hard disk or a flash memory). The map DB5 stores map information MAP. The map information MAP includes a general navigation map. Examples of the information included in the navigation map include information on the position of a road, information on the shape of the road (for example, the type of a curve or a straight line), and information on the position of an intersection and a structure. As information specific to the present disclosure, the map information MAP includes information on a road intersection angle θ indicating an angle formed by two roads intersecting at an intersection CP.

[0016] Note that examples of the components of the recognition device according to the embodiment include the above-described camera 11, GNSS sensor 12, information processing device 3, and map DB5.

[0017] 2. Features of the Embodiment FIG. 2 is a diagram for explaining the focus of the embodiment. In FIG. 2, roads R1 and R2 are drawn. The roads R1 and R2 intersect at an intersection CP. The road R1 is also the driving road DR of the vehicle 1. The road R1 as the driving road DR is an example of the "first road" of the present disclosure, and the road R2 intersecting with the driving road DR is an example of the "second road" of the present disclosure.

[0018] Traffic signs TS1 and TS2 are installed at the intersection CP. In a general intersection, the road intersection angle θ is designed to be 90° or an angle close to it (for example, 80°), but the road intersection angle θ between road R1 and road R2 is an acute angle (for example, 30°). When the road intersection angle θ is a narrow angle, the traffic sign related to road R1 (that is, traffic sign TS1) and the traffic sign related to road R2 (that is, traffic sign TS2) may be recognized from the image information IMG of the same vehicle. Then, there may be incorrect information in the information output from the display device 4 as the traffic rule TRR.

[0019] Therefore, in the embodiment, when a traffic sign TS is recognized at the intersection where the vehicle 1 is scheduled to enter, it is determined whether the recognized sign RTS corresponds to a traffic sign related to the driving road RD (hereinafter also referred to as the "target sign"). FIG. 3 is a diagram for explaining the outline of the determination method of the recognized sign RTS. The same intersection as the intersection CP shown in FIG. 2 is drawn in FIG. 3. This intersection CP is the intersection where the vehicle 1 is scheduled to enter.

[0020] In the determination of the recognized sign RTS, first, the inter-road angle θ* is calculated. The inter-road angle θ* indicates the angle formed by road R2 and the virtual road R3. The virtual road R3 is set as a road orthogonal to road R1 at the intersection CP. The road intersection angle θ formed by road R1 and road R2 is known from the map information MAP. Therefore, the inter-road angle θ* can be easily calculated using the road intersection angle θ (θ* = 90° - θ).

[0021] In the determination of the recognized sign RTS, in addition, the inter-virtual-plane angle φ is calculated. The inter-virtual-plane angle φ indicates the angle formed by the virtual plane VS1 and the virtual plane VS2. The virtual plane VS1 is an example of the "first virtual plane" of the present disclosure, and the virtual plane VS2 is an example of the "second virtual plane" of the present disclosure.

[0022] The virtual plane VS1 is calculated using the image of the traffic sign TS recognized from the image information IMG. The recognition of the traffic sign TS is performed by using an image recognition AI (Artificial Intelligence). The image recognition AI is pre-generated through a learning method such as deep learning. The image recognition AI is trained, for example, to detect various traffic signs TS from the image information IMG and further extract the images of the detected traffic signs TS. When the image of the traffic sign TS is recognized, the virtual plane VS1 is calculated as a plane orthogonal to the information display surface DS of the recognized sign RTS. The calculation of the virtual plane VS1 is performed, for example, by using an image conversion model to generate a depth image of the information display surface DS from the traffic sign TS.

[0023] The virtual plane VS2 includes the reference line RF (for example, the center line of the road R1) in the traffic direction of the driving road RD (that is, the road R1) and is a plane perpendicular to the driving road RD. The virtual plane VS2 is calculated using the reference line RF. The reference line RF is estimated, for example, from the lane lines of the road R1 recognized from the image information IMG.

[0024] In the example shown in FIG. 3, the traffic signs TS1 and TS2 are recognized. The virtual plane angle φ1 between the virtual planes of the traffic sign TS1 is calculated as the angle formed by the virtual plane VS2 and the virtual plane VS1 orthogonal to the information display surface DS1 of the recognized sign RTS1. The virtual plane angle φ2 between the virtual planes of the traffic sign TS2 is calculated as the angle formed by the virtual plane VS2 and the virtual plane VS1 orthogonal to the information display surface DS2 of the recognized sign RTS2.

[0025] Here, the sign of the virtual plane angle φ is negative when the left end LE of the information display surface DS is located closer to the vehicle 1 than the right end RE, and positive when the right end RE is located closer to the vehicle 1 than the left end LE. In the example shown in FIG. 3, since the left end LE of the information display surface DS1 is located closer to the vehicle 1 than the right end RE, the sign of the virtual plane angle φ1 is negative. On the other hand, since the right end RE of the information display surface DS2 is located closer to the vehicle 1 than the left end LE, the sign of the virtual plane angle φ2 is positive.

[0026] When the angle θ* between roads and the angle φ between virtual planes are calculated, the difference D (= θ* - φ) between the two is calculated. And when the difference D is less than or equal to a predetermined acute angle (for example, 30°), it is determined that the recognition sign RTS corresponds to the target sign. On the other hand, when the difference D is not less than or equal to the predetermined acute angle, it is determined that the recognition sign RTS does not correspond to the target sign. For example, let the angles (θ*, φ1, φ2) = (60, 30, -30). Then, the value of the difference D1 (= θ* - φ1) is 30°, and the value of the difference D2 (= θ* - φ2) is 90°.

[0027] According to the determination method described above, it is determined that the recognition sign RTS1 corresponds to the target sign, and it is determined that the recognition sign RTS2 corresponds to the target sign. This determination result is consistent with the premise of the embodiment that the traffic sign TS1 is a traffic sign related to the road R1 (that is, the driving road DR), and the traffic sign TS2 is a traffic sign related to the road R2. Therefore, according to the embodiment of calculating the angle θ* between roads and the angle φ between virtual planes, it is possible to determine whether the recognition sign RTS corresponds to the target sign.

[0028] 3. Information Processing Example FIG. 4 is a flowchart showing a processing example particularly related to the embodiment. The flowchart shown in FIG. 4 is repeatedly executed by the processor 31 shown in FIG. 1 at a predetermined control cycle.

[0029] In the routine shown in FIG. 4, first, the intersection that the vehicle 1 is scheduled to enter is specified (step S11). The intersection that the vehicle is scheduled to enter is specified based on the position information POS and the map information MAP. For example, the current position and the driving road DR of the vehicle 1 are specified from the history of the position information POS. Subsequently, by referring to the map information MAP using the current position and the driving road DR, the intersection that the vehicle is scheduled to enter is specified.

[0030] Following the processing of step S11, it is determined whether the recognition sign RTS has been acquired (step S12). The recognition sign RTS is acquired when a traffic sign TS is recognized from the image information IMG. That is, when the traffic sign TS has not been recognized, the recognition sign RTS is not acquired.

[0031] When the determination result in step S12 is affirmative, the angle φ between virtual planes and the angle θ* between roads are calculated (steps S13 and S14). The angle φ between virtual planes is calculated as the angle formed by virtual plane VS1 and virtual plane VS2. Virtual plane VS1 is calculated using the image of recognition sign RTS. Virtual plane VS2 is calculated using reference line RF. The angle θ* between roads is calculated using the road intersection angle θ (θ* = 90° - θ).

[0032] Subsequent to the process of step S14, it is determined whether or not the angle θ* between roads is 30° or more (step S15). As already explained, the angle θ* between roads indicates the angle formed by road R2 and virtual road R3. Therefore, the fact that the angle θ* between roads is not 30° or more means that the road intersection angle θ between road R2 and the driving road DR (that is, road R1) is 60° or more. When the road intersection angle θ is 60° or more, there is a high possibility that the problems described from the perspective of the embodiment do not occur. Therefore, when the determination result in step S15 is negative, the recognition sign RTS is determined to correspond to the target sign (step S17).

[0033] When the determination result in step S15 is affirmative, it is determined whether or not the difference D (= θ* - φ) is 30° or less (step S16). And when the determination result in step S16 is affirmative, the recognition sign RTS is determined to correspond to the target sign (step S17). On the other hand, when the determination result in step S16 is negative, the recognition sign RTS is determined not to correspond to the target sign (step S18). Incidentally, when the process of step S17 is performed, the recognition sign RTS is used for the generation of traffic rule TRR. On the other hand, when the process of step S18 is performed, the recognition sign RTS is rejected.

Explanation of Signs

[0034] 1…Vehicle, 11…Camera, 12…GNSS sensor, 2…Driving assistance system, 3…Information processing device, 4…Display device, 5…Map DB, 31…Processor, 32…Memory device, θ…Road intersection angle, θ*…Angle between roads, φ, φ1, φ2…Angle between virtual planes, R1, R2…Roads, R3…Virtual road, CR…Intersection, D…Difference, DS, DS1, DS2…Information display surface, DR…Driving road, LE…Left end, RE…Right end, TS, TS1, TS2…Traffic signs, US…User, VS1, VS2…Virtual planes, IMG…Image information, RTS, RTS1, RTS2…Recognized signs, MAP…Map information, POS…Position information, TRR…Traffic rules

Claims

1. An apparatus for recognizing traffic signs provided on a road, comprising: a camera mounted on a vehicle for acquiring image information in front of the vehicle; a storage device storing map information including information on a road intersection angle indicating an angle formed by two intersecting roads at an intersection; a processor for performing various information processing; wherein the processor identifies an intersection where the vehicle is scheduled to enter based on the map information, when a traffic sign is recognized from the image information acquired around the intersection where the vehicle is scheduled to enter, calculates a virtual surface angle between a first virtual surface orthogonal to an information display surface of the recognized traffic sign and a second virtual surface perpendicular to a first road on which the vehicle travels and including a reference line in a traffic direction of the first road, calculates a road angle between a second road intersecting the first road at the intersection where the vehicle is scheduled to enter and a virtual road perpendicular to the first road at the intersection where the vehicle is scheduled to enter based on the information on the road intersection angle at the intersection where the vehicle is scheduled to enter, and determines whether the recognized traffic sign corresponds to a traffic sign related to the first road based on the calculated virtual surface angle and the calculated road angle. A traffic sign recognition apparatus characterized by the above.

2. The apparatus according to claim 1, wherein the processor, in determining the recognized traffic sign, determines whether the calculated road angle is 30° or more, and when it is determined that the calculated road angle is less than 30°, determines that the recognized traffic sign corresponds to a traffic sign related to the first road. A traffic sign recognition apparatus characterized by the above.

3. The apparatus according to claim 1 or 2, wherein the sign of the virtual surface angle is negative when the left end of the information display surface is located closer to the vehicle than the right end of the information display surface, and positive when the right end is located closer to the vehicle than the left end, and the processor, in determining the recognized traffic sign, determines whether the calculated road angle is 30° or more, when it is determined that the calculated road angle is 30° or more, determines whether the difference between the road angle and the virtual surface angle is 30° or less, and when it is determined that the difference is 30° or less, determines that the recognized traffic sign corresponds to a traffic sign related to the first road. A traffic sign recognition apparatus characterized by the above.

4. The apparatus according to claim 3, wherein when it is determined that the difference is not 30° or more, it is determined that the recognized traffic sign does not correspond to the traffic sign related to the first road A traffic sign recognition apparatus characterized by the above.

Images