Sign recognition method, sign recognition device, driving assistance method and driving assistance device

Abstract

To suppress erroneous recognition of a limited speed sign based on an image photographed in a situation where a branch line joins with a main line or a situation where a branch line branches from a main line.SOLUTION: A sign recognition method includes: determining whether a travel lane on which an own vehicle travels is a main line or a branch line (S1, S5); recognizing a limited speed sign from a photographed image in an area in front of the own vehicle by a camera mounted on the own vehicle (S3); determining whether one-way sign is jointly provided beside the limited speed sign by recognizing the one-way sign from the photographed image (S4); adopting a speed designated by the limited speed sign as speed information when the travel lane is determined to be the branch line and also the one-way sign is determined to be jointly provided beside the limited speed sign (S6); and not adopting the speed designated by the limited speed sign as speed information when the travel lane is determined to be the main line and also the on-way sign is determined to be jointly provided besides the limited speed sign (S7).SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a sign recognition method, a sign recognition device, a driving assistance method, and a driving assistance device. [Background technology]

[0002] Patent Document 1 describes a technology that recognizes speed limit signs from an image captured by a camera that captures the area ahead of the vehicle, detects speed limit information, and displays the speed limit information on a display device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-060456 Summary of the Invention [Problem to be solved by the invention]

[0004] When recognizing a speed limit sign from an image taken in a situation where a branch lane merges with a main lane or where a branch lane branches off from a main lane, there is a risk that a speed limit sign installed in a lane other than the lane in which the vehicle is traveling may be mistaken for a speed limit sign installed in the lane in which the vehicle is traveling. The present invention aims to prevent misidentification of speed limit signs based on images captured in situations where a branch lane merges with a main lane or where a branch lane branches off from a main lane. [Means for solving the problem]

[0005] In a sign recognition method according to one aspect of the present invention, a controller executes the following processes: determining whether the lane in which the vehicle is traveling is a main lane or a branch lane that is a merging lane or a branch lane; acquiring an image of the area in front of the vehicle using a camera mounted on the vehicle; recognizing a speed limit sign from the image; determining whether a one-way sign is attached to the speed limit sign by recognizing a one-way sign from the image; and, if it is determined that the lane in which the vehicle is traveling is a branch lane and that a one-way sign is attached to the speed limit sign, adopting the speed specified by the speed limit sign as speed information; and, if it is determined that the lane in which the vehicle is traveling is a main lane and that a one-way sign is attached to the speed limit sign, not adopting the speed specified by the speed limit sign as speed information. [Effects of the Invention]

[0006] According to the present invention, it is possible to suppress erroneous recognition of speed limit signs based on images captured in situations where a branch lane merges with a main lane or where a branch lane branches off from a main lane. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a diagram illustrating an example of a schematic configuration of a vehicle equipped with a driving assistance device according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a part of the input device of FIG. [Figure 3] FIG. 2 is a schematic diagram showing a state in which the host vehicle is traveling in a merging lane; [Figure 4] FIG. 2 is a schematic diagram showing a state in which the host vehicle is traveling on a main lane. [Figure 5] FIG. 2 is a schematic diagram showing a state in which the host vehicle is traveling on a branching lane; [Figure 6] FIG. 2 is a schematic diagram showing a state in which the host vehicle is traveling on a main lane. [Figure 7] 1 is a flowchart illustrating an example of a driving assistance method according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the drawings are schematic and may differ from the actual product. Furthermore, the embodiments of the present invention shown below are examples of devices and methods for embodying the technical concept of the present invention, and the technical concept of the present invention does not limit the structure, arrangement, etc. of component parts to those described below. The technical concept of the present invention can be modified in various ways within the technical scope defined by the claims.

[0009] (composition) 1 is a diagram showing an example of a schematic configuration of a vehicle equipped with a driving assistance device according to an embodiment. The driving assistance device 10 equipped in the vehicle 1 includes a sensor 11, a positioning device 12, a map database (map DB) 13, on-board equipment 14, a navigation system 15, a display device 16, an audio output device 17, an input device 18, a vehicle behavior control device 19, and a controller 20. These devices are connected to each other via, for example, a CAN (Controller Area Network) or other on-board LAN for transmitting and receiving information to and from each other.

[0010] The sensor 11 detects the traveling state of the host vehicle 1. For example, the sensor 11 includes cameras such as a front camera that captures an image in front of the host vehicle 1, a rear camera that captures an image behind the host vehicle 1, and a side camera that captures images on the left and right sides of the host vehicle 1. The sensor 11 also includes radars such as a front radar that detects obstacles in front of the host vehicle 1, a rear radar that detects obstacles behind the host vehicle 1, and a side radar that detects obstacles present on the left and right sides of the host vehicle 1. The sensor 11 also includes a vehicle speed sensor that detects the vehicle speed of the host vehicle 1.

[0011] The positioning device 12 includes a GPS unit, a gyro sensor, a vehicle speed sensor, etc. The positioning device 12 detects radio waves transmitted from multiple satellite communications using the GPS unit and periodically acquires position information of the vehicle 1. The positioning device 12 also detects the current position of the vehicle 1 based on the acquired position information of the vehicle 1, angle change information acquired from the gyro sensor, and vehicle speed acquired from the vehicle speed sensor.

[0012] The map database 13 is a memory that stores three-dimensional high-precision map information including position information of various facilities and specific points and is accessible from the controller 20. The three-dimensional high-precision map information is three-dimensional map information based on road shapes detected when a data acquisition vehicle travels on actual roads. The three-dimensional high-precision map information is map information that associates, as three-dimensional information, detailed and highly accurate position information such as curved roads and the magnitude of the curves (for example, curvature or curvature radius), road junctions, branching points, toll gates, and positions where the number of lanes decreases, with the map information.

[0013] The 3D high precision map information includes lane-level information that is more detailed than road-level information. Hereinafter, the lane-level information included in the 3D high precision map information may be referred to as "lane information." For example, the 3D high-precision map information includes, as lane information, information on lane nodes that indicate reference points on lane reference lines (e.g., the center line within a lane), information on lane links that indicate the section states of the lanes between the lane nodes, and lane type information. The lane type information may include information indicating whether each lane is a main lane, a merging lane that merges into the main lane, or a branch lane that branches off from the main lane. In the following description, a merging lane that merges into the main lane and a branch lane that branches off from the main lane may be collectively referred to as a "branch lane."

[0014] The three-dimensional high precision map information also includes speed limit information that indicates the speed limit of a road. The speed limit information may include, for example, information about speed limit signs (hereinafter referred to as "speed limit sign information"). The speed limit sign information includes the installation location of speed limit signs installed on roads and lanes and information on the speed limit. Furthermore, the speed limit sign information includes linking information that links (i.e., associates or associates) the roads and lanes on which the speed limit signs are installed (i.e., roads and lanes on which the speed limit is regulated or specified by the speed limit signs) with the speed limit signs. The speed limit information may also specify the speed limit based on speed information associated with a road or lane, for example.

[0015] The in-vehicle devices 14 are various devices mounted on the vehicle 1 and are operated by the driver. Examples of such in-vehicle devices include a steering wheel, an accelerator pedal, a brake pedal, a turn signal, wipers, lights, a horn, and other specific switches. The navigation system 15 acquires current position information of the vehicle 1 from the positioning device 12, and displays the position of the vehicle 1 on a display or the like, superimposing it on map information for navigation. The navigation system 15 also has a navigation function that, when a destination is set, sets a target driving route to the destination and guides the occupant along the set target driving route. This navigation function displays the target driving route on a map on the display and notifies the occupant of the target driving route by voice or the like. The target driving route set by the navigation system 15 is also used by a route driving assistance function provided in the controller 20. The route driving assistance function is a function that causes the vehicle 1 to autonomously drive to the destination based on the set target driving route.

[0016] The display device 16 includes various displays, such as a display provided in the navigation system 15, a display incorporated in a rearview mirror, a display incorporated in a meter section, a head-up display projected on the windshield, etc. The display device 16 notifies the occupant of various pieces of presented information under the control of the controller 20. The audio output device 17 is a device that outputs auditory information such as a speaker provided in the navigation system 15, a speaker of an audio device, a buzzer, etc. The audio output device 17 notifies the occupants of various pieces of presentation information under the control of the controller 20.

[0017] The input device 18 is, for example, a button switch that can be manually operated by the occupant to input information, a touch panel arranged on the display screen, or a microphone that can be used for voice input by the occupant. By operating the input device 18, the occupant can input setting information for the information presented by the display device 16 and the audio output device 17. 2 is a diagram showing a part of the input device 18 of this embodiment. The input device 18 may be, for example, a group of button switches arranged on the spokes of the steering wheel. The input device 18 is used to set the on / off status of the autonomous driving control function provided in the controller 20. The input device 18 includes a main switch 181, a resume / accelerate switch 182, a set / coast switch 183, a cancel switch 184, a distance adjustment switch 185, and a lane change assist switch 186.

[0018] The main switch 181 is a switch that turns the autonomous driving control function of the controller 20 on and off. The resume / accelerate switch 182 is a switch that sets the autonomous driving control function to resume at the set speed before the autonomous driving control function was turned off after being turned off, or that increases the set speed. The set / coast switch 183 is a switch that starts the autonomous driving control function. To start the autonomous driving control function, the set / coast switch 183 is pressed after the autonomous driving control function is turned on with the main switch 181. The set / coast switch 183 is also a switch that decreases the set speed. The cancel switch 184 is a switch that cancels the autonomous driving control function. The distance adjustment switch 185 is a switch that sets the distance from the preceding vehicle. The lane change assist switch 186 is a switch that instructs (agrees to) start a lane change when the controller 20 has confirmed with the occupant that they want to start a lane change. 2, a turn signal lever of a turn signal or other switches of the in-vehicle device 14 can also be used as the input device 18. For example, when the controller 20 suggests whether or not to automatically change lanes, if the occupant operates the turn signal lever, the vehicle will change lanes in the direction in which the turn signal lever was operated, rather than the suggested lane change.

[0019] The vehicle behavior control device 19 controls the vehicle behavior of the host vehicle 1. For example, when the host vehicle 1 travels at a constant speed at a set speed using the autonomous driving control function, the vehicle behavior control device 19 controls the operation of the drive mechanism and the brakes to achieve acceleration / deceleration and a travel speed so that the host vehicle 1 reaches the set speed. The vehicle behavior control device 19 also controls the operation of the drive mechanism and the brakes in the same way when the host vehicle 1 travels following a preceding vehicle using the autonomous driving control function. Note that the control of the operation of the drive mechanism includes the operation of the internal combustion engine in an engine vehicle and the operation of the traction motor in an electric vehicle. In addition, in a hybrid vehicle, it includes the torque distribution between the internal combustion engine and the traction motor. In addition, when the vehicle behavior control device 19 uses the autonomous driving control function to execute the lane keeping control, lane change assistance function, overtaking assistance function, or route driving assistance function described below, it controls the operation of the steering actuator in addition to controlling the operation of the drive mechanism and brakes, thereby performing steering control of the vehicle 1.

[0020] The controller 20 is one or more electronic control units (ECUs) for controlling the traveling of the host vehicle 1, and includes a processor 21 and peripheral components such as a storage device 22. The processor 21 may be, for example, a central processing unit (CPU) or a micro-processing unit (MPU). The storage device 22 may include a semiconductor storage device, a magnetic storage device, an optical storage device, or the like. The storage device 22 may include memories such as a register, a cache memory, a read-only memory (ROM) used as a main storage device, and a random access memory (RAM). The functions of the controller 20 described below are realized by, for example, the processor 21 executing a computer program stored in the storage device 22. The controller 20 may be formed of dedicated hardware for executing each of the information processes described below. For example, the controller 20 may include a functional logic circuit configured in a general-purpose semiconductor integrated circuit. For example, the controller 20 may include a programmable logic device (PLD) such as a field-programmable gate array (FPGA).

[0021] The controller 20 realizes a driving information acquisition function that acquires information about the driving state of the host vehicle 1 and an autonomous driving control function that autonomously controls the driving speed and / or steering of the host vehicle 1. The driving information acquisition function of the controller 20 is a function that acquires driving information about the driving state of the host vehicle 1. For example, the controller 20 may acquire image information of the outside of the vehicle captured by the front camera, rear camera, and side camera of the sensor 11 as the driving information. The controller 20 also acquires detection results by the front radar, rear radar, and side radar as the driving information. Furthermore, the controller 20 also acquires vehicle speed information of the host vehicle 1 detected by the vehicle speed sensor of the sensor 11 as the driving information.

[0022] Furthermore, the controller 20 acquires current position information of the vehicle 1 from the positioning device 12 as travel information. The controller 20 also acquires a set destination and a target travel route to the destination as travel information from the navigation system 15. The controller 20 also acquires position information such as curved roads and the magnitude of the curves (e.g., curvature or curvature radius), merging points, branching points, toll gates, and positions where the number of lanes decreases from the map database 13 as travel information. In addition, the controller 20 acquires operation information of the in-vehicle device 14 by the occupant from the in-vehicle device 14 as travel information.

[0023] The autonomous driving control function of the controller 20 is a function that autonomously controls the driving of the host vehicle 1 without relying on the operation of the occupant. The autonomous driving control function of the controller 20 includes an autonomous speed control function that autonomously controls the driving speed of the host vehicle 1, and an autonomous steering control function that autonomously controls the steering of the host vehicle 1. The autonomous speed control function and the autonomous steering control function of this embodiment will be described below. Note that in the present invention, it is sufficient for the autonomous driving control function to include only the autonomous speed control function, and the autonomous driving control function does not necessarily have to include the autonomous steering control function.

[0024] Autonomous speed control function The autonomous speed control function is a function that, when a preceding vehicle is detected, follows the preceding vehicle while performing vehicle distance control to maintain a distance according to vehicle speed, with the upper limit set to the speed limit of the road or lane on which the vehicle 1 is traveling, or the speed set by the occupant. On the other hand, if no preceding vehicle is detected, the autonomous speed control function performs constant speed driving at the speed limit of the road or lane on which the vehicle 1 is traveling, or at a speed set by the occupant. The former is also called inter-vehicle control, and the latter is called constant speed control. The controller 20 has a function of recognizing a speed limit sign from an image captured by the camera of the sensor 11 and setting the speed specified by the speed limit sign as the speed limit. The method of recognizing a speed limit sign will be described later.

[0025] The constant speed control is executed when the forward radar of the sensor 11 detects that there is no preceding vehicle ahead in the vehicle's own lane. In the constant speed control, the vehicle behavior control device 19 controls the operation of the engine, brakes, and other drive mechanisms while feeding back vehicle speed data from the vehicle speed sensor so as to maintain a set traveling speed. The vehicle distance control is executed when the forward radar of the sensor 11 detects the presence of a preceding vehicle ahead in the vehicle's own lane. In the vehicle distance control, the vehicle behavior control device 19 controls the operation of the engine, brakes, and other drive mechanisms while feeding back the vehicle distance data detected by the forward radar so as to maintain the set vehicle distance with the set travel speed as the upper limit.

[0026] <Autonomous steering control function> The autonomous steering control function is a function that executes steering control of the host vehicle 1 by controlling the operation of the steering actuator. The autonomous steering control function includes, for example, a lane keeping function, a lane change assist function, an overtaking assist function, and a route driving assist function. The lane keeping function is a function that assists the driver in steering the vehicle by controlling the steering actuator so that the vehicle stays near the center of the lane, for example.

[0027] When the set / coast switch 183 is pressed after the autonomous driving control function is turned on by the main switch 181, the controller 20 determines whether or not a predetermined autonomous driving control start condition is met. If it is determined that the autonomous driving control start condition is met, the controller 20 executes the lane keeping function of the autonomous steering control function. Alternatively, when the autonomous driving control start condition is met after the autonomous driving control function is turned on by the main switch 181, the controller 20 waits for the set / coast switch 183 to be pressed. When the set / coast switch 183 is pressed, the controller 20 starts the autonomous driving control and executes the lane keeping function of the autonomous steering control function.

[0028] Lane change assist function The lane change assist function turns on the turn signal when the driver operates the turn signal lever, and when predetermined lane change start conditions are met, it starts a lane change operation, which is a series of processes for changing lanes on an automobile.The lane change assist function determines whether the predetermined lane change start conditions are met based on various driving information acquired by the driving information acquisition function. The lane change assist function starts a lane change operation when the lane change start conditions are met. In the lane change operation, the vehicle 1 moves sideways to the adjacent lane, and when the movement to the adjacent lane is completed, the turn signal is turned off and the lane keeping function in the adjacent lane is started.

[0029] Overtaking Assist Function The overtaking assistance function presents overtaking information to the occupant on the display device 16 when there is a preceding vehicle slower than the host vehicle 1 ahead in the host vehicle's lane and predetermined overtaking suggestion conditions are met. Here, the overtaking information is information for suggesting to the occupant that they should overtake the preceding vehicle. When the occupant accepts the presentation of the overtaking information by operating the lane change assistance switch 186 of the input device 18 and predetermined overtaking start conditions are met, the overtaking assistance function turns on the turn signal and starts the above-mentioned lane change operation. The overtaking assistance function determines whether the overtaking suggestion conditions and the overtaking start conditions are met based on various driving information acquired by the driving information acquisition function.

[0030] Route driving support function The route driving assistance function presents route driving information on the display device 16 and suggests a lane change to the driving direction change point when a branch point, junction point, exit, toll gate, or other driving direction change point exists on the set target driving route, the distance to the driving direction change point is within a predetermined distance, and predetermined route driving suggestion conditions are met. In addition, the route driving assistance function starts a lane change operation when the lane change suggestion is accepted by operating the lane change assistance switch 186 and predetermined route driving start conditions are met. The route driving assistance function determines whether the route driving suggestion conditions and the route driving start conditions are met based on various driving information acquired by the driving information acquisition function.

[0031] How to recognize speed limit signs As described above, the controller 20 has the function of recognizing a speed limit sign from an image captured by the camera of the sensor 11. The recognition result of the speed limit sign can be used for driving assistance control of the host vehicle 1. For example, the controller 20 sets the speed specified by the speed limit sign as the speed limit, and executes driving assistance control of the host vehicle 1 based on the set speed limit. The driving assistance control may be, for example, autonomous speed control of the host vehicle 1 based on the speed limit. In this case, the controller 20 may perform constant speed control to maintain the traveling speed of the host vehicle 1 at the speed limit. Furthermore, for example, the controller 20 may perform inter-vehicle control using the speed limit as an upper limit speed.

[0032] The driving assistance control may also be, for example, presenting the speed limit to the driver. For example, the controller 20 may display the recognized speed limit on the display device 16. For example, the speed limit may be displayed on a display incorporated in the meter unit or a head-up display projected on the windshield. Furthermore, the controller 20 may output a voice message based on the recognized speed limit from the voice output device 17. For example, the controller 20 may output from the voice output device 17 a voice message informing the driver of the speed limit, a voice message informing the driver that the speed limit has changed, or a voice message informing the driver that the speed of the vehicle 1 exceeds the speed limit.

[0033] When recognizing a speed limit sign, the controller 20 acquires a captured image showing the area ahead of the vehicle 1 from the front camera of the on-board sensor 11 that captures an image of the area ahead of the vehicle 1. The up / down and left / right angles of the front camera are set or the angle of view is adjusted so that the camera can recognize at least speed limit signs installed in lanes adjacent to the vehicle's own lane. Multiple cameras that recognize speed limit signs may also be installed. The controller 20 recognizes the speed limit sign from the acquired captured image. For example, the controller 20 may recognize the speed limit sign from the captured image using known image recognition processing such as pattern matching. The controller also estimates the distance and direction from the vehicle 1 to the speed limit sign. The distance from the vehicle 1 to the speed limit sign can be estimated, for example, by detecting the coordinates of the ground position of the support structure of the speed limit sign from the captured image.

[0034] As described above, when recognizing a speed limit sign from an image captured in a situation where a branch lane (merging lane) merges into a main lane or a branch lane (branching lane) branches off from a main lane, there is a risk that a speed limit sign installed in a lane other than the lane in which the vehicle is traveling may be mistaken for a speed limit sign installed in the lane in which the vehicle is traveling. For example, if the lane in which the vehicle 1 is traveling is a main lane, there is a risk that a speed limit sign installed for a branch lane may be mistaken for a speed limit sign for the lane in which the vehicle 1 is traveling (ie, the main lane). Similarly, if the lane in which the vehicle 1 is traveling is a branch lane, there is a risk that a speed limit sign installed for the main lane may be mistaken for the speed limit sign for the lane in which the vehicle 1 is traveling (that is, the branch lane).

[0035] Therefore, the controller 20 recognizes one-way signs from the captured image to determine whether the speed limit sign recognized from the captured image is a sign installed for a branch lane (i.e., a sign that regulates or specifies the speed limit for the branch lane). The controller 20 may recognize one-way signs from the captured image by known image recognition processing such as pattern matching. The system then determines whether a one-way sign is attached to the speed limit sign, because a speed limit sign attached to a one-way sign can be assumed to be a sign installed for a branch lane.

[0036] Here, a one-way sign "located next to" a speed limit sign refers to a state in which a one-way sign is installed close to the speed limit sign, including, for example, a state in which the speed limit sign and the one-way sign are fixed to the same support (a pillar or beam) or a state in which the distance between the speed limit sign and the one-way sign is less than a predetermined distance. The controller 20 may determine whether a one-way sign is located next to a speed limit sign by performing image recognition processing on a captured image in which the speed limit sign and the one-way sign are recognized.

[0037] The controller 20 also determines whether the lane in which the vehicle 1 is traveling is a main lane or a branch lane. For example, the controller 20 may identify the lane in which the host vehicle 1 is currently traveling as the traveling lane in which the host vehicle 1 is traveling, based on the current position of the host vehicle 1 measured by the positioning device 12 and the three-dimensional high-precision map information stored in the map database 13. The controller 20 may determine whether the traveling lane in which the host vehicle 1 is traveling is a main lane or a branch lane, based on the lane type information of the lane identified as the traveling lane in which the host vehicle 1 is traveling.

[0038] For example, the controller 20 may determine whether the lane in which the vehicle 1 is traveling is a main lane or a branch lane based on the target driving route set by the navigation system 15 and the three-dimensional high-precision map information. For example, if the target driving route set by the navigation system 15 is a route that continues on a road with a main lane, the system may determine that the driving lane on which the host vehicle 1 is traveling is the main lane. On the other hand, if the target driving route set by the navigation system 15 is a route that proceeds from a road with a main lane to a branch lane, the system may determine that the driving lane on which the host vehicle 1 is traveling is a branch lane (a branch lane). Also, if the target driving route set by the navigation system 15 is a route that proceeds from a merging road to a road with a main lane, the system may determine that the driving lane on which the host vehicle 1 is traveling is a merging lane (a branch lane).

[0039] When the controller 20 determines that the lane in which the vehicle 1 is traveling is a branch lane and that a one-way sign is attached to the speed limit sign, the controller 20 determines that the speed limit sign recognized from the captured image is the speed limit sign installed in the lane in which the vehicle 1 is traveling, and adopts the speed specified by the speed limit sign as the speed information of the speed limit to be used for driving assistance control. For example, the controller 20 executes driving assistance control of the vehicle 1 based on the speed limit specified by the speed limit sign.

[0040] 3 and 4 are schematic diagrams showing a situation where merging lane L3 merges with main lanes L1 and L2 at merging section Sm. Point Pms is the merging start point where merging section Sm begins, and point Pme is the merging end point where merging section Sm ends. The speed limit sign Sg1 is installed in the merging section Sm, and even if the vehicle 1 is traveling in the merging lane L3 (see Figure 3) or the main lane L1 (see Figure 4), the speed limit sign Sg1 may be recognized from the image of the area ahead of the vehicle 1 captured by the forward camera of the sensor 11. When the controller 20 determines that the vehicle 1 is traveling in the merging lane L3 (see Figure 3) and that a one-way sign Sg2 is located next to the speed limit sign Sg1, the controller 20 adopts the speed specified by the speed limit sign Sg1 as the speed information for the speed limit and performs driving assistance control based on the adopted speed limit.

[0041] 5 and 6 are schematic diagrams showing a situation where branch lane L4 branches off from main lanes L1 and L2 at branch section Sj. Point Pjs is the branch start point where branch section Sj begins, and point Pje is the branch end point where branch section Sj ends. The speed limit sign Sg1 is installed in the branch section Sj, and even when the vehicle 1 is traveling in the branch lane L4 (see Figure 5) or the main lane L1 (see Figure 6), the speed limit sign Sg1 may be recognized from the captured image of the area ahead of the vehicle 1 taken by the forward camera of the sensor 11. When the controller 20 determines that the vehicle 1 is traveling in the branch lane L4 (see Figure 5) and that a one-way sign Sg2 is located next to the speed limit sign Sg1, the controller 20 adopts the speed specified by the speed limit sign Sg1 as the speed information for the speed limit, and performs driving assistance control based on the adopted speed limit.

[0042] 4 and 6, if the controller 20 determines that the lane in which the vehicle 1 is traveling is the main lane L1 and that a one-way sign Sg2 is attached to the speed limit sign Sg1, the controller 20 does not use the speed limit specified by the speed limit sign Sg1 recognized from the captured image as the speed information for the speed limit used in the driving assistance control. For example, the controller 20 does not use the speed limit specified by the speed limit sign Sg1 in the driving assistance control. In this case, for example, the controller 20 may continue to use the speed limit currently used for the driving assistance control for the driving assistance control. Alternatively, the controller 20 may obtain speed limit information for the lane in which the host vehicle 1 is traveling from the three-dimensional high-precision map information in the map database 13, and perform driving assistance control based on the speed limit indicated by the speed limit information.

[0043] If it is determined that the speed limit sign recognized from the captured image is not accompanied by a one-way sign, the controller 20 may determine whether the speed limit sign is an electronic sign and whether the speed specified by the speed limit sign is equal to or greater than a threshold speed. The controller 20 may determine whether the speed limit sign is an electronic sign from the captured image using a known image recognition process such as pattern matching. If it is determined that the lane in which the host vehicle 1 is traveling is a main lane, the speed limit sign is an electronic sign, and the speed specified by the speed limit sign is equal to or greater than a threshold speed, the speed specified by the speed limit sign may be used as speed information for the speed limit to be used for driving assistance control. For example, the controller 20 may execute driving assistance control for the host vehicle 1 based on the speed limit specified by the speed limit sign. The reason is that electronic speed limit signs are often installed on main lanes, and a minimum speed is set for main lanes of expressways and other motorway-only roads. Therefore, the threshold speed may be set to, for example, the minimum speed for main lanes. For example, in the case of a main lane of an expressway, the threshold speed may be 60 kilometers per hour.

[0044] If the speed limit sign recognized from the captured image is not an electronic sign, or if the speed specified by the speed limit sign is not above the threshold speed, the controller 20 may refer to the speed limit sign information in the three-dimensional high-precision map information in the map database 13 to determine whether the speed limit sign recognized from the captured image is linked to the lane in which the vehicle 1 is traveling in the three-dimensional high-precision map information. When the speed limit sign recognized from the captured image is associated with the lane in which the host vehicle 1 is traveling, the controller 20 may use the speed specified by the speed limit sign as speed information of the speed limit to be used for driving assistance control. For example, the controller 20 may execute driving assistance control of the host vehicle 1 based on the speed limit specified by the speed limit sign.

[0045] For example, the controller 20 calculates the installation position of the speed limit sign based on the distance and direction to the speed limit sign recognized from the captured image and the current position of the vehicle 1 measured by the positioning device 12. The controller 20 may acquire speed limit sign information for the speed limit sign recognized from the captured image by comparing the installation position of the speed limit sign recognized from the captured image with the installation position of the speed limit sign represented by the speed limit sign information in the three-dimensional high-precision map information. The controller 20 may determine whether the lane linked to the speed limit sign is the driving lane in which the vehicle 1 is traveling, based on the linking information of the speed limit sign information.

[0046] If the speed limit sign recognized from the captured image is not linked to the lane in which the vehicle 1 is traveling, the controller 20 may determine, based on the distance from the vehicle 1 to the speed limit sign, whether or not to adopt the speed specified by the speed limit sign as the speed limit information to be used for driving assistance control. For example, if the distance from the host vehicle 1 to the speed limit sign is less than or equal to a predetermined distance, the controller 20 adopts the speed specified by the speed limit sign. If the distance is greater than the predetermined distance, the controller 20 does not adopt the speed specified by the speed limit sign. In this case, for example, the controller 20 may continue to use the speed limit currently being used for driving assistance control for driving assistance control. Alternatively, the controller 20 may obtain speed limit information for the lane in which the host vehicle 1 is traveling from the three-dimensional high-precision map information in the map database 13, and perform driving assistance control based on the speed limit indicated by the speed limit information.

[0047] (operation) 1 is a flowchart illustrating an example of a driving assistance method according to an embodiment. In step S1, the controller 20 determines whether the lane in which the host vehicle 1 is traveling is a main lane or a branch lane. In step S2, the controller 20 acquires a captured image of the area in front of the host vehicle 1 from the front camera of the sensor 11. In step S3, the controller 20 recognizes a speed limit sign from the acquired captured image.

[0048] In step S4, the controller 20 determines whether a one-way sign is attached to the speed limit sign. If a one-way sign is attached to the speed limit sign (step S4: Y), the process proceeds to step S5. If a one-way sign is not attached to the speed limit sign (step S4: N), the process proceeds to step S8. In step S5, if the lane in which the vehicle 1 is traveling is a branch lane (step S5: Y), the process proceeds to step S6. If the lane in which the vehicle 1 is traveling is a main lane (step S5: N), the process proceeds to step S7.

[0049] In step S6, the controller 20 adopts the speed specified by the speed limit sign as the speed limit information to be used for driving assistance control. Then, the process proceeds to step S13. In step S7, the controller 20 does not adopt the speed specified by the speed limit sign as the speed limit information to be used for driving assistance control. Then, the process proceeds to step S13.

[0050] In step S8, controller 20 determines whether the speed limit sign is an electronic sign and the speed specified by the speed limit sign is equal to or greater than the threshold speed. If the speed limit sign is an electronic sign and the speed specified by the speed limit sign is equal to or greater than the threshold speed (step S8: Y), the process proceeds to step S9. If the speed limit sign is not an electronic sign or the speed specified by the speed limit sign is not equal to or greater than the threshold speed (step S8: N), the process proceeds to step S10. In step S9, if the lane in which the vehicle 1 is traveling is the main lane (step S9: Y), the process proceeds to step S6. If the lane in which the vehicle 1 is traveling is not the main lane (step S9: N), the process proceeds to step S10.

[0051] In step S10, the controller 20 determines whether the speed limit sign recognized from the captured image is linked to the lane in which the host vehicle 1 is traveling in the map database 13. If the speed limit sign is linked to the lane in which the host vehicle 1 is traveling (step S10: Y), the process proceeds to step S6. If the speed limit sign is not linked to the lane in which the host vehicle 1 is traveling (step S10: N), the process proceeds to step S11.

[0052] In step S11, the controller 20 determines whether the distance from the vehicle 1 to the speed limit sign is equal to or less than a predetermined distance. If the distance from the vehicle 1 to the speed limit sign is equal to or less than the predetermined distance (step S11: Y), the process proceeds to step S6. If the distance from the vehicle 1 to the speed limit sign is not equal to or less than the predetermined distance (step S11: N), the process proceeds to step S12. In step S12, the controller 20 does not adopt the speed specified by the speed limit sign as the speed limit information to be used for driving assistance control, and then the process proceeds to step S13.

[0053] In step S13, the controller 20 executes driving assistance control of the host vehicle 1 based on the set speed limit. If the speed specified by the speed limit sign is not adopted in step S7 or S12, the speed limit currently being used for driving assistance control may continue to be used for driving assistance control. Alternatively, the controller 20 may obtain speed limit information for the lane in which the host vehicle 1 is traveling from the three-dimensional high-precision map information in the map database 13, and execute driving assistance control based on the speed limit indicated by the speed limit information. The process then ends.

[0054] (Effects of the embodiment) (1) The controller 20 performs the following processes: determining whether the lane in which the vehicle 1 is traveling is a main lane or a branch lane that is a merging lane or a branch lane; acquiring an image of the area in front of the vehicle 1 using a camera mounted on the vehicle; recognizing a speed limit sign from the image; determining whether a one-way sign is attached to the speed limit sign by recognizing a one-way sign from the image; and, if it determines that the lane in which the vehicle is traveling is a branch lane and that a one-way sign is attached to the speed limit sign, adopting the speed specified by the speed limit sign as speed information; and, if it determines that the lane in which the vehicle is traveling is a main lane and that a one-way sign is attached to the speed limit sign, not adopting the speed specified by the speed limit sign as speed information. This makes it possible to prevent speed limit signs from being mistakenly recognized based on images captured in situations where a branch lane merges with a main lane or where a branch lane branches off from a main lane.

[0055] (2) When the controller 20 determines that a one-way sign is not attached to the speed limit sign, the controller 20 may execute the following process: determining whether the speed limit sign is an electronic sign and whether the speed specified by the speed limit sign is equal to or greater than a threshold speed; and when the controller 20 determines that the traveling lane is a main lane, the speed limit sign is an electronic sign, and the speed specified by the speed limit sign is equal to or greater than a threshold speed, adopting the speed specified by the speed limit sign as speed information. This makes it possible to prevent the speed limit sign from being mistakenly identified based on an image taken in a situation where a branch lane merges with a main lane or where a branch lane branches off from a main lane when a one-way sign is not attached to the speed limit sign.

[0056] (3) The controller 20 may execute driving assistance control of the host vehicle 1 based on the speed information adopted in (1) or (2) above. This makes it possible to prevent inappropriate driving assistance control from being performed based on a speed limit sign installed in a lane other than the lane in which the vehicle 1 is traveling, when a speed limit sign is recognized based on an image taken in a situation where a branch lane merges with a main lane or where a branch lane branches off from a main lane. [Explanation of symbols]

[0057] 1... Vehicle, 10... Driving assistance device, 11... Sensor, 12... Positioning device, 13... Map database, 14... In-vehicle equipment, 15... Navigation system, 16... Display device, 17... Audio output device, 18... Input device, 19... Vehicle behavior control device, 20... Controller

Claims

1. A process of determining whether the lane in which the vehicle is traveling is a main lane or a branch lane that is a merging lane or a branch lane; A process of acquiring a captured image of an area ahead of the host vehicle by a camera mounted on the host vehicle; A process of recognizing a speed limit sign from the captured image; a process of determining whether a one-way sign is present alongside the speed limit sign by recognizing a one-way sign from the captured image; a process of adopting the speed specified by the speed limit sign as speed information when it is determined that the travel lane is a branch lane and that a one-way sign is attached to the speed limit sign, and not adopting the speed specified by the speed limit sign as speed information when it is determined that the travel lane is a main lane and that a one-way sign is attached to the speed limit sign; to the controller, A sign recognition method characterized by:

2. The controller a process for determining whether the speed limit sign is an electronic sign and whether the speed specified by the speed limit sign is equal to or greater than a threshold speed when it is determined that a one-way sign is not attached to the speed limit sign; a process of adopting the speed specified by the speed limit sign as the speed information when it is determined that the travel lane is a main lane, the speed limit sign is an electronic sign, and the speed specified by the speed limit sign is equal to or greater than a threshold speed; 2. The sign recognition method according to claim 1, further comprising:

3. A driving assistance method, in which the controller assists the driving of the host vehicle based on the speed information adopted by the sign recognition method according to claim 1 or 2.

4. a camera mounted on the vehicle and configured to capture an image of an area in front of the vehicle; a controller that executes the following processes: determining whether the lane in which the vehicle is traveling is a main lane or a branch lane that is a merging lane or a branch lane; recognizing a speed limit sign from an image captured by the camera; determining whether a one-way sign is attached to the speed limit sign by recognizing a one-way sign from the image captured; and, if it is determined that the lane in which the vehicle is traveling is a branch lane and that a one-way sign is attached to the speed limit sign, adopting the speed specified by the speed limit sign as speed information; and, if it is determined that the lane in which the vehicle is traveling is a main lane and that a one-way sign is attached to the speed limit sign, not adopting the speed specified by the speed limit sign as the speed information; A sign recognition device comprising:

5. A driving assistance device comprising the sign recognition device according to claim 4, wherein the controller assists the driving of the host vehicle based on the adopted speed information.

Images