Speed limit determination device

Abstract

The present invention provides a speed limit determination device that recognizes an external environment from a camera image and acquires a speed limit of a road from a map database on the basis of a current position of a vehicle, and that is configured to include at least one computer. The at least one computer acquires a speed limit of a current position on a current travel path from the map database, acquires, from the map database, a speed limit of a read-ahead section set in front of the current position on the travel path, specifies a first speed limit which is the speed limit of the current position on the basis of recognized data of the external environment based on the camera image and the speed limit of the current position, specifies a second speed limit which is the speed limit of the read-ahead section on the basis of the recognized data of the external environment based on the camera image and the speed limit of the read-ahead section, outputs the first speed limit to an output device, and outputs the previously specified second speed limit to the output device when traveling in the read-ahead section.

Description

Speed ​​limit determination device

[0001] The present invention relates to a speed limit determination device using an in-vehicle camera and a map device.

[0002] Patent document 1 (JP 2021-157712 A) discloses a technology that compares the speed limit obtained by recognizing signs from camera images with the speed limit for the current location obtained from map data, and displays the obtained speed limit if they match, and then stops displaying the speed limit if they contradict each other.

[0003] Japanese Patent Application Laid-Open No. 2021-157712

[0004] In recent years, there has been a growing demand for highly accurate in-vehicle display of speed limit information for the current road. For example, the revised General Safety Regulation (GSR), a new European vehicle safety regulation, requires high accuracy for automatic speed control devices (Intelligent Speed ​​Assist: ISA), requiring constant speed limits regardless of road conditions, weather conditions, or time of day. Therefore, sign recognition using a camera alone cannot meet these requirements, and it is necessary to determine the speed limit for the road in conjunction with map information.

[0005] The technology in Patent Document 1 recognizes road signs from camera images and acquires speed limit data for the current road from map data. However, there are errors in positioning data from the Global Navigation Satellite System (GNSS), and depending on the timing of updates to map data for the current location acquired while moving, it is possible that at some point the speed limit displayed on the road signs in the camera image will not match the speed limit acquired from the map data.

[0006] An object of the present invention is to provide a speed limit determination device that can improve the accuracy of displaying a speed limit.

[0007] In order to achieve the above-mentioned object, the present invention provides a speed limit determination device comprising at least one computer that recognizes the outside world from images captured by an on-board camera and obtains the road speed limit from a map database based on the current position of the vehicle, wherein the at least one computer obtains the speed limit for the current position of the current driving path from the map database, obtains from the map database the speed limit for a pre-read section set ahead of the current position on the driving path from the map database, identifies a first speed limit that is the speed limit for the current position based on the external world recognition data based on the on-board camera image and the speed limit for the current position, identifies a second speed limit that is the speed limit for the pre-read section based on the external world recognition data based on the on-board camera image and the speed limit for the pre-read section, outputs the first speed limit to the output device, and outputs the previously determined second speed limit to the output device when traveling through the pre-read section.

[0008] According to the present invention, the accuracy of displaying speed limits can be improved.

[0009] FIG. 1 is a schematic diagram showing an example of a vehicle equipped with a speed limit recognition device according to a first embodiment of the present invention; FIG. 2 is a schematic diagram showing an example of the configuration of a speed limit recognition device according to a first embodiment of the present invention; FIG. 3 is a schematic diagram showing another example of the configuration of a speed limit recognition device according to a first embodiment of the present invention; FIG. 4 is a diagram showing an example of map data stored in a map database; FIG. 5 is an explanatory diagram showing the positional relationship between the current position of the vehicle and a look-ahead section in the first embodiment; FIG. 6 is an explanatory diagram showing the basic concept of an arbitration function; FIG. 7 is an explanatory diagram showing an example of an arbitration situation other than the example shown in FIG. 5; FIG. 7 is an explanatory diagram showing another example of an arbitration situation other than the example shown in FIG. 5; FIG. 8 is a flowchart showing an example of the processing procedure of a speed limit determination device according to a first embodiment of the present invention; and FIG. 9 is a flowchart showing an extracted main part of an example of the processing procedure of a speed limit determination device according to a second embodiment of the present invention.

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] 1 is a schematic diagram showing an example of a vehicle equipped with a speed limit recognition device according to a first embodiment of the present invention. The vehicle 1 is an automobile or the like, and is configured to include a vehicle control system 2, a communication device 3, a drive device 5, a recognition device 6, an output device 7, an input device 8, and a notification device 9.

[0012] The vehicle control system 2 includes an in-vehicle network 10 and at least one computer 11 connected to the in-vehicle network 10. The in-vehicle network 10 is, for example, a Controller Area Network (CAN) or a CAN with Flexible Data-rate (CANFD). The computer 11 may include, for example, an Electronic Control Unit (ECU) 11A and a Map Positioning Unit (MPU) 11B. The vehicle control system 2 is connected to a communication device 3, a drive device 5, a recognition device 6, an output device 7, an input device 8, and a notification device 9, and communicates with these devices to exchange data.

[0013] The communication device 3 is a device that performs wireless communication with devices outside the vehicle 1, and includes, for example, a GNSS (Global Navigation Satellite System) mobile station that acquires data on the current position and direction of the vehicle 1, and other communication devices.

[0014] The drive device 5 is a device, such as an actuator, that drives mechanical and electrical equipment (e.g., motor, engine, transmission, wheels, brakes, steering device, etc.) related to the operation of the vehicle (vehicle 1) in response to control commands from the vehicle control system 2.

[0015] The recognition device 6 is a device that acquires data related to the outside world of the vehicle 1 and the state of the vehicle 1. The recognition device 6 includes, for example, external sensors such as an on-board camera (monocular camera or stereo camera), radar, LIDAR, and ultrasonic sensors, dynamic system sensors that recognize the state of the vehicle 1 (operating state, acceleration, vehicle speed, etc.), and other sensors. In this embodiment, in the process described below of recognizing signs and the like by image processing, a stereo camera is used as the on-board camera 21 ( FIG. 2 ) to acquire distance information, but distance measurement is also possible with other recognition devices 6 such as a monocular camera or LIDAR.

[0016] The output device 7 is connected to the in-vehicle network 10 by wire or wirelessly, and displays or outputs message information (e.g., video or sound) necessary for the driver based on data received via the in-vehicle network 10. The output device 7 includes, for example, a liquid crystal display, a warning light, a speaker, and other notification devices.

[0017] The input device 8 is a device for generating input signals that the driver uses to input operational intentions and instructions to the vehicle control system 2, and includes, for example, a steering wheel, accelerator pedal, brake pedal, shift lever, buttons, levers, touch panels, and other operating devices.

[0018] The notification device 9 is a device that the vehicle 1 uses to notify the outside world (following vehicles, surrounding people, etc.) of the state of the vehicle 1. The notification device 9 includes, for example, a lamp, an LED (Light Emitting Diode), a speaker, and other notification devices.

[0019] -Speed ​​Limit Determination Device- Figure 2 is a schematic diagram showing one configuration example of a speed limit recognition device according to this embodiment, and Figure 3 is a schematic diagram showing another configuration example. The speed limit determination device 100 is configured to include at least one computer having an external environment recognition function F1, a map interpretation function F2, and an arbitration function F3.

[0020] The external environment recognition function F1 is a function for recognizing the external environment from an image captured by the vehicle-mounted camera 21. In this embodiment, the vehicle-mounted camera 21 is, for example, a stereo camera included in the recognition device 6 (FIG. 1).

[0021] The map interpretation function F2 is a function that acquires road speed limits from the map database DB based on the current position of the vehicle 1 (host vehicle) input from the GNSS mobile station (communication device 3). The MPU 11B uses the map interpretation function F2 to acquire host vehicle surroundings map data based on the current position of the vehicle 1. The MPU 11B may also be provided with a function that communicates with a server via the communication device 3 and updates the data in the map database DB.

[0022] The arbitration function F3 is a function that arbitrates between the recognition data obtained by the external recognition function F1 and the data obtained from the map database DB by the map interpretation function F2.

[0023] 2, the external environment recognition function F1 is installed in the computer of the in-vehicle camera 21, the map interpretation function F2 and the map database DB are installed in the MPU 11B, and the arbitration function F3 is installed in the ECU 11A. However, as in the example of FIG. 3, the location of each function and database can be changed as appropriate, such as by providing the arbitration function F3 in the computer of the in-vehicle camera 21. In some cases, a single computer may have the external environment recognition function F1, the map interpretation function F2, and the arbitration function F3.

[0024] The ECU 11A may include an ADAS-ECU (Advanced Driver Assistance Systems) and an AD-ECU (Autonomous Driving). The ADAS-ECU has a driving assistance function and a preventive safety function that implements vehicle speed limiting, brake assistance, etc. based on data from the recognition device 6 and the input device 8. The AD-ECU has an autonomous driving function for the vehicle 1.

[0025] -Map Data- Figure 4 shows an example of map data stored in the map database. In the map database DB, roads shown on the left side of the figure are stored in the format shown on the right side of the figure.

[0026] In the example shown in the figure, link information of the road network is expressed by a combination of nodes N and links L connecting the nodes N together.

[0027] Each node N is a node representing a road network such as an intersection, and includes data such as an ID and location coordinates. In addition, each node N may be associated with attribute data such as a connecting node ID, node type, number of connecting links, intersection name, etc. as needed.

[0028] A link L is a road section between two connected nodes N, and includes data such as an ID, traffic regulations such as speed limits, and the number of lanes. Other attribute data, such as the road administrator, road type, route number, link length, and administrative district code, may be associated with each link L as needed. For example, if a default speed limit that applies to an administrative district code or road type is associated with that district or road type, it is possible to read the default speed limit from attribute data such as the administrative district or road type for a link L that does not contain explicit speed limit data. In other words, some attribute data, such as the administrative district code or road type, can serve as implicit speed limit data.

[0029] In the example of FIG. 4, shape line segment data M representing road shapes (straight lines, curves, etc.) by connecting line segments is stored in the map database DB in association with location information along with link information of the road network.

[0030] 5 is an explanatory diagram of the positional relationship between the current position of the vehicle 1 (host vehicle) and the look-ahead section in the first embodiment. In the figure, an image of the time and distance required for sign recognition based on the image from the onboard camera 21 is shown to the right of the image, and an image of map data with the same scale is also shown.

[0031] As shown in Fig. 5, the look-ahead section P is a section along which the vehicle 1 will travel. For example, in Fig. 5, if a sign candidate S, such as a speed limit sign, is newly detected by the on-board camera 21 at a certain point during travel and recognition processing is initiated, and the position of the vehicle 1 at that time is defined as the current position O and the location of the sign candidate S is defined as a first point α, the look-ahead section P is set as a section on the current travel route that includes the first point α. Fig. 5 shows an example in which the section from the current position O to a second point β that is a predetermined distance beyond the first point α is set as the look-ahead section P.

[0032] However, there may be cases where a sign is not detected even after traveling a certain distance. In such a case, when vehicle 1 travels a preset distance without detecting a sign, speed limit determination device 100 sets look-ahead section P by applying a preset default value to distance L2 (described later) based on, for example, current position O at the time the set distance has been traveled.

[0033] The sign candidate S is, for example, data of a predetermined size and a specific shape (e.g., circle) that is presumed to be a road sign or the like, and that is stored as a model in the memory of the speed limit determination device 100 through pre-setting, learning, or the like. The position data of the first point α is measured by the on-board camera 21 at the time when the sign candidate S is detected within the field of view of the on-board camera 21. For example, the distance L1 from the current position O to the first point α may vary depending on the focal length, angle of view, resolution, etc. of the on-board camera 21. The distance L2 from the current position O to the third point γ, which is the end point of the look-ahead section P, is set based on the distance L1. Preferably, the distance L2 is set longer than the distance L1, and may be a value that varies depending on the distance L1, or a fixed value may be adopted.

[0034] Note that section R shown in Figure 5 is a section corresponding to the time distance required from the current position O to complete recognition of the sign candidate S. Section R is determined, for example, by the current traveling speed of the vehicle 1 and the time (e.g., a set time) required for the recognition process of the road sign candidate S. The faster the traveling speed, the longer section R becomes, and section A from the second point β where the recognition process of the sign candidate S is expected to be completed to the first point α where the sign candidate S is located becomes shorter. Conversely, the slower the traveling speed, the shorter section R becomes, and section A becomes longer or shorter. The optical conditions and external environment recognition processing capability of the on-board camera 21 are set so that the above-mentioned distance L1 is sufficiently longer than section R.

[0035] --Overview of Processing by Speed ​​Limit Determination Device-- In summary, the speed limit determination device 100 of this embodiment executes the following processing using at least one computer that constitutes it (in the example of FIG. 2, the computer of the vehicle-mounted camera 21, the ECU 11A, and the MPU 11B).

[0036] First, when a sign candidate S is detected by the on-board camera 21 while the vehicle 1 is traveling, a look-ahead section P is set ahead of the current position O on the current road as shown in Figure 5, and the computer of the on-board camera 21 starts processing to recognize the speed limit from the sign candidate S captured in the image from the on-board camera 21.

[0037] Next, the map interpretation function F2 of the MPU 11B obtains the speed limit for the current position O on the current driving route as pre-read data from the map database DB based on the current position O input from the GNSS mobile station.

[0038] Similarly, the map interpretation function F2 of the MPU 11B obtains the speed limit of the pre-read section P set ahead of the current position O on the current driving route as pre-read data from the map database DB before the vehicle 1 reaches the pre-read section P.

[0039] Then, the adjustment function F3 determines the first speed limit, which is the speed limit for the current position O, based on the recognition data of the sign candidate S based on the image from the in-vehicle camera 21 and the speed limit for the current position O obtained by the MPU 11B.

[0040] Similarly, the adjustment function F3 determines the second speed limit, which is the speed limit of the predicted section P, based on the recognition data (e.g., recognition data of the speed limit) of the sign candidate S based on the image from the in-vehicle camera 21 and the speed limit of the predicted section P obtained by MPU 11B.

[0041] In addition, when the map database DB does not contain data on the speed limit itself for the pre-read section P or the current position O obtained from the map database DB, if attribute data linked to the speed limit such as an administrative area code can be referenced, the implicit speed limit linked to such attribute data can be obtained instead.

[0042] The speed limit determination device 100 repeatedly executes these processes while driving, outputs the identified speed limit (first speed limit) for the current position O to the output device 7, and outputs the previously identified speed limit (second speed limit) for the pre-read section P to the output device 7 when driving through the pre-read section P thereafter.

[0043] At this time, at least one computer (such as ECU 11B) constituting speed limit determination device 100 has the function of recognizing the speed limit from signs captured in images from on-board camera 21, as described above, and uses arbitration function F3 to reconcile the speed limit recognized from images from on-board camera 21 with the speed limit acquired from map database DB, and after this arbitration, identifies the second speed limit and the first speed limit. The arbitration method will be described later.

[0044] 5, by ensuring that the distance L1 is longer than the section R, the arbitration process is completed by the time the vehicle 1 reaches the first point α where the candidate sign S (a recognized sign) is located. The speed limit (first speed limit) for the pre-read section P is output to the output device 7 based on the timing when the vehicle 1 passes the first point α (for example, at the time when the vehicle 1 passes the first point α, or at a set time or distance before or after this).

[0045] By the above processing of the speed limit determination device 100, for example, if the second speed limit output to the output device 7 in the processing one cycle before has not changed from the first speed limit specified in the current cycle, the display on the output device 7 will not change until moving from the current position O to the look-ahead section P. Conversely, if the second speed limit output to the output device 7 in the processing one cycle before is different from the first speed limit specified in the current cycle, the display on the output device 7 will be switched to the newly specified first speed limit during the time it takes to move from the current position O to the look-ahead section P.

[0046] Furthermore, if the first and second speed limits specified in the current cycle are the same, the display on the output device 7 does not change when traveling through the look-ahead section P. Conversely, if the first and second speed limits specified in the current cycle are different, the display on the output device 7 switches when traveling through the look-ahead section P.

[0047] 5, for example, shows a scene in which the vehicle is traveling on a road with a speed limit of 100 km / h, the speed limit of 100 km / h is displayed on the output device 7, and a sign candidate S is detected by the in-vehicle camera 21 at the current position O. Furthermore, the sign candidate S is a sign that displays a speed limit of 90 km / h, and in the map database DB, the speed limit changes from 100 km / h to 90 km / h at point δ corresponding to the first point α in the look-ahead section P that was set in response to the detection of the sign candidate S.

[0048] In this case, the speed limit determination device 100 performs a process of acquiring the speed limit from the map database DB while performing a process of recognizing the sign candidate S while the vehicle 1 is traveling through section R. While the vehicle 1 is traveling through section R, the sign candidate S is recognized as a sign indicating a speed limit of 90 km / h.

[0049] On the other hand, data on the speed limit of 100 km / h, the speed limit of 90 km / h, and the point δ where these speed limits change are obtained from the map database DB for the pre-read section P. The timing at which this data is obtained from the map database DB depends on the processing cycle of the MPU 11B (for example, a cycle of 0.5 s to 1.0 s), and it may be while the vehicle is traveling in section R or while the vehicle is traveling in section A.

[0050] Then, while the vehicle 1 is traveling through section A, as soon as the speed limit is acquired from the map database DB, the speed limit determination device 100 reconciles the speed limit recognized by the in-vehicle camera 21 with the speed limit acquired from the map database DB to identify the first speed limit and the second speed limit. In this example, the data from the in-vehicle camera 21 and the data from the map database DB match for both the first speed limit and the second speed limit.

[0051] The speed limit determination device 100 then displays the identified first speed limit on the output device 7, and thereafter, while traveling through the look-ahead section P, displays the second speed limit on the output device 7, for example, when the vehicle passes a sign (first point α) recognized by the in-vehicle camera 21. In this example, the speed limit displayed on the output device 7 remains unchanged at 100 km / h until the vehicle 1 reaches the first point α from the current position O, and then switches from 100 km / h to 90 km / h when the vehicle 1 passes the first point α.

[0052] - Arbitration - Figure 6 is an explanatory diagram of the basic concept of arbitration function F3. The arbitration function F3 of the computer (ECU 11A in the example of Figure 2, the computer of on-board camera 21 in the example of Figure 3) that constitutes speed limit determination device 100 classifies situations as follows, and performs arbitration processing between the speed limit recognized from the image of on-board camera 21 and the speed limit obtained from the map database DB depending on the situation. This arbitration processing is performed for at least the look-ahead section P, and preferably also for the current position O.

[0053] 5, when a speed limit is acquired from the map database DB and recognized from an image from the on-board camera 21, the speed limit that matches between the two is adopted as the speed limit to be displayed on the output device 7. It is assumed that the speed limit data acquired by the MPU 11B and the on-board camera 21 will match, but if there is a contradiction, the display of the speed limit on the output device 7 is controlled by a predetermined algorithm, for example, by hiding the speed limit on the output device 7, or by prioritizing the recognition result from the image when the external lighting conditions are good and the recognition result from the on-board camera 21 is highly reliable.

[0054] Next, for example, after passing through a speed limit area or while driving in an implicit speed limit area (certain urban areas, residential areas, etc.), if the speed limit is obtained from the map database DB but the speed limit cannot be recognized from the image from the in-vehicle camera 21, the speed limit obtained from the map database DB is adopted as the speed limit to be displayed on the output device 7.

[0055] Furthermore, for example, when the speed limit is not obtained from the map database DB but can only be recognized from the image of the in-vehicle camera 21, such as when traveling through an area where the speed limit is displayed electronically or in an area where temporary restrictions are imposed due to construction or the like, the speed limit obtained from the image of the in-vehicle camera 21 is adopted as the speed limit displayed on the output device 7. Generally, the speed limit displayed electronically is given priority. Therefore, when the speed limit is recognized from the camera image, even if the speed limit for that area can be obtained from the map database DB, the speed limit from the camera image can be adopted.

[0056] Finally, if no speed limit is obtained from the map database DB and no speed limit is recognized from the image of the in-vehicle camera 21, the default speed limit defined for the current area (the area to which the current position O belongs) is adopted as the speed limit to be displayed on the output device 7. Alternatively, the speed limit display on the output device 7 may be hidden.

[0057] --Application 1 to a case where there are multiple speed limits in the look-ahead section-- Figure 7 is an explanatory diagram showing an example of an arbitration situation that can be envisioned in addition to the example shown in Figure 5. In Figure 7, as in Figure 5, an image of the time and distance required for sign recognition based on the image from the on-board camera 21 is shown to the right of the image, along with an image of map data with the same scale. In Figure 7, elements that are the same as or correspond to those in Figure 5 are given the same reference numerals as in Figure 5.

[0058] As shown in Figure 7, roads that branch off or merge into each other, such as a main lane and a branch lane on a highway, may have different speed limits after the roads branch off or merge. In the example of Figure 7, when a sign candidate S, which is a branch lane sign, is detected while the vehicle 1 is traveling on the main lane, the branch lane branch point is included in the set pre-read section P, and multiple speed limit data (in this example, 100 km / h for the main lane and 40 km / h for the branch lane) can be pre-read from the map database DB. When passing this branch point, if the vehicle continues traveling on the main lane, it is desirable that the 100 km / h speed limit for the main lane continues to be displayed on the output device 7. However, if the vehicle enters the branch lane, it is desirable that the speed limit displayed on the output device 7 for the main lane be switched from 100 km / h for the main lane to 40 km / h for the branch lane in a timely manner.

[0059] Therefore, in this embodiment, at least one computer (e.g., ECU 11A) constituting the speed limit determination device 100, when multiple speed limits are obtained from the map database DB for the pre-read section P, uses its adjustment function F3 to selectively select a speed limit (second speed limit) for the pre-read section P from the multiple speed limits based on the behavior of the vehicle 1 (host vehicle).

[0060] Specifically, the behavior of the vehicle 1 refers to data related to the path of the vehicle 1, and may include, for example, operation signals from the input devices 8 ( FIG. 1 ), such as the steering wheel, brake pedal, accelerator pedal, and shift lever; the azimuth angle of the vehicle 1 and its changes acquired from the communication device 3 (GNSS); and the operation amounts of the drive devices 5, such as the motor, engine, transmission, wheels, brakes, and steering device. For example, by applying parameters such as the steering operation amount and brake operation amount to a predetermined judgment formula, if it is predicted that the vehicle will enter a branch line from the main line based on the steering amount, deceleration amount, etc., the speed limit determination device 100 assumes that the branch line is the look-ahead section P, performs the above-mentioned arbitration process for the branch line, and determines the speed limit (second speed limit) for the look-ahead section P. Conversely, if it is predicted that the vehicle will continue traveling on the main line, the speed limit determination device 100 assumes that the main line is the look-ahead section P, performs the above-mentioned arbitration process for the main line, and determines the speed limit (second speed limit) for the look-ahead section P.

[0061] The process of predicting whether the vehicle 1 will travel on the main line or on the branch line in the pre-read section P is not limited to an algorithm using a judgment formula, and may be performed by machine learning, for example. For example, machine learning is performed using the link shape and road shape of map data as input in addition to the data related to the above behavior, and outputting whether the vehicle 1 continued traveling on the main line or entered the branch line after passing the branch point, thereby making it possible to predict the traveling route in the pre-read section P from the behavior of the vehicle 1 and map data in a situation such as that shown in FIG. 5 .

[0062] Even if an error occurs in the above prediction, the speed limit determination device 100 also identifies the speed limit for the current position O during the arbitration process, so when the next arbitration process is executed soon, the speed limit display on the output device 7 will be correctly switched when the speed limit (first speed limit) for the road currently being traveled is identified.

[0063] --Application 2 to a case where there are multiple speed limits in the look-ahead section-- Figure 8 is an explanatory diagram showing another example of a possible arbitration situation in addition to the example shown in Figure 5. In Figure 8, as in Figure 5, an image of the map data, showing the time and distance required for sign recognition based on the image from the in-vehicle camera 21, is shown to the right of this image, along with a scaled image of the map data. In Figure 8, elements that are the same as or correspond to those in Figure 5 are given the same reference numerals as in Figure 5.

[0064] As shown in FIG. 8 , for example, the speed limit may differ depending on the lane even in the same section of the same road. In such a case, when a sign candidate S is detected while the vehicle 1 is traveling, multiple speed limit data (80 km / h, 90 km / h, and 100 km / h in the example of FIG. 8 ) for the set pre-read section P may be pre-read from the map database DB. However, as shown in FIG. 4 , the speed limit data in the map database DB is linked to the link L of each road, and the map database DB may not be able to determine which lane on the road each of the multiple speed limits linked to the same link L corresponds to. Even in such a case, it is desirable that when the vehicle 1 enters the pre-read section P, the speed limit specified for the traveling lane be alternatively displayed on the output device 7.

[0065] Therefore, at least one computer (e.g., ECU 11A) constituting the speed limit determination device 100, when multiple speed limits are obtained from the map database DB for the pre-read section P, selects a speed limit (second speed limit) for the pre-read section P from the multiple speed limits based on the image from the on-board camera 21 using its adjustment function F3.

[0066] Specifically, in the example of FIG. 8 , when three speed limits (80 km / h, 90 km / h, 100 km / h) are acquired from the map database DB for the same road in the pre-read section P, the speed limit determination device 100 compares them with three signs (100 km / h, 90 km / h, 80 km / h) recognized from the image of the on-board camera 21, identifies the lane in which the vehicle 1 is currently traveling, and performs the arbitration process described above. To give a simple example, of the three signs recognized by the on-board camera 21, the sign closest to the vehicle 1 (the 90 km / h sign in the example of FIG. 8 ) can be estimated as the sign related to the lane in which the vehicle 1 is currently traveling. Since the lane in which the vehicle is currently traveling is identified in this way, the 90 km / h speed limit recognized from the image can be compared with the three speed limits acquired from the map database DB. If 90 km / h is included in the three speed limits acquired from the map database DB, the speed limit for the pre-read section P is identified as 90 km / h.

[0067] The process of identifying the driving lane while driving is not limited to algorithms based on the distance to the sign, but can also apply an algorithm that recognizes white lines (lane dividing lines) depending on the conditions, such as when the field of view is open. In this case, the speed limit can be determined by reading the signs on the recognized lane.

[0068] Furthermore, even if an error occurs in the lane estimation, the speed limit determination device 100 also identifies the speed limit for the current position O during the arbitration process, so when the next arbitration process is executed soon, the speed limit display on the output device 7 will be correctly switched when the speed limit (first speed limit) for the road currently being traveled is identified.

[0069] - Flowchart - Figure 9 is a flowchart showing an example of the processing procedure of the speed limit determination device 100. The speed limit determination device 100 repeatedly executes the flow of Figure 9 while the vehicle 1 is traveling. In this embodiment, the processing of steps S11 to S17 in the figure is executed by a computer of the on-board camera 21 or a computer equipped with the arbitration function F3. The processing of steps S21 to S25 is executed by the MPU 11B. The processing of steps S11 to S17 and the processing of steps S21 to S25 are executed in parallel.

[0070] During operation, the on-board camera 21 constantly captures images of the outside world (such as the area in front of the vehicle 1) using its computer (step S11), and performs image processing of the captured images in real time (step S12). The on-board camera 21 detects, for example, potential signs S (FIG. 5) in the outside world through image processing (step S13), and recognizes the signs (step S14).

[0071] On the other hand, during operation, the MPU 11B sequentially acquires data on the position and direction of the vehicle 1 acquired by GNSS (step S21). For example, when a sign candidate S is detected in step S13, the MPU 11B acquires from the map database DB the speed limit for the current position O (step S22), supplementary data such as the speed limit, link shape, and road shape for the pre-read section P (step S23), and interpolated data for the speed limit, such as a default speed limit linked to an administrative area code (step S24), and outputs these data (step S25).

[0072] Based on the data output by the MPI 11B in step S25 and the recognition results obtained in step S14, the computer executing the arbitration function F3 extracts a first point α where a sign is present (i.e., a point where the speed limit may change in the pre-read section P). If multiple speed limits are obtained from the map database DB, the computer selects one speed limit as shown in Figures 7 and 8 (step S15). Furthermore, the computer executing the arbitration function F3 arbitrates the speed limit obtained from the image recognition and the speed limit obtained from the map data according to the situation, as described in Figure 6, and determines the speed limits (first speed limit and second speed limit) that are estimated to be true values ​​based on the current position O and the pre-read section P (step S16). The determined first speed limit, which is the speed limit for the current position O, is promptly reflected on the display of the output device 7. If the second speed limit, which is the speed limit for the pre-read section P, differs from the first speed limit, the speed limit display on the output device 7 is switched to the second speed limit, for example, when the vehicle passes through the first point α (step S17).

[0073] - Effect - (1) According to this embodiment, by using the image from the onboard camera 21 and the data from the map database DB, it is possible to accurately identify the current location and the speed limit data for the look-ahead section P. Furthermore, by looking ahead and identifying the speed limit for the look-ahead section P in advance using the camera image and the map database DB, it is possible to absorb time errors due to GNSS communication delays and the data output cycle of the MPU 11B, and to timely control the display of the speed limit on the output device 7, for example, at the timing of passing a speed sign. This improves the accuracy of the speed limit display and reduces the discomfort felt by the driver due to a time lag in the provision of information.

[0074] Furthermore, since the speed limit for the current position O is also specified, even if the predicted speed limit is incorrect, the speed limit display will soon be switched to the correct one based on the speed limit for the current position O that is subsequently specified.

[0075] In addition, since the speed limit data is identified accurately and in advance relative to the position of the vehicle 1, it is expected that this will also contribute to improving the accuracy of vehicle speed control in AD and ASDAS.

[0076] (2) Simply taking the AND of the speed limit recognized by the camera image and the speed limit obtained from the map database DB does not allow the computer to determine how to identify the speed limit unless matching data is obtained from both the image processing and the map data.

[0077] In contrast, in the case of this embodiment, cases are divided into those in which image recognition is effective, those in which map data is effective, etc., as shown in the example of FIG. 6, and data is selected according to the respective strengths of image recognition and map data, thereby enabling a more flexible response to various situations than when simply ANDing data, and also improving the accuracy of presenting speed limits.

[0078] In addition, the system can accurately present the speed limit to drivers even in areas where speed limit data can only be obtained through image processing or map data, such as areas without signs or areas where the speed limit is displayed electronically (areas where the speed limit changes).

[0079] Furthermore, by reconciling image recognition and map data, it is possible to respond to even more situations, such as when an electronic speed limit is recognized, it takes precedence over the speed limit in the map data, or when data on implicit speed regulations is available, the implicit speed limit is applied even if an explicit speed limit cannot be identified.

[0080] (3) As in the examples of Figures 7 and 8, even when there are multiple speed limits for the look-ahead section P, it is possible to select the speed limit to be subjected to arbitration processing based on the behavior of the vehicle 1 and the image recognition results. By providing the look-ahead section P and determining the speed limit in advance, for example, when approaching an interchange on a highway, if the prediction of whether to continue on the main lane or enter a branch lane is incorrect, the speed limit may be displayed incorrectly. However, by estimating the driving path from the behavior of the vehicle 1, it is possible to prevent the speed limit from being displayed incorrectly. However, it is also possible that the estimated driving path may be incorrect due to driver operation. However, because the speed limit for the current position O is confirmed each time arbitration processing is performed, the speed limit will soon be displayed correctly.

[0081] (4) Furthermore, even in areas where explicit speed limit data is not specified in the map database DB, the speed limit of the predicted section P can be interpolated by, for example, referring to a default speed limit linked to an administrative area code.

[0082] Second Embodiment Fig. 10 is a flowchart illustrating an example of a main part of a processing procedure of a speed limit determination device according to a second embodiment of the present invention. In this embodiment, the hardware configuration of the speed limit determination device 100 can be the same as that of the first embodiment. Furthermore, the speed limit determination device 100 of this embodiment can also execute the processing described in the first embodiment.

[0083] This embodiment differs from the first embodiment in that it adds a function for recognizing roads without white lines (demarcation lines). In this embodiment, at least one computer constituting the speed limit determination device 100, when recognizing the current road, executes a process of comparing the shape of the external environment recognized from the image of the onboard camera 21 with data on a map of the surrounding area of ​​the current position O stored in the map database DB. This process is executed, for example, in the image processing of step S12 in FIG. 9 . While the example of FIG. 8 mentioned an algorithm for recognizing white lines to recognize roads, roads come in various forms. For example, some roads are not clearly demarcated by white lines, such as roads with only a barrier, guardrails, or rows of trees on either side of the asphalt road surface, roads with only an asphalt road surface running through grass or sand, and roads with only a step between the left and right sidewalks. In this embodiment, an advantage of using a stereo camera suitable for shape measurement as the onboard camera 21 is that it is possible to recognize roads without such white lines.

[0084] In the case of a road surrounded on both sides by barriers or guardrails, or a road with a step between the road and the sidewalk, such elevated features can be detected as road edges from the parallax images of the stereo camera. Furthermore, because the stereo camera also acquires color information, even if there is no step at the road edge because the road is simply covered with asphalt on a grassy area, the road edge can be recognized by detecting the texture boundary. By detecting the road edge, the road is recognized, and by comparing this road with the surrounding map of the current location O in the map database DB, speed limit data can be obtained. Of course, if a road and a sign are recognized in the camera image, the speed limit determined by image recognition and the speed limit determined by map data are compared as described above (the example of "Speed ​​Information Available" and "Signs Available" in Figure 6).

[0085] The flowchart in Figure 10 shows a simplified road edge detection algorithm. The speed limit determination device 100 executes this process when the process in step S12 in Figure 9 requires recognition of the roadway and no white lines or other features are recognized. Specifically, when the flow in Figure 10 starts with the process in step S12, the speed limit determination device 100 creates a parallax image from the onboard camera 21 (step S31) and sets a processing region (e.g., the lower side of the screen) (step S32). Furthermore, the speed limit determination device 100 extracts boundary features by recognizing the shape and texture described above (step S33), detects the road edge of the roadway (step S34), and ends the flow in Figure 10 (proceeds to step S13).

[0086] In this embodiment, the same processing as in the first embodiment is executed, and therefore the same effects as in the first embodiment can be obtained. Additionally, there is an advantage in that the road can be recognized even when the road is obscured by white lines, etc. In this case, even in an environment where the road shape data in the map database DB is simple and the GNSS positioning accuracy is low, the road shape recognized from the camera image can be compared with the simple road shape in the map database DB to identify the road with high accuracy, and an accurate speed limit can be displayed.

[0087] (Modifications) The present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment has been specifically described in order to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to an embodiment having all of the described configurations.

[0088] Furthermore, a part of the configuration of one embodiment can be replaced with a part of the configuration of another embodiment. Also, a part of the configuration of another embodiment can be added to the configuration of one embodiment. Also, a part of the configuration of each embodiment can be deleted, and a part of another configuration can be added or replaced with a part of another configuration.

[0089] For example, in the processing of the speed limit determination device 100 that displays the speed limit on the output device 7, the on-board camera 21 may be used exclusively for external shape recognition to accurately identify the driving route by comparing it with map data, and the speed limit may be obtained only from the map database DB. In other words, rather than mediating between the on-board camera 21 and the map database DB, it is possible to configure the two to share roles.

[0090] 1...vehicle (own vehicle), 7...output device, 11A...ECU (computer), 11B...MPU (computer), 21...vehicle-mounted camera (computer), 100...speed limit determination device, A...first point (point where sign candidate exists), DB...map database, F3...arbitration function, O...current position, P...preview section, S...sign candidate

Claims

1. A speed limit determination device comprising at least one computer that recognizes the outside world from images captured by an on-board camera and obtains the road speed limit from a map database based on the vehicle's current position, wherein the at least one computer: obtains the speed limit for the current position of the current driving path from the map database; obtains the speed limit for the look-ahead section set ahead of the current position on the driving path from the map database; identifies a first speed limit that is the speed limit for the current position based on the outside world recognition data based on the images from the on-board camera and the speed limit for the current position; identifies a second speed limit that is the speed limit for the look-ahead section based on the outside world recognition data based on the images from the on-board camera and the speed limit for the look-ahead section; outputs the first speed limit to an output device, and outputs the second speed limit previously determined to the output device when traveling through the look-ahead section.

2. A speed limit determination device according to claim 1, wherein the at least one computer recognizes the speed limit from a sign captured in an image captured by the in-vehicle camera, and determines the second speed limit and the first speed limit by reconciling the speed limit recognized from the image captured by the in-vehicle camera with the speed limit obtained from the map database.

3. A speed limit determination device according to claim 2, wherein the look-ahead section is a section including a point on the current road where a sign candidate detected by the vehicle-mounted camera exists.

4. A speed limit determination device according to claim 3, wherein the at least one computer executes the arbitration process before the vehicle reaches a point where the sign candidate is located, and outputs the speed limit of the look-ahead section to the output device based on the timing at which the vehicle passes the point where the sign candidate is located.

5. A speed limit determination device according to claim 2, wherein the at least one computer: when the speed limit is obtained from the map database and the speed limit is recognized from the dash-mounted camera image, adopts the speed limit that matches between the two; when the speed limit is obtained from the map database and the speed limit is not recognized from the dash-mounted camera image, adopts the speed limit obtained from the map database; when the speed limit is not obtained from the map database and the speed limit is recognized from the dash-mounted camera image, adopts the speed limit obtained from the dash-mounted camera image; and when the speed limit is not obtained from the map database and the speed limit is not recognized from the dash-mounted camera image, adopts the default speed limit specified for the current area or is not displayed.

6. A speed limit determination device according to claim 1, wherein when multiple speed limits are obtained from the map database for the look-ahead section, the at least one computer selects the second speed limit alternatively from the multiple speed limits based on an image from the on-board camera or the behavior of the vehicle.

7. A speed limit determination device according to claim 1, wherein the speed limit of the look-ahead section obtained from the map database includes attribute data associated therewith.

8. A speed limit determination device according to claim 1, wherein said at least one computer recognizes the current driving route by comparing the shape of the outside world recognized from the image taken by said vehicle-mounted camera with the current position data in said map database.

Images