Driving assistance method and driving assistance device

Abstract

The present invention is a computer-based driving assistance method that involves: detecting a regulatory sign that is placed ahead of an ego vehicle and that regulates at least one of a specific road, vehicle, and time period; recognizing regulatory content from the detected regulatory sign; recognizing, from the regulatory content, first regulatory content that applies to the ego vehicle and second regulatory content that applies to other vehicles around the ego vehicle; and notifying, if it is determined that the first regulatory content and the second regulatory content are different, a driver that regulations applying to the ego vehicle and the other vehicles are different.

Description

Driving assistance method and driving assistance apparatus

[0001] The present invention relates to a driving assistance method and a driving assistance apparatus.

[0002] Patent Document 1 discloses a technology that notifies a driver that a regulatory sign installed on a road or the like has been updated since the driver traveled the road in the past.

[0003] Japanese Patent No. 710795

[0004] Incidentally, some regulatory signs indicate regulatory contents for different vehicles, and it may be difficult to determine whether the regulation applies to the own vehicle. The present invention has been made to solve the above problem, and an object of the present invention is to provide a driving assistance method and a driving assistance apparatus that can determine whether a regulation applies to the own vehicle when a regulatory sign indicates regulatory contents for different vehicles.

[0005] The present invention is a driving assistance method implemented by a computer, the method comprising: detecting a regulatory sign disposed in front of the own vehicle that regulates at least one of a specific road, a specific vehicle type, and a specific time zone; recognizing regulatory content from the detected regulatory sign; recognizing, from the regulatory content, first regulatory content that applies to the own vehicle and second regulatory content that applies to other vehicles around the own vehicle; and notifying the driver that the applicable regulations differ between the own vehicle and the other vehicles when it is determined that the first regulatory content and the second regulatory content are different.

[0006] According to the present invention, when a regulatory sign indicates regulatory contents for different vehicles, it is possible to determine whether the regulation applies to the own vehicle.

[0007] FIG. 1 is a block diagram showing a hardware configuration of a vehicle. FIG. 2 is a block diagram showing a software configuration of a vehicle. FIGS. 3 and 4 are examples of regulatory signs. FIG. 5 is a flowchart showing an example of the driving assistance method.

[0008] Hereinafter, an autonomous driving vehicle including a driving assistance method and a driving assistance apparatus according to an embodiment of the present invention will be described with reference to the drawings.

[0009] <1. Vehicle Hardware Configuration> Figure 1 is a block diagram showing the vehicle hardware configuration. As shown in Figure 1, the vehicle 2 comprises an external environment acquisition unit 21, a vehicle control unit 22, a storage unit 23, a drive unit 24, a display unit 25, a speaker 26, a transmitting / receiving unit 27, and a navigation device 28. In this embodiment, the vehicle control unit 22 corresponds to the driver assistance device of this disclosure.

[0010] The external environment acquisition unit 21 includes at least a shooting unit 211, a distance detection unit 212, a position detection unit 213, a direction detection unit 214, and an object detection unit 215, and acquires information regarding the position and orientation of the vehicle 2, captured images (including video) of the area around the vehicle 2, and obstacles around the vehicle 2.

[0011] The imaging units 211 are provided in multiple locations and capture images of the area around the vehicle 2, including the front, rear, and sides of the vehicle 2, and output them as image data. These units are, for example, CCD cameras, CMOS cameras, or HD cameras. The imaging units 211 continuously capture images of the area around the vehicle 2 and output the captured images to the display unit 25 as real-time video showing the area around the vehicle 2.

[0012] The distance detection unit 212 measures the distance to objects in front of and around the vehicle 2, and can use various sensors such as a laser rangefinder or an ultrasonic sensor. Examples of objects to be detected include pedestrians, bicycles, motorcycles, automobiles, and obstacles on the road or sidewalk (construction sites, traffic control materials, median strips, traffic signals, commercial facilities, residential facilities, utility poles, signs, trees, etc.). The state of the objects to be detected is not particularly limited, and they can be detected whether they are moving or stationary.

[0013] The position detection unit 213 detects the current position of the vehicle 2, and can use, for example, a GPS device. The information regarding the current position of the vehicle 2 detected by the position detection unit 213 is output to the vehicle control unit 22.

[0014] The direction detection unit 214 detects the direction of travel of the vehicle 2 (the direction the vehicle is facing), and for example, a gyro sensor can be used.

[0015] The object detection unit 215 detects moving and stationary objects around the vehicle (in front, to the side, and behind), and can use, for example, a camera, laser radar, millimeter-wave radar, optical sensor, or infrared sensor. Examples of objects to be detected include various types of equipment, pedestrians, bicycles, motorcycles, automobiles, and obstacles on the road or sidewalk (construction sites, traffic control materials, median strips, traffic signals, commercial facilities, residential facilities, signs, utility poles, trees, etc.). The state of the object to be detected is not particularly limited, and it can detect objects that are moving or stationary. In addition, multiple object detection units 215 are provided to detect objects in front of, to the side of, and behind the vehicle 2.

[0016] The vehicle control unit 22 controls the movement of the vehicle 2 through automatic driving. For example, the vehicle control unit 22 controls the vehicle 2 to move along the input driving path based on various environmental information obtained by the external environment acquisition unit 21. It also performs processing based on the software configuration described later. For example, a computer such as an ECU (Electronic Control Unit) can be used as the vehicle control unit 22.

[0017] The vehicle control unit 22 uses various environmental information obtained by the external environment acquisition unit 21 to control the drive unit 24 in accordance with traffic laws and regulations and the surrounding conditions of the vehicle 2, thereby driving the vehicle 2. Specific operations include, for example, braking, turning signal operation, accelerator operation, steering operation, and shift lever operation. The automatic driving functions provided by the vehicle control unit 22 are not particularly limited, and known automatic driving technologies can be appropriately utilized.

[0018] The memory unit 23 is a storage medium for storing various types of information, and can be, for example, ROM, RAM, SRAM, HDD, SSD, or a combination thereof. The memory unit 23 stores various types of information necessary for the vehicle control unit 22 to perform various processes. For example, it stores control programs, map information, and information about the vehicle itself. The road information included in the map information includes information about various objects that restrict the movement of the vehicle 2, such as road intersections, traffic lights, pedestrian crossings, sidewalks, median strips on the road, guardrails along the road, and regulatory signs (including supplementary signs).

[0019] Map information may be provided by a navigation device 28 installed in the vehicle, or by an external server, etc.

[0020] The drive unit 24 includes a drive mechanism such as an electric motor and / or an internal combustion engine that are the driving source for the vehicle, a power transmission device including a drive shaft and an automatic transmission that transmits the output from these driving sources to the drive wheels, and a braking device that brakes the wheels. Based on the control signals input from the vehicle control unit 22, the drive unit 24 generates control signals for each of these drive mechanisms and performs driving control, including acceleration and deceleration of the vehicle. The drive unit 24 may also include other equipment necessary for the vehicle 2 to run, such as headlights, turn signals, hazard lights, and wipers.

[0021] The display unit 25 is a display device such as a screen installed inside the vehicle. The display unit 25 displays, for example, instruments, maps from the navigation system, operation screens for in-vehicle equipment, images of the outside of the vehicle taken by the camera unit 211, and information related to notifications to the driver, which will be described later.

[0022] Speaker 26 is a well-known device installed inside the vehicle, and when the aforementioned notification to the driver is made by voice, it is done through speaker 26. Speaker 26 is also used as an audio output device from audio equipment installed inside the vehicle.

[0023] The transmitting / receiving unit 27 transmits and receives various information and signals to and from devices such as servers outside the vehicle. For example, it can receive map information from a server outside the vehicle. The transmitting / receiving unit 27 includes, for example, a wireless device which is an interface such as a wireless LAN. The wireless device includes, for example, wireless modules such as Wi-Fi® and Bluetooth®. As a result, the server is connected to a network such as a wireless LAN, a public network such as the Internet, or a wireless network such as a mobile phone network including a communication base station.

[0024] The navigation device 28 is a well-known device that displays road information and other data to the input destination on its display unit based on map information. The navigation device 28 may be an in-vehicle unit, or it may use a navigation application provided from an external server or the like.

[0025] <2. Vehicle Software Configuration> Figure 2 is a block diagram showing the software configuration of the vehicle control unit 22 of the vehicle. The program loaded into the RAM of the vehicle control unit 22 is interpreted and executed by the ECU, etc. As a result, as shown in Figure 2, the vehicle control unit 22 functions as a computer equipped with a sign acquisition unit 221, another vehicle information acquisition unit 222, determination unit 223, route generation unit 224, drive control unit 225, and notification unit 226.

[0026] <2-1. Sign Acquisition Unit> The sign acquisition unit 221 detects regulatory signs (including supplementary signs) under the Road Traffic Act that are placed in front of the vehicle 2 on the road while it is traveling, using the external environment acquisition unit 21. For example, when using the external environment acquisition unit 21, regulatory signs in front of the vehicle 2 are detected by image recognition of the imaging unit 211. In addition to using the imaging unit 211, regulatory signs placed in front of the vehicle 2 can also be detected by referring to map information stored in the storage unit 23, the navigation device 28, or map information acquired from an external server via the transmission / reception unit 27.

[0027] Furthermore, the sign acquisition unit 221 acquires the regulatory content indicated by the regulatory sign. Figures 3A and 3B are examples of regulatory signs (including supplementary signs) that can be detected. The sign acquisition unit 221 recognizes the regulatory content indicated by the detected regulatory sign. For example, the regulatory sign in Figure 3A indicates that on the road ahead, from 7:00 to 9:00 and from 16:00 to 18:00, only "large vehicles, etc.", i.e., large vehicles, medium-sized vehicles, and large special vehicles are allowed to go straight or turn left, and right turns are prohibited.

[0028] On the other hand, vehicles other than "large vehicles," such as regular passenger cars, are not subject to such restrictions and can go straight, turn right, and turn left at all times. Therefore, it can be understood that this regulatory sign displays two types of restrictions: one for "large vehicles" and another for regular passenger cars.

[0029] The regulatory sign in Figure 3B indicates that, on the road ahead, large vehicles and certain medium-sized vehicles are only permitted to proceed straight. However, even large vehicles and certain medium-sized vehicles, such as school and kindergarten buses, are permitted to proceed straight, turn right, and turn left. Vehicles other than large vehicles and certain medium-sized vehicles are also permitted to proceed straight, turn right, and turn left. Therefore, it can be recognized that this regulatory sign displays three types of regulations: for large vehicles and certain medium-sized vehicles other than school and kindergarten buses, for school and kindergarten buses that are large vehicles and certain medium-sized vehicles, and for vehicles other than large vehicles and certain medium-sized vehicles (such as regular passenger cars).

[0030] The regulatory information indicated by such regulatory signs is stored in the storage unit 23. Therefore, the regulatory information corresponding to a detected regulatory sign can be recognized by referring to the storage unit 23. Alternatively, the image of the detected regulatory sign can be transmitted to an external server via the transmitting / receiving unit 27, and the corresponding regulatory information can be obtained from the server. This acquisition of regulatory information may be done by extracting it from the acquired image of the regulatory sign using image recognition, or the regulatory information of the regulatory sign may be stored in the storage unit 23 (or an external server), and the regulatory information may be obtained by comparing the stored regulatory information with the image of the detected regulatory sign.

[0031] <2-2. Other Vehicle Information Acquisition Unit> The Other Vehicle Information Acquisition Unit 222 acquires information about other vehicles in the vicinity of the vehicle 2. For example, the shooting unit 211 of the external environment acquisition unit 21 photographs other vehicles traveling around the vehicle 2 and acquires information about those vehicles. Information about other vehicles refers to information about vehicles that are subject to regulations indicated by regulatory signs, such as the vehicle classification, vehicle type, vehicle height, and vehicle width under the Road Traffic Act. The range of other vehicles to be detected around the vehicle 2 is not particularly limited, but for example, vehicles traveling in the same direction as the vehicle 2 in front of, diagonally in front of, to the side of, behind, and diagonally behind the vehicle 2 are included.

[0032] Information about such vehicles is stored in the storage unit 23. For example, information about other vehicles can be obtained from the storage unit 23 based on images of other vehicles that have been photographed. Alternatively, by transmitting images of other vehicles that have been acquired to an external server via the transmitting / receiving unit 27, information about the corresponding vehicle can be obtained from the server.

[0033] Furthermore, if multiple other vehicles are present in the vicinity of your vehicle, you can obtain information about each of them.

[0034] <2-3. Determination Unit> The determination unit 223 determines the correspondence between the regulatory content indicated by the regulatory sign and the vehicle itself 2 and other vehicles. For example, suppose that at 8:00 a.m., on a certain road, the vehicle itself and other vehicles are driving around it, and the determination unit 223 determines that the vehicle itself 2 is a regular motor vehicle under the Road Traffic Act, and the other vehicles are heavy motor vehicles under the Road Traffic Act. If the vehicle itself detects the regulatory sign shown in Figure 3A in front of the vehicle itself 2, the determination unit 223 makes the following determination: ・Vehicle itself: It is possible to go straight, turn left, or turn right at the intersection (or fork in the road) ahead. ・Other vehicles: It is possible to go straight or turn left at the intersection (or fork in the road) ahead. In other words, it determines that the regulatory content differs between the vehicle itself 2 and the other vehicles.

[0035] Furthermore, if multiple other vehicles are present around your own vehicle 2, you can perform a check on each of the other vehicles.

[0036] <2-4. Route Generation Unit> The route generation unit 224 can generate a route for the vehicle 2 based on at least one element, such as road conditions including lanes and traffic lights, surrounding objects including other vehicles and pedestrians, and a route to the destination of the vehicle 2 that is pre-set in the navigation device 28, etc. Such a route can be generated, for example, by a machine learning model that has been trained using each of the above elements as input. It is also possible to obtain a route generated by the server by communicating with the server.

[0037] Furthermore, the vehicle 2 can generate a route based on regulatory signs acquired by the sign acquisition unit 221. For example, if a regulatory sign different from the one the vehicle 2 possesses is installed, and the vehicle cannot proceed along the pre-set route due to the regulations indicated by the regulatory sign, a new route following the regulatory sign can be generated.

[0038] <2-5. Drive Control Unit> The drive control unit 225 normally controls the drive unit 24 to proceed along a pre-set route. The drive control unit 225 also controls the drive of the vehicle 2 based on regulatory signs acquired by the sign acquisition unit 221. For example, when a new route is generated by the route generation unit 224, the drive control unit 225 controls the vehicle 2's movement, such as deciding whether to start or stop and controlling the speed, according to the regulatory content of the recognized regulatory sign.

[0039] <2-6. Notification Unit> The notification unit 226 notifies the driver according to the determination of the determination unit 223 described above. For example, it notifies the driver that there is a difference in the direction of travel regulations at the upcoming intersection (or fork in the road) between the vehicle 2 and other vehicles traveling around it. It can also notify the driver in more detail that "vehicle 2 can go straight, turn left, and turn right, but other vehicles can only go straight and turn left." In this case, the display unit 25 can show the location of the other vehicle that is the subject of the notification, or the location of the other vehicle can be notified by voice.

[0040] Alternatively, according to the determination, a notification can be issued that another vehicle is predicted to exhibit a behavior different from that of the host vehicle 2. For example, when the host vehicle 2 turns right at the upcoming intersection based on a preset route, the notification unit 226 can also notify that "unlike the host vehicle, the other vehicle will not turn right".

[0041] Note that no notification is required when the regulation content is the same for the host vehicle 2 and another vehicle.

[0042] The notification method is not particularly limited, and may be, for example, notification by voice from the in-vehicle speaker 26, notification by displaying characters, graphics, or the like indicating the relevant content on the display unit 25, notification by light emission, or a combination of two or more of these methods.

[0043] Furthermore, when there are a plurality of other vehicles around the host vehicle 2, the determination can be performed for each other vehicle as described above, and a notification can be issued that the regulation content differs for each. In this case, the position of each other vehicle to be notified can be indicated on the display unit 25, or the position of another vehicle can be notified by voice.

[0044] <3. Driving Support Method> Next, a driving support method for a vehicle configured as described above will be described with reference to the flowchart in FIG. 4. Here, the flowchart in FIG. 4 will be described with reference to the example of FIG. 3A.

[0045] When the sign acquisition unit 221 of the host vehicle 2 detects a regulation sign ahead of the host vehicle 2 (step S11), the sign acquisition unit 221 recognizes the regulation content indicated by the regulation sign. For example, when the regulation sign in FIG. 3A is detected, the sign acquisition unit 221 recognizes that two regulation contents are indicated as described above (step S12). Then, when the host vehicle 2 is a normal passenger vehicle, the determination unit 223 recognizes, from the recognized regulation content, that the host vehicle is allowed to go straight, turn left, and turn right at the upcoming intersection (step S13).

[0046] Further, when the other vehicle information acquisition unit detects another vehicle around the host vehicle, the unit acquires information of said other vehicle (step S14). Then, when the other vehicle is a large automobile, the determination unit 223 recognizes from the identified regulation content that the other vehicle is allowed to go straight and turn left at the intersection ahead (step S15). Subsequently, the determination unit 223 determines whether or not there is a difference in regulation content between the host vehicle 2 and the other vehicle (step S16). In this example, since there is a difference in the regulation content (YES in step S16), the drive control unit 225 controls the driving of the host vehicle 2 as described above, as necessary (step S17). In addition, when traveling along a preset route, no special drive control is performed if the regulation content does not affect the driving of the host vehicle 2.

[0047] Thereafter, the notification unit 226 notifies the driver that there is a difference in the regulation content regarding the traveling direction at the intersection ahead between the host vehicle 2 and the other vehicle (step S18). As described above, more detailed notification can also be provided. In addition, when there is no difference in the regulation content (NO in step S16), the process ends.

[0048] <4. Features> According to the driving support method of the present embodiment, when a regulation sign is detected, if the regulation sign indicates regulation content for different vehicles, the regulation content is recognized, and it is determined whether the regulation content is applicable to the host vehicle 2. Therefore, even if the regulation sign indicates regulation content for different vehicles, it is possible to easily determine whether the regulation is applicable to the host vehicle.

[0049] Further, when the regulation content differs between the host vehicle 2 and another vehicle traveling in the surrounding area, the driver may feel uncomfortable if the other vehicle exhibits behavior different from that of the host vehicle 2. Therefore, the present embodiment is configured to notify the driver of this fact when the regulation content differs between the host vehicle and the other vehicle.

[0050] <5. Modifications> Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the invention. For example, the following modifications are possible. Furthermore, the gist of the following modifications can be combined with each other as appropriate, and can also be combined with the above embodiment.

[0051] (1) In the above embodiment, regulatory signs under the Road Traffic Act have been described, but the present invention is also applicable to regulatory signs stipulated in various laws and regulations other than the Road Traffic Act.

[0052] (2) In the above embodiment, the determination unit 223 determines whether the regulatory sign applies to the vehicle 2 and other vehicles, but this function may also be performed by the sign acquisition unit 221. Therefore, the determination unit only needs to determine whether the regulatory content differs between the vehicle 2 and other vehicles.

[0053] (3) If the determination unit 223 determines that another vehicle has exhibited behavior that violates the regulations, the notification unit 225 may notify the driver and at least one of the other vehicle of the violation. Also, for example, if the regulations for the own vehicle 2 and the other vehicle are the same, and the other vehicle is driving in violation of the regulations (for example, when the other vehicle enters a road that is subject to regulations), the driver may be notified that the own vehicle will comply with the regulations and will not enter the road in question (will take a detour).

[0054] (4) In the above embodiment, the vehicle control unit 22, which is the driving assistance device of the present invention, is configured with a computer mounted on the vehicle itself. However, the driving assistance device can also be located elsewhere than the vehicle itself. For example, it can be located on a server that can communicate with the vehicle itself or on another vehicle. Alternatively, at least some of the functions of the external environment acquisition unit 21 can be implemented on a server that can communicate with the vehicle itself, on another vehicle, or on infrastructure equipment.

[0055] (5) In the above embodiment, if the regulations differ between the vehicle 2 and other vehicles, a notification is given to that effect. However, it is also possible to set cases in which notification is not required. For example, after determining that the regulations differ between the vehicle 2 and other vehicles, if the behavior of the other vehicle differs from that of the vehicle 2, a notification may be given when the in-vehicle camera detects a change in at least one of the driver's (or other occupant's) gaze, facial expression, and behavior. In such cases, it is assumed that the driver is concerned about the other vehicle's behavior differing from that of the vehicle 2, so a notification is necessary. However, if the above-mentioned change cannot be detected, it is assumed that the driver is not concerned about the other vehicle's behavior, so a notification may be deemed unnecessary.

[0056] (6) In the above embodiment, if the regulations differ between the vehicle 2 and other vehicles, a notification is given to that effect. However, it is also possible to give a notification only if the driver has given prior permission. For example, a notification may be given only when the driver gives permission using the touch panel or other operation switches on the display unit 25, or when permission is given by voice.

[0057] (7) In the above embodiment, the case in which the vehicle 2 performs autonomous driving has been described, but the present invention can also be applied to vehicles that do not perform autonomous driving. That is, even when the driver is driving the vehicle themselves, if there is a difference in regulations between the vehicle 2 and other vehicles, the driver may notify the other party of that fact.

[0058] 2: Vehicle 2: Own vehicle 22: Vehicle control unit 226: Notification unit

Claims

1. A computer-based driving assistance method comprising: detecting a regulatory sign positioned in front of the vehicle that regulates at least one of a specific road, vehicle, and time period; recognizing the content of the regulation from the detected regulatory sign; recognizing a first regulatory content applicable to the vehicle and at least one second regulatory content applicable to other vehicles in the vicinity of the vehicle from the regulatory content; and, if it is determined that the first regulatory content and the second regulatory content are different, notifying the driver that the regulations applicable to the vehicle and the other vehicles are different.

2. The driving assistance method according to claim 1, wherein if it is determined that the first regulatory content and the second regulatory content are different, the method notifies the driver that it is predicted that the vehicle itself and the other vehicle will exhibit different behaviors based on the different regulations.

3. The driving assistance method according to claim 1, which controls the driving of the vehicle in accordance with the first regulatory content.

4. The driving assistance method according to claim 1, wherein the difference between the first regulatory content and the second regulatory content is whether or not entry to the branch road is permitted.

5. The driving assistance method according to claim 1, wherein the difference between the first regulatory content and the second regulatory content is due to at least one of the legal vehicle classification, vehicle type, vehicle height, and vehicle width.

6. The driver assistance method according to claim 1, which determines that the other vehicle has exhibited behavior in violation of the second regulation, and notifies the driver of that fact.

7. The driving assistance method according to claim 1, wherein the notification is made only if the driver has given prior permission.

8. The driving assistance method according to claim 1, wherein when the other vehicle exhibits behavior different from that of the own vehicle, the driver's gaze, facial expression, and actions are detected, and the notification is given when there is a change.

9. The driving assistance method according to claim 1, which, when it is determined that the other vehicle is exhibiting a violation, notifies the other vehicle of that fact.

10. A driver assistance device having a vehicle control unit, the vehicle control unit is configured to: detect a regulatory sign positioned in front of the vehicle that regulates at least one of a specific road, vehicle, and time period; recognize the content of the regulation from the detected regulatory sign; recognize a first regulatory content applicable to the vehicle and a second regulatory content applicable to other vehicles in the vicinity of the vehicle from the content of the regulation; and, if it is determined that the first regulatory content and the second regulatory content are different, notify the driver that the regulations applicable to the vehicle and the other vehicles are different.

Images