Driving assistance control device, driving assistance control method and program
The driving support system uses onboard and external cameras to detect and notify drivers of obscured pedestrians, addressing blind spots caused by parked vehicles, thus improving road safety.
Patent Information
- Application Number
- JP2021050106
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-24
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-03-24
AI Technical Summary
Drivers face blind spots due to parked or stopped vehicles, which obstruct their view of pedestrians crossing the road, particularly at bus stops, leading to safety hazards.
A driving support system that utilizes both onboard and external cameras to capture and analyze pedestrian crossing areas, comparing the data to identify any discrepancies and notify the driver of obscured pedestrians through a notification system.
Enhances driving safety by providing real-time awareness of blind spots, ensuring drivers are alerted to pedestrians hidden from view, thereby reducing the risk of accidents.
Smart Images

Figure 0007585919000001 
Figure 0007585919000002 
Figure 0007585919000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a driving assistance control device, a driving assistance control method, and a program. [Background technology]
[0002] When multiple vehicles are traveling on a road, a driver of the vehicle may be obstructed by parked or stopped vehicles, resulting in blind spots in the driver's field of vision. The following patent documents describe a technology for improving safety in the event of blind spots occurring. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2008-041058 A [Patent Document 1] JP 2009-199532 A Summary of the Invention [Problem to be solved by the invention]
[0004] For example, when a vehicle is traveling on a road, there may be a bus stop in front of a crosswalk with oncoming traffic, and the bus may be stopped at the bus stop. In this case, the driver of the vehicle is blocked by the bus, and part of the crosswalk behind the bus is in the blind spot. As a result, the driver cannot see pedestrians crossing the crosswalk who are hidden by the bus.
[0005] The present invention has been made in consideration of the above, and has an object to improve driving safety by obtaining information on areas that are blind spots for the driver and assisting the driver. [Means for solving the problem]
[0006] In order to solve the above-mentioned problems and achieve the object, the driving assistance control device of the present invention comprises a first image data acquisition unit that acquires image data of a pedestrian crossing area captured by a vehicle camera, a second image data acquisition unit that acquires image data captured by an external camera, a comparison unit that compares a first form of a pedestrian in the crossing area estimated based on the image data captured by the vehicle camera with a second form of a pedestrian in the crossing area estimated based on the image data captured by the external camera, and a notification unit that provides an alert when the first form and the second form differ.
[0007] The driving assistance control method of the present invention includes the steps of acquiring image data of a pedestrian crossing area captured by a vehicle camera, acquiring image data captured by an external camera, comparing a first form of the pedestrian in the crossing area estimated based on the image data captured by the vehicle camera with a second form of the pedestrian in the crossing area estimated based on the image data captured by the external camera, and issuing an alert when the first form and the second form differ.
[0008] The program of the present invention causes a computer to execute the steps of acquiring image data of a pedestrian crossing area photographed by a vehicle camera, acquiring image data photographed by an external camera, comparing a first form of a pedestrian in the crossing area estimated based on the image data photographed by the vehicle camera with a second form of a pedestrian in the crossing area estimated based on the image data photographed by the external camera, and issuing an alert when the first form and the second form differ. Effect of the Invention
[0009] According to the present invention, it is possible to obtain information on areas that are blind spots for the driver and thereby assist the driver, thereby improving driving safety. [Brief description of the drawings]
[0010] [Figure 1]FIG. 1 is a block diagram showing a navigation system as a driving assistance device according to the first embodiment. [Diagram 2] FIG. 2 is an explanatory diagram illustrating the driving support control process. [Diagram 3] FIG. 3 is an explanatory diagram for explaining the driver's field of vision. [Figure 4] FIG. 4 is an explanatory diagram illustrating an example of a notification determination in the driving support control process. [Diagram 5] FIG. 5 is an explanatory diagram illustrating another example of the notification determination in the driving support control process. [Figure 6] FIG. 6 is a flowchart showing the driving assistance control process. [Figure 7] FIG. 7 is a block diagram showing a navigation system as a driving support device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A driving assistance control device, a driving assistance control method, and a program according to the present invention will be described in detail below with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiments.
[0012] First Embodiment [Navigation system] FIG. 1 is a block diagram showing a navigation system as a driving assistance device according to the first embodiment.
[0013] 1, the navigation system 1 includes a GPS (Global Positioning System) receiving unit 11, which is an example of a GNSS (Global Navigation Satellite System), an operation unit 12, a communication unit 13, a map information storage unit 14, a camera 15, a notification unit 16, and a control unit (driving assistance control device) 20. The navigation system 1 provides route guidance from a current position to a destination.
[0014] In the following, the navigation system 1 will be described as being mounted on a vehicle, but is not limited to this. For example, the navigation system 1 may be configured as a mobile phone, a smartphone, a tablet terminal, a computer (personal computer), etc.
[0015] The GPS receiving unit 11 is composed of a GPS receiving circuit, a GPS receiving antenna, etc., and receives a GPS signal. The GPS receiving unit 11 outputs the received GPS signal to the position information acquisition unit 21.
[0016] The operation unit 12 accepts various operations to the control unit 20. The operation unit 12 allows a user to input a departure point, departure date and time, a destination, intermediate points, and the like. The operation unit 12 is configured with, for example, a physical switch and a touch panel. The operation unit 12 outputs an electrical signal corresponding to the accepted operation to the operation information acquisition unit 22.
[0017] The communication unit 13 is composed of, for example, a communication circuit. The communication unit 13 is wirelessly connected to a network (for example, the Internet). The communication unit 13 transmits and receives information to and from other information communication devices, etc. via the network in accordance with the control of the communication control unit 23. The communication unit 13 transmits and receives information to and from, for example, a smartphone owned by a user. Any communication method can be applied as the communication method by which the communication unit 13 communicates. The communication unit 13 transmits and receives information to and from the communication control unit 23.
[0018] The map information storage unit 14 stores various map information. The map information is road map data including various road information such as the positions of traffic lights and rest points. The map information includes, for example, information on main roads and so-called shortcuts and back roads. The map information storage unit 14 is not included in the configuration of the navigation system 1, and may be capable of acquiring map information from an external server using the communication unit 13. The map information storage unit 14 outputs the acquired map information to the map information acquisition unit 24.
[0019] The camera 15 is a camera disposed on the vehicle side. The camera 15 is disposed, for example, at the front of the vehicle interior. The camera 15 constantly captures images of the outside from when the engine starts until it is stopped, that is, while the vehicle is operating. The camera 15 captures images of the outside while the accessory power of the vehicle is ON. The camera 15 outputs the captured image data to the image data acquisition unit 25.
[0020] The notification unit 16 notifies various kinds of information. For example, the notification unit 16 notifies various kinds of information according to the control of the notification control unit 27 of the control unit 20. For example, the notification unit 16 displays information on route guidance from the current position to the destination. For example, the notification unit 16 is a display including a liquid crystal display (LCD) or an organic electro-luminescence (EL) display. When the operation unit 12 is configured with a touch panel, the notification unit 16 is provided integrally with the operation unit 12. The notification unit 16 may also include a speaker and outputs the information to be notified by voice. For example, the notification unit 16 outputs voice of route guidance from the current position to the destination. The notification unit 16 receives data to be notified from the notification control unit 27.
[0021] The control unit 20 is realized, for example, by a CPU (Central Processing Unit), MPU (Micro Processing Unit), etc., executing a program (for example, the program according to the present invention) stored in a storage unit (not shown) using a RAM (Random Access Memory) etc. as a working area. The control unit 20 is a controller, and may be realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).
[0022] The control unit 20 includes a position information acquisition unit 21, an operation information acquisition unit 22, a communication control unit 23, a map information acquisition unit 24, an image data acquisition unit (first image data acquisition unit) 25, an image data analysis unit 26, and a notification control unit 27. The control unit 20 also includes an external image data acquisition unit (second image data acquisition unit) 31, an external image data analysis unit 32, a comparison unit 33, a route setting unit 34, and a route guidance unit 35.
[0023] The position information acquisition unit 21 acquires the current position information of the vehicle. The position information acquisition unit 21 acquires the current position information of the vehicle based on the GPS signal received by the GPS receiving unit 11, for example.
[0024] The operation information acquiring unit 22 receives various information from the operation unit 12. The operation information acquiring unit 22 receives information from the operation unit 12, such as the departure point, departure date and time, destination, and stopover points.
[0025] The communication control unit 23 controls various communications via the communication unit 13. For example, the communication control unit 23 controls communications with a smartphone or the like owned by the user. The smartphone or the like owned by the user is registered in advance as a communication destination by the communication unit 13.
[0026] The map information acquisition unit 24 acquires map information from the map information storage unit 14. Based on the current position of the vehicle and the position information of the intermediate points and the destination acquired by the position information acquisition unit 21, the map information acquisition unit 24 acquires map information relating to a route from the current position of the vehicle to the destination via the intermediate points from the map information storage unit 14.
[0027] The image data acquisition unit 25 acquires image data obtained by capturing images of the surroundings (particularly, the front) of the vehicle. More specifically, the image data acquisition unit 25 acquires image data output by the camera 15.
[0028] The image data analysis unit 26 analyzes the image data acquired by the image data acquisition unit 25. The image data analysis unit 26 outputs the analyzed image data to the comparison unit 33. More specifically, the image data analysis unit 26 identifies a crossing area where a pedestrian crosses a road, for example, a crosswalk, based on the acquired image data. At this time, the image data analysis unit 26 identifies the position of the identified crosswalk as absolute coordinate data (for example, a GPS area number, an address, or a block number). Then, the image data analysis unit 26 identifies pedestrians moving on the identified crosswalk (crossing area) and calculates the number of pedestrians. In addition, the image data analysis unit 26 identifies pedestrians who are about to enter the crosswalk from the sidewalk based on the acquired image data, and calculates the total number of pedestrians, including pedestrians moving on the crosswalk and pedestrians who are about to enter the crosswalk from the sidewalk.
[0029] The crossing area is not limited to a crosswalk, but may be an intersection with a traffic light, a road without a crosswalk but where it is possible to cross, etc. The image data analysis unit 26 may also identify a pet (e.g., a dog, etc.) moving with a pedestrian on the crosswalk, or a person riding a vehicle such as a bicycle or a wheelchair, and identify them as pedestrians. Here, machine learning such as deep learning may be used as a method for identifying the crosswalk, pedestrian, pet, etc., but other machine learning or other identification methods may also be used.
[0030] The notification control unit 27 causes the notification unit 16 to display various types of information. For example, the notification control unit 27 causes the notification unit 16 to display information on a current position or route guidance from a starting point to an intermediate point or a destination. The notification control unit 27 also causes the notification unit 16 to notify, as a comparison result of the comparison unit 33, that a pedestrian that the driver cannot visually recognize is present at a crosswalk.
[0031] The external image data acquisition unit 31 acquires image data of a specific crossing area, for example, a specific crosswalk. An external camera 41 is installed near the crosswalk, and an external image data communication unit 42 is connected to the external camera 41. The external camera 41 can capture at least the entire area of the crosswalk, preferably the crosswalk and both sidewalks and the surrounding roads. The external image data communication unit 42 transmits the image data output by the external camera 41 to the outside. The image data may include position information and time information based on the image data captured by the external camera 41. The external image data acquisition unit 31 can acquire the image data output by the external camera 41 and transmitted by the external image data communication unit 42. The installation location of the external camera 41 is not limited to the vicinity of the crosswalk, but may be the vicinity of an intersection with a traffic light, or the vicinity of a road that does not have a crosswalk but is crossable.
[0032] The external image data analysis unit 32 analyzes the image data acquired by the external image data acquisition unit 31. The external image data analysis unit 32 outputs the analyzed image data to the comparison unit 33. More specifically, the external image data analysis unit 32 identifies a crossing area where a pedestrian crosses a road, for example, a crosswalk, based on the acquired image data. At this time, the external image data analysis unit 32 identifies the position of the identified crosswalk as absolute coordinate data (for example, a GPS area number, an address, or a block number). Then, the external image data analysis unit 32 identifies pedestrians moving on the identified crosswalk (crossing area) and calculates the number of pedestrians. In addition, the external image data analysis unit 32 identifies pedestrians who are about to enter the crosswalk from the sidewalk based on the acquired image data, and calculates the total number of pedestrians, including pedestrians moving on the crosswalk and pedestrians who are about to enter the crosswalk from the sidewalk.
[0033] The external image data analysis unit 32 may identify a pet (e.g., a dog) moving with a pedestrian on the crosswalk, or a person riding a vehicle such as a bicycle or a wheelchair, and identify them as a pedestrian. Here, machine learning such as deep learning may be used as a method for identifying the crosswalk, pedestrian, pet, etc., but other machine learning or other identification methods may also be used.
[0034] The comparison unit 33 compares a first number (first form) of pedestrians on the crosswalk (crossing area) estimated based on image data acquired by the image data acquisition unit 25 and analyzed by the image data analysis unit 26, with a second number (second form) of pedestrians on the crosswalk (crossing area) estimated based on image data acquired by the external image data acquisition unit 31 and analyzed by the external image data analysis unit 32.
[0035] That is, while the vehicle is traveling, the image data analysis unit 26 receives and analyzes image data from the camera 15 from the image data acquisition unit 25, and if there is a crosswalk ahead of the vehicle, it identifies the crosswalk based on the image data from the camera 15, identifies the position of the identified crosswalk as absolute coordinate data, and identifies the number of pedestrians on the crosswalk. At this time, if an external camera 41 is installed near the crosswalk, the external image data analysis unit 32 receives and analyzes image data from the external camera 41 from the external image data acquisition unit 31. That is, the external image data analysis unit 32 identifies the crosswalk based on the image data from the external camera 41, identifies the position of the identified crosswalk as absolute coordinate data, and identifies the number of pedestrians on the crosswalk.
[0036] At this time, the comparison unit 33 checks whether the absolute coordinate data of the crosswalk identified by the image data analysis unit 26 matches the absolute coordinate data of the crosswalk identified by the external image data analysis unit 32. The comparison unit 33 compares the number of pedestrians on the crosswalk identified by the external image data analysis unit 32 at the same time with the number of pedestrians on the crosswalk identified by the image data analysis unit 26. Since the external image data analysis unit 32 identifies the absolute coordinate data of the crosswalk based on image data from the external camera 41, the image data analysis unit 26 may identify a crosswalk having the same absolute coordinate data as the absolute coordinate data of the crosswalk identified by the external image data analysis unit 32, and identify the number of pedestrians on the crosswalk.
[0037] In addition, the comparison unit 33 may compare a first total number of pedestrians on the crosswalk and pedestrians attempting to enter the crosswalk estimated based on the image data acquired by the image data acquisition unit 25 and analyzed by the image data analysis unit 26, with a second total number of pedestrians on the crosswalk and pedestrians attempting to enter the crosswalk estimated based on the image data acquired by the external image data acquisition unit 31 and analyzed by the external image data analysis unit 32.
[0038] When the first number of people and the second number of people differ, the comparison unit 33 outputs a command to the notification control unit 27 to notify that there is a pedestrian on the crosswalk that the driver cannot visually recognize. Furthermore, when the first total number of people and the second total number of people differ, the comparison unit 33 may output a command to the notification control unit 27 to notify that there is a pedestrian on the crosswalk that the driver cannot visually recognize.
[0039] The image data analysis unit 26 and the external image data analysis unit 32 may identify a crosswalk based on the image data and identify the positions of pedestrians on the identified crosswalk as absolute coordinate data. The comparison unit 33 may then compare the absolute coordinate data of pedestrians on the crosswalk analyzed by the image data analysis unit 26 with the absolute coordinate data of pedestrians on the crosswalk analyzed by the external image data analysis unit 32 to determine the difference in the number of pedestrians.
[0040] The route setting unit 34 sets a route for the vehicle to travel from the departure point or the current position to the intermediate point and the destination. For example, the route setting unit 34 acquires map information related to the route from the current position to the destination from the map information storage unit 14, and sets a route from the current position to the intermediate point and the destination based on the acquired map information.
[0041] The route setting unit 34 acquires information about congestion on the route from the current position to the intermediate points and the destination. The route setting unit 34 acquires, for example, information indicating whether the route from the current position to the destination is congested, information indicating the degree of congestion, etc. from an external server or the like via the communication unit 13 and the communication control unit 23.
[0042] For example, when a waypoint is set between the current position and the destination, the route setting unit 34 may set a route with different conditions for each section. Specifically, when a waypoint is set between the current position and the destination, the route setting unit 34 may set a route with different conditions for the section from the current position to the waypoint and the section from the waypoint to the destination.
[0043] The route guidance unit 35 performs route guidance. For example, the route guidance unit 35 causes the notification control unit 27 to display map information or the like on the notification unit 16 for the route from the current position to the intermediate point and the destination set by the route setting unit 34, or performs route guidance using voice.
[0044] [Description of driving assistance processing] Here, the specific contents of the driving support process will be described. Fig. 2 is an explanatory diagram for explaining the driving support control process.
[0045] As shown in Fig. 2, sidewalks 52, 53 are provided on both sides of road 51, and a crosswalk 54 is provided to connect sidewalks 52, 53 across road 51. A bus stop 55 is provided on one side of sidewalk 52. An external camera 41 and an external image data communication unit 42 are also provided on one side of sidewalk 52. At this time, vehicle 61 travels on the left side of road 51 toward crosswalk 54. Bus 62 travels on the right side of road 51, passes crosswalk 54, and stops at bus stop 55.
[0046] At this time, the driver of the vehicle 61 and the camera 15 of the navigation system 1 have a visual angle θ1 toward the crosswalk 54. However, for the driver of the vehicle 61 and the camera 15 of the navigation system 1, a blind spot θ1a occurs in the visual angle θ1 due to the bus 62. The driver of the vehicle 61 can visually recognize pedestrian M1 moving on the crosswalk 54 within the visual angle θ1. On the other hand, although the driver of the vehicle 61 is within the visual angle θ1, he cannot visually recognize pedestrian M2 moving on the crosswalk 54 in the blind spot θ1a. Furthermore, he cannot visually recognize pedestrian M3 who is on the sidewalk 52 in the blind spot θ1a and heading toward the crosswalk 54.
[0047] FIG. 3 is an explanatory diagram for explaining the driver's field of vision.
[0048] 3, the driver or camera 15 can see pedestrian M1 moving from left to right on the crosswalk 54. However, the driver or camera 15 cannot see pedestrian M2 moving from right to left on the crosswalk 54 because a blind spot area 62A is formed by the bus 62. In addition, the driver or camera 15 cannot see pedestrian M3 entering the crosswalk 54 from the sidewalk 52 because the bus 62 creates a blind spot.
[0049] As shown in FIG. 2, in the first embodiment, an external camera 41 that captures an image of the crosswalk 54 is installed near the crosswalk 54, and an external image data communication unit 42 is connected to the external camera 41. The external camera 41 captures image data of the crosswalk, and the external image data communication unit 42 transmits the image data of the crosswalk captured by the external camera 41 to the outside at all times and when pedestrians move across the crosswalk. The control unit 20 compares a first number of pedestrians on the crosswalk (first total number of pedestrians) estimated based on the image data of the camera 15 with a second number of pedestrians on the crosswalk (second total number of pedestrians) estimated based on the image data of the external camera 41, and when the numbers differ, the notification unit 16 warns the driver. That is, the number of pedestrians on the crosswalk estimated based on the image data of the camera 15 is one person (pedestrian M1), and the number of pedestrians on the crosswalk estimated based on the image data of the external camera 41 is three people (pedestrians M1, M2, and an entering pedestrian M3). Therefore, the control unit 20 warns the driver.
[0050] A specific example will be described below. Fig. 4 is an explanatory diagram for explaining an example of a notification determination in the driving support control process.
[0051] As shown in FIG. 4, a pedestrian M11 actually moves across the crosswalk 54 from the sidewalk 53 side to the sidewalk 52 side, and pedestrians M12 and M13 move across the crosswalk 54 from the sidewalk 52 side to the sidewalk 53 side. At this time, the driver and the camera 15 cannot see the pedestrian M13 because the bus 62 forms a blind spot area 62A. On the other hand, the external camera 41 can capture all the pedestrians M11, M12, and M13 without being obstructed by the bus 62. Therefore, the number of pedestrians on the crosswalk estimated based on the image data of the camera 15 (2 pedestrians M11 and M12) differs from the number of pedestrians on the crosswalk estimated based on the image data of the external camera 41 (3 pedestrians M11, M12, and M13), and the driver is alerted.
[0052] FIG. 5 is an explanatory diagram illustrating another example of the notification determination in the driving support control process.
[0053] As shown in FIG. 5, pedestrians M11 and M14 actually move across the crosswalk 54 from the sidewalk 53 side to the sidewalk 52 side, and pedestrian M12 moves across the crosswalk 54 from the sidewalk 52 side to the sidewalk 53 side. At this time, the driver and camera 15 cannot see the pedestrian M14 because the bus 62 forms a blind spot area 62A. On the other hand, the external camera 41 can capture all the pedestrians M11, M12, and M14 without being obstructed by the bus 62. Therefore, the number of pedestrians on the crosswalk estimated based on the image data of the camera 15 (2 pedestrians M11 and M12) differs from the number of pedestrians on the crosswalk estimated based on the image data of the external camera 41 (3 pedestrians M11, M12, and M14), and the driver is alerted.
[0054] However, a pedestrian M14 moving on the crosswalk 54 from the sidewalk 53 side to the sidewalk 52 side will soon reach the sidewalk 52, and the possibility of danger is low. In this case, the control unit 20 may not include the pedestrian M14 in the number of pedestrians. As shown in FIG. 1 and FIG. 5, the external image data acquisition unit 31 continuously acquires image data from the external camera 41, and the external image data analysis unit 32 continuously processes the image data. Therefore, the external image data analysis unit 32 can obtain time-series image data and obtain the movement direction of the pedestrian moving on the crosswalk 54. Therefore, the number of pedestrians on the crosswalk (2 people) (pedestrians M11, M12) estimated based on the image data of the camera 15 matches the number of pedestrians on the crosswalk (2 people) (pedestrians M11, M12) estimated based on the image data of the external camera 41, and the driver is not warned.
[0055] Also, as shown in FIG. 5, in addition to the pedestrians M11, M12, and M14 who are actually moving on the crosswalk 54, a pedestrian M15 on the sidewalk 52 is about to move on the crosswalk 54. The pedestrian M15 is in the blind spot area 62A and is moving to the crosswalk 54 on the sidewalk 53 that is not in the blind spot area 62A, so there is a possibility of danger. In this case, it is preferable that the control unit 20 add the pedestrian M15 to the number of pedestrians. As shown in FIG. 1 and FIG. 5, the image data analysis unit 26 and the external image data analysis unit 32 identify the crosswalk 54 based on the acquired image data. At this time, the image data analysis unit 26 and the external image data analysis unit 32 identify the number of pedestrians on the sidewalk 52 on the side where the blind spot occurs in addition to the identified crosswalk 54.
[0056] However, as described above, the external image data analysis unit 32 determines the movement direction of the pedestrian moving on the sidewalk 52 from the time-series image data, and identifies the pedestrian M15 who is about to enter the crosswalk 54 from the sidewalk 52. The external image data analysis unit 32 also identifies the pedestrian M16 who is stopped on the sidewalk 52 as a pedestrian to be compared, since there is a possibility that the pedestrian M16 has stopped in an attempt to enter the crosswalk 54. On the other hand, the external image data analysis unit 32 does not identify the pedestrian M17 who is moving on the sidewalk 52 and not moving toward the crosswalk 54, as a pedestrian to be compared.
[0057] [Driving assistance control method] A specific driving assistance control method will be described below. Fig. 6 is a flowchart showing a driving assistance control process.
[0058] As shown in FIG. 1 and FIG. 6, in step S11, when an external camera 41 is installed near a crosswalk while a vehicle is traveling, the external image data acquisition unit 31 receives image data from the external camera 41, and the external image data analysis unit 32 analyzes the image data from the external camera 41. In step S12, the external image data analysis unit 32 identifies the crosswalk based on the image data from the external camera 41, and estimates the number of pedestrians on the crosswalk based on the image from the external camera 41. In step S13, the image data acquisition unit 25 receives image data from the vehicle-side camera 15, and the image data analysis unit 26 analyzes the image data from the camera 15. In step S14, the image data analysis unit 26 identifies the crosswalk based on the image data from the camera 15 from the image data acquisition unit 25, and estimates the number of pedestrians on the crosswalk.
[0059] In step S15, the comparison unit 33 compares the number of pedestrians on the crosswalk identified by the external image data analysis unit 32 with the number of pedestrians on the crosswalk identified by the image data analysis unit 26. The comparison unit 33 may compare a first total number of pedestrians on the crosswalk and pedestrians about to enter the crosswalk estimated based on the image data acquired by the image data acquisition unit 25 and analyzed by the image data analysis unit 26 with a second total number of pedestrians on the crosswalk and pedestrians about to enter the crosswalk estimated based on the image data acquired by the external image data acquisition unit 31 and analyzed by the external image data analysis unit 32.
[0060] In step S16, the comparison unit 33 judges whether the first number of people (first total number of people) and the second number of people (second total number of people) are different. Here, if it is judged that the first number of people and the second number of people are not different (No), the routine is exited without doing anything. That is, it is presumed that the driver has visually recognized all pedestrians moving on the crosswalk, and there is no need to warn the driver. On the other hand, if it is judged that the first number of people and the second number of people are different (Yes), an output command is issued to the notification control unit 27 in step S17, and the notification control unit 27 displays a warning that there is a hidden pedestrian on the notification unit 16. That is, it is presumed that the driver has not visually recognized some of the pedestrians moving on the crosswalk, and a warning is issued to the driver.
[0061] <Second embodiment> 7 is a block diagram showing a navigation system as a driving support device according to the second embodiment. Note that members having the same functions as those in the first embodiment described above are given the same reference numerals and detailed descriptions thereof are omitted.
[0062] In the second embodiment, as shown in FIG. 7, the control unit 20 includes a location information acquisition unit 21, an operation information acquisition unit 22, a communication control unit 23, a map information acquisition unit 24, an image data acquisition unit (first image data acquisition unit) 25, an image data analysis unit 26, a notification control unit 27, an external image data acquisition unit (second image data acquisition unit) 31, a comparison unit 33, a route setting unit 34, and a route guidance unit 35.
[0063] Here, the location information acquisition unit 21, the operation information acquisition unit 22, the communication control unit 23, the map information acquisition unit 24, the image data acquisition unit 25, the image data analysis unit 26, the notification control unit 27, the comparison unit 33, the route setting unit 34, and the route guidance unit 35 are the same as in the first embodiment.
[0064] Meanwhile, an external camera 41 is installed near the crosswalk, and an external image data analysis unit 32 is connected to the external camera 41, and an external image data communication unit 42 is connected to the external image data analysis unit 32. The external camera 41 is capable of photographing the crosswalk, the sidewalks on both sides, and the surrounding roads.
[0065] The external image data analysis unit 32 analyzes the image data captured by the external camera 41. The external image data analysis unit 32 outputs the analyzed image data to the external image data communication unit 42. More specifically, the external image data analysis unit 32 identifies a crossing area where pedestrians cross the road, for example, a crosswalk, based on the image data captured by the external camera 41. At this time, the external image data analysis unit 32 identifies the position of the identified crosswalk as absolute coordinate data (for example, a GPS area number, an address, or a block number). Then, the external image data analysis unit 32 identifies pedestrians moving on the identified crosswalk (crossing area) and calculates the number of pedestrians. In addition, the external image data analysis unit 32 identifies pedestrians who are about to enter the crosswalk from the sidewalk based on the acquired image data, and calculates the total number of pedestrians, including pedestrians moving on the crosswalk and pedestrians who are about to enter the crosswalk from the sidewalk.
[0066] The external image data communication unit 42 transmits to the outside the image data analyzed by the external image data analysis unit 32. The external image data acquisition unit 31 can acquire the image data analyzed by the external image data analysis unit 32 and transmitted by the external image data communication unit 42. The external image data acquisition unit 31 outputs the acquired image data analyzed by the external image data analysis unit 32 to the comparison unit 33.
[0067] That is, in the first embodiment, the control unit 20 has an external image data analysis unit 32. Therefore, the external image data acquisition unit 31 acquires image data captured by the external camera 41 and transmitted by the external image data communication unit 42, and the external image data analysis unit 32 analyzes the acquired image data. On the other hand, in the second embodiment, the control unit 20 does not have an external image data analysis unit 32, and the external image data analysis unit 32 is connected to the external camera 41. Therefore, the external image data analysis unit 32 analyzes image data captured by the external camera 41, and the external image data acquisition unit 31 acquires image data analyzed by the external image data analysis unit 32 and transmitted by the external image data communication unit 42. Other processes are similar between the first embodiment and the second embodiment.
[0068] [Effects of this embodiment] The driving assistance control device of this embodiment includes an image data acquisition unit (first image data acquisition unit) 25 that acquires image data of a pedestrian crosswalk (crossing area) captured by the vehicle's camera 15, an external image data acquisition unit (second image data acquisition unit) 31 that acquires image data captured by an external camera 41, a comparison unit 33 that compares a first number (first form) of pedestrians on the crosswalk estimated based on image data captured by the vehicle's camera 15 with a second number (second form) of pedestrians on the crosswalk estimated based on image data captured by the external camera 41, and an alarm unit 16 that issues an alarm when the first number and the second number differ.
[0069] Therefore, when the first number of pedestrians on the crosswalk 54 estimated based on image data captured by the camera 15 mounted on the vehicle differs from the second number of pedestrians on the crosswalk 54 estimated based on image data captured by the external camera 41 capable of capturing the entire area of the crosswalk, a notification is issued, so that the driver notices the pedestrian in the blind spot and drives or stops the vehicle. As a result, driving safety can be improved by obtaining information on the area in the driver's blind spot and supporting the driver.
[0070] In the driving assistance control device of this embodiment, the comparison unit 33 compares the first number and the second number of pedestrians on the crosswalk 54 having the same absolute coordinates based on image data captured by the vehicle camera 15 and image data captured by the external camera 41, and the notification unit 16 issues a notification when the first number and the second number differ. Therefore, the actual number of pedestrians on the crosswalk can be accurately captured and notified to the driver.
[0071] In the driving assistance control device of this embodiment, the comparison unit 33 compares a first total number of pedestrians estimated based on image data captured by the vehicle's camera 15 by adding the number of pedestrians on the crosswalk 54 and the number of pedestrians entering the crosswalk 54 with a second total number of pedestrians estimated based on image data captured by the external camera 41 by adding the number of pedestrians on the crosswalk 54 and the number of pedestrians entering the crosswalk 54, and the notification unit 16 issues a notification when the first total number of pedestrians and the second total number of pedestrians differ. Therefore, it is possible to capture not only pedestrians on the crosswalk 54 but also pedestrians about to enter the crosswalk 54, and the crosswalk situation can be accurately notified to the driver, thereby further improving driving safety.
[0072] The driving assistance control device according to the present invention has been described above, but the driving assistance control device may be embodied in various different forms other than the above-described embodiment.
[0073] Each component of the illustrated driving assistance control device is a functional concept, and does not necessarily have to be physically configured as illustrated. In other words, the specific form of each device is not limited to that shown in the figure, and all or part of the device may be functionally or physically distributed or integrated in any unit depending on the processing load and usage status of each device.
[0074] The configuration of the driving assistance control device is realized, for example, as software, by a program loaded into a memory. In the above embodiment, the functional blocks are described as being realized by cooperation of these hardware and software. In other words, these functional blocks can be realized in various forms by only hardware, only software, or a combination of both.
[0075] The above-mentioned components include those that can be easily imagined by a person skilled in the art and those that are substantially the same. Furthermore, the above-mentioned configurations can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the configurations are possible without departing from the scope of the present invention.
[0076] In the above embodiment, the driving assistance control device has been described as being applied to a navigation system, but it may also be applied to a drive recorder, or may be used as a standalone device. [Explanation of symbols]
[0077] 1. Navigation system 11 GPS receiver 12 Control section 13. Communications Department 14 Map information storage unit 15 Camera 16. Notification Department 20 Control unit (driving assistance control device) 21 Location information acquisition unit 22 Operation information acquisition section 23 Communication control section 24 Map information acquisition section 25 Image data acquisition section 26 Image data analysis section 27 Notification control section 31 External image data acquisition unit 32 External image data analysis unit 33 Comparison section 34 Route setting section 35 Route Guidance Department 41 External Camera 42 External image data communication unit
Claims
1. a first image data acquisition unit that acquires image data of a pedestrian crossing area captured by a vehicle camera; A second image data acquisition unit that acquires image data captured by an external camera installed on the roadside; a comparison unit that compares a first total number of people, which is the sum of the number of pedestrians in the crossing area estimated based on image data captured by the vehicle's camera and the number of pedestrians entering the crossing area, with a second total number of people, which is the sum of the number of pedestrians in the crossing area estimated based on image data captured by the external camera and the number of pedestrians entering the crossing area; a notification unit that notifies when the first total number of people and the second total number of people differ; A driving assistance control device comprising:
2. The comparison unit compares a first number of pedestrians and a second number of pedestrians in the crossing area having the same absolute coordinates based on image data captured by the camera of the vehicle and image data captured by the external camera, and the notification unit issues a notification when the first number of pedestrians and the second number of pedestrians differ from each other. The driving assistance control device according to claim 1 .
3. A computer-implemented method for detecting a plurality of acquiring image data of a pedestrian crossing area captured by a vehicle camera; acquiring image data captured by an external camera installed on the roadside; comparing a first total number of people, which is a sum of the number of pedestrians in the crossing area estimated based on image data captured by the camera of the vehicle and the number of pedestrians entering the crossing area, with a second total number of people, which is a sum of the number of pedestrians in the crossing area estimated based on image data captured by the external camera and the number of pedestrians entering the crossing area; a step of notifying when the first total number of people and the second total number of people differ; A driving assistance control method comprising:
4. acquiring image data of a pedestrian crossing area captured by a vehicle camera; acquiring image data captured by an external camera installed on the roadside; comparing a first total number of people, which is a sum of the number of pedestrians in the crossing area estimated based on image data captured by the camera of the vehicle and the number of pedestrians entering the crossing area, with a second total number of people, which is a sum of the number of pedestrians in the crossing area estimated based on image data captured by the external camera and the number of pedestrians entering the crossing area; a step of notifying when the first total number of people and the second total number of people differ; A program that causes a computer to execute the following.
Citation Information
Patent Citations
Device for detecting pedestrian
JP2001195698A
Pedestrian detector
JP2002024986A
Traveling support device for vehicle
JP2006350613A
System for notifying blind mobing body, image processor, on-vehicle device and notification method
JP2008041058A
Intersection operation support system, on-vehicle equipment, and roadside device
JP2009199532A
