Driving assistance system, driving assistance method, and program

Abstract

This driving assistance system comprises: a storage means that stores image information of a prescribed object including at least one of road demarcation lines indicating vehicle travel lanes, stop lines where vehicles temporarily stop, crosswalk lines where pedestrians cross, road signs, and demarcation lines that demarcate parking spaces in a parking lot, which are provided within a prescribed area; an image acquisition means that acquires image information within the prescribed area; a vehicle detection means that detects information about a vehicle traveling within the prescribed area, on the basis of the image information within the prescribed area; an image generation means that, if it is determined that the prescribed object is not visible due to an obstacle on the basis of the image information within the prescribed area, generates a second image in which the prescribed object and the vehicle are superimposed on the image within the prescribed area, on the basis of the image information of the prescribed object in the storage means, the image information within the prescribed area acquired by the image acquisition means, and the information about the vehicle; and a display means that displays the second image in the vehicle.

Description

Driving support system, driving support method, and program

[0001] The present disclosure relates to a driving support system, a driving support method, and a program for supporting the driving of a vehicle.

[0002] There is known a driving support device that detects a landmark indicating a parking section in a parking lot and supports parking in the parking section based on the position of the detected landmark (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2020-192905

[0004] In the above driving support device, for example, when the landmark is hidden by snow or the like, the landmark cannot be detected, and thus there is a possibility that the driving support for the parking section cannot be performed.

[0005] An object of the present disclosure is to provide a driving support system, a driving support method, and a program that solve any of the above-described problems.

[0006] One aspect of the present disclosure for achieving the above objective is a driving assistance system comprising: a storage means for storing image information of a predetermined object, which includes at least one of the following provided within a predetermined area: road lane markings indicating vehicle driving lanes, stop lines where vehicles must temporarily stop, pedestrian crossing lines where pedestrians must cross, road signs, and markings that demarcate parking spaces in a parking lot; an image acquisition means for acquiring image information within the predetermined area; a vehicle detection means for detecting information of a vehicle traveling within the predetermined area based on the image information within the predetermined area acquired by the image acquisition means; an image generation means for generating a second image in which the predetermined object and the vehicle are superimposed on the image within the predetermined area, based on the image information within the predetermined area acquired by the image acquisition means and when it is determined that the predetermined object is not visible due to an obstacle; and a display means for displaying the second image generated by the image generation means in the vehicle. One aspect of the present disclosure for achieving the above objective is a driving assistance method comprising: storing image information of a predetermined object, which includes at least one of the following provided within a predetermined area: road markings indicating vehicle driving lanes, stop lines where vehicles must temporarily stop, pedestrian crossing lines where pedestrians must cross, road signs, and markings that demarcate parking spaces in a parking lot; acquiring image information within the predetermined area; detecting information of a vehicle traveling within the predetermined area based on the acquired image information within the predetermined area; generating a second image in which the predetermined object and the vehicle are superimposed on the image within the predetermined area, based on the stored image information of the predetermined object, the acquired image information within the predetermined area, and the detected vehicle information, when the predetermined object is not visible due to an obstacle; and displaying the generated second image in the vehicle.One aspect of this disclosure for achieving the above objective is a program that causes a computer to execute: a process of storing image information of a predetermined object which includes at least one of the following provided within a predetermined area: road markings indicating vehicle driving lanes, stop lines where vehicles stop temporarily, pedestrian crossing lines where pedestrians cross, road signs, and markings that demarcate parking spaces in a parking lot; a process of acquiring image information within the predetermined area; a process of detecting information of a vehicle driving within the predetermined area based on the acquired image information within the predetermined area; a process of generating a second image which superimposes the predetermined object and the vehicle onto the image within the predetermined area based on the stored image information of the predetermined object, the acquired image information within the predetermined area, and the detected vehicle information, when the predetermined object is not visible due to an obstacle; and a process of displaying the generated second image in the vehicle.

[0007] This disclosure provides a driver assistance system, a driver assistance method, and a program that solve any of the above-mentioned problems.

[0008] This is a block diagram showing the schematic system configuration of the driver assistance system according to this embodiment. This is a block diagram showing the schematic system configuration of the driver assistance device according to this embodiment. This is a flowchart showing an example of the processing flow when detecting and storing image information of lane markings. This is a diagram showing an example of an overhead view image. This is a diagram showing an example of an overhead view image. This is a diagram showing an example of the processing flow when generating an overhead view image. This is a diagram showing the extraction of overhead view images including each parked vehicle. This is a flowchart showing an example of the processing flow when the display unit of a parked vehicle displays a second overhead view image. This is a diagram showing an empty space in the parking lot. This is a diagram showing an empty parking space in the parking lot.

[0009] The driver assistance system according to this embodiment assists in driving a vehicle within a predetermined area. For example, the driver assistance system assists in parking a vehicle when it is parked within the designated parking space lines in a parking lot.

[0010] A driver assistance system, for example, can display parking space markings on the parked vehicle even if the driver cannot see them due to snow or other factors, allowing the driver to accurately recognize the location of the markings. This prevents accidents and confusion during parking. Furthermore, it can keep installation and operating costs lower compared to implementing physical measures (such as snow melting systems).

[0011] Figure 1 is a block diagram showing a schematic system configuration of the driver assistance system according to this embodiment. The driver assistance system 1 according to this embodiment includes at least one camera 2 installed in a parking lot, a driver assistance device 3 that assists in parking by a parked vehicle, and a display unit 4 installed in the parked vehicle.

[0012] Each camera 2, the driver assistance device 3, and the parked vehicle are configured to send and receive data from each other, for example, by wireless communication.

[0013] Each camera 2 is installed in the parking lot to film the area within the parking lot. While multiple cameras 2 are typically installed in the parking lot to film the entire area, this is not the only option. A single camera 2 may be installed in the parking lot as long as it can film the entire area, and the number, location, and method of installation are arbitrary. Camera 2 may also be installed on vehicles within the parking lot. Camera 2 may also be an existing security camera in the parking lot or a dashcam camera installed on a vehicle within the parking lot.

[0014] Each camera 2 takes pictures of the parking lot. Each camera 2 transmits the acquired image information of the parking lot to the driver assistance system 3.

[0015] The driver assistance device 3 has a hardware configuration similar to that of a normal computer, comprising, for example, a processor 3a such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit), internal memory 3b such as RAM (Random Access Memory) or ROM (Read Only Memory), storage devices 3c such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), input / output I / F 3d for connecting peripheral devices such as a display, and communication I / F 3e for communicating with devices outside the device.

[0016] Figure 2 is a block diagram showing a schematic system configuration of the driver assistance device according to this embodiment. The driver assistance device 3 according to this embodiment includes an image acquisition unit 31, a storage unit 32, a vehicle detection unit 33, and an image generation unit 34.

[0017] The driver assistance device 3 is installed near the parking lot, but is not limited to this location; it may also be installed at a location away from the parking lot. The image acquisition unit 31 and the image generation unit 34 may be configured to be installed on the parked vehicle.

[0018] The image acquisition unit 31 is a specific example of an image acquisition means. The image acquisition unit 31 acquires image information of the parking lot captured by each camera 2.

[0019] The memory unit 32 is a specific example of a storage means. The memory unit 32 is composed of, for example, a storage device 3c. The memory unit 32 stores image information of the parking space markings within the parking lot.

[0020] Figure 3 is a flowchart showing an example of a processing flow when detecting and storing image information of road markings.

[0021] For example, the driver assistance device 3 starts the initial setup process in response to an initial setup start instruction signal from the vehicle's control panel or the like (step S101). It is assumed that the parking lot manager or the like confirms that the parking lot lines are not hidden by obstacles such as snow before issuing the initial setup start instruction.

[0022] Each camera 2 takes a picture of the parking lot and transmits the image information to the driver assistance device 3 (step S102). The driver assistance device 3 detects image information of the parking lot lines from the transmitted image information of the parking lot (step S103).

[0023] The memory unit 32 stores the image information of the parking space lines detected within the parking lot (step S104). The driver assistance device 3 terminates the initial setup process in response to an instruction signal from the parking vehicle's control panel or the like.

[0024] The driver assistance device 3 may perform the above processing at predetermined time intervals (e.g., once every XX days), or each time it communicates with a parked vehicle entering the parking lot.

[0025] The vehicle detection unit 33 is a specific example of a vehicle detection means. Based on the image information of the parking lot acquired by the image acquisition unit 31, the vehicle detection unit 33 performs predetermined image processing to detect information about parked vehicles parked in parking spaces. The information about parked vehicles includes identification information of the parked vehicle (license plate information of the parked vehicle, color information of the parked vehicle, vehicle type information of the parked vehicle, etc.) and location information of the parked vehicle (coordinate information, etc.).

[0026] The vehicle detection unit 33 may use GNSS (Global Navigation Satellite System) information of the parked vehicle in addition to image information of the parking lot to detect the location of the parked vehicle with greater accuracy. Furthermore, if the position accuracy of the GNSS information of the parked vehicle is low, the vehicle detection unit 33 may combine the GNSS information of the parked vehicle with the radio wave strength transmitted from the parked vehicle to detect the location of the parked vehicle with greater accuracy.

[0027] The vehicle detection unit 33 may detect information about a parked vehicle by identifying that parked vehicle in the parking space using a learning model that has been pre-trained to take images of parked vehicles parked in the parking space.

[0028] Furthermore, the vehicle detection unit 33 may detect information about the parked vehicle based on a parking start signal transmitted from the parked vehicle.

[0029] The image generation unit 34 is a specific example of an image generation means. Based on the image information of the parking lot acquired by the image acquisition unit 31, the image generation unit 34 generates a second image based on the first image of the parking lot when it determines that the parking space lines are hidden by obstacles and cannot be seen. The image generation unit 34 can determine whether or not the parking space lines are hidden by obstacles and cannot be seen by comparing the image information of the parking lot acquired by the image acquisition unit 31 with the image information of the parking space lines stored in the storage unit 32.

[0030] When the above determination is made, the image generation unit 34 generates a second image by superimposing the parking lot lines and parked vehicles onto the first image of the parking lot, based on the image information of the parking lot lines stored in the storage unit 32, the image information of the parking lot acquired by the image acquisition unit 31, and the information of parked vehicles detected by the vehicle detection unit 33.

[0031] Furthermore, the image generation unit 34 may generate a second overhead image by superimposing the parking lot lines and parked vehicles onto a first overhead image of the parking lot, based on the image information of the parking lot lines stored in the storage unit 32, the image information of the parking lot acquired by the image acquisition unit 31, and the information of parked vehicles detected by the vehicle detection unit 33. By viewing such an overhead image, drivers can more clearly confirm the relative positions of parked vehicles and parking lot lines.

[0032] For example, the image generation unit 34 generates a first overhead view image of the parking lot, as shown in Figure 4, based on the image information of the parking lot acquired by the image acquisition unit 31 and the parked vehicles detected by the vehicle detection unit 33. At this time, the image generation unit 34 determines, based on the first overhead view image, that the parking lot lines are hidden and not visible due to obstacles such as snow.

[0033] If the above determination is made, the image generation unit 34 generates a second overhead image, as shown in Figure 5, by superimposing the parking lot lines onto the first overhead image, based on the image information of the parking lot lines stored in the storage unit 32.

[0034] In this second overhead view, the lane markings are clearly visible and not obscured by obstacles such as snow. Therefore, drivers can clearly recognize the relative position of parked vehicles to the lane markings.

[0035] As described above, the image generation unit 34 generates a second overhead image based on the image information of the parking lot lines stored in the storage unit 32, the image information of the parking lot acquired by the image acquisition unit 31, and the information of parked vehicles detected by the vehicle detection unit 33, but is not limited to this. The image generation unit 34 may also generate a second image other than an overhead image, such as a two-dimensional or three-dimensional image, as long as the parking lot lines and parked vehicles can be clearly recognized.

[0036] Figure 6 shows an example of the processing flow when generating an overhead view image. The parked vehicle transmits a parking start signal to the driver assistance device 3 to indicate the start of parking (step S201). The parking start signal includes identification information of the parked vehicle.

[0037] The image acquisition unit 31 of the driver assistance device 3 acquires image information of the parking lot from each camera 2 (step S202). The vehicle detection unit 33 detects information of parked vehicles parked in parking spaces based on the image information of the parking lot acquired by the image acquisition unit 31 (step S203).

[0038] If the image generation unit 34 determines that the parking space lines are hidden, it generates a second overhead view image based on the image information of the parking space lines within the parking lot stored in the memory unit 32, the image information of the parking lot acquired by the image acquisition unit 31, and the information of parked vehicles detected by the vehicle detection unit 33 (step S204).

[0039] The driver assistance device 3 transmits the second overhead view image generated by the image generation unit 34 to the parked vehicle (step S205).

[0040] When the parked vehicle has completed parking, it transmits a parking completion signal to the driver assistance device 3 (step S206). When the driver assistance device 3 receives the parking completion signal from the parked vehicle, it stops generating and transmitting the second overhead view image.

[0041] In addition, when there are a plurality of parked vehicles present simultaneously, the image generation unit 34 may cut out from the generated second bird's-eye view image, bird's-eye view images each including the parked vehicles. For example, as shown in FIG. 7, for example, the image generation unit 34 cuts out from the second bird's-eye view image, a bird's-eye view image A1 including the parked vehicle A and a bird's-eye view image B1 including the parked vehicle B respectively. The driving support device 3 transmits the cut-out bird's-eye view image A1 including the parked vehicle A to the parked vehicle A, and transmits the cut-out bird's-eye view image B1 including the parked vehicle B to the parked vehicle A.

[0042] The parked vehicle includes a display unit 4 that displays the second bird's-eye view image transmitted from the driving support device 3. The display unit 4 is a specific example of display means. The display unit 4 is provided, for example, on the front side of the parked vehicle and is a display device that performs navigation, vehicle information display, and the like. The display device is composed of, for example, a liquid crystal display device, an organic EL (electro-luminescence) display device, an LED (Light Emitting Diode) display device, and the like.

[0043] FIG. 8 is a flowchart showing an example of a processing flow when the display unit of the parked vehicle displays the second bird's-eye view image.

[0044] When starting parking, the driver of the parked vehicle performs an operation (button operation, touch panel operation, switch operation, etc.) serving as a trigger on the operation unit of the parked vehicle (step S301).

[0045] The parked vehicle transmits a parking start signal indicating parking start to the driving support device 3 (step S302). Note that the parked vehicle may automatically determine the start of parking and transmit a parking start signal to the driving support device 3. For example, when communication between the driving support device 3 and the parked vehicle is established, the parked vehicle may determine the start of parking and transmit a parking start signal to the driving support device 3.

[0046] The parked vehicle receives the second bird's-eye view image from the driving support device 3 (step S303). The display unit 4 of the parked vehicle displays the second bird's-eye view image received from the driving support device 3 (step S304).

[0047] When parking is completed, the driver of the parked vehicle performs an operation serving as a trigger on the operation unit of the parked vehicle (step S305). The parked vehicle transmits a parking completion signal to the driving support device 3 (step S306).

[0048] Note that the parked vehicle may automatically determine that parking is completed and transmit a parking completion signal to the driving support device 3. For example, when communication between the driving support device 3 and the parked vehicle is established, the parked vehicle may determine that parking is completed and transmit a parking completion signal to the driving support device 3.

[0049] As described above, according to the driving support system 1, even when the boundary line becomes invisible due to snow or the like, the driver can accurately recognize the boundary line, so that accidents and confusion during parking can be prevented.

[0050] Embodiment 2 In this embodiment, the driving support system 1 may support the driving of a vehicle within a predetermined area other than a parking lot.

[0051] For example, it is assumed that road boundary lines (such as roadside lines) indicating the driving lanes of vehicles, stop lines where vehicles temporarily stop, crosswalk lines where pedestrians cross, road signs, etc., provided within a predetermined area such as in the vicinity of an intersection, are hidden and invisible due to obstacles such as snow.

[0052] In the case described above, the driving support system 1 according to this embodiment can support the driving of the vehicle by causing the hidden road boundary lines, stop lines, crosswalk lines, road signs, etc. to be displayed on the display unit 4 of the vehicle.

[0053] For example, the image acquisition unit 31 of the driving support device 3 acquires image information within a predetermined area from each camera 2. The vehicle detection unit 33 detects information on a traveling vehicle traveling within the predetermined area based on the image information within the predetermined area acquired by the image acquisition unit 31.

[0054] When the image generation unit 34 determines that road markings, stop lines, pedestrian crossing lines, road signs, etc. are hidden, it generates a second image by superimposing the road markings, stop lines, pedestrian crossing lines, road signs, etc. onto the first image within the predetermined area, based on image information of road markings, stop lines, pedestrian crossing lines, road signs, etc. within a predetermined area stored in the memory unit 32, image information of the predetermined area acquired by the image acquisition unit 31, and information of moving vehicles detected by the vehicle detection unit 33.

[0055] The driver assistance device 3 transmits the second image generated by the image generation unit 34 to the vehicle. The vehicle receives the second image from the driver assistance device 3, and the vehicle's display unit 4 displays the second image received from the driver assistance device 3.

[0056] Embodiment 3 Figure 9 is a block diagram showing a schematic system configuration of the driver assistance device according to this embodiment. The driver assistance device 20 according to this embodiment may further include a guide unit 35 that provides guidance to parked vehicles in a parking lot. The guide unit 35 is one specific example of a guide means.

[0057] If the guidance unit 35 determines that a parked vehicle has entered the parking lot, it may guide the parked vehicle to an available parking space (hereinafter referred to as an available parking space). For example, the guidance unit 35 guides the parked vehicle to park by transmitting location information of an available parking space to the parked vehicle. The display unit 4 of the parked vehicle displays the location information of the available parking space transmitted from the guidance unit 35 (location on the map and direction of the available parking space).

[0058] For example, the guidance unit 35 detects available parking spaces in the parking lot based on image information of the parking lot acquired by the image acquisition unit 31, as shown in Figure 10. The vehicle detection unit 33 detects information about parked vehicles based on image information of the parking lot acquired by the image acquisition unit 31.

[0059] The guidance unit 35 identifies a parked vehicle that has entered the parking lot based on the detected vacant parking space and the information of the parked vehicle, and provides guidance to that parked vehicle regarding a vacant parking space. The guidance unit 35 may also provide guidance to a vacant parking space by, for example, transmitting the location of the vacant parking space to the identified parked vehicle.

[0060] Furthermore, the guidance unit 35 may detect the size of a parked vehicle entering the parking lot and, based on the detected vehicle size, provide information about the availability of parking spaces to that vehicle.

[0061] Based on the image information of the parking lot acquired by the image acquisition unit 31, the guidance unit 35 detects, for example, available parking spaces in the parking lot and the size of those available parking spaces, as shown in Figure 10.

[0062] The vehicle detection unit 33 detects information about parked vehicles based on image information of the parking lot acquired by the image acquisition unit 31, and detects the size of the parked vehicles based on the detected information about the parked vehicles.

[0063] The guidance unit 35 identifies a parked vehicle that is close in size to the detected vacant parking space, based on the size of the detected vacant parking space and the size of the parked vehicle. The guidance unit 35 then guides the identified parked vehicle to park in that vacant parking space. For example, in Figure 10, the guidance unit 35 guides the parked vehicle (a small vehicle) to the small vehicle section P2.

[0064] In this case, if the guidance unit 35 determines, based on the image information of the parking lot acquired by the image acquisition unit 31, that there are no available parking spaces in the parking lot, it may send a notification to other parked vehicles in the parking lot informing them that the parking lot is full.

[0065] The driving assistance device 20 according to this embodiment may further include a parking determination unit 36 ​​that determines whether or not a parked vehicle is properly parked in a parking space within the parking lot based on image information of the parking lot acquired by the image acquisition unit 31.

[0066] The parking determination unit 36 ​​is one specific example of a parking determination means. If the parking determination unit 36 ​​determines, for example, that a parked vehicle is not properly parked within a parking space, such as when the parked vehicle is protruding from the parking space, it may notify a designated terminal such as an administrator or driver of this fact. The designated terminal is, for example, a mobile device such as a smartphone or tablet.

[0067] The guidance unit 35 may dynamically change the size and position of the parking spaces demarcated by the parking lines in the parking lot. The guidance unit 35 may dynamically change the size and position of the parking spaces in the parking lot according to date and time information and the degree of vehicle congestion in the parking lot.

[0068] For example, in the parking lots of highway rest areas / service areas, there are many regular cars on Saturdays and Sundays, and many large vehicles on weekdays. Therefore, if the guidance unit 35 determines, based on the date and time information, that it is a Saturday or Sunday, it will allocate more parking spaces for regular cars in the parking lot. On the other hand, if the guidance unit 35 determines, based on the date and time information, that it is a weekday, it will allocate more parking spaces for large vehicles in the parking lot.

[0069] The guidance unit 35 provides guidance to parked vehicles in the designated parking spaces, as described above, based on the parking spaces of the parking lot set out above.

[0070] Furthermore, this disclosure can also be implemented, for example, by having a processor execute a computer program to perform the processes shown in Figures 3, 6, or 8.

[0071] Programs can be stored and supplied to a computer using various types of non-transitory computer-readable medium. Non-transitory computer-readable medium includes various types of tangible storage medium. Examples of non-transitory computer-readable medium include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memory (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (random access memory)).

[0072] Programs may be supplied to a computer by various types of transient computer-readable medium. Examples of transient computer-readable medium include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable medium can be supplied to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0073] Each component of the driver assistance devices 3 and 20 according to the above-described embodiment can be implemented not only by program, but also partially or entirely by dedicated hardware such as ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array).

[0074] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0075] Each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments, rather than being associated with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps described in any of the drawings may be changed as appropriate.

[0076] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) A driving assistance system comprising: a storage means for storing image information of a predetermined object which includes at least one of the following provided within a predetermined area: road markings indicating vehicle driving lanes, stop lines where vehicles must temporarily stop, pedestrian crossing lines where pedestrians must cross, road signs, and markings that demarcate parking spaces in a parking lot; an image acquisition means for acquiring image information within the predetermined area; a vehicle detection means for detecting information of a vehicle traveling within the predetermined area based on the image information within the predetermined area acquired by the image acquisition means; an image generation means for generating a second image which superimposes the predetermined object and the vehicle onto the image within the predetermined area when it is determined that the predetermined object is not visible due to an obstacle based on the image information within the predetermined area acquired by the image acquisition means; and a display means for displaying the second image generated by the image generation means in the vehicle. (Note 2) A driving assistance system as described in Note 1, wherein the storage means stores image information of parking space lines within a parking lot; the image acquisition means acquires image information of the parking lot; the vehicle detection means detects information of a parked vehicle parked in the parking space based on the image information of the parking lot acquired by the image acquisition means; the image generation means, when it determines that the parking space lines are not visible due to an obstacle based on the image information of the parking lot acquired by the image acquisition means, generates a second image in which the parking space lines and the parked vehicle are superimposed on a first image of the parking lot based on the image information of the parking lot stored in the storage means, the image information of the parking lot acquired by the image acquisition means, and the information of the parked vehicle detected by the vehicle detection means; and the display means displays the second image generated by the image generation means in the parked vehicle.(Note 3) A driving support system according to Note 1, wherein the image generation means generates a second overhead image by superimposing the parking lot lines and the parked vehicle onto a first overhead image of the parking lot, based on the image information of the parking lot lines stored in the storage means, the image information of the parking lot acquired by the image acquisition means, and the information of the parked vehicle detected by the vehicle detection means, and the display means displays the second overhead image generated by the image generation means on the parked vehicle. (Note 4) A driving support system according to Note 1, further comprising a guidance means for providing guidance to a parked vehicle in a parking lot. (Note 5) A driving assistance system as described in Note 4, wherein the guidance means detects an available parking space in the parking lot based on image information of the parking lot acquired by the image acquisition means; the vehicle detection means detects information of a parked vehicle based on image information of the parking lot acquired by the image acquisition means; and the guidance means identifies a parked vehicle that has entered the parking lot based on the available parking space and the information of the parked vehicle, and guides the identified parked vehicle to the available parking space. (Note 6) A driving assistance system as described in Note 4, wherein the guidance means detects the size of an available parking space in the parking lot based on image information of the parking lot acquired by the image acquisition means; the vehicle detection means detects information of a parked vehicle based on image information of the parking lot acquired by the image acquisition means, and detects the size of the parked vehicle based on the detected information of the parked vehicle; the guidance means identifies a parked vehicle that is close in size to the size of the available parking space based on the size of the available parking space and the size of the parked vehicle, and guides the identified parked vehicle to park in the available parking space. (Note 7) A driving assistance system as described in Note 4, wherein the guidance means, based on image information of the parking lot acquired by the image acquisition means, determines that there are no available parking spaces in the parking lot and provides a notification to parked vehicles in the parking lot informing them that the parking lot is full.(Note 8) A driving assistance system as described in Note 4, further comprising a parking determination means that, based on image information of the parking lot acquired by the image acquisition means, determines that a parked vehicle is not properly parked in a parking space within the parking lot and notifies a predetermined terminal of that fact. (Note 9) A driving assistance method comprising: storing image information of a predetermined object which includes at least one of the following provided within a predetermined area: road lane markings indicating a vehicle's driving lane, a stop line where a vehicle should stop temporarily, a pedestrian crossing line where a pedestrian should cross, a road sign, and markings that demarcate parking spaces in a parking lot; acquiring image information within the predetermined area; detecting information of a vehicle traveling within the predetermined area based on the acquired image information within the predetermined area; generating a second image which superimposes the predetermined object and the vehicle onto the image within the predetermined area, based on the stored image information of the predetermined object, the acquired image information within the predetermined area, and the detected vehicle information, when the predetermined object is not visible due to an obstacle; and displaying the generated second image in the vehicle. (Note 10) A program that causes a computer to execute the following: a process of storing image information of a predetermined object which includes at least one of the following provided within a predetermined area: road lane markings indicating vehicle driving lanes, stop lines where vehicles must stop temporarily, pedestrian crossing lines where pedestrians must cross, road signs, and markings that demarcate parking spaces in a parking lot; a process of acquiring image information within the predetermined area; a process of detecting information of a vehicle driving within the predetermined area based on the acquired image information within the predetermined area; a process of generating a second image which superimposes the predetermined object and the vehicle onto the image within the predetermined area based on the stored image information of the predetermined object, the acquired image information within the predetermined area, and the detected vehicle information, when the predetermined object is not visible due to an obstacle; and a process of displaying the generated second image in the vehicle.

[0077] Some or all of the elements (e.g., configuration and function) described in Appendices 2 to 8 that are dependent on Appendice 1 {e.g., device} may also be dependent on Appendices 9 {e.g., method} and 10 {e.g., program} in the same way as in Appendices 2 to 8. Some or all of the elements described in any appendice may be applied to various hardware, software, recording means, systems, and methods for recording software.

[0078] 1. Driving assistance system 2. Camera 3. Driving assistance device 3a. Processor 3b. Internal memory 3c. Storage device 20. Driving assistance device 31. Image acquisition unit 32. Storage unit 33. Vehicle detection unit 34. Image generation unit 35. Guidance unit 36. Parking determination unit

Claims

1. A driving assistance system comprising: a storage means for storing image information of a predetermined object, which includes at least one of the following within a predetermined area: road lane markings indicating vehicle driving lanes, stop lines where vehicles must temporarily stop, pedestrian crossing lines where pedestrians must cross, road signs, and markings that demarcate parking spaces in a parking lot; an image acquisition means for acquiring image information within the predetermined area; a vehicle detection means for detecting information of a vehicle traveling within the predetermined area based on the image information within the predetermined area acquired by the image acquisition means; an image generation means for generating a second image in which the predetermined object and the vehicle are superimposed on the image within the predetermined area, based on the image information within the predetermined area acquired by the image acquisition means and when it is determined that the predetermined object is not visible due to an obstacle; and a display means for displaying the second image generated by the image generation means in the vehicle.

2. A driving assistance system according to claim 1, wherein the storage means stores image information of parking space lines within a parking lot; the image acquisition means acquires image information of the parking lot; the vehicle detection means detects information of a parked vehicle parked in the parking space based on the image information of the parking lot acquired by the image acquisition means; the image generation means, when it determines that the parking space lines are not visible due to an obstacle based on the image information of the parking lot acquired by the image acquisition means, generates a second image in which the parking space lines and the parked vehicle are superimposed on a first image of the parking lot based on the image information of the parking lot stored in the storage means, the image information of the parking lot acquired by the image acquisition means, and the information of the parked vehicle detected by the vehicle detection means; and the display means displays the second image generated by the image generation means in the parked vehicle.

3. A driving assistance system according to claim 1, wherein the image generation means generates a second overhead image in which the parking lot lines and the parked vehicle are superimposed on a first overhead image of the parking lot, based on image information of the parking lot lines stored in the storage means, image information of the parking lot acquired by the image acquisition means, and information of the parked vehicle detected by the vehicle detection means; and the display means displays the second overhead image generated by the image generation means on the parked vehicle.

4. A driver assistance system according to claim 1, further comprising guidance means for providing guidance to a parked vehicle in a parking lot.

5. A driving assistance system according to claim 4, wherein the guidance means detects an available parking space in the parking lot based on image information of the parking lot acquired by the image acquisition means; the vehicle detection means detects information of a parked vehicle based on image information of the parking lot acquired by the image acquisition means; and the guidance means identifies a parked vehicle that has entered the parking lot based on the available parking space and the information of the parked vehicle, and guides the identified parked vehicle to the available parking space.

6. A driving assistance system according to claim 4, wherein the guidance means detects the size of an empty parking space in the parking lot based on image information of the parking lot acquired by the image acquisition means; the vehicle detection means detects information of a parked vehicle based on image information of the parking lot acquired by the image acquisition means, and detects the size of the parked vehicle based on the detected information of the parked vehicle; and the guidance means identifies a parked vehicle that is close in size to the size of the empty parking space based on the size of the empty parking space and the size of the parked vehicle, and guides the identified parked vehicle to park in the empty parking space.

7. A driving assistance system according to claim 4, wherein the guidance means, based on image information of the parking lot acquired by the image acquisition means, determines that there are no available parking spaces in the parking lot and provides a notification to parked vehicles in the parking lot indicating that the parking lot is full.

8. A driving assistance system according to claim 4, further comprising a parking determination means that, based on image information of the parking lot acquired by the image acquisition means, determines that a parked vehicle is not properly parked in a parking space within the parking lot, and notifies a predetermined terminal of that fact.

9. A driving assistance method comprising: storing image information of a predetermined object which includes at least one of the following within a predetermined area: road markings indicating vehicle driving lanes, stop lines where vehicles must temporarily stop, pedestrian crossing lines where pedestrians must cross, road signs, and markings that demarcate parking spaces in a parking lot; acquiring image information within the predetermined area; detecting information of a vehicle traveling within the predetermined area based on the acquired image information within the predetermined area; generating a second image which superimposes the predetermined object and the vehicle onto the image within the predetermined area, based on the stored image information of the predetermined object, the acquired image information within the predetermined area, and the detected vehicle information, when the predetermined object is obscured by an obstacle; and displaying the generated second image in the vehicle.

10. A program that causes a computer to perform the following steps:

10. A process of storing image information of a predetermined object which includes at least one of the following within a predetermined area: road markings indicating vehicle driving lanes, stop lines where vehicles must temporarily stop, pedestrian crossing lines where pedestrians must cross, road signs, and markings that demarcate parking spaces in a parking lot; 10. A process of acquiring image information within the predetermined area; 11. A process of detecting information of a vehicle traveling within the predetermined area based on the acquired image information within the predetermined area; 22. A process of generating a second image in which the predetermined object and the vehicle are superimposed on the image within the predetermined area, based on the stored image information of the predetermined object, the acquired image information within the predetermined area, and the detected vehicle information, when the predetermined object is not visible due to an obstacle; 33. A process of displaying the generated second image in the vehicle.

Images