Vehicle surrounding information display device and vehicle surrounding information display method

Abstract

To display peripheral information in a range wider than an overhead view image.SOLUTION: A vehicle peripheral information display device 100 comprises: a peripheral information image generation unit 131 that generates a peripheral information image PS indicating peripheral information of a vehicle 1 on the basis of point group data that indicates a distance from the vehicle 1, obtained by a distance sensor 40; a map image acquisition unit 132 that acquires a circumference map image PG of the vehicle 1; an overhead image acquisition unit 133 that acquires an overhead view image PB in the circumference of the vehicle 1; and a display image generation unit 135 that synthesizes the map image PG, a vehicle mark PM that indicates a position of vehicle 1, the overhead view image PB arranged in the circumference of the vehicle mark PM, and the peripheral information image PS arranged in the circumference of the overhead view image PB to generate a display image PD displayed on a display panel 61.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a vehicle surroundings information display device and a vehicle surroundings information display method. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for displaying an image showing information about the surroundings of a vehicle. For example, Patent Document 1 describes a technique for reducing the influence of smears on a displayed image even when smears occur due to the influence of a highly bright subject when an image of the surroundings of a vehicle is displayed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-134586 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 merely displays an overhead image (so-called surround view) based on an image captured by a camera, and only displays information about the surroundings of the vehicle at a short distance. An object of the present invention is to provide a vehicle surroundings information display device and a vehicle surroundings information display method that can display surroundings information over a wider range than a bird's-eye view image. [Means for solving the problem]

[0005] In order to achieve the above object, the vehicle surroundings information display device of the present invention includes a surroundings information image generation unit that generates a surroundings information image that shows surrounding information of the vehicle based on point cloud data that indicates a distance from the vehicle obtained from a sensor, a map image acquisition unit that acquires a map image of the periphery of the vehicle, and a display image generation unit that generates a display image to be displayed on a display by combining the map image, a vehicle mark that indicates the position of the vehicle, the overhead image arranged around the vehicle mark, and the surroundings information image arranged around the overhead image, wherein the area in which the overhead image is displayed is a circular area centered on the vehicle mark, and the display image generation unit includes a risk detection unit that detects a moving object that is at risk of contact with the vehicle based on the point cloud data, and the display image generation unit highlights a part of a circumferential boundary line between the circular area and the area in which the surroundings information image is displayed in a direction in which there is a risk of the vehicle coming into contact with the moving object, using the vehicle as a reference. death , The display image generation unit generates a collision risk image, which is an image showing the direction of travel of a first vehicle that is at risk of colliding with the vehicle and the area in which the vehicle is at risk of colliding with the first vehicle, and the risk detection unit determines a first direction display indicating the direction in which the vehicle is at risk of colliding with the first vehicle as the area in which the collision risk image overlaps with the circular boundary line, and the display image generation unit highlights the first direction display by superimposing it on part of the boundary line.

[0006] In order to achieve the above object, the vehicle surroundings information display method of the present invention includes a surroundings information image generation step of generating a surroundings information image showing surrounding information of the vehicle based on point cloud data showing a distance from the vehicle obtained from a sensor, a map image acquisition step of acquiring a map image of the periphery of the vehicle, an overhead image acquisition step of acquiring an overhead image of the periphery of the vehicle, and a display image generation step of synthesizing the map image, a vehicle mark showing the position of the vehicle, the overhead image arranged around the vehicle mark, and the surroundings information image arranged around the overhead image to generate a display image to be displayed on a display, wherein in the display image generation step, a part of a circumferential boundary line between the circular area and an area where the surroundings information image is displayed is highlighted in a direction in which there is a risk of the vehicle coming into contact with the moving object, with the vehicle as a reference. death , In the display image generation step, a collision risk image is generated, which is an image showing the direction of travel of a first vehicle that is at risk of colliding with the vehicle and the area in which the vehicle is at risk of colliding with the first vehicle.In the risk detection step, a first direction display indicating the direction in which the vehicle is at risk of colliding with the first vehicle is obtained as the area in which the collision risk image overlaps with the circular boundary line.In the display image generation step, the first direction display is superimposed on part of the boundary line and highlighted. [Effects of the Invention]

[0007] According to the vehicle surroundings information display device and vehicle surroundings information display method of the present invention, a display image to be displayed on a display is generated by combining a map image, a vehicle mark indicating the position of the vehicle, an overhead image arranged around the vehicle mark, and a surroundings information image arranged around the overhead image, thereby making it possible to display surroundings information over a wider range than with an overhead image. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of the configuration of an in-vehicle device including a vehicle surroundings information display device. [Figure 2] FIG. 10 is a screen diagram showing an example of a first screen displaying a display image in a first state. [Figure 3] FIG. 10 is a screen diagram showing an example of a second screen displaying a display image in a second state. [Figure 4] FIG. 10 is a screen diagram showing an example of a third screen displaying a display image in a third state. [Figure 5] FIG. 10 is a screen diagram showing an example of a fourth screen displaying a display image in a fourth state. [Figure 6] FIG. 10 is a screen diagram showing an example of a fifth screen displaying a display image in a fifth state. [Figure 7] FIG. 10 is a screen diagram showing an example of a sixth screen displaying a display image in a sixth state. [Figure 8] FIG. 10 is a screen diagram showing an example of a seventh screen displaying a display image in a seventh state. [Figure 9] FIG. 10 is a screen diagram showing an example of an eighth screen displaying a display image in an eighth state. [Figure 10] FIG. 10 is a screen diagram showing an example of a ninth screen displaying a display image in a ninth state. [Figure 11] 4 is a flowchart illustrating an example of processing performed by the vehicle surrounding information display device. [Figure 12] 10 is a flowchart showing an example of a danger display process of the vehicle surrounding information display device. [Figure 13] 10 is a flowchart showing an example of a danger display process of the vehicle surrounding information display device. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. 1 is a diagram showing an example of the configuration of an in-vehicle device 3 including a vehicle surroundings information display device 100. The in-vehicle device 3 is mounted on a vehicle 1. The in-vehicle device 3 includes a position detection unit 10, a detection section 20, an operation section 50, a display section 60, and a vehicle surrounding information display device 100.

[0010] The position detection unit 10 detects the position of the vehicle 1. The position detection unit 10 includes a GNSS (Global Navigation Satellite System) receiver that receives GNSS signals, and a processor that calculates the position of the vehicle 1 based on the GNSS signals received by the GNSS receiver. The GNSS receiver and processor are not shown in the figure. The position detection unit 10 outputs position information indicating the position of the vehicle 1 to the vehicle surrounding information display device 100.

[0011] The detection unit 20 captures images of the surroundings of the vehicle 1 and detects point cloud data indicating the distance between the vehicle 1 and moving objects MS that exist within a range of a predetermined distance from the vehicle 1. The moving objects MS include other vehicles C, bicycles, and pedestrians. The predetermined distance is, for example, 20 m. The detection unit 20 includes an imaging unit 30 and a distance sensor 40 .

[0012] The photographing unit 30 photographs images of the surroundings of the vehicle 1. The photographing unit 30 includes a front camera 31 that photographs the area in front of the vehicle 1, a rear camera 33 that photographs the area behind the vehicle 1, a left side camera 35 that photographs the area to the left of the vehicle 1, and a right side camera 37 that photographs the area to the right of the vehicle 1. Each of these cameras includes an image sensor such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor), and a data processing circuit that generates a photographed image from the image sensor. The imaging unit 30 adjusts the angle of view of each of the cameras facing in four directions so that they can capture a 360° range centered on the vehicle 1. Each of the front camera 31, rear camera 33, left side camera 35, and right side camera 37 captures the capture range at a predetermined frame rate to generate a captured image. The imaging unit 30 outputs the generated captured image to the vehicle periphery information display device 100. The vehicle periphery information display device 100 stores the input captured image in memory 140. Each of the front camera 31, rear camera 33, left side camera 35, and right side camera 37 may be configured with one camera or multiple cameras.

[0013] The distance sensor 40 detects point cloud data indicating the distance between the vehicle 1 and a moving object MS that exists within a range of a predetermined distance from the vehicle 1. The distance sensor 40 is equipped with LiDAR (Light Detection and Ranging) at multiple locations, for example, in front, behind, left and right sides of the vehicle 1, and acquires the point cloud data using electromagnetic waves. Each point data constituting the point cloud data indicates the distance between the vehicle 1 and a moving object MS that exists within a range of a predetermined distance from the vehicle 1. In this embodiment, the distance sensor 40 is a LiDAR, but is not limited to this. The distance sensor 40 may be, for example, a radar or sonar sensor. Furthermore, in this embodiment, a case will be described in which the LiDAR is disposed at a plurality of locations such as the front, rear, left side, and right side of the vehicle 1, but the present invention is not limited to this. The LiDAR may be disposed on the roof of the vehicle 1, and point cloud data of the entire periphery (i.e., the front, rear, left side, and right side) may be acquired. The distance sensor 40 corresponds to an example of a "sensor."

[0014] The operation unit 50 accepts operations from a user in the vehicle 1. The user is, for example, the driver. The operation unit 50 outputs an operation signal corresponding to the accepted operation to the vehicle periphery information display device 100. The operations accepted by the operation unit 50 include, for example, an operation to instruct the start of the vehicle periphery information display process, an operation to end the vehicle periphery information display process, etc. The operation unit 50 includes, for example, a vehicle periphery information display ON switch (not shown) and a vehicle periphery information display OFF switch (not shown), and when the vehicle periphery information display ON switch is pressed, the vehicle periphery information display device 100 accepts an operation to instruct the start of vehicle periphery information display processing. Also, when the vehicle periphery information display OFF switch is pressed during execution of the vehicle periphery information display processing, the vehicle periphery information display device 100 accepts an operation to end the vehicle periphery information display processing. The "vehicle surrounding information display process" is a process in which the vehicle surrounding information display device 100 synthesizes the map image PG, the vehicle mark PM, the overhead image PB, and the surrounding information image PS to generate a display image PD, and displays the generated display image PD on the display panel 61. The "vehicle surroundings information display process" will be described with reference to FIG. 1 as to the functional configuration of the vehicle surroundings information display device 100, and will be further described with reference to FIGS. The map image PG, the vehicle mark PM, the overhead image PB, the surrounding information image PS, and the display image PD will be described with reference to FIGS.

[0015] The display unit 60 includes a display panel 61 and a touch sensor 63 . A liquid crystal display, an organic EL display, or the like is used for the display panel 61. The display unit 60 displays images such as a display image PD on the display panel 61 based on display data input from the vehicle surrounding information display device 100. The display panel 61 corresponds to an example of a "display." The display image PD will be described with reference to FIGS. The touch sensor 63 may be a resistive sensor, a capacitive sensor, or the like. The display unit 60 detects a touch operation of the user's finger on the display panel 61 using the touch sensor 63, and generates a position signal indicating the operation position of the detected touch operation. The display unit 60 outputs the generated position signal to the vehicle surrounding information display device 100.

[0016] The vehicle surrounding information display device 100 is a computer including a processor 130 such as a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit), and a memory 140 such as a ROM (Read Only Memory) or a RAM (Random Access Memory). In addition to these devices, the vehicle surrounding information display device 100 also includes a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), an interface circuit for connecting sensors and peripheral devices, and an in-vehicle network communication circuit for communicating with other in-vehicle devices via an in-vehicle network. The vehicle surrounding information display device 100 realizes various functional configurations by having the processor 130 execute control programs stored in the memory 140 or the storage device.

[0017] The vehicle surrounding information display device 100 includes a surrounding information image generation unit 131, a map image acquisition unit 132, an overhead image acquisition unit 133, a danger detection unit 134, a display image generation unit 135, a display control unit 136, a map memory unit 141, and an image memory unit 142. Specifically, processor 130 executes a control program stored in memory 140 or the storage device, causing processor 130 to function as a surrounding information image generation unit 131, a map image acquisition unit 132, an overhead image acquisition unit 133, a risk detection unit 134, a display image generation unit 135, and a display control unit 136. Furthermore, processor 130 executes a control program stored in memory 140 or the storage device, causing memory 140 to function as a map storage unit 141 and an image storage unit 142.

[0018] The map storage unit 141 stores a map image PG. The map image PG is read by the map image acquisition unit 132.

[0019] The image storage unit 142 stores images such as the vehicle mark PM, the first vehicle image CR1, the first vehicle mark CM1, the second vehicle image CR2, the second vehicle mark CM2, and the pedestrian mark P. The vehicle mark PM indicates the vehicle 1. The vehicle mark PM is placed at the position of the vehicle 1 on the map image PG by the display image generating unit 135. That is, the vehicle mark PM indicates the position of the vehicle 1 on the map image PG. The first vehicle image CR1 is an image of the first vehicle C1 detected by the distance sensor 40. The first vehicle image CR1 is placed at the position of the first vehicle C1 on the map image PG by the display image generation unit 135. The first vehicle C1 is, for example, a four-wheeled passenger car. The first vehicle mark CM1 indicates the first vehicle C1 detected by the distance sensor 40. The first vehicle mark CM1 is placed on the road in the traveling direction of the first vehicle C1 on the map image PG by the display image generation unit 135. The first vehicle mark CM1 is, for example, a pentagonal mark. The second vehicle image CR2 is an image of the second vehicle C2 detected by the distance sensor 40. The second vehicle image CR2 is placed at the position of the second vehicle C2 on the map image PG by the display image generation unit 135. The second vehicle C2 is, for example, a motorcycle. The second vehicle mark CM2 indicates the second vehicle C2 detected by the distance sensor 40. The second vehicle mark CM2 is placed on the road in the traveling direction of the second vehicle C2 on the map image PG by the display image generation unit 135. The second vehicle mark CM2 is, for example, a pentagonal mark. The pedestrian mark P indicates a pedestrian detected by the distance sensor 40. The pedestrian mark P is placed at the position of the pedestrian on the map image PG by the display image generation unit 135. The pedestrian mark P is, for example, a circular mark. The first vehicle mark CM1 and the second vehicle mark CM2 will be further described with reference to FIGS. The first vehicle image CR1 and the second vehicle image CR2 will be further described with reference to FIG. In the following description, the first vehicle C1 and the second vehicle C2 may be referred to as vehicle C when they are not to be distinguished from each other. The pedestrian mark P will be further explained with reference to FIG.

[0020] The surrounding information image generation unit 131 acquires point cloud data from the distance sensor 40, and generates a surrounding information image PS based on the point cloud data, which shows information about the surroundings of the vehicle 1. Each point data constituting the point cloud data indicates the distance from the vehicle 1 to a moving object MS located within a predetermined distance from the vehicle 1. The predetermined distance is, for example, 20 m. The surrounding information image PS includes a first vehicle image CR1, a first vehicle mark CM1, a second vehicle image CR2, a second vehicle mark CM2, and a pedestrian mark P.

[0021] When the first vehicle C1 is included in the area indicated by the map image PG displayed on the display panel 6, the first vehicle image CR1 is placed by the display image generating unit 135 at the position of the first vehicle C1 on the map image PG. When the second vehicle C2 is included in the area indicated by the map image PG displayed on the display panel 6, the second vehicle image CR2 is placed by the display image generating unit 135 at the position of the second vehicle C2 on the map image PG.

[0022] The first vehicle mark CM1 is placed by the display image generation unit 135 on the road in the direction of travel of the first vehicle C1 on the map image PG when the first vehicle C1 is not included in the area indicated by the map image PG displayed on the display panel 6 and the risk detection unit 134 detects that there is a risk of contact with vehicle 1. The second vehicle mark CM2 is placed by the display image generation unit 135 on the road in the direction of travel of the second vehicle C2 on the map image PG when the second vehicle C2 is not included in the area indicated by the map image PG displayed on the display panel 6 and the risk detection unit 134 detects that there is a risk of contact with the second vehicle C2. When a pedestrian detected by the distance sensor 40 is included in the area indicated by the map image PG displayed on the display panel 6, the pedestrian mark P is placed at the position of the pedestrian on the map image PG.

[0023] The map image acquisition unit 132 acquires location information indicating the location of the vehicle 1 from the position detection unit 10, and acquires a map image PG of the surroundings of the vehicle 1 by reading out the map image PG corresponding to the acquired location information from the map memory unit 141.

[0024] In the present embodiment, a case will be described in which the map image acquisition unit 132 acquires a map image PG of the periphery of the vehicle 1 by reading out a map image PG corresponding to the location information from the map storage unit 141, but this is not limiting. The vehicle periphery information display device 100 may be communicably connected to a server device, and the map image acquisition unit 132 may acquire a map image PG of the periphery of the vehicle 1 by receiving the map image PG corresponding to the location information from the server device. In this case, the map storage unit 141 does not need to store the map image PG, and therefore the configuration of the vehicle periphery information display device 100 can be simplified.

[0025] The overhead image acquisition unit 133 generates an overhead image PB of the surroundings of the vehicle 1 by combining the images captured by the front camera 31, the rear camera 33, the left side camera 35, and the right side camera 37, and acquires the overhead image PB of the surroundings of the vehicle 1.

[0026] The risk detection unit 134 detects the risk of contact between the moving object MS and the vehicle 1 based on the point cloud data from the distance sensor 40. The risk detection unit 134 detects, for example, the position of a moving body MS that is at risk of coming into contact with the vehicle 1. The risk detection unit 134 detects, for example, the attributes of the moving body MS that is at risk of coming into contact with the vehicle 1. The attributes of the moving body MS indicate, for example, whether the moving body MS is a vehicle C or a pedestrian. Furthermore, the risk detection unit 134 detects, for example, a direction in which the moving body MS is at risk of coming into contact with the vehicle 1, based on the vehicle 1. Furthermore, the risk detection unit 134 detects, for example, the degree of risk of the moving body MS coming into contact with the vehicle 1. The moving objects MS include other vehicles C, bicycles, and pedestrians.

[0027] The display image generation unit 135 synthesizes the map image PG, the vehicle mark PM indicating the position of the vehicle 1, the overhead image PB arranged around the vehicle mark PM, and the surrounding information image PS arranged around the overhead image PB to generate the display image PD to be displayed on the display panel 61. The surrounding information image PS includes a first vehicle image CR1, a first vehicle mark CM1, a second vehicle image CR2, a second vehicle mark CM2, and a pedestrian mark P.

[0028] On the display panel 61, the area in which the overhead image PB is displayed is a circular first area AR1 centered on the vehicle mark PM. The first area AR1 corresponds to an example of a "circular area." The first area AR1 will be further described with reference to FIG.

[0029] The display image generation unit 135 highlights the circular boundary line CF between the first area AR1 and the second area AR2 in which the surrounding information image PS is displayed, in the direction in which there is a risk of the vehicle 1 coming into contact with the moving body MS, using the vehicle 1 as a reference. For example, the display image generation unit 135 highlights the boundary line CF by displaying the color of the boundary line CF in a color (e.g., red, yellow, etc.) different from other parts of the boundary line CF in a direction where there is a risk of contact with vehicle 1, based on vehicle 1. The boundary line CF will be further described with reference to Figures 2 to 10. In Figures 2 to 10, the highlighted display of the boundary line CF is described as a directional display CFE. The directional display CFE includes a first directional display CFR and a second directional display CFY.

[0030] In this embodiment, a case will be described in which the boundary line CF in the direction where there is a risk of contact with the vehicle 1 is highlighted in a color different from the color of the other parts of the boundary line CF, but this is not limiting. For example, the boundary line CF in the direction where there is a risk of contact with the vehicle 1 may be highlighted by flashing. In this case, the visibility of the boundary line CF in the direction where there is a risk of contact with the vehicle 1 to the user can be improved.

[0031] The display control unit 136 displays the display image PD generated by the display image generation unit 135 on the display panel 61. The display image PD will be further described with reference to FIGS. In this embodiment, a case will be described in which the display control unit 136 displays the display image PD over the entire display surface of the display panel 61. However, the present invention is not limited to this. The display control unit 136 may also display the display image PD in a partial region of the display surface of the display panel 61.

[0032] Next, specific examples of the display image PD displayed on the display panel 61 will be described with reference to FIGS. 2 to 10 each show directions. The upper direction of the figure corresponds to the north direction, the lower direction of the figure corresponds to the south direction, the right direction of the figure corresponds to the east direction, and the left direction of the figure corresponds to the west direction.

[0033] 2 is a screen diagram showing an example of a first screen 700 that displays a display image PD in the first state. The first screen 700 is displayed on the display panel 61 by the display control unit 136. In the first screen 700, a vehicle mark PM and an overhead image PB are displayed in the first area AR1, a map image PG is displayed in the second area AR2, and a boundary line CF is displayed at the boundary between the first area AR1 and the second area AR2.

[0034] The map image PG includes a road image RW and a sidewalk image SW. The road image RW includes a first road image RW1 showing a first road R1 extending in an east-west direction, a second road image RW2 showing a second road R2 extending in a north-south direction, and an intersection image RW3 showing an intersection R3 where the first road R1 and the second road R2 intersect. In this embodiment, there is no traffic light at the intersection R3. As shown in FIG. 2, the vehicle 1 is traveling north on the second road R2 and is approaching an intersection R3.

[0035] In the first state, the distance sensor 40 does not detect any moving object MS. Therefore, the risk detection unit 134 detects that there is no moving object MS that poses a risk of contact with the vehicle 1. In this case, the display image generation unit 135 displays the boundary line CF in a first color that indicates that there is no risk of the vehicle 1 coming into contact with the moving object MS. The display image generating unit 135 displays the boundary line CF in green, for example. Green corresponds to an example of a first color. In other words, displaying the boundary line CF in green indicates that there is no risk of the vehicle 1 coming into contact with the moving object MS. In FIG. 2, the fact that the boundary line CF is green is shown by a dashed line for convenience. The boundary line CF displayed in green may be referred to as the first boundary line CF1.

[0036] The overhead image PB is displayed in a first area AR1. The first area AR1 is a circular area that includes the vehicle 1. The overhead image PB is generated by projectively transforming the images captured by the front camera 31, the rear camera 33, the left-side camera 35, and the right-side camera 37, and then combining the images. Therefore, when the overhead image PB is displayed as a rectangular image, the image is significantly distorted at the positions corresponding to the four corners of the rectangular image, and a discontinuous image is generated at the junction with the map image PG. In contrast, in this embodiment, of the rectangular image generated as the overhead image PB, the image at the four corners is truncated to generate a circular overhead image PB. Therefore, the circular overhead image PB does not include any areas with significant distortion. Therefore, the overhead image PB can be seamlessly connected to the map image PG.

[0037] As explained with reference to Figure 2, the display control unit 136 displays the boundary line CF as the first boundary line CF1, which indicates that there is no risk of the vehicle 1 coming into contact with the moving body MS, so that the user can visually confirm that there is no risk of the vehicle 1 coming into contact with the moving body MS.

[0038] 3 is a screen diagram showing an example of a second screen 710 that displays the display image PD in the second state. The second screen 710 is displayed on the display panel 61 by the display control unit 136. In second screen 710, a vehicle mark PM and an overhead image PB are displayed in first area AR1, similarly to first screen 700 shown in FIG. On the second screen 710, the map image PG, the first vehicle mark CM1, the second vehicle mark CM2, the first hazard detection display ED1, the second hazard detection display ED2, and the collision hazard image ARC are displayed in the second area AR2 by the display control unit 136.

[0039] The first vehicle mark CM1 indicates a first vehicle C1 traveling east on a first road R1 extending east-west and entering an intersection R3. The first vehicle C1 corresponds to an example of a moving object MS detected by the distance sensor 40. The first vehicle C1 is not included in the area shown by the map image PG displayed on the display panel 6, and the risk detection unit 134 detects that there is a risk of contact with the vehicle 1. In this case, the display image generation unit 135 places the first vehicle mark CM1 on the road in the traveling direction of the first vehicle C1 on the map image PG. The display image generation unit 135 places the first vehicle mark CM1, for example, at the western end (left end in FIG. 3) of the first road image RW1.

[0040] When the risk detection unit 134 detects that the first vehicle C1 detected by the distance sensor 40 is at risk of contact with the vehicle 1, the display control unit 136 displays the first risk detection display ED1. The display control unit 136 displays the first risk detection display ED1 at an end of the display panel 61 in the direction of the risk of contact with the vehicle 1 (in this embodiment, the left direction) relative to the vehicle 1. In this embodiment, the display control unit 136 notifies the user of the direction in which a risk of contact with the vehicle 1 is detected by flashing the left edge of the display panel 61 in red for a predetermined period as the first danger detection display ED1. The predetermined period is, for example, three seconds. The color of the first danger detection display ED1 is determined according to the degree of risk that the first vehicle C1 will come into contact with the vehicle 1. The degree of risk that the first vehicle C1 will come into contact with the vehicle 1 is detected by the risk detection unit 134. In this embodiment, it is detected that there is a high risk that the first vehicle C1 will come into contact with the vehicle 1, and the color of the first danger detection display ED1 is determined to be, for example, red.

[0041] The collision risk image ARC is an image that is generated by the image generation unit 135 and displayed by the display control unit 136 when the traveling speed of the first vehicle C1 is equal to or greater than a first speed. The collision risk image ARC is an image that indicates the traveling direction of the first vehicle C1 and the area where there is a risk of collision. The first speed is, for example, 70 km / h. The traveling speed of the first vehicle C1 is, for example, 80 km / h. The collision risk image ARC is displayed on the first road image RW1 in, for example, a semi-transparent red color. The collision risk image ARC is displayed, for example, as a rectangular image extending in the east-west direction. The width of the collision risk image ARC (the size in the short side direction, i.e., the north-south direction) indicates, for example, the width of the first vehicle C1.

[0042] The second vehicle mark CM2 indicates a second vehicle C2 traveling north on a second road R2 extending north-south and entering an intersection R3. The second vehicle C2 corresponds to an example of a moving object MS detected by the distance sensor 40. The traveling speed of the second vehicle C2 is less than a first speed and greater than or equal to a second speed. The second speed is, for example, 30 km / h. The traveling speed of the second vehicle C2 is, for example, 40 km / h. The second vehicle C2 is not included in the area shown in the map image PG displayed on the display panel 6, and the risk detection unit 134 detects that there is a risk of contact with the vehicle 1. In this case, the surrounding information image generation unit 131 places the second vehicle mark CM2 on the road in the traveling direction of the second vehicle C2 on the map image PG. The surrounding information image generation unit 131 places the second vehicle mark CM2, for example, at the southern end (the bottom end in FIG. 3) of the second road image RW2.

[0043] When the risk detection unit 134 detects that the second vehicle C2 detected by the distance sensor 40 is at risk of contact with the vehicle 1, the image generation unit 135 generates a second risk detection display ED2, and the display control unit 136 displays the second risk detection display ED2. The display control unit 136 displays the second risk detection display ED2 at an end of the display panel 61 in the direction of the risk of contact with the vehicle 1 (downward in this embodiment), with the vehicle 1 as the reference. In this embodiment, the display control unit 136 notifies the user of the direction in which a risk of contact with the vehicle 1 is detected by flashing the vicinity of the bottom edge of the display panel 61 in yellow for a predetermined period as the second danger detection display ED2. The predetermined period is, for example, three seconds. The color of the second danger detection display ED2 is determined according to the degree of risk that the second vehicle C2 will come into contact with the vehicle 1. The degree of risk that the second vehicle C2 will come into contact with the vehicle 1 is detected by the risk detection unit 134. In this embodiment, the risk that the second vehicle C2 will come into contact with the vehicle 1 is detected to be medium, and the color of the second danger detection display ED2 is determined to be yellow. When there is no need to distinguish between the first danger detection display ED1 and the second danger detection display ED2, they may be referred to as danger detection display ED in the following description.

[0044] Moreover, the second screen 710 further displays a first direction indicator CFR and a second direction indicator CFY superimposed on a part of the boundary line CF.

[0045] The first direction display CFR is arranged by the display image generation unit 135 on a circumferential boundary line CF in a direction in which there is a risk of the vehicle 1 coming into contact with the first vehicle C1, with the vehicle 1 as the reference. The direction in which there is a risk of the vehicle 1 coming into contact with the first vehicle C1 is determined by the risk detection unit 134, for example, as the range in which the boundary line CF overlaps with the collision risk image ARC. The first direction display CFR is displayed by the display image generating unit 135 as, for example, a red arc-shaped image. The color of the first direction indicator CFR is determined according to the degree of risk that the first vehicle C1 will come into contact with the vehicle 1. The degree of risk that the first vehicle C1 will come into contact with the vehicle 1 is detected by the risk detection unit 134. In this embodiment, it is detected that there is a high risk that the first vehicle C1 will come into contact with the vehicle 1, and the color of the first direction indicator CFR is determined to be red. The fact that the color of the first direction indicator CFR is red is indicated by dark hatching.

[0046] The second direction display CFY is arranged by the display image generation unit 135 on a circumferential boundary line CF in a direction in which there is a risk of vehicle 1 coming into contact with the second vehicle C2, with vehicle 1 as the reference. The direction in which there is a risk of vehicle 1 coming into contact with the second vehicle C2 is determined by the risk detection unit 134 as, for example, the direction in which the second vehicle C2 is located (here, southward) with vehicle 1 as the reference. The second direction display CFY is displayed by the display image generating unit 135 as, for example, a yellow arc-shaped image. The color of the second direction indicator CFY is determined according to the degree of risk that the second vehicle C2 will come into contact with vehicle 1. The degree of risk that the second vehicle C2 will come into contact with vehicle 1 is detected by the risk detection unit 134. In this embodiment, the risk that the second vehicle C2 will come into contact with vehicle 1 is detected to be medium, and the color of the second direction indicator CFY is determined to be yellow. The yellow color of the second direction indicator CFY is indicated by light hatching. When there is no need to distinguish between the first direction indicator CFR and the second direction indicator CFY, they may be referred to as the direction indicator CFE in the following description.

[0047] Since the first direction display CFR and the second direction display CFY are displayed superimposed on a portion of the boundary line CF, the color of the boundary line CF is displayed in an inconspicuous color by the display image generation unit 135. The boundary line CF is displayed in gray, for example, by the display image generation unit 135. The boundary line CF displayed in gray may be referred to as a second boundary line CF2.

[0048] As explained with reference to Figure 3, the display control unit 136 displays the first danger detection display ED1 near the left edge of the display panel 61, so that the user can visually see that a moving object MS has been detected west of vehicle 1 that poses a risk of contact with vehicle 1. Furthermore, the display control unit 136 displays the first danger detection display ED1 in red, so that the user can visually recognize that the risk of contact with the vehicle 1 is high.

[0049] As explained with reference to Figure 3, the display control unit 136 displays the second danger detection display ED2 near the bottom edge of the display panel 61, so that the user can visually see that a moving object MS has been detected south of vehicle 1 that poses a risk of contact with vehicle 1. Furthermore, the display control unit 136 displays the second danger detection display ED2 in yellow, so that the user can visually recognize that the risk of contact with the vehicle 1 is medium.

[0050] As explained with reference to Figure 3, the display control unit 136 displays the collision risk image ARC in a semi-transparent red on the first road image RW1, so that the user can visually recognize that the traveling speed of the first vehicle C1 corresponding to the collision risk image ARC is equal to or greater than the first speed.

[0051] 3, the display control unit 136 displays the first direction display CFR as an arc-shaped image in the range where the boundary line CF overlaps with the collision risk image ARC, so that the user can visually recognize the direction in which there is a risk of contact with the vehicle 1. Furthermore, the display control unit 136 displays the first direction display CFR as a red arc-shaped image, so that the user can visually recognize that there is a high risk of contact with the vehicle 1 in the direction in which the first direction display CFR is displayed.

[0052] 3, the display control unit 136 displays the second direction display CFY as an arc-shaped image in the direction in which the second vehicle C2 is located, relative to the vehicle 1, so that the user can visually recognize the direction in which there is a risk of contact with the vehicle 1. Furthermore, the display control unit 136 displays the second direction display CFY as a yellow arc-shaped image, so that the user can visually recognize that there is a medium risk of contact with the vehicle 1 in the direction in which the second direction display CFY is displayed.

[0053] 4 is a screen diagram showing an example of a third screen 720 that displays the display image PD in the third state. The third screen 720 is displayed on the display panel 61 by the display control unit 136. The third screen 720 differs from the second screen 710 shown in Fig. 3 in that the first danger detection display ED1 and the second danger detection display ED2 are not displayed on the third screen 720. In other words, the third state is a state after a predetermined period has elapsed since the second state.

[0054] After flashing the first danger detection display ED1 and the second danger detection display ED2 for a predetermined period of time, the display control unit 136 ends the display of the first danger detection display ED1 and the second danger detection display ED2 on the display panel 61. As a result, as shown in the third screen 720 of FIG. 4, the first danger detection display ED1 and the second danger detection display ED2 are not displayed.

[0055] 4, the display control unit 136 blinks the first danger detection display ED1 and the second danger detection display ED2 for a predetermined period of time, and then ends display of the first danger detection display ED1 and the second danger detection display ED2 on the display panel 61. Therefore, the blinking display of the first danger detection display ED1 and the second danger detection display ED2 focuses the user's attention on the first danger detection display ED1 and the second danger detection display ED2, and it is possible to prevent a decrease in visibility of the entire display image PD.

[0056] 5 is a screen diagram showing an example of a fourth screen 730 that displays a display image PD in the fourth state. The fourth screen 730 is displayed on the display panel 61 by the display control unit 136. The fourth screen 730 differs from the third screen 720 in that a first vehicle image CR1 is displayed instead of the first vehicle mark CM1, and a second vehicle image CR2 is displayed instead of the second vehicle mark CM2.

[0057] In the fourth state, the first vehicle C1 moves from a position that is not included in the area indicated by the map image PG displayed on the display panel 6 to a position that is included in the area indicated by the map image PG displayed on the display panel 6. Then, the display control unit 136 displays the first vehicle image CR1 in place of the first vehicle mark CM1.

[0058] The first vehicle image CR1 is generated by the surrounding information image generation unit 131 based on the shape and size of the first vehicle C1 detected by the distance sensor 40. The surrounding information image generation unit 131 generates the first vehicle image CR1 by, for example, reading out the first vehicle image CR1 from images of multiple vehicles stored in the image storage unit 142 based on the shape and size of the first vehicle C1 detected by the distance sensor 40. In this embodiment, the first vehicle C1 is, for example, a four-wheeled passenger car, and therefore the first vehicle image CR1 is an image showing the four-wheeled passenger car.

[0059] In the fourth state, the second vehicle C2 moves from a position that is not included in the area indicated by the map image PG displayed on the display panel 6 to a position that is included in the area indicated by the map image PG displayed on the display panel 6. Therefore, the display control unit 136 displays a second vehicle image CR2 in place of the second vehicle mark CM2.

[0060] The second vehicle image CR2 is generated by the surrounding information image generation unit 131 based on the shape and size of the second vehicle C2 detected by the distance sensor 40. The surrounding information image generation unit 131 generates the second vehicle image CR2 by, for example, reading out the second vehicle image CR2 from a plurality of vehicle images stored in the image storage unit 142 based on the shape and size of the second vehicle C2 detected by the distance sensor 40. In this embodiment, the second vehicle C2 is, for example, a motorcycle, and therefore the second vehicle image CR2 is an image showing the motorcycle.

[0061] As described with reference to FIG. 5, when the first vehicle C1 moves from a position that is not included in the area indicated by the map image PG displayed on the display panel 6 to a position that is included in the area indicated by the map image PG displayed on the display panel 6, the display control unit 136 displays the first vehicle image CR1 instead of the first vehicle mark CM1. This allows the user to visually recognize the position of the first vehicle C1. Furthermore, the first vehicle image CR1 allows the user to visually recognize that the first vehicle C1 is a four-wheeled passenger vehicle.

[0062] As described with reference to FIG. 5, when the second vehicle C2 moves from a position that is not included in the area indicated by the map image PG displayed on the display panel 6 to a position that is included in the area indicated by the map image PG displayed on the display panel 6, the display control unit 136 displays the second vehicle image CR2 in place of the second vehicle mark CM2. This allows the user to visually recognize the position of the second vehicle C2. Furthermore, the second vehicle image CR2 allows the user to visually recognize that the second vehicle C2 is a motorcycle.

[0063] 6 is a screen diagram showing an example of a fifth screen 740 that displays a display image PD in the fifth state. The fifth screen 740 is displayed on the display panel 61 by the display control unit 136. The fifth screen 740 showing the fifth state is different from the fourth screen 730 showing the fourth state in that the first vehicle C1 is passing through the intersection R3.

[0064] In the fifth state, the first vehicle C1 is passing through the intersection R3, so the display image generating unit 135 positions the first vehicle image CR1 to the east of the intersection image RW3. Furthermore, because the first vehicle C1 is passing through the intersection R3, the risk detection unit 134 detects that there is no risk of the first vehicle C1 coming into contact with the vehicle 1. Therefore, the display image generation unit 135 does not display the collision risk image ARC and the first direction display CFR.

[0065] 6, when the first vehicle C1 is passing through the intersection R3, the collision risk image ARC and the first direction display CFR are not displayed, so the user can visually confirm that there is no risk of contact with the first vehicle C1.

[0066] 7 is a screen diagram showing an example of a sixth screen 750 that displays a display image PD in a sixth state. The sixth screen 750 is displayed on the display panel 61 by the display control unit 136. The sixth screen 750 showing the sixth state is different from the fifth screen 740 showing the fifth state in that a part of the second vehicle C2 is located inside the boundary line CF.

[0067] In the sixth state, a portion of the second vehicle C2 is located inside the boundary line CF, so the display image generation unit 135 does not display the second direction display CFY. Furthermore, the risk detection unit 134 detects that there is a high risk of the second vehicle C2 coming into contact with the vehicle 1. Therefore, the display control unit 136 highlights the boundary line CF. For example, the display control unit 136 displays the boundary line CF in red. The boundary line CF displayed in red may be referred to as the third boundary line CF3.

[0068] 7, when a portion of the second vehicle C2 is located inside the boundary line CF, the boundary line CF is displayed in red, allowing the user to visually recognize that there is a high risk of contact with the second vehicle C2.

[0069] FIG. 8 is a screen diagram showing an example of a seventh screen 800 that displays a display image PD in a seventh state. The seventh screen 800 is displayed on the display panel 61 by the display control unit 136.

[0070] In the second state shown in Figure 3 to the sixth state shown in Figure 7, we will explain the case where the distance sensor 40 detects at least one of the first vehicle C1 and the first vehicle C1, but in the seventh state shown in Figure 8 to the ninth state shown in Figure 10, we will explain the case where the distance sensor 40 does not detect the vehicle C but detects a pedestrian.

[0071] 8, the surrounding information image generating unit 131 generates a first pedestrian mark P1, a second pedestrian mark P2, a third pedestrian mark P3, and a fourth pedestrian mark P4 as the surrounding information image PS. Each of the first pedestrian mark P1 to the fourth pedestrian mark P4 is a circular mark.

[0072] The first pedestrian mark P1 is located west of the intersection image RW3 in the first road image RW1. Because the first pedestrian mark P1 is located in the first road image RW1, it is displayed in red, for example, to indicate that the user needs to be more careful. The second pedestrian mark P2 is located on the sidewalk image SW within the first area AR1, and is therefore displayed in yellow, for example, which indicates that the user needs to be moderately careful. The third pedestrian mark P3 and the fourth pedestrian mark P4 are located on the sidewalk image SW, and are therefore displayed in gray, for example, which indicates that the user does not need to pay much attention to them.

[0073] The risk detection unit 134 detects that the pedestrians corresponding to each of the first to fourth pedestrian marks P1 to P4 are at a low risk of coming into contact with the vehicle 1. Therefore, the display control unit 136 displays the boundary line CF in, for example, green, which indicates a low risk. In Fig. 8, the fact that the boundary line CF is green is indicated by a dashed line for convenience.

[0074] FIG. 9 is a screen diagram showing an example of an eighth screen 810 that displays a display image PD in an eighth state. The eighth screen 810 is displayed on the display panel 61 by the display control unit 136.

[0075] The difference between the seventh state shown in Figure 8 and the eighth state shown in Figure 9 is that the first pedestrian mark P1 is located in the second area AR2, whereas the difference between the seventh state shown in Figure 8 and the eighth state shown in Figure 9 is that the first pedestrian mark P1 is located in the first area AR1. Because the first pedestrian mark P1 is located in the first area AR1, the risk detection unit 134 detects that there is a high risk that the pedestrian corresponding to the first pedestrian mark P1 will come into contact with the vehicle 1. Therefore, the display control unit 136 displays the boundary line CF in, for example, red, which indicates a high risk. In Fig. 9, the red color of the boundary line CF is shown by a solid line for convenience.

[0076] FIG. 10 is a screen diagram showing an example of a ninth screen 820 displaying a display image PD in a ninth state. The ninth screen 820 is displayed on the display panel 61 by the display control unit 136.

[0077] The difference is that in the eighth state shown in Figure 9, the first pedestrian mark P1 is located on the first road image RW1, whereas in the ninth state shown in Figure 10, the first pedestrian mark P1 is located on the sidewalk image SW. Since the first pedestrian mark P1 is located on the sidewalk image SW within the first area AR1, it is displayed in yellow, for example, which indicates that the user needs to be moderately careful. Because the first pedestrian mark P1 is located on the sidewalk image SW within the first area AR1, the risk detection unit 134 detects that the pedestrian corresponding to the first pedestrian mark P1 is at a low risk of contact with the vehicle 1. Therefore, the display control unit 136 displays the boundary line CF in, for example, green, which indicates a low risk. In Fig. 10, the fact that the boundary line CF is green is indicated by a dashed line for convenience.

[0078] As described with reference to FIGS. 8 to 10, the risk detection unit 134 detects the risk of the pedestrian coming into contact with the vehicle 1, based on the position of the pedestrian detected by the distance sensor 40. Then, the display control unit 136 displays the boundary line CF in a color that corresponds to the risk detected by the risk detection unit 134. Therefore, the user can visually recognize the degree of risk of the pedestrian coming into contact with the vehicle 1 based on the color of the boundary line CF.

[0079] Next, an example of the vehicle surrounding information display process of the vehicle surrounding information display device 100 will be described with reference to FIGS. FIG. 11 is a flowchart showing an example of processing performed by the vehicle surrounding information display device 100.

[0080] First, as shown in FIG. 11, in step S101, the map image acquisition unit 132 acquires position information indicating the position of the vehicle 1 from the position detection unit 10. Next, in step S103, the map image acquisition unit 132 reads out from the map storage unit 141 the map image PG corresponding to the position information acquired in step S101, thereby acquiring a map image PG of the surroundings of the vehicle 1. Next, in step S105, the surrounding information image generating unit 131 acquires point cloud data from the distance sensor 40. Next, in step S107, the surroundings information image generating unit 131 generates a surroundings information image PS showing surroundings information of the vehicle 1 based on the point cloud data acquired in step S105.

[0081] Next, in step S109, the overhead image acquisition unit 133 acquires the images captured by the front camera 31, the rear camera 33, the left side camera 35, and the right side camera 37, respectively. Next, in step S111, the overhead image acquisition unit 133 generates an overhead image PB of the surroundings of the vehicle 1 by combining the captured images acquired in step S109, and acquires the overhead image PB of the surroundings of the vehicle 1. Next, in step S113, the display image generation unit 135 synthesizes the map image PG, the vehicle mark PM indicating the position of the vehicle 1, the overhead image PB arranged around the vehicle mark PM, and the surrounding information image PS arranged around the overhead image PB to generate the display image PD to be displayed on the display panel 61. Next, in step S115, the display control unit 136 displays on the display panel 61 the display image PD generated by the display image generation unit 135 in step S113. Next, in step S117, the display image generation unit 135 and the display control unit 136 execute a "danger display process." The "danger display process" is a process of generating an image showing the detection result of the danger detection unit 134 and displaying it on the display panel 61. The image showing the detection result of the danger detection unit 134 includes a boundary line CF, a direction display CFE, a danger detection display ED, and a collision risk image ARC. The direction display CFE includes a first direction display CFR and a second direction display CFY. The danger detection display ED includes a first danger detection display ED1 and a second danger detection display ED2. The "danger display process" will be further explained with reference to FIGS.

[0082] Next, in step S119, the vehicle periphery information display device 100 accepts a user's operation on the operation unit 50 and determines whether or not to end the vehicle periphery information display process. If the vehicle surrounding information display device 100 determines not to end the vehicle surrounding information display process (step S119; NO), the process returns to step S101. If the vehicle surrounding information display device 100 determines to end the vehicle surrounding information display process (step S119; YES), the process then ends.

[0083] Steps S105 and S107 correspond to an example of a "periphery information image generating step." Steps S101 and S103 correspond to an example of a "map image acquiring step." Steps S109 and S111 correspond to an example of a "bird's-eye view image acquiring step." Step S113 corresponds to an example of a "display image generating step."

[0084] 11, the display image generation unit 135 combines the map image PG, the vehicle mark PM indicating the position of the vehicle 1, the overhead image PB arranged around the vehicle mark PM, and the surrounding information image PS arranged around the overhead image PB to generate the display image PD to be displayed on the display panel 61. Then, the display control unit 136 displays the display image PD generated by the display image generation unit 135 on the display panel 61. Therefore, it is possible to display surrounding information over a wider range than the overhead image PB as the display image PD.

[0085] Next, the danger display process executed by the vehicle surrounding information display device 100 will be described with reference to FIGS. 12 and 13 are flowcharts showing an example of the danger display process of the vehicle surrounding information display device 100. In FIG.

[0086] First, as shown in FIG. 12, in step S201, the risk detection unit 134 determines, based on the point cloud data from the distance sensor 40, whether or not a moving object MS has been detected. If the risk detection unit 134 determines that a moving object MS has not been detected (step S201; NO), the process proceeds to step S203. Then, in step S203, the risk detection unit 134 detects that there is a low risk of the moving object MS coming into contact with the vehicle 1. The display image generation unit 135 displays the boundary line CF in a color, for example, green, indicating that there is a low risk of the vehicle 1 coming into contact with the moving object MS. The display control unit 136 displays the generated green first boundary line CF1 on the display panel 61. Thereafter, the process returns to step S119 in FIG. 11. If the risk detection unit 134 determines that a moving object MS has been detected (step S201; YES), the process proceeds to step S205. Then, in step S205, the surrounding information image generating unit 131 determines, based on the point cloud data from the distance sensor 40, whether or not the moving object MS is a pedestrian.

[0087] If the surrounding information image generating unit 131 determines that the moving object MS is not a pedestrian (step S205; NO), the process proceeds to step S207. Then, in step S207, the display image generation unit 135 displays the boundary line CF, for example, in gray. The display control unit 136 displays the generated gray second boundary line CF2 on the display panel 61. In addition, the risk detection unit 134 detects that the moving object MS is a vehicle C. Thereafter, the process proceeds to step S219 shown in FIG. If the surrounding information image generating unit 131 determines that the moving object MS is a pedestrian (step S205; YES), the process proceeds to step S209. Then, in step S209, the danger detection unit 134 determines whether or not a pedestrian is on the roadway. If the risk detection unit 134 determines that no pedestrian is on the roadway (step S209; NO), the process proceeds to step S211. Then, in step S211, the risk detection unit 134 determines that the pedestrian is on the sidewalk and detects that the risk of the pedestrian coming into contact with the vehicle 1 is low. The display image generation unit 135 displays the boundary line CF in a color, for example, green, indicating that the risk of the vehicle 1 coming into contact with the pedestrian is low. The display control unit 136 displays the generated green first boundary line CF1 on the display panel 61. Thereafter, the process returns to step S119 in FIG. 11.

[0088] If the risk detection unit 134 determines that a pedestrian is on the roadway (step S209; YES), the process proceeds to step S213. Then, in step S213, the danger detection unit 134 determines whether or not a pedestrian is present in the first area AR1. If the risk detection unit 134 determines that no pedestrian is present in the first area AR1 (step S213; NO), the process proceeds to step S215. Then, in step S215, the risk detection unit 134 detects that there is a low risk of the pedestrian coming into contact with the vehicle 1. The display image generation unit 135 displays the boundary line CF in a color, for example, green, indicating that there is a low risk of the vehicle 1 coming into contact with the pedestrian. The display control unit 136 displays the generated green first boundary line CF1 on the display panel 61. Thereafter, the process returns to step S119 in FIG. 11. If the risk detection unit 134 determines that a pedestrian is in the first area AR1 (step S213; YES), the process proceeds to step S217. Then, in step S217, the risk detection unit 134 detects that there is a high risk of the pedestrian coming into contact with the vehicle 1. The display image generation unit 135 displays the boundary line CF in a color, for example, red, indicating a high risk of the vehicle 1 coming into contact with the pedestrian. The display control unit 136 displays the generated red third boundary line CF3 on the display panel 61. Thereafter, the process returns to step S119 in FIG. 11.

[0089] If the answer is NO in step S205, in step S207, the moving body MS is detected to be vehicle C, and as shown in FIG. 13, in step S219, the risk detection unit 134 determines whether the traveling speed of vehicle C is high, equal to or greater than a first speed (e.g., 70 km / h). If the risk detection unit 134 determines that the traveling speed of vehicle C is high (step S219; YES), the process proceeds to step S231. If the risk detection unit 134 determines that the traveling speed of vehicle C is not high (step S219; NO), the process proceeds to step S221. Then, in step S221, the risk detection unit 134 determines whether the traveling speed of the vehicle C is a medium speed equal to or greater than a second speed (for example, 30 km / hour). If the risk detection unit 134 determines that the traveling speed of the vehicle C is not medium (step S221; NO), the process then returns to step S119 in Fig. 11. If the risk detection unit 134 determines that the traveling speed of the vehicle C is medium (step S221; YES), the process proceeds to step S223. Then, in step S223, the display control unit 136 determines whether or not the flashing of the hazard detection display ED corresponding to the medium speed vehicle C has finished. Here, the hazard detection display ED indicates the second hazard detection display ED2 shown in FIG. If the display control unit 136 determines that the flashing of the danger detection display ED has ended (step S223; YES), the process proceeds to step S225. Then, in step S225, the risk detection unit 134 detects a direction in which there is a risk of contact with the vehicle 1. The display image generation unit 135 generates the direction display CFE as a yellow arc-shaped image and places it on the circumferential boundary line CF. The display control unit 136 displays the direction display CFE on the display panel 61. The direction display CFE corresponds to the second direction display CFY shown in FIG. 3. Thereafter, the process returns to step S119 in FIG. 11.

[0090] In step S219, the risk detection unit 134 determines whether the traveling speed of the vehicle C is high, but is not limited to this. In addition to or instead of determining the traveling speed of the vehicle C, the risk detection unit 134 may determine the risk based on whether the difference in the traveling speed of the vehicle C from the traveling speed of the vehicle 1 is equal to or greater than a specified value. For example, the risk detection unit 134 may determine the risk based on whether the speed ratio between the traveling speed of vehicle 1 and the traveling speed of vehicle C is 1:2 or more. Alternatively, for example, the risk detection unit 134 may determine the risk based on whether the difference between the traveling speed of vehicle 1 and the traveling speed of vehicle C is 60 km / hour or more. In this way, for example, when vehicle 1 and vehicle C are traveling in the same direction on a highway or the like, it is possible to exclude from risk determination cases where the traveling speeds of both vehicles are approximately the same and high. Also, when vehicle 1 and vehicle C are traveling in the same direction, it is possible to detect the risk of the two vehicles suddenly approaching each other.

[0091] If the display control unit 136 determines that the flashing of the danger detection display ED has not ended (step S223; NO), the process proceeds to step S227. Then, in step S227, the risk detection unit 134 detects that there is a medium risk of contact with the vehicle 1. The display control unit 136 determines the color of the risk detection display ED to be, for example, yellow. The display control unit 136 displays the risk detection display ED in a flashing manner at the end of the display panel 61 in the direction of the risk of contact with the vehicle 1, based on the vehicle 1. The risk detection display ED corresponds to the second risk detection display ED2 shown in FIG. 3. Next, in step S229, the risk detection unit 134 detects a direction in which there is a risk of contact with the vehicle 1. The display image generation unit 135 generates a direction display CFE as a yellow arc-shaped image and places it on the circumferential boundary line CF. The display control unit 136 displays the direction display CFE on the display panel 61. The direction display CFE corresponds to the second direction display CFY shown in FIG. 3. Then, the process proceeds to step S243.

[0092] If the determination in step S219 is YES, in step S231, the display control unit 136 determines whether or not the flashing of the danger detection display ED has ended. Here, the danger detection display ED indicates the first danger detection display ED1 shown in FIG. If the display control unit 136 determines that the flashing of the danger detection display ED has ended (step S231; YES), the process proceeds to step S233. Then, in step S233, the image generation unit 135 generates a collision risk image ARC, and the display control unit 136 displays the collision risk image ARC. The collision risk image ARC is an image that shows the traveling direction of the vehicle C and the area where there is a risk of collision. The collision risk image ARC is the collision risk image ARC shown in FIG. Next, in step S235, the risk detection unit 134 detects a direction in which there is a risk of contact with the vehicle 1. The display image generation unit 135 generates a direction display CFE as a red arc-shaped image and places it on the circumferential boundary line CF. The display control unit 136 displays the direction display CFE on the display panel 61. The direction display CFE corresponds to the first direction display CFR shown in FIG. 3. Thereafter, the process returns to step S119 in FIG. 11.

[0093] If the determination in step S231 is NO, in step S237, the risk detection unit 134 detects that there is a high risk of contact with the vehicle 1. The display control unit 136 determines the color of the risk detection display ED to be, for example, red. The display control unit 136 displays the risk detection display ED in a flashing manner at the end of the display panel 61 in the direction of the risk of contact with the vehicle 1, based on the vehicle 1. The risk detection display ED corresponds to the first risk detection display ED1 shown in FIG. 3. Next, in step S239, the image generation unit 135 generates a collision risk image ARC, and the display control unit 136 displays the collision risk image ARC. Next, in step S241, the risk detection unit 134 detects a direction in which there is a risk of contact with the vehicle 1. The display image generation unit 135 generates a direction display CFE as a red arc-shaped image and places it on the circumferential boundary line CF. The display control unit 136 displays the direction display CFE on the display panel 61. The direction display CFE corresponds to the first direction display CFR shown in FIG. 3.

[0094] Next, in step S243, the display control unit 136 determines whether or not a predetermined period (for example, 3 seconds) has elapsed since the flashing of the danger detection display ED was started in step S227 or step S237. If the display control unit 136 determines that the predetermined period has not elapsed (step S243; NO), the process goes to a standby state. If the display control unit 136 determines that the predetermined period has elapsed (step S243; YES), the process proceeds to step S245. Then, in step S245, the display control unit 136 ends the flashing of the danger detection display ED, after which the process returns to step S119 in FIG.

[0095] As explained with reference to Figures 12 and 13, when it is detected that the moving body MS is vehicle C, that the traveling speed of vehicle C is high, and that there is a high risk of contact with vehicle 1, the first danger detection display ED1, the collision risk image ARC, and the first direction display CFR are displayed. Since the first danger detection display ED1 is displayed, the user can visually recognize that a vehicle C that poses a high risk of contact with the vehicle 1 has been detected in the direction in which the first danger detection display ED1 is displayed. In addition, the collision risk image ARC is displayed, allowing the user to visually recognize the area where there is a risk of collision with vehicle C traveling at high speed. Since the first direction display CFR is displayed, the user can visually identify the direction in which there is a high possibility of collision with the vehicle C traveling at high speed.

[0096] As described above, the vehicle surrounding information display device 100 of this embodiment includes a surrounding information image generation unit 131 that generates a surrounding information image PS showing surrounding information of the vehicle 1 based on point cloud data showing the distance from the vehicle 1 obtained from the distance sensor 40, a map image acquisition unit 132 that acquires a map image PG of the area around the vehicle 1, an overhead image acquisition unit 133 that acquires an overhead image PB of the area around the vehicle 1, and a display image generation unit 135 that synthesizes the map image PG, the vehicle mark PM indicating the position of the vehicle 1, the overhead image PB arranged around the vehicle mark PM, and the surrounding information image PS arranged around the overhead image PB to generate a display image PD to be displayed on the display panel 61. Therefore, the display image PD, in which the map image PG, the vehicle mark PM, the overhead image PB, and the surrounding information image PS are combined, is displayed on the display panel 61, so that surrounding information over a wider range than the overhead image PB can be displayed.

[0097] The first area AR1 in which the overhead image PB is displayed is a circular area with the vehicle mark PM at its center. Therefore, the overhead image PB generated as a rectangular image does not include edge areas with large distortion, and can be smoothly connected to the map image PG, thereby making it possible to display a display image PD with good visibility.

[0098] It also includes a risk detection unit 134 that detects a moving body MS that is at risk of coming into contact with the vehicle 1 using point cloud data, and the display image generation unit 135 highlights the circumferential boundary line CF between a circular first area AR1 and a second area AR2 in which the surrounding information image PS is displayed, in the direction in which there is a risk of the vehicle 1 coming into contact with the moving body MS (e.g., vehicle C), using the vehicle 1 as a reference. Therefore, the direction of the boundary line CF highlighted relative to the vehicle 1 allows the user to easily see the direction in which there is a risk of the vehicle 1 coming into contact with the moving object MS.

[0099] The highlighting also includes displaying the color of the boundary line CF in a direction in which there is a risk of vehicle 1 coming into contact with a moving body MS (e.g., vehicle C) relative to vehicle 1 in a color different from other parts of the boundary line CF. Therefore, the user can easily see the direction in which there is a risk of the vehicle 1 coming into contact with the moving object MS by looking at the direction of the boundary line CF, which is displayed in a different color from the other parts of the boundary line CF relative to the vehicle 1. For example, as described with reference to Fig. 3, the color of the boundary line CF in the direction in which there is a risk of the vehicle 1 coming into contact with the moving object MS may be displayed in red, and the other parts of the boundary line CF may be displayed in gray. The first direction display CFR shown in Fig. 3 indicates the direction in which there is a risk of the vehicle 1 coming into contact with the moving object MS.

[0100] Further, the highlighting may be performed by flashing the boundary line CF in the direction of the danger with the vehicle 1 as the reference point. In this case, by displaying the boundary line CF in a flashing manner, the user can easily visually identify the direction in which there is a risk of the vehicle 1 coming into contact with the moving object MS.

[0101] Furthermore, the map image acquisition unit 132 may be communicably connected to a server device and acquire a map image PG from the server device. In this case, the map image acquisition unit 132 acquires the map image PG from the server device, so that the processing of the map image acquisition unit 132 can be simplified.

[0102] The distance sensor 40 also includes a LiDAR (Light Detection and Ranging). Therefore, since the surrounding information image PS showing surrounding information is generated based on point cloud data showing the distance from the vehicle 1 obtained from the LiDAR, an image showing appropriate surrounding information can be generated as the surrounding information image PS.

[0103] In addition, the vehicle surrounding information display method of this embodiment includes a surrounding information image generation step of generating a surrounding information image PS showing surrounding information of the vehicle 1 based on point cloud data showing the distance from the vehicle 1 obtained from the distance sensor 40, a map image acquisition step of acquiring a map image PG of the area around the vehicle 1, an overhead image acquisition step of acquiring an overhead image PB of the area around the vehicle 1, and a display image generation step of synthesizing the map image PG, a vehicle mark PM showing the position of the vehicle 1, the overhead image PB arranged around the vehicle mark PM, and the surrounding information image PS arranged around the overhead image PB to generate a display image PD to be displayed on the display panel 61. Therefore, the display image PD, in which the map image PG, the vehicle mark PM, the overhead image PB, and the surrounding information image PS are combined, is displayed on the display panel 61, so that surrounding information over a wider range than the overhead image PB can be displayed.

[0104] The above-described embodiment is merely an example of one embodiment of the present invention, and any modifications and applications are possible without departing from the spirit of the present invention.

[0105] For example, in order to facilitate understanding of the present invention, the components are classified according to their main processing content in Fig. 1, but the components can be further classified into more components according to the processing content, or one component can be classified to perform more processes. Furthermore, the processing of each component may be executed by a single piece of hardware, or may be executed by multiple pieces of hardware. Furthermore, the processing of each component may be realized by one program or by multiple programs.

[0106] In addition, in FIG. 1, the vehicle surrounding information display device 100 may be provided with at least one of the detection unit 20 and the display unit 50 integrated into one unit.

[0107] In the present embodiment, vehicle periphery information display device 100 includes, but is not limited to, periphery information image generation unit 131, map image acquisition unit 132, overhead image acquisition unit 133, risk detection unit 134, display image generation unit 135, and display control unit 136. A server device communicably connected to vehicle periphery information display device 100 via a network such as the Internet may include at least one of periphery information image generation unit 131, map image acquisition unit 132, overhead image acquisition unit 133, risk detection unit 134, display image generation unit 135, and display control unit 136. The server device may include, for example, map image acquisition unit 132 and overhead image acquisition unit 133. In this case, overhead image acquisition unit 133 may acquire an overhead image PB captured by an artificial satellite.

[0108] In addition, in this embodiment, the case where the moving object MS is a vehicle C and the case where the moving object MS is a pedestrian will be described, but the present invention is not limited to this. The moving object MS may also be, for example, a bicycle.

[0109] In addition, in this embodiment, the overhead image PB is displayed in the first area AR1, which is a circular area, but the present invention is not limited to this, and the overhead image PB may be displayed in a rectangular area.

[0110] In addition, in this embodiment, the hazard detection display ED is displayed by flashing the edge of the display panel 61 for a predetermined period of time, but this is not limiting. The hazard detection display ED may also be displayed by flashing the direction display CFE for a predetermined period of time. In this case, since there is no need to display the hazard detection display ED, processing can be simplified.

[0111] Furthermore, when the vehicle surrounding information display method of the present invention is realized using a computer, the control program executed by the computer can be configured in the form of a recording medium or a transmission medium for transmitting the control program. The recording medium may be a magnetic or optical recording medium or a semiconductor memory device. Specific examples include portable or fixed recording media such as a flexible disk, HDD, CD-ROM (Compact Disk Read Only Memory), DVD, Blu-ray (registered trademark) Disc, magneto-optical disk, flash memory, and card-type recording medium. The recording medium may also be a non-volatile storage device such as RAM, ROM, or HDD included in the vehicle surrounding information display device 100. The control program may also be downloaded by the vehicle surrounding information display device 100 from a server device communicably connected to the vehicle surrounding information display device 100 via a network.

[0112] 11, 12, and 13 are divided according to the main processing content in order to facilitate understanding of the processing of the vehicle periphery information display device 100, and the method of dividing the processing units and their names do not limit the present invention. The processing of the vehicle periphery information display device 100 may be divided into more processing units according to the processing content. Furthermore, the processing of the vehicle periphery information display device 100 may be divided so that one processing unit includes more processes. [Explanation of symbols]

[0113] 100 Vehicle surrounding information display device 1 vehicle 10 Position detection unit 20 Detector 30 Photography Department 40 Distance sensor (sensor) 50 Control section 60 Display section 61 Display panel (display) 130 processors 131 Surrounding information image generation unit 132 Map image acquisition unit 133 Bird's-eye view image acquisition unit 134 Risk detection unit 135 Image Generation Unit 136 Display control unit 140 memory 141 Map memory section 142 Image storage unit AR1 1st area AR2 2nd area ARC Collision Risk Image C vehicle C1 1st car C2 2nd vehicle CF border CFE direction display CFR 1st direction display CFY 2nd direction display ED Hazard Detection Display ED1 First danger detection indicator ED2 Second danger detection display MS Mobile PM vehicle mark PD display image PB overhead view image PG map image PS Surrounding area information image

Claims

1. a surrounding information image generating unit that generates a surrounding information image showing surrounding information of the vehicle based on point cloud data indicating a distance from the vehicle acquired from a sensor; a map image acquisition unit that acquires a map image of the area around the vehicle; an overhead image acquisition unit that acquires an overhead image of the surroundings of the vehicle; a display image generating unit that generates a display image to be displayed on a display by combining the map image, a vehicle mark indicating the position of the vehicle, the overhead image arranged around the vehicle mark, and the surrounding information image arranged around the overhead image; Equipped with the area in which the overhead image is displayed is a circular area centered on the vehicle mark, a risk detection unit that detects a moving object that is at risk of contact with the vehicle based on the point cloud data; the display image generation unit highlights a part of a circumferential boundary line between the circular area and an area in which the surrounding information image is displayed, in a direction in which there is a risk of the vehicle coming into contact with the moving object, based on the vehicle; the display image generation unit generates a collision risk image that is an image showing a traveling direction of a first vehicle that is at risk of colliding with the vehicle and a range in which the vehicle is at risk of colliding with the first vehicle; the risk detection unit determines a first direction indication indicating a direction in which there is a risk of collision between the vehicle and the first vehicle as a range in which the collision risk image overlaps with the circumferential boundary line; the display image generation unit superimposes the first direction indication on a part of the boundary line to highlight it. Vehicle surrounding information display device.

2. The highlighting includes displaying the boundary line in a direction in which the risk exists relative to the vehicle in a color different from that of other portions of the boundary line. The vehicle surrounding information display device according to claim 1 .

3. The highlighting includes flashing the boundary line in the direction of the risk relative to the vehicle. The vehicle surrounding information display device according to claim 1 or 2.

4. Communicatively connected to the server device, the map image acquisition unit acquires the map image from the server device; The vehicle surrounding information display device according to any one of claims 1 to 3.

5. The sensor includes a LiDAR (Light Detection and Ranging), The vehicle surrounding information display device according to any one of claims 1 to 4.

6. a surrounding information image generating step of generating a surrounding information image showing surrounding information of the vehicle based on point cloud data indicating a distance from the vehicle acquired from a sensor; a map image acquisition step of acquiring a map image of the surroundings of the vehicle; an overhead image acquisition step of acquiring an overhead image of the surroundings of the vehicle; a display image generating step of generating a display image to be displayed on a display by combining the map image, a vehicle mark indicating the position of the vehicle, the overhead image arranged around the vehicle mark, and the peripheral information image arranged around the overhead image; Including, the area in which the overhead image is displayed is a circular area centered on the vehicle mark, a risk detection step of detecting a moving object that is at risk of contact with the vehicle based on the point cloud data; In the display image generating step, a part of a circumferential boundary line between the circular area and an area where the surrounding information image is displayed is highlighted in a direction in which there is a risk of the vehicle coming into contact with the moving object, with the vehicle as a reference; In the display image generating step, a collision risk image is generated which is an image showing a traveling direction of a first vehicle which is at risk of colliding with the vehicle and a range in which the vehicle is at risk of colliding with the first vehicle, In the risk detection step, a first direction indication is obtained, which indicates a direction in which there is a risk of collision between the vehicle and the first vehicle, as a range in which the collision risk image overlaps with the circumferential boundary line; In the display image generating step, the first direction indication is superimposed on a part of the boundary line to be highlighted. A method for displaying information about the vehicle's surroundings.

Images