Vehicle surrounding information display device and vehicle surrounding information display method

Abstract

To switch an image to be displayed on a display panel between a first display image and a second display image.SOLUTION: A vehicle peripheral information display device 100 comprises: a map image acquisition unit 132 for acquiring a map image PG of a periphery of a vehicle 1; a first display image generation unit 135 for generating a first display image PD1 by combining the map image PG with a self vehicle mark PM; a peripheral information image generation unit 131 for generating a peripheral information image PS indicating peripheral information of the vehicle 1 based on dot group data indicating a distance from the vehicle 1 acquired from a distance sensor 40; an overview image acquisition unit 133 for acquiring an overview image PB of the periphery of the vehicle 1; a second display image generation unit 136 for generating a second display image PD2 by combining the map image PG, a self vehicle image PR, the overview image PB and the peripheral information image PS; and a display control unit 138 for switching an image to be displayed on a display panel 61 between the first display image PD1 and the second display image PD2 in response to an operation from a user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a vehicle surrounding information display device and a vehicle surrounding information display method. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a technique for displaying an image showing information about the periphery of a vehicle. For example, Patent Document 1 describes a technique for reducing the effect of smears in a displayed image even when smears occur due to the effect of a highly luminous subject when an image of the surroundings of a vehicle is displayed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-134586 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 merely displays an overhead image (a so-called surround view) based on an image captured by a camera, and only displays peripheral information within a short distance from the vehicle. Furthermore, for example, when it is possible to display a normal navigation image and an image showing surrounding information, it is preferable to be able to switch between the two. An object of the present invention is to provide a vehicle surroundings information display device and a vehicle surroundings information display method capable of switching between a plurality of display images. [Means for solving the problem]

[0005] In order to achieve the above-mentioned object, the vehicle surrounding information display device of the present invention includes a map image acquisition unit that acquires a map image of the surroundings of the vehicle, a first display image generation unit that synthesizes the map image with a vehicle mark indicating the position of the vehicle to generate a first display image to be displayed on a display, a surrounding information image generation unit that generates a surrounding information image indicating surrounding information of the vehicle based on point cloud data indicating the distance from the vehicle acquired from a sensor, an overhead image acquisition unit that acquires an overhead image of the surroundings of the vehicle, a second display image generation unit that synthesizes the map image, the vehicle mark, the overhead image arranged around the vehicle mark, and the surrounding information image arranged around the overhead image to generate a second display image to be displayed on a display, and a display control unit that accepts operation from a user and switches the image to be displayed on the display between the first display image and the second display image in accordance with the operation.

[0006] In order to achieve the above-mentioned object, the vehicle surrounding information display method of the present invention includes a map image acquisition step of acquiring a map image of the surroundings of the vehicle, a first display image generation step of synthesizing the map image with a vehicle mark indicating the position of the vehicle to generate a first display image to be displayed on a display, a surrounding information image generation step of generating a surrounding information image indicating surrounding information of the vehicle based on point cloud data indicating a distance from the vehicle acquired from a sensor, an overhead image acquisition step of acquiring an overhead image of the surroundings of the vehicle, a second display image generation step of synthesizing the map image, the vehicle mark, the overhead image arranged around the vehicle mark, and the surrounding information image arranged around the overhead image to generate a second display image to be displayed on a display, and a display control step of accepting an operation from a user and switching the image to be displayed on the display between the first display image and the second display image in accordance with the operation. Effect of the Invention

[0007] According to the vehicle surrounding information display device and vehicle surrounding information display method of the present invention, a map image and a vehicle mark are combined to generate a first display image. Also, a second display image is generated by combining the map image, the vehicle mark, an overhead image, and a surrounding information image showing surrounding information of the vehicle. Furthermore, the image displayed on the display is switched between the first display image and the second display image in response to an operation from the user. Therefore, the user can switch between the first display image and the second display image. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing an example of a configuration of an in-vehicle device including a vehicle surrounding information display device; [Diagram 2] 10 is a screen diagram showing an example of a first screen displaying a second display image in a first state. FIG. [Diagram 3] FIG. 11 is a screen diagram showing an example of a second screen displaying a second display image in a second state. [Figure 4] FIG. 11 is a screen diagram showing an example of a third screen displaying a second display image in a third state. [Diagram 5] FIG. 13 is a screen diagram showing an example of a fourth screen displaying a second display image in a fourth state. [Figure 6] FIG. 13 is a screen diagram showing an example of a fifth screen displaying a second display image in a fifth state. [Figure 7] FIG. 13 is a screen diagram showing an example of a sixth screen displaying a second display image in a sixth state. [Figure 8] FIG. 13 is a screen diagram showing an example of a seventh screen displaying a second display image in a seventh state. [Figure 9] FIG. 13 is a screen diagram showing an example of an eighth screen displaying a second display image in an eighth state. [Figure 10] FIG. 13 is a screen diagram showing an example of a ninth screen displaying a second display image in a ninth state. [Figure 11] 4 is a flowchart showing an example of a process of the vehicle surrounding information display device. [Figure 12] 5 is a flowchart showing an example of a danger display process of the vehicle surrounding information display device. [Figure 13]5 is a flowchart showing an example of a danger display process of the vehicle surrounding information display device. [Figure 14] FIG. 13 is a screen diagram showing an example of a tenth screen displaying a first display state of a transition from a first display image to a second display image. [Figure 15] FIG. 13 is a screen diagram showing an example of an eleventh screen displaying a second display state of transition from the first display image to the second display image. [Figure 16] FIG. 15 is a screen diagram showing an example of a twelfth screen displaying a third display state of transition from the first display image to the second display image. [Figure 17] FIG. 13 is a screen diagram showing an example of a thirteenth screen displaying a fourth display state of transition from the first display image to the second display image. [Figure 18] FIG. 17 is a screen diagram showing an example of a fourteenth screen displaying a fifth display state of transition from the first display image to the second display image. [Figure 19] 10 is a state transition diagram showing an example of transition between a first display image, a second display image, and a third display image. FIG. [Figure 20] 13 is a flowchart showing an example of a display image determination process when changing the display image from a first display image to a second display image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. 1 is a diagram showing an example of the configuration of an in-vehicle device 3 including a vehicle surrounding information display device 100. The in-vehicle device 3 is mounted on a vehicle 1. The in-vehicle device 3 includes a position detection unit 10, a detection section 20, an operation section 50, a display section 60, and a vehicle surrounding information display device 100.

[0010] The position detection unit 10 detects the position of the vehicle 1. The position detection unit 10 includes a GNSS (Global Navigation Satellite System) receiver that receives GNSS signals, and a processor that calculates the position of the vehicle 1 based on the GNSS signals received by the GNSS receiver. The GNSS receiver and the processor are not shown in the drawings. The position detection unit 10 outputs position information indicating the position of the vehicle 1 to the vehicle surrounding information display device 100.

[0011] The detection unit 20 captures images of the surroundings of the vehicle 1 and detects point cloud data indicating the distance between the vehicle 1 and moving objects MS that exist within a range of a predetermined distance from the vehicle 1. The moving objects MS include other vehicles C, bicycles, and pedestrians. The predetermined distance is, for example, 20 m. The detection unit 20 includes an imaging unit 30 and a distance sensor 40 .

[0012] The photographing unit 30 photographs images of the surroundings of the vehicle 1. The photographing unit 30 includes a front camera 31 that photographs the area in front of the vehicle 1, a rear camera 33 that photographs the area behind the vehicle 1, a left side camera 35 that photographs the area to the left of the vehicle 1, and a right side camera 37 that photographs the area to the right of the vehicle 1. Each of these cameras includes an image sensor such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor), and a data processing circuit that generates a photographed image from the image sensor. The photographing unit 30 adjusts the angle of view of each of the cameras facing in four directions (i.e., forward, backward, left, and right) so that the cameras can photograph a 360° range centered on the vehicle 1. Each of the front camera 31, rear camera 33, left side camera 35, and right side camera 37 photographs the photographing range at a predetermined frame rate to generate a photographed image. The photographing unit 30 outputs the generated photographed image to the vehicle surrounding information display device 100. The vehicle surrounding information display device 100 stores the input photographed image in the memory 140. Each of the front camera 31, the rear camera 33, the left side camera 35, and the right side camera 37 may be configured with one camera or multiple cameras.

[0013] The distance sensor 40 detects point cloud data indicating the distance between the vehicle 1 and a moving object MS that exists within a range of a predetermined distance from the vehicle 1. The distance sensor 40 is provided with LiDAR (Light Detection and Ranging) at multiple locations, such as the front, rear, left side, and right side, on the body of the vehicle 1, and acquires point cloud data using electromagnetic waves. Each point data constituting the point cloud data indicates the distance between the vehicle 1 and a moving object MS that exists within a range of a predetermined distance from the vehicle 1. In this embodiment, the distance sensor 40 is a LiDAR, but is not limited to this. The distance sensor 40 may be, for example, a radar or a sonar sensor. In addition, in this embodiment, a case will be described in which the LiDAR is disposed at a plurality of locations, such as the front, rear, left side, and right side, on the vehicle body constituting the vehicle 1, but is not limited thereto. The LiDAR may be disposed on the roof of the vehicle 1 to acquire point cloud data of the entire periphery (i.e., the front, rear, left side, and right side). The distance sensor 40 corresponds to an example of a "sensor."

[0014] The operation unit 50 accepts an operation of a user who is in the vehicle 1. The user is, for example, a driver. The operation unit 50 outputs an operation signal corresponding to the accepted operation to the vehicle surrounding information display device 100. The operations accepted by the operation unit 50 include, for example, an operation to instruct the start of image display processing, an operation to end image display processing, and the like. The operation unit 50 includes, for example, an unillustrated image display on switch and an unillustrated image display off switch, and when the image display on switch is pressed, the vehicle surrounding information display device 100 accepts an operation to instruct the start of image display processing. Also, when the image display off switch is pressed during execution of the image display processing, the vehicle surrounding information display device 100 accepts an operation to end the image display processing. The "image display process" refers to a process in which the vehicle surrounding information display device 100 generates a second display image PD2 by, for example, synthesizing a map image PG, a vehicle image PR, an overhead image PB, and a surrounding information image PS, and displays the generated second display image PD2 on the display panel 61. The second display image PD2 corresponds to an example of a display image PD. The display image PD indicates an image displayed on the display panel 61. The "image display process" will be described with reference to FIG. 1 as to the functional configuration of the vehicle surrounding information display device 100, and will be further described with reference to FIGS. Moreover, the map image PG, the host vehicle mark PM, the overhead image PB, the surrounding information image PS, and the second display image PD2 will be described with reference to FIGS.

[0015] The display unit 60 includes a display panel 61 and a touch sensor 63 . A liquid crystal display, an organic EL display, or the like is used for the display panel 61. The display unit 60 displays images such as a display image PD on the display panel 61 based on display data input from the vehicle surrounding information display device 100. The display panel 61 corresponds to an example of a "display." The second display image PD2 will be described with reference to FIGS. The touch sensor 63 may be a resistive sensor, a capacitive sensor, or the like. The touch sensor 63 is disposed on the display surface of the display panel 61. The display unit 60 detects a touch operation of the user's finger on the display panel 61 by the touch sensor 63, and generates a position signal indicating the operation position of the detected touch operation. That is, the display panel 61 and the touch sensor 63 constitute a so-called "touch panel." The touch sensor 63 is, for example, a capacitive pressure sensor. The display unit 60 detects a pressure value PV of a touch operation of a user's finger on the display panel 61 using the touch sensor 63, and generates a pressure signal indicating the pressure value PV of the detected touch operation. The display unit 60 outputs the generated position signal and pressure signal to the vehicle surrounding information display device 100.

[0016] The vehicle surrounding information display device 100 is a computer including a processor 130 such as a Central Processing Unit (CPU) or a Micro-Processing Unit (MPU), and a memory 140 such as a Read Only Memory (ROM) or a Random Access Memory (RAM). In addition to these devices, the vehicle surrounding information display device 100 is equipped with storage devices such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), an interface circuit for connecting sensors and peripheral devices, and an in-vehicle network communication circuit for communicating with other in-vehicle devices via the in-vehicle network. The vehicle surrounding information display device 100 realizes various functional configurations by the processor 130 executing control programs stored in the memory 140 or the storage device.

[0017] The vehicle surrounding information display device 100 includes, for example, as functional units, a surrounding information image generation unit 131, a map image acquisition unit 132, an overhead image acquisition unit 133, a danger detection unit 134, a first display image generation unit 135, a second display image generation unit 136, a third display image generation unit 137, a display control unit 138, a map memory unit 141, and an image memory unit 142. Specifically, processor 130 executes a control program stored in memory 140 or the storage device, causing processor 130 to function as surrounding information image generating section 131, map image acquiring section 132, bird's-eye view image acquiring section 133, risk detecting section 134, first display image generating section 135, second display image generating section 136, third display image generating section 137, and display control section 138. Processor 130 also executes a control program stored in memory 140 or the storage device, causing memory 140 to function as map storage section 141 and image storage section 142.

[0018] The map storage unit 141 stores a map image PG. The map image PG is read out by the map image acquisition unit 132.

[0019] The image storage unit 142 stores images of the host vehicle image PR, the host vehicle mark PM, the first vehicle image CR1, the first vehicle mark CM1, the second vehicle image CR2, the second vehicle mark CM2, the pedestrian mark P, and the like. The host vehicle image PR shows the vehicle 1. When the second display image generating unit 136 displays the overhead image PB, the host vehicle image PR is disposed at the position of the vehicle 1. That is, the host vehicle image PR shows the position of the vehicle 1 in the map image PG. The vehicle 1 is, for example, a four-wheeled passenger car. Thus, the host vehicle image PR is an image showing the four-wheeled passenger car. The subject vehicle image PR corresponds to an example of a "vehicle mark." The host vehicle mark PM indicates the vehicle 1. When the overhead image PB is not displayed, the host vehicle mark PM is placed at the position of the vehicle 1 in the map image PG. That is, the host vehicle mark PM indicates the position of the vehicle 1 in the map image PG. The host vehicle mark PM is, for example, a triangular mark. The host vehicle mark PM corresponds to an example of a "vehicle mark". The host vehicle image PR will be further described with reference to Fig. 3. The host vehicle mark PM will be further described with reference to Fig. 14.

[0020] The first vehicle image CR1 is an image of the first vehicle C1 detected by the distance sensor 40. The first vehicle image CR1 is placed at the position of the first vehicle C1 in the map image PG by the second display image generating unit 136. The first vehicle C1 is, for example, a four-wheeled passenger car. The first vehicle mark CM1 indicates the first vehicle C1 detected by the distance sensor 40. The first vehicle mark CM1 is placed on a road in the traveling direction of the first vehicle C1 on the map image PG by the second display image generating unit 136. The first vehicle mark CM1 is, for example, a pentagonal mark.

[0021] The second vehicle image CR2 is an image of the second vehicle C2 detected by the distance sensor 40. The second vehicle image CR2 is placed at the position of the second vehicle C2 in the map image PG by the second display image generating unit 136. The second vehicle C2 is, for example, a motorcycle. The second vehicle mark CM2 indicates the second vehicle C2 detected by the distance sensor 40. The second vehicle mark CM2 is placed on a road in the traveling direction of the second vehicle C2 on the map image PG by the second display image generating unit 136. The second vehicle mark CM2 is, for example, a pentagonal mark.

[0022] The pedestrian mark P indicates a pedestrian detected by the distance sensor 40. The pedestrian mark P is placed at the position of the pedestrian on the map image PG by the second display image generating unit 136. The pedestrian mark P is, for example, a circular mark. The first vehicle mark CM1 and the second vehicle mark CM2 will be further described with reference to FIGS. The first vehicle image CR1 and the second vehicle image CR2 will be further described with reference to FIG. In the following description, the first vehicle C1 and the second vehicle C2 may be referred to as vehicle C when there is no need to distinguish between them. The pedestrian mark P will be further explained with reference to FIG.

[0023] The surrounding information image generating unit 131 acquires point cloud data from the distance sensor 40, and generates a surrounding information image PS showing surrounding information of the vehicle 1 based on the point cloud data. Each point data constituting the point cloud data indicates a distance between the vehicle 1 and a moving object MS existing within a range of a predetermined distance from the vehicle 1. The predetermined distance is, for example, 20 m. Each point data constituting the point cloud data indicates the direction of the moving object MS. That is, the distance sensor 40 detects the direction of the moving object MS by receiving the electromagnetic waves reflected by the moving object MS. Each point data constituting the point cloud data indicates the position of the moving object MS with respect to the vehicle 1 as a reference. The surrounding information image PS includes a first vehicle image CR1, a first vehicle mark CM1, a second vehicle image CR2, a second vehicle mark CM2, and a pedestrian mark P.

[0024] When the first vehicle C1 is included in the area indicated by the map image PG displayed on the display panel 61, the first vehicle image CR1 is placed by the second display image generating unit 136 at the position of the first vehicle C1 on the map image PG. When the second vehicle C2 is included in the area indicated by the map image PG displayed on the display panel 61, the second vehicle image CR2 is placed by the second display image generating unit 136 at the position of the second vehicle C2 on the map image PG.

[0025] The first vehicle mark CM1 is placed on the road in the direction of travel of the first vehicle C1 on the map image PG by the second display image generation unit 136 when the first vehicle C1 is not included in the area indicated by the map image PG displayed on the display panel 61 and the risk detection unit 134 detects that there is a risk of contact with vehicle 1. The second vehicle mark CM2 is placed by the second display image generating unit 136 on the road in the direction of travel of the second vehicle C2 on the map image PG when the second vehicle C2 is not included in the area indicated by the map image PG displayed on the display panel 61 and the risk detection unit 134 detects that there is a risk of contact with the second vehicle C2. When a pedestrian detected by the distance sensor 40 is included in the area indicated by the map image PG displayed on the display panel 61, the pedestrian mark P is placed at the position of the pedestrian on the map image PG.

[0026] The map image acquisition unit 132 acquires location information indicating the position of the vehicle 1 from the position detection unit 10, and acquires a map image PG of the surroundings of the vehicle 1 by reading out a map image PG corresponding to the acquired location information from the map memory unit 141.

[0027] In the present embodiment, a case will be described in which the map image acquisition unit 132 acquires a map image PG of the periphery of the vehicle 1 by reading out a map image PG corresponding to the position information from the map storage unit 141, but the present invention is not limited to this. The vehicle surrounding information display device 100 may be communicably connected to a server device, and the map image acquisition unit 132 may acquire a map image PG of the periphery of the vehicle 1 by receiving a map image PG corresponding to the position information from the server device. In this case, the map storage unit 141 does not need to store the map image PG, so that the configuration of the vehicle surrounding information display device 100 can be simplified.

[0028] The overhead image acquisition unit 133 generates an overhead image PB of the surroundings of the vehicle 1 by combining images captured by the front camera 31, the rear camera 33, the left side camera 35, and the right side camera 37, and acquires the overhead image PB of the surroundings of the vehicle 1.

[0029] The risk detection unit 134 detects the risk of contact between the moving body MS and the vehicle 1 based on the point cloud data from the distance sensor 40. The risk detection unit 134 detects, for example, the position of a moving body MS that is at risk of contact with the vehicle 1. The risk detection unit 134 detects, for example, the attributes of the moving body MS that is at risk of contact with the vehicle 1. The attributes of the moving body MS indicate, for example, whether the moving body MS is a vehicle C or a pedestrian. Furthermore, the risk detection unit 134 detects, for example, a direction in which the moving body MS is at risk of contact with the vehicle 1, based on the vehicle 1. Furthermore, the risk detection unit 134 detects, for example, the degree of risk of the moving body MS contacting the vehicle 1. The moving objects MS include other vehicles C, bicycles, and pedestrians.

[0030] The first display image generating unit 135 generates a first display image PD1 to be displayed on the display panel 61 by combining the map image PG with a host vehicle mark PM indicating the position of the vehicle 1. The first display image PD1 is an image displayed on a so-called "navigation screen."

[0031] The third display image generation unit 137 synthesizes the map image PG, the host vehicle image PR showing the position of the vehicle 1, and an overhead image PB arranged around the host vehicle image PR, to generate the third display image PD3 to be displayed on the display panel 61.

[0032] The second display image generation unit 136 synthesizes the map image PG, the vehicle image PR indicating the position of the vehicle 1, the overhead image PB arranged around the vehicle image PR, and the surrounding information image PS arranged around the overhead image PB, to generate the second display image PD2 to be displayed on the display panel 61. The surrounding information image PS includes a first vehicle image CR1, a first vehicle mark CM1, a second vehicle image CR2, a second vehicle mark CM2, and a pedestrian mark P.

[0033] On the display panel 61, the area in which the overhead image PB is displayed is a circular first area AR1 centered on the host vehicle mark PM. The first area AR1 will be further described with reference to FIG.

[0034] The second display image generation unit 136 highlights the circular boundary line CF between the first area AR1 and the second area AR2 in which the surrounding information image PS is displayed, in the direction in which there is a risk of the vehicle 1 coming into contact with the moving body MS, based on the vehicle 1. For example, the second display image generating unit 136 highlights the boundary line CF by displaying the color of the boundary line CF in a color (e.g., red, yellow, etc.) different from other parts of the boundary line CF in a direction in which there is a risk of contact with vehicle 1, based on vehicle 1. The boundary line CF will be further described with reference to Figures 2 to 10. Note that in Figures 2 to 10, the highlighted display of the boundary line CF is described as a directional indication CFE. The directional indication CFE includes a first directional indication CFR and a second directional indication CFY.

[0035] In this embodiment, a case will be described in which the color of the boundary line CF in the direction where there is a risk of contact with the vehicle 1 is displayed in a color different from the color of the other parts of the boundary line CF as the highlighting of the boundary line CF, but the present invention is not limited to this. For example, the boundary line CF in the direction where there is a risk of contact with the vehicle 1 may be displayed in a blinking manner as the highlighting of the boundary line CF. In addition, for example, the direction display CFE may be displayed in a blinking manner as the highlighting of the boundary line CF. In these cases, the visibility of the direction where there is a risk of contact with the vehicle 1 for the user can be improved.

[0036] The display control unit 138 displays the first display image PD1 generated by the first display image generation unit 135, the second display image PD2 generated by the second display image generation unit 136, or the third display image PD3 generated by the third display image generation unit 137 on the display panel 61. The second display image PD2 will be further described with reference to FIGS. The first display image PD1 will be further described with reference to FIG. The third display image PD3 will be further described with reference to FIG. 2 to 10, a case will be described in which the display control unit 138 displays the second display image PD2 over the entire display surface of the display panel 61. However, the present invention is not limited to this. The display control unit 138 may display the second display image PD2 in a partial region of the display surface of the display panel 61.

[0037] Next, a specific example of the second display image PD2 displayed on the display panel 61 will be described with reference to FIGS. 2 to 10 each show directions. The upper direction of the figure corresponds to the north direction, the lower direction of the figure corresponds to the south direction, the right direction of the figure corresponds to the east direction, and the left direction of the figure corresponds to the west direction.

[0038] 2 is a screen diagram showing an example of a first screen 700 displaying a second display image PD2 in a first state. The first screen 700 is displayed on the display panel 61 by the display control unit 138. In the first screen 700, a host vehicle image PR and an overhead image PB are displayed in a first area AR1, a map image PG is displayed in a second area AR2, and a boundary line CF is displayed at the boundary between the first area AR1 and the second area AR2.

[0039] The map image PG includes a road image RW and a sidewalk image SW. The road image RW includes a first road image RW1 showing a first road R1 extending in an east-west direction, a second road image RW2 showing a second road R2 extending in a north-south direction, and an intersection image RW3 showing an intersection R3 where the first road R1 and the second road R2 intersect. In this embodiment, no traffic light is provided at the intersection R3. As shown in FIG. 2, the vehicle 1 is traveling north on the second road R2 and is approaching an intersection R3.

[0040] In the first state, the distance sensor 40 does not detect the moving object MS. Therefore, the risk detection unit 134 detects that there is no moving object MS that is at risk of contact with the vehicle 1. In this case, the second display image generation unit 136 displays the boundary line CF in a first color indicating that there is no risk of the vehicle 1 contacting the moving object MS. The second display image generating unit 136 displays the boundary line CF in green, for example. Green corresponds to an example of the first color. In other words, the boundary line CF being displayed in green indicates that there is no risk of the vehicle 1 coming into contact with the moving body MS. In FIG. 2, the boundary line CF is shown in green by a dashed line for the sake of convenience. The boundary line CF displayed in green may be referred to as the first boundary line CF1.

[0041] The overhead image PB is displayed in a first area AR1. The first area AR1 is a circular area including the vehicle 1. Meanwhile, the overhead image PB is generated by projecting and synthesizing the images captured by the front camera 31, the rear camera 33, the left side camera 35, and the right side camera 37. Therefore, when the overhead image PB is displayed as a rectangular image, the image is significantly distorted at the positions corresponding to the four corners of the rectangular image, and a discontinuous image is generated at the joint with the map image PG. In contrast, in this embodiment, the image at the four corners of the rectangular image generated as the overhead image PB is truncated to generate a circular overhead image PB. Therefore, the circular overhead image PB does not include any areas with large distortion. Therefore, the overhead image PB can be seamlessly connected to the map image PG.

[0042] As explained with reference to Figure 2, the display control unit 138 displays the boundary line CF as the first boundary line CF1, which indicates that there is no risk of the vehicle 1 coming into contact with the moving body MS, so that the user can visually confirm that there is no risk of the vehicle 1 coming into contact with the moving body MS.

[0043] 3 is a screen diagram showing an example of a second screen 710 displaying a second display image PD2 in the second state. The second screen 710 is displayed on the display panel 61 by the display control unit 138. In the second screen 710, an image of the vehicle itself PR and an overhead image PB are displayed in the first area AR1, similarly to the first screen 700 shown in FIG. In the second screen 710, the display control unit 138 displays a map image PG, a first vehicle mark CM1, a second vehicle mark CM2, a first danger detection display ED1, a second danger detection display ED2, and a collision risk image ARC in the second area AR2.

[0044] The first vehicle mark CM1 indicates a first vehicle C1 traveling east on a first road R1 that extends in an east-west direction and entering an intersection R3. The first vehicle C1 corresponds to an example of a moving object MS detected by the distance sensor 40. The first vehicle C1 is not included in the area indicated by the map image PG displayed on the display panel 61, and is detected by the risk detection unit 134 as being at risk of contact with the vehicle 1. In this case, the second display image generation unit 136 places the first vehicle mark CM1 on the road in the traveling direction of the first vehicle C1 on the map image PG. The second display image generation unit 136 places the first vehicle mark CM1, for example, at the western end (the left end in FIG. 3) of the first road image RW1.

[0045] When the risk detection unit 134 detects that the first vehicle C1 detected by the distance sensor 40 is at risk of contact with the vehicle 1, the display control unit 138 displays the first risk detection display ED1. The display control unit 138 displays the first risk detection display ED1 at an end of the display panel 61 in a direction (in this embodiment, the left direction) where there is a risk of contact with the vehicle 1, based on the vehicle 1. In this embodiment, the display control unit 138 notifies the user of the direction in which a risk of contact with the vehicle 1 is detected by flashing the vicinity of the left side of the display panel 61, for example, in red for a predetermined period as the first danger detection display ED1. The predetermined period is, for example, three seconds. The color of the first danger detection indication ED1 is determined according to the degree of risk that the first vehicle C1 will come into contact with the vehicle 1. The degree of risk that the first vehicle C1 will come into contact with the vehicle 1 is detected by the risk detection unit 134. In this embodiment, it is detected that there is a high risk that the first vehicle C1 will come into contact with the vehicle 1, and the color of the first danger detection indication ED1 is determined to be, for example, red.

[0046] The collision risk image ARC is an image that is generated by the second display image generating unit 136 and displayed by the display control unit 138 when the traveling speed of the first vehicle C1 is equal to or higher than the first speed. The collision risk image ARC is an image that indicates the traveling direction of the first vehicle C1 and an area where there is a risk of collision. The first speed is, for example, 70 km / hour. The traveling speed of the first vehicle C1 is, for example, 80 km / hour. The collision risk image ARC is displayed, for example, in a semi-transparent red color on the first road image RW1. The collision risk image ARC is displayed, for example, as a rectangular image extending in the east-west direction. The width of the collision risk image ARC (the size in the short side direction, i.e., the north-south direction) indicates, for example, the width of the first vehicle C1.

[0047] The second vehicle mark CM2 indicates a second vehicle C2 traveling north on a second road R2 extending in a north-south direction and entering an intersection R3. The second vehicle C2 corresponds to an example of a moving body MS detected by the distance sensor 40. The traveling speed of the second vehicle C2 is less than a first speed and greater than or equal to a second speed. The second speed is, for example, 30 km / hour. The traveling speed of the second vehicle C2 is, for example, 40 km / hour. The second vehicle C2 is not included in the area indicated by the map image PG displayed on the display panel 61, and is detected by the risk detection unit 134 as being at risk of contact with the vehicle 1. In this case, the surrounding information image generation unit 131 places the second vehicle mark CM2 on the road in the traveling direction of the second vehicle C2 on the map image PG. The surrounding information image generation unit 131 places the second vehicle mark CM2, for example, at the southern end (the lower end in FIG. 3) of the second road image RW2.

[0048] When the risk detection unit 134 detects that the second vehicle C2 detected by the distance sensor 40 is at risk of contact with the vehicle 1, the second display image generation unit 136 generates the second risk detection display ED2, and the display control unit 138 displays the second risk detection display ED2. The display control unit 138 displays the second risk detection display ED2 on an end of the display panel 61 in a direction (downward in this embodiment) where there is a risk of contact with the vehicle 1, based on the vehicle 1. In this embodiment, the display control unit 138 notifies the user of the direction in which a risk of contact with the vehicle 1 is detected by flashing the vicinity of the lower side of the display panel 61, for example, in yellow for a predetermined period as the second danger detection display ED2. The predetermined period is, for example, three seconds. The color of the second danger detection indication ED2 is determined according to the degree of risk that the second vehicle C2 will come into contact with the vehicle 1. The degree of risk that the second vehicle C2 will come into contact with the vehicle 1 is detected by the risk detection unit 134. In this embodiment, it is detected that the risk that the second vehicle C2 will come into contact with the vehicle 1 is medium, and the color of the second danger detection indication ED2 is determined to be yellow. When there is no need to distinguish between the first danger detection display ED1 and the second danger detection display ED2, they may be referred to as the danger detection display ED in the following description.

[0049] Moreover, in the second screen 710, a first direction indication CFR and a second direction indication CFY are further displayed so as to be superimposed on a part of the boundary line CF.

[0050] The first direction display CFR is arranged on a circumferential boundary line CF by the second display image generating unit 136 with respect to a direction in which there is a risk of the vehicle 1 coming into contact with the first vehicle C1, with the vehicle 1 as a reference. The direction in which there is a risk of the vehicle 1 coming into contact with the first vehicle C1 is obtained by the risk detecting unit 134 as, for example, a range in which the boundary line CF overlaps with the collision risk image ARC. The first direction display CFR is displayed by the second display image generating unit 136 as, for example, a red arc-shaped image. The color of the first direction indication CFR is determined according to the degree of risk that the first vehicle C1 will come into contact with the vehicle 1. The degree of risk that the first vehicle C1 will come into contact with the vehicle 1 is detected by the risk detection unit 134. In this embodiment, it is detected that there is a high risk that the first vehicle C1 will come into contact with the vehicle 1, and the color of the first direction indication CFR is determined to be red. In FIG. 3, the fact that the color of the first direction indication CFR is red is indicated by dark hatching.

[0051] The second direction display CFY is arranged on a circumferential boundary line CF by the second display image generating unit 136 in a direction in which there is a risk of the vehicle 1 coming into contact with the second vehicle C2, with the vehicle 1 as a reference. The direction in which there is a risk of the vehicle 1 coming into contact with the second vehicle C2 is obtained by the risk detecting unit 134 as, for example, the direction in which the second vehicle C2 is located (here, southward), with the vehicle 1 as a reference. The second direction display CFY is displayed by the second display image generating section 136 as, for example, a yellow arc-shaped image. The color of the second direction indication CFY is determined according to the degree of risk that the second vehicle C2 will come into contact with the vehicle 1. The degree of risk that the second vehicle C2 will come into contact with the vehicle 1 is detected by the risk detection unit 134. In this embodiment, it is detected that the risk that the second vehicle C2 will come into contact with the vehicle 1 is medium, and the color of the second direction indication CFY is determined to be yellow. In FIG. 3, the color of the second direction indication CFY is indicated by light hatching to be yellow. When there is no need to distinguish between the first direction indication CFR and the second direction indication CFY, they may be referred to as the direction indication CFE in the following description.

[0052] Since the first direction indication CFR and the second direction indication CFY are displayed superimposed on a portion of the boundary line CF, the second display image generating unit 136 displays the boundary line CF in a second color, which is an inconspicuous color. The second color is, for example, gray. That is, the boundary line CF is displayed in, for example, gray by the second display image generating unit 136. The boundary line CF displayed in gray may be referred to as a second boundary line CF2.

[0053] As explained with reference to Figure 3, the display control unit 138 displays the first danger detection display ED1 near the left edge of the display panel 61, so that the user can visually recognize that a moving body MS that poses a risk of contact with vehicle 1 has been detected in the west direction relative to vehicle 1. In addition, the display control unit 138 displays the first danger detection display ED1 in red, so that the user can visually recognize that the danger of contact with the vehicle 1 is high.

[0054] As explained with reference to Figure 3, the display control unit 138 displays the second danger detection display ED2 near the bottom edge of the display panel 61, so that the user can visually recognize that a moving object MS that poses a risk of contact with vehicle 1 has been detected in the south direction relative to vehicle 1. Furthermore, the display control unit 138 displays the second danger detection display ED2 in yellow, so that the user can visually recognize that the risk of contact with the vehicle 1 is medium.

[0055] As described with reference to Figure 3, the display control unit 138 displays the collision risk image ARC in a semi-transparent red color on the first road image RW1, so that the user can visually recognize that the traveling speed of the first vehicle C1 corresponding to the collision risk image ARC is equal to or greater than the first speed.

[0056] 3, the display control unit 138 displays the first direction display CFR as an arc-shaped image in the range where the boundary line CF overlaps with the collision risk image ARC, so that the user can visually recognize the direction in which there is a risk of contact with the vehicle 1. In addition, the display control unit 138 displays the first direction display CFR as a red arc-shaped image, so that the user can visually recognize that there is a high risk of contact with the vehicle 1 in the direction in which the first direction display CFR is displayed.

[0057] 3, the display control unit 138 displays the second direction display CFY as an arc-shaped image in the direction in which the second vehicle C2 is located with respect to the vehicle 1 as a reference, so that the user can visually recognize the direction in which there is a risk of contact with the vehicle 1. In addition, the display control unit 138 displays the second direction display CFY as a yellow arc-shaped image, so that the user can visually recognize that there is a medium risk of contact with the vehicle 1 in the direction in which the second direction display CFY is displayed.

[0058] 4 is a screen diagram showing an example of a third screen 720 displaying the second display image PD2 in the third state. The third screen 720 is displayed on the display panel 61 by the display control unit 138. The third screen 720 differs from the second screen 710 shown in Fig. 3 in that the first danger detection display ED1 and the second danger detection display ED2 are not displayed on the third screen 720. In other words, the third state is a state after a predetermined period has elapsed from the second state.

[0059] The display control unit 138 blinks each of the first danger detection display ED1 and the second danger detection display ED2 for a predetermined period of time, and then ends the display of the first danger detection display ED1 and the second danger detection display ED2 on the display panel 61. As a result, as shown in the third screen 720 of Fig. 4, the first danger detection display ED1 and the second danger detection display ED2 are not displayed.

[0060] 4, the display control unit 138 blinks the first danger detection display ED1 and the second danger detection display ED2 for a predetermined period of time, and then ends the display of the first danger detection display ED1 and the second danger detection display ED2 on the display panel 61. Therefore, the blinking display of the first danger detection display ED1 and the second danger detection display ED2 focuses the user's attention on the first danger detection display ED1 and the second danger detection display ED2, and it is possible to suppress a decrease in visibility of the entire second display image PD2.

[0061] 5 is a screen diagram showing an example of a fourth screen 730 displaying the second display image PD2 in the fourth state. The fourth screen 730 is displayed on the display panel 61 by the display control unit 138. The fourth screen 730 differs from the third screen 720 in that a first vehicle image CR1 is displayed instead of the first vehicle mark CM1, and a second vehicle image CR2 is displayed instead of the second vehicle mark CM2.

[0062] In the fourth state, the first vehicle C1 moves from a position not included in the area indicated by the map image PG displayed on the display panel 61 to a position included in the area indicated by the map image PG displayed on the display panel 61. Then, the display control unit 138 displays the first vehicle image CR1 in place of the first vehicle mark CM1.

[0063] The first vehicle image CR1 is generated by the surrounding information image generating unit 131 based on the shape and size of the first vehicle C1 detected by the distance sensor 40. The surrounding information image generating unit 131 generates the first vehicle image CR1 by reading out the first vehicle image CR1 from images of multiple vehicles stored in the image storage unit 142 based on, for example, the shape and size of the first vehicle C1 detected by the distance sensor 40. In this embodiment, the first vehicle C1 is, for example, a four-wheeled passenger car, and therefore the first vehicle image CR1 is an image showing the four-wheeled passenger car.

[0064] In the fourth state, the second vehicle C2 moves from a position not included in the area indicated by the map image PG displayed on the display panel 61 to a position included in the area indicated by the map image PG displayed on the display panel 61. Then, the display control unit 138 displays the second vehicle image CR2 in place of the second vehicle mark CM2.

[0065] The second vehicle image CR2 is generated by the surrounding information image generating unit 131 based on the shape and size of the second vehicle C2 detected by the distance sensor 40. The surrounding information image generating unit 131 generates the second vehicle image CR2 by reading out the second vehicle image CR2 from the images of multiple vehicles stored in the image storage unit 142 based on, for example, the shape and size of the second vehicle C2 detected by the distance sensor 40. In this embodiment, the second vehicle C2 is, for example, a motorcycle, and therefore the second vehicle image CR2 is an image showing the motorcycle.

[0066] As described with reference to Fig. 5, when the first vehicle C1 moves from a position not included in the area indicated by the map image PG displayed on the display panel 61 to a position included in the area indicated by the map image PG displayed on the display panel 61, the display control unit 138 displays the first vehicle image CR1 instead of the first vehicle mark CM1. Therefore, the user can visually recognize the position of the first vehicle C1. In addition, the first vehicle image CR1 allows the user to visually recognize that the first vehicle C1 is a four-wheeled passenger vehicle.

[0067] As described with reference to Fig. 5, when the second vehicle C2 moves from a position not included in the area indicated by the map image PG displayed on the display panel 61 to a position included in the area indicated by the map image PG displayed on the display panel 61, the display control unit 138 displays the second vehicle image CR2 instead of the second vehicle mark CM2. Therefore, the user can visually recognize the position of the second vehicle C2. In addition, the second vehicle image CR2 allows the user to visually recognize that the second vehicle C2 is a motorcycle.

[0068] 6 is a screen diagram showing an example of a fifth screen 740 displaying the second display image PD2 in the fifth state. The fifth screen 740 is displayed on the display panel 61 by the display control unit 138. The fifth screen 740 showing the fifth state is different from the fourth screen 730 showing the fourth state in that the first vehicle C1 is passing through the intersection R3.

[0069] In the fifth state, since the first vehicle C1 is passing through the intersection R3, the second display image generating unit 136 positions the first vehicle image CR1 to the east of the intersection image RW3. In addition, since the first vehicle C1 is passing through the intersection R3, the risk detection unit 134 detects that there is no risk of the first vehicle C1 coming into contact with the vehicle 1. Therefore, the second display image generation unit 136 does not display the collision risk image ARC and the first direction display CFR.

[0070] 6, when the first vehicle C1 is passing through the intersection R3, the collision risk image ARC and the first direction display CFR are not displayed. Therefore, the user can visually confirm that there is no risk of contact with the first vehicle C1.

[0071] 7 is a screen diagram showing an example of a sixth screen 750 displaying the second display image PD2 in the sixth state. The sixth screen 750 is displayed on the display panel 61 by the display control unit 138. The sixth screen 750 showing the sixth state is different from the fifth screen 740 showing the fifth state in that a part of the second vehicle C2 is located inside the boundary line CF.

[0072] In the sixth state, a part of the second vehicle C2 is located inside the boundary line CF, so the second display image generating unit 136 does not display the second direction display CFY. In addition, the risk detecting unit 134 detects that there is a high risk of the second vehicle C2 coming into contact with the vehicle 1. Therefore, the display control unit 138 highlights the boundary line CF. For example, the display control unit 138 displays the boundary line CF in a third color indicating a high risk of coming into contact with the vehicle 1. The third color is, for example, red. The boundary line CF displayed in red may be referred to as the third boundary line CF3.

[0073] 7, when a portion of the second vehicle C2 is located inside the boundary line CF, the boundary line CF is displayed in red, so that the user can visually recognize that there is a high risk of contact with the second vehicle C2.

[0074] FIG. 8 is a screen diagram showing an example of a seventh screen 800 displaying the second display image PD2 in the seventh state. The seventh screen 800 is displayed on the display panel 61 by the display control unit 138.

[0075] In the second state shown in Figure 3 to the sixth state shown in Figure 7, we will explain the cases where the distance sensor 40 detects at least one of the first vehicle C1 and the first vehicle C1, but in the seventh state shown in Figure 8 to the ninth state shown in Figure 10, we will explain the cases where the distance sensor 40 does not detect a vehicle C but detects a pedestrian.

[0076] 8, the surrounding information image generating unit 131 generates a first pedestrian mark P1, a second pedestrian mark P2, a third pedestrian mark P3, and a fourth pedestrian mark P4 as the surrounding information image PS. Each of the first pedestrian mark P1 to the fourth pedestrian mark P4 is a circular mark.

[0077] The first pedestrian mark P1 is located in the west direction of the intersection image RW3 in the first road image RW1. Since the first pedestrian mark P1 is located in the first road image RW1, it is displayed in red, for example, which indicates that the user needs to be careful. In FIG. 8, the first pedestrian mark P1 is shown in red by dark hatching. Since the second pedestrian mark P2 is located in the sidewalk image SW within the first area AR1, it is displayed in yellow, which indicates that the user needs to be moderately careful. In Fig. 8, the second pedestrian mark P2 is shown in yellow with medium-thickness hatching. The third pedestrian mark P3 and the fourth pedestrian mark P4 are located on the sidewalk image SW, and therefore are displayed in gray, which indicates that the user does not need to pay much attention to them. In Fig. 8, the third pedestrian mark P3 and the fourth pedestrian mark P4 are shown in gray by applying light hatching.

[0078] The risk detection unit 134 detects that the pedestrians corresponding to each of the first pedestrian mark P1 to the fourth pedestrian mark P4 are at a low risk of contact with the vehicle 1. Therefore, the display control unit 138 displays the boundary line CF in, for example, a first color (e.g., green) indicating a low risk. In Fig. 8, the boundary line CF is shown by a dashed line to indicate that it is green for the sake of convenience.

[0079] FIG. 9 is a screen diagram showing an example of an eighth screen 810 displaying the second display image PD2 in the eighth state. The eighth screen 810 is displayed on the display panel 61 by the display control unit 138.

[0080] The difference between the seventh state shown in FIG. 8 and the eighth state shown in FIG. 9 is that the first pedestrian mark P1 is located in the second area AR2, whereas the eighth state shown in FIG. 9 is that the first pedestrian mark P1 is located in the first area AR1. Since the first pedestrian mark P1 is located in the first area AR1, the risk detection unit 134 detects that there is a high risk of the pedestrian corresponding to the first pedestrian mark P1 coming into contact with the vehicle 1. Therefore, the display control unit 138 displays the boundary line CF in, for example, a third color (e.g., red) indicating a high risk. In Fig. 9, the fact that the boundary line CF is red is shown by a solid line for the sake of convenience.

[0081] FIG. 10 is a screen diagram showing an example of a ninth screen 820 displaying the second display image PD2 in the ninth state. The ninth screen 820 is displayed on the display panel 61 by the display control unit 138.

[0082] The difference between the eighth state shown in FIG. 9 and the ninth state shown in FIG. 10 is that the first pedestrian mark P1 is located on the first road image RW1, whereas the first pedestrian mark P1 is located on the sidewalk image SW. Since the first pedestrian mark P1 is located on the sidewalk image SW within the first area AR1, it is displayed in yellow, for example, which indicates that the user needs to be moderately careful. Since the first pedestrian mark P1 is located on the sidewalk image SW within the first area AR1, the risk detection unit 134 detects that the risk of the pedestrian corresponding to the first pedestrian mark P1 coming into contact with the vehicle 1 is low. Therefore, the display control unit 138 displays the boundary line CF in, for example, a first color (e.g., green) indicating a low risk. In Fig. 10, the fact that the boundary line CF is green is shown by a dashed line for the sake of convenience.

[0083] As described with reference to Figs. 8 to 10, the risk detection unit 134 detects the risk of the pedestrian coming into contact with the vehicle 1, depending on the position of the pedestrian detected by the distance sensor 40. Then, the display control unit 138 displays the boundary line CF in a color according to the risk detected by the risk detection unit 134. Therefore, the user can visually recognize the degree of risk of the pedestrian coming into contact with the vehicle 1, depending on the color of the boundary line CF.

[0084] Next, an example of an image display process of the vehicle surrounding information display device 100 will be described with reference to FIGS. FIG. 11 is a flowchart showing an example of processing performed by the vehicle surrounding information display device 100.

[0085] First, as shown in FIG. 11, in step S101, the map image acquisition unit 132 acquires position information indicating the position of the vehicle 1 from the position detection unit 10. Next, in step S103, the map image acquisition unit 132 reads out the map image PG corresponding to the position information acquired in step S101 from the map storage unit 141, thereby acquiring the map image PG of the surroundings of the vehicle 1. Next, in step S105, the surrounding information image generating unit 131 acquires point cloud data from the distance sensor 40. Next, in step S107, the surrounding information image generating unit 131 generates a surrounding information image PS showing surrounding information of the vehicle 1, based on the point cloud data acquired in step S105.

[0086] Next, in step S109, the overhead image acquisition unit 133 acquires images captured by the front camera 31, the rear camera 33, the left side camera 35, and the right side camera 37, respectively. Next, in step S111, the overhead image acquisition unit 133 generates an overhead image PB of the surroundings of the vehicle 1 by synthesizing the captured images acquired in step S109, and acquires the overhead image PB of the surroundings of the vehicle 1. Next, in step S113, the second display image generation unit 136 synthesizes the map image PG, the vehicle image PR indicating the position of the vehicle 1, the overhead image PB arranged around the vehicle image PR, and the surrounding information image PS arranged around the overhead image PB, to generate the second display image PD2 to be displayed on the display panel 61. Further, the first display image generating unit 135 generates a first display image PD1 to be displayed on the display panel 61 by combining the map image PG with the host vehicle image PR indicating the position of the vehicle 1. In addition, the third display image generation unit 137 synthesizes the map image PG, the host vehicle image PR showing the position of the vehicle 1, and an overhead image PB arranged around the host vehicle image PR, to generate the third display image PD3 to be displayed on the display panel 61. Next, in step S115, the second display image generating unit 136 and the display control unit 138 execute a “danger display process.” The “danger display process” is a process of generating an image showing the detection result of the danger detection unit 134 and displaying it on the display panel 61. The image showing the detection result of the risk detection unit 134 includes a boundary line CF, a direction display CFE, a risk detection display ED, and a collision risk image ARC. The direction display CFE includes a first direction display CFR and a second direction display CFY. The risk detection display ED includes a first risk detection display ED1 and a second risk detection display ED2. The "danger display process" will be further described with reference to FIGS.

[0087] Next, in step S117, the display control unit 138 executes a display image determination process. The “display image determination process” refers to a process of determining which image is to be displayed among the first display image PD1, the second display image PD2, and the third display image PD3 generated in step S113. The display control unit 138 displays the determined display image PD on the display panel 61. The "display image determination process" will be further described with reference to FIGS. Next, in step S119, the vehicle surrounding information display device 100 accepts a user's operation on the operation unit 50 and determines whether or not to end the image display process. When the vehicle surrounding information display device 100 determines not to end the image display process (step S119; NO), the process returns to step S101. When the vehicle surrounding information display device 100 determines that the image display process is to end (step S119; YES), the process ends thereafter.

[0088] Steps S105 and S107 correspond to an example of a "periphery information image generating step". Steps S101 and S103 correspond to an example of a "map image acquiring step". Steps S109 and S111 correspond to an example of a "bird's-eye image acquiring step". Step S113 corresponds to an example of a "first display image generating step" and a "second display image generating step". Step S117 corresponds to an example of a "display control step".

[0089] 11, the second display image generating unit 136 synthesizes the map image PG, the host vehicle image PR indicating the position of the vehicle 1, the overhead image PB arranged around the host vehicle image PR, and the surrounding information image PS arranged around the overhead image PB to generate the second display image PD2 to be displayed on the display panel 61. Then, the display control unit 138 displays the second display image PD2 generated by the second display image generating unit 136 on the display panel 61. Therefore, it is possible to display a wider range of surrounding information as the second display image PD2 than the overhead image PB.

[0090] Next, the danger display process executed by the vehicle surrounding information display device 100 will be described with reference to FIG. 12 and FIG. 12 and 13 are flowcharts showing an example of a danger display process of the vehicle surrounding information display device 100. Note that, in Fig. 12 and Fig. 13, a case will be described in which the display control unit 138 displays the second display image PD2 on the display panel 61.

[0091] First, as shown in FIG. 12, in step S201, the risk detection unit 134 determines, based on the point cloud data from the distance sensor 40, whether or not a moving object MS has been detected. When the risk detection unit 134 determines that a moving object MS has not been detected (step S201; NO), the process proceeds to step S203. Then, in step S203, the risk detection unit 134 detects that there is a low risk of the moving object MS coming into contact with the vehicle 1. The second display image generation unit 136 generates the boundary line CF in a first color (e.g., green) indicating that there is a low risk of the vehicle 1 coming into contact with the moving object MS. The display control unit 138 displays the generated first boundary line CF1 in green on the display panel 61. Thereafter, the process returns to step S117 in FIG. 11. When the risk detection unit 134 determines that a moving object MS has been detected (step S201; YES), the process proceeds to step S205. Then, in step S205, the surrounding information image generating unit 131 determines, based on the point cloud data from the distance sensor 40, whether or not the moving object MS is a pedestrian.

[0092] When the surrounding information image generating unit 131 determines that the moving object MS is not a pedestrian (step S205; NO), the process proceeds to step S207. Then, in step S207, the second display image generating unit 136 generates the boundary line CF in an inconspicuous second color (for example, gray). The display control unit 138 displays the generated gray second boundary line CF2 on the display panel 61. In addition, the risk detection unit 134 detects that the moving object MS is a vehicle C. Then, the process proceeds to step S219 shown in FIG. When the surrounding information image generating unit 131 determines that the moving object MS is a pedestrian (step S205; YES), the process proceeds to step S209. Then, in step S209, the risk detection unit 134 determines whether or not a pedestrian is on the roadway. When the risk detection unit 134 determines that a pedestrian is not on the roadway (step S209; NO), the process proceeds to step S211. Then, in step S211, the risk detection unit 134 determines that the pedestrian is on the sidewalk and detects that the risk of the pedestrian coming into contact with the vehicle 1 is low. The second display image generation unit 136 generates the boundary line CF in a first color (e.g., green) indicating that the risk of the vehicle 1 coming into contact with the pedestrian is low. The display control unit 138 displays the generated first boundary line CF1 in green on the display panel 61. Thereafter, the process returns to step S117 in FIG. 11.

[0093] When the risk detection unit 134 determines that a pedestrian is on the roadway (step S209; YES), the process proceeds to step S213. Then, in step S213, the risk detection unit 134 determines whether or not a pedestrian is present in the first area AR1. When the risk detection unit 134 determines that no pedestrian is present in the first area AR1 (step S213; NO), the process proceeds to step S215. Then, in step S215, the risk detection unit 134 detects that there is a low risk of the pedestrian coming into contact with the vehicle 1. The second display image generation unit 136 generates the boundary line CF in a first color (e.g., green) indicating that there is a low risk of the vehicle 1 coming into contact with the pedestrian. The display control unit 138 displays the generated first boundary line CF1 in green on the display panel 61. Thereafter, the process returns to step S117 in FIG. 11. When the risk detection unit 134 determines that a pedestrian is in the first area AR1 (step S213; YES), the process proceeds to step S217. Then, in step S217, the risk detection unit 134 detects that there is a high risk of the pedestrian coming into contact with the vehicle 1. The second display image generation unit 136 generates the boundary line CF in a third color (e.g., red) indicating that there is a high risk of the vehicle 1 coming into contact with the pedestrian. The display control unit 138 displays the generated red third boundary line CF3 on the display panel 61. Thereafter, the process returns to step S117 in FIG. 11.

[0094] If the answer is NO in step S205, in step S207, the moving body MS is detected to be vehicle C, and as shown in FIG. 13, in step S219, the risk detection unit 134 determines whether the traveling speed of vehicle C is high, that is, equal to or higher than a first speed (e.g., 70 km / h). When the risk detection unit 134 determines that the traveling speed of the vehicle C is high (step S219; YES), the process proceeds to step S231. When the risk detection unit 134 determines that the traveling speed of the vehicle C is not high (step S219; NO), the process proceeds to step S221. Then, in step S221, the risk detection unit 134 determines whether the traveling speed of the vehicle C is a medium speed equal to or higher than a second speed (for example, 30 km / hour). When the risk detection unit 134 determines that the traveling speed of the vehicle C is not medium (step S221; NO), the process then returns to step S117 in Fig. 11. When the risk detection unit 134 determines that the traveling speed of the vehicle C is medium (step S221; YES), the process proceeds to step S223. Then, in step S223, the display control unit 138 determines whether or not the flashing of the danger detection display ED corresponding to the medium speed vehicle C has ended. Here, the danger detection display ED indicates the second danger detection display ED2 shown in FIG. When the display control unit 138 determines that the flashing of the danger detection display ED has ended (step S223; YES), the process proceeds to step S225. Then, in step S225, the risk detection unit 134 detects a direction in which there is a risk of contact with the vehicle 1. The second display image generation unit 136 generates the direction display CFE as a yellow arc-shaped image and places it on the circumferential boundary line CF. The display control unit 138 displays the direction display CFE on the display panel 61. The direction display CFE corresponds to the second direction display CFY shown in FIG. 3. After that, the process returns to step S117 in FIG. 11.

[0095] In step S219, the risk detection unit 134 determines whether or not the traveling speed of the vehicle C is high, but is not limited to this. In addition to or instead of the determination based on the traveling speed of the vehicle C, the risk detection unit 134 may determine the risk based on whether or not the difference in the traveling speed of the vehicle C from the traveling speed of the vehicle 1 is equal to or greater than a specified value. For example, the risk detection unit 134 may determine the risk based on whether or not the speed ratio between the traveling speed of the vehicle 1 and the traveling speed of the vehicle C is 1:2 or more. Also, for example, the risk may be determined based on whether or not the difference between the traveling speed of the vehicle 1 and the traveling speed of the vehicle C is 60 km / hour or more. In this way, for example, when the vehicle 1 and the vehicle C are traveling in the same direction on a highway or the like, the cases where the traveling speeds of both vehicles are approximately the same and high can be excluded from the risk determination. Also, when the vehicle 1 and the vehicle C are traveling in the same direction, the risk of the two vehicles approaching each other suddenly can be detected.

[0096] When the display control unit 138 determines that the flashing of the danger detection display ED has not ended (step S223; NO), the process proceeds to step S227. Then, in step S227, the risk detection unit 134 detects that the risk of contact with the vehicle 1 is medium. The display control unit 138 determines the color of the risk detection display ED to be, for example, yellow. The display control unit 138 displays the risk detection display ED in a flashing manner at the end of the display panel 61 in the direction in which there is a risk of contact with the vehicle 1, based on the vehicle 1. The risk detection display ED corresponds to the second risk detection display ED2 shown in FIG. Next, in step S229, the risk detection unit 134 detects a direction in which there is a risk of contact with the vehicle 1. The second display image generation unit 136 generates a direction display CFE as a yellow arc-shaped image and places it on the circumferential boundary line CF. The display control unit 138 displays the direction display CFE on the display panel 61. The direction display CFE corresponds to the second direction display CFY shown in FIG. 3. Then, the process proceeds to step S243.

[0097] If the result of step S219 is YES, then in step S231, the display control unit 138 determines whether or not the flashing of the danger detection display ED has ended. Here, the danger detection display ED indicates the first danger detection display ED1 shown in FIG. When the display control unit 138 determines that the flashing of the danger detection display ED has ended (step S231; YES), the process proceeds to step S233. Then, in step S233, the second display image generating unit 136 generates a collision risk image ARC, and the display control unit 138 displays the collision risk image ARC. The collision risk image ARC is an image showing the traveling direction of the vehicle C and an area where there is a risk of collision. The collision risk image ARC is the collision risk image ARC shown in FIG. Next, in step S235, the risk detection unit 134 detects a direction in which there is a risk of contact with the vehicle 1. The second display image generation unit 136 generates a direction display CFE as a red arc-shaped image and places it on the circumferential boundary line CF. The display control unit 138 displays the direction display CFE on the display panel 61. The direction display CFE corresponds to the first direction display CFR shown in FIG. 3. After that, the process returns to step S117 in FIG. 11.

[0098] If the result is NO in step S231, in step S237, the risk detection unit 134 detects that there is a high risk of contact with the vehicle 1. The display control unit 138 determines the color of the risk detection display ED to be, for example, red. The display control unit 138 displays the risk detection display ED in a flashing manner at the end of the display panel 61 in the direction in which there is a risk of contact with the vehicle 1, based on the vehicle 1. The risk detection display ED corresponds to the first risk detection display ED1 shown in FIG. Next, in step S239, the second display image generating unit 136 generates a collision risk image ARC, and the display control unit 138 displays the collision risk image ARC. Next, in step S241, the risk detection unit 134 detects a direction in which there is a risk of contact with the vehicle 1. The second display image generation unit 136 generates a direction display CFE as a red arc-shaped image and places it on the circumferential boundary line CF. The display control unit 138 displays the direction display CFE on the display panel 61. The direction display CFE corresponds to the first direction display CFR shown in FIG. 3.

[0099] Next, in step S243, the display control unit 138 determines whether or not a predetermined period (for example, 3 seconds) has elapsed since the flashing of the danger detection display ED was started in step S227 or step S237. If the display control unit 138 determines that the predetermined period has not elapsed (step S243; NO), the process goes to a standby state. If the display control unit 138 determines that the predetermined period has elapsed (step S243; YES), the process proceeds to step S245. Then, in step S245, the display control unit 138 ends the blinking of the danger detection display ED. After that, the process returns to step S117 in FIG.

[0100] As explained with reference to Figures 12 and 13, when the moving body MS is vehicle C, the traveling speed of vehicle C is high, and it is detected that there is a high risk of contact with vehicle 1, the first danger detection display ED1, the collision risk image ARC, and the first direction display CFR are displayed. Since the first danger detection display ED1 is displayed, the user can visually recognize that a vehicle C that is at high risk of contact with the vehicle 1 has been detected in the direction in which the first danger detection display ED1 is displayed. In addition, the collision risk image ARC is displayed, allowing the user to visually recognize the area where there is a risk of collision with vehicle C traveling at high speed. Since the first direction indication CFR is displayed, the user can visually confirm the direction in which there is a high possibility of collision with the vehicle C traveling at high speed.

[0101] Next, with reference to FIGS. 14 to 18, an example of the transition of the display image PD displayed on the display panel 61 when transitioning from the first display image PD1 to the second display image PD2 will be described. 14 to 18 each show directions. The upper direction in the figure corresponds to the north direction, the lower direction in the figure corresponds to the south direction, the right direction in the figure corresponds to the east direction, and the left direction in the figure corresponds to the west direction.

[0102] 14 is a screen diagram showing an example of a tenth screen displaying a first display state ST1 of a transition from a first display image PD1 to a second display image PD2. The first display image PD1 is displayed on a part of the tenth screen 900 by the display control unit 138. Specifically, the first display image PD1 is displayed on the center of the tenth screen 900 in the horizontal direction, and is displayed on the lower part of the tenth screen 900 in the vertical direction.

[0103] The first display image PD1 is generated by the first display image generating unit 135. The first display image PD1 includes a map image PG and a host vehicle mark PM indicating the position of the vehicle 1. The map image PG is the same as the map image PG described with reference to Fig. 2. That is, the map image PG includes a road image RW and a sidewalk image SW. The road image RW includes a first road image RW1 showing a first road R1 extending in an east-west direction, a second road image RW2 showing a second road R2 extending in a north-south direction, and an intersection image RW3 showing an intersection R3 where the first road R1 and the second road R2 intersect.

[0104] The vehicle 1 is traveling north on the second road R2 and is approaching the intersection R3. The first display image generating unit 135 places the host vehicle mark PM on the second road image RW2, to the south of the intersection image RW3. The host vehicle mark PM is, for example, a triangular mark. The host vehicle mark PM is, for example, an isosceles triangular mark, and the apex with the smaller apex angle is disposed in the traveling direction of the vehicle 1. In other words, of the three apexes constituting the host vehicle mark PM, the apex with the smaller apex angle is disposed in the north direction on the host vehicle mark PM.

[0105] 14, the user touches the host vehicle mark PM with a finger. The touch sensor 63 detects a pressure value PV of the touch operation of the user's finger on the display panel 61, and generates a pressure signal indicating the pressure value PV of the detected touch operation. The display unit 60 then outputs the pressure signal indicating the pressure value PV to the vehicle surrounding information display device 100. The display control unit 138 transitions the state of the display image PD displayed on the display panel 61 based on the pressure value PV.

[0106] When the pressure value PV is less than the second pressure value PV2, the display control unit 138 displays an eleventh screen 910 shown in FIG. 15 is a screen diagram showing an example of an eleventh screen 910 displaying a second display state ST2 in which the first display image PD1 transitions to the second display image PD2. In the second display state ST2, the pressure value PV is less than the second pressure value PV2.

[0107] The display control unit 138 arranges a circular area AR11 centered on the position of the vehicle 1 in the first display image PD1, and displays the overhead image PB and the host vehicle image PR within the circular area AR11. The size of the circular region AR11 for the first display image PD1 is determined by the display control unit 138 in accordance with the pressure value PV. The size of the circular region AR11 for the first display image PD1 is determined by the display control unit 138 to be larger as the pressure value PV is larger. However, the size of the circular region AR11 for the first display image PD1 is smaller than the first region AR1 for the first display image PD1 shown in FIGS. 2 to 10. The host vehicle image PR is placed at the position of the vehicle 1 by the display control unit 138 in place of the host vehicle mark PM shown in FIG.

[0108] When the touch operation is ended while the eleventh screen 910 shown in FIG. 15 is being displayed on the display panel 61, the display control unit 138 displays the tenth screen 900 shown in FIG. When the 11th screen 910 shown in FIG. 15 is displayed on the display panel 61, i.e., in the second display state ST2, if the pressure value PV caused by the touch operation of the user's finger on the vehicle image PR reaches the second pressure value PV2, the display control unit 138 displays the 12th screen 920 shown in FIG. 16 on the display panel 61.

[0109] FIG. 16 is a screen diagram showing an example of a twelfth screen 920 displaying a third display state ST3 in which the first display image PD1 transitions to the second display image PD2. When the pressure value PV reaches the second pressure value PV2, the display control unit 138 displays the twelfth screen 920 on the display panel 61.

[0110] On the twelfth screen 920, the third display image PD3 is displayed by the display control unit 138. The third display image PD3 is generated by the third display image generating unit 137. The third display image PD3 is generated by the third display image generating unit 137 by combining a map image PG, a host vehicle image PR indicating the position of the vehicle 1, and an overhead image PB arranged around the host vehicle image PR. A boundary line CF is displayed at the boundary between the overhead image PB and the map image PG.

[0111] When the touch operation is ended while the twelfth screen 920 shown in FIG. 16 is being displayed on the display panel 61, the display control unit 138 continues to display the twelfth screen 920 shown in FIG. 16 on the display panel 61. When the pressure value PV reaches the first pressure value PV1, the display control unit 138 displays a thirteenth screen 930 shown in Fig. 17 on the display panel 61. The first pressure value PV1 is greater than the second pressure value PV2.

[0112] FIG. 17 is a screen diagram showing an example of a thirteenth screen 930 displaying a fourth display state ST4 in which the first display image PD1 transitions to the second display image PD2. The second display image PD2 is displayed on the thirteenth screen 930 by the display control unit 138. The second display image PD2 is generated by the second display image generating unit 136. The second display image PD2 is generated by the second display image generating unit 136 by synthesizing a map image PG, a host vehicle image PR indicating the position of the vehicle 1, an overhead image PB arranged around the host vehicle image PR, and a surrounding information image PS arranged around the overhead image PB. The second display image PD2 also includes a boundary line CF. The surrounding information image PS includes the first vehicle mark CM1 and the second vehicle mark CM2 described with reference to FIG. Further, the first direction display CFR, the second direction display CFY, and the collision risk image ARC are displayed on the thirteenth screen 930. The first direction display CFR, the second direction display CFY, and the collision risk image ARC are displayed on the display panel 61 by the display control unit 138.

[0113] When the touch operation is ended while the 13th screen 930 shown in FIG. 17 is being displayed on the display panel 61, the display control unit 138 continues to display the 13th screen 930 shown in FIG. 17 on the display panel 61. When the pressure value PV reaches the third pressure value PV3, the display control unit 138 displays a fourteenth screen 940 shown in Fig. 18 on the display panel 61. The third pressure value PV3 is greater than the first pressure value PV1.

[0114] FIG. 18 is a screen diagram showing an example of a fourteenth screen 940 displaying a fifth display state ST5 in which the first display image PD1 transitions to the second display image PD2. In the fourteenth screen 940, the second display image PD2 is displayed in full screen. Further, the first direction display CFR, the second direction display CFY, and the collision risk image ARC are displayed on the fourteenth screen 940. The first direction display CFR, the second direction display CFY, and the collision risk image ARC are displayed on the display panel 61 by the display control unit 138. The fourteenth screen 940 corresponds to the third screen 720 shown in FIG.

[0115] When the touch operation is ended while the fourteenth screen 940 shown in FIG. 18 is being displayed on the display panel 61, the display control unit 138 continues to display the fourteenth screen 940 shown in FIG. 18 on the display panel 61.

[0116] 14 to 18, the user can perform a touch operation with a finger on the host vehicle mark PM or the host vehicle image PR and increase the pressure value PV by the touch operation to a first pressure value PV1, thereby transitioning the display image PD displayed on the display panel 61 from the first display image PD1 to the second display image PD2. Therefore, the image displayed on the display panel 61 can be switched from the first display image PD1 to the second display image PD2 with a simple operation.

[0117] Furthermore, the display control unit 138 switches from the first display image PD1 to the second display image PD2 via the third display image PD3. Thus, the third display image PD3 can be displayed on the display panel 61. Therefore, for example, when the vehicle 1 is traveling at a low speed (specifically, when the vehicle 1 is about to be parked in reverse into a parking position), the third display image PD3 can be displayed on the display panel 61.

[0118] 19 is a state transition diagram showing an example of transition among the first display image PD1, the second display image PD2, and the third display image PD3. The display control unit 138 executes the "display image determination process", and as shown in FIG. 19, the display state displayed on the display panel 61 transitions. First, a description will be given of a transition from a first display state ST1 in which the first display image PD1 is displayed on the display panel 61 to a fifth display state ST5 in which the second display image PD2 is displayed in full screen on the display panel 61. Note that, for convenience, Fig. 19 describes a case in which the touch operation is continued until the pressure value PV reaches the third pressure value PV3 after the touch operation is started.

[0119] When a touch operation with the user's finger is detected and the pressure value PV due to the touch operation is less than the second pressure value PV2, the first display state ST1 transitions to the second display state ST2. In the second display state ST2, as described with reference to Fig. 15, a circular area AR11 is arranged in the first display image PD1, and the overhead image PB and the host vehicle image PR are displayed within the circular area AR11. The size of the circular area AR11 is smaller than that of the first area AR1.

[0120] When the pressure value PV increases and reaches the second pressure value PV2, the second display state ST2 transitions to the third display state ST3. In the third display state ST3, the third display image PD3 is displayed on the display panel 61.

[0121] When the pressure value PV further increases and reaches the first pressure value PV1, the third display state ST3 transitions to a fourth display state ST4. In the fourth display state ST4, the second display image PD2 is displayed on the display panel 61.

[0122] Furthermore, when the pressure value PV further increases and reaches the third pressure value PV3, the fourth display state ST4 transitions to a fifth display state ST5. In the fifth display state ST5, the second display image PD2 is displayed on the display panel 61 in full screen.

[0123] In this way, the user can cause a transition from the first display state ST1 to the third display state ST3 and display the third display image PD3 on the display panel 61 by increasing the pressure value PV by the touch operation to the second pressure value PV2. Moreover, the user can cause a transition from the third display state ST3 to the fourth display state ST4 by increasing the pressure value PV to the first pressure value PV1, thereby causing the display panel 61 to display the second display image PD2. Furthermore, the user can change the fourth display state ST4 to a fifth display state ST5 by increasing the pressure value PV to the third pressure value PV3, and display the second display image PD2 on the display panel 61 in full screen.

[0124] Next, a description will be given of a case where the touch operation is continued from the fifth display state ST5 in which the second display image PD2 is displayed full-screen on the display panel 61 to the first display state ST1 in which the first display image PD1 is displayed on the display panel 61. For convenience, Fig. 19 describes a case where the touch operation is continued until the pressure value PV reaches the third pressure value PV3 after the touch operation is started.

[0125] When a touch operation with the user's finger is detected and the pressure value PV of the touch operation is equal to or greater than the second pressure value PV2, a transition is made from the fifth display state ST5 to the fourth display state ST4. When the pressure value PV increases and reaches the first pressure value PV1, the fourth display state ST4 transitions to the third display state ST3. In the third display state ST3, the third display image PD3 is displayed on the display panel 61.

[0126] If the pressure value PV further increases and becomes greater than the first pressure value PV1 and less than the third pressure value PV3, the third display state ST3 transitions to the second display state ST2. In the second display state ST2, a circular area AR11 smaller than the first area AR1 is arranged in the first display image PD1, and the overhead image PB and the host vehicle image PR are displayed within the circular area AR11. When the pressure value PV further increases and reaches a third pressure value PV3, the second display state ST2 transitions to the first display state ST1. In the first display state ST1, the first display image PD1 is displayed on the display panel 61.

[0127] In this way, the user can transition from the fifth display state ST5 to the fourth display state ST4 and display the second display image PD2 on part of the display panel 61 by increasing the pressure value PV by the touch operation to the second pressure value PV2. Moreover, the user can cause a transition from the fourth display state ST4 to the third display state ST3 by increasing the pressure value PV to the first pressure value PV1, and cause the display panel 61 to display the third display image PD3. Furthermore, the user can cause a transition from the third display state ST3 to the first display state ST1 by increasing the pressure value PV to the third pressure value PV3, thereby causing the display panel 61 to display the first display image PD1.

[0128] Fig. 20 is a flowchart showing an example of a display image determination process when changing the display image PD from the first display image PD1 to the second display image PD2. The process shown in Fig. 20 corresponds to a part of the "display image determination process" executed in step S117 in Fig. 11. For convenience, Fig. 20 describes a case where the touch operation is continued until the pressure value PV reaches the third pressure value PV3 after the touch operation is started.

[0129] First, in step S301, the display control unit 138 receives a user's operation on the operation unit 50 and displays a first display image PD1 on the display panel 61. Next, in step S303, the display control unit 138 determines whether or not the touch sensor 63 of the display unit 60 has detected a touch operation from the user on the host vehicle mark PM displayed on the first display image PD1. When the display control unit 138 determines that the user's touch operation on the vehicle mark PM has not been detected (step S303; NO), the process returns to step S301. When the display control unit 138 determines that the user's touch operation on the vehicle mark PM has been detected (step S303; YES), the process proceeds to step S305. Then, in step S305, the display control unit 138 acquires the detection result of the pressure value PV due to the touch operation from the touch sensor 63.

[0130] Next, in step S307, the display control unit 138 determines whether or not the pressure value PV is less than the second pressure value PV2. When the display control unit 138 determines that the pressure value PV is less than the second pressure value PV2 (step S307; YES), the process proceeds to step S309. Then, in step S309, the display control unit 138 displays the overhead image PB and the host vehicle image PR in the circular area AR11 according to the pressure value PV. After that, the process returns to step S119 shown in FIG. When the display control unit 138 determines that the pressure value PV is not less than the second pressure value PV2 (step S307; NO), the process proceeds to step S311.

[0131] Then, in step S311, the display control unit 138 determines whether or not the pressure value PV is the second pressure value PV2. When the display control unit 138 determines that the pressure value PV is the second pressure value PV2 (step S311; YES), the process proceeds to step S313. Then, in step S313, the display control unit 138 displays the third display image PD3 on the display panel 61. After that, the process returns to step S119 shown in FIG. When the display control unit 138 determines that the pressure value PV is not the second pressure value PV2 (step S311; NO), the process proceeds to step S315.

[0132] Then, in step S315, the display control unit 138 determines whether or not the pressure value PV is greater than the second pressure value PV2 and less than the first pressure value PV1. If the display control unit 138 determines that the pressure value PV is greater than the second pressure value PV2 and less than the first pressure value PV1 (step S315; YES), the process proceeds to step S317. Then, in step S317, the display control unit 138 displays the second display image PD2 in an area corresponding to the pressure value PV. The size of the area corresponding to the pressure value PV is equal to or smaller than the area in which the first display image PD1 is displayed in step S301. The area corresponding to the pressure value PV is, for example, a rectangular area. The rectangular area is, for example, a quadrilateral area similar to the rectangular area in which the first display image PD1 is displayed in step S301. In this state, in the area in which the first display image PD1 is displayed in step S301, the second display image PD2 is displayed inside the rectangular area, and the first display image PD1 is displayed outside the rectangular area. Then, the process returns to step S119 shown in FIG. 11. When the display control unit 138 determines that the pressure value PV is greater than the second pressure value PV2 and is not less than the first pressure value PV1 (step S315; NO), the process proceeds to step S319.

[0133] Then, in step S319, the display control unit 138 determines whether the pressure value PV is equal to or greater than the first pressure value PV1 and less than the third pressure value PV3. When the display control unit 138 determines that the pressure value PV is equal to or greater than the first pressure value PV1 and less than the third pressure value PV3 (step S319; YES), the process proceeds to step S321. Then, in step S321, the display control unit 138 displays the second display image PD2 on the display panel 61. After that, the process returns to step S119 shown in FIG. When the display control unit 138 determines that the pressure value PV is equal to or greater than the first pressure value PV1 and is not less than the third pressure value PV3 (step S319; NO), the process proceeds to step S323.

[0134] Then, in step S323, the display control unit 138 determines whether or not the pressure value PV is equal to or greater than the third pressure value PV3. When the display control unit 138 determines that the pressure value PV is equal to or greater than the third pressure value PV3 (step S323; YES), the process proceeds to step S325. Then, in step S325, the display control unit 138 displays the second display image PD2 in full screen on the display panel 61. After that, the process returns to step S119 shown in FIG. When the display control unit 138 determines that the pressure value PV is not equal to or greater than the third pressure value PV3 (step S323; NO), the process returns to step S119 shown in FIG.

[0135] 20, the display control unit 138 transitions the image displayed on the display panel 61 from the first display image PD1 to the second display image PD2 via the third display image PD3 based on the pressure value PV due to the touch operation. Therefore, the user can easily display the third display image PD3 and the second display image PD2 by adjusting the pressure value PV.

[0136] As described above, the vehicle surrounding information display device 100 according to this embodiment includes a map image acquisition unit 132 that acquires a map image PG of the surroundings of the vehicle 1, a first display image generation unit 135 that combines the map image PG with a host vehicle mark PM indicating the position of the vehicle 1 to generate a first display image PD1 to be displayed on the display panel 61, a surrounding information image generation unit 131 that generates a surrounding information image PS showing surrounding information of the vehicle 1 based on point cloud data indicating the distance from the vehicle 1 acquired from the distance sensor 40, and an overhead image PB of the surroundings of the vehicle 1. the map image PG, the vehicle's own vehicle image PR indicating the position of the vehicle 1, the bird's-eye view image PB arranged around the vehicle's own vehicle image PR, and the surrounding information image PS arranged around the bird's-eye view image PB to generate a second display image PD2 to be displayed on the display panel 61; and a display control unit 138 that receives an operation from a user and switches the image to be displayed on the display panel 61 between the first display image PD1 and the second display image PD2 in accordance with the operation.

[0137] Therefore, in response to user operation, the image displayed on the display panel 61 is switched between the first display image PD1 and the second display image PD2, so that the user can switch the image displayed on the display panel 61 between the first display image PD1 and the second display image PD2. The first display image PD1 is generated by combining the map image PG with a host vehicle mark PM indicating the position of the vehicle 1. The first display image PD1 corresponds to a so-called navigation screen. Therefore, when the user desires to display the navigation screen, the user can display the first display image PD1. The second display image PD2 is generated by combining the map image PG, the vehicle image PR indicating the position of the vehicle 1, the overhead image PB arranged around the vehicle image PR, and the surrounding information image PS arranged around the overhead image PB. Therefore, the second display image PD2 includes the surrounding information image PS in a wider range than the overhead image PB. Therefore, when there is a risk of contact with another vehicle C or a pedestrian, the user can display the second display image PD2.

[0138] The display control unit 138 also includes a third display image generation unit 137 that synthesizes the map image PG, the vehicle image PR, and the overhead image PB arranged around the vehicle image PR to generate a third display image PD3 to be displayed on the display panel 61, and the display control unit 138 switches from the first display image PD1 to the second display image PD2 via the third display image PD3. Therefore, the user can select the image to be displayed on the display panel 61 from among the first display image PD1, the second display image PD2, and the third display image PD3. This improves user convenience. Moreover, the third display image PD3 is generated by synthesizing the map image PG, the host vehicle image PR, and the overhead image PB arranged around the host vehicle image PR. Therefore, for example, when the vehicle 1 is traveling at a low speed (specifically, when the vehicle 1 is about to be parked in a parking position, etc.), the third display image PD3 can be displayed on the display panel 61.

[0139] In addition, the display panel 61 is configured as a touch panel equipped with a touch sensor 63, and the operation is a touch operation on the vehicle mark PM or the vehicle image PR displayed on the touch panel, and the display control unit 138 switches between the first display image PD1 and the second display image PD2 depending on the pressure value PV detected by the touch sensor 63 by the touch operation. Therefore, since the first display image PD1 and the second display image PD2 are switched in accordance with the pressure value PV detected by the touch sensor 63 by a touch operation, it is possible to easily switch between the first display image PD1 and the second display image PD2.

[0140] In this embodiment, a case will be described in which the display control unit 138 switches between the first display image PD1 and the second display image PD2 according to the pressure value PV, but the present invention is not limited to this. The display control unit 138 may switch between the first display image PD1 and the second display image PD2 according to a touch time indicating the time during which a touch operation continues. In this case, the touch sensor 63 does not need to detect the pressure value PV, so that a touch panel can be realized with a simple configuration.

[0141] Furthermore, when a touch operation on the host vehicle mark PM is received while the first display image PD1 is being displayed, the display control unit 138 enlarges the area in which the second display image PD2 is displayed in accordance with the pressure value PV. Therefore, the second display image PD2 can be displayed in a partial area of ​​the first display image PD1. This improves user convenience. In addition, since the area in which the second display image PD2 is displayed is enlarged according to the pressure value PV, the user can display the second display image PD2 in a desired size.

[0142] In the present embodiment, the display control unit 138 expands the area in which the second display image PD2 is displayed in accordance with the pressure value PV, but is not limited to this. The display control unit 138 may expand the area in which the second display image PD2 is displayed in accordance with the touch duration indicating the duration of the touch operation.

[0143] Furthermore, the display control unit 138 displays a circumferential boundary line CF indicating the area in which the second display image PD2 is to be displayed. Therefore, the user can visually recognize that the second display image PD2 is displayed inside the boundary line CF, which improves user convenience.

[0144] In addition, when the pressure value PV is equal to or greater than the first pressure value PV1 and the state changes to one in which the touch operation is not detected, the display control unit 138 displays the second display image PD2 on the display panel 61, and when the pressure value PV is less than the first pressure value PV1 and equal to or greater than a second pressure value PV2 that is smaller than the first pressure value PV1 and the state changes to one in which the touch operation is not detected, the display control unit 138 displays the third display image PD3 on the display panel 61. Therefore, the user can display the second display image PD2 on the display panel 61 by stopping the touch operation (i.e., removing the finger from the touch panel) when the pressure value PV is equal to or greater than the first pressure value PV1. Also, the user can display the third display image PD3 on the display panel 61 by stopping the touch operation (i.e., removing the finger from the touch panel) when the pressure value PV is less than the first pressure value PV1 and equal to or greater than a second pressure value PV2 that is smaller than the first pressure value PV1. Therefore, the second display image PD2 and the third display image PD3 can be easily displayed.

[0145] In the present embodiment, the display control unit 138 displays the second display image PD2 or the third display image PD3 based on the pressure value PV before the touch operation is stopped, but the present invention is not limited to this. The display control unit 138 may display the second display image PD2 or the third display image PD3 based on the touch time before the touch operation is stopped. For example, the display control unit 138 may display the second display image PD2 on the display panel 61 when the touch time is a first time or more and the state changes to a state where the touch operation is not detected, and may display the third display image PD3 on the display panel 61 when the touch time is less than the first time and is a second time or more that is shorter than the first time and the state changes to a state where the touch operation is not detected.

[0146] In addition, when the display control unit 138 detects a touch operation on the vehicle image PR while the second display image PD2 or the third display image PD3 is displayed on the display panel 61, it switches the image displayed on the display panel 61 to the first display image PD1. Therefore, the user can easily display the first display image PD1 on the display panel 61 from a state in which the second display image PD2 or the third display image PD3 is displayed on the display panel 61. This improves user convenience.

[0147] The vehicle surrounding information display method according to this embodiment includes a map image acquisition step (steps S101 and S103) of acquiring a map image PG of the surroundings of the vehicle 1, a first display image generation step (step S113) of synthesizing the map image PG with a host vehicle mark PM indicating the position of the vehicle 1 to generate a first display image PD1 to be displayed on the display panel 61, a surrounding information image generation step (steps S105 and S107) of generating a surrounding information image PS indicating surrounding information of the vehicle 1 based on point cloud data indicating the distance from the vehicle 1 acquired from the distance sensor 40, and The method includes an overhead image acquisition step (steps S109 and S111) of acquiring an overhead image PB, a second display image generation step (step S113) of synthesizing a map image PG, an own vehicle image PR, the overhead image PB arranged around the own vehicle image PR, and a surrounding information image PS arranged around the overhead image PB to generate a second display image PD2 to be displayed on the display panel 61, and a display control step (step S117) of accepting an operation from a user and switching the image to be displayed on the display panel 61 between the first display image PD1 and the second display image PD2 in accordance with the operation. Therefore, the vehicle surrounding information display method according to this embodiment has the same effects as those of the vehicle surrounding information display device 100 according to this embodiment.

[0148] The above-described embodiment is merely an example of one embodiment of the present invention, and any modifications and applications are possible without departing from the spirit and scope of the present invention.

[0149] For example, in order to facilitate understanding of the present invention, the components are classified according to the main processing content in Fig. 1, but the components can be further classified into more components according to the processing content. Also, one component can be classified so that it executes more processes. Furthermore, the processing of each component may be executed by a single piece of hardware, or may be executed by multiple pieces of hardware. Furthermore, the processing of each component may be realized by one program, or may be realized by multiple programs.

[0150] In addition, in FIG. 1, the vehicle surrounding information display device 100 may include at least one of the detection unit 20 and the display unit 50 integrally.

[0151] In the present embodiment, the vehicle surrounding information display device 100 includes, but is not limited to, the surrounding information image generating unit 131, the map image acquiring unit 132, the overhead image acquiring unit 133, the risk detecting unit 134, the first display image generating unit 135, the second display image generating unit 136, the third display image generating unit 137, and the display control unit 138. A server device communicably connected to the vehicle surrounding information display device 100 via a network such as the Internet may include at least one of the surrounding information image generating unit 131, the map image acquiring unit 132, the overhead image acquiring unit 133, the risk detecting unit 134, the first display image generating unit 135, the second display image generating unit 136, the third display image generating unit 137, and the display control unit 138. The server device may include, for example, the map image acquiring unit 132 and the overhead image acquiring unit 133. In this case, the overhead image acquisition unit 133 may acquire an overhead image PB captured by an artificial satellite.

[0152] In addition, in this embodiment, as described with reference to Figure 19, a case has been described in which the display control unit 138 transitions from a state in which the first display image PD1 is displayed on the display panel 61 (i.e., the first display state ST1), via a state in which the third display image PD3 is displayed (i.e., the third display state ST3), to a state in which the second display image PD2 is displayed on the display panel 61 (i.e., the fourth display state ST4), but this is not limited to this. The display control unit 138 may transition from a state in which the first display image PD1 is displayed on the display panel 61 (i.e., the first display state ST1) to a state in which the second display image PD2 is displayed on the display panel 61 (i.e., the fourth display state ST4) without passing through a state in which the third display image PD3 is displayed (i.e., the third display state ST3). For example, when the display control unit 138 detects a touch operation on the host vehicle mark PM in a state in which the first display image PD1 is displayed on the display panel 61, the display control unit 138 may transition to a state in which the second display image PD2 is displayed on the display panel 61. In this case, the processing can be simplified.

[0153] In addition, in this embodiment, as described with reference to Figure 19, a case has been described in which the display control unit 138 transitions from a state in which the second display image PD2 is displayed on the display panel 61 (i.e., the fourth display state ST4), via a state in which the third display image PD3 is displayed (i.e., the third display state ST3), to a state in which the first display image PD1 is displayed on the display panel 61 (i.e., the first display state ST1), but this is not limited to this. The display control unit 138 may transition from a state in which the second display image PD2 is displayed on the display panel 61 (i.e., the fourth display state ST4) to a state in which the first display image PD1 is displayed on the display panel 61 (i.e., the first display state ST1) without passing through a state in which the third display image PD3 is displayed (i.e., the third display state ST3). For example, when the display control unit 138 detects a touch operation on the host vehicle image PR in a state in which the second display image PD2 is displayed on the display panel 61, the display control unit 138 may transition to a state in which the first display image PD1 is displayed on the display panel 61. In this case, the processing can be simplified.

[0154] Furthermore, in this embodiment, as described with reference to FIG. 19, a case has been described in which the display control unit 138 displays the overhead image PB in an area corresponding to the pressure value PV when the first display image PD1 is displayed on the display panel 61 (i.e., the first display state ST1) and the pressure value PV is equal to or less than the second pressure value PV2, but this is not limited to the above. For example, when the first display image PD1 is displayed on the display panel 61 (i.e., the first display state ST1) and the pressure value PV is less than or equal to the second pressure value PV2, the display control unit 138 may maintain the first display state ST1. Also, for example, when the display control unit 138 is in a state where the first display image PD1 is displayed on the display panel 61 (i.e., the first display state ST1) and the pressure value PV is equal to or less than the second pressure value PV2, it may transition to a state where the third display image PD3 is displayed on the display panel 61 (i.e., the third display state ST3) or a state where the second display image PD2 is displayed on the display panel 61 (i.e., the fourth display state ST4).

[0155] Furthermore, in this embodiment, as described with reference to FIG. 19, when the pressure value PV increases in a state in which the second display image PD2 is displayed on the display panel 61 (i.e., the fourth display state ST4), the display control unit 138 transitions to a state in which the first display image PD1 is displayed on the display panel 61 (i.e., the first display state ST1) via the third display state ST3 and the second display state ST2 in sequence, but this is not limited to the above. For example, when the display control unit 138 detects a touch operation in a state in which the second display image PD2 is displayed on the display panel 61 (i.e., the fourth display state ST4), the display control unit 138 may transition to a state in which the first display image PD1 is displayed on the display panel 61 (i.e., the first display state ST1).

[0156] Furthermore, when the vehicle surrounding information display method of the present invention is realized using a computer, the control program executed by the computer can be configured in the form of a recording medium or a transmission medium for transmitting the control program. The recording medium may be a magnetic or optical recording medium or a semiconductor memory device. Specifically, the recording medium may be a portable or fixed recording medium such as a flexible disk, HDD, CD-ROM (Compact Disk Read Only Memory), DVD, Blu-ray (registered trademark) Disc, magneto-optical disk, flash memory, or card-type recording medium. The recording medium may be a non-volatile storage device such as a RAM, ROM, or HDD provided in the vehicle surrounding information display device 100. The control program may be downloaded by the vehicle surrounding information display device 100 from a server device communicably connected to the vehicle surrounding information display device 100 via a network.

[0157] 11, 12, 13, and 20 are divided according to the main processing contents in order to facilitate understanding of the processing of the vehicle surrounding information display device 100, and the present invention is not limited by the manner in which the processing units are divided or the names of the processing units. The processing of the vehicle surrounding information display device 100 may be divided into more processing units according to the processing contents. Furthermore, the processing of the vehicle surrounding information display device 100 may be divided so that one processing unit includes more processes. [Explanation of symbols]

[0158] 100 Vehicle surrounding information display device 1 vehicle 10 Position detection unit 20 Detection unit 30 Photography Department 40 Distance sensor (sensor) 50 Control section 60 Display section 61 Display panel (display) 130 processors 131 Surrounding information image generation unit 132 Map image acquisition unit 133 Bird's-eye view image acquisition unit 134 Danger detection unit 135 First image generating unit 136 Second image generating unit 137 Third Image Generation Unit 138 Display control section 140 Memory 141 Map memory section 142 Image storage unit AR1 1st area AR2 2nd area CF border MS Mobile PM Vehicle mark (vehicle mark) PR Vehicle image (vehicle mark) PD1 1st display image PD2 2nd display image PD3 3rd display image PB overhead view PG map image PS Peripheral Information Image PV Pressure Value PV1 First pressure value PV2 Second pressure value PV3 3rd pressure value

Claims

1. a map image acquisition unit that acquires a map image of the surroundings of the vehicle; a first display image generating unit that generates a first display image to be displayed on a display by combining the map image with a vehicle mark indicating the position of the vehicle; a surrounding information image generating unit that generates a surrounding information image showing surrounding information of the vehicle based on point cloud data indicating a distance from the vehicle obtained from a sensor; an overhead image acquisition unit that acquires an overhead image of the surroundings of the vehicle; a second display image generating unit that generates a second display image to be displayed on a display by synthesizing the map image, the vehicle mark, the overhead image arranged around the vehicle mark, and the peripheral information image arranged around the overhead image; a display control unit that receives an operation from a user and switches an image to be displayed on the display between the first display image and the second display image in response to the operation; A vehicle surrounding information display device comprising:

2. a third display image generating unit that generates a third display image to be displayed on a display by synthesizing the map image, the vehicle mark, and the overhead image arranged around the vehicle mark; the display control unit switches from the first display image to the second display image via the third display image; The vehicle surrounding information display device according to claim 1 .

3. The display is configured as a touch panel having a touch sensor; the operation is a touch operation on the vehicle mark displayed on the touch panel, the display control unit switches between the first display image and the second display image in response to at least one of a pressure value detected by the touch sensor due to the touch operation and a touch time indicating a time during which the touch operation is continuing. The vehicle surrounding information display device according to claim 2 .

4. When a touch operation on the vehicle mark is received while the first display image is being displayed, the display control unit enlarges an area in which the overhead image is displayed in accordance with the pressure value or the touch time. The vehicle surrounding information display device according to claim 3.

5. the display control unit displays a circumferential boundary line indicating an area in which the overhead image is displayed. The vehicle surrounding information display device according to claim 4.

6. The display control unit is When the pressure value is equal to or greater than a first pressure value or the touch time is equal to or greater than a first time, the touch operation is changed to a state in which the touch operation is not detected. When the state where the touch operation is not detected is changed in a state where the pressure value is less than the first pressure value and is equal to or greater than a second pressure value smaller than the first pressure value, or in a state where the touch time is less than the first time and is equal to or greater than a second time shorter than the first time, the third display image is displayed on the display. The vehicle surrounding information display device according to any one of claims 3 to 5.

7. When the display control unit detects a touch operation on the vehicle mark while the second display image or the third display image is being displayed on the display, the display control unit switches the image to be displayed on the display to the first display image. The vehicle surrounding information display device according to any one of claims 3 to 6.

8. A map image acquisition step of acquiring a map image of a surrounding area of ​​the vehicle; a first display image generating step of generating a first display image to be displayed on a display by combining the map image with a vehicle mark indicating the position of the vehicle; a surrounding information image generating step of generating a surrounding information image showing surrounding information of the vehicle based on point cloud data showing a distance from the vehicle obtained from a sensor; an overhead image acquisition step of acquiring an overhead image of the surroundings of the vehicle; a second display image generating step of synthesizing the map image, the vehicle mark, the overhead image arranged around the vehicle mark, and the peripheral information image arranged around the overhead image to generate a second display image to be displayed on a display; a display control step of accepting an operation from a user and switching an image to be displayed on the display between the first display image and the second display image in response to the operation; A method for displaying information surrounding a vehicle, comprising:

Images