Vehicle peripheral image display device

Abstract

To effectively display surrounding images according to the progress of automatic parking.SOLUTION: A vehicle peripheral image display device displays images of the surroundings of a vehicle capable of executing automatic parking control to automatically enter or exit a specified parking space. The device includes an image generation unit that can generate multiple peripheral images from different viewpoints on the basis of images captured by a camera mounted on the vehicle, and an image display control unit that, while the automatic parking control is being executed, selects a peripheral image corresponding to the progress of the automatic parking control from the multiple peripheral images that the image generation unit can generate, and displays the selected peripheral image on the vehicle's display device.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present disclosure relates to a vehicle peripheral image display device. [Background technology]

[0002] For example, Patent Document 1 discloses a vehicle surroundings display device that changes the display area of ​​an overhead image according to the direction of travel of the vehicle when displaying the overhead image of the surroundings of the vehicle on the touch panel of a navigation device installed in the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-076645 Summary of the Invention

[0004] In recent years, parking assistance systems have been put into practical use that can perform automatic parking, in which a vehicle is driven automatically to enter or leave a designated parking space. Information that a driver wants to know while such automatic parking is being performed (for example, an image of the vehicle's surroundings displayed on a display device) changes depending on the progress of the automatic parking. The technology described in Patent Document 1 only changes the display area of ​​the overhead image, so there is a possibility that the information that the driver wants to know may not be accurately provided.

[0005] The present disclosure has been made to solve the above-mentioned problem, and aims to provide a technology that can effectively display surrounding images according to the progress of automatic parking.

[0006] The vehicle peripheral image display device of the present disclosure includes: A vehicle surroundings image display device that displays an image of the surroundings of a vehicle capable of executing automatic parking control to automatically enter a predetermined parking space or automatically exit a vehicle from the parking space, an image generating means capable of generating a plurality of surrounding images from different viewpoints based on images captured by a camera mounted on the vehicle; and an image display control means for selecting a surrounding image according to the progress of the automatic parking control from the plurality of surrounding images that can be generated by the image generation means during execution of the automatic parking control, and displaying the selected surrounding image on a display device of the vehicle. It is characterized by: [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram showing a hardware configuration of a vehicle according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram showing a software configuration of the control device according to the present embodiment. [Figure 3] FIG. 2 is a schematic diagram showing an example of a target movement path set by the automatic parking control unit according to the present embodiment. [Figure 4] 3 is a schematic diagram showing an example of a first overhead view image generated by a surrounding image generating unit according to the present embodiment and a camera viewpoint image of a view in front of the vehicle. FIG. [Figure 5] 10 is a schematic diagram showing an example of a second overhead view image generated by a surrounding image generating unit according to the present embodiment and a camera viewpoint image behind the vehicle. FIG. [Figure 6] 10 is a schematic diagram showing an example of a third overhead view image generated by a surrounding image generating unit according to the present embodiment, and a virtual viewpoint image in which the vehicle is viewed from above and diagonally forward to the right. FIG. [Figure 7] 10 is a schematic diagram showing an example of a fourth overhead view image generated by the surrounding image generating unit according to the present embodiment, and a virtual viewpoint image in which the vehicle is viewed from above and diagonally rear left of the vehicle. FIG. [Figure 8] 10 is a flowchart illustrating a routine for processing peripheral image display control according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, a vehicle surroundings image display device according to this embodiment will be described with reference to the drawings.

[0009] [Hardware configuration] 1 is a schematic diagram showing the hardware configuration of a vehicle VH according to this embodiment. Hereinafter, the vehicle VH may also be referred to as the host vehicle when it is necessary to distinguish it from other vehicles.

[0010] The vehicle VH has an ECU (Electronic Control Unit) 10. The ECU 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, and an interface device 14. The CPU 11 is a processor that executes various programs stored in the ROM 12. The ROM 12 is a non-volatile memory that stores data and the like required for the CPU 11 to execute the various programs. The RAM 13 is a volatile memory that provides a working area into which the various programs are expanded when the CPU 11 executes them. The interface device 14 is a communication device for communicating with external devices.

[0011] The ECU 10 is a central control device that performs automatic entry control, which causes the vehicle VH to automatically enter a designated parking space, and automatic exit control, which causes the vehicle VH to automatically exit a designated parking space (hereinafter, the automatic entry control and automatic exit control may be collectively referred to as "automatic parking control"). The ECU 10 is communicatively connected to a drive unit 20, a steering unit 21, a braking unit 22, a transmission 23, an internal sensor unit 30, an external sensor unit 40, an HMI (Human Machine Interface) 60, an automatic parking switch 70, and the like.

[0012] The drive unit 20 generates a drive force to be transmitted to the drive wheels of the vehicle VH. Examples of the drive unit 20 include an electric motor and an engine. In this embodiment, the vehicle VH may be a hybrid electric vehicle (HEV), a plug-in hybrid electric vehicle (PHEV), a fuel cell electric vehicle (FCEV), an electric vehicle (BEV), or an engine vehicle. The steering unit 21 applies a steering force to the wheels of the vehicle VH. The braking unit 22 applies a braking force to the wheels of the vehicle VH. The transmission 23 is configured to be selectively switchable among a parking range that locks the rotation of the drive wheels, a reverse range that drives the vehicle VH backward, a neutral range that cuts off the transmission of power, and a drive range that drives the vehicle VH forward.

[0013] The internal sensor device 30 is a group of sensors that detect the state of the vehicle VH. Specifically, the internal sensor device 30 includes a vehicle speed sensor 31, an accelerator sensor 32, a brake sensor 33, a steering angle sensor 34, a shift sensor 35, and the like. The vehicle speed sensor 31 detects the traveling speed of the vehicle VH, i.e., the vehicle speed. The accelerator sensor 32 detects the amount of operation of an accelerator pedal (not shown) by the driver. The brake sensor 33 detects the amount of operation of a brake pedal (not shown) by the driver. The steering angle sensor 34 detects the rotation angle of a steering wheel or steering shaft (not shown), i.e., the steering angle. The shift sensor 35 detects the shift position (parking P, reverse R, neutral N, drive D, etc.) of the transmission 23. The internal sensor device 30 transmits the state of the vehicle VH detected by each of the sensors 31 to 35 to the ECU 10 at a predetermined interval.

[0014] The external sensor device 40 is an example of an obstacle detection means of the present disclosure, and is a type of sensor that recognizes target information related to targets around the vehicle VH. Specifically, the external sensor device 40 includes a radar sensor 41, a sonar sensor 42, a camera sensor 43, etc.

[0015] The radar sensor 41 includes a front radar sensor that recognizes the area ahead of the vehicle VH, a rear radar sensor that recognizes the area behind the vehicle VH, a left side radar sensor that recognizes the area to the left of the vehicle VH, and a right side radar sensor that recognizes the area to the right of the vehicle VH. The radar sensor 41 includes a millimeter-wave radar and / or a lidar. The millimeter-wave radar emits radio waves in the millimeter wave band and receives the waves reflected by targets within the emission range to obtain the relative distance, relative speed, etc. between the vehicle VH and the target. The lidar sequentially scans a pulsed laser light with a wavelength shorter than millimeter waves in multiple directions and receives the reflected light reflected by the target to obtain the shape of the target and the relative distance, relative speed, etc. between the vehicle VH and the target.

[0016] The sonar sensor 42 includes a front sonar sensor that recognizes the area ahead of the vehicle VH, a rear sonar sensor that recognizes the area behind the vehicle VH, a left side sonar sensor that recognizes the area to the left of the vehicle VH, a right side sonar sensor that recognizes the area to the right of the vehicle VH, etc. The sonar sensor 42 emits ultrasonic waves and receives reflected waves from targets present within the emission range to obtain the relative distance, relative speed, etc. between the vehicle VH and the target.

[0017] The camera sensor 43 includes a front camera sensor that captures images in front of the vehicle VH, a rear camera sensor that captures images behind the vehicle VH, a left side camera sensor that captures images on the left side of the vehicle VH, and a right side camera sensor that captures images on the right side of the vehicle VH. The camera sensor 43 is, for example, a stereo camera or a monocular camera, and a digital camera having an imaging element such as a CMOS or CCD can be used. The camera sensor 43 processes the captured image data to obtain the shape of the target, the relative distance and relative speed between the vehicle VH and the target, etc.

[0018] The external sensor device 40 transmits the acquired target information at a predetermined cycle to the ECU 10. Note that the external sensor device 40 does not necessarily have to include all of the radar sensor 41, the sonar sensor 42, and the camera sensor 43, and may include, for example, only the camera sensor 43.

[0019] The HMI 60 is an interface for inputting and outputting information between the ECU 10 and the driver, and includes an input device and an output device. Examples of the input device include a touch panel liquid crystal display, a switch, and a voice pickup microphone. Examples of the output device include a display device 61 and a speaker 62. The display device 61 is, for example, a center display or a multi-information display. The speaker 62 is, for example, a speaker of an audio system or a navigation system.

[0020] The automatic parking switch 70 is provided, for example, on the center console or instrument panel of the vehicle VH, and is a switch that is turned on by an occupant of the vehicle VH (for example, the driver). When the automatic parking switch 70 is turned on, the ECU 10 receives an automatic parking start request. Note that the automatic parking switch 70 is not limited to a physical switch, and may be a touch-type switch image displayed on the display device 61. The automatic parking start request may also be acquired by voice recognition using a voice pickup microphone, or by gesture recognition using a driver camera.

[0021] [Software configuration] 2 is a schematic diagram showing the software configuration of the ECU 10 according to this embodiment. As shown in FIG. 2, the ECU 10 includes, as functional elements, an automatic parking control unit 100, a surroundings image generation unit 110, a surroundings image display control unit 120, and the like. These functional elements 100 to 120 are realized by the CPU 11 of the ECU 10 reading a program stored in the ROM 12 into the RAM 13 and executing the program. Note that all or part of the functional elements 100 to 120 may be provided in another ECU separate from the ECU 10, or in an information processing device in a facility (such as a management center) that can communicate with the vehicle VH.

[0022] The automatic parking control unit 100 executes automatic parking control to automatically enter or exit a predetermined parking space by causing the vehicle VH to travel along a target travel path. Note that the automatic parking control basically involves the same processing when entering the vehicle VH and when exiting the vehicle VH, so the following will only describe the case when entering the vehicle VH, and will omit the description of the case when exiting the vehicle VH.

[0023] The automatic parking control unit 100 detects a parking space (hereinafter referred to as a parking space) around the vehicle VH where the vehicle VH can be parked, based on the target object information transmitted from the external sensor device 40. When the automatic parking control unit 100 detects a parking space and the driver turns on the automatic parking switch 70, the automatic parking control unit 100 displays the parking space on the display device 61. If the automatic parking control unit 100 has detected multiple parking spaces, the automatic parking control unit 100 displays these multiple parking spaces on the display device 61. When the driver selects a specific parking space by, for example, touching the screen of the display device 61, the automatic parking control unit 100 sets the parking space as the target parking space. The driver's selection may be made by operating a dial switch provided on the center console or the like. Alternatively, the driver's selection may be obtained based on voice recognition using a voice pickup microphone or the like.

[0024] Once the target parking space is set, the automatic parking control unit 100 displays a parking mode selection screen on the display device 61. Examples of parking modes include reverse parallel parking, forward parallel parking, reverse parallel parking, and forward parallel parking. As with the selection of an available parking space described above, the parking mode can be selected by touch operation, dial operation, or voice recognition. The following describes an example in which the driver selects reverse parallel parking as the parking mode.

[0025] When reverse parallel parking is selected, the automatic parking control unit 100 sets a target movement path for the vehicle VH to enter the target parking space. Fig. 3 shows an example of the target movement path Rt set by the automatic parking control unit 100. The target movement path Rt includes a forward movement path R1 for the vehicle VH to move forward (forward movement control of the present disclosure) from the parking start position S1, a turning position P1 for stopping the vehicle VH (stop control of the present disclosure) and switching the traveling direction from forward to reverse, a reverse movement path R2 for the vehicle VH to move backward (reverse movement control of the present disclosure) from the turning position P1, and a target stop position P2 for stopping the vehicle VH at a predetermined position within the target parking space PL.

[0026] Once the target movement path Rt is set, the automatic parking control unit 100 displays a confirmation screen G1 (shown in FIG. 3) including the target movement path Rt on the display device 61. When the driver touches a start button B1 on the confirmation screen G1, the automatic parking control unit 100 starts automatic parking control. The automatic parking control is performed, for example, by feedback controlling the operation of the drive unit 20, the steering unit 21, etc. based on the deviation between the target movement path Rt and the actual movement path of the vehicle VH. The actual movement path of the vehicle VH may be obtained, for example, by odometry based on the detection results of the vehicle speed sensor 31 and the steering angle sensor 34, or by optical flow based on road surface images around the vehicle VH captured by the camera sensor 43. The driver's start command is not limited to a touch operation, but may also be a dial operation, voice recognition, etc.

[0027] The surrounding image generating unit 110 generates an image of the surroundings of the vehicle VH (hereinafter referred to as the surrounding image) based on the target information transmitted from the external sensor device 40. The surrounding image is an image corresponding to at least a part of the range of the area surrounding the vehicle VH, and includes a camera viewpoint image and a composite image, etc. The camera viewpoint image is an image whose viewpoint is the position where each lens of the camera sensor 43 is disposed. One of the composite images is an image (hereinafter referred to as the virtual viewpoint image) of the surroundings of the vehicle VH viewed from a virtual viewpoint set at any position around the vehicle VH.

[0028] This method of generating a virtual viewpoint video is well known (see, for example, Japanese Patent Application Laid-Open No. 2012-217000, Japanese Patent Application Laid-Open No. 2016-192772, and Japanese Patent Application Laid-Open No. 2018-107754, etc.). The surroundings video generation unit 110 generates a video by further combining (superimposing) on ​​each of the camera viewpoint video and the virtual viewpoint video a vehicle image (for example, a polygon showing the shape of the vehicle VH) and a graphic image constituting lines that support the parking operation.

[0029] To briefly outline the method for generating a virtual viewpoint video, the peripheral video generation unit 110 projects pixels included in the front image, rear image, left side image, and right side image captured by the camera sensor 43 onto a predetermined projection surface (e.g., a hemispherical curved surface) in a virtual three-dimensional space. The center of the projection surface is defined as the position of the vehicle VH. Portions other than the center of the projection surface correspond to the front image, rear image, left side image, and right side image. The peripheral video generation unit 110 projects information about pixels included in the front image, rear image, left side image, and right side image onto portions other than the center of the projection surface. The peripheral video generation unit 110 places a polygon representing the shape of the vehicle VH at the center of the projection surface. The peripheral video generation unit 110 then sets a virtual viewpoint in the virtual three-dimensional space and cuts out a predetermined region of the projection surface that is within a predetermined viewing angle as viewed from the virtual viewpoint. Furthermore, the peripheral video generation unit 110 superimposes a polygon representing the shape of the vehicle VH that is within a predetermined viewing angle as viewed from the virtual viewpoint onto the cut-out image (video). This generates a virtual viewpoint video.

[0030] FIG. 4 is a schematic diagram illustrating the first overhead view image GB1 generated by the surroundings image generating unit 110 and a camera viewpoint image GF in front of the vehicle VH (hereinafter referred to as a front camera viewpoint image).

[0031] The first overhead image GB1 is a virtual viewpoint image obtained by cutting out an area of ​​the projection curved surface included in a predetermined field of view when viewing the projection curved surface from a virtual viewpoint set directly above the vehicle VH. The polygon SP of the host vehicle VH, the target parking space PL, etc. are superimposed and displayed on the first overhead image GB1. The front camera viewpoint image GF is an image captured mainly by the front camera sensor of the camera sensor 43. The polygon SP of the host vehicle VH, the target movement path Rt, the target parking space PL, obstacle highlight marks EF1, EF2, etc. are superimposed and displayed on the front camera viewpoint image GF. The obstacle highlight marks EF1, EF2 are figures that surround obstacles OB1, OB2, such as other vehicles ahead of the host vehicle VH, which are detected by the external environment sensor device 40.

[0032] The surrounding image generating unit 110 displays the obstacle highlighting mark EF1 of the obstacle OB1, which has a large impact if the predicted trajectory intersects with the target travel route Rt, in a first color (e.g., red) to call the driver's attention. Also, the surrounding image generating unit 110 displays the obstacle highlighting mark EF2 of the obstacle OB2, which has a small impact if the predicted trajectory does not intersect with the target travel route Rt, in a second color (e.g., amber) that is lighter than the first color.

[0033] FIG. 5 is a schematic diagram illustrating the second overhead view image GB2 generated by the surroundings image generating unit 110 and a camera viewpoint image (hereinafter referred to as rear camera viewpoint image) GR behind the vehicle VH.

[0034] Similar to the first overhead image GB1 described above, the second overhead image GB2 displays the polygon SP of the host vehicle VH, the target parking space PL, etc. superimposed thereon. The rear camera viewpoint image GR is an image captured primarily by the rear camera sensor of the camera sensor 43. The rear camera viewpoint image GR displays the polygon SP of the host vehicle VH, the target movement path Rt, the target parking space PL, obstacle highlight marks EF3, EF4, etc. superimposed thereon. The obstacle highlight marks EF3, EF4 are figures or the like superimposed on obstacles OB3, OB4, such as poles behind the host vehicle VH, which are detected by the external sensor device 40. The surrounding image generation unit 110 extracts obstacles OB3, OB4 that exist within a predetermined distance range from the target movement path Rt from among the obstacles, and superimposes the obstacle highlight marks EF3, EF4 on the extracted obstacles OB3, OB4. The colors of the obstacle highlighting marks EF3 and EF4 may all be the same color, or may change depending on the distance from the target movement route Rt.

[0035] FIG. 6 is a schematic diagram illustrating the third overhead view video GB3 generated by the surroundings video generating unit 110 and a virtual viewpoint video GFD in which the vehicle VH is viewed from above diagonally forward right (hereinafter referred to as diagonally forward right upper virtual viewpoint video).

[0036] Similar to the first overhead view image GB1 and the like, the third overhead view image GB3 displays the host vehicle VH's polygon SP, target parking space PL, etc. superimposed thereon. The right-front-upper virtual viewpoint image GFD is a virtual viewpoint image obtained by cutting out an area on the projection curved surface included in a predetermined viewing angle when viewing the projection curved surface from a virtual viewpoint set diagonally above and to the right of the vehicle VH. Note that the virtual viewpoint image of the vehicle VH viewed from above and to the left is the right-front-upper virtual viewpoint image GFD reversed left and right, and therefore a description thereof will be omitted. The right-front-upper virtual viewpoint image GFD displays the host vehicle VH's polygon SP, target movement path Rt, target parking space PL, obstacle highlight mark EF5, predicted outer turning passage line LO, etc. superimposed thereon.

[0037] The predicted outside passing line LO is a line that indicates the portion (boundary) of the area through which the body of the vehicle VH will pass that is predicted to protrude most to the outside of the turning. The obstacle highlighting mark EF5 is a figure or the like that is superimposed on an obstacle OB5, such as a pole, that is present on the outside of the turning of the vehicle VH and is detected by the external sensor device 40. The surrounding image generating unit 110 extracts, from among the obstacles, obstacle OB5 that is present within a predetermined distance range from the predicted outside passing line LO, and superimposes the obstacle highlighting mark EF5 on the extracted obstacle OB5.

[0038] FIG. 7 is a schematic diagram illustrating a fourth overhead view image GB4 generated by the surroundings image generating unit 110 and a virtual viewpoint image GRD in which the vehicle VH is viewed from above and diagonally rear left (hereinafter referred to as a left diagonally rear upper virtual viewpoint image).

[0039] The fourth overhead view image GB4, like the first overhead view image GB1 and the like, displays the host vehicle VH's polygon SP, target parking space PL, etc. superimposed thereon. The left-rear-upper virtual viewpoint image GRD is a virtual viewpoint image obtained by cutting out an area on the projection curved surface included in a predetermined viewing angle when viewing the projection curved surface from a virtual viewpoint set diagonally above and to the left of the vehicle VH. Note that the virtual viewpoint image viewed from above and to the right of the vehicle VH is a left-rear-upper virtual viewpoint image GRD that is a mirror image of the left-rear-upper virtual viewpoint image GRD, and therefore a description thereof will be omitted. The left-rear-upper virtual viewpoint image GRD displays the host vehicle VH's polygon SP, target movement path Rt, target parking space PL, obstacle highlight mark EF6, predicted inside turning passage line LI, etc. superimposed thereon.

[0040] The predicted inside passing line LI is a line that indicates the portion (boundary) of the area through which the body of the vehicle VH will pass that is predicted to protrude most toward the inside of the turn. The obstacle highlighting mark EF6 is a figure or the like that is superimposed on an obstacle OB6, such as a pole, that is present on the inside of the turn of the host vehicle VH and is detected by the external sensor device 40. The surrounding image generating unit 110 extracts, from among the obstacles, an obstacle OB6 that is present within a predetermined distance range from the predicted inside passing line LI, and superimposes the obstacle highlighting mark EF6 on the extracted obstacle OB6.

[0041] 2 again, the surrounding image display control unit 120 executes surrounding image display control to appropriately display the image generated by the surrounding image generation unit 110 on the display device 61 according to the progress of the automatic parking control while the automatic parking control unit 100 is executing the automatic parking control. Specific details of the surrounding image display control will be described below.

[0042] When the vehicle VH travels forward on the forward path R1 of the target movement path Rt from the parking start position S1 to the turning position P1 under automatic parking control, it is considered that the driver's line of sight is directed forward of the vehicle VH. While the vehicle VH travels on the forward path R1 under automatic parking control (forward control), the surrounding image display control unit 120 displays the front camera viewpoint image GF and the first overhead image GB1 (see FIG. 4) generated by the surrounding image generation unit 110 on the display device 61. In this case, the display area of ​​the front camera viewpoint image GF may be expanded as the vehicle speed of the vehicle VH increases, and may be reduced as the vehicle speed of the vehicle VH decreases.

[0043] In this way, while the vehicle VH is traveling forward under automatic parking control, by displaying the front camera viewpoint image GF on the display device 61, the driver can appropriately recognize from the front camera viewpoint image GF the presence of an obstacle OB1 that may affect the traveling of the host vehicle VH, and the presence of an obstacle OB2 that does not affect the traveling of the host vehicle VH. In other words, the driver can easily determine whether or not to continue automatic parking while the host vehicle VH is traveling forward.

[0044] When the vehicle VH reaches the turning position P1 due to automatic parking control, it is considered that the driver's line of sight is directed to the rear of the vehicle VH via the rearview mirror or the like. When the vehicle VH stops at the turning position P1 due to automatic parking control (stop control), the surroundings image display control unit 120 displays the rear camera viewpoint image GR and the second overhead image GB2 (see FIG. 5) generated by the surroundings image generation unit 110 on the display device 61. The timing for displaying the rear camera viewpoint image GR on the display device 61 (i.e., the timing for switching from the front camera viewpoint image GF) may be the timing when the vehicle VH reaches the turning position P1, or may be a predetermined time before the vehicle VH reaches the turning position P1.

[0045] In this way, when the vehicle VH is stopped at the turning position P1 by the automatic parking control, the driver can effectively grasp the situation behind the vehicle VH from the rear camera viewpoint image GR by displaying the rear camera viewpoint image GR on the display device 61. In other words, the driver can easily determine whether or not it is okay to start reversing the vehicle VH by the automatic parking.

[0046] When the vehicle VH travels backward while turning from a turning position P1 to a target stop position P2 on a reverse route R2 of a target travel route Rt under automatic parking control, it is considered that the driver's line of sight is directed to the side of the vehicle VH (the inside or outside of the turn) via the side mirrors, etc. While the vehicle VH travels while turning on the reverse route R2 under automatic parking control (reverse control), the peripheral image display control unit 120 displays on the display device 61 the right front upper diagonal virtual viewpoint image GFD and the third overhead-view image GB3 (see FIG. 6) or the left rear upper diagonal virtual viewpoint image GRD and the fourth overhead-view image GB4 (see FIG. 7, with the target travel route Rt in the reverse direction) generated by the peripheral image generation unit 110.

[0047] When the distance between the outside predicted passing line LO and obstacle OB5 is closer than the distance between the inside predicted passing line LI and obstacle OB6, the peripheral image display control unit 120 displays the right front upper diagonal virtual viewpoint image GFD and the third overhead image GB3 on the display device 61. On the other hand, when the distance between the inside predicted passing line LI and obstacle OB6 is closer than the distance between the outside predicted passing line LO and obstacle OB5, the peripheral image display control unit 120 displays the left rear upper diagonal virtual viewpoint image GRD and the fourth overhead image GB4 on the display device 61.

[0048] In this way, while the vehicle VH is traveling backward under automatic parking control, the display device 61 displays the right-front-upper virtual viewpoint image GFD or the rear-upper virtual viewpoint image GRD, thereby enabling the driver to appropriately determine from the right-front-upper virtual viewpoint image GFD or the rear-upper virtual viewpoint image GRD whether the host vehicle VH can pass through the obstacles OB5 and OB6 without coming into contact with them. In other words, the driver can easily determine whether or not to continue automatic parking while the host vehicle VH is traveling backward.

[0049] Next, a processing routine for controlling the display of peripheral images executed by the CPU 11 of the ECU 10 will be described with reference to the flowchart shown in Fig. 8. This routine is started after the driver turns on the automatic parking switch 70 and selects a target parking space and a parking mode. For convenience, the following describes an example in which the driver selects reverse parallel parking as the parking mode, in which the vehicle VH moves backward while turning left from a switching position P1 toward a target stopping position P2, and the automatic parking is not canceled until the vehicle VH reaches the target stopping position P2.

[0050] In step S100, the ECU 10 displays a confirmation screen G1 (see FIG. 3) including the target movement route Rt and the start button B1 on the display device 61. Next, in step S110, the ECU 10 determines whether or not the driver has touched the start button B1. If the driver has touched the start button B1 (Yes), the ECU 10 proceeds to the processing of step S120. On the other hand, if the driver has not touched the start button B1 (No), the ECU 10 returns from this routine.

[0051] In step S120, the ECU 10 determines whether the vehicle VH has started traveling along the forward route R1 due to automatic parking control. Whether the vehicle VH has started traveling along the forward route R1 may be determined based on the detection results of the vehicle speed sensor 31 and the shift sensor 35. If the vehicle VH has started traveling along the forward route R1 (Yes), the ECU 10 proceeds to the processing of step S125. On the other hand, if the vehicle VH has not started traveling along the forward route R1 (No), the ECU 10 returns to the processing of step S120.

[0052] In step S125, ECU 10 displays the front camera viewpoint image GF and the first overhead image GB1 (see FIG. 4) on display device 61. At this time, the display area of ​​the front camera viewpoint image GF may be expanded as the vehicle speed of vehicle VH increases, and may be reduced as the vehicle speed of vehicle VH decreases.

[0053] Next, in step S130, the ECU 10 determines whether the vehicle VH has reached the turning-back position P1 due to the automatic parking control. Whether the vehicle VH has reached the turning-back position P1 may be determined based on the detection results of the vehicle speed sensor 31 and the shift sensor 35. If the vehicle VH has reached the turning-back position P1 (Yes), the ECU 10 proceeds to the processing of step S135. On the other hand, if the vehicle VH has not reached the turning-back position P1 (No), the ECU 10 returns to the processing of step S125.

[0054] In step S135, the ECU 10 displays the rear camera viewpoint image GR and the second overhead image GB2 (see FIG. 5) on the display device 61.

[0055] Next, in step S140, the ECU 10 determines whether the vehicle VH has started traveling along the reverse route R2 due to automatic parking control. Whether the vehicle VH has started traveling along the reverse route R2 may be determined based on the detection results of the vehicle speed sensor 31 and the shift sensor 35. If the vehicle VH has started traveling along the reverse route R2 (Yes), the ECU 10 proceeds to the processing of step S145. On the other hand, if the vehicle VH has not started traveling along the reverse route R2 (No), the ECU 10 returns to the processing of step S135.

[0056] In step S145, the ECU 10 determines whether the distance between the outside predicted passing line LO and the obstacle OB5 is closer than the distance between the inside predicted passing line LI and the obstacle OB6. If the distance between the outside predicted passing line LO and the obstacle OB5 is closer than the distance between the inside predicted passing line LI and the obstacle OB6 (Yes), the ECU 10 proceeds to the processing of step S150 and displays the right front upper diagonal virtual viewpoint image GFD and the third overhead image GB3 (see FIG. 6 ) on the display device 61. On the other hand, if the distance between the outside predicted passing line LO and obstacle OB5 is not closer than the distance between the inside predicted passing line LI and obstacle OB6 (No), that is, if the distance between the inside predicted passing line LI and obstacle OB6 is closer than the distance between the outside predicted passing line LO and obstacle OB5, ECU10 proceeds to processing of step S160 and displays the left rear upper diagonal virtual viewpoint image GRD and the fourth overhead image GB4 (see Figure 7, but with the target movement path Rt in the backward direction) on the display device 61.

[0057] In step S170, the ECU 10 determines whether the vehicle VH has reached the target stop position P2 through the automatic parking control. If the vehicle VH has not reached the target stop position P2 (No), the ECU 10 returns to the processing of step S145. On the other hand, if the vehicle VH has reached the target stop position P2 (Yes), the ECU 10 returns (ends) this routine.

[0058] The present disclosure is not limited to the above-described embodiment, and various modifications are possible without departing from the object of the present invention.

[0059] For example, in the above embodiment, the automatic parking control has been described as an example of so-called route setting type automatic parking in which a target movement route Rt is set, but the technology of the present disclosure can also be applied to so-called route memory type automatic parking in which the movement route when the driver manually parks the vehicle VH is stored and the stored movement route is used the next time the vehicle is parked in the same parking space. The technology of the present disclosure can also be applied to an autonomous vehicle in which some or all of the driving operations are performed automatically.

Claims

1. A vehicle surroundings image display device that displays an image of the surroundings of a vehicle capable of executing automatic parking control to automatically enter a predetermined parking space or automatically exit a vehicle from the parking space, an image generating means capable of generating a plurality of surrounding images from different viewpoints based on images captured by a camera mounted on the vehicle; and an image display control means for selecting a surrounding image according to the progress of the automatic parking control from the plurality of surrounding images that can be generated by the image generation means during execution of the automatic parking control, and displaying the selected surrounding image on a display device of the vehicle. A vehicle peripheral image display device characterized by:

2. The vehicle surroundings image display device according to claim 1, an obstacle detection means for detecting an obstacle present around the vehicle; The image generating means When the obstacle is detected by the obstacle detection means, an image in which the obstacle is emphasized is generated as the surrounding image. A vehicle peripheral image display device characterized by:

3. 3. The vehicle surroundings image display device according to claim 1 or 2, The video display control means When the automatic parking control includes forward control for driving the vehicle forward to a predetermined turning position, stop control for stopping the vehicle at the turning position, and reverse control for driving the vehicle backward from the turning position to a predetermined position within the parking space, Different surrounding images are displayed on the display device during the execution of the forward control, the execution of the stop control, and the execution of the reverse control. A vehicle peripheral image display device characterized by:

4. 4. The vehicle surroundings image display device according to claim 3, The image generating means As the plurality of surrounding images, a front viewpoint image capturing an area in front of the vehicle, a rear viewpoint image capturing an area behind the vehicle, and an obliquely upward viewpoint image capturing the vehicle from an obliquely upward position can be generated, The video display control means The front viewpoint image is displayed on the display device while the forward control is being executed, the rear viewpoint image is displayed on the display device while the stop control is being executed, and the obliquely upper viewpoint image is displayed on the display device while the reverse control is being executed. A vehicle peripheral image display device characterized by:

5. 5. The vehicle surroundings image display device according to claim 4, The image generating means As the obliquely upper viewpoint image, an image is generated on which a predicted passing line indicating a boundary of an area through which the body of the vehicle is predicted to pass by due to the reverse control is superimposed. A vehicle peripheral image display device characterized by:

Images