Image display method and image display device

Abstract

An image display device (100) uses a processor to display, on a monitor (210), a bird's-eye view image including a host vehicle. An image acquisition unit (101) of the image display device (100) acquires imaging data obtained by imaging the surrounding region of the host vehicle with a camera (201) provided in the host vehicle, and a bird's-eye view image generation unit (102) generates a first image, which is a bird's-eye view image of the surrounding region, on the basis of the imaging data. When the area of a parking candidate space displayed in the first image is less than a predetermined first area, a display control unit (105) displays an icon indicating the presence of the parking candidate space. When it is determined that the icon has been selected, the bird's-eye view image generation unit (102) generates, on the basis of imaging data acquired in the past, a second image, which is a bird's-eye view image of a region different from the first image and in which the parking candidate space is displayed in an area equal to or greater than a predetermined second area, and the display control unit (105) causes the second image to be displayed on the monitor (210).

Description

Image display method and image display device

[0001] The present invention relates to an image display method and an image display device.

[0002] In a driving support technology for automatically controlling vehicle parking, there is a technology for identifying a parking space based on the result of detecting the surroundings of the vehicle by a sensor and displaying an image of the parking space (for example, Patent Document 1). The parking support device described in Patent Document 1 extracts a predetermined number or less of candidate parking positions from the detected parking positions based on the distances between the current position of the host vehicle and each detected parking position and displays them on a display device when a plurality of parking positions are detected. It is explained that this can convey the parking positions to the user in an easy-to-understand manner.

[0003] Japanese Patent Application Laid-Open No. 2021-94927

[0004] The parking support device of Patent Document 1 acquires the imaging data of a camera group that captures the surroundings of the host vehicle and the detection data of a sonar group, and displays an image including the parking space specified based on the imaging data and the detection data. As a parking space display method, there is a method of generating and displaying an overhead image viewed from a virtual viewpoint above the host vehicle based on the imaging data. When the target parking space is far from the position of the host vehicle in such an overhead image, there is a problem that most of the parking space may not be displayed, or the parking space may be distorted and displayed at the edge of the display image, and the image of the peripheral area including the parking space cannot be accurately displayed.

[0005] The present invention has been made in view of the above circumstances, and an object thereof is to provide an image display method and an image display device capable of suitably displaying an image of a peripheral area including a parking space.

[0006] To achieve the above objective, the image display method according to the present invention is an image display method that uses a processor to display an overhead view image including the vehicle on a monitor. The processor acquires imaging data of the area surrounding the vehicle using a camera installed on the vehicle, generates a first image which is an overhead view image of the surrounding area generated based on the imaging data, and displays it on the monitor. If the area of ​​the parking candidate space displayed in the first image is less than a predetermined first area, the processor displays an icon indicating that a parking candidate space exists, and when it is determined that the icon has been selected, it generates a second image which is an overhead view image of a different area from the first image, in which the parking candidate space is displayed with an area of ​​a predetermined second area or more, based on imaging data acquired in the past, and displays it on the monitor.

[0007] According to the present invention, a second image is generated and displayed in which the candidate parking space is shown with an area of ​​two or more areas based on previously acquired imaging data, making it possible to suitably display an image of the surrounding area including the parking space.

[0008] This is a block diagram showing an example of the functional configuration of a display system equipped with an image display device according to an embodiment of the present invention. This is a diagram showing an example of the hardware configuration of the image display device according to the embodiment. This is a top view showing the state of the vehicle before it is parked. This is a diagram showing an example of an automatic parking screen including a first image. This is a top view showing the state of the vehicle before it is parked. This is a diagram showing an example of an automatic parking screen including a first image. This is a diagram showing an example of an automatic parking screen including a second image. This is a diagram showing an example of an automatic parking screen including a parking position adjustment screen. This is a diagram showing an example of an automatic parking screen after parking has started. This is a flowchart of the image display process. This is a diagram showing another example of an automatic parking screen including a first image. This is a diagram showing another example of an automatic parking screen including a second image. This is a top view showing the state of the vehicle before it is parked. This is a diagram showing another example of an automatic parking screen including a second image.

[0009] An image display method and an image display device according to embodiments of the present invention will be described with reference to the drawings. In each drawing, the same or equivalent parts are denoted by the same reference numerals.

[0010] (Embodiment) The image display device 100 according to an embodiment of the present invention is a device that displays an overhead image representing a parking candidate space on a monitor when a vehicle with a parking control function performs parking. Figure 1 is a block diagram showing an example of the functional configuration of a display system 10 equipped with the image display device 100 according to this embodiment. The display system 10 is mounted on the vehicle and realizes the function of presenting a parking candidate space to the occupants.

[0011] As shown in Figure 1, the display system 10 includes a plurality of cameras 201 that capture images of the area around the vehicle, a short-range sensor 202 that detects the surrounding environment of the vehicle, and a vehicle sensor 203 that detects the operating state, operational state, etc. of the vehicle. The display system 10 further includes a human-machine interface (HMI) 204 that exchanges information between the vehicle and the occupant, a monitor 210 that displays images, text information, etc., and an image display device 100 that controls the generation of an overhead view image including the vehicle based on images captured by the cameras 201 and displays it on the monitor 210.

[0012] Camera 201 is any camera installed on the outer perimeter of the vehicle and capturing images of the area surrounding the vehicle. For example, it may be an around-view camera that outputs captured images that can be converted into an overhead view image (a so-called around-view monitor image) including the vehicle. Camera 201 includes, for example, a front camera that captures an area including the front of the vehicle, a right side camera and a left side camera that capture an area including the right and left sides of the vehicle, respectively, and a rear camera that captures an area including the rear of the vehicle.

[0013] Multiple short-range sensors 202 are installed on the vehicle to detect the position, shape, and size of any object in the vicinity of the vehicle, such as buildings, plants, parked vehicles, and other objects. Examples include SONAR (Sound Navigation and Ranging), LiDAR (Light Detection and Ranging), RADAR (Radio Detection and Ranging), LRF (Laser Range-Finder), and cameras. The output of the short-range sensors 202 can be used to determine the presence or absence of parking spaces, parked vehicles, and objects such as pylons and barricades.

[0014] The vehicle sensor 203 is an arbitrary sensor that detects various vehicle information such as the operating state and operational state of the vehicle. The vehicle sensor 203 may include, for example, a vehicle speed sensor that detects the vehicle's driving speed, a wheel speed sensor that detects the rotational speed of each tire on the vehicle, a three-axis acceleration sensor that detects the vehicle's acceleration (including deceleration) in the three axes, a steering angle sensor that detects the steering angle of the steering wheel, a steering angle sensor that detects the steering angle of the steering wheels, a gyro sensor that detects the angular velocity occurring in the vehicle, and a yaw rate sensor that detects the yaw rate. The vehicle sensor 203 may further include a driver camera that photographs the occupants.

[0015] The HMI 204 includes controls that receive input from the occupant of the vehicle. These controls include, for example, buttons, switches, levers, dials, and keyboards. The HMI 204 may include, for example, a deadman's switch, steering wheel switches, turn signal levers, shift levers, or shift switches. The monitor 210 is a display device visible to the occupant of the vehicle and displays images generated by the image display device 100. The monitor 210 may include, for example, a display screen that displays information from the navigation system, or a display screen located near the meter in front of the driver's seat. The HMI 204 may also include a touch panel integrated with the monitor 210. The HMI 204 may also include a voice recognition (VR) system for voice-based operation.

[0016] Figure 2 shows an example of the hardware configuration of the image display device 100. In the example in Figure 2, the image display device 100 comprises a processor 1011, a storage unit 1012, and a communication interface 1013, all connected to each other via a bus 1010.

[0017] The processor 1011 comprises, for example, one or more CPUs (Central Processing Units) and their peripheral circuits, and performs various arithmetic operations. The processor 1011 executes control programs stored in the memory unit 1012. The processor 1011 may also include volatile semiconductor memory such as RAM (Random Access Memory) that functions as the CPU's working memory. Furthermore, the processor 1011 may also include arithmetic circuits such as a logical operation unit and a numerical operation unit.

[0018] The storage unit 1012 includes, for example, a non-volatile semiconductor memory such as EEPROM (Electrically Erasable and Programmable Read Only Memory) or flash memory. The storage unit 1012 stores control programs executed by the processor 1011 and various data used in the arithmetic processing of the processor 1011.

[0019] The communication interface 1013 includes an interface circuit for connecting the processor 1011 to an in-vehicle network compliant with standards such as CAN (Controller Area Network). The communication interface 1013 receives signals from in-vehicle components such as the camera 201, near-field sensor 202, vehicle sensor 203, and HMI 204, and passes them to the processor 1011. The communication interface 1013 also outputs images generated by the processor 1011 to the monitor 210.

[0020] For example, the image display device 100 implements the functions shown in Figure 1 by having the processor 1011 of the image display device 100 execute an image display processing program. Specifically, the image display device 100 includes an image acquisition unit 101 that acquires imaging data captured by the camera 201, an overhead image generation unit 102 that generates an overhead image based on the imaging data, and an operation detection unit 103 that detects occupant operations based on the output of the vehicle sensor 203 or HMI 204. The image display device 100 further includes a parking candidate space selection unit 104 that selects a parking candidate space based on the results of analyzing the imaging data and the operations detected by the operation detection unit 103, and a display control unit 105 that displays the overhead image generated by the overhead image generation unit 102 and icons indicating parking candidate spaces attached to the overhead image on the monitor 210.

[0021] The image acquisition unit 101 acquires imaging data from each of the multiple cameras 201 installed around the vehicle's perimeter and stores it in the storage unit 1012. The imaging data is retained in the storage unit 1012 for a predetermined period of time or longer, but may be deleted thereafter. The imaging data stored in the storage unit 1012 is read out when needed by the overhead image generation unit 102 and the parking candidate space selection unit 104.

[0022] The overhead image generation unit 102 generates an overhead image including the vehicle 1 based on the imaging data stored in the storage unit 1012. The overhead image is an image created by combining imaging data of the area surrounding the vehicle 1 and superimposing an image of the vehicle 1 onto that image. The timing of generating the overhead image is arbitrary. The overhead image generation unit 102 may generate the overhead image at predetermined time intervals and store it in the storage unit 1012, or it may generate it when the operation detection unit 103 detects an operation to display the automatic parking screen.

[0023] The overhead image generation unit 102 is capable of generating overhead images of two or more different regions. For example, the overhead image generation unit 102 can generate a first image, which is the initial overhead image when an operation to instruct automatic parking is detected, and a second image, which is an overhead image of a region different from the region of the first image.

[0024] The operation detection unit 103 detects occupant operations based on the output of the vehicle sensor 203 or HMI 204. For example, the operation detection unit 103 detects the switching of the shift switch, the pressing of the deadman switch or steering wheel switch, the touch of the touch panel, voice input to the VR system, etc., and generates operation signals for executing automatic parking, selecting a parking candidate space, correcting the parking position, switching screens, etc., and outputs them to the overhead image generation unit 102, the parking candidate space selection unit 104, or the display control unit 105. For the sake of simplicity, the following explanation will mainly focus on cases where operations are performed by touching the touch panel, but it is possible to replace touching the touch panel with pressing other switches or voice input.

[0025] The display control unit 105 controls the display on the monitor 210 based on the operation signal output from the operation detection unit 103, displaying the overhead image generated by the overhead image generation unit 102, an icon indicating the parking candidate space selected by the parking candidate space selection unit 104, as well as image data, text information, operation buttons, etc.

[0026] The screen that the display control unit 105 displays on the monitor 210 will be explained in detail. Figures 3A and 4 are top views illustrating the state of the vehicle 1 before parking, and Figures 3B and 5A are diagrams illustrating the automatic parking screen 211 on the monitor 210 when the automatic parking operation is performed. As shown in Figure 3A, the vehicle 1 is driven by the occupant, moves forward past an empty parking candidate space 31 and comes to a stop, at which point the occupant may instruct the display of the automatic parking screen 211 by pressing the automatic parking operation button or the like. In this case, the overhead image generation unit 102 generates a first image 225, which is an overhead image showing the surrounding area centered on the vehicle 1, based on the image data at that time, as shown in the left screen 221 of Figure 3B. The display control unit 105 displays the first image 225 as the initial image on the left screen 221 of the monitor 210.

[0027] The area displayed in the first image 225 is an area in which parking candidate spaces 31 appearing in the first image 225 can be displayed when their suitability as a parking space, calculated based on the output of the camera 201 or near-field sensor 202 installed in the vehicle 1, is equal to or greater than a predetermined value. The suitability is a value calculated based on, for example, the thickness or length of the white line indicating that it is a parking space, the width or shape of the parking space, and the presence or absence of objects in the parking space. For example, the suitability will be low if the parking space is smaller than a predetermined standard value corresponding to the size of the vehicle 1. Also, the suitability will be low if objects such as other vehicles, pylons, or barricades are detected in the parking space, or if there are restriction markings that exclude the vehicle 1, such as a light vehicle mark or a priority mark. In other words, in the first image 225, parking spaces with a suitability of a predetermined value or higher can be presented to the occupants as parking candidate spaces.

[0028] The parking space selection unit 104 determines, based on the output of the proximity sensor 202, that there is an available parking space 31 behind the vehicle 1. Here, the parking space 31 is a space that can be parked in by reversing without having to maneuver back and forth. For example, the parking space selection unit 104 can detect the parking space 31 based on the output of the proximity sensor 202, based on the thickness or length of the white line indicating that it is a parking space, the width or shape of the parking space, the presence or absence of objects in the parking space, etc. The parking space selection unit 104 calculates the area of ​​the parking space 31 displayed in the first image 225, and selects the parking space 31 if that area is equal to or greater than a predetermined first area. The display control unit 105 displays an icon 301 with a circled "P" at the location of the parking space 31 on the left screen 221. The display control unit 105 also displays the image captured by the rear camera on the right screen 222 as is, and displays an icon 302 with a circled "P" at the location of the parking space 31. Note that the shapes of icons 301 and 302 are just examples, and any shape is acceptable.

[0029] On the other hand, as shown in Figure 4, the occupant may, while driving the vehicle 1, pass a position slightly away from an available parking candidate space 32 and come to a stop, press the automatic parking operation button or perform other actions to instruct the display of the automatic parking screen 211. In this case, the overhead image generation unit 102 generates a first image 225, which is an overhead image showing the surrounding area centered on the vehicle 1, based on the captured data at that time, as shown in the left screen 221 of Figure 5A. The display control unit 105 displays the first image 225 as the initial image on the left screen 221 of the monitor 210. The image captured by the rear camera is displayed as is on the right screen 222.

[0030] The parking space selection unit 104 determines, based on the output of the proximity sensor 202, that there is an available parking space 32 behind the vehicle 1, and selects the parking space 32. The parking space 32 is a space that can be parked in reverse without having to maneuver back. The parking space selection unit 104 calculates the area of ​​the parking space 32 displayed in the first image 225. If the area of ​​the parking space 32 in the first image 225 is less than the first area, and most of the parking space 32 is not within the first image 225, the display control unit 105 displays an indication that the parking space 32 is outside the frame of the first image 225. Here, the first area is a portion of the total area of ​​the first image 225, and is a predetermined area that is sufficiently smaller than, for example, the area of ​​the region in which the vehicle 1 is visible.

[0031] For example, the display control unit 105 displays an icon 311, which is a square-enclosed "P" pointing downwards, near the location of the parking candidate space 32 on the left screen 221. In the left screen 221 of Figure 5A, the icon 311 is located in the lower right, indicating that the parking candidate space 32 is to the right rear. If there are multiple parking candidate spaces 32, multiple icons 311 may be displayed according to the location of the parking candidate spaces 32. The shape of the icon 311 is just an example, and any shape is acceptable as long as it can be distinguished from the icon 301 shown in Figure 3B.

[0032] When it is determined that icon 311 has been selected, the overhead image generation unit 102 generates a second image 226, which is an overhead image representing a different area from the first image 225, based on imaging data previously acquired and stored in the storage unit 1012. In the second image 226, the parking candidate space 32 is displayed with an area equal to or greater than a predetermined second area. The second image 226 is an overhead image in which an image of the vehicle 1 is superimposed on an image generated by the image acquisition unit 101 based on imaging data previously acquired and stored in the storage unit 1012.

[0033] More specifically, the second image 226 is an overhead view image created by superimposing an image representing the vehicle 1 onto an image generated based on imaging data captured when passing the parking candidate space 32. Normally, reverse parking is performed after passing the parking candidate space 32, so the imaging data of the parking candidate space 32 acquired immediately beforehand can be used to generate the second image 226. In the second image 226, the area in which the parking candidate space 32 appears is a part of the second image 226, and the area of ​​the area in which the parking candidate space 32 appears is greater than or equal to the second area. The second area is a predetermined area that is a part of the total area of ​​the second image 226, for example, an area within a predetermined range that is different from the area in which the vehicle 1 appears.

[0034] Here, the operation detection unit 103 may determine that icon 311 has been selected when it detects an operation such as touching icon 311 on the touch panel, pressing a preset deadman switch, making a voice input to the VR system, or a predetermined time elapsed without any operation. Alternatively, it may determine that icon 311 has been selected by detecting that the occupant's gaze is directed toward the automatic parking screen 211 based on the image captured by the driver camera.

[0035] Figure 5B is an example showing the state in which the second image 226 is displayed on the left screen 221. In Figure 5B, the candidate parking spaces 32 and 33 are both parking spaces that can be reversed into without reversing, and are displayed with an area of ​​2 or more. The display format of the second image 226 is arbitrary, but for example, as shown in Figure 5B, the display direction of the vehicle 1 may be rotated relative to the first image 225, and the candidate parking spaces 32 and 33 may be displayed closer to the center of the left screen 221. Such a display can clearly show the positional relationship between the vehicle 1 and the candidate parking spaces 32 and 33. The number of candidate parking spaces 32 and 33 is one or more, any number.

[0036] The display control unit 105 may display an icon 312 with a "P" enclosed in a double circle for parking candidate spaces 32 that are determined to be closer to the vehicle 1, and an icon 313 with a "P" enclosed in a single circle for other parking candidate spaces 33. Icons 312 and 313 may be changeable from one another based on user operation. The display control unit 105 may also display the image captured by the rear camera on the right screen 222 as is, and display an icon 314 with a "P" enclosed in a circle at the location of parking candidate space 32. The shapes of icons 312, 313, and 314 are examples, and any shape is acceptable as long as icons 312 and 313 can be distinguished from each other, and icons 312 and 313 may be given different colors from each other.

[0037] As shown in Figures 3B, 5A, and 5B, the automatic parking screen 211 displayed on the monitor 210 also displays a button 350 for adjusting the parking position. When the occupant selects the button 350, the display control unit 105 displays the parking position adjustment screen 223 on the right screen 222, as shown in Figure 6A. On the parking position adjustment screen 223, the occupant can adjust the parking position 321 in the parking candidate space 32 by selecting the left / right / up / down or rotate buttons. In the parking candidate space 32, a frame indicating the parking position 321 is automatically displayed based on white lines, etc., but if, for example, the occupant wants to park further forward, the frame for the parking position 321 can be moved forward, as shown in Figure 6A.

[0038] In this way, by switching from the first image 225, in which most of the parking candidate space 32 is not visible, to the second image 226, the parking position 321 can be adjusted while the parking candidate space 32 is visible. The parking position adjustment screen 223 can be closed by selecting the "Back" button.

[0039] As shown in Figures 3B, 5A, 5B, and 6A, the automatic parking screen 211 displayed on the monitor 210 shows a parking start button 250. When the parking start button 250 is selected by the occupant, automatic parking control is initiated. Automatic parking control is performed by any conventional method, by controlling various actuators of the vehicle 1. When automatic parking control is being performed, the left screen 221 may be returned to the first image 225, as shown in Figure 6B. By returning to the first image 225, the area surrounding the vehicle 1 can be enlarged and viewed.

[0040] The operation of the image display device 100 configured as described above will be explained in detail using the flowchart in Figure 7. Figure 7 is a flowchart of the image display process performed by the image display device 100 of the vehicle 1.

[0041] The image acquisition unit 101 uses a camera 201 installed on the vehicle 1 to capture images of the area surrounding the vehicle 1 and acquires the image data (step S101). If the operation detection unit 103 does not detect an operation to execute automatic parking (step S102: No), the process returns to step S101, and the image data of the area surrounding the vehicle 1 is repeatedly acquired and stored in the storage unit 1012. The image data stored in the storage unit 1012 may be updated to store image data captured in the past within a predetermined time from the present.

[0042] As shown in FIGS. 3A and 4, when the driver moves the host vehicle 1 forward and then stops the host vehicle 1 with the parking candidate spaces 31 and 32 located behind the host vehicle 1, the driver performs an operation to execute automatic parking, such as pressing an automatic parking operation button. When the operation detection unit 103 detects an operation to execute automatic parking (step S102: Yes), the display control unit 105 displays an automatic parking screen 211 (step S103). A first image 225, which is an initial image, is displayed on the left screen 221 of the automatic parking screen 211. The first image 225 is an overhead image representing a peripheral area within a predetermined range centered on the host vehicle 1, and is generated based on the imaging data acquired in step S101.

[0043] The parking candidate space selection unit 104 determines whether there is a parking candidate space where reverse parking is possible without switching (step S104). If it is determined that there is no parking candidate space where reverse parking is possible without switching (step S104: No), the image display device 100 displays this fact on the monitor 210 or notifies the driver by voice, and returns to step S101. The driver moves the host vehicle 1 forward and moves the host vehicle 1 to a position where reverse parking is possible, and performs an operation to execute automatic parking again.

[0044] If it is determined that there is a parking candidate space 31 or a parking candidate space 32 where reverse parking is possible without switching (step S104: Yes), the display control unit 105 displays an icon on at least one of the parking candidate space 31 and the parking candidate space 32 (step S105). Here, different icons are displayed for the parking candidate space 31 in which the area of the portion displayed in the first image 225 is greater than or equal to the first area, and the parking candidate space 32 in which the area of the portion displayed in the first image 225 is less than the first area.

[0045] In the example of FIG. 3B, since the area of the portion displayed in the first image 225 of the parking candidate space 31 is greater than or equal to the first area, an icon 301 with "P" surrounded by a circle is displayed. In the example of FIG. 5A, since the area of the portion displayed in the first image 225 of the parking candidate space 32 is less than the first area, an icon 311 with "P" surrounded by a square and pointing downward is displayed.

[0046] As shown in FIG. 5A, when the area of the portion displayed in the first image 225 of the parking candidate space 32 is less than the first area and an icon 311 indicating that the parking candidate space 32 exists outside the first image 225 is displayed, and when that icon 311 is selected (step S106: Yes), the display control unit 105 switches the image on the left screen 221 to the second image 226 (step S107). The switching from the first image 225 to the second image 226 may be performed at once, or an animation that gradually or smoothly transitions from the first image 225 to the second image 226 may be displayed.

[0047] The second image 226 is an overhead image of a region different from the first image 225, in which the parking candidate space 32 is displayed with an area equal to or greater than a predetermined second area. The second image 226 is an overhead image generated based on the imaging data acquired when passing by the parking candidate space 32 in step S101.

[0048] As shown in FIG. 3B, when the icon of the parking candidate space 31 in which the area of the portion displayed in the first image 225 is equal to or greater than the first area is selected, or when no selection operation is performed while the icons of the parking candidate spaces 31 and 32 are displayed (step S106: No), the switching to the second image 226 is not performed, and the process proceeds to step S108.

[0049] When the parking position adjustment button 350 is selected to correct the target parking position while the first image 225 or the second image 226 is displayed on the left screen 221 (step S108: Yes), the display control unit 105 displays the parking position adjustment screen 223 shown in FIG. 6A (step S109), and the operation detection unit 103 receives an operation for adjusting the parking position.

[0050] If the button 350 for adjusting the parking position is not selected (step S108: No), the process proceeds to step S110. When the first image 225 or the second image 226 is displayed on the left screen 221 and the image captured by the camera 201 or the parking position adjustment screen 223 is displayed on the right screen 222, if the parking start button 250 is selected (step S110: Yes), the image display device 100 notifies the automatic driving system to start automatic parking and terminates the post-parking completion process. At this time, it is preferable to display the first image 225 on the image of the left screen 221 (step S111) to display an image with a high degree of appropriateness during actual parking.

[0051] If the parking start button is not selected (step S110: No), the process returns to step S105, and the image display device 100 waits for the occupant to select an icon, adjust the parking position, or start parking. If the exit button 251 is selected during the screen display process shown in Figure 7, the display of the automatic parking screen 211 on the monitor 210 is terminated, and the process ends.

[0052] As described above, in the image display device 100 according to this embodiment, the image acquisition unit 101 acquires imaging data of the area surrounding the vehicle 1 using a camera 201 provided on the vehicle 1, the overhead image generation unit 102 generates a first image 225 which is an overhead image of the surrounding area generated based on the imaging data, and the display control unit 105 displays the first image 225 on the monitor 210. If the area of ​​the parking candidate space displayed in the first image 225 is less than a predetermined first area, the display control unit 105 displays an icon 311 indicating that a parking candidate space 32 exists, and when the operation detection unit 103 determines that the icon 311 has been selected, the overhead image generation unit 102 generates a second image 226 which is an overhead image of a different area from the first image 225, in which the parking candidate space 32 is displayed with an area of ​​a predetermined second area or more, based on imaging data acquired in the past, and displays it on the monitor 210. This makes it possible to suitably display the surrounding image including the parking candidate space 32.

[0053] The hardware configuration and flowchart shown in the above embodiment are examples and can be arbitrarily changed or adapted. For example, in the above embodiment, the second image 226 is an overhead view image in which the display direction of the vehicle 1 is rotated relative to the first image 225 and the parking candidate space 32 is displayed centered on the left screen 221. However, other image configurations are acceptable as long as the positional relationship between the vehicle 1 and the parking candidate space 32 can be clearly represented. For example, as shown in Figure 8A, an icon 311 is displayed indicating that the parking candidate space 32 is outside the frame of the first image 225. When it is determined that the icon 311 has been selected, the overhead view image generation unit 102 may generate a second image 226, as shown in Figure 8B, in which the display direction of the vehicle 1 is the same as in the first image 225, and a wider area is displayed than in the first image 225. Such image configurations may be made selectable by the occupant.

[0054] Furthermore, although the above embodiment describes the case of reversing and parking in the reverse direction, it is also applicable to parallel parking when reversing. As shown in Figure 9A, a second image 226, which is an overhead view, is generated based on imaging data taken while passing by a candidate parking space 34 for parallel parking. If an icon indicating that the candidate parking space 34 is outside the frame of the first image 225 is selected, the automatic parking screen 211 including the second image 226 illustrated in Figure 9B may be displayed on the monitor 210.

[0055] Furthermore, in the above embodiment, the automatic parking screen 211 is displayed on a monitor 210 installed in the vehicle 1, but the automatic parking screen 211 may be displayed on another display device. For example, it may be displayed on an external display device such as a smartphone, tablet terminal, or personal computer. This allows the user to select icons 301, 311-313 or start parking operations from outside the vehicle 1 using the automatic parking screen 211 on any display device.

[0056] Furthermore, although the above embodiment describes an example in which each function is realized by the processor 1011 executing a control program, the image display device 100 may also be configured with dedicated hardware to realize each function.

[0057] Furthermore, an image display device 100 that can realize each function may be configured by distributing a control program for executing the operations of the above embodiment on a recording medium such as a computer-readable CD-ROM (Compact Disc Read-Only Memory), DVD (Digital Versatile Disc), MO (Magneto Optical Disc), or memory card, and installing the program on a computer. In cases where each function is realized through a division of labor between the OS (Operating System) and the application, or through cooperation between the OS and the application, only the parts other than the OS may be stored on the recording medium.

[0058] The present invention allows for various embodiments and modifications without departing from the broad spirit and scope of the invention. Furthermore, the embodiments described above are for illustrative purposes only and do not limit the scope of the invention. In other words, the scope of the invention is indicated by the claims, not by the embodiments. Various modifications made within the scope of the claims and the equivalent significance of disclosure are considered to be within the scope of the invention.

[0059] 1. Own vehicle, 10. Display system, 31, 32, 33, 34. Parking candidate space, 100. Image display device, 101. Image acquisition unit, 102. Overhead image generation unit, 103. Operation detection unit, 104. Parking candidate space selection unit, 105. Display control unit, 201. Camera, 202. Near-range sensor, 203. Vehicle sensor, 204. Human-machine interface (HMI), 210. Monitor, 211. Automatic parking screen, 221. Left screen, 222. Right screen, 223. Parking position adjustment screen, 225. First image, 226. Second image, 301, 302, 311, 312, 313, 314. Icons, 250. Start parking button, 251. End button, 350. Button, 321. Parking position, 1010. Bus, 1011. Processor, 1012. Memory unit, 1013. Communication interface.

Claims

1. An image display method for displaying an overhead image including the vehicle on a monitor using a processor, wherein the processor acquires imaging data of the area surrounding the vehicle using a camera installed on the vehicle, generates a first image which is an overhead image of the surrounding area generated based on the imaging data and displays it on the monitor, displays an icon indicating the existence of the parking candidate space if the area of ​​the parking candidate space displayed in the first image is less than a predetermined first area, and when it is determined that the icon has been selected, generates a second image which is an overhead image of a different area from the first image in which the parking candidate space is displayed with an area of ​​a predetermined second area or more, based on the imaging data acquired in the past, and displays it on the monitor.

2. The image display method according to claim 1, wherein the second image is an overhead view image obtained by rotating the display direction of the vehicle relative to the first image and centering the parking candidate space in the image.

3. The image display method according to claim 1, wherein the second image is an overhead view image in which the display direction of the vehicle is the same as that of the first image, and a wider area is displayed than that of the first image.

4. The image display method according to any one of claims 1 to 3, wherein the area displayed in the first image is an area in which the candidate parking space appearing in the first image can be displayed in a state in which the suitability as a parking space, calculated based on the output of the camera or sensor provided in the vehicle, is equal to or greater than a predetermined value.

5. The image display method according to any one of claims 1 to 4, wherein the parking candidate space is a space in which the vehicle can park by reversing without making any adjustments, and the processor generates the second image based on the image data captured when the vehicle passes beside the parking candidate space.

6. The image display method according to claim 5, wherein the second image is an overhead view image obtained by superimposing an image of the vehicle itself onto an image generated based on the image data captured when passing beside the candidate parking space.

7. The image display method according to any one of claims 1 to 6, wherein the processor, when displaying the second image, receives an operation from an occupant to adjust the parking position in the candidate parking space, and sets the adjusted parking position as the target parking position.

8. An image display device for displaying an overhead image including the vehicle on a monitor, comprising: a storage unit for storing imaging data obtained by a camera installed on the vehicle from the area surrounding the vehicle; and a processor for generating a first image, which is an overhead image of the surrounding area generated based on the imaging data, and displaying it on the monitor; displaying an icon indicating the existence of a parking candidate space if the area of ​​the parking candidate space displayed in the first image is less than a predetermined first area; and, when it is determined that the icon has been selected, generating a second image, which is an overhead image of a different area from the first image, based on previously acquired imaging data, in which the parking candidate space is displayed with an area of ​​a predetermined second area or more, and displaying it on the monitor.

Images