Vehicle display control device and vehicle display control method

Abstract

Even when changing the display area of ​​the overhead view image, it makes it easier for the driver of the vehicle to check the area they want to see. [Solution] The system includes an obstacle detection unit 101 that detects obstacles around the vehicle based on sensing results from a surrounding monitoring sensor 20, a predicted trajectory identification unit 103 that identifies the predicted trajectory of the vehicle, a panoramic image generation unit 102 that generates a panoramic image of the vehicle and its surrounding area from above using images captured by a surrounding monitoring camera 201, and a display control unit 104 that displays the panoramic image generated by the panoramic image generation unit 102 on the display screen of a display device 40. The display control unit 104 changes the range of the surrounding area relative to the vehicle's position that is displayed as a panoramic image on the display screen, according to the predicted trajectory identified by the predicted trajectory identification unit 103 and the position of obstacles detected by the obstacle detection unit 101.

Description

Technical Field

[0001] The present disclosure relates to a vehicle display control device and a vehicle display control method.

Background Art

[0002] Techniques for displaying an overhead image that looks down on the surroundings of the vehicle from above are known. For example, Patent Document 1 discloses a technique for expanding the display area of an overhead image with respect to the area behind the vehicle image when the vehicle switches from forward movement to reverse movement. Patent Document 1 describes that the determination of the forward and reverse states is made from the shift position. Further, Patent Document 2 discloses a technique for trimming the range displayed on the monitor device in the overhead image so that many search areas are included. The search area is an area on the peripheral image set as an area that is likely to be close to a three-dimensional object.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is considered that there is a demand for vehicle drivers to particularly check the vicinity of the predicted travel route of their own vehicle. In contrast, neither of the techniques of Patent Documents 1 and 2 takes into account the predicted travel route of the vehicle. Therefore, when changing the display area of the overhead image, there is a risk that the range that the driver of the vehicle wants to check may become invisible or difficult to see.

[0005] One purpose of this disclosure is to provide a vehicle display control device and a vehicle display control method that make it easier for the driver of the vehicle to check the range they want to check, even when the display area of ​​the overhead image is changed. [Means for solving the problem]

[0006] The above objectives are achieved by a combination of features described in the independent claims, and the subordinate claims provide further advantageous specific examples of the disclosure. The reference numerals in parentheses in the claims indicate correspondences with specific means described in the embodiments described later as one aspect, and do not limit the technical scope of this disclosure.

[0007] To achieve the above objective, the vehicle display control device of this disclosure is a vehicle display control device that can be used in a vehicle and comprises: an obstacle detection unit (101) that detects obstacles around the vehicle from sensing results from a surrounding monitoring sensor (20) that monitors the area around the vehicle; a predicted trajectory identification unit (103) that identifies the predicted trajectory of the vehicle; a panoramic image generation unit (102) that generates a panoramic image of the vehicle and the area around the vehicle from above using captured images taken by a surrounding monitoring camera (201) that captures images of the area around the vehicle; and a display control unit (104) that displays the panoramic image generated by the panoramic image generation unit on the display screen of a display device (40). The display control unit changes the range of the area around the vehicle that is displayed as a panoramic image on the display screen according to the predicted trajectory identified by the predicted trajectory identification unit and the position of obstacles detected by the obstacle detection unit.

[0008] To achieve the above objective, the vehicle display control method of this disclosure is a vehicle display control device that can be used in a vehicle and is executed by at least one of a processor and a circuit, and includes an obstacle detection step for detecting obstacles around the vehicle from sensing results from a surrounding monitoring sensor (20) that monitors the area around the vehicle, a predicted trajectory identification step for identifying a predicted trajectory of the vehicle, a panoramic image generation step for generating a panoramic image of the vehicle and the area around the vehicle from above using captured images taken by a surrounding monitoring camera (201) that captures images of the area around the vehicle, and a display control step for displaying the panoramic image generated in the panoramic image generation step on the display screen of a display device (40), wherein the display control step changes the range of the surrounding area with respect to the position of the vehicle to be displayed as a panoramic image on the display screen according to the predicted trajectory identified in the predicted trajectory identification step and the position of obstacles detected in the obstacle detection step.

[0009] With the above configuration, the range of the surrounding area displayed as an overhead view on the display screen will change according to the predicted trajectory of the vehicle and the positions of obstacles detected around the vehicle. Therefore, it becomes possible to change the range of the surrounding area displayed as an overhead view on the display screen according to the predicted trajectory and the positions of obstacles, so that the area the driver of the vehicle wants to check can be easily confirmed. As a result, even when the display area of ​​the overhead view is changed, it becomes possible to easily confirm the area the driver of the vehicle wants to check. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example of a schematic configuration for a vehicle system. [Figure 2] This diagram illustrates an example where the configuration of this embodiment is not applied. [Figure 3] This diagram illustrates an example of applying the configuration of this embodiment. [Figure 4] This diagram illustrates an example of applying the configuration of this embodiment. [Figure 5]This flowchart shows an example of the processing flow related to the display of an overhead image in the image generation ECU. [Modes for carrying out the invention]

[0011] Multiple embodiments for disclosure will be described with reference to the drawings. For the sake of clarity, in some embodiments, parts having the same function as those shown in the drawings used in previous descriptions will be denoted by the same reference numerals, and their descriptions may be omitted. For parts denoted by the same reference numerals, refer to the descriptions in other embodiments.

[0012] (Embodiment 1) <Outline configuration of vehicle system 1> Hereinafter, Embodiment 1 of the present disclosure will be described with reference to the drawings. The vehicle system 1 shown in Figure 1 is usable in a vehicle. As shown in Figure 1, the vehicle system 1 includes an image generation ECU 10, a surrounding monitoring sensor 20, a vehicle status sensor 30, a display device 40, and a user input device 50.

[0013] The surrounding monitoring sensor 20 monitors the area around the vehicle. In other words, the surrounding monitoring sensor 20 monitors the environment around the vehicle. The surrounding monitoring sensor 20 includes a surrounding monitoring camera 201. The surrounding monitoring camera 201 captures images of the area around the vehicle. The surrounding monitoring camera 201 can be configured to capture images of the entire area around the vehicle by using multiple cameras. For example, a front camera, a rear camera, a left-side camera, and a right-side camera can be used. The front camera should capture images of a predetermined area in front of the vehicle. The rear camera should capture images of a predetermined area behind the vehicle. The left-side camera should capture images of a predetermined area to the left of the vehicle. The right-side camera should capture images of a predetermined area to the right of the vehicle. The surrounding monitoring camera 201 sequentially outputs the captured images of the entire area around the vehicle as sensing results to the image generation ECU 10.

[0014] The surrounding monitoring sensor 20 may include a probe sensor that transmits probe waves within a predetermined range around the vehicle. Examples of probe sensors include millimeter-wave radar, sonar, and LIDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging). The predetermined range may include at least partially the front, rear, left, and right sides of the vehicle. The probe sensor sequentially outputs the scanning result based on the received signal obtained when it receives reflected waves reflected by obstacles as a sensing result to the image generation ECU 10. The surrounding monitoring sensor 20 may also output the sensing result to the in-vehicle LAN and output the sensing result to the image generation ECU 10 via the in-vehicle LAN.

[0015] The vehicle status sensor 30 is a group of sensors for detecting various states of the vehicle. The vehicle status sensor 30 includes a vehicle speed sensor 301 and a steering angle sensor 302. The vehicle speed sensor is a sensor that detects the speed of the vehicle. The steering angle sensor 302 is a sensor that detects the steering angle of the vehicle. The vehicle status sensor 30 sequentially outputs the detected sensing results to the image generation ECU 10. The vehicle status sensor 30 may also output the sensing results to the in-vehicle LAN and output the sensing results to the vehicle status sensor 30 via the in-vehicle LAN.

[0016] The display device 40 displays an image according to the instructions of the image generation ECU 10. For example, the display device 40 can be a meter MID (Multi Information Display), a CID (Center Information Display), or a HUD (Head-Up Display). The meter MID is located in front of the driver's seat inside the vehicle. For example, the meter MID may be installed on the meter panel. The CID is located in the center of the vehicle's instrument panel. The HUD is installed, for example, on the instrument panel inside the vehicle. The HUD projects a display image formed by a projector onto a predetermined projection area on the front windshield, which acts as a projection member. The light from the image reflected into the vehicle interior by the front windshield is perceived by the driver seated in the driver's seat. This allows the driver to see the virtual image of the display formed in front of the front windshield superimposed on a part of the foreground. The HUD may also be configured to project the display image onto a combiner located in front of the driver's seat instead of the front windshield. For example, a CID can be used as the display device 40.

[0017] The user input device 50 receives input from the occupants of the vehicle. The user input device 50 can be an operating device that receives operation input from the occupants. The operating device may be a mechanical switch or a touch switch integrated with a display. Examples of mechanical switches include steering wheel switches provided on the steering wheel. However, the user input device 50 is not limited to an operating device that receives operation input, as long as it is a device that receives input from the occupants. For example, it may be a voice input device that receives voice commands from the occupants.

[0018] The image generation ECU 10 is mainly composed of, for example, a computer including a processor, a volatile memory, a non-volatile memory, I / O, and a bus connecting these components. The image generation ECU 10 executes various processes related to image display by executing a control program stored in the non-volatile memory. This image generation ECU 10 corresponds to a vehicle display control device. Note that at least a part of the functions performed by the processor in the image generation ECU 10 may be performed by a circuit. The circuit referred to here is a hardware circuit. The configuration of the image generation ECU 10 will be described in detail below.

[0019] <Schematic Configuration of Image Generation ECU 10> Subsequently, the schematic configuration of the image generation ECU 10 will be described using FIG. 1. As shown in FIG. 1, the image generation ECU 10 includes an obstacle detection unit 101, an overhead image generation unit 102, a predicted trajectory specifying unit 103, and a display control unit 104 as functional blocks. Also, the execution of the processing of each functional block of the image generation ECU 10 by a computer corresponds to the execution of a vehicle display control method. Note that part or all of the functions executed by the image generation ECU 10 may be configured hardware-wise by one or more circuits or the like. Also, part or all of the functional blocks included in the image generation ECU 10 may be realized by a combination of software execution by a processor and a hardware circuit.

[0020] The obstacle detection unit 101 detects obstacles around the host vehicle from the sensing results of the surrounding monitoring sensor 20. The processing in this obstacle detection unit 101 corresponds to the obstacle detection step. Examples of obstacles include moving objects such as pedestrians and other vehicles. Examples of obstacles also include stationary objects such as fallen objects on the road and roadside installations. The obstacle detection unit 101 only needs to detect obstacles, including the position of the obstacles relative to the host vehicle. When the obstacle detection unit 101 detects the position of an obstacle using the sensing results of the probing wave sensor, it may proceed as follows. The obstacle detection unit 101 only needs to detect the orientation of the obstacle relative to the host vehicle from the direction in which the probing wave that has received the reflected wave from the obstacle was transmitted. The obstacle detection unit 101 only needs to detect the distance from the host vehicle to the obstacle based on the time until the probing wave is reflected by the obstacle and returns. The obstacle detection unit 101 only needs to detect the position of the obstacle relative to the host vehicle from the detected orientation and distance.

[0021] The obstacle detection unit 101 may also detect the presence and position of an obstacle using the captured image captured by the surrounding monitoring camera 201. In this case, it may proceed as follows. The obstacle detection unit 101 only needs to detect the obstacle from the captured image using image recognition technology. The obstacle detection unit 101 only needs to detect the position of the obstacle in the captured image and the position of the obstacle relative to the host vehicle from the camera parameters of the surrounding monitoring camera 201 from which the captured image was obtained. The position of the obstacle may be detected, for example, as a position in a coordinate system based on the position of the host vehicle. Regarding the position of the host vehicle, it may be sequentially specified from the positioning results of the vehicle locator. The camera parameters may be the installation position and the direction of the optical axis of the surrounding monitoring camera 201 on the host vehicle.

[0022] The locator may, for example, be equipped with a GNSS (Global Navigation Satellite System) receiver and an inertial sensor. The GNSS receiver receives positioning signals from multiple positioning satellites. The inertial sensor may include, for example, a gyroscope and an accelerometer. The locator sequentially determines the vehicle's position by combining the positioning signals received by the GNSS receiver with the measurement results from the inertial sensor. The vehicle's position may be expressed as latitude and longitude coordinates. Alternatively, the vehicle's position may be expressed as coordinates in a vehicle coordinate system with a predetermined reference point of the vehicle at a given time as the origin. In this case, the locator can determine the coordinates in the vehicle coordinate system from the measurement results from the inertial sensor.

[0023] The overhead image generation unit 102 generates an overhead image of the vehicle and its surrounding area from above, using the images captured by the surrounding surveillance camera 201. The processing in this overhead image generation unit 102 corresponds to the overhead image generation process. The overhead image generation unit 102 should generate the overhead image as follows. Here, we will explain using the case where the images captured by the surrounding surveillance camera 201 are images captured by the front camera, rear camera, left side camera, and right side camera as an example. The overhead image generation unit 102 converts the images captured by the front camera, rear camera, left side camera, and right side camera into image data in the ground surface coordinate system, which is the coordinate system on the road surface, using a coordinate transformation formula, with a virtual viewpoint above the vehicle as the viewpoint position. For example, it converts it into an overhead image that looks down vertically on the ground surface. Next, the overhead image generation unit 102 rotates and translates the overhead images from the front camera, rear camera, left-side camera, and right-side camera using a transformation formula, and places them on a single coordinate plane. In other words, it arranges the overhead images so that they can be considered as a single image. Then, the overhead image generation unit 102 combines these overhead images to generate a composite image, which is the overhead image of the area surrounding the vehicle.

[0024] The overhead image generation unit 102 may also be configured to perform necessary image processing, such as lens distortion correction, on the captured image before converting it into an overhead image. Furthermore, when generating this composite image, the overhead image generation unit 102 places an image of the vehicle at the location corresponding to the vehicle's position and composites it into the overhead image. The image of the vehicle can be read from a memory of the image generation ECU 10 that has been stored in advance.

[0025] The position of the vehicle in the composite image can be determined by identifying the vehicle's position from the portion of the vehicle included in the captured image, if that portion is included in the captured image. Even if the vehicle's position is not included in the captured image, it is still possible to determine the vehicle's position in the composite image. For details, see below. Once the installation position and optical axis orientation of the surrounding surveillance camera 201 relative to the vehicle are determined, the correspondence between the position in the captured image and the installation position of the surrounding surveillance camera 201 can be determined. Therefore, the position of the surrounding surveillance camera 201 relative to the vehicle can be determined from the installation position and optical axis orientation of the surrounding surveillance camera 201 relative to the vehicle. Furthermore, the position of the vehicle relative to the installation position of the surrounding surveillance camera 201 can be determined from the installation position of the surrounding surveillance camera 201 relative to the vehicle. Therefore, the position of the vehicle in the composite image can be determined from the position of the surrounding surveillance camera 201 relative to the position in the composite image.

[0026] The predicted trajectory identification unit 103 identifies the predicted trajectory of the vehicle. The processing in this predicted trajectory identification unit 103 corresponds to the predicted trajectory identification process. The predicted trajectory identification unit 103 should identify the predicted trajectory of the vehicle from the vehicle speed, the vehicle steering angle, and vehicle information of the vehicle. The vehicle speed can be the one detected by the vehicle speed sensor 301. The vehicle steering angle can be the one detected by the steering angle sensor 302. Vehicle information can include the steering gear ratio and vehicle width. The steering gear ratio is the gear ratio of the tire steering angle to the steering angle. Vehicle information can be pre-stored in the memory of the image generation ECU 10. The predicted trajectory identification unit 103 should determine the steering angle from the steering angle and the steering gear ratio. Then, from this steering angle and vehicle speed, it should calculate the predicted position of the vehicle at regular intervals and identify the predicted trajectory from the calculated predicted position group. The predicted trajectory may be a line extending from the center of the vehicle's width, or it may be the vehicle's width extended in the direction of the vehicle's path. The predicted trajectory identification unit 103 only needs to identify the predicted trajectory up to the predicted position of the vehicle after a predetermined time. The predetermined time is an arbitrarily set time, such as 3 seconds.

[0027] The predicted trajectory identification unit 103 does not need to identify a predicted trajectory if the vehicle is stationary and the shift position has not been switched to the forward position "D" or the reverse position "R". On the other hand, even if the vehicle is stationary, the predicted trajectory identification unit 103 should identify a predicted trajectory if the shift position has been switched to the forward position "D" or the reverse position "R". In this case, the predicted trajectory should be identified assuming that the vehicle speed is a predetermined speed. The predetermined speed is an arbitrarily set speed, such as a slow speed of 10 km / h. Therefore, it becomes possible to identify a predicted trajectory even when the vehicle is stationary if there is a high probability that the vehicle will start moving. According to this, if there is a high probability that the vehicle will start moving, the offset processing described later becomes possible even when the vehicle is stationary.

[0028] The display control unit 104 displays the overhead image generated by the overhead image generation unit 102 on the display screen of the display device 40. The size of the overhead image generated by the overhead image generation unit 102 should be larger than the size of the image to be displayed on the display screen of the display device 40. The display control unit 104 then performs trimming to cut out a portion of the overhead image generated by the overhead image generation unit 102 and displays the trimmed overhead image on the display screen. In trimming, the overhead image generation unit 102 cuts out an image of the size to be displayed on the display screen. By default, the display control unit 104 should perform trimming so that the position of the vehicle included in the overhead image is centered on the display screen.

[0029] The display control unit 104 can initiate the display of the overhead image on the display screen of the display device 40 using the following triggers. For example, the trigger can be the reception of an instruction input from the user input device 50 to turn on the overhead image display function (hereinafter referred to as the overhead function on input). Alternatively, the trigger can be the start of a vehicle function that involves the display of an overhead image on the display screen of the display device 40. An example of such a vehicle function is the automatic parking function that automatically parks the vehicle. In the following explanation, we will use the case where the overhead image display function is also turned on when the automatic parking function is started as an example.

[0030] The display control unit 104 changes the range of the surrounding area relative to the vehicle's position that is displayed as an overhead view image on the display screen, according to the predicted trajectory identified by the predicted trajectory identification unit 103 and the position of obstacles detected by the obstacle detection unit 101. The range of the surrounding area relative to the vehicle's position that is displayed as an overhead view image on the display screen will be referred to as the display range below. The display control unit 104 changes the display range by offsetting the position that serves as the center of the display screen in the aforementioned trimming from the position of the vehicle included in the overhead view image. A change in the display range can also be described as a change in the positional relationship between the vehicle's position in the overhead view image and the center position of that overhead view image. Below, the process of changing the display range according to the predicted trajectory identified by the predicted trajectory identification unit 103 and the position of obstacles detected by the obstacle detection unit 101 will be referred to as the offset process. The processing in the display control unit 104 corresponds to the display control process.

[0031] With the above configuration, the display range will change according to the predicted trajectory of the vehicle and the positions of obstacles detected around the vehicle. Therefore, it becomes possible to change the display range so that the area the driver of the vehicle wants to check can be easily viewed, according to the predicted trajectory and the positions of obstacles. As a result, even when changing the display area of ​​the overhead view image, it becomes possible to easily view the area the driver of the vehicle wants to check.

[0032] The display control unit 104 preferably changes the display range in the direction of the predicted trajectory identified by the predicted trajectory identification unit 103. In other words, it is preferable to change the range of the surrounding area displayed as an overhead image on the display screen so that the surrounding area in the direction of the predicted trajectory relative to the position of the vehicle is widely displayed on the display screen. In addition, when an obstacle detected by the obstacle detection unit 101 is located within a set distance from the predicted trajectory, it is preferable for the display control unit 104 to suppress the amount of change in the display range in the direction of the predicted trajectory compared to when the obstacle is not located within a set distance from the predicted trajectory. The set distance can be any distance that can be set arbitrarily. The predicted trajectory can be, for example, the width of the vehicle extended in the direction of the vehicle's path.

[0033] It is assumed that the driver of the vehicle has a particular need to confirm the direction in which their vehicle's predicted trajectory is heading. With the above configuration, the area surrounding the vehicle in the direction of the predicted trajectory can be displayed more broadly on the display screen. Therefore, even when changing the display area of ​​the overhead image, it becomes easier for the driver of the vehicle to confirm the range they want to check. Furthermore, if an obstacle is detected near the predicted trajectory, it is assumed that the driver of the vehicle has a stronger need to check the obstacle than the area ahead in the predicted trajectory. With the above configuration, when an obstacle is detected near the predicted trajectory, the amount of change in the display range in the direction of the predicted trajectory is kept to a minimum. Therefore, it becomes possible to suppress changes in the displayed range that extend beyond the driver's desired range.

[0034] Here, using Figures 2 to 4, an example of how the display range changes according to the predicted trajectory of the vehicle and the position of obstacles detected around the vehicle will be explained. Figure 2 is a diagram illustrating an example where the configuration of this embodiment is not applied. Figures 3 and 4 are diagrams illustrating an example where the configuration of this embodiment is applied. Figure 3 is a diagram illustrating an example where no obstacles within a set distance from the predicted trajectory of the vehicle are detected. Figure 4 is a diagram illustrating an example where obstacles are detected within a set distance from the predicted trajectory of the vehicle. In Figures 2 to 4, BVI shows the overhead image generated by the overhead image generation unit 102. In Figures 2 to 4, DI shows the image to be displayed on the display screen (hereinafter, display image) which is cropped from this overhead image BVI. In Figures 2 to 4, TR shows the area to be cropped. In Figures 2 to 4, HVI shows the image of the vehicle, and OI shows the image of the obstacle. In Figures 2 to 4, FT shows the predicted trajectory of the vehicle. The predicted trajectory FT does not necessarily have to be displayed on the display screen. The display of the predicted trajectory FT may be linear or strip-shaped with a width corresponding to the vehicle width.

[0035] If the configuration of this embodiment is not applied, the display range will be as follows. The display image DI in Figures 2 to 4 corresponds to the display range. In the example shown in Figure 2, the image is cropped so that the position of the vehicle image HVI included in the overhead image BVI is at the center of the display image DI. The cropped display image DI is then displayed on the display screen. In the display image DI shown in Figure 2, the area beyond the predicted trajectory of the vehicle is not included, making it difficult to grasp the situation beyond the predicted trajectory.

[0036] When applying the configuration of this embodiment, the display range is as follows. In the examples shown in Figures 3 and 4, the position of the vehicle image HVI included in the overhead image BVI is trimmed so that it is offset from the center of the display image DI in the opposite direction to the direction of the vehicle's predicted trajectory FT. In other words, the trimming is performed so that a larger area of ​​the surrounding region in the direction of the predicted trajectory FT is included from the perspective of the vehicle image HVI. The display image DI extracted by the trimming is then displayed on the display screen. In the display image DI shown in Figures 3 and 4, a larger area beyond the vehicle's predicted trajectory is included compared to the case in Figure 2, making it easier to grasp the situation beyond the predicted trajectory.

[0037] Furthermore, as can be seen by comparing Figure 3 and Figure 4, when an obstacle is detected within a set distance from the vehicle's predicted trajectory, the offset amount described above is kept smaller than when no obstacle is detected within that distance. In other words, in Figure 4, the trimming is performed in a way that reduces the amount of the surrounding area in the direction of the predicted trajectory FT from the vehicle's HVI image compared to Figure 3. As a result, when an obstacle is detected near the predicted trajectory FT, the display range does not change to an area beyond the predicted trajectory FT that the driver does not want.

[0038] It is preferable that the display control unit 104 switches whether or not to perform offset processing according to a selection input received by the user input device 50 for selecting whether or not to perform offset processing. This makes it possible for the user to choose whether or not to change the display range by performing offset processing according to their preference. The selection input for selecting whether or not to perform offset processing will be referred to as the offset selection input below.

[0039] It is preferable that the display control unit 104 overlays an image showing the predicted trajectory identified by the predicted trajectory identification unit 103 onto the overhead view image. This makes it easier for the driver of their vehicle to understand the range they need to check.

[0040] The display control unit 104 may switch whether or not to display the image showing the predicted trajectory according to a selection input received by the user input device 50 for selecting whether or not to display the image showing the predicted trajectory. This makes it possible for the user to choose whether or not to display the image showing the predicted trajectory according to their preference. The selection input for selecting whether or not to display the image showing the predicted trajectory will be referred to as the trajectory display selection input below.

[0041] When the display range is changed, it is preferable for the display control unit 104 to change the display range so that at least a part of the vehicle is included in the overhead view image. In other words, it is preferable to change the display range so that at least a part of the image of the vehicle is included on the display screen of the display device 40. This makes it possible for the occupants of the vehicle to always judge distance based on the image of their own vehicle, even when the display range is changed. Therefore, even when the display range is changed, it becomes less likely for the occupants of the vehicle to lose their sense of distance in the overhead view image.

[0042] Furthermore, the display control unit 104 may be configured to change the display range according to the predicted trajectory of the vehicle and the positions of obstacles detected around the vehicle, as follows: The display control unit 104 may change the display range to include the area corresponding to the predicted trajectory identified by the predicted trajectory identification unit 103 and obstacles detected by the obstacle detection unit 101 within a set distance from that area. This makes it possible to change the display range to include the predicted trajectory and obstacles close to that trajectory in accordance with the changes in the predicted trajectory and obstacles close to that trajectory. The area corresponding to the predicted trajectory and obstacles close to that area are considered to be areas that the driver of the vehicle would most want to check. Therefore, even with the above configuration, it is possible to make it easier for the driver of the vehicle to check the area they want to check, even when changing the display range of the overhead image.

[0043] <Processing related to displaying overhead images in the image generation ECU10> Here, using the flowchart in Figure 5, we will explain an example of the flow of processing related to the display of the overhead image in the image generation ECU 10 (hereinafter referred to as overhead image display-related processing). The flowchart in Figure 5 can be started, for example, when the overhead function ON input is received by the user input device 50. Alternatively, it may be configured to start when the automatic parking function is activated.

[0044] First, in step S1, the predicted trajectory identification unit 103 identifies the predicted trajectory. The predicted trajectory identification unit 103 does not identify a trajectory if the vehicle is stationary and there is no change in the shift position to the forward position "D" or the reverse position "R".

[0045] In step S2, the overhead image generation unit 102 acquires the images captured by the surrounding surveillance camera 201 and generates an overhead image. Specifically, it acquires the images from the front camera, rear camera, left side camera, and right side camera and generates an overhead image that can be considered as a single image. The overhead image also incorporates an image of the vehicle at its own position. In step S3, the obstacle detection unit 101 detects obstacles around the vehicle based on the sensing results from the surrounding surveillance sensor 20.

[0046] In step S4, the display control unit 104 determines whether or not offset processing is necessary. The display control unit 104 should determine whether or not offset processing is necessary according to the offset selection input received by the user input device 50. In addition, the display control unit 104 may determine that offset processing is unnecessary if the vehicle is stationary and the shift position has not been switched to the forward position "D" or the reverse position "R". If it is determined in S4 that offset processing is necessary (YES in S4), the process moves to step S5. On the other hand, if it is determined that offset processing is unnecessary (NO in S4), the process moves to step S6.

[0047] In step S5, the display control unit 104 performs the aforementioned offset processing according to the predicted trajectory identified in S1 and the position of the obstacle detected in S3. In step S6, the display control unit 104 causes the display screen of the display device 40 to display the overhead view image. In S6, if the offset processing was performed in S5, the display control unit 104 causes the display of the overhead view image with the offset processing applied. In S6, if the offset processing was not performed in S5, the display control unit 104 causes the display of the overhead view image without the offset processing applied. In S6, the display control unit 104 may also superimpose an image showing the predicted trajectory onto the overhead view image. In S6, the display control unit 104 can switch whether or not to superimpose an image showing the predicted trajectory onto the overhead view image according to the trajectory display selection input received by the user input device 50.

[0048] In step S7, the image generation ECU 10 determines the vehicle's position based on the positioning results from the vehicle's locator. The vehicle's position can be used, for example, in the automatic parking function to determine whether the vehicle has reached the target parking position. In step S8, if it is the end of the overhead image display related processing (YES in S8), the overhead image display related processing is terminated. On the other hand, if it is not the end of the overhead image display related processing (NO in S8), the process returns to S1 and is repeated. The end of the overhead image display related processing can be attributed to receiving an instruction input from the user input device 50 to turn off the overhead image display function. Alternatively, if the start of the automatic parking function was used as the start trigger in the flowchart of Figure 5, the completion of automatic parking can also be attributed. Other examples include turning off the power switch. The power switch is a switch used to start the vehicle's internal combustion engine or motor generator.

[0049] (Embodiment 2) In Embodiment 1, the image generation ECU 10 is shown as corresponding to a vehicle display control device, but the embodiment is not necessarily limited to this. For example, an ECU other than the image generation ECU 10 may be configured to correspond to a vehicle display control device.

[0050] (Disclosed technical ideas) This specification discloses several technical concepts, as set forth in the following paragraphs. Some paragraphs may be written in a multiple dependent form, where subsequent paragraphs alternately refer to preceding paragraphs. Furthermore, some paragraphs may be written in a multiple dependent form, referring to other multiple dependent forms. These paragraphs written in multiple dependent forms define several technical concepts.

[0051] (Technical thought 1) A vehicle display control device that can be used in a vehicle, An obstacle detection unit (101) detects obstacles around the vehicle based on the sensing results from the surrounding monitoring sensor (20) that monitors the area around the vehicle, A predicted trajectory identification unit (103) that identifies the predicted trajectory of the vehicle, An overhead image generation unit (102) generates an overhead image of the vehicle and the area surrounding the vehicle, using images captured by an overhead surveillance camera (201) that captures images of the area around the vehicle, The system includes a display control unit (104) that displays the overhead image generated by the overhead image generation unit on the display screen of the display device (40), The display control unit is a vehicle display control device that changes the range of the surrounding area with respect to the position of the vehicle, which is displayed on the display screen as an overhead view image, according to the predicted trajectory identified by the predicted trajectory identification unit and the position of the obstacle detected by the obstacle detection unit.

[0052] (Technical thought 2) A vehicle display control device as described in Technical Concept 1, The display control unit changes the range of the surrounding area with respect to the position of the vehicle displayed in the overhead image in the direction of the predicted trajectory identified by the predicted trajectory identification unit, and when an obstacle detected by the obstacle detection unit is located within a set distance from the predicted trajectory, the amount of change in the direction of the predicted trajectory is reduced compared to when the obstacle is not located within a set distance from the predicted trajectory.

[0053] (Technical Thought 3) A vehicle display control device described in Technical Concept 1 or 2, The display control unit is a vehicle display control device that receives input from the occupants of the vehicle and switches according to a selection input to determine whether or not there is a change in the range of the surrounding area with respect to the position of the vehicle, which is displayed on the display screen as an overhead image, according to the predicted trajectory identified by the predicted trajectory identification unit and the position of the obstacle detected by the obstacle detection unit.

[0054] (Technical Thought 4) A vehicle display control device described in any one of the technical concepts 1 to 3, The display control unit is a vehicle display control device that superimposes an image showing the predicted trajectory identified by the predicted trajectory identification unit onto the overhead view image.

[0055] (Technical Thought 5) A vehicle display control device described in any one of the technical concepts 1 to 4, The display control unit is a vehicle display control device that changes the range of the surrounding area with respect to the position of the vehicle to be displayed in the overhead view image, so that at least a part of the vehicle is included in the overhead view image.

[0056] In this disclosure or claims, the term "processor" refers to one or more hardware processors configured to execute processing defined by computer program code (i.e., one or more instructions of a computer program) contained in a computer program by reading the code each time. In other words, a "processor" is a hardware device that executes one or more programmed processes. Therefore, computer program code can also be considered software that can define the processing of the processor according to its content. For example, a "processor" may be a general-purpose or specific-purpose processor, and may be, but is not limited to, a CPU, microprocessor, GPU, and DFP (Data Flow Processor).

[0057] In this disclosure or claims, the term “memory” means one or more hardware memories that are non-transitional tangible recording media configured to record computer program code and / or data in a manner accessible from a processor. “Memory” can be implemented by memory technologies such as SRAM, SDRAM, non-volatile / flash type memory, or other types of memory. The computer program code that constitutes the program is recorded in memory and executed by a processor, thereby enabling the processor to perform the various functions described above.

[0058] In this disclosure or claims, the term “circuit” refers to one or more logic circuits as hardware, configured to perform specific processing defined by a pre-designed circuit configuration. In other words (and, in contrast to “processor”), “circuit” in this disclosure or claims refers to a hardware device that performs specific processing based on a circuit configuration, rather than processing defined by software such as the computer program code described above. For example, “circuit” may include custom ICs such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays) designed with Hardware Description Language (HDL). That is, “circuit” in this disclosure or claims includes all hardware circuits except for the processor described above, which performs processing by reading computer program code.

[0059] In this disclosure or claims, the expression "at least one of a processor and a circuit" should be interpreted as a disjunctive (logical OR) and not as at least one processor and at least one circuit. Therefore, in this disclosure or claims, "at least one of a processor and a circuit" includes cases where the circuit alone performs all functions. Also, in this disclosure or claims, "at least one of a processor and a circuit" includes cases where the processor alone performs all functions. In this disclosure or claims, "at least one of a processor and a circuit" includes cases where the circuit performs some functions and the processor performs the remaining functions. [Explanation of Symbols]

[0060] 1 Vehicle system, 10 Image generation ECU (vehicle display control device), 20 Surroundings monitoring sensors, 40 Display device, 101 Obstacle detection unit, 102 Overhead image generation unit, 103 Predicted trajectory identification unit, 104 Display control unit, 201 Surroundings monitoring camera

Claims

1. A vehicle display control device that can be used in a vehicle, An obstacle detection unit (101) detects obstacles around the vehicle based on the sensing results from the surrounding monitoring sensor (20) that monitors the area around the vehicle, A predicted trajectory identification unit (103) that identifies the predicted trajectory of the vehicle, An overhead image generation unit (102) generates an overhead image of the vehicle and the area surrounding the vehicle, using images captured by an overhead surveillance camera (201) that captures images of the area around the vehicle, The system includes a display control unit (104) that displays the overhead image generated by the overhead image generation unit on the display screen of the display device (40), The display control unit is a vehicle display control device that changes the range of the surrounding area with respect to the position of the vehicle, which is displayed on the display screen as an overhead view image, according to the predicted trajectory identified by the predicted trajectory identification unit and the position of the obstacle detected by the obstacle detection unit.

2. A vehicle display control device according to claim 1, The display control unit changes the range of the surrounding area with respect to the position of the vehicle displayed in the overhead image in the direction of the predicted trajectory identified by the predicted trajectory identification unit, and when an obstacle detected by the obstacle detection unit is located within a set distance from the predicted trajectory, the amount of change in the direction of the predicted trajectory is reduced compared to when the obstacle is not located within a set distance from the predicted trajectory.

3. A vehicle display control device according to claim 1, The display control unit is a vehicle display control device that receives input from the occupants of the vehicle and switches according to a selection input to determine whether or not there is a change in the range of the surrounding area with respect to the position of the vehicle, which is displayed on the display screen as an overhead image, according to the predicted trajectory identified by the predicted trajectory identification unit and the position of the obstacle detected by the obstacle detection unit.

4. A vehicle display control device according to claim 1, The display control unit is a vehicle display control device that superimposes an image showing the predicted trajectory identified by the predicted trajectory identification unit onto the overhead view image.

5. A vehicle display control device according to claim 1, The display control unit is a vehicle display control device that changes the range of the surrounding area with respect to the position of the vehicle to be displayed in the overhead view image, so that at least a part of the vehicle is included in the overhead view image.

6. Executed by at least one of a processor and a circuit, A vehicle display control device that can be used in a vehicle, An obstacle detection step is performed to detect obstacles around the vehicle based on the sensing results from a surrounding monitoring sensor (20) that monitors the area around the vehicle, A predicted trajectory identification step for identifying the predicted trajectory of the vehicle, An overhead image generation step is performed to generate an overhead image of the vehicle and the area surrounding the vehicle, using images captured by a surrounding surveillance camera (201) that captures images of the area around the vehicle, and to generate an overhead image of the vehicle and the area surrounding the vehicle, viewed from above. The process includes a display control step that causes the overhead image generated in the overhead image generation step to be displayed on the display screen of a display device (40), The display control step involves a vehicle display control method that changes the range of the surrounding area with respect to the vehicle's position, which is displayed on the display screen as an overhead view image, according to the predicted trajectory identified in the predicted trajectory identification step and the position of the obstacle detected in the obstacle detection step.

Images