Driving assistance systems

Abstract

The present invention provides a driver assistance device that, when displaying wheel images superimposed on captured images, achieves both visibility of the wheel images and visibility of blind spots hidden by the wheel images. [Solution] A support image is generated by superimposing a wheel image 55, which represents the wheels of the vehicle, onto an image captured of the area around the vehicle's wheels. The support image is displayed on the liquid crystal display 4, and the wheel image 55 in the support image is rotated around the wheel axle in accordance with the vehicle's movement. The wheel image 55 is set to have a higher transmittance in the upper part, which is further from the road surface, than in the lower part, which is closer to the road surface. Furthermore, when the wheel image 55 is rotated, the transmittance of the wheel image 55 is updated in accordance with the rotation of the wheel image 55 according to the transmittance setting of the wheel image 55.

Description

Technical Field

[0001] The present invention relates to a driving support device that performs driving support for a vehicle.

Background Art

[0002] Conventionally, various means have been used as information providing means for providing various kinds of information for performing driving support of a vehicle, such as route guidance and warning of obstacles, to a passenger of the vehicle. For example, display on a liquid crystal display installed in the vehicle, voice output from a speaker, and the like. Here, there is an area that is a blind spot difficult to visually recognize from the position of the driver around the vehicle. In particular, when a special operation such as a parking operation is performed, in order to make the driver grasp the situation of such a blind spot, it has been conventionally performed to display a peripheral image captured by a camera installed in the vehicle on a liquid crystal display.

[0003] As an example, Japanese Patent No. 5134504 discloses a technique of displaying an image of the periphery of a vehicle captured by a camera on a liquid crystal display installed in the vehicle, and in order to clarify in which direction the image of the periphery of the vehicle displayed on the liquid crystal display is an image captured with respect to the vehicle, displaying an image of a wheel at the position of the wheel of the host vehicle in the image. On the other hand, when an image of a wheel is displayed, a blind spot will occur in the image of the periphery of the vehicle displayed on the liquid crystal display due to the image of the wheel, but a technique of making a user visually recognize a blind spot portion hidden by the image of the wheel by making the image of the wheel semi-transparent is also disclosed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] ​In the above-mentioned Patent Document 1, visibility of blind spots is ensured by making the wheel image semi-transparent. However, if the transparency of the vehicle image is made too high, visibility of blind spots improves, but the visibility of the wheels themselves decreases, and the original purpose of displaying the wheel image is not achieved. On the other hand, if the transparency is kept low so as not to become too high in order to make the wheel image visible, there is a problem in that the visibility of blind spots decreases.

[0006] The present invention was made to solve the aforementioned problems of the conventional invention, and aims to provide a driving assistance device that achieves both visibility of the wheel image and visibility of blind spots hidden by the wheel image when the wheel image is superimposed on the captured image. [Means for solving the problem]

[0007] To achieve the above objective, the driving assistance device according to the present invention includes: an image acquisition means for acquiring an image of the area surrounding the vehicle; an assistance image generation means for generating an assistance image by superimposing a wheel image showing the wheels onto the image of the area where the wheels of the vehicle are located; and an image display means for displaying the assistance image on a display device, wherein the wheel image is set to have a higher transmittance in the upper part, which is further from the road surface, than in the lower part, which is closer to the road surface. Furthermore, the "images taken of the area around the vehicle" may be the images themselves taken by an imaging device such as a camera, or they may be images that have been processed from captured images. For example, they may be images that have been combined from images taken by multiple cameras, or images that have had their viewpoint transformed from captured images. [Effects of the Invention]

[0008] According to the driver assistance device of the present invention having the above configuration, when displaying a wheel image superimposed on an image captured of the area around the vehicle, visibility of the blind spots can be ensured by setting a high transmittance above the wheel image, where there is a large overlap with the captured image and many blind spots, while visibility of the wheel image can be ensured by setting a low transmittance below the wheel image. Therefore, it is possible to achieve both visibility of the wheel image and visibility of the blind spots hidden by the wheel image. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram of the vehicle according to this embodiment. [Figure 2] This is a block diagram showing the configuration of the driver assistance system according to this embodiment. [Figure 3] This is a flowchart of the driver assistance processing program according to this embodiment. [Figure 4] This diagram illustrates the method for converting captured images into overhead and bird's-eye view images. [Figure 5] This diagram illustrates the method for generating overhead and bird's-eye view images. [Figure 6] This is a diagram showing an image of a wheel. [Figure 7] This figure shows an image created by combining a bird's-eye view image with an image of wheels. [Figure 8] This figure shows an example of a support image displayed on a liquid crystal display. [Figure 9] This diagram illustrates an example of rotating the wheel image in accordance with the vehicle's movement. [Figure 10] This is a flowchart of the sub-processing programs for the transmittance setting process and the transmittance update process. [Figure 11] This figure shows the polygons that form the surface of the wheel image. [Figure 12] This diagram illustrates the method for calculating vertex normal vectors. [Figure 13] This figure shows an example of a reference vector. [Figure 14] This diagram illustrates how to calculate the transparency of a polygon's vertices. [Figure 15] This diagram illustrates how to set the transparency of the faces between polygon vertices. [Figure 16] This figure shows the transparency settings applied to the wheel image. [Modes for carrying out the invention]

[0010] Hereinafter, an embodiment of the driving support device according to the present invention will be described in detail with reference to the drawings. First, the vehicle 2 equipped with the driving support device 1 according to this embodiment will be described below. FIG. 1 is a schematic configuration diagram of the vehicle 2 according to this embodiment.

[0011] Here, the vehicle 2 may be, for example, an automobile (internal combustion engine vehicle) having an internal combustion engine (engine, etc.) as a drive source, or an automobile (electric vehicle, fuel cell vehicle, etc.) having an electric motor (motor, etc.) as a drive source, or an automobile (hybrid vehicle) having both of them as drive sources. Also, regardless of the vehicle type, it may be an ordinary vehicle, or a commercial large truck, bus, construction machinery, etc. Further, in the following description, it is assumed to be a four-wheel vehicle, but it may also be a two-wheel or three-wheel vehicle.

[0012] However, in addition to the manual driving in which the vehicle 2 travels based on the user's driving operation, the vehicle is a vehicle capable of support driving by automatic driving support in which the vehicle automatically travels without depending on the user's driving operation.

[0013] Also, the automatic driving support may be performed only in specific situations such as when parking or leaving the warehouse, or may be performed for all road sections, or may be configured to be performed only while the vehicle travels on a specific road section (for example, a highway provided with a gate (regardless of manned or unmanned, toll or free) at the boundary). In the following description, the automatic driving section where the automatic driving support of the vehicle is performed includes all road sections including general roads and highways as well as parking lots. Further, it is selected by the user to perform automatic driving support (for example, turning on the automatic driving start button), and it is determined that driving by automatic driving support is possible. It will be performed only in the situation. On the other hand, the vehicle 2 may be a vehicle capable of only support driving by automatic driving support. Alternatively, the support driving by automatic driving support may be performed only for the driving to the parking space when the vehicle parks (that is, parking support).

[0014] In the vehicle control for automatic driving support in this embodiment, for example, the current position of the vehicle, the lane in which the vehicle is traveling, and the positions of surrounding obstacles are detected at any time, and along the generated driving trajectory, the vehicle control such as steering, drive source, and brake is automatically performed at a speed according to the also generated speed plan. Particularly when performing parking support, the detection results of sensors and cameras are used to check the parking space that the vehicle is to park in and the surrounding situation, calculate the parking trajectory to the parking space, and automatically perform vehicle control to enter the vehicle into the parking space along the calculated parking trajectory and complete the parking. However, in the above parking support, only the steering operation may be automatically performed, and the control of the drive source and brake may be performed based on manual operation. Alternatively, only the guidance of the parking trajectory to the parking space or the guidance of vehicle operation may be performed, and the parking operation to the parking space may be left to the user manually.

[0015] Also, during the automatic vehicle control such as steering, drive source, and brake, the vehicle occupants can cancel the automatic driving support at any timing according to their own will, and it is also possible to execute a brake operation to stop the vehicle. When performing the above automatic driving support, as will be described later, guidance is provided to display an imaging image of the vehicle surroundings captured by a camera on an in-vehicle display, and the vehicle occupants can also interrupt the automatic driving support and stop the vehicle by performing a brake operation if necessary while viewing the display.

[0016] As shown in FIG. 1, the vehicle 2 includes an operation unit 3 that receives operations from the occupants, a liquid crystal display 4 that displays an aerial image, an overhead image, or other information related to driving support around the vehicle to the occupants, a speaker 5 that outputs voice guidance related to driving support, front cameras 6, rear cameras 7, side cameras 8A and 8B for imaging the vehicle surroundings, ultrasonic sensors 9A to 9F that detect obstacles around the vehicle, and a driving support ECU (Electronic Control Unit) 10 that performs various arithmetic processes based on the input information. Note that the driving support device 1 includes each component including the above driving support ECU 10.

[0017] The following describes the various components of vehicle 2. First, the control unit 3 is located, for example, in front of the steering wheel and includes control buttons that are operated when starting the automated driving assistance system. By operating the control unit 3, the user can switch between manual driving, where the vehicle moves based on the user's driving input, and automated driving assistance, where the vehicle moves automatically without user input. The control unit 3 may also have a touch panel located in front of the liquid crystal display 4. It may also have a microphone and a voice recognition device.

[0018] The liquid crystal display 4 is a type of display device mounted on the instrument panel of the vehicle 2. During the execution of automated driving assistance, it displays bird's-eye and overhead views of the vehicle's surroundings, generated by performing viewpoint transformation and synthesis processing on images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B. In particular, for the bird's-eye view, wheel images (tire images) indicating the location of the vehicle's wheels are also added and displayed. Details of the wheel images will be described later. Note that the liquid crystal display 4 may also be used for the navigation system.

[0019] Furthermore, speaker 5 is mounted on the instrument panel of vehicle 2 and outputs voice guidance and warning sounds related to driver assistance. Speaker 5 may also be used for the navigation system.

[0020] Furthermore, the front camera 6 is an imaging device that has a camera using a solid-state image sensor such as a CCD, and is installed, for example, above the front bumper of the vehicle 2 or behind the rearview mirror, with the optical axis facing forward in the direction of travel of the vehicle.

[0021] The rear camera 7 is an imaging device that also has a camera using a solid-state image sensor such as a CCD, and is mounted, for example, near the center above the license plate attached to the rear of the vehicle 2, with the optical axis facing the rear of the vehicle.

[0022] Furthermore, the side cameras 8A and 8B are imaging devices that also have cameras using solid-state image sensors such as CCDs, and are mounted, for example, on the left and right side mirrors of vehicle 2, with the optical axis facing the side of the vehicle.

[0023] The driver assistance ECU 10 generates bird's-eye and overhead views of the vehicle's surroundings by performing viewpoint transformation and synthesis processing on the images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B. Furthermore, during automated driving assistance, it performs image recognition processing on the captured images to detect lane markings and obstacles (other vehicles, pedestrians, bicycles, walls, guardrails, and other structures) around the vehicle, and performs automated driving assistance based on the detection results. In particular, when performing parking assistance, it also uses the obstacle detection results from the cameras to check the parking space and its surroundings.

[0024] On the other hand, ultrasonic sensors 9A to 9L are arranged at predetermined intervals on the front, rear, and sides of the vehicle, respectively. They transmit ultrasonic waves as probe waves around the vehicle 2 and detect objects that reflected the probe waves by receiving reflected waves from objects around the vehicle. Specifically, they are a type of distance measuring sensor capable of detecting the distance (measured distance value) to the object that reflected the probe waves by measuring the time from transmission to reception. Furthermore, ultrasonic sensors 9A to 9L are configured to generate an output signal (including the distance to the detected object) corresponding to the reception result of the received wave and output it to the control unit. The objects to be detected by ultrasonic sensors 9A to 9L include, for example, people, bicycles, other vehicles, walls, and other obstacles that the vehicle 2 needs to avoid when driving, or obstacles that form a parking space. In addition, millimeter-wave sensors or laser sensors may be used as distance measuring sensors instead of ultrasonic sensors.

[0025] Furthermore, while the installation position and direction of each ultrasonic sensor 9A to 9L can be set as appropriate, in this embodiment, in order to make the detection range of the target object encompass all directions in front of, behind, and to the left and right of the vehicle's direction of travel, for example, ultrasonic sensors 9A to 9D are installed on the front of the vehicle 2 facing the direction of travel so that the direction of transmission of the probe wave is in front of the vehicle's direction of travel. Ultrasonic sensors 9E and 9F are installed on the left side of the vehicle 2 facing left so that the direction of transmission of the probe wave is to the left of the vehicle's direction of travel. Ultrasonic sensors 9G and 9H are installed on the right side of the vehicle 2 facing right so that the direction of transmission of the probe wave is to the right of the vehicle's direction of travel. Ultrasonic sensors 9I to 9L are installed on the rear of the vehicle 2 facing the opposite direction of travel so that the direction of transmission of the probe wave is to the rear of the vehicle. The height of each ultrasonic sensor 9A to 9L from the ground surface is approximately the same.

[0026] In this embodiment, among the ultrasonic sensors 9A to 9L, the ultrasonic sensors 9A to 9D on the front of the vehicle 2 and the ultrasonic sensors 9I to 9L on the rear of the vehicle 2 are installed in positions where they can receive reflected waves as indirect waves from adjacent sensors. By receiving both direct and indirect waves, it is possible to determine not only the distance to the object but also the specific position of the object (relative position to the vehicle) using triangulation. The ultrasonic sensors 9E to 9H on the sides are installed spaced apart from each other and cannot receive indirect waves, but as the vehicle moves, it is possible to determine the specific position of the object (relative position to the vehicle) using triangulation with respect to the distance measured at the previous position, the distance measured at the current position, and the distance traveled in between.

[0027] On the other hand, the driver assistance ECU 10 is an electronic control unit that performs various processes related to automated driving assistance. For example, it continuously detects the vehicle's current position, the lane it is traveling in, and the positions of surrounding obstacles, and controls the vehicle, such as steering, drivetrain, and brakes, to ensure that the vehicle travels along a generated driving trajectory at a speed according to a similarly generated speed plan. In particular, when performing parking assistance, it uses the detection results from the aforementioned front camera 6, rear camera 7, side cameras 8A, 8B, and ultrasonic sensors 9A to 9L to confirm the parking space and its surroundings, calculates a parking trajectory to the parking space, and controls the vehicle to enter the parking space along the calculated parking trajectory and complete the parking. Furthermore, the LCD display 4 displays bird's-eye and overhead images of the area around the vehicle generated from the images captured by the aforementioned cameras, and also adds wheel images to the area where the vehicle's wheels are located. The driver assistance ECU 10 is connected to the aforementioned control unit 3, LCD display 4, speaker 5, front camera 6, rear camera 7, side cameras 8A, 8B, and ultrasonic sensors 9A-9L via an in-vehicle network such as CAN. It is also connected to various sensors mounted on the vehicle 2, such as the vehicle speed sensor, acceleration sensor, gyro sensor, steering sensor, and shift position sensor, as well as in-vehicle devices such as the navigation system. The detailed configuration of the driver assistance ECU 10 will be described later.

[0028] In addition to the components shown in Figure 1, Vehicle 2 also has other basic components as Vehicle 2, but only the configuration related to the control of the automated driving assistance system and the control related to said configuration will be explained.

[0029] Next, we will describe in detail the driver assistance ECU 10, which is part of the driver assistance system 1 provided by the vehicle 2 described above. Figure 2 is a block diagram showing the configuration of the driver assistance system 1 according to this embodiment.

[0030] As shown in Figure 2, the driver assistance ECU (Electronic Control Unit) 10 is an electronic control unit that controls the entire driver assistance device 1, and includes a CPU 31 as a calculation device and control device, a RAM 32 which is used as working memory when the CPU 31 performs various calculations and stores driving trajectory data when the driving trajectory is calculated, a ROM 33 which stores control programs as well as the driver assistance processing program described later (see Figure 3), and a flash memory 34 which stores programs read from the ROM 33. The driver assistance ECU 10 also has various means as processing algorithms. For example, the image acquisition means acquires an image of the area around the vehicle. The support image generation means generates a support image by superimposing wheel images showing the wheels in the area where the vehicle's wheels are located on the image. The image display means displays the support image on a display device. The behavior acquisition means acquires the behavior of the vehicle. The image update means rotates the wheel images in the support image around the wheel axle in accordance with the behavior of the vehicle. In other words, the driver assistance ECU 10 is an example of an image acquisition means, an assistance image generation means, an image display means, a behavior acquisition means, and an image update means.

[0031] Furthermore, the driver assistance ECU 10 is connected to various sensors 37 for detecting the vehicle's behavior, such as a vehicle speed sensor, wheel speed sensor, acceleration sensor, gyro sensor, steering sensor, and shift position sensor, as well as to various drive units 38 of the vehicle, such as the steering, brakes, accelerator, and transmission. Based on the detection results of these sensors 37, the ECU 10 detects the vehicle's current behavior and controls each drive unit 38 to provide automatic driving assistance for the vehicle 2. Specifically, the automatic driving assistance includes, for example, continuously detecting the vehicle's current position, the lane it is traveling in, and the positions of surrounding obstacles, and controlling the vehicle, such as the steering, drive source, and brakes, to travel along the generated driving trajectory at a speed according to the generated speed plan. However, it is also possible to perform only the steering operation automatically, while controlling the drive source and brakes is done manually.

[0032] Furthermore, the flash memory 34 includes vehicle information DB 35 and wheel image DB 36. Vehicle information DB 35 stores various information about vehicle 2. For example, it stores the installation positions (height from the ground, left-right position) and detection axes (optical axis for cameras) of cameras and ultrasonic sensors 9A to 9L installed on vehicle 2, as well as the overall length, vehicle width, wheelbase, and minimum turning radius. It also stores the type and tread pattern of the tires that the vehicle is initially equipped with. This information is entered in advance by the occupants or personnel from the vehicle manufacturer.

[0033] On the other hand, the wheel image DB36 stores various information used when rendering a 3D model image (hereinafter referred to as the wheel image) that shows the appearance of the wheels (basically only the tire portion, but the wheel may also be included) of the vehicle. Specifically, the wheel image DB36 stores information necessary for executing various processes to create 3DCG, such as modeling, scene layout settings, and rendering, as well as texture images to be applied to the created 3DCG model. The wheel image DB36 may also store a completed wheel image. Furthermore, when storing a completed wheel image, it is desirable to store a wheel image that faithfully reproduces the type of tire and tread pattern of the vehicle, but it is also acceptable to store a common vehicle image regardless of the type of tire and tread pattern of the vehicle.

[0034] Next, the driver assistance processing program executed by the driver assistance ECU 10 in the driver assistance device 1 having the above configuration will be explained with reference to Figure 3. Figure 3 is a flowchart of the driver assistance processing program according to this embodiment. Here, the driver assistance processing program is executed after the ACC power supply (accessory power supply) of the vehicle 2 is turned ON, and is a program that provides user assistance using bird's-eye view images and overhead view images of the area around the vehicle while the vehicle's automatic driving assistance is being performed.

[0035] However, in the following embodiment, bird's-eye and overhead views of the vehicle's surroundings are displayed while the vehicle's automated driving assistance is being performed. However, it is not necessary to display bird's-eye and overhead views of the vehicle's surroundings only while automated driving assistance is being performed. The images may also be displayed while the vehicle is being driven manually. Alternatively, bird's-eye and overhead views of the vehicle's surroundings may be displayed only while specific automated driving assistance, such as parking assistance, is being performed. The programs shown in the flowcharts in Figures 3 and 10 below are stored in the RAM 32 and ROM 33 of the driving assistance device 1 and executed by the CPU 31.

[0036] First, in step 1 (hereinafter abbreviated as S), the CPU 31 determines whether or not assisted driving is being performed by an automated driving support system. As mentioned above, the determination condition may be whether or not a specific automated driving support system, such as parking assistance, is being performed.

[0037] In this embodiment, the user selects to perform automatic driving assistance by operating the control unit 3, and when it is determined that it is possible to perform driving with automatic driving assistance, the automatic driving assistance is performed. The automatic driving assistance includes detecting the vehicle's current position, the lane it is traveling in, and the positions of surrounding obstacles, and automatically controlling the vehicle, such as the steering, drivetrain, and brakes, to travel along the generated driving trajectory at a speed according to the generated speed plan. In particular, for parking assistance, the vehicle control is performed automatically until it is parked in the parking space. However, it is also possible to perform only the steering operation automatically, and control of the drivetrain and brakes based on manual operation.

[0038] If it is determined that assisted driving is being performed by the automated driving assistance system (S1:YES), the process proceeds to S2. Conversely, if it is determined that assisted driving is not being performed by the automated driving assistance system (S1:NO), the driving assistance processing program is terminated.

[0039] In S2, the CPU 31 generates a bird's-eye view image looking diagonally downwards from above around the vehicle, and an overhead view image looking vertically downwards from above around the vehicle, based on real-time images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B. As an example, the method for generating the overhead view image looking vertically downwards from above around the vehicle is described below. As shown in Figure 4, the real-time images captured by each camera are projected onto a virtual projection plane, which is a horizontal plane corresponding to the height of the ground surface. The images projected onto the virtual projection plane are then converted into images viewed from a virtual viewpoint looking vertically downwards from above the vehicle 2, thereby generating the overhead view images for each camera. The conversion to an image viewed from a virtual viewpoint (viewpoint conversion) is performed by first converting each coordinate in the image coordinate system, which is set along a plane perpendicular to the optical axis of the camera, to each coordinate in the ground coordinate system, which is set along the ground surface, and then to each coordinate in the overhead view image coordinate system. The conversion formulas used for each coordinate conversion are already publicly known, so their explanation is omitted. Then, as shown in Figure 5, an overhead view image is generated by combining (stitching together) an overhead view image 41 obtained by changing the viewpoint of the image captured by the front camera 6, an overhead view image 42 obtained by changing the viewpoint of the image captured by the rear camera 7, an overhead view image 43 obtained by changing the viewpoint of the image captured by the side camera 8A, and an overhead view image 44 obtained by changing the viewpoint of the image captured by the side camera 8B. Furthermore, an illustration image 45 schematically showing the vehicle is inserted between each of the overhead view images 41 to 44.

[0040] Regarding the bird's-eye view image, the process is basically the same as generating the overhead view image, with only the angle of the line of sight direction differing during viewpoint conversion, so the explanation will be omitted. However, the virtual viewpoint for the bird's-eye view image will be the interior of the vehicle (i.e., the occupant's viewpoint). Accordingly, the overhead view image obtained by converting the viewpoint of the rear camera 7's images will be excluded from the synthesis. Also, if the bird's-eye view image is to be a view looking diagonally down at the rear of the vehicle when reversing, the overhead view image obtained by converting the viewpoint of the front camera 6's images will be excluded from the synthesis. Furthermore, as shown in Figure 5, the bird's-eye view image 51 obtained by converting the viewpoint of the front camera 6's images, the bird's-eye view image 52 obtained by converting the viewpoint of the side camera 8A's images, and the bird's-eye view image 53 obtained by converting the viewpoint of the side camera 8B's images will be synthesized (stitched together). The area between each bird's-eye view image 51 to 53 is the region where the vehicle's body is located, but in order to display the wheel image in this region for the user to see, as described later, the image of the vehicle's body will not be displayed, and instead, the road surface image 54 will be displayed. Furthermore, it is desirable that the road surface image 54 be an image that does not look out of place with the surroundings. For example, it is possible to extract the current vehicle position from an image previously captured by the front camera 6 and composite it. In the example shown in Figure 5, the virtual viewpoint of the bird's-eye view image is set to the inside of the vehicle (i.e., the occupant's viewpoint), but the position of the virtual viewpoint can be changed as appropriate.

[0041] Subsequently, in S3, the CPU 31 generates a wheel image 55 showing the appearance of the vehicle's wheels. Here, the wheel image 55 is a three-dimensional polygon image. Figure 6 shows an example of the wheel image 55 generated in S3. The vehicle has four wheels: the right front wheel, left front wheel, right rear wheel, and left rear wheel. However, the wheel images are basically all the same in appearance. For example, in the example shown in Figure 6, the wheel images 55 of the right front wheel and left front wheel are shown. In addition, the wheel image 55 is basically an image of only the tire part excluding the wheel, but it may also include the wheel. It may also include the axle. On the other hand, the wheel image 55 includes at least an image corresponding to the tread part (contact surface part) of the wheel that is in contact with the road surface (hereinafter referred to as the tread image 56) and an image corresponding to the side of the wheel that is not in contact with the road surface (non-contact surface part) (hereinafter referred to as the side image 57). As shown in Figure 6, the tread pattern is depicted in the tread image 56. Here, the vehicle information DB35 stores information about the type of tires and tread pattern that the vehicle is equipped with as initial equipment, and it is desirable that the wheel image 55 be an image that reproduces the type of tires and tread pattern that the vehicle is equipped with using that information.

[0042] Furthermore, various material settings and mapping processes may be applied to the wheel image 55. Mapping processes include texture mapping, which applies a texture image to the surface of an object, and bump mapping, which changes the direction of light reflection to create fine irregularities. For example, a texture image is an image that depicts effects such as shading, transparency, and reflection. When the wheel image 55 is placed in the bird's-eye view image in S5, described later, each effect of the texture image is rendered in a way that does not look unnatural, taking into account the orientation of placement, the surrounding brightness, and the position of light sources (e.g., sunlight, streetlights).

[0043] In addition, in S3, the CPU 31 may use the information stored in the wheel image DB 36 to perform processes such as modeling, scene layout setting, and rendering to create the wheel image 55 shown in Figure 6, or it may store a pre-completed wheel image 55 in the wheel image DB 36 and retrieve it in S3 by reading it from the wheel image DB 36.

[0044] Next, in S4, the transparency setting process (Figure 10), described later, is performed. In this transparency setting process, the transparency of the wheel image 55 is set. Specifically, different transparency values ​​are set for the tread image 56 and the side image 57. For the tread image 56, the transparency is set to be higher at the top, further from the road surface than at the bottom, closer to the road surface, while for the side image 57, the transparency is set to be the same regardless of the distance from the road surface. Further details will be described later.

[0045] Next, in S5, the CPU 31 composites the wheel image 55, which was generated in S3 and whose transparency was set in S4, onto the bird's-eye view image generated in S2. Specifically, the wheel image 55 is added to the bird's-eye view image in the region where the vehicle's wheels are located. Figure 7 shows the bird's-eye view image 61 with the wheel image 55 composited onto it.

[0046] The example shown in Figure 7 illustrates the case where a wheel image 55 is superimposed on a bird's-eye view image 61 of the area in front of the vehicle's direction of travel, which is generated when the vehicle is moving forward. The wheel image 55 for the right front wheel and the wheel image 55 for the left front wheel are added to the areas corresponding to the vehicle's right front wheel and left front wheel, respectively. The wheel image 55 for the right front wheel and the wheel image 55 for the left front wheel are symmetrical and essentially the same image. Furthermore, the area enclosed by the dashed line in Figure 7 between the wheel image 55 for the right front wheel and the wheel image 55 for the left front wheel is the area where the vehicle's body is located, as mentioned above. However, in order to make the wheel image 55 visible, the image of the vehicle's body is not displayed, and instead, an image 54 of the vehicle's current position is displayed.

[0047] While details are omitted, when the vehicle is moving in reverse, the image of the right rear wheel 55 and the image of the left rear wheel 55 are similarly composited onto the bird's-eye view image 61 of the rear of the vehicle. Furthermore, the designs of the front wheel image 55 and the rear wheel image 55 may be different to distinguish them.

[0048] Subsequently, in S6, the CPU 31 displays real-time bird's-eye view and overhead view images, which represent the current environment around the vehicle, on the liquid crystal display 4. For the bird's-eye view image, the image after the wheel image 55 has been composited in S5 is displayed. In this embodiment, both the bird's-eye view and overhead view images are displayed simultaneously on the liquid crystal display 4, but only one of them may be displayed, or they may be displayed in a way that allows the user to switch between them. The bird's-eye view and overhead view images displayed on the liquid crystal display 4 will be referred to as support images 65 below.

[0049] Here, Figure 8 shows an example of the support image 65 displayed on the liquid crystal display 4 in S6. As shown in Figure 8, the support image 65 is divided into two screens, with a bird's-eye view image 61 displayed on the left and a bird's-eye view image 62 displayed on the right. Figure 8 specifically shows the case when the vehicle is moving forward, and the bird's-eye view image 61 is a bird's-eye view image looking diagonally downwards in the direction of travel. When the vehicle is moving backward, the bird's-eye view image 61 is a bird's-eye view image looking diagonally downwards behind. As a result, the user can clearly understand the environment around the vehicle by clarifying the relative relationship with the vehicle's position, including areas that are difficult to see directly. The CPU 31 may also calculate the future movement trajectory of the vehicle based on detected values ​​from the vehicle speed sensor and steering sensor, and superimpose the movement trajectory onto the bird's-eye view image 61 and bird's-eye view image 62. From there, the support image 65 will continue to be displayed until the support driving by the automatic driving assistance is terminated (S11: YES), and will be updated at a preset frame rate to display a real-time image of the area around the vehicle. Furthermore, as described later, the wheel image 55 included in the support image 65 rotates in accordance with the actual movement of the vehicle. For example, if the vehicle moves forward, the wheel image 55 also rotates in the forward direction, and if the vehicle moves backward, the wheel image 55 also rotates in the backward direction.

[0050] Next, in S7, the CPU31 acquires the behavior of its own vehicle. Specifically, it determines and acquires the current direction of travel and vehicle speed based on the vehicle speed pulse output from the vehicle speed sensor and the shift position. The direction of travel and vehicle speed also correspond to the rotation direction and rotation speed of the vehicle's wheels. The turning angle of the vehicle may also be acquired using the steering sensor. The turning angle of the vehicle also corresponds to the angle of the vehicle's wheels.

[0051] Next, in S8, the CPU 31 determines whether the vehicle is moving forward or backward based on the vehicle behavior acquired in S7.

[0052] If it is determined that the vehicle is moving forward or backward (S8: YES), the process proceeds to S9. Conversely, if it is determined that the vehicle is stopped (S8: NO), the process proceeds to S11.

[0053] In S9, the CPU 31 updates the wheel image 55 included in the support image 65 in accordance with the vehicle's behavior. Specifically, it rotates the wheel image 55 displayed in the support image 65 around the wheel axle (center of the wheel image 55) in accordance with the actual rotation direction and speed of the vehicle's wheels. However, the rotation speed of the wheel image 55 does not necessarily have to be the same as the actual rotation speed of the wheels; for example, it may be rotated at a predetermined ratio (e.g., 1 / 2) of the actual rotation speed. Alternatively, an upper limit may be set on the rotation speed. Along with the rotation of the wheel image 55, the tread pattern included in the tread image is also rotated together. When rotating the wheel image 55, it is updated at a preset frame rate.

[0054] As a result, for example, when the vehicle is moving forward, the wheel image 55 rotates in the forward direction (towards the front of the screen) as shown in Figure 9. On the other hand, when the vehicle is moving backward, the wheel image 55 rotates in the backward direction. Note that when the vehicle is moving backward, a bird's-eye view image 61 of the rear of the vehicle, which is the opposite direction to when it is moving forward, is displayed, so the rotation direction of the wheel image 55 included in the bird's-eye view image 61 is the same as when it is moving forward (the direction towards the front of the screen as shown in Figure 9).

[0055] Furthermore, if the vehicle's turning angle is obtained in S7 as part of the vehicle's behavior, the angle (direction of travel) of the wheel image 55 displayed in the support image 65 may be changed to match the actual wheel angle of the vehicle.

[0056] Next, in S10, the transmittance update process (Figure 10), described later, is performed. In the transmittance update process, the transmittance is reset (updated) for the wheel image 55 after rotation in S9. Of the wheel image 55, the side image 57 has the same transmittance regardless of the distance from the road surface, so the transmittance update is performed only on the tread image 56 that is in contact with the road surface. However, the transmittance update process in S10 may also be performed on the side image 57. Even in that case, the transmittance of the side image 57 will not change as a result. On the other hand, for the tread image 56, after rotation, the transmittance is updated so that the transmittance is higher at the top, which is further from the road surface, than at the bottom, which is closer to the road surface. Details will be described later.

[0057] Subsequently, in S11, the CPU 31 determines whether or not to terminate the assisted driving provided by the automated driving assistance system. Here, the termination of the assisted driving assistance provided by the automated driving assistance system is determined by, for example, the completion of parking in the parking space in the case of parking assistance. Alternatively, the termination may be determined by the user performing a predetermined termination operation on the control unit 3, or by the shift position being moved to "P" or the engine being turned off. Furthermore, the termination may be determined by the fact that it is no longer possible to continue the automated driving assistance system.

[0058] Then, if it is determined that the assisted driving by the automated driving assistance system has ended (S11:YES), the driving assistance processing program is terminated. On the other hand, if it is determined that the assisted driving by the automated driving assistance system has not ended (S11:NO), the program returns to S7 and continues to display the assistance image on the liquid crystal display 4.

[0059] Next, the subprocesses of the transmittance setting process executed in S4 and the transmittance update process executed in S10 will be explained with reference to Figure 10. Since the transmittance setting process executed in S4 and the transmittance update process executed in S10 are almost identical processes, they will be explained together below. Figure 10 is a flowchart of the subprocessing programs for the transmittance setting process and the transmittance update process.

[0060] Here, the transparency setting process and transparency update process are processes for setting the transparency of the wheel image 55, and in particular, the transparency is set for each polygon included in the wheel image 55. The following processes S21 to S24 are performed on the polygons included in the wheel image 55 generated in S3, in particular on the polygons included in the tread image 56, and the processes S21 to S24 are performed on all target polygons, and after the transparency is set, the process proceeds to S25. As mentioned above, the wheel image 55 is a three-dimensional polygon image, and as shown in Figure 11, the constituent surface (surface) of the wheel image 55 is formed by multiple polygons PL. That is, the wheel image 55 is a three-dimensional shape formed by the combination of multiple polygons PL. The processes S21 to S24 are performed on each of these polygons PL.

[0061] However, the processing in S21 to S24 may be limited to polygons PL that are within the range visible to the user (display range) when the wheel image 55 is composited with the bird's-eye view image in S5. In other words, polygons PL that are not within the range visible to the user, even if they are included in the wheel image 55, such as polygons PL located on the opposite side of the virtual viewpoint from the wheel axle, are not drawn in the support image 65 and therefore do not need to have their transparency set. Accordingly, such polygons PL may be excluded from processing. Furthermore, among the polygons included in the wheel image 55, the transparency will be set in S25, which will be described later, for polygons that are included in the side view image 57 in particular.

[0062] First, in S21, the CPU31 obtains the current normal vectors of each vertex of the polygon PL to be processed (hereinafter referred to as vertex normal vectors). In the transparency update process, the vertex normal vectors obtained are obtained after the rotation process in S9 has been performed.

[0063] The following provides a specific example of the process in S21. For the sake of simplicity, the polygon PL in the following example is assumed to be a triangle, but the shape of the polygon PL is not limited to a triangle; it can be any polygon, such as a quadrilateral. Furthermore, while the vertices of the polygon PL are assumed to be three vertices touching each other, four or more vertices may also be touching.

[0064] For example, as shown in Figure 12, a triangular polygon PL has three vertices a, b, and c, so the vertex normal vectors obtained are the vertex normal vector na of vertex a, the vertex normal vector nb of vertex b, and the vertex normal vector nc of vertex c. Here, the vertex normal vector is calculated from the face normal vectors of each polygon PL to which the vertex belongs. For example, taking the vertex normal vector na in Figure 12 as an example, the vertex normal vector na has three different polygon PLs belonging to that vertex, and the face normal vectors of each polygon PL are N1, N2, and N3. Note that the face normal vector is a vector perpendicular to the face and can be calculated from the vectors connecting the vertices that make up the face. For example, the face normal vector N1 can be calculated by calculating the cross product of the vector extending from vertex a to vertex b and the vector extending from vertex a to vertex c. Similarly, the face normal vectors N2 and N3 can also be calculated.

[0065] The CPU 31 then calculates the vertex normal vector na by first calculating the face normal vectors N1 to N3 for each face, and then adding them all together and normalizing them. Similarly, the vertex normal vector nb for vertex b and the vertex normal vector nc for vertex c are calculated by adding all the face normal vectors of each polygon PL to which the vertex belongs and normalizing them. Figure 12 illustrates an example where three polygon PLs belong to a vertex, but the calculation method is basically the same even when four or more polygon PLs belong to a vertex, only the number of face normal vectors to be added increases. Note that the normal vectors na to nc for each vertex may be calculated using results from prior calculations. Since the normal vectors for each vertex are determined by the polygon shape, they can be calculated in advance. Furthermore, from S21 onward, each vector is calculated in a three-dimensional Cartesian coordinate system with the direction of travel of the vehicle as the Z axis and the vertical direction as the Y axis, as shown in Figure 12.

[0066] On the other hand, in the transparency update process performed in S10, it is also possible to calculate vertex normal vectors corresponding to the wheel's rotation by using the normal vectors of each vertex and the rotation matrix, which have been calculated in advance based on the polygon shape.

[0067] Next, in S22, the CPU 31 obtains a reference vector. The reference vector is a vector that serves as the basis for setting the transparency of each polygon, and it is a vector that indicates the direction in which the wheel image 55 should be most transparent. In this embodiment, the reference vector is either a "vector corresponding to the vertical direction" or a "vector corresponding to the line of sight from the virtual viewpoint of the bird's-eye view image". As shown in Figure 13, the "vector corresponding to the vertical direction" is more specifically a vertically upward vector, and the "vector corresponding to the line of sight from the virtual viewpoint of the bird's-eye view image" is more specifically a vector indicating the direction from each vertex a to c of the polygon PL to be processed to the virtual viewpoint.

[0068] Subsequently, in S23, the CPU 31 calculates the transparency for each vertex of the polygon PL to be processed, based on the vertex normal vector obtained in S21 and the reference vector obtained in S22. Basically, vertices with vertex normal vectors that differ little from the reference vector have a high transparency, while vertices with vertex normal vectors that differ greatly from the reference vector have a low transparency.

[0069] The following describes the process in S23 with specific examples. As shown in Figure 14, if θ is the angle between the vertex normal vector of the vertex for which the transmittance is to be calculated and the reference vector obtained in S22, the transmittance t can be calculated, for example, by the following equation (1). t = cosθ (when cosθ > 0), t = 0 (when cosθ ≤ 0) ... (1) Furthermore, we assume that the transmittance is 100% (complete transmission) at t=1 and 0% (opaque) at t=0. That is, transmittance = t × 100%.

[0070] Furthermore, if the reference vector is a vector indicating the direction from each vertex of polygon PL to the virtual viewpoint, the reference vector should be a vector indicating the direction from the vertex for which the transmittance is to be calculated to the virtual viewpoint. In addition, an upper or lower limit may be set for the transmittance t calculated in equation (1) above. This prevents the visibility of the captured image from decreasing due to lowering the transmittance too much. Similarly, it prevents the visibility of the wheel image 55 from decreasing due to raising the transmittance too much. For example, it is possible to set the upper limit to 90% and the lower limit to 10%. Furthermore, the value may be adjusted by multiplying the transmittance t calculated in equation (1) above by a predetermined coefficient (for example, the transmittance of the entire polygon).

[0071] Next, in S24, the CPU 31 sets the transparency of the faces of the polygon PL to be processed based on the transparency of each vertex calculated in S23. Specifically, the transparency of the faces of the polygon PL is set by the transparency of each vertex set in S23 and the distance from each vertex. For example, as shown in Figure 15, if the transparency of vertex a is set to 40%, the transparency of vertex b is set to 35%, and the transparency of vertex c is set to 30%, the transparency between vertex b and vertex a is set so that it changes linearly and smoothly from 35% to 40% as you move from vertex b to vertex a. Between vertex a and vertex c, the transparency is set so that it changes linearly and smoothly from 40% to 30% as you move from vertex a to vertex c. Between vertex b and vertex c, the transparency is set so that it changes linearly and smoothly from 35% to 30% as you move from vertex b to vertex c. In other words, the CPU 31 sets the transparency of each vertex of the polygon PL to be processed to the transparency calculated in S23, and sets the transparency of the faces between vertices to change as smoothly as possible without causing abrupt changes.

[0072] Then, the processes described in S21 to S24 are performed on all polygons in the wheel image 55 generated in S3, particularly those included in the tread image 56, and after the transparency is set, the process proceeds to S25.

[0073] In S25, the CPU 31 sets the transparency of the polygons included in the wheel image 55, particularly those included in the side image 57. In this embodiment, the transparency of the side image 57 is set to the same level regardless of the distance from the road surface. As an example, the transparency is set to 25%. If the side image 57 is divided into multiple regions, the transparency may be set differently for each region. Also, although the processing in S25 is performed in the transparency setting process in S4, it is not necessary to perform it in the transparency update process in S10 (the transparency of the side image 57 does not change even when the wheel rotates), so the processing in S25 may be omitted. After that, the process proceeds to S5 or S11.

[0074] However, the transparency of the side image 57 may also be set and updated in the same way as the tread image 56 using the processing in S21 to S24. Even if the transparency of the side image 57 is calculated in the same way as the tread image 56, the direction of the vertex normal vectors of the polygons in the side image 57 is all the same (i.e., θ in equation (1) is the same), so as a result, the same transparency will be set regardless of the distance from the road surface. Furthermore, even if the wheel image 55 rotates, the direction of the vertex normal vectors of the polygons in the side image 57 does not change before and after rotation, and the same transparency will be set.

[0075] In the above transparency setting and transparency update processes, the transparency of the wheel image 55 is set and updated. As a result, for each polygon forming the surface of the wheel image 55, the transparency of the polygon is set and updated such that the transparency is higher for polygons indicated by vertex normal vectors with a smaller difference from the reference vector (except for the side image 57). That is, as shown in Figure 16, the wheel image 55 basically has a higher transparency set in the upper part, which is further from the road surface, than in the lower part, which is closer to the road surface. Furthermore, if the wheel image 55 is rotated according to the behavior of the vehicle, the transparency is updated as needed so that even after rotation, the transparency is higher in the upper part, which is further from the road surface, than in the lower part, which is closer to the road surface. In this embodiment, the reference vector is either a "vector corresponding to the vertical direction" or a "vector corresponding to the line of sight from the virtual viewpoint of the bird's-eye view image". As shown in Figure 16, if the reference vector is a "vector corresponding to the vertical direction", the range with high transparency is centered on the vertices of the wheel image 55. On the other hand, if the reference vector is defined as "a vector corresponding to the line of sight direction from the virtual viewpoint of the bird's-eye view image," then the area with high transmittance will be centered on the surface opposite the virtual viewpoint. In other words, when viewed from the virtual viewpoint, defining the reference vector as "a vector corresponding to the line of sight direction from the virtual viewpoint of the bird's-eye view image" will result in a wider area of ​​high transmittance.

[0076] Furthermore, by setting a high transparency in the upper part of the wheel image 55, which has a large overlap with the bird's-eye view image 61 and creates many blind spots, visibility of the blind spots can be ensured. On the other hand, by setting a low transparency in the lower part of the wheel image 55, visibility of the wheel image 55 can be ensured. In addition, for example, the difference between the vertex normal vector and the reference vector is larger near the left and right edges of the tread surface of the wheel image 55 compared to the central area, and the transparency is set lower compared to the central area. As a result, even in the upper area where the transparency of the wheel image 55 is high, the left and right edges of the tread surface (i.e., the outline of the tire) remain displayed, making it possible to ensure the visibility of the wheel image 55.

[0077] As described in detail above, according to the driving support device 1 and the computer program executed by the driving support device 1 according to this embodiment, a support image is generated by superimposing a wheel image 55 indicating the wheels on an image captured of the area around the vehicle, in the region where the vehicle's wheels are located (S3), and the support image is displayed on the liquid crystal display 4 (S6). The wheel image 55 in the support image is rotated around the wheel axle in accordance with the vehicle's movement (S9). The wheel image 55 has a higher transmittance set in the upper part, which is further from the road surface, than in the lower part, which is closer to the road surface. On the other hand, when the wheel image 55 is rotated, the transmittance of the wheel image 55 is updated according to the rotation of the wheel image 55 in accordance with the transmittance setting of the wheel image 55 (S10). Therefore, by setting a high transmittance in the upper part of the wheel image 55, which has a wide overlap with the captured image and creates many blind spots, visibility of the blind spots can be ensured, while by setting a low transmittance in the lower part of the wheel image 55, visibility of the wheel image 55 can also be ensured. Therefore, it is possible to achieve both visibility of the wheel image 55 and visibility of the blind spot hidden by the wheel image 55. Furthermore, the wheel image 55 includes a tread image 56 that is in contact with the road surface and a side view image 57 that is not in contact with the road surface. The transparency settings for the wheel image 55 are such that the transparency is higher in the upper part of the wheel that is further from the road surface than in the lower part that is closer to the road surface, while the transparency of the side view image 57 is set to the same level regardless of the distance from the road surface. Therefore, even if the transparency of the tread image 56 is increased by the side view image 57 which is displayed with a fixed transparency, the wheel shape can be maintained for the wheel image 55 as a whole. Furthermore, the three-dimensional shape of the surface of the wheel image 55 is formed by a combination of multiple polygons. The vertex normal vectors of each polygon are compared with a reference vector, and the transparency is updated so that the transparency of the polygons indicated by normal vectors with smaller differences is higher (S23, S24). Therefore, even when the wheel image 55 is rotated, the transparency of each polygon can be easily calculated by using the normal vectors of the rotated polygons. Furthermore, the support image 65 is a bird's-eye view of the area around the vehicle, viewed diagonally from a virtual viewpoint above. The reference vector is either a vector corresponding to the vertical direction or a vector corresponding to the line of sight from the virtual viewpoint. By defining the direction in which the most transparency is desired for the wheel image 55 as the reference vector, it becomes possible to calculate the transparency for each polygon by comparing the reference vector with the normal vector.

[0078] It should be noted that the present invention is not limited to the embodiments described above, and various improvements and modifications are possible without departing from the spirit of the invention. For example, in this embodiment, the bird's-eye view and overhead view images generated from images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B are displayed on the liquid crystal display 4 as images of the scenery around the vehicle. However, instead of processed images, the raw images captured by the front camera 6 or rear camera 7 may be displayed on the liquid crystal display 4, and the wheel images 55 may be added to those images. Also, in this embodiment, the wheel images 55 are only added to the bird's-eye view images, but the wheel images 55 may also be added to the overhead view images.

[0079] Furthermore, in this embodiment, the transmittance is calculated using the vertex normal vectors located at the vertices of the polygon, but the transmittance may also be calculated using the face normal vectors instead of the vertex normal vectors. For example, in S23, the transmittance t can be calculated from equation (1) using the angle θ between the face normal vector of the polygon PL to be processed and the reference vector, and the calculated transmittance t can be set as the transmittance of the entire polygon PL to be processed.

[0080] Furthermore, in this embodiment, the transparency is calculated using the vertex normal vectors located at the vertices of the polygon, but the transparency may also be set by the distance D from the virtual viewpoint instead of the vertex normal vectors. For example, the distance D from the virtual viewpoint to each vertex of the polygon is calculated, and then the transparency is calculated for each vertex from the calculated distance D, and the processing from S24 onwards is performed using the calculated transparency. Note that the calculation formula should be such that a shorter distance D will result in a higher transparency. Even when setting the transparency using the distance D as described above, the same effects as when the reference vector is "a vector corresponding to the line of sight direction from the virtual viewpoint" can be expected.

[0081] Furthermore, the scenery around the vehicle displayed on the liquid crystal display 4 may be a schematicly generated virtual scenery image (for example, a 3D map image) rather than an image captured by the camera.

[0082] Furthermore, in this embodiment, it is assumed that the support image 65 shown in Figure 8 is displayed while driving is being performed with automated driving assistance. However, the support image 65 shown in Figure 8 may also be displayed while driving is being performed manually.

[0083] Furthermore, in this embodiment, the driver assistance ECU 10 of the driver assistance device 1 executes the processing of the driver assistance processing program (Figure 3), but the execution entity can be changed as appropriate. For example, the control unit of the liquid crystal display 4, the vehicle control ECU, the control unit of the navigation device, or other in-vehicle devices may be used to execute the processing. [Explanation of symbols]

[0084] 1...Driving support system, 2...Vehicle, 3...Operation unit, 4...Liquid crystal display (display device), 6...Front camera, 7...Rear camera, 8A,8B...Side cameras, 10...Driving support ECU (Example of means for acquiring captured images, generating support images, displaying images, acquiring behavior, and updating images), 55...Wheel image, 56...Tread image (image of the contact surface), 57...Side image (image of the non-contact surface), 61...Bird's-eye view image, 65...Support image, PL...Polygon, na,nb,nc...Vertex normal vectors

Claims

1. Image acquisition means for acquiring captured images of the area around the vehicle, Support image generation means for generating a support image by superimposing a wheel image showing the wheel in the region where the vehicle's wheels are located on the aforementioned captured image, It has an image display means for displaying the aforementioned support image on a display device, The aforementioned wheel image is a driver assistance device in which higher transparency is set in the upper part, which is further from the road surface, than in the lower part, which is closer to the road surface.

2. The wheel image includes an image of the contact surface that is in contact with the road surface and an image of the non-contact surface that is not in contact with the road surface. The driving assistance device according to claim 1, wherein the transparency of the wheel image is set such that the transparency of the contact surface image is higher at the top, further from the road surface than at the bottom, closer to the road surface, while the transparency of the non-contact surface image is set to the same regardless of the distance from the road surface.

3. The three-dimensional shape of the surface of the wheel image is formed by a combination of multiple polygons. A transparency level is set for each of the aforementioned multiple polygons. A means for acquiring the behavior of the vehicle itself, It includes an image updating means that rotates the wheel image in the support image around the wheel axle in accordance with the behavior of the vehicle, The image updating means is For each of the aforementioned plurality of polygons, the normal vector represented by the polygon is rotated in accordance with the rotation of the wheel image, and then the rotated normal vector is compared with the reference vector. The driving support device according to claim 1 or claim 2, which updates the transmittance of the wheel image in accordance with the rotation of the wheel image by updating the transmittance such that the transmittance of the polygon indicated by the normal vector has a smaller difference from the reference vector.

4. The aforementioned support image is a bird's-eye view of the area around the vehicle, viewed diagonally from a virtual viewpoint above. The driving support device according to claim 3, wherein the reference vector is a vector corresponding to the vertical direction or a vector corresponding to the line of sight direction from the virtual viewpoint.

Images