Drive support device

Abstract

To provide a drive support device capable of making a user correctly recognize a rotation direction of a wheel image even when the wheel image is rotated according to travel of a vehicle.SOLUTION: A drive support device acquires a captured image obtained by imaging a periphery of an own vehicle, generates a support image 51 in which a wheel image 46 showing a wheel is added to a region where a wheel of the own vehicle is located with respect to the captured image, while displays the generated support image 51 on a liquid crystal display 4, acquires behavior of the own vehicle, and rotates the wheel image 46 in the support image 51 around a wheel shaft according to the behavior of the own vehicle. The wheel image 46 includes a tread image corresponding to a tread portion of the wheel, a pattern image 48 showing a pattern different from a tread pattern is displayed on a surface, and the pattern image 48 is configured to be integrally rotated with the wheel image 46 when the wheel image 46 is rotated.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a driving assistance device that assists driving of a vehicle. [Background technology]

[0002] Conventionally, various means have been used as information providing means for providing vehicle occupants with various information for providing vehicle driving support such as route guidance and obstacle warnings. For example, such means include display on a liquid crystal display installed in the vehicle and audio output from a speaker. However, there are blind spots around the vehicle that are difficult to see from the driver's position. In particular, when performing special operations such as parking, a surrounding image captured by a camera installed in the vehicle has conventionally been displayed on a liquid crystal display to allow the driver to understand the situation in such blind spots.

[0003] For example, Japanese Patent Application Laid-Open Publication No. 2018-191242 discloses a technology for displaying an image of the vehicle's surroundings captured by a camera on an LCD display installed inside the vehicle, and displaying wheel images including tread patterns at the position of the vehicle's wheels in the image to clarify the direction from which the image of the vehicle's surroundings displayed on the LCD display was captured. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-191242 A (paragraphs 0049-0051, Figure 10) Summary of the Invention [Problem to be solved by the invention]

[0005] In the above-mentioned Patent Document 1, a wheel image including a tread pattern is displayed against an image of the vehicle's surroundings. However, if the wheel image is rotated in accordance with the vehicle's movement to achieve a more realistic representation, the tread pattern will also rotate. Here, since the tread pattern is basically a repeated pattern, there is a problem that the rotation direction of the wheel image cannot be correctly recognized due to a strobe effect depending on the frame rate and rotation speed of the displayed image. As a result, a sense of incongruity may occur, such as a discrepancy between the direction of travel and the rotation direction of the wheel image, and it may be difficult to correctly understand which direction the displayed image of the vehicle's surroundings was captured relative to the vehicle.

[0006] The present invention has been made to solve the above-mentioned problems in the conventional art, and aims to provide a driving assistance device that allows the user to correctly recognize the rotation direction of the wheel image even when the wheel image is rotated in accordance with the vehicle's movement by adding and displaying a pattern image that shows a pattern different from the tread pattern to the wheel image. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, the driving assistance device of the present invention comprises an image acquisition unit that acquires an image of the periphery of the vehicle, an assistance image generation unit that generates an assistance image by adding wheel images showing the wheels of the vehicle to the area of ​​the image where the wheels are located, an image display unit that displays the assistance image on a display device, a behavior acquisition unit that acquires the behavior of the vehicle, and an image update unit that rotates the wheel image in the assistance image around the wheel axis in accordance with the behavior of the vehicle, wherein the wheel image includes a tread image corresponding to the tread portion of the wheel, and a pattern image showing a pattern different from the tread pattern is displayed on the surface, and when rotating the wheel image, the image update unit rotates the pattern image together with the wheel image. The "captured image of the surroundings of the vehicle" may be an image captured by an imaging device such as a camera, or may be an image obtained by processing the captured image. For example, it may be an image obtained by combining images captured by multiple cameras. [Effects of the Invention]

[0008] According to the driving assistance device of the present invention having the above configuration, a captured image of the surroundings of the vehicle is displayed, a wheel image is displayed at the position of the vehicle in the captured image, and a pattern image showing a pattern different from the tread pattern is additionally displayed on the wheel image, so that the user can correctly recognize the rotation direction of the wheel image even when the wheel image is rotated in accordance with the running of the vehicle. As a result, the user can accurately understand the direction of the displayed captured image relative to the vehicle, without causing any discomfort such as a discrepancy between the direction of travel and the rotation direction of the wheel image. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic configuration diagram of a vehicle according to an embodiment of the present invention. [Figure 2] 1 is a block diagram showing a configuration of a driving assistance device according to an embodiment of the present invention. [Figure 3] 4 is a flowchart of a driving assistance processing program according to the present embodiment. [Figure 4] 10A and 10B are diagrams illustrating a method for converting a captured image into a bird's-eye view image. [Figure 5] FIG. 10 is a diagram illustrating a method for generating an overhead image. [Figure 6] FIG. 10 is a diagram showing a wheel image. [Figure 7] FIG. 10 is a diagram showing a pattern image displayed on a side of a wheel image. [Figure 8] FIG. 10 is a diagram showing an image in which a wheel image is synthesized with a bird's-eye view image. [Figure 9] FIG. 10 is a diagram showing an example of a support image displayed on a liquid crystal display. [Figure 10] 10A and 10B are diagrams showing an example of rotating wheel images in accordance with vehicle behavior. DETAILED DESCRIPTION OF THE INVENTION

[0010] A detailed description will be given below of a specific embodiment of a driving assistance device according to the present invention with reference to the drawings. First, a vehicle 2 equipped with a driving assistance device 1 according to this embodiment will be described below. Fig. 1 is a schematic diagram of the vehicle 2 according to this embodiment.

[0011] Here, the vehicle 2 may be, for example, an automobile (internal combustion engine automobile) that uses an internal combustion engine (engine, etc.) as a drive source, an automobile (electric automobile, fuel cell automobile, etc.) that uses an electric motor (motor, etc.) as a drive source, or an automobile that uses both of these as a drive source (hybrid automobile). Furthermore, the vehicle type is not limited, and it may be a standard car, or a large commercial truck, bus, construction machinery, etc. Furthermore, although the following description will be of a four-wheeled automobile, it may also be a two-wheeled or three-wheeled vehicle.

[0012] However, vehicle 2 is a vehicle capable of manual driving, in which the vehicle drives based on the driving operation of the user, as well as assisted driving using automatic driving assistance, in which the vehicle drives automatically without the driving operation of the user.

[0013] Furthermore, autonomous driving assistance may be performed only under specific circumstances, such as when parking, or may be performed for all road sections, or may be performed only while the vehicle is traveling on a specific road section (for example, a highway with a gate (manned or unmanned, toll or free) at the boundary). In the following description, the autonomous driving section in which autonomous driving assistance is performed includes all road sections, including general roads and highways, as well as parking lots, and is performed only when the user selects to perform autonomous driving assistance (for example, turns on the autonomous driving start button) and it is determined that autonomous driving assistance is possible. On the other hand, vehicle 2 may be a vehicle that is only capable of assisted driving with autonomous driving assistance. Alternatively, autonomous driving assistance may be performed only when the vehicle is traveling to a parking space when parking (i.e., parking assistance).

[0014] In the vehicle control in the autonomous driving assistance of this embodiment, for example, the current position of the vehicle, the lane the vehicle is traveling on, and the positions of surrounding obstacles are detected at any time, and vehicle control such as steering, drive source, and braking is automatically performed so that the vehicle travels along the generated travel trajectory at a speed in accordance with the generated speed plan. However, it is also possible to perform only the steering operation automatically, and control the drive source and brakes based on manual operation. Furthermore, when the autonomous driving assistance is performed, as described below, the scenery (actual scene) around the vehicle captured by a camera installed in the vehicle is displayed on an in-vehicle display, and in particular, wheel images showing the wheels are added to the area of ​​the scenery where the wheels of the host vehicle are located.

[0015] 1, the vehicle 2 has an operation unit 3 that accepts operations from the occupant, a liquid crystal display 4 that displays bird's-eye and overhead images of the vehicle's surroundings and other driving assistance-related information to the occupant, a speaker 5 that outputs audio guidance related to the driving assistance, a front camera 6, a rear camera 7, and side cameras 8A and 8B for capturing images of the vehicle's surroundings, ultrasonic sensors 9A to 9F that detect obstacles around the vehicle, and a driving assistance ECU (electronic control unit) 10 that performs various calculations based on input information. The driving assistance ECU 10 and other components are collectively referred to as a driving assistance device 1.

[0016] Each component of the vehicle 2 will be described below. First, the operation unit 3 is arranged, for example, on the front of the handle (also called the steering wheel), and includes operation buttons and the like that are operated when starting automatic driving assistance. By operating the operation unit 3, the user can switch between manual driving, in which the vehicle travels based on the user's driving operation, and automatic driving assistance, in which the vehicle travels automatically without the user's driving operation. The operation unit 3 may have a touch panel provided on the front of the liquid crystal display 4. It may also have a microphone and a voice recognition device.

[0017] The liquid crystal display 4 is mounted on the instrument panel of the vehicle 2, and displays bird's-eye and overhead images of the vehicle surroundings generated by performing viewpoint conversion and synthesis processing on images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B during autonomous driving assistance. Furthermore, in particular, wheel images (tire images) showing the wheels of the vehicle are also displayed in the bird's-eye view image in the area where the wheels are located. Details of the wheel images will be described later. The liquid crystal display 4 may also be used for a navigation device.

[0018] The speaker 5 is mounted on the instrument panel of the vehicle 2 and outputs voice guidance and warning sounds related to driving assistance. The speaker 5 may also be used for a navigation device.

[0019] The forward camera 6 is an imaging device having a camera using a solid-state imaging element such as a CCD, and is installed, for example, above the front bumper of the vehicle 2 or behind the rearview mirror, with its optical axis facing forward in the direction of travel of the vehicle.

[0020] The rear camera 7 is an imaging device having a camera that also uses a solid-state imaging element such as a CCD, and is installed, for example, near the upper center of the license plate attached to the rear of the vehicle 2, with the optical axis facing toward the rear of the vehicle.

[0021] Furthermore, the side cameras 8A and 8B are imaging devices each having a camera using a solid-state imaging element such as a CCD, and are attached to the left and right side mirrors of the vehicle 2, for example, with their optical axes directed to the sides of the vehicle.

[0022] The driving assistance ECU 10 generates bird's-eye and overhead images of the surroundings of the vehicle by performing viewpoint conversion and synthesis processing on the captured images taken by the front camera 6, rear camera 7, and side cameras 8A and 8B. During execution of autonomous driving assistance, the driving assistance ECU 10 also performs image recognition processing on the captured images to detect lane lines and obstacles (other vehicles, pedestrians, bicycles, walls, guardrails, and other structures) around the vehicle, and executes autonomous driving assistance based on the detection results.

[0023] The ultrasonic sensors 9A-9F are positioned at a predetermined interval at the front and rear of the vehicle 2. They transmit ultrasonic waves as probe waves around the vehicle 2 and receive reflected waves from objects around the vehicle, thereby detecting the objects that reflect the probe waves. Specifically, the distance to the object that reflected the probe wave is detected by measuring the time from transmission to reception. Note that objects detected by the ultrasonic sensors 9A-9F include obstacles that the vehicle 2 must avoid while traveling, such as people, bicycles, other vehicles, and walls. Note that millimeter-wave sensors may be used instead of ultrasonic sensors to detect obstacles. Furthermore, in this embodiment, obstacles can also be detected by performing image recognition on images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B, as described above. Therefore, if obstacle detection is performed using these cameras, the ultrasonic sensors 9A-9F may be omitted.

[0024] Meanwhile, the driving assistance ECU 10 is an electronic control unit that performs various processes related to autonomous driving assistance. For example, it constantly detects the current position of the vehicle, the lane the vehicle is traveling on, and the positions of surrounding obstacles, and controls the vehicle, such as steering, drive source, and braking, so that the vehicle travels along the generated travel trajectory at a speed according to the generated speed plan. It also displays the scenery (real scene) around the vehicle on the LCD display 4, and further displays wheel images representing the vehicle's wheels in the area where the wheels are located. The driving assistance ECU 10 is connected to the operation unit 3, LCD display 4, speaker 5, front camera 6, rear camera 7, side cameras 8A and 8B, and ultrasonic sensors 9A to 9F via an in-vehicle network such as a CAN. It is also connected to various sensors installed in the vehicle 2, such as a vehicle speed sensor, acceleration sensor, and steering sensor, as well as an in-vehicle device such as a navigation system. The detailed configuration of the driving assistance ECU 10 will be described later.

[0025] In addition, vehicle 2 has basic components as vehicle 2 in addition to the components shown in Figure 1, but we will only explain the configuration related to the control of automatic driving assistance and the control related to that configuration.

[0026] Next, a detailed description will be given of the driving assistance ECU 10 in particular of the driving assistance device 1 provided in the vehicle 2. Fig. 2 is a block diagram showing the configuration of the driving assistance device 1 according to this embodiment.

[0027] As shown in FIG. 2, the driving assistance ECU (electronic control unit) 10 is an electronic control unit that controls the entire driving assistance device 1. It includes a CPU 31, which functions as a calculation device and a control device; a RAM 32, which is used as a working memory when the CPU 31 performs various calculation processes and stores route data and the like when a route is searched; a ROM 33, which stores control programs as well as a driving assistance processing program (see FIG. 3 ), which will be described later; and a flash memory 34, which stores programs read from the ROM 33. The driving assistance ECU 10 includes various means for processing algorithms. For example, the captured image acquisition unit acquires captured images of the surroundings of the vehicle. The support image generation unit generates a support image by adding wheel images representing the wheels of the vehicle to the captured image in the area where the wheels are located. The image display unit displays the support image on the LCD display 4. The behavior acquisition unit acquires the behavior of the vehicle. The image update unit rotates the wheel images in the support image around the wheel axis in accordance with the behavior of the vehicle.

[0028] The driving assistance ECU 10 is also connected to various sensors 36 for detecting vehicle behavior, such as a vehicle speed sensor, an acceleration sensor, and a steering sensor, as well as various vehicle drive units 37, such as the steering, brakes, and accelerator, and detects the current vehicle behavior based on the detection results of these sensors 36, while controlling the various drive units 37 to provide automatic driving assistance for the vehicle 2. Specific details of the automatic driving assistance include, for example, detecting the current vehicle position, the lane the vehicle is traveling on, and the positions of surrounding obstacles at any time, and controlling the vehicle, such as the steering, drive source, and brakes, so that the vehicle travels along a generated travel trajectory at a speed in accordance with a speed plan that is also generated. However, it is also possible to automatically perform only the steering operation, while manually controlling the drive source and brakes.

[0029] The flash memory 34 also includes a vehicle information DB 35, which stores various information related to the vehicle 2. For example, the installation positions (height from the ground and left-right positions) of the cameras and ultrasonic sensors 9A-9F installed on the vehicle 2, the detection axis X, the overall length, the vehicle width, the wheelbase, the minimum turning radius, etc. are stored. Furthermore, the type and tread pattern of tires that the vehicle is initially equipped with are also stored. This information is input in advance by the occupant or a person from the vehicle manufacturer.

[0030] Next, a driving assistance processing program executed by the driving assistance ECU 10 in the driving assistance device 1 having the above configuration will be described with reference to Fig. 3. Fig. 3 is a flowchart of the driving assistance processing program according to this embodiment. Here, the driving assistance processing program is executed after the ACC power supply (accessory power supply) of the vehicle 2 is turned on, and is a program that provides assistance to the user using bird's-eye view images and overhead view images of the area around the vehicle while automatic driving assistance is being performed for the vehicle.

[0031] However, although the following embodiment is characterized in that a bird's-eye view image or a bird's-eye view image of the surroundings of the vehicle is displayed while automatic driving assistance is being performed for the vehicle, it is not necessary to display a bird's-eye view image or a bird's-eye view image of the surroundings of the vehicle only while automatic driving assistance is being performed, and a bird's-eye view image or a bird's-eye view image of the surroundings of the vehicle may also be displayed while the vehicle is being driven manually. Alternatively, a bird's-eye view image or a bird's-eye view image of the surroundings of the vehicle may be displayed only while special automatic driving assistance such as parking assistance is being performed. The program shown in the flowchart in FIG. 3 below is stored in RAM 32 or ROM 33 provided in driving assistance device 1, and is executed by CPU 31.

[0032] First, in step (hereinafter abbreviated as S) 1, the CPU 31 determines whether assisted driving by automatic driving assistance is being performed. In this embodiment, assisted driving by automatic driving assistance is performed when the user selects to perform automatic driving assistance by operating the operation unit 3 and it is determined that driving by automatic driving assistance is possible. The automatic driving assistance includes detecting the current position of the vehicle, the lane the vehicle is traveling on, and the positions of surrounding obstacles at any time, and automatically controlling the vehicle, including steering, drive source, and brakes, so that the vehicle travels along the generated travel trajectory at a speed in accordance with the generated speed plan. In particular, in the case of parking assistance, vehicle control up to parking into a parking space is performed automatically. However, it is also possible to automatically perform only steering operation, and manually control the drive source and brakes.

[0033] If it is determined that assisted driving by the autonomous driving assistance is being performed (S1: YES), the process proceeds to S2. On the other hand, if it is determined that assisted driving by the autonomous driving assistance is not being performed (S1: NO), the driving assistance processing program is terminated.

[0034] In S2, the CPU 31 generates a bird's-eye image looking diagonally downward from the sky and a bird's-eye image looking vertically downward from the sky of the vehicle's surroundings based on real-time captured images captured by the front camera 6, the rear camera 7, and the side cameras 8A and 8B. The bird's-eye image generation method will be described below as an example. As shown in FIG. 4, the real-time captured images captured by each camera are projected onto a virtual projection plane, which is a horizontal plane corresponding to the height of the ground surface. The captured images projected on the virtual projection plane are then converted into images viewed from a virtual viewpoint looking vertically downward from above the vehicle 2, thereby generating bird's-eye images of each camera. The conversion to the image viewed from the virtual viewpoint (viewpoint conversion) is performed by first converting each coordinate in the captured image coordinate system, which is set along a plane perpendicular to the optical axis of the camera, into each coordinate in a ground coordinate system set along the ground surface, and then converting it into each coordinate in the bird's-eye image coordinate system. The conversion formulas used for each coordinate conversion are already known, so a description thereof will be omitted. Then, as shown in FIG. 5 , overhead image 41 obtained by viewpoint-converting an image captured by front camera 6, overhead image 42 obtained by viewpoint-converting an image captured by rear camera 7, overhead image 43 obtained by viewpoint-converting an image captured by side camera 8A, and overhead image 44 obtained by viewpoint-converting an image captured by side camera 8B are synthesized (spliced ​​together), and an illustration image 45 that schematically depicts the vehicle is inserted between each of the overhead images 41 to 44 to generate an overhead image. Note that the process for generating the bird's-eye view image is essentially the same as that for generating the overhead image, except for the angle of the line of sight during viewpoint conversion, and therefore a detailed description is omitted. However, it is preferable that the virtual viewpoint for the bird's-eye view image be the interior of the vehicle (i.e., the viewpoint of the occupant). Furthermore, if the bird's-eye view image is a bird's-eye view image looking down diagonally ahead in the direction of travel, then overhead image 42 obtained by viewpoint-converting an image captured by rear camera 7 may be excluded from the synthesis target (i.e., an image of only the area ahead in the direction of travel from the vehicle position may be used). Conversely, if a bird's-eye view image looking diagonally down behind the vehicle when reversing is to be obtained, the bird's-eye view image 41 obtained by converting the viewpoint of the image captured by the front camera 6 may be excluded from the synthesis target (i.e., the image may be only behind the vehicle position).

[0035] Thereafter, in S3, the CPU 31 generates a wheel image 46 showing the wheels of the host vehicle. FIG. 6 is a diagram showing an example of the wheel image 46 generated in S3. The host vehicle has four wheels: a right front wheel, a left front wheel, a right rear wheel, and a left rear wheel. All four wheels have essentially the same appearance. For example, the example shown in FIG. 6 shows wheel images 46 of the right front wheel and the left front wheel. Basically, the wheel image 46 is an image of only the tire portion, excluding the wheel. However, the wheel may also be included. The axle may also be included. Meanwhile, the wheel image 46 includes at least a tread image corresponding to the tread portion of the wheel (the portion that comes into contact with the ground). As shown in FIG. 6, the tread image depicts a tread pattern 47. The wheel image 46 is a semi-transparent image (for example, a transmittance of 50%). However, the transmittance can be set appropriately, and may be 30% or 70%.

[0036] Here, information regarding the type and tread pattern of tires initially equipped on the vehicle is stored in the vehicle information DB 35, and the wheel image 46 is an image that faithfully reproduces the type and tread pattern of tires equipped on the vehicle using this information. If the tires have been changed from the initial equipment, the image may differ from the actual tires, but if information about the changed tires can be input, it is possible to create a wheel image 46 that faithfully reproduces the type and tread pattern of tires equipped on the vehicle even after the tires have been changed.

[0037] Thereafter, in S4, the CPU 31 displays a pattern image 48, which shows a pattern different from the tread pattern, on the surface of the wheel image generated in S3. The pattern image 48 may be any pattern as long as it is visible to the user. For example, as shown in FIG. 6, the pattern image 48 may be an arc-shaped line segment displayed in white along the circumferential direction on the inner side of the wheel image 46. The display position of the pattern image 48 does not necessarily have to be the inner side of the wheel image 46 as long as it is visible; for example, it may be on the outer side or tread surface. Furthermore, the display color is preferably white, which is easily visible to the user in contrast to the tire display color (i.e., black), but is not limited to white and may be yellow or red. Furthermore, the pattern image 48 may be a semi-transparent image like the wheel image 46, or it may be an opaque image.

[0038] FIG. 7 particularly illustrates a pattern image 48 displayed on the inner side of the wheel image 46. As shown in FIG. 7, the pattern image 48 of this embodiment is a pattern of arc-shaped line segments displayed in white along the circumferential direction and spaced at a predetermined distance apart. The length L of each line segment is determined based on the frame rate of the support image (see FIG. 9 ) displayed on the LCD display 4, which will be described later. Here, the support image, which includes the bird's-eye view image and overhead view image generated in S2 and the wheel image 46 generated in S3, as will be described later, is updated at a preset frame rate. The wheel image 46 included in the support image is rotated in accordance with the actual behavior of the host vehicle. For example, if the vehicle moves forward, the wheel image 46 also rotates in the forward direction, and if the vehicle moves backward, the wheel image 46 also rotates in the backward direction. Therefore, if the length L of the line segment is shorter than the amount of rotation of the wheel image 46 between frames (more specifically, the movement distance of the line segment between frames as the wheel image 46 rotates), the rotation direction of the pattern image 48 may be perceived as being different from the rotation direction of the wheel image 46 due to the stroboscopic effect. Therefore, the length L of the line segment is set to be longer than the maximum amount of rotation of the wheel image 46 between frames. Specifically, the maximum amount of rotation of the wheel image 46 is r × (ω / f), where ω [rad / s] is the maximum angular velocity when rotating the wheel image 46, f [fps] is the frame rate, and r [m] is the distance from the center of the wheel image to the line segment. For example, when the frame rate is 30 fps, it is desirable to set the length to about 1 / 3 of the tire circumference. Furthermore, the length L of the line segment of the pattern image 48 may be determined based on the tread pattern displayed on the wheel image 46. For example, the length L of the line segment may be set to be longer than the spacing between grooves in the tread pattern on the wheel rotation direction side.

[0039] Next, in S5, the CPU 31 combines the wheel image 46 (including the pattern image 48) generated in S3 with the bird's-eye image generated in S3. Specifically, the wheel image 46 is added to the bird's-eye image in an area where the wheels of the host vehicle are located. Fig. 8 is a diagram showing a bird's-eye image 52 combined with the wheel image 46.

[0040] The example shown in FIG. 8 illustrates a case where a wheel image 46 is superimposed on a bird's-eye view image 52 of the front side in the traveling direction of the vehicle, which is generated when the vehicle is moving forward. The wheel image 46 of the right front wheel and the wheel image 46 of the left front wheel are added to the areas corresponding to the right and left front wheels of the vehicle. The wheel image 46 of the right front wheel and the wheel image 46 of the left front wheel are bilaterally symmetrical and are basically identical images. In addition, in an area 55 surrounded by a dashed line in FIG. 8 between the wheel image 46 of the right front wheel and the wheel image 46 of the left front wheel, the body of the vehicle is located in this area, but an image of the body of the vehicle is not displayed in order to allow the wheel image 46 to be visually recognized, and an image of the road surface is displayed instead. It is desirable that the image of the road surface displayed in area 55 be an image that blends in with the surroundings. For example, it is possible to cut out and superimpose an image of the road surface captured by the front camera 6.

[0041] Although details are omitted, when the vehicle is reversing, the wheel image 46 of the right rear wheel and the wheel image 46 of the left rear wheel are similarly synthesized with the bird's-eye view image 52 of the rear of the vehicle. Note that the designs of the wheel image 46 of the front wheel and the wheel image 46 of the rear wheel may be different so that the front wheels and the rear wheels can be distinguished.

[0042] Thereafter, in S6, the CPU 31 displays the real-time bird's-eye view image and overhead view image, which are generated in S2 and show the current environment around the vehicle, on the liquid crystal display 4. The bird's-eye view image is the image that has been synthesized with the wheel image 46 in S5. In this embodiment, both the bird's-eye view image and the overhead view image are displayed simultaneously on the liquid crystal display 4, but it is also possible to display only one of them, or to display them in a switchable manner by a user operation.

[0043] FIG. 9 shows an example of the support image 51 displayed on the LCD display 4 in S6. As shown in FIG. 9, the support image 51 is divided into two screens, with a bird's-eye view image 52 displayed on the left and a bird's-eye view image 53 displayed on the right. Note that FIG. 9 particularly shows a case where the vehicle is moving forward, and bird's-eye view image 52 is displayed as a bird's-eye view looking diagonally down ahead in the direction of travel. Note that when the vehicle is reversing, bird's-eye view image 52 is displayed as a bird's-eye view looking diagonally down behind. As a result, the user can clearly understand the environment around the vehicle, including areas that are difficult to see directly, relative to the vehicle's position. Furthermore, the CPU 31 may calculate the vehicle's future movement trajectory based on detection values ​​from a vehicle speed sensor, a steering sensor, etc., and superimpose the movement trajectory on the bird's-eye view image 52 and the bird's-eye view image 53. Thereafter, the support image 51 continues to be displayed until assisted driving by autonomous driving assistance is terminated (S10: YES). It is updated at a preset frame rate, and real-time images of the vehicle's surroundings are displayed.

[0044] Next, in S7, the CPU 31 acquires the behavior of the host vehicle. Specifically, the current traveling direction and vehicle speed of the vehicle are identified and acquired based on the vehicle speed pulse output from the vehicle speed sensor and the shift position. The traveling direction and vehicle speed of the vehicle also correspond to the rotation direction and rotation speed of the wheels equipped on the vehicle. The turning angle of the vehicle may also be acquired using a steering sensor. The turning angle of the vehicle also corresponds to the angle of the wheels equipped on the vehicle.

[0045] Next, in S8, the CPU 31 determines whether the vehicle is moving forward or backward based on the vehicle behavior acquired in S7.

[0046] If it is determined that the vehicle is moving forward or backward (S8: YES), the process proceeds to S9. On the other hand, if it is determined that the vehicle is stopped (S8: NO), the process proceeds to S10.

[0047] In S9, the CPU 31 updates the wheel images 46 included in the support image 51 in accordance with the vehicle behavior. Specifically, the wheel images 46 displayed in the support image 51 are also rotated around the wheel axis (the center of the wheel images 46) in accordance with the rotation direction and rotation speed of the wheels of the actual vehicle. However, the rotation speed of the wheel images 46 does not necessarily have to be the same as the rotation speed of the actual wheels, and may be rotated at a speed that is a predetermined ratio (e.g., 1 / 2) of the actual rotation speed, for example. Alternatively, an upper limit may be set for the rotation speed. As the wheel images 46 rotate, the tread pattern 47 and pattern images 48 included in the tread image are also rotated together. When the wheel images 46 are rotated, they are updated at a preset frame rate.

[0048] As a result, for example, when the vehicle is moving forward, the wheel image 46 is rotated in the forward direction (toward the front of the screen) as shown in Fig. 10. In this case, if the user only refers to the tread pattern 47, there is a possibility that the user will not be able to correctly recognize the rotation direction of the wheel image 46 due to the strobe effect. However, in this embodiment, by referring to the pattern image 48 displayed on the inner side surface, the rotation direction of the wheel image 46 can be correctly recognized. As described in Fig. 7, the length of the line segments of the pattern image 48 is set in advance to prevent erroneous recognition of the rotation direction due to the strobe effect. Furthermore, because the wheel image 46 is displayed semi-transparently, the tread pattern 47 is not clearly visible to the user, and the influence of the strobe effect can be reduced.

[0049] When the vehicle is moving backward, the wheel image 46 is rotated in the backward direction. When the vehicle is moving backward, a bird's-eye view image 52 of the rear of the vehicle is displayed, which is the opposite direction to when the vehicle is moving forward, so that the rotation direction of the wheel image 46 included in the bird's-eye view image 52 is the same as when the vehicle is moving forward (the direction moving forward on the screen shown in FIG. 9).

[0050] Furthermore, if the turning angle of the vehicle is acquired as the behavior of the vehicle in S7, the angle (direction of travel) of the wheel image 46 displayed in the support image 51 may be changed to match the angle of the actual wheel of the vehicle.

[0051] Thereafter, in S10, the CPU 31 determines whether or not to end assisted driving by the automatic driving assistance. Here, the assisted driving by the automatic driving assistance may be ended, for example, on the condition that the user performs a predetermined end operation on the operation unit 3, or on the condition that the shift position is shifted to "P" or the engine is turned off. Alternatively, the end condition may be on the condition that the automatic driving assistance cannot be continued.

[0052] If it is determined that the assisted driving by the autonomous driving assistance has ended (S10: YES), the driving assistance processing program is terminated. On the other hand, if it is determined that the assisted driving by the autonomous driving assistance has not ended (S10: NO), the process returns to S7, and the display of the assistance image on the LCD display 4 continues.

[0053] As described above in detail, the driving assistance device 1 and the computer program executed by the driving assistance device 1 according to this embodiment acquire an image of the periphery of the vehicle (S2), generate a support image 51 by adding wheel images 46 representing the wheels of the vehicle to areas of the captured image where the wheels are located (S5), and display the generated support image 51 on the LCD display 4 (S6), while acquiring the behavior of the vehicle (S7), and rotating the wheel images 46 in the support image 51 around the wheel axis in accordance with the behavior of the vehicle (S9). Furthermore, the wheel image 46 includes a tread image corresponding to the tread portion of the wheel, and a pattern image 48 showing a pattern different from the tread pattern is displayed on the surface, and when the wheel image 46 is rotated, the pattern image 48 also rotates together with the wheel image 46. Therefore, even when the wheel image is rotated in accordance with the traveling of the vehicle, the user can correctly recognize the rotation direction of the wheel image. As a result, it is possible for the user to accurately understand the direction in which the displayed captured image was taken relative to the vehicle, without causing any sense of incongruity such as a discrepancy between the direction of travel and the direction of rotation of the wheel image. Furthermore, a tread pattern 47 is displayed on the tread image of the wheel image 46, and the pattern image 48 is displayed in white on the side of the wheel image 46, making it easier for the user to recognize the pattern image in contrast to the tire color. Furthermore, since the pattern image is displayed on the side of the wheel different from the tread portion where the tread pattern is displayed, even if a strobe effect occurs in the tread pattern, the pattern image allows the user to correctly recognize the rotation direction of the wheel image without being affected by this effect. Furthermore, the support image displayed on the liquid crystal display 4 is updated at a preset frame rate, and the pattern image 48 is a line segment displayed circumferentially on the side of the wheel image 46, with the length of the line segment being longer than the maximum amount of rotation of the wheel image 46 between frames, so that the pattern image can be designed so that a strobe effect does not occur due to the rotation of the wheel image. Furthermore, since the wheel image 46 is displayed in a semi-transparent state in the support image 51, the tread image is not clearly visible to the user, and the influence of the strobe effect can be reduced.

[0054] The present invention is not limited to the above-described embodiment, and it goes without saying that various improvements and modifications are possible within the scope of the present invention. For example, in this embodiment, as the landscape images around the vehicle, bird's-eye images and overhead images generated from images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B are displayed on the liquid crystal display 4, but it is also possible to display the image captured by the front camera 6 or rear camera 7 as is, rather than a processed image, on the liquid crystal display 4, and add the wheel image 46 to that image. Also, in this embodiment, the image to which the wheel image 46 is added is only a bird's-eye image, but the wheel image 46 may be added to an overhead image.

[0055] Furthermore, the scenery around the vehicle displayed on the liquid crystal display 4 may not be an image captured by a camera, but may be a schematic virtual scenery image (for example, a three-dimensional map image).

[0056] In addition, in this embodiment, it is assumed that the support image 51 shown in Figure 14 is displayed while driving is being performed using automatic driving assistance, but the support image 51 shown in Figure 9 may also be displayed while driving is being performed using manual driving.

[0057] In this embodiment, the driving assistance processing program (FIG. 3) is executed by the driving assistance ECU 10 of the driving assistance device 1, but the execution entity can be changed as appropriate. For example, the program may be executed by the control unit of the liquid crystal display 4, the vehicle control ECU, the control unit of the navigation device, or other in-vehicle device. [Explanation of symbols]

[0058] 1...driving assistance device, 2...vehicle, 3...operation unit, 4...liquid crystal display, 6...front camera, 7...rear camera, 8A, 8B...side cameras, 10...driving assistance ECU, 46...wheel image, 47...tread pattern, 48...pattern image, 51...assistance image, 52...bird's-eye view image, 53...bird's-eye view image

Claims

1. a captured image acquisition unit that acquires captured images of the surroundings of the vehicle; a support image generating unit that generates a support image by adding wheel images showing the wheels of the host vehicle to an area of ​​the captured image where the wheels are located; an image display unit that displays the support image on a display device; a behavior acquisition unit that acquires the behavior of the host vehicle; an image updating unit that rotates the wheel image in the support image around a wheel axis in accordance with the behavior of the vehicle; the wheel image includes a tread image corresponding to a tread portion of the wheel, and a pattern image showing a pattern different from the tread pattern is displayed on the surface; The driving assistance device wherein the image update unit rotates the pattern image together with the wheel image when rotating the wheel image.

2. The tread image displays a tread pattern, The driving assistance device according to claim 1 , wherein the pattern image is displayed in white on a side of the wheel image.

3. the support image displayed on the display device is updated at a preset frame rate; The pattern image is a line segment displayed along a circumferential direction on a side surface of the wheel image, The driving assistance device according to claim 1 , wherein the length of the line segment is set to be longer than a maximum value of the rotation amount of the wheel image between frames.

4. The driving support device according to claim 1 , wherein the wheel image is displayed in a semi-transparent manner in the support image.

Images