Peripheral monitoring device

Abstract

The present invention provides a surrounding monitoring device. It is possible to easily identify whether a detection unit is in an operating state based on an image displayed on a display unit. As an example, a surrounding monitoring device of an embodiment includes: an acquisition unit that acquires the current steering angle of a vehicle; an image acquisition unit that acquires a captured image from a capture unit that captures the surroundings of the vehicle; and a control unit that causes a composite image including a vehicle image representing the vehicle and a surrounding image representing the surroundings of the vehicle based on the captured image to be displayed on the display unit, and when the detection unit capable of detecting an object in contact with the vehicle is in an operating state, a virtual vehicle image representing the shape of the vehicle is overlapped with the position of the vehicle represented by the vehicle image in the composite image and displayed after the vehicle has traveled a specified distance from the current steering angle acquired by the acquisition unit, based on the position of the vehicle represented by the vehicle image in the composite image.

Description

Technical Field

[0001] The present invention relates to a perimeter monitoring device. Background Art

[0002] A technology has been developed that generates a composite image including a vehicle image and a peripheral image of the vehicle and its surroundings based on a captured image obtained by a camera unit capturing the surroundings of the vehicle, displays a display screen including the generated composite image on a display unit, and provides the driver with the surrounding conditions of the vehicle.

[0003] Patent Document 1: Japanese Patent No. 5529058

[0004] Patent Document 2: Japanese Patent No. 5605606

[0005] Patent Document 3: Japanese Patent No. 5182137

[0006] However, although the vehicle has a detection unit that detects an object that may come into contact with the vehicle, it is desired to be able to easily recognize whether the detection unit is in an operating state from a display screen displayed on the display unit. Summary of the invention

[0007] Therefore, as one of the subjects of the embodiment, it is to provide a surrounding monitoring device that can easily recognize whether a detection unit is in an operating state from an image displayed on a display unit.

[0008] As an example, the surrounding monitoring device of the embodiment includes: an acquisition unit that acquires the current steering angle of the vehicle; an image acquisition unit that acquires a captured image from a capture unit that captures the surroundings of the vehicle; and a control unit that causes a composite image including a vehicle image representing the vehicle and a surrounding image representing the surroundings of the vehicle based on the captured image to be displayed on a display unit, and when a detection unit capable of detecting an object in contact with the vehicle is in an operating state, a virtual vehicle image representing the shape of the vehicle is displayed in an overlapping manner on a position of the vehicle after the current steering angle acquired by the acquisition unit has traveled a predetermined distance, based on the position of the vehicle represented by the vehicle image in the composite image. Therefore, as an example, the driver of the vehicle can easily recognize whether the detection unit is in an operating state based on whether the virtual vehicle image is displayed on the display unit and the image displayed on the display unit.

[0009] In addition, in the surrounding monitoring device of the embodiment, as an example, when the object is detected by the detection unit, the control unit changes the display mode of the partial image in contact with the object in the virtual vehicle image, and stops the movement of the virtual vehicle image at the contact position between the vehicle and the object in the synthetic image. Therefore, as an example, since the driver of the vehicle can recognize the contact position of the vehicle body with the object from the virtual vehicle image and drive the vehicle, it is possible to easily avoid the contact between the vehicle and the detected object.

[0010] In addition, in the surrounding monitoring device of the embodiment, as an example, the virtual vehicle image is an image showing the shape of the vehicle composed of polygons, and the partial image is a polygon of a portion of the polygons constituting the virtual vehicle image that contacts the object. Therefore, as an example, since the driver of the vehicle can recognize the position of the vehicle body that contacts the object from the virtual vehicle image and drive the vehicle, it is possible to easily avoid the vehicle from contacting the detected object.

[0011] In addition, in the surrounding monitoring device of the embodiment, as an example, the control unit changes the display mode of the partial image according to the distance between the position of the object and the position of the vehicle represented by the virtual vehicle image. Therefore, as an example, by confirming the change in the display mode of the partial image, the positional relationship between the vehicle and the object can be grasped in more detail, so that the vehicle can be driven more easily to avoid contact between the vehicle and the detected object.

[0012] In addition, in the surrounding monitoring device of the embodiment, as an example, when the detection unit is in the operating state, the control unit displays a mark that can identify the direction in which the object approaches the vehicle relative to the traveling direction of the vehicle represented by the vehicle image in the synthetic image, and changes the display mode of the mark when the object approaching the vehicle is detected by the detection unit. Therefore, as an example, the driver of the vehicle can easily recognize whether the object that may contact the vehicle is approaching from a certain direction by visually confirming the approaching object mark whose display mode has been changed.

[0013] In addition, in the surrounding monitoring device of the embodiment, as an example, the control unit causes the virtual vehicle image to be superimposed and displayed at the contact position where the vehicle contacts the object, or at the position when the vehicle moves to the position before the contact, as the position after the vehicle moves a predetermined distance. Therefore, as an example, the driver of the vehicle can easily recognize at which position the vehicle contacts the object.

[0014] In addition, in the surrounding monitoring device of the embodiment, as an example, the vehicle image is an overhead image of the vehicle. Therefore, as an example, the positional relationship between the vehicle and the objects around it can be accurately grasped.

[0015] In addition, in the surrounding monitoring device of the embodiment, as an example, the virtual vehicle image is an image showing the three-dimensional shape of the vehicle. Therefore, as an example, a more realistic virtual vehicle image can be displayed on the display unit.

[0016] In addition, in the surrounding monitoring device of the embodiment, as an example, the virtual vehicle image is a semi-transparent image showing the shape of the vehicle. Therefore, as an example, the driver of the vehicle can easily distinguish the virtual vehicle image from the vehicle image and can intuitively recognize that the virtual vehicle image is an image showing the future position of the vehicle.

[0017] In addition, in the surrounding monitoring device of the embodiment, as an example, the virtual vehicle image is an image that emphasizes the outline of the vehicle. Therefore, as an example, the driver of the vehicle can easily recognize the future position of the vehicle based on the virtual vehicle image.

[0018] In addition, in the surrounding monitoring device of the embodiment, as an example, the virtual vehicle image is an image whose transmittance increases from the outline of the vehicle toward the inside. Therefore, as an example, the driver of the vehicle can easily recognize the future position of the vehicle based on the virtual vehicle image. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a perspective view showing an example of a state in which a part of a vehicle interior of a vehicle equipped with the surrounding monitoring device according to the first embodiment is viewed through perspective.

[0020] Figure 2 It is a plan view of an example of the vehicle according to the first embodiment.

[0021] Figure 3 This is a block diagram showing an example of the functional configuration of the vehicle according to the first embodiment.

[0022] Figure 4 This is a block diagram showing an example of the functional configuration of the ECU included in the vehicle according to the first embodiment.

[0023] Figure 5 It is a diagram showing a display example of a display screen displayed by the ECU included in the vehicle according to the first embodiment.

[0024] Figure 6 It is a diagram showing a display example of a display screen displayed by the ECU included in the vehicle according to the first embodiment.

[0025] Figure 7 This is a diagram for explaining an example of a display method of a virtual vehicle image displayed by the ECU included in the vehicle according to the first embodiment.

[0026] Figure 8 This is a diagram for explaining an example of a display method of a virtual vehicle image displayed by the ECU included in the vehicle according to the first embodiment.

[0027] Fig. 9 This is a diagram for explaining an example of a method for determining the colors of polygons constituting a virtual vehicle image by the ECU included in the vehicle according to the first embodiment.

[0028] Fig.10 This is a diagram showing an example of a display screen displayed by the ECU included in the vehicle according to the first embodiment.

[0029] Fig.11 This is a diagram for explaining an example of a process of emphasizing and displaying a partial image by the ECU included in the vehicle according to the first embodiment.

[0030] Fig.12 This is a diagram for explaining an example of a process of emphasizing and displaying a partial image by the ECU included in the vehicle according to the first embodiment.

[0031] Fig.13 This is a diagram showing an example of a display screen displayed by the ECU included in the vehicle according to the second embodiment.

[0032] Fig.14 This is a diagram showing an example of a display screen displayed by the ECU included in the vehicle according to the second embodiment.

[0033] Fig.15 This is a diagram showing an example of a display screen displayed by the ECU included in the vehicle according to the second embodiment.

[0034] Fig.16 This is a diagram showing an example of a display screen displayed by the ECU included in the vehicle according to the second embodiment.

[0035] Description of reference numerals:

[0036] 1…vehicle; 8…display device; 14…ECU; 14a…CPU; 14b…ROM; 14c…RAM; 14d…display control unit; 14f…SSD; 15…shooting unit; 16…radar; 17…sonar; 400…image acquisition unit; 401…acquisition unit; 402…detection unit; 403…control unit. DETAILED DESCRIPTION

[0037] The following discloses an exemplary embodiment of the present invention. The structure of the embodiment shown below and the action, result and effect brought about by the structure are an example. The present invention can be implemented by a structure other than the structure disclosed in the following embodiment, and at least one of the various effects and derived effects based on the basic structure can be obtained.

[0038] The vehicle equipped with the surrounding monitoring device of this embodiment can be a car that uses an internal combustion engine (engine) as a driving source (internal combustion engine car), a car that uses an electric motor (motor) as a driving source (electric car, fuel cell car, etc.), or a car that uses both an internal combustion engine and an electric motor as driving sources (hybrid electric car). In addition, the vehicle can be equipped with various devices (systems, components, etc.) required for driving various transmissions, internal combustion engines, and electric motors. In addition, the method, number, layout, etc. of the devices related to the driving of the wheels in the vehicle can be variously set.

[0039] (First Embodiment)

[0040] Figure 1 1 is a perspective view showing an example of a state in which a part of a vehicle compartment of a vehicle equipped with the surrounding monitoring device according to the first embodiment is viewed through a perspective view. Figure 1 As shown, the vehicle 1 includes a vehicle body 2, a steering control unit 4, an acceleration operating unit 5, a brake operating unit 6, a speed shift operating unit 7, and a monitoring device 11. The vehicle body 2 has a vehicle compartment 2a for passengers to ride in. In the vehicle compartment 2a, a steering control unit 4, an acceleration operating unit 5, a brake operating unit 6, a speed shift operating unit 7, etc. are provided in a state where a driver as a passenger sits in a seat 2b. The steering control unit 4 is, for example, a steering wheel highlighted from the instrument panel 24. The acceleration operating unit 5 is, for example, an accelerator pedal located at a position under the driver's feet. The brake operating unit 6 is, for example, a brake pedal located at a position under the driver's feet. The speed shift operating unit 7 is, for example, a shift lever highlighted from the center console.

[0041] The monitoring device 11 is, for example, provided at the center of the vehicle width direction (i.e., the left-right direction) of the instrument panel 24. The monitoring device 11 may also have functions such as a navigation system or an audio system. The monitoring device 11 includes a display device 8, a sound output device 9, and an operation input unit 10. In addition, the monitoring device 11 may also include various operation input units such as switches, dials, joysticks, and buttons.

[0042] The display device 8 is composed of LCD (Liquid Crystal Display), OELD (Organic Electroluminescent Display), etc., and can display various images based on image data. The sound output device 9 is composed of speakers, etc., and outputs various sounds based on sound data. The sound output device 9 can also be installed in a different position in the vehicle cabin 2a other than the monitoring device 11.

[0043] The operation input unit 10 is composed of a touch panel or the like, and the passenger can input various information. In addition, the operation input unit 10 is provided on the display screen of the display device 8, and can make the image displayed on the display device 8 transparent. Thus, the operation input unit 10 can make the passenger visually confirm the image displayed on the display screen of the display device 8. The operation input unit 10 receives the input of various information by the passenger by detecting the touch operation of the passenger on the display screen of the display device 8.

[0044] Figure 2 FIG. 1 is a top view of an example of a vehicle according to the first embodiment. Figure 1 as well as Figure 2 As shown, the vehicle 1 is a four-wheeled vehicle or the like, and has two left and right front wheels 3F and two left and right rear wheels 3R. All or part of the four wheels 3 are steerable.

[0045] The vehicle 1 is equipped with a plurality of photographing units 15 (on-board cameras). In the present embodiment, the vehicle 1 is equipped with, for example, four photographing units 15a to 15d. The photographing unit 15 is a digital camera having a photographing element such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor). The photographing unit 15 can photograph the surroundings of the vehicle 1 at a predetermined frame rate. Furthermore, the photographing unit 15 outputs a photographed image obtained by photographing the surroundings of the vehicle 1. The photographing unit 15 has a wide-angle lens or a fisheye lens, respectively, and can photograph, for example, a range of 140° to 220° in the horizontal direction. In addition, there is a case where the optical axis of the photographing unit 15 is set obliquely downward.

[0046] Specifically, the camera unit 15a is, for example, located at the end 2e on the rear side of the vehicle body 2 and is provided on the wall below the rear window of the door 2h of the rear compartment. Furthermore, the camera unit 15a can capture the area behind the vehicle 1 in the surroundings of the vehicle 1. The camera unit 15b is, for example, located at the end 2f on the right side of the vehicle body 2 and is provided on the right door mirror 2g. Furthermore, the camera unit 15b can capture the area on the side of the vehicle in the surroundings of the vehicle 1. The camera unit 15c is, for example, located at the front side of the vehicle body 2, that is, the end 2c on the front side in the front-rear direction of the vehicle 1, and is provided on the front bumper, the front grille, etc. Furthermore, the camera unit 15c can capture the area in front of the vehicle 1 in the surroundings of the vehicle 1. The camera unit 15d is, for example, located at the left side of the vehicle body 2, that is, the end 2d on the left side in the vehicle width direction, and is provided on the left door mirror 2g. Furthermore, the camera unit 15d can capture the area on the side of the vehicle 1 in the surroundings of the vehicle 1.

[0047] In addition, the vehicle 1 has a plurality of radars 16 that can measure the distance to an object existing outside the vehicle 1. The radar 16 is a millimeter wave radar or the like, and can measure the distance to an object existing in the traveling direction of the vehicle 1. In the present embodiment, the vehicle 1 has a plurality of radars 16a to 16d. The radar 16c is provided at the end portion on the right side of the front bumper of the vehicle 1, and can measure the distance to an object existing in front of the right side of the vehicle 1. In addition, the radar 16d is provided at the end portion on the left side of the front bumper of the vehicle 1, and can measure the distance to an object existing in front of the left side of the vehicle 1. The radar 16b is provided at the end portion on the right side of the rear bumper of the vehicle 1, and can measure the distance to an object existing behind the right side of the vehicle 1. In addition, the radar 16a is provided at the end portion on the left side of the rear bumper of the vehicle 1, and can measure the distance to an object existing behind the left side of the vehicle 1.

[0048] In addition, the vehicle 1 has a sonar 17 that can measure the distance to an external object that is close to the vehicle 1. In the present embodiment, the vehicle 1 has a plurality of sonars 17a to 17h. The sonars 17a to 17d are provided on the rear bumper of the vehicle 1 and can measure the distance to an object that is behind the vehicle. The sonars 17e to 17h are provided on the front bumper of the vehicle 1 and can measure the distance to an object that is in front of the vehicle 1.

[0049] Figure 3 FIG. 1 is a block diagram showing an example of the functional structure of the vehicle according to the first embodiment. Figure 3As shown, the vehicle 1 includes a steering system 13, a brake system 18, a steering angle sensor 19, an acceleration sensor 20, a shift sensor 21, a wheel speed sensor 22, a GPS (Global Positioning System) receiver 25, an in-vehicle network 23, and an ECU (Electronic Control Unit) 14. The monitoring device 11, the steering system 13, the radar 16, the sonar 17, the brake system 18, the steering angle sensor 19, the acceleration sensor 20, the shift sensor 21, the wheel speed sensor 22, the GPS receiver 25, and the ECU 14 are electrically connected via the in-vehicle network 23 as an electrical communication line. The in-vehicle network 23 is composed of a CAN (Controller Area Network) or the like.

[0050] The steering system 13 is an electric power steering system, a steer by wire (SBW) system, etc. The steering system 13 includes an actuator 13a and a torque sensor 13b. The steering system 13 is electrically controlled by the ECU 14, etc., so that the actuator 13a is actuated to apply torque to the steering unit 4 to compensate for the steering force, thereby steering the wheels 3. The torque sensor 13b detects the torque applied to the steering unit 4 by the driver, and sends the detection result to the ECU 14.

[0051] The brake system 18 includes an ABS (Anti-lock Brake System) for controlling the locking of the brakes of the vehicle 1, an anti-skid device (ESC: Electronic Stability Control) for suppressing the side slip of the vehicle 1 when turning, an electric brake system for enhancing the braking force auxiliary brake, and a brake-by-wire system (BBW: Brake By Wire). The brake system 18 has an actuator 18a and a brake sensor 18b. The brake system 18 is electrically controlled by the ECU 14 and the like, and applies a braking force to the wheel 3 via the actuator 18a. The brake system 18 detects the locking of the brake, the idling of the wheel 3, and the signs of side slip based on the rotation difference between the left and right wheels 3, and performs control to suppress the locking of the brake, the idling of the wheel 3, and the side slip. The brake sensor 18b is a displacement sensor that detects the position of the brake pedal, which is a movable part of the brake operating unit 6, and sends the detection result of the position of the brake pedal to the ECU 14.

[0052] The steering angle sensor 19 is a sensor for detecting the steering amount of the steering control unit 4 such as a steering wheel. In the present embodiment, the steering angle sensor 19 is composed of a Hall element or the like, detects the rotation angle of the rotating part of the steering control unit 4 as the steering amount, and sends the detection result to the ECU 14. The acceleration sensor 20 is a displacement sensor for detecting the position of the accelerator pedal as the movable part of the acceleration operation unit 5, and sends the detection result to the ECU 14. The GPS receiver 25 acquires the current position of the vehicle 1 based on the radio waves received from the artificial satellite.

[0053] The shift sensor 21 is a sensor that detects the position of the movable part (lever, arm, button, etc.) of the shift operating unit 7, and sends the detection result to the ECU 14. The wheel speed sensor 22 is a sensor that has a Hall element, etc., and detects the rotation amount of the wheel 3 and the rotation speed of the wheel 3 per unit time, and sends the detection result to the ECU 14.

[0054] The ECU 14 is composed of a computer and the like, and manages the overall control of the vehicle 1 through the cooperation of hardware and software. Specifically, the ECU 14 includes a CPU (Central Processing Unit) 14a, a ROM (Read only Memory) 14b, a RAM (Random Access Memory) 14c, a display control unit 14d, a sound control unit 14e, and an SSD (Solid State Drive) 14f. The CPU 14a, the ROM 14b, and the RAM 14c may be provided in the same circuit board.

[0055] The CPU 14a reads a program stored in a nonvolatile storage device such as the ROM 14b and performs various calculations according to the program. For example, the CPU 14a performs image processing on image data displayed on the display device 8 and controls the vehicle 1 to travel along a target route to a target position such as a parking position.

[0056] ROM14b stores various programs and parameters required for executing the programs. RAM14c temporarily stores various data used in the operation of CPU14a. Display control unit 14d mainly performs image processing on image data obtained from camera 15 and output to CPU14a, conversion of image data obtained from CPU14a into image data for display displayed on display device 8, etc. in the operation of ECU14. Sound control unit 14e mainly performs processing of sound obtained from CPU14a and output to sound output device 9 in the operation of ECU14. SSD14f is a rewritable non-volatile storage unit, which continues to store data obtained from CPU14a even after the power of ECU14 is turned off.

[0057] Figure 4 FIG. 1 is a block diagram showing an example of the functional structure of the ECU included in the vehicle of the first embodiment. Figure 4 As shown, the ECU 14 includes an image acquisition unit 400, an acquisition unit 401, a detection unit 402, and a control unit 403. For example, the ECU 14 implements the functions of the image acquisition unit 400, the acquisition unit 401, the detection unit 402, and the control unit 403 by a processor such as a CPU 14a mounted on a circuit board executing a peripheral monitoring program stored in a storage medium such as a ROM 14b or an SSD 14f. A part or all of the image acquisition unit 400, the acquisition unit 401, the detection unit 402, and the control unit 403 may also be configured by hardware such as a circuit.

[0058] The image acquisition unit 400 acquires an image of the surroundings of the vehicle 1 captured by the image capture unit 15. The acquisition unit 401 acquires the current steering angle of the vehicle 1. In this embodiment, the acquisition unit 401 acquires the steering amount detected by the steering angle sensor 19 as the current steering angle of the vehicle 1.

[0059] The detection unit 402 can detect an object that is in contact with the vehicle 1. In the present embodiment, the detection unit 402 detects an object that is likely to be in contact with the vehicle 1 based on a captured image obtained by capturing the moving direction of the vehicle 1 by the camera 15, a distance measured by the radar 16 (the distance between the vehicle 1 and an object that is present in the moving direction of the vehicle 1), and the like. In addition, in the present embodiment, the detection unit 402 detects both a stationary object that is likely to be in contact with the vehicle 1 and a moving object that is approaching the vehicle 1 and is likely to be in contact with the vehicle 1 as objects that are likely to be in contact with the vehicle 1.

[0060] For example, the detection unit 402 detects an object that may come into contact with the vehicle 1 by performing image processing (e.g., optical flow) on the captured image obtained by the imaging unit 15. Alternatively, the detection unit 402 detects an object that may come into contact with the vehicle 1 based on a change in the distance measured by the radar 16.

[0061] In this embodiment, the detection unit 402 detects an object that may come into contact with the vehicle 1 based on the captured image obtained by the shooting unit 15 or the measurement result of the distance measured by the radar 16, but when detecting an object that is at a relatively close distance to the vehicle 1, the object that may come into contact with the vehicle 1 can also be detected based on the measurement result of the distance measured by the sonar 17.

[0062] In addition, in the present embodiment, the detection unit 402 is converted to an operating state (on) or a non-operating state (off) in response to the operation of a main switch (not shown) of the vehicle 1. Here, the operating state is a state in which an object in contact with the vehicle 1 is detected. On the other hand, the non-operating state is a state in which an object in contact with the vehicle 1 is not detected.

[0063] In the present embodiment, the detection unit 402 is converted to an operating state or a non-operating state in accordance with the operation of the main switch, but the present invention is not limited thereto. For example, when the speed of the vehicle 1 based on the detection result of the rotation speed of the wheel 3 by the wheel speed sensor 22 becomes less than a preset speed (e.g., 12 km / h), the detection unit 402 may be automatically (independent of the operation of the main switch) converted to an operating state. In addition, when the speed of the vehicle 1 is faster than the preset speed, the detection unit 402 may be automatically (independent of the operation of the main switch) converted to a non-operating state.

[0064] The control unit 403 causes the display screen to be displayed on the display device 8 via the display control unit 14d, and the display screen includes a captured image obtained by the camera unit 15 capturing the traveling direction of the vehicle 1, and a composite image including the vehicle image and the surrounding image. In addition, in the present embodiment, the control unit 403 causes the display screen including the composite image and the captured image to be displayed on the display device 8, but it is sufficient to cause at least the display screen including the composite image to be displayed on the display device 8. Therefore, for example, the control unit 403 may cause the display screen including the composite image to be displayed on the display device 8 without including the captured image.

[0065] Here, the vehicle image is an image representing the vehicle 1. In the present embodiment, the vehicle image is a bird's-eye view image obtained by observing the vehicle 1 from above. Thus, the positional relationship between the vehicle 1 and its surrounding objects can be correctly grasped. In addition, in the present embodiment, the vehicle image can be an image in the form of a bitmap, or an image representing the shape of the vehicle composed of a plurality of polygons. Here, the vehicle image composed of a plurality of polygons refers to the shape of the three-dimensional vehicle 1 expressed by a plurality of polygons (in the present embodiment, triangular polygons).

[0066] The surrounding image is an image showing the surroundings (surroundings) of the vehicle 1, and is generated based on a captured image obtained by capturing the surroundings of the vehicle 1 by the camera 15. In the present embodiment, the surrounding image is an overhead image obtained by observing the surroundings (surroundings) of the vehicle 1 from above. In the present embodiment, the surrounding image is an overhead image of the surroundings of the vehicle 1 centered on the center of the rear wheel axle of the vehicle image.

[0067] In addition, when the detection unit 402 is in an operating state, the control unit 403 uses the position of the vehicle 1 represented by the vehicle image in the synthetic image as a reference, and displays the virtual vehicle image superimposed on the position of the vehicle 1 after the vehicle 1 has traveled a predetermined distance at the current steering angle acquired by the acquisition unit 401. On the other hand, when the detection unit 402 is in an inoperative state, the control unit 403 does not display the virtual vehicle image. Thus, the driver of the vehicle 1 can easily recognize whether the detection unit 402 is in an operating state based on whether the virtual vehicle image is included in the display screen displayed on the display device 8, according to the display screen displayed on the display device 8.

[0068] Here, the predetermined distance is a distance that has been set in advance, for example, 1.0 to 2.0 m. In addition, the current steering angle is the steering angle of the vehicle 1 at the current position. In this embodiment, the control unit 403 obtains the steering angle obtained by the acquisition unit 401 as the steering angle of the vehicle 1 at the current position.

[0069] In addition, the virtual vehicle image is a virtual image showing the shape of the vehicle 1. In the present embodiment, the virtual vehicle image is an image showing the shape of the vehicle 1 composed of a plurality of polygons. Here, the virtual vehicle image composed of a plurality of polygons refers to the shape of the three-dimensional vehicle 1 (the three-dimensional shape of the vehicle 1) displayed by a plurality of polygons (in the present embodiment, triangular polygons). Thus, a more realistic virtual vehicle image can be displayed on the display device 8.

[0070] In this embodiment, although the control unit 403 includes an image representing the shape of the vehicle 1 composed of multiple polygons as a virtual vehicle image in the synthesized image, for example, an image representing the shape of the vehicle 1 in bitmap form may be included in the synthesized image as a virtual vehicle image.

[0071] In this embodiment, when the detection unit 402 is in operation and the shift sensor 21 detects that the position of the speed change operation unit 7 is in the D range, the control unit 403 displays a virtual vehicle image in front of the vehicle 1. This notifies the driver that an object approaching from the front of the vehicle 1 may be detected.

[0072] On the other hand, when the detection unit 402 is in operation and the shift sensor 21 detects that the position of the speed change operation unit 7 is in the R range, the control unit 403 displays a virtual vehicle image behind the vehicle 1. This notifies the driver that an object approaching from the rear of the vehicle 1 may be detected.

[0073] In addition, when the detection unit 402 does not detect an object in contact with the vehicle 1, the control unit 403 continues to use the position of the vehicle 1 represented by the vehicle image in the synthetic image as a reference, and displays the virtual vehicle image in a superimposed manner at a position after the vehicle 1 has traveled a predetermined distance at the current steering angle. That is, when the detection unit 402 does not detect an object in contact with the vehicle 1, the control unit 403 moves the position of the virtual vehicle image in the synthetic image as the vehicle 1 moves.

[0074] On the other hand, when the detection unit 402 detects an object in contact with the vehicle 1, the control unit 403 changes the display mode of the image of the portion in contact with the detected object (hereinafter referred to as the partial image) in the virtual vehicle image. As a result, the driver of the vehicle 1 can recognize the position in contact with the object in the body of the vehicle 1 from the virtual vehicle image and drive the vehicle 1, so that the contact between the vehicle 1 and the detected object can be easily avoided.

[0075] In this embodiment, the control unit 403 flashes the partial image, changes the color, or emphasizes the outline of the partial image, so that the partial image is displayed differently from other parts of the virtual vehicle image. In addition, in this embodiment, when the virtual vehicle image is composed of polygons, the control unit 403 determines the polygons of the polygons constituting the virtual vehicle image that are in contact with the object as the partial image. Then, the control unit 403 changes the display method of the determined polygons.

[0076] In addition, when an object in contact with the vehicle 1 is detected, the control unit 403 moves the virtual vehicle image to the contact position where the vehicle 1 and the object are in contact within the synthetic image, and then does not move the virtual vehicle image from the contact position. In the present embodiment, when an object in contact with the vehicle 1 is detected, the control unit 403 does not move the virtual vehicle image from the contact position where the vehicle 1 and the object are in contact within the synthetic image to fix it, but the present invention is not limited thereto. For example, the control unit 403 may also stop the movement of the virtual vehicle image at a position near the contact position, and does not move the virtual vehicle image from the position to fix it. That is, the control unit 403 displays the virtual vehicle image in an overlapping manner at the contact position where the vehicle 1 and the object are in contact, or at the position before the contact, as the position after the vehicle 1 has traveled a predetermined distance. As a result, the driver of the vehicle 1 can easily recognize at which position the vehicle 1 is in contact with the object.

[0077] In addition, in the present embodiment, after the object in contact with the vehicle 1 is detected and the control unit 403 fixes the virtual vehicle image at the contact position, the driver of the vehicle 1 performs steering operation on the steering control unit 4 to change the traveling direction of the vehicle 1, and when the object in contact with the vehicle 1 is no longer detected by the detection unit 402, the control unit 403 releases the fixation of the virtual vehicle image at the contact position. Furthermore, the control unit 403 moves the position of the virtual vehicle image in the synthetic image again as the vehicle 1 moves.

[0078] Furthermore, when the detection unit 402 is in the operating state, the control unit 403 displays the approaching object mark for the traveling direction of the vehicle 1 represented by the vehicle image in the composite image. Thus, the driver of the vehicle 1 can easily recognize whether the detection unit 402 is in the operating state based on whether the approaching object mark is included in the display screen displayed on the display device 8 and the display screen displayed on the display device 8.

[0079] Here, the approaching object marker is a marker that can identify the direction in which the object is approaching the vehicle 1 (hereinafter referred to as the approaching direction). In this embodiment, the approaching object marker is an arrow indicating the approaching direction. In addition, in this embodiment, the approaching object marker is a marker that can identify the approaching direction of a moving body among objects that may come into contact with the vehicle 1.

[0080] Furthermore, when the detection unit 402 detects an object approaching the vehicle 1, the control unit 403 changes the display mode of the approaching object indicator. Thus, the driver of the vehicle 1 can easily recognize whether an object that may contact the vehicle 1 is approaching from a certain direction by visually confirming the approaching object indicator with a changed display mode. In this embodiment, the control unit 403 changes the color of the approaching object indicator or makes it flash so that the display mode of the approaching object indicator is different from the display mode of the approaching object indicator when the object approaching the vehicle 1 is not detected.

[0081] In this embodiment, when a stationary object is detected as an object that may contact the vehicle 1, the control unit 403 changes the display mode of a portion of the image in the virtual vehicle image, and when a moving object is detected as an object that may contact the vehicle 1, the control unit 403 changes the display mode of the approaching object mark. However, the display mode of a portion of the image in the virtual vehicle image may be changed when a moving object is detected as an object that may contact the vehicle 1. In this case, the control unit 403 may include the approaching object mark in the synthesized image, or may not include the approaching object mark in the synthesized image.

[0082] Next, use Figures 5 to 12 A specific example of the display screen displayed on the display device 8 by the control unit 403 will be described.

[0083] Figure 51 is a diagram showing an example of a display screen displayed by the ECU of the vehicle of the first embodiment. Here, the display processing of the display screen when the position of the speed change operation unit 7 is detected by the shift sensor 21 to be in the D gear is described. In this embodiment, Figure 5 As shown, the control unit 403 displays a display screen G on the display device 8, wherein the display screen G includes a composite image G3 including a vehicle image G1 and a peripheral image G2, and a captured image G4 obtained by capturing the traveling direction of the vehicle 1 (eg, the front of the vehicle 1) by the camera 15.

[0084] Furthermore, when the detection unit 402 is in an operating state, as shown in FIG. Figure 5 As shown, the control unit 403 uses the position of the vehicle 1 represented by the vehicle image G1 in the surrounding image G2 as a reference, and causes the virtual vehicle image G5 to be superimposed and displayed at the position P2 after the vehicle 1 has traveled a predetermined distance at the steering angle of the current position P1 (the steering angle acquired by the acquisition unit 401). Figure 5 As shown, the virtual vehicle image G5 is a semi-transparent image showing the shape of the vehicle 1. Thus, the driver of the vehicle 1 can easily distinguish the virtual vehicle image G5 from the vehicle image G1 and can intuitively recognize that the virtual vehicle image G5 is an image showing the future position P2 of the vehicle 1.

[0085] In addition, in the present embodiment, the control unit 403 displays an image whose transmittance increases as it moves from the outline of the vehicle 1 toward the inside as the virtual vehicle image G5. Thus, the driver of the vehicle 1 can easily distinguish the virtual vehicle image G5 from the vehicle image G1, and can easily and more intuitively recognize that the virtual vehicle image G5 is an image indicating the future position of the vehicle 1.

[0086] Furthermore, the control unit 403 may also display the outline of the virtual vehicle image G5 in a different display mode (e.g., different color, flashing, overlapping of frame lines) from other parts of the virtual vehicle image G5 to emphasize the outline. Thus, the driver of the vehicle 1 can easily recognize the future position of the vehicle 1 based on the virtual vehicle image G5.

[0087] In addition, when the detection unit 402 is in an operating state, as shown in FIG. Figure 5 As shown, the control unit 403 displays the approaching object marker G6 in the surrounding image G2 at a predetermined position (e.g., the right and left sides of the virtual vehicle image G5) located in the traveling direction of the vehicle 1 (e.g., in front of the vehicle 1) and based on the position of the virtual vehicle image G5. At this time, in the present embodiment, the control unit 403 displays the approaching object marker G6 in grayscale.

[0088] Figure 61 is a diagram showing an example of a display screen displayed by the ECU of the vehicle of the first embodiment. In this embodiment, when the object in contact with the vehicle 1 is not detected by the detection unit 402, as shown in FIG. Figure 5 As shown in FIG. 1 , as the vehicle 1 moves, the control unit 403 also moves the position of the virtual vehicle image G5 in the surrounding image G2. On the other hand, when the detection unit 402 detects an object O (such as a wall or a wall base) in contact with the vehicle 1, as shown in FIG. Figure 6 As shown in FIG. 4 , the control unit 403 moves the virtual vehicle image G5 to the contact position P3 where the vehicle 1 contacts the detected object O in the surrounding image G2. Figure 6 As shown, the control unit 403 does not move the virtual vehicle image G5 but keeps it fixed even if the vehicle 1 moves, starting from the contact position P3.

[0089] At this time, if Figure 6 As shown, the control unit 403 makes the display mode of the partial image PG in the virtual vehicle image G5 that contacts the detected object O different from the display mode of other parts of the virtual vehicle image G5. For example, the control unit 403 displays the partial image PG in red and displays the parts other than the partial image PG in the virtual vehicle image G5 in white. As a result, since the driver of the vehicle 1 can understand the position in the body 2 of the vehicle 1 that contacts the detected object O when the vehicle 1 is driven at the current steering angle, it is easier to drive the vehicle 1 in a manner that the vehicle 1 does not contact the detected object O.

[0090] In addition, in the present embodiment, the control unit 403 may also change the display mode of the partial image PG in accordance with the distance between the position of the detected object O and the current position P1 of the vehicle 1 represented by the virtual vehicle image G5. Thus, by confirming the change in the display mode of the partial image PG, the positional relationship between the vehicle 1 and the object O can be grasped in more detail, so that the vehicle 1 can be driven more easily in a manner that the vehicle 1 and the detected object O do not contact each other. Specifically, as the distance between the position of the detected object O and the current position P1 of the vehicle 1 represented by the virtual vehicle image G5 decreases, the control unit 403 increases the degree of red of the partial image PG displayed in red, or causes the partial image PG to flash, thereby emphasizing the display of the partial image PG.

[0091] On the other hand, as the distance between the position of the detected object O and the current position P1 of the vehicle 1 represented by the virtual vehicle image G5 increases, the control unit 403 reduces the degree of redness of the partial image PG or extends the interval of flashing of the partial image PG to cancel the emphasized display of the partial image PG. After that, the driver of the vehicle 1 performs a steering operation on the steering control unit 4 to change the direction of travel of the vehicle 1. When the object O in contact with the vehicle 1 is no longer detected by the detection unit 402, the control unit 403 returns the display mode of the partial image PG to the same display mode as the other parts of the virtual vehicle image G5. In addition, the control unit 403 cancels the fixation of the virtual vehicle image G5 at the contact position P3, and moves the position of the virtual vehicle image G5 in the synthetic image G3 again as the vehicle 1 moves.

[0092] Here, since it is assumed that the object O detected by the detection unit 402 is a stationary object such as a wall or a base of a wall, the control unit 403 changes the display mode of the partial image PG in the virtual vehicle image G5, but does not change the display mode of the approaching object indicator G6. Thus, the driver of the vehicle 1 can recognize whether the object detected by the detection unit 402 is a stationary object or a moving object approaching the vehicle 1.

[0093] In the present embodiment, the control unit 403 displays the approaching object marker G6 in a gray scale, which is the display mode of the approaching object marker G6 when the detection unit 402 does not detect a moving body that may come into contact with the vehicle 1. However, when the object detected by the detection unit 402 is a moving body such as another vehicle or a pedestrian, the control unit 403 changes the display mode of the approaching object marker G6 included in the surrounding image G2, which is located in the direction in which the detected moving body is detected. At this time, the control unit 403 may also change the display mode of the approaching object marker G6, and also change the display mode of the partial image PG in the virtual vehicle image G5 that comes into contact with the detected moving body.

[0094] In addition, when the object detected by the detection unit 402 is a moving body approaching the vehicle 1, as described above, the control unit 403 changes the display mode of the approaching object marker G6 that exists in the direction in which the detected object is approaching. For example, the control unit 403 changes the color of the approaching object marker G6 that exists in the direction in which the detected object is approaching to yellow or flashes the approaching object marker G6. Alternatively, when each approaching object marker G6 includes a plurality of arrows, the control unit 403 displays the plurality of arrows included in the approaching object marker G6 displayed in the direction in which the detected moving body exists, in an animation that sequentially changes the display mode starting from the arrow that is away from the virtual vehicle image G5.

[0095] Figure 7 as well as Figure 8 FIG. 1 is a diagram for explaining an example of a method for displaying a virtual vehicle image displayed by an ECU included in a vehicle according to the first embodiment. Figure 7 In FIG. 1 , the X axis corresponds to the width direction of the vehicle 1, the Z axis corresponds to the traveling direction of the vehicle 1, and the Y axis corresponds to the height direction of the vehicle 1. When the virtual vehicle image G5 is composed of a plurality of polygons PL, as shown in FIG. Figure 7 as well as Figure 8 As shown, the control unit 403 obtains the value of the normal vector n of the vertices V1, V2, and V3 of each polygon PL in the Y-axis direction (the direction perpendicular to the road surface) (hereinafter referred to as the Y component). Furthermore, the control unit 403 determines the transmittance of the polygon PL based on the Y component of the normal vector n.

[0096] Specifically, the control unit 403 obtains the Y component of the normal vector n of the vertices V1, V2, and V3 of the polygon PL. Next, the control unit 403 determines the pixels contained in the polygon PL based on the Y component of the normal vector n of the vertices V1, V2, and V3. At this time, the control unit 403 increases the transmittance as the Y component of the normal vector n increases. As a result, the control unit 403 can display an image that is transmitted from the outline of the vehicle 1 toward the inside as a virtual vehicle image G5. In this embodiment, the color of the pixels constituting the polygon PL is white, but it is not limited to this. For example, it can also be any color such as the color of the body of the vehicle 1.

[0097] Fig. 9 FIG. 1 is a diagram for explaining an example of a method for determining the color of a polygon constituting a virtual vehicle image by the ECU of the vehicle according to the first embodiment. Fig. 9 In the figure, the horizontal axis represents the type of color (e.g., RGB) of the vertices of the polygons constituting the virtual vehicle image, and the vertical axis represents the value of the color (e.g., RGB value) of the vertices of the polygons constituting the virtual vehicle image. In the present embodiment, when the detection unit 402 detects an object in contact with the vehicle 1, the control unit 403 makes the display mode of the portion of the virtual vehicle image in contact with the detected object different from the display mode of the other portions of the virtual vehicle image.

[0098] Specifically, when the object in contact with the vehicle 1 is not detected by the detection unit 402, as shown in FIG. Fig. 9 As shown in FIG. 4 , the control unit 403 makes the values ​​of the RGB colors of the vertices of the polygon constituting the virtual vehicle image equal. Furthermore, the control unit 403 interpolates the colors of the areas within the polygons enclosed by the vertices by linear interpolation or the like based on the values ​​of the RGB colors of the vertices of the polygons to determine the color of the entire polygon. Thus, the control unit 403 displays the virtual vehicle image in white.

[0099] On the other hand, when the object in contact with the vehicle 1 is detected by the detection unit 402, as shown in FIG. Fig. 9 As shown in FIG. 1 , the control unit 403 makes the value of GB of each vertex of the polygon constituting the partial image among the polygons constituting the virtual vehicle image smaller than the value of R of each vertex. At this time, the control unit 403 also interpolates the color of the area within the polygon enclosed by each vertex by linear interpolation or the like based on the value of each RGB color of each vertex of the polygon, and determines the color of the entire polygon. As a result, the control unit 403 displays the partial image in red.

[0100] In the present embodiment, the control unit 403 displays the partial image in red and emphasizes the display of the partial image by making the value of GB of each vertex of the polygon constituting the partial image smaller than the value of R of each vertex, but the present invention is not limited to this. For example, the control unit 403 may also display the partial image in green and emphasize the display of the partial image by making the value of RB of each vertex of the polygon constituting the partial image smaller than the value of G of each vertex.

[0101] At this time, the control unit 403 may also reduce the GB value of each vertex of the polygon constituting the partial image as the distance between the position of the detected object and the position of the vehicle 1 represented by the virtual vehicle image shortens. Thus, the control unit 403 increases the degree of redness of the partial image and emphasizes the display of the partial image. Thus, since the part of the body of the vehicle 1 that contacts the external object can be easily identified, it is easy for the driver of the vehicle 1 to avoid contact with the external object. In addition, the control unit 403 maintains the state in which the RGB values ​​of each vertex of the polygon other than the partial image among the polygons constituting the virtual vehicle image are equal. Thus, the control unit 403 displays the polygon other than the partial image in white.

[0102] Fig.10 FIG. 1 is a diagram showing an example of a display screen displayed by the ECU of the vehicle according to the first embodiment. Fig.10 As shown, at time t1, when the object O (stationary object) in contact with the vehicle 1 is detected by the detection unit 402, the control unit 403 highlights the partial image PG in contact with the detected object O in the virtual vehicle image G5 in red.

[0103] Specifically, the control unit 403 obtains the XZ plane parallel to the road surface (refer to Figure 7) is the Euclidean distance from each vertex of the polygon constituting the partial image PG to the object O. Next, the control unit 403 makes the value of GB of each vertex smaller than the value of R of each vertex, corresponding to the Euclidean distance between each vertex of the polygon constituting the partial image PG and the object O. Moreover, the control unit 403 determines the color of the pixels contained in the polygon constituting the partial image PG through the fragment shader based on the RGB value of each vertex of the polygon constituting the partial image PG. In addition, the control unit 403 interpolates the color of the area within the polygon enclosed by each vertex through linear interpolation or the like based on the RGB color values ​​of each vertex of the polygon. Thus, the control unit 403 displays the partial image PG in red. In addition, when calculating the RGB value of the polygon constituting the virtual vehicle image, the RGB value of the polygon constituting the partial image can also be calculated simultaneously corresponding to the distance between each vertex of the polygon and the object. Thus, the virtual vehicle image and the partial image can be generated without being separated.

[0104] Afterwards, the vehicle 1 continues to move, and at time t2 after time t1, when the virtual vehicle image PG reaches the contact position P3 with the object O, or the position that is about to be the contact position P3, as shown in FIG. Fig.10 As shown, the control unit 403 does not move the virtual vehicle image PG from the contact position P3, and keeps displaying the virtual vehicle image PG at the contact position P3.

[0105] Furthermore, after time t2 and before time t3, the steering angle of the vehicle 1 is changed, and when the vehicle 1 is no longer likely to contact the object O (that is, when the object O is no longer detected by the detection unit 402), as shown in FIG. Fig.10 As shown, the control unit 403 releases the fixation of the virtual vehicle image PG at the contact position P3 and displays the virtual vehicle image G5 at a position P2 after the vehicle 1 has traveled a predetermined distance based on the position of the vehicle 1 indicated by the vehicle image G1 at time t3.

[0106] At this time, the control unit 403 may also cancel the highlighted display of the partial image PG in the virtual vehicle image G5 in red. That is, the control unit 403 returns the display mode of the partial image PG in the virtual vehicle image G5 at time t3 to the same display mode as the other parts of the virtual vehicle image G5. Thus, the driver of the vehicle 1 can recognize that the current steering angle of the vehicle 1 can avoid contact with the object 901.

[0107] Fig.11 as well as Fig.12 4 is a diagram for explaining an example of processing of emphasizing a portion of an image by the ECU of the vehicle of the first embodiment. In this embodiment, if the object O is detected by the detection unit 402, Fig.11As shown, the control unit 403 first obtains the coordinates of the XZ plane 1100 ( Figure 7 The control unit 403 finds the point V′ at which a perpendicular line 1101 to a plane defined by the X-axis and the Z-axis intersects the XZ plane 1100. Next, the control unit 403 obtains the Euclidean distance L between the point V′ and the position of the object O in the XZ plane 1100.

[0108] Next, the control unit 403 Fig.12 The intensity distribution 1200 shown determines the degree of emphasis display corresponding to the obtained Euclidean distance L. Here, the intensity distribution 1200 is a distribution of the degree of emphasis display when emphasizing a part of the image, and as the Euclidean distance L decreases, the degree of emphasis display increases.

[0109] In the present embodiment, the intensity distribution 1200 is a concentric intensity distribution, with the position of the object O as the center, and the degree of emphasis is reduced slightly as the Euclidean distance L becomes farther. In the present embodiment, the intensity distribution 1200 is represented by a high-order curve, and when the Euclidean distance L becomes less than a preset distance (e.g., 1.7 to 3.0 m), the degree of emphasis increases sharply. For example, the intensity distribution 1200 is an intensity distribution in which the value of GB drops sharply and R is emphasized when the Euclidean distance L becomes less than a preset distance.

[0110] Therefore, if Fig.12 As shown in FIG. 4 , among the polygons constituting the virtual vehicle image G5, the polygons whose Euclidean distance L is closer to the object O are highlighted in red by the control unit 403. As a result, Fig.12 As shown, the control unit 403 can highlight the polygons constituting the partial image PG among the polygons constituting the virtual vehicle image G5 in red.

[0111] Thus, according to the vehicle 1 of the first embodiment, the driver of the vehicle 1 can easily recognize whether the detection unit 402 is in operation based on whether the virtual vehicle image is included in the display screen displayed on the display device 8 .

[0112] (Second Embodiment)

[0113] This embodiment is an example in which a display screen including a three-dimensional image of the periphery of the vehicle is displayed on a display device instead of an image captured by an image capturing unit in the direction of travel of the vehicle. In the following description, the same configuration as that of the first embodiment is omitted.

[0114] Fig.13 as well as Fig.14FIG. 4 is a diagram showing an example of a display screen displayed by the ECU of the vehicle according to the second embodiment. In the present embodiment, when the detection unit 402 is in a non-operating state, as shown in FIG. Fig.13 As shown, the control unit 403 causes the display screen G including the synthesized image G3 and the three-dimensional image G7 of the vehicle 1 and its surroundings (hereinafter referred to as the three-dimensional surrounding image) to be displayed on the display device 8. Thus, in addition to the synthesized image G3, the three-dimensional surrounding image G7 is also visually confirmed, so that the positional relationship between the vehicle 1 and the objects in its surroundings can be grasped in more detail.

[0115] Here, as described above, the three-dimensional surrounding image G7 is a three-dimensional image of the vehicle 1 and its surroundings. In the present embodiment, the three-dimensional surrounding image G7 is an image obtained by pasting an image of the surroundings of the vehicle 1 captured by the imaging unit 15 onto a three-dimensional surface in a bowl or cylinder shape. Fig.13 As shown, the three-dimensional surrounding image G7 includes a three-dimensional image of the vehicle 1, namely a three-dimensional vehicle image G8. In this embodiment, the three-dimensional vehicle image G8 is composed of a plurality of polygons and represents the three-dimensional shape of the vehicle 1, similarly to the virtual vehicle image G5.

[0116] In addition, in the present embodiment, the control unit 403 displays vehicle position information I that can identify the position of the three-dimensional vehicle image G8 on the road surface within the three-dimensional surrounding image G7. For example, the vehicle position information I is information that displays the position of the three-dimensional vehicle image G8 on the road surface within the three-dimensional surrounding image G7 by a gray scale and a line (e.g., a dotted line) surrounding the position where the three-dimensional vehicle image G8 exists.

[0117] Furthermore, when the detection unit 402 is in an operating state, as shown in FIG. Fig.14 As shown in FIG. 4 , similar to the first embodiment, the control unit 403 displays the proximity object marker G6 in the surrounding image G2 and also displays the proximity object marker G9 in the three-dimensional surrounding image G7. Fig.14 As shown, the control unit 403 displays the approaching object marker G9 included in the three-dimensional surrounding image G7 with a gray scale. In this embodiment, since it is easy to grasp the positional relationship between the vehicle image G1 and the surrounding objects, the control unit 403 does not display the virtual vehicle image G5 in the three-dimensional surrounding image G7, but it is not limited to this, and the virtual vehicle image G5 may also be displayed in the three-dimensional surrounding image G7.

[0118] Fig.15 as well as Fig.16 1 is a diagram showing an example of a display screen displayed by the ECU of the vehicle of the second embodiment. In this embodiment, when the detection unit 402 detects an object approaching the vehicle 1 (for example, an object approaching from the left side of the traveling direction of the vehicle 1), as shown in FIG. Fig.15As shown, the control unit 403 changes the display mode of the approaching object markers G6 and G9 included in the surrounding image G2 and the three-dimensional surrounding image G7 , which are located in the direction in which the detected object approaches (eg, the left side).

[0119] In addition, when the detection unit 402 detects that the vehicle 1 is approaching from the left and right sides of the traveling direction, as shown in FIG. Fig.16 As shown, the control unit 403 changes the display color of the approaching object markers G6 and G9 existing on the left and right sides of the traveling direction (e.g., the front) of the vehicle 1 to yellow or the like, or makes the approaching object markers G6 and G9 flash. Alternatively, when the approaching object markers G6 and G9 include multiple arrows, the control unit 403 may also display the multiple arrows included in the approaching object markers G6 and G9 displayed in the direction where the detected moving object exists in an animation that sequentially changes the display mode starting from the arrow away from the virtual vehicle image G5.

[0120] In the present embodiment, when the detection unit 402 is in the operating state, the approaching object indicators G6 and G9 are displayed in advance by gray scales, etc., so that when the detection unit 402 detects an object approaching the vehicle 1 and the display mode of the approaching object indicators G6 and G9 is changed, the driver of the vehicle 1 can easily recognize that the object approaching the vehicle 1 has been detected by the change in the display mode. In the present embodiment, when the detection unit 402 is in the operating state, the control unit 403 displays both the virtual vehicle image G5 and the approaching object indicators G6 and G9 on the display screen G, but at least the virtual vehicle image G5 is displayed.

[0121] As described above, according to the vehicle 1 of the second embodiment, by visually checking the three-dimensional surrounding image G7 in addition to the synthesized image G3 , it is possible to grasp the positional relationship between the vehicle 1 and the objects in its surroundings in more detail.

Claims

1. A perimeter monitoring device, characterized in that: have: An acquisition unit (401) for acquiring a current steering angle of a vehicle; an image acquisition unit (400) that acquires a captured image from a capturing unit (15) that captures the periphery of the vehicle; and A control unit (403) causes a composite image including a vehicle image representing the vehicle and a peripheral image representing the periphery of the vehicle based on the captured image to be displayed on a display unit, and when a detection unit (402) capable of detecting an object in contact with the vehicle is in an operating state, a virtual vehicle image representing the shape of the vehicle is displayed in an overlapping manner on a position after the vehicle has traveled a predetermined distance at the current steering angle acquired by the acquisition unit (401), based on the position of the vehicle represented by the vehicle image in the composite image. When the object is detected by the detection unit, the control unit changes the display mode of the portion of the image in contact with the object in the virtual vehicle image and stops the movement of the virtual vehicle image at a contact position where the vehicle and the object are in contact in the synthetic image.

2. The perimeter monitoring device according to claim 1, characterized in that: The virtual vehicle image is an image representing the shape of the vehicle composed of polygons. The partial image is the polygon that is in contact with the object among the polygons constituting the virtual vehicle image.

3. The perimeter monitoring device according to claim 1, characterized in that: The control unit changes a display mode of the partial image in accordance with a distance between a position of the object and a position of the vehicle represented by the virtual vehicle image.

4. The perimeter monitoring device according to claim 2, characterized in that: The control unit changes a display mode of the partial image in accordance with a distance between a position of the object and a position of the vehicle represented by the virtual vehicle image.

5. The perimeter monitoring device according to any one of claims 1 to 4, characterized in that: The vehicle image is a bird's-eye view image of the vehicle.

6. The perimeter monitoring device according to claim 5, characterized in that: The virtual vehicle image is a semi-transparent image representing the shape of the vehicle.

7. The perimeter monitoring device according to claim 5, characterized in that: The virtual vehicle image is an image in which the outline of the vehicle is emphasized.

8. The perimeter monitoring device according to claim 5, characterized in that: The virtual vehicle image is an image whose transmittance increases from the outline of the vehicle toward the inner side.

9. The perimeter monitoring device according to any one of claims 1 to 4, characterized in that: The virtual vehicle image is an image representing a three-dimensional shape of the vehicle.

10. The perimeter monitoring device according to claim 9, characterized in that: The virtual vehicle image is a semi-transparent image representing the shape of the vehicle.

11. The perimeter monitoring device according to claim 9, characterized in that: The virtual vehicle image is an image whose transmittance increases from the outline of the vehicle toward the inner side.

12. The perimeter monitoring device according to claim 9, characterized in that: The virtual vehicle image is an image in which the outline of the vehicle is emphasized.

13. A perimeter monitoring device, characterized in that: have: An acquisition unit (401) for acquiring a current steering angle of a vehicle; an image acquisition unit (400) that acquires a captured image from a capturing unit (15) that captures the periphery of the vehicle; and A control unit (403) causes a composite image including a vehicle image representing the vehicle and a peripheral image representing the periphery of the vehicle based on the captured image to be displayed on a display unit, and when a detection unit (402) capable of detecting an object in contact with the vehicle is in an operating state, a virtual vehicle image representing the shape of the vehicle is displayed in an overlapping manner on a position after the vehicle has traveled a predetermined distance at the current steering angle acquired by the acquisition unit (401), based on the position of the vehicle represented by the vehicle image in the composite image. When the detection unit is in an action state, the control unit displays a mark capable of identifying the direction in which the object is approaching the vehicle relative to the traveling direction of the vehicle represented by the vehicle image in the composite image, and when the object approaching the vehicle is detected by the detection unit, the control unit changes the display mode of the mark.

14. The perimeter monitoring device according to claim 13, characterized in that: The control unit displays the virtual vehicle image in a superimposed manner at a contact position between the vehicle and the object or at a position when the vehicle has traveled to a position before the contact as a position after the vehicle has traveled the predetermined distance.

15. The perimeter monitoring device according to claim 13, characterized in that: The vehicle image is a bird's-eye view image of the vehicle.

16. The perimeter monitoring device according to claim 13, characterized in that: The virtual vehicle image is an image representing a three-dimensional shape of the vehicle.

17. The perimeter monitoring device according to claim 13, characterized in that: The virtual vehicle image is a semi-transparent image representing the shape of the vehicle.

18. The perimeter monitoring device according to claim 13, characterized in that: The virtual vehicle image is an image in which the outline of the vehicle is emphasized.

19. The perimeter monitoring device according to claim 13, characterized in that: The virtual vehicle image is an image whose transmittance increases from the outline of the vehicle toward the inner side.

20. A perimeter monitoring device, characterized in that: have: An acquisition unit (401) for acquiring a current steering angle of a vehicle; an image acquisition unit (400) that acquires a captured image from a capturing unit (15) that captures the periphery of the vehicle; and A control unit (403) causes a composite image including a vehicle image representing the vehicle and a peripheral image representing the periphery of the vehicle based on the captured image to be displayed on a display unit, and when a detection unit (402) capable of detecting an object in contact with the vehicle is in an operating state, a virtual vehicle image representing the shape of the vehicle is displayed in an overlapping manner on a position after the vehicle has traveled a predetermined distance at the current steering angle acquired by the acquisition unit (401), based on the position of the vehicle represented by the vehicle image in the composite image. The control unit displays the virtual vehicle image in a superimposed manner at a contact position between the vehicle and the object or at a position when the vehicle has traveled to a position before the contact as a position after the vehicle has traveled the predetermined distance.

21. The perimeter monitoring device according to claim 20, characterized in that: The vehicle image is a bird's-eye view image of the vehicle.

22. The perimeter monitoring device according to claim 20, characterized in that: The virtual vehicle image is an image representing a three-dimensional shape of the vehicle.

23. The perimeter monitoring device according to claim 20, characterized in that: The virtual vehicle image is a semi-transparent image representing the shape of the vehicle.

24. The perimeter monitoring device according to claim 20, characterized in that: The virtual vehicle image is an image in which the outline of the vehicle is emphasized.

25. The perimeter monitoring device according to claim 20, characterized in that: The virtual vehicle image is an image whose transmittance increases from the outline of the vehicle toward the inner side.

Images