Periphery monitoring device

Abstract

To allow for easily checking a change in the situation on the left or right side of the vehicle's direction of travel.SOLUTION: A peripheral monitoring device according to an embodiment of the present disclosure includes: a display processing unit that displays a peripheral image indicating a virtual target view area including a virtual target viewpoint viewed from a virtual viewpoint in a virtual space that reflects a situation around a vehicle on a display unit on the basis of a captured image obtained by imaging surroundings of the vehicle; and a change processing unit that changes a viewing radius corresponding to a distance between the virtual target viewpoint and a virtual viewpoint and a viewing angle corresponding to a range of the virtual target area viewed from the virtual viewpoint according to change of the situation on the left side or the right side of a vehicle traveling direction.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to a perimeter monitoring device. [Background technology]

[0002] A technology is known for monitoring the situation around a vehicle by using a surrounding image that represents a virtual gaze area including a virtual gaze point seen from a virtual viewpoint in a virtual space in which the situation around the vehicle is reflected. Such a surrounding image is generated based on an image captured by an imaging device provided in the vehicle for capturing an image of the surroundings of the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 3300334 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned technology, if the viewpoint radius, which corresponds to the distance between the virtual gaze point and the virtual viewpoint, and the field of view angle, which corresponds to the range of the virtual gaze area as seen from the virtual viewpoint, are always set constant, it may be difficult to monitor changes in the situation on the left or right side of the vehicle's direction of travel.

[0005] Therefore, one of the objects of the present disclosure is to provide a surroundings monitoring device that makes it possible to easily check changes in the situation on the left or right side of the vehicle's traveling direction. [Means for solving the problem]

[0006] A surroundings monitoring device as an example of the present disclosure includes a display processing unit that displays on a display unit a surrounding image representing a virtual gaze area including a virtual gaze point seen from a virtual viewpoint in a virtual space in which the situation around the vehicle is reflected, based on an image obtained by imaging the surroundings of the vehicle, and a change processing unit that changes a viewpoint radius corresponding to the distance between the virtual gaze point and the virtual viewpoint and a field of view angle corresponding to the range of the virtual gaze area seen from the virtual viewpoint in response to changes in the situation on the left or right side of the vehicle's traveling direction.

[0007] According to the above-mentioned surroundings monitoring device, the sense of distance to the left and right of the vehicle on the surroundings image can be appropriately adjusted by changing the viewpoint radius and the viewing angle in response to changes in the situation on the left or right side of the vehicle's traveling direction. Therefore, changes in the situation on the left or right side of the vehicle's traveling direction can be easily confirmed.

[0008] In the above-mentioned surroundings monitoring device, when the situation on the left or right side in the traveling direction of the vehicle changes from a first situation to a second situation in which the degree to which the sense of distance to the left and right of the vehicle should be emphasized on the surroundings image is greater than in the first situation, the change processing unit changes the viewpoint radius to a second viewpoint radius larger than the first viewpoint radius used in the first situation, and changes the viewing angle to a second viewing angle smaller than the first viewing angle used in the first situation. With this configuration, the viewpoint radius and the viewing angle can be appropriately changed according to the situation.

[0009] In the above-mentioned surroundings monitoring device, when a three-dimensional object is detected on the left or right side of the vehicle's traveling direction under a first situation, the change processing unit determines that the situation on the left or right side of the vehicle's traveling direction has changed from the first situation to a second situation, and changes the viewpoint radius and the viewing angle. With this configuration, it is possible to appropriately determine whether or not to change the viewpoint radius and the viewing angle depending on whether or not a three-dimensional object is present on the left or right side of the vehicle's traveling direction.

[0010] In the above-mentioned surroundings monitoring device, when the movement of the vehicle is detected under a first condition, the change processing unit determines that the condition on the left or right side of the traveling direction of the vehicle has changed from the first condition to a second condition, and changes the viewpoint radius and the viewing angle. With this configuration, it is possible to appropriately determine whether or not to change the viewpoint radius and the viewing angle depending on whether or not the vehicle is moving.

[0011] In the above-mentioned surroundings monitoring device, when a change in the steering angle of the vehicle is detected in a first situation, the change processing unit determines that the situation on the left or right side of the vehicle's traveling direction has changed from the first situation to a second situation, and changes the viewpoint radius and the viewing angle. With this configuration, it is possible to appropriately determine whether or not to change the viewpoint radius and the viewing angle depending on whether or not the steering angle of the vehicle has changed by the predetermined amount or more.

[0012] In the above-mentioned surroundings monitoring device, when the vehicle is stopped, the change processing unit sets the viewpoint radius to a third viewpoint radius equal to or smaller than the first viewpoint radius, and sets the viewing angle to a third viewing angle equal to or larger than the first viewing angle. With this configuration, it is possible to prevent the sense of distance to the left and right of the vehicle from being emphasized on the surroundings image when it is not necessary.

[0013] In the above-mentioned surroundings monitoring device, the display processing unit gradually changes the surroundings image in an animation in response to the change in the viewpoint radius and the viewing angle by the change processing unit. With this configuration, the change in the surroundings image accompanying the change in the viewpoint radius and the viewing angle can be executed in an intuitive and easy-to-understand manner.

[0014] In the above-described surroundings monitoring device, the change processing unit maintains the viewpoint radius and the viewing angle until a predetermined time has elapsed after changing the viewpoint radius and the viewing angle. With this configuration, it is possible to prevent the surroundings image from frequently changing due to frequent changes in the viewpoint radius and the viewing angle, which makes it difficult to perform surroundings monitoring. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is an exemplary schematic diagram showing a configuration of a vehicle interior of a vehicle according to an embodiment. [Diagram 2] FIG. 2 is an exemplary schematic diagram showing the external appearance of the vehicle according to the embodiment as viewed from above. [Diagram 3] FIG. 3 is an exemplary schematic block diagram showing a system configuration of a vehicle according to an embodiment. [Figure 4] FIG. 4 is an exemplary schematic block diagram illustrating functions of the periphery monitoring device according to the embodiment. [Diagram 5] FIG. 5 is an illustrative schematic diagram showing an example of a peripheral image according to the embodiment. [Figure 6] FIG. 6 is an exemplary schematic diagram for explaining viewpoint parameters according to the embodiment. [Figure 7] FIG. 7 is an illustrative schematic diagram showing another example of the peripheral image according to the embodiment. [Figure 8] FIG. 8 is an exemplary schematic flowchart showing a series of processes executed by the surroundings monitoring device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. The configuration of the embodiment described below and the actions and effects brought about by the configuration are merely examples and are not limited to the contents described below.

[0017] First, a schematic configuration of a vehicle 1 according to an embodiment will be described with reference to Figures 1 and 2. Figure 1 is an exemplary and schematic diagram showing the configuration inside a vehicle compartment 2a of a vehicle 1 according to an embodiment, and Figure 2 is an exemplary and schematic diagram showing the external appearance of the vehicle 1 according to the embodiment as viewed from above.

[0018] 1, a vehicle 1 according to an embodiment has a cabin 2a in which passengers including a driver as a user ride. A braking unit (braking operation unit) 301a, an acceleration unit (acceleration operation unit) 302a, a steering unit 303a, a gear shift unit (gear shift operation unit) 304a, and the like are provided in the cabin 2a and can be operated by a user from a seat 2b.

[0019] The braking unit 301a is, for example, a brake pedal provided under the driver's feet, and the acceleration unit 302a is, for example, an accelerator pedal provided under the driver's feet. The steering unit 303a is, for example, a steering wheel protruding from a dashboard (instrument panel), and the transmission unit 304a is, for example, a shift lever protruding from a center console. The steering unit 303a may be a handle.

[0020] A monitor device 11 having a display unit 8 capable of outputting various images and an audio output unit 9 capable of outputting various sounds is provided in the vehicle interior 2a. The monitor device 11 is provided, for example, in the center of the width direction (left-right direction) of the dashboard in the vehicle interior 2a. The display unit 8 is, for example, configured with an LCD (liquid crystal display) or an OELD (organic electroluminescence display).

[0021] Here, the display screen as the area where the image is displayed on the display unit 8 is provided with an operation input unit 10. The operation input unit 10 is configured as a touch panel capable of detecting the coordinates of a position where an indicator such as a finger or a stylus approaches (including contacts) the display unit 8. This allows the user (driver) to visually recognize the image displayed on the display screen of the display unit 8, and also allows the user to perform various operation inputs by performing a touch (tap) operation using the indicator on the operation input unit 10.

[0022] In the embodiment, the operation input unit 10 may be various physical interfaces such as a switch, a dial, a joystick, a push button, etc. In the embodiment, another audio output device may be provided at a position different from the position of the monitor device 11 in the vehicle interior 2a. In this case, various sound information can be output from both the audio output unit 9 and the other audio output device. In the embodiment, the monitor device 11 may be configured to be capable of displaying information on various systems such as a navigation system and an audio system.

[0023] 1 and 2, the vehicle 1 according to the embodiment is configured as a four-wheeled vehicle having two front wheels 3F (left and right) and two rear wheels 3R ​​(left and right). For simplification, the front wheels 3F and the rear wheels 3R ​​may be collectively referred to as wheels below. In the embodiment, the sideslip angles of some or all of the four wheels are changed (steered) in response to the operation of the steering unit 303a, etc.

[0024] The vehicle 1 is also equipped with a plurality of on-board cameras 15a to 15d (four in the example shown in FIG. 1 and FIG. 2) as imaging devices for periphery monitoring. The on-board camera 15a is provided at the rear end 2e of the vehicle body 2 (for example, below the door 2h of the rear trunk) and images the area behind the vehicle 1. The on-board camera 15b is provided at the door mirror 2g at the right end 2f of the vehicle body 2 and images the area to the right of the vehicle 1. The on-board camera 15c is provided at the front end 2c of the vehicle body 2 (for example, the front bumper) and images the area in front of the vehicle 1. The on-board camera 15d is provided at the door mirror 2g at the left end 2d of the vehicle body 2 and images the area to the left of the vehicle 1. Hereinafter, for simplicity, the on-board cameras 15a to 15d may be collectively referred to as the on-board camera 15.

[0025] The vehicle-mounted camera 15 is a so-called digital camera having an imaging element such as a CCD (charge-coupled device) or a CIS (CMOS (complementary metal-oxide semiconductor) image sensor). The vehicle-mounted camera 15 captures images of the surroundings of the vehicle 1 at a predetermined frame rate and outputs image data of the captured images obtained by the imaging. The image data obtained by the vehicle-mounted camera 15 can be used as frame images to form a moving image.

[0026] Next, a system configuration provided for implementing various controls in the vehicle 1 according to the embodiment will be described with reference to Fig. 3. Note that the system configuration shown in Fig. 3 is merely an example and can be set (changed) in various ways.

[0027] 3 is an exemplary schematic block diagram showing a system configuration of the vehicle 1 according to the embodiment. As shown in FIG. 3, the vehicle 1 according to the embodiment includes a braking system 301, an acceleration system 302, a steering system 303, a gear shift system 304, an obstacle sensor 305, a driving state sensor 306, an in-vehicle camera 15, a monitor device 11, an ECU (Electronic Control Unit) 310, and an in-vehicle network 350.

[0028] The braking system 301 controls the deceleration of the vehicle 1. The braking system 301 includes a braking unit 301a, a braking control unit 301b, and a braking unit sensor 301c.

[0029] The braking unit 301a is a device for decelerating the vehicle 1, such as the above-mentioned brake pedal.

[0030] The braking control unit 301b is configured as a microcomputer having a hardware processor such as a CPU (Central Processing Unit), etc. The braking control unit 301b drives an actuator (not shown) based on an instruction input via the in-vehicle network 350, for example, to operate the braking unit 301a, thereby controlling the deceleration rate of the vehicle 1.

[0031] The braking unit sensor 301c is a sensing device for detecting the state of the braking unit 301a. For example, when the braking unit 301a is configured as a brake pedal, the braking unit sensor 301c detects the position of the brake pedal or the pressure acting on the brake pedal as the state of the braking unit 301a. The braking unit sensor 301c outputs the detected state of the braking unit 301a to the in-vehicle network 350.

[0032] The acceleration system 302 controls the acceleration of the vehicle 1. The acceleration system 302 has an acceleration unit 302a, an acceleration control unit 302b, and an acceleration sensor 302c.

[0033] The acceleration unit 302a is a device for accelerating the vehicle 1, such as the above-mentioned accelerator pedal.

[0034] The acceleration control unit 302b is configured as, for example, a microcomputer having a hardware processor such as a CPU. The acceleration control unit 302b drives an actuator (not shown) based on an instruction input via, for example, the in-vehicle network 350 to operate the acceleration unit 302a, thereby controlling the acceleration rate of the vehicle 1.

[0035] The acceleration sensor 302c is a sensing device for detecting the state of the acceleration unit 302a. For example, if the acceleration unit 302a is configured as an accelerator pedal, the acceleration sensor 302c detects the position of the accelerator pedal or the pressure acting on the accelerator pedal. The acceleration sensor 302c outputs the detected state of the acceleration unit 302a to the in-vehicle network 350.

[0036] The steering system 303 controls the traveling direction of the vehicle 1. The steering system 303 has a steering unit 303a, a steering control unit 303b, and a steering unit sensor 303c.

[0037] The steering unit 303a is a device for steering the steered wheels of the vehicle 1, such as the above-mentioned steering wheel or handle.

[0038] The steering control unit 303b is configured as, for example, a microcomputer having a hardware processor such as a CPU. The steering control unit 303b drives an actuator (not shown) based on an instruction input via, for example, the in-vehicle network 350, and operates the steering unit 303a to control the traveling direction of the vehicle 1.

[0039] The steering unit sensor 303c is a sensing device for detecting the state of the steering unit 303a, that is, a steering angle sensor for detecting the steering angle of the vehicle 1. For example, when the steering unit 303a is configured as a steering wheel, the steering unit sensor 303c detects the position of the steering wheel or the rotation angle of the steering wheel. When the steering unit 303a is configured as a steering wheel, the steering unit sensor 303c may detect the position of the steering wheel or the pressure acting on the steering wheel. The steering unit sensor 303c outputs the detected state of the steering unit 303a to the in-vehicle network 350.

[0040] The gear shift system 304 controls the gear ratio of the vehicle 1. The gear shift system 304 has a gear shift section 304a, a gear shift control section 304b, and a gear shift section sensor 304c.

[0041] The transmission unit 304a is a device for changing the gear ratio of the vehicle 1, such as the above-mentioned shift lever.

[0042] The gear shift control unit 304b is configured as a computer having a hardware processor such as a CPU, for example. The gear shift control unit 304b drives an actuator (not shown) based on an instruction input via the in-vehicle network 350, for example, to operate the gear shift unit 304a, thereby controlling the gear ratio of the vehicle 1.

[0043] The transmission sensor 304c is a sensing device for detecting the state of the transmission 304a. For example, when the transmission 304a is configured as a shift lever, the transmission sensor 304c detects the position of the shift lever or the pressure acting on the shift lever. The transmission sensor 304c outputs the detected state of the transmission 304a to the in-vehicle network 350.

[0044] The obstacle sensor 305 is a sensing device for detecting information related to objects (obstacles) that may exist around the vehicle 1. The obstacle sensor 305 includes, for example, a distance measurement sensor that acquires the distance to objects existing around the vehicle 1. Examples of distance measurement sensors that can be used include sonar that transmits sound waves and acquires the distance by receiving the sound waves reflected by objects existing around the vehicle 1, and laser radar that transmits radio waves such as light and acquires the distance by receiving the radio waves reflected by objects existing around the vehicle 1. The obstacle sensor 305 outputs the detected information to the in-vehicle network 350.

[0045] The running condition sensor 306 is a device for detecting the running condition of the vehicle 1. The running condition sensor 306 includes, for example, a wheel speed sensor for detecting the wheel speed of the vehicle 1, an acceleration sensor for detecting the acceleration in the forward / rearward or left / right direction of the vehicle 1, a gyro sensor for detecting the turning speed (angular velocity) of the vehicle 1, and the like. The running condition sensor 306 outputs the detected running condition to the in-vehicle network 350.

[0046] The ECU 310 is a device that comprehensively controls various systems provided in the vehicle 1. Although details will be described later, the ECU 310 according to the embodiment has a function of generating a surrounding image that shows the situation around the vehicle 1 based on image data captured by the vehicle-mounted camera 15, and displaying the generated surrounding image on the display unit 8. The surrounding image is, for example, a three-dimensional image (a specific example will be described later) that shows a virtual gaze area including a virtual gaze point seen from a virtual viewpoint in a virtual space in which the situation around the vehicle is reflected. Note that the surrounding image may include, in addition to the three-dimensional image, a bird's-eye image that shows the situation around the vehicle 1 from above, and a single camera image that is an image based only on one image data acquired from one vehicle-mounted camera 15.

[0047] The ECU 310 is configured as a microcomputer having a CPU (Central Processing Unit) 310a, a ROM (Read Only Memory) 310b, a RAM (Random Access Memory) 310c, a Solid State Drive (SSD) 310d, a display control unit 310e, and an audio control unit 310f.

[0048] The CPU 310a is a hardware processor that performs overall control of the ECU 310. The CPU 310a reads out various control programs (computer programs) stored in the ROM 310b or the like, and realizes various functions according to instructions defined in the various control programs. The various control programs mentioned here include a surroundings monitoring program for realizing a surroundings monitoring process that outputs the above-mentioned surroundings image to the display unit 8.

[0049] The ROM 310b is a non-volatile main storage device that stores parameters and the like required for executing the various control programs described above.

[0050] The RAM 310c is a volatile main storage device that provides a working area for the CPU 310a.

[0051] The SSD 310d is a rewritable non-volatile auxiliary storage device. Note that, in the ECU 310 according to the embodiment, a hard disk drive (HDD) may be provided as the auxiliary storage device instead of the SSD 310d (or in addition to the SSD 310d).

[0052] Of the various types of processing that can be executed by the ECU 310, the display control unit 310e mainly controls image processing of images captured by the vehicle-mounted camera 15, generation of image data to be output to the display unit 8 of the monitor device 11, and the like.

[0053] Of the various processes that can be executed by the ECU 310, the audio control unit 310f mainly controls the generation of audio data to be output to the audio output unit 9 of the monitor device 11 and the like.

[0054] The in-vehicle network 350 communicatively connects the braking system 301, the acceleration system 302, the steering system 303, the gear shift system 304, the obstacle sensor 305, the driving condition sensor 306, the operation input unit 10 of the monitor device 11, and the ECU 310.

[0055] Various technologies have been considered for the past for monitoring the situation around vehicle 1 using peripheral images such as the three-dimensional image described above. However, in such technologies, if the viewpoint radius corresponding to the distance between the virtual gaze point and the virtual viewpoint and the field of view angle corresponding to the range of the virtual gaze area as seen from the virtual viewpoint are always set constant, it may be difficult to monitor changes in the situation on the left or right side of the vehicle 1's direction of travel.

[0056] Therefore, in this embodiment, a surroundings monitoring device 400 having functions as shown in the following FIG. 4 is implemented in the ECU 310, thereby making it possible to easily monitor changes in the situation on the left or right side of the traveling direction of the vehicle 1.

[0057] FIG. 4 is an exemplary schematic block diagram showing the functions of a periphery monitoring device 400 according to the embodiment.

[0058] The functions shown in Fig. 4 are realized in the ECU 310 by cooperation between software and hardware. That is, the functions shown in Fig. 4 are realized as a result of the CPU 310a of the ECU 310 reading and executing a surroundings monitoring program stored in the ROM 310b or the SSD 310d. Note that in the embodiment, at least some of the functions shown in Fig. 4 may be realized by dedicated hardware (circuitry).

[0059] As shown in FIG. 4, the surroundings monitoring device 400 according to the embodiment includes a captured image acquisition unit 401, a surroundings image generation unit 402, a display processing unit 403, a detection processing unit 404, and a change processing unit 405.

[0060] The captured image acquisition unit 401 acquires a captured image from the vehicle-mounted camera 15. Then, the surrounding image generation unit 402 generates a surrounding image, for example, as shown in the following example of FIG.

[0061] FIG. 5 is an illustrative schematic diagram showing an example of a peripheral image according to the embodiment.

[0062] 5, image IM500 is an example of the above-mentioned three-dimensional image as a peripheral image. In this image IM500, the situation around the vehicle 1 is displayed three-dimensionally from a predetermined viewpoint and field of view together with a vehicle image V0 representing the vehicle 1. Image IM500 can be generated by setting various viewpoint parameters in a virtual space obtained by projecting a captured image onto a bowl-shaped or cylindrical model.

[0063] The viewpoint parameters in the embodiment include the coordinates of the virtual viewpoint and the virtual gaze point in the virtual space, a viewpoint radius corresponding to the distance between the virtual viewpoint and the virtual gaze point, and a viewing angle corresponding to the range of the virtual parking area as seen from the virtual viewpoint, as shown in FIG. 6 below.

[0064] FIG. 6 is an exemplary schematic diagram for explaining viewpoint parameters according to the embodiment.

[0065] In the example shown in Fig. 6, points P0 and P1 correspond to a virtual viewpoint and a virtual gaze point, respectively, in a virtual space in which a predetermined coordinate system is set. Fig. 6 illustrates, as an example, a virtual space in which an XYZ coordinate system is set, the X axis extending horizontally from the right to the left of the vehicle image V0, the Y axis extending vertically from the bottom to the top of the vehicle image V0, and the Z axis extending horizontally from the rear to the front of the vehicle image V0.

[0066] In the embodiment, for example, a viewpoint radius as the distance D between points P0 and P1, a horizontal angle indicating the angle of point P0 with respect to the horizontal direction with point P1 as a reference, and a vertical angle indicating the angle of point P0 with respect to the vertical direction with point P1 as a reference are set as viewpoint parameters, and as a result, the coordinates of points P0 and P1 in the virtual space are set. Note that in the embodiment, the viewpoint radius, horizontal angle, and vertical angle may be set by setting the coordinates of points P0 and P1 in the virtual space as viewpoint parameters.

[0067] 6, the angle θ corresponds to a viewing angle from a virtual viewpoint. In the embodiment, when the angle θ is set as a viewpoint parameter, the range of the virtual gaze area R including the point P1 that is a virtual gaze point as viewed from the point P0 that is a virtual viewpoint is determined as a range to be displayed in the peripheral image.

[0068] Here, as described above, if the viewpoint radius and the viewing angle are always set to be constant, it may be difficult to monitor changes in the situation on the left or right side of the traveling direction of the vehicle 1. For example, in a situation where the vehicle 1 passes a three-dimensional object such as another vehicle, it is desirable to appropriately change the viewpoint radius and the viewing angle so that the sense of distance to the left and right of the vehicle 1 is emphasized in the peripheral image more than when the three-dimensional object does not exist.

[0069] 4, the detection processing unit 404 detects the situation on the left or right side of the traveling direction of the vehicle 1 based on the output of various sensors (see FIG. 3) mounted on the vehicle 1. Then, the change processing unit 405 dynamically changes the viewpoint radius and the viewing angle of the peripheral image output to the display unit 8 by the display processing unit 403 according to the result of detection by the detection processing unit 404, that is, according to a change in the situation on the left or right side of the traveling direction of the vehicle 1.

[0070] More specifically, the change processing unit 405 determines whether the degree to which the sense of left and right distance from the vehicle 1 should be emphasized on the peripheral image has increased from the current state, according to the result of detection by the detection processing unit 404. Then, when the change processing unit 405 determines that the degree to which the sense of left and right distance from the vehicle 1 should be emphasized on the peripheral image has increased from the current state, the change processing unit 405 changes the viewpoint radius to be larger than the current viewpoint radius, and changes the viewing angle to be smaller than the current viewing angle. Hereinafter, for convenience of explanation, the situation before the change of the viewpoint radius and the viewing angle is sometimes expressed as a first situation, and the situation after the change of the viewpoint radius and the viewing angle is sometimes expressed as a second situation.

[0071] For example, when a three-dimensional object is detected on the left or right side of the traveling direction of vehicle 1 under a first situation, change processing unit 405 may determine that the situation on the left or right side of the traveling direction of vehicle 1 has changed from the first situation to a second situation. In this case, change processing unit 405 may change the viewpoint radius to a second viewpoint radius that is larger than the first viewpoint radius used in the first situation, and change the viewing angle to a second viewing angle that is smaller than the first viewing angle used in the first situation.

[0072] Also, when the movement of the vehicle 1 is detected under the first situation, the change processing unit 405 may determine that the situation on the left or right side of the traveling direction of the vehicle 1 has changed from the first situation to the second situation, and may execute the above-mentioned change in the viewpoint radius and the viewing angle. Furthermore, when the change in the steering angle of the vehicle 1 of a predetermined amount or more is detected under the first situation, the change processing unit 405 may determine that the situation on the left or right side of the traveling direction of the vehicle 1 has changed from the first situation to the second situation, and may execute the above-mentioned change in the viewpoint radius and the viewing angle.

[0073] Taking the above into consideration, when the situation shown in Figure 5, in which there is no three-dimensional object on either the left or right side of the vehicle 1's traveling direction, changes to a situation in which a three-dimensional object is present on the left or right side of the vehicle 1's traveling direction, the surrounding image output by the display processing unit 403 will change, for example, as shown in the example in Figure 7 below.

[0074] FIG. 7 is an illustrative schematic diagram showing another example of the peripheral image according to the embodiment.

[0075] Image IM700 shown in Fig. 7 is an example of a surrounding image (three-dimensional image) that is switched from image IM500 shown in Fig. 5 and output to display unit 8 in response to the above-mentioned change of viewpoint radius and viewing angle by change processing unit 405. In this image IM700, the situation around vehicle 1 is displayed three-dimensionally from a farther viewpoint and with a narrower field of view than the example shown in Fig. 5, together with vehicle image V0 representing vehicle 1 and vehicle image V1 representing another vehicle as a three-dimensional object existing to the right of vehicle 1 in the traveling direction. As a result, image IM700 shown in Fig. 7 makes it easier to understand the sense of distance to the left and right of vehicle 1, which should be noted when vehicle 1 passes another vehicle, than the example shown in Fig. 5.

[0076] In this way, according to the embodiment, a surrounding image that allows the sense of distance to the left and right of the vehicle 1 to be easily understood depending on the situation can be provided.

[0077] Incidentally, the above description illustrates a situation in which the viewpoint radius is increased and the field of view is narrowed to make it easier to understand the sense of distance to the left and right of the vehicle 1. However, depending on the situation, there may be cases where it is more appropriate to reduce the viewpoint radius and increase the field of view. For example, when the vehicle 1 is stopped, particularly before the vehicle 1 starts, it may be appropriate to monitor a wider range from a closer viewpoint.

[0078] Therefore, in an embodiment, when the vehicle is stopped, the change processing unit 405 may set the viewpoint radius to a third viewpoint radius that is less than or equal to the first viewpoint radius used in the above-mentioned first situation, and set the field of view angle to a third field of view angle that is greater than or equal to the first field of view angle used in the above-mentioned first situation.

[0079] In addition, in the embodiment, if the viewpoint radius and the viewing angle are changed frequently, the surrounding image is also changed frequently. If the surrounding image changes frequently, it is difficult to perform surrounding monitoring using the surrounding image. Therefore, in the embodiment, after changing the viewpoint radius and the viewing angle, the change processing unit 405 can maintain the viewpoint radius and the viewing angle until a predetermined time has elapsed.

[0080] In addition, in the embodiment, when the viewpoint radius and the viewing angle are changed, a configuration in which the peripheral image changes gradually is more intuitive than a configuration in which the peripheral image changes significantly all at once. Therefore, in the embodiment, the display processing unit 403 can gradually change the peripheral image with an animation in response to the change of the viewpoint radius and the viewing angle by the change processing unit 405.

[0081] In the embodiment, the change of the viewpoint radius and the viewing angle may take into consideration the distance to the detected three-dimensional object, etc. In other words, in the embodiment, the change processing unit 405 may change the degree of change of the viewpoint radius and the viewing angle depending on the distance to the detected three-dimensional object.

[0082] Based on the above configuration, the surroundings monitoring device 400 according to the embodiment executes a series of processes as shown in FIG.

[0083] 8 is an exemplary schematic flowchart showing a series of processes executed by the surroundings monitoring device 400 according to the embodiment. The series of processes shown in FIG. 8 is repeatedly executed under a situation where the surroundings monitoring of the vehicle 1 is required.

[0084] As shown in FIG. 8, in this embodiment, first, in S801, the captured image acquisition unit 401 of the periphery monitoring device 400 acquires an image of the periphery of the vehicle 1 from the in-vehicle camera 15.

[0085] Then, in S802, the surrounding image generation unit 402 of the surrounding monitoring device 400 generates a surrounding image (three-dimensional image) representing a virtual gaze area including a virtual gaze point seen from a virtual viewpoint in a virtual space reflecting the situation around the vehicle 1, based on the captured image acquired in S401.

[0086] Then, in S803, the detection processing unit 404 of the periphery monitoring device 400 detects the situation on the left or right side of the traveling direction of the vehicle 1 based on the output of various sensors (see FIG. 3) mounted on the vehicle 1. For example, the detection processing unit 404 detects whether or not a three-dimensional object exists on the left or right side of the traveling direction of the vehicle 1, whether or not the vehicle 1 is moving, whether or not the steering angle of the vehicle 1 has changed by a predetermined amount or more, and the like.

[0087] Then, in S804, the change processing unit 405 of the surroundings monitoring device 400 determines, based on the result of the detection in S803, whether or not the situation on the left or right side in the direction of travel of the vehicle 1 has changed to a situation in which the sense of distance to the left and right of the vehicle 1 should be emphasized on the surroundings image.

[0088] If it is determined in S804 that the situation on the left or right side of the traveling direction of the vehicle 1 has not changed to a situation in which the sense of distance to the left and right of the vehicle 1 should be emphasized on the peripheral image, there is no need to change the viewpoint radius and the viewing angle of the peripheral image. Therefore, in this case, the process ends.

[0089] On the other hand, if it is determined in S804 that the situation on the left or right side of the traveling direction of the vehicle 1 has changed to one in which the sense of distance to the left and right of the vehicle 1 should be emphasized on the peripheral image, the viewpoint radius and the viewing angle of the peripheral image may be required. Therefore, in this case, the process proceeds to S805, and the change processing unit 405 determines whether a predetermined time has elapsed since the previous change of the viewpoint radius and the viewing angle, prior to changing the viewpoint radius and the viewing angle of the peripheral image.

[0090] If it is determined in S805 that the predetermined time has elapsed, the viewpoint radius and the viewing angle should be changed. Therefore, in this case, the process proceeds to S806, where the change processing unit 405 changes the viewpoint radius to a larger one than the currently used viewpoint radius, and changes the viewing angle to a smaller one than the currently used viewing angle. Then, the process ends.

[0091] On the other hand, if it is determined in S805 that the predetermined time has not elapsed, the determination in S804 is executed again.

[0092] As described above, the periphery monitoring device 400 according to the embodiment includes the display processing unit 403 and the change processing unit 405. The display processing unit 403 displays, on the display unit 8, a peripheral image representing a virtual gaze area including a virtual gaze point seen from a virtual viewpoint in a virtual space reflecting the situation of the periphery of the vehicle 1, based on an image obtained by imaging the periphery of the vehicle 1. The change processing unit 405 changes a viewpoint radius corresponding to the distance between the virtual gaze point and the virtual viewpoint and a viewing angle corresponding to the range of the virtual gaze area seen from the virtual viewpoint, in response to a change in the situation on the left or right side of the traveling direction of the vehicle 1.

[0093] According to the above configuration, it is possible to appropriately adjust the sense of distance to the left and right of the vehicle 1 on the peripheral image by changing the viewpoint radius and the viewing angle in response to changes in the situation on the left or right side of the traveling direction of the vehicle 1. Therefore, it is possible to easily confirm changes in the situation on the left or right side of the traveling direction of the vehicle 1.

[0094] The surroundings monitoring program executed in the ECU 310 according to the embodiment does not necessarily need to be stored in advance in the ROM 310b or the SSD 310d. For example, the surroundings monitoring program according to the embodiment may be provided as a computer program product recorded in an installable or executable format on a computer-readable recording medium such as various magnetic disks such as a flexible disk (FD) or various optical disks such as a digital versatile disk (DVD).

[0095] The periphery monitoring program according to the embodiment may be provided or distributed via a network such as the Internet. That is, the periphery monitoring program according to the embodiment may be provided in a state where it is stored on a computer connected to a network such as the Internet and is downloaded via the network.

[0096] Although the embodiments of the present disclosure have been described above, the above-mentioned embodiments are presented as examples and are not intended to limit the scope of the invention. The above-mentioned novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. The above-mentioned embodiments and their modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0097] 1 vehicle 8 Display 400 Periphery monitoring device 403 Display processing unit 405 Change Processing Section

Claims

1. a display processing unit that displays, on a display unit, a peripheral image that represents a virtual gaze area including a virtual gaze point seen from a virtual viewpoint in a virtual space in which a situation surrounding the vehicle is reflected, based on an image obtained by imaging the periphery of the vehicle; and a change processing unit that, when a three-dimensional object is detected on the left or right side of the traveling direction of the vehicle, changes a viewpoint radius corresponding to a distance between the virtual gaze point and the virtual viewpoint from a first viewpoint radius to a second viewpoint radius larger than the first viewpoint radius, and changes a viewing angle corresponding to a range of the virtual gaze area seen from the virtual viewpoint from the first viewing angle to a second viewing angle smaller than the first viewing angle; A perimeter monitoring device comprising:

2. a display processing unit that displays, on a display unit, a peripheral image that represents a virtual gaze area including a virtual gaze point seen from a virtual viewpoint in a virtual space in which a situation surrounding the vehicle is reflected, based on an image obtained by imaging the periphery of the vehicle; and a change processing unit that, when movement of the vehicle is detected, changes a viewpoint radius corresponding to a distance between the virtual gaze point and the virtual viewpoint from a first viewpoint radius to a second viewpoint radius larger than the first viewpoint radius, and changes a viewing angle corresponding to a range of the virtual gaze area seen from the virtual viewpoint from a first viewing angle to a second viewing angle smaller than the first viewing angle; A perimeter monitoring device comprising:

3. a display processing unit that displays, on a display unit, a peripheral image that represents a virtual gaze area including a virtual gaze point seen from a virtual viewpoint in a virtual space in which a situation surrounding the vehicle is reflected, based on an image obtained by imaging the periphery of the vehicle; and a change processing unit that, when a change of a steering angle of the vehicle equal to or larger than a predetermined value is detected, changes a viewpoint radius corresponding to a distance between the virtual gaze point and the virtual viewpoint from a first viewpoint radius to a second viewpoint radius larger than the first viewpoint radius, and changes a viewing angle corresponding to a range of the virtual gaze area seen from the virtual viewpoint from the first viewing angle to a second viewing angle smaller than the first viewing angle; A perimeter monitoring device comprising:

4. the change processing unit, when the vehicle is stopped, sets the viewpoint radius to a third viewpoint radius that is equal to or smaller than the first viewpoint radius, and sets the viewing angle to a third viewing angle that is equal to or larger than the first viewing angle. The surroundings monitoring device according to any one of claims 1 to 3.

5. the display processing unit gradually changes the peripheral image in an animation in response to the change of the viewpoint radius and the viewing angle by the change processing unit. The surroundings monitoring device according to any one of claims 1 to 4.

6. the change processing unit maintains the viewpoint radius and the viewing angle until a predetermined time has elapsed after the viewpoint radius and the viewing angle are changed. The surroundings monitoring device according to any one of claims 1 to 5.

Images