Environmental display device and environmental display system

Abstract

To provide an environmental display that can improve the accuracy of the driver's sense of vehicle speed and vehicle body regarding his / her own vehicle, while eliminating information on the surrounding environment that is of little importance for driving.SOLUTION: An environmental display device includes: an image processing unit that generates an environmental image from an image of the surrounding environment of an own vehicle, including the occupant's blind spots as viewed from the viewpoint of occupants of the own vehicle; and a display control unit that displays the environmental image by means of a display device. The image processing unit displays in the environmental image contour lines indicating the range of the bottom, side and front of the own vehicle's body as viewed from the occupants, displays an ambient image, which is an image of the surrounding environment of the own vehicle, in the image portion of the environmental image other than inside the contour lines indicating the range of the bottom surface, and does not display the ambient image in the image portion inside the contour lines indicating the range of the bottom surface.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an environmental display device and an environmental display system. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transport systems that take into consideration vulnerable transport participants have been gaining momentum. To achieve this, we are focusing on research and development into driver assistance technologies to further improve road safety and convenience.

[0003] Patent Document 1 discloses a driving support display device. This driving support display device displays a turning trajectory icon of the vehicle on an overhead image including the road around the vehicle and the left and right front ends of the vehicle, and displays a vehicle width icon on a bird's-eye image including the road ahead of the road around the vehicle. The inside of the vehicle width of the turning trajectory icon and the vehicle width icon are highlighted by hatching.

[0004] Patent Document 2 discloses a vehicle trajectory display system that can provide the driver with a sense of vehicle width. This vehicle trajectory display system adjusts the display position of an AR (Augmented Reality)-HUD based on the driver's viewpoint estimated from an image of the driver's face, and displays guidelines that indicate the outer edge of the vehicle in the vehicle width direction aligned with the road surface position using the AR-HUD. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2012-133614 A [Patent Document 2] Patent Publication No. 2021-86559 Summary of the Invention [Problem to be solved by the invention]

[0006] Incidentally, in driving assistance technology, it is an issue to assist a passenger in improving the accuracy of his / her sense of vehicle speed and vehicle body regarding the vehicle. In order to solve the above problems, the present application aims to provide an environmental display that can improve the accuracy of the vehicle speed and vehicle body sense of a passenger who may become a driver, while eliminating information about the surrounding environment that is less important for driving the vehicle, and ultimately contribute to the development of a sustainable transportation system. [Means for solving the problem]

[0007] One aspect of the present invention is an environmental display device that includes an image processing unit that generates an environmental image including a blind spot of a passenger of the vehicle from an image of the environment surrounding the vehicle, as seen from the viewpoint of the passenger of the vehicle, and a display control unit that displays the environmental image on a display device, wherein the image processing unit displays a contour line indicating the range of the bottom, sides, and front of the body of the vehicle as seen by the passenger in the environmental image, and displays a surrounding image, which is an image of the environment surrounding the vehicle, in an image portion of the environmental image other than the inside of the contour line indicating the range of the bottom surface, and does not display the surrounding image in an image portion inside the contour line indicating the range of the bottom surface. According to another aspect of the present invention, when the vehicle speed of the vehicle exceeds a predetermined first threshold, the image processing unit extends the contour line forward of the vehicle along a future trajectory of the vehicle, and displays the surrounding image in an image portion of the environmental image other than an image portion showing at least the extended range of the bottom surface of the vehicle within the extended range defined by the extended contour line, and does not display the surrounding image in an image portion of the extended bottom surface range. According to another aspect of the present invention, the distance over which the contour line is extended is longer as the vehicle speed of the host vehicle increases. According to another aspect of the present invention, when the vehicle speed of the vehicle exceeds a predetermined second threshold, the image processing unit displays the surrounding image in an image portion of the environmental image other than the image portion showing the bottom and side ranges of the vehicle within the range defined by the contour line, and does not display the surrounding image in the image portion showing the bottom and side ranges. According to another aspect of the present invention, when the vehicle speed of the vehicle exceeds a predetermined second threshold, the image processing unit displays the surrounding image in an image portion of the environmental image other than the image portion showing the extended bottom and side ranges of the vehicle within the range defined by the extended contour line, and does not display the surrounding image in the image portion showing the extended bottom and side ranges. According to another aspect of the present invention, the image processing unit displays an image of wheels in the environmental image at a position of the wheels of the host vehicle as seen by the passenger. According to another aspect of the present invention, the image processing unit superimposes and displays a moving graphic, which is a group of fixed or irregular graphic elements that move in accordance with the movement of the surrounding environment flowing from the front to the rear of the vehicle, on the image portion of the environmental image where the surrounding image is displayed. According to another aspect of the present invention, the group of graphic elements of the moving graphic is a group of fixed or irregular points in the environmental image that move from a vanishing point of the image of the surrounding environment across the horizon on which the vehicle is traveling. According to another aspect of the present invention, the group of graphic elements of the movement graphic is a group of lines parallel to the horizon in the environmental image, which move from a position on the horizon along the ground on which the host vehicle is traveling. According to another aspect of the present invention, the display device is a projector that displays an image on an inner surface of a passenger compartment of the host vehicle. According to another aspect of the present invention, the display control unit instructs the display device, which is the projector, to display the environmental image on the inner surface of the vehicle cabin so that the position of the outer edge of the bottom, side, and front areas of the vehicle's body as seen by the occupant coincides with the display position of the contour line. According to another aspect of the present invention, the display device is a head-mounted display that is worn on a person's head. According to another aspect of the present invention, the head-mounted display, which is the display device, provides an augmented reality display in which an image is superimposed on the real world viewed by the occupant, and the display control unit instructs the display device, which is the head-mounted display, to display the environmental image so that the position of the outer edge of the bottom, side, and front areas of the vehicle's body as seen by the occupant coincides with the display position of the contour line. According to another aspect of the present invention, the display device is a monitor device mounted on the host vehicle. According to another aspect of the invention, the display device is a head-up display. Another aspect of the present invention is an environment display system including the above-mentioned display device, an imaging device that captures an image of the surrounding environment of the vehicle, and any one of the above-mentioned environment display devices. Effect of the Invention

[0008] According to the present invention, it is possible to improve the accuracy of a potential driver's sense of vehicle speed and vehicle body sensation regarding the vehicle, while eliminating information about parts of the surrounding environment that are less important to driving the vehicle. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a vehicle equipped with an environmental display device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of the configuration of the interior of a vehicle. [Diagram 3] FIG. 3 is a diagram showing the configuration of an environment display device and an environment display system according to an embodiment of the present invention. [Figure 4] FIG. 4 is a flow chart showing the procedure of the environment display method executed by the processor of the environment display device. [Diagram 5] FIG. 5 is a diagram showing an example of a displayed environmental image. [Figure 6] FIG. 6 is a diagram showing an example of a displayed environmental image. [Figure 7]FIG. 7 is a diagram showing an example of a displayed environmental image. [Figure 8] FIG. 8 is a diagram showing an example of a displayed environmental image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [1. Vehicle configuration] FIG. 1 is a diagram showing an example of the configuration of a host vehicle 2 equipped with an environmental display device 1 according to one embodiment of the present invention, and FIG. 2 is a diagram showing an example of the configuration of the interior of the host vehicle 2.

[0011] The vehicle 2 includes an environment display device 1 and a front camera 3 that captures an image of the surrounding environment in front of the vehicle 2. The front camera 3 is disposed, for example, above the windshield inside the vehicle cabin. Note that multiple cameras may be used as the front camera 3.

[0012] The front camera 3 captures an image of the environment surrounding the vehicle 2, including the blind spot of the passenger D who will be the driver of the vehicle 2. Here, the front camera 3 corresponds to an imaging device that captures an image of the environment surrounding the vehicle in this disclosure. The vehicle 2 may further include a rear camera (not shown) that captures an image of the environment behind the vehicle.

[0013] The vehicle 2 may also be equipped with an object detection device 5 that detects objects present in the surrounding environment. The host vehicle 2 is further equipped with a vehicle monitoring device 6 that collects at least information on the traveling speed of the host vehicle 2 and information on the steering angle based on the operation of the steering wheel 10 or the like.

[0014] A projector 7 that projects and displays an image on an inner surface (or an inner wall surface) of the vehicle interior is provided on the ceiling of the vehicle interior of the vehicle 2. The projector 7 corresponds to the display device in this disclosure. In the vehicle interior, a general-purpose monitor device 11 used by an in-vehicle navigation device or an in-vehicle infotainment device (both not shown) may be provided on an instrument panel 12. A head-up display (HUD) 17 that projects and displays a virtual image on a windshield 13 may also be provided.

[0015] [2. Configuration of the environmental display device] FIG. 3 is a diagram showing the configuration of the environment display device 1 and the environment display system 8. As shown in FIG. The environment display system 8 includes at least a front camera 3 which is an imaging device, a projector 7 which is a display device, and an environment display device 1.

[0016] The environmental display device 1 acquires an image of the surrounding environment of the vehicle 2, and displays an environmental image of the vehicle 2, including blind spots of a passenger D of the vehicle 2, viewed from the viewpoint of the passenger D, on the inner surface of the cabin of the vehicle 2 by a projector 7. In this embodiment, the projector 7 displays the environmental image, for example, in a range DA (hereinafter, referred to as a display range DA) that is a part of the inner surface of the cabin, as shown by a dashed line in FIG.

[0017] The display range DA is a range of the interior surface of the vehicle cabin that includes the windshield 13 and the instrument panel 12, and extends beyond the left pillar 16a and the right pillar 16b to a part of the left door window 14a and a part of the right door window. The interior surface of the vehicle cabin within the display range DA may be coated to increase the reflectance of light in a predetermined light wavelength range so that the image projected by the projector 7 is emphasized.

[0018] 3, the environment display device 1 has a processor 20 and a memory 21. The memory 21 is, for example, composed of a volatile and / or non-volatile semiconductor memory and / or a hard disk device. The processor 20 is, for example, a computer equipped with a CPU and the like. The processor 20 may be configured to have a ROM in which a program is written, a RAM for temporarily storing data, and the like. The processor 20 has, as functional elements or functional units, an image acquisition section 23, an image processing section 24, and a display control section 25.

[0019] These functional elements of the processor 20 are realized, for example, by the processor 20, which is a computer, executing a display program 22 stored in the memory 21. The display program 22 can be stored in any computer-readable storage medium. Alternatively, all or part of the functional elements of the processor 20 can be configured by hardware including one or more electronic circuit components.

[0020] The image acquisition unit 23 acquires an image of the surrounding environment of the vehicle 2, including the blind spot of the passenger D of the vehicle 2, from the front camera 3. In this embodiment, the image acquired from the front camera 3 is a moving image at a predetermined frame rate.

[0021] The image processing unit 24 generates an environmental image including the blind spots of the occupant D of the vehicle 2 as seen from the viewpoint of the occupant D from the image of the surrounding environment of the vehicle 2 acquired by the image acquisition unit 23 from the front camera 3. In this embodiment, in particular, the image processing unit 24 displays, in the environmental image, a contour line indicating the range of the bottom, side, and front of the body of the vehicle 2 as seen by the passenger D. Information about the range of the bottom, side, and front of the body of the vehicle 2 (for example, coordinate information with the position of the projector 7 as the origin) may be stored in advance in the memory 21, for example.

[0022] Furthermore, the image processor 24 displays a surrounding image, which is an image of the environment surrounding the vehicle 2, in the image portion of the environmental image other than the inside of the contour line indicating the bottom range of the vehicle 2, and does not display the surrounding image inside the contour line indicating the bottom range. This provides a sense of security and a sense of the vehicle based on the mask width (such as the width of the bottom of the vehicle 2 indicated by the contour line) by masking the lower part of the vehicle body that is less important for driving and by allowing an object to move in front of the vehicle 2 along with the vehicle's own movement.

[0023] In addition, in the image portion of the environmental image where the surrounding image is displayed, a moving graphic, which is a group of fixed or irregular graphic elements that move in accordance with the movement of the surrounding environment flowing from the front to the rear of the vehicle 2, may be superimposed and displayed.

[0024] For example, the group of graphic elements of the moving graphic is a group of fixed or irregular points (e.g., a plurality of points having a finite size) that move radially from the vanishing point of the image of the surrounding environment in the environmental image on the ground plane on which the vehicle 2 is traveling. This can improve the accuracy of the passenger D's sense of vehicle speed regarding the forward / rearward and left / right movements of the vehicle 2.

[0025] The image processor 24 may also display images of the wheels of the vehicle 2 in the environmental image at the positions of the wheels of the vehicle 2 as seen by the passenger D. This makes it easier to grasp the positions of the wheels of the vehicle 2, and further supports the passenger D in making decisions regarding driving behavior.

[0026] When the vehicle speed of the vehicle 2 exceeds a predetermined first threshold, the image processor 24 may also extend the above-mentioned contour line forward along the future trajectory of the vehicle 2 and display it in the environmental image. The image processor 24 displays a surrounding image in an image portion of the environmental image other than the extended bottom surface range including at least the bottom surface of the vehicle 2 within the extended range defined (or "divided"; the same applies below) by the extended contour line, and does not display a surrounding image in the image portion of the extended bottom surface range. The future trajectory may be estimated based on the steering angle of the vehicle 2 acquired from the vehicle monitoring device 6, for example.

[0027] As a result, when the vehicle speed of the vehicle 2 exceeds the first threshold, the contour line is extended along the future trajectory of the vehicle, and the environmental information in the direction of the extended bottom surface of the vehicle body indicated by the extended contour line is masked. As a result, the passenger D can easily direct his / her gaze forward along the future trajectory that should be focused on when traveling at high speed, and can obtain a sense of the vehicle based on the mask width.

[0028] Here, when the contour line is extended as described above, the extension distance of the contour line can be set to be longer as the vehicle speed of the host vehicle 2 increases. As a result, the extension distance of the contour line along the future trajectory of the host vehicle 2 increases as the vehicle speed of the host vehicle 2 increases, so that the line of sight of the passenger D can be guided further forward in accordance with the vehicle speed.

[0029] The image processing unit 24 may also, when the vehicle speed of the host vehicle 2 exceeds a predetermined second threshold, display a surrounding image in an image portion of the environmental image that is within the range defined by the contour line or the extended contour line, other than the bottom and side ranges of the host vehicle 2, or other than the extended bottom and side ranges, and may not display a surrounding image in the image portion of the bottom and side ranges or the image portion of the extended bottom and side ranges. As a result, when the vehicle speed of the vehicle 2 exceeds the second threshold value, environmental information from the side of the vehicle 2 is also eliminated, allowing the occupant D to be more clearly aware of the vehicle sensation based on the area of ​​the bottom and both sides of the vehicle 2 where the surrounding image is masked from the environmental image.

[0030] The display control unit 25 uses the projector 7, which is a display device, to display the environmental image generated by the image processing unit on the inner surface of the cabin of the vehicle 2. The display control unit 25 instructs the projector 7 to display the environmental image on the inner surface of the cabin so that the positions of the outer edges of the range of the bottom, side, and front of the vehicle 2 as seen by the passenger D coincide with the display position of the contour line.

[0031] This makes it possible to further improve the accuracy of the occupant D's sense of the vehicle body within the range of the vehicle's body centered on the occupant D.

[0032] [3. Example of displayed environment image] Next, an example of an environmental image displayed by the environmental display device 1 will be described with reference to Fig. 5, Fig. 6, and Fig. 7. The environmental images shown in Fig. 5, Fig. 6, and Fig. 7 are images projected from the projector 7 and displayed in the range of the inner surface of the vehicle interior shown as the display range DA in Fig. 2. In Fig. 5, Fig. 6, and Fig. 7, the display range DA extends downward from the dashed line shown at the upper edge of the windshield 13. In addition, in the range shown in Fig. 5, Fig. 6, and Fig. 7, the display control unit 25 of the environmental display device 1 controls the projector 7 so that the environmental image is not projected onto the ceiling, the rearview mirror 15, the right pillar 16b, and the right door mirror 4b of the vehicle 2.

[0033] FIG. 5 is an example of an environmental image that is displayed when the vehicle speed Ve of the host vehicle 2 is equal to or lower than the first threshold value Vth1 and equal to or lower than the second threshold value Vth2 (when Ve≦Vth1, Vth2).

[0034] As shown in Fig. 5, the environmental image includes a display of the surrounding environment of the host vehicle 2 acquired from the forward camera 3. In the example of Fig. 5, a surrounding image including another vehicle 33 traveling to the left front of the host vehicle 2, a utility pole 35, and a building 36 is displayed.

[0035] The environmental image also displays a contour line 31 indicating the bottom range AB, left side range AL, front range AF, and right side range AR of the vehicle body of the vehicle 2 as seen by the passenger D. By displaying this contour line 31, the accuracy of the passenger D's vehicle body sense (sense of how far the vehicle body extends) of the vehicle 2 is improved.

[0036] 5, in addition to the contour line 31, wheels 34 are displayed at the positions of the two front wheels of the vehicle 2 as seen by the passenger D. This further improves the passenger D's sense of the vehicle body.

[0037] Furthermore, in the environmental image, a moving graphic consisting of a plurality of dots 32 of indefinite shape that move in accordance with the movement of the surrounding environment flowing from the front to the rear of the vehicle 2 is displayed in the image portion where the surrounding image is displayed. Here, a "dot" refers to a point having a finite size. In other words, the moving graphic in FIG. 5 is composed of a point cloud.

[0038] Note that in Figures 5, 6, and 7, only one dot is given the symbol "32," but it should be understood that the various black dots of different sizes shown below the horizon GL are dots 32 that make up the moving graphic.

[0039] These dots 32 move radially in a direction spreading out from the vanishing point P on the horizontal ground surface on which the vehicle 2 is traveling, downward from any position in the environmental image to the left or right of the vanishing point P and the horizon GL indicated by the dotted line in the drawing. The moving speed of the dots 32 can be displayed to be faster as the vehicle speed Ve of the vehicle 2 is faster, and can also be displayed to be slower as the dots are closer to the horizon GL (i.e., faster as the dots are closer to the vehicle 2). This allows the passenger D to intuitively understand the speed of the vehicle 2 from the movement of the dots 32, improving the accuracy of the passenger D's sense of vehicle speed.

[0040] Here, the positions of the vanishing point P and the horizon GL can be estimated according to conventional technology based on the movement direction of each object in the image of the surrounding environment acquired from the front camera 3. For example, the image processing unit 24 can estimate the position of the vanishing point P as the intersection of each line indicating the direction of movement of each of the above-mentioned objects, and estimate the horizontal line passing through the vanishing point P or its vicinity as the horizon GL.

[0041] 5, the surrounding image is displayed in an image portion of the environmental image other than the bottom range AB of the vehicle 2 defined by the contour line 31. This allows the passenger D to feel secure and get a sense of the vehicle based on the width of the mask by masking the image of the bottom of the vehicle body, which is less important for driving, and by having something moving along with the passenger D in front of the vehicle 2. The bottom range AB where the surrounding image is not displayed may be a non-display area where no image is displayed, or may be an area where a predetermined image, such as a static pattern having a specific geometric shape, is displayed.

[0042] Fig. 6 is an example of an environmental image that is displayed when the first threshold value Vth1 is greater than the second threshold value Vth2 (Vth1>Vth2) and the vehicle speed Ve of the vehicle 2 exceeds the second threshold value Vth2 but does not exceed the first threshold value Vth1 (Vth1≧Ve>Vth2). In Fig. 6, the same elements as those in the environmental image shown in Fig. 5 are indicated by the same reference numerals as those in Fig. 5, and the explanation of Fig. 5 above is incorporated. The same applies to Figs. 7 and 8 described below.

[0043] In FIG. 6, since the vehicle speed Ve of the host vehicle 2 exceeds the second threshold value Vth2, the surrounding image is displayed in the image portion of the environmental image other than the image portion showing the bottom surface range AB, the left side surface range AL, and the right side surface range AR within the range defined by the contour line 31. As a result, environmental information in the lateral direction of the host vehicle 2 is also excluded, so that the passenger D can be more clearly aware of the vehicle sensation based on the range of the bottom surface and both sides of the host vehicle 2 in which the surrounding image is masked in the environmental image. Note that the bottom surface range AB, the left side surface range AL, and the right side surface range AR in which the surrounding image is not displayed can be non-display areas in which no image is displayed, or areas in which a predetermined image such as a specific geometric pattern is displayed.

[0044] When the surrounding image is displayed in an image portion other than the bottom range AB, the left side range AL, and the right side range AR, the above-mentioned moving graphic can also be displayed in the image portion other than the bottom range AB, the left side range AL, and the right side range AR. In the example of Fig. 6, the surrounding image and the moving graphic are displayed only in the image portion other than the bottom range AB, the left side range AL, and the right side range AR.

[0045] FIG. 7 is an example of an environmental image displayed when the first threshold Vth1 is smaller than the second threshold Vth2 and the vehicle speed Ve of the host vehicle 2 exceeds the first threshold Vth1 but does not exceed the second threshold Vth2 (when Vth2≧Ve>Vth1).

[0046] In Fig. 7, since the vehicle speed Ve exceeds the first threshold value Vth1, the contour line 31 shown in Fig. 5 is extended forward of the host vehicle 2 along the future trajectory of the host vehicle 2 and displayed as a virtual contour line 41. In Fig. 7, the bottom range AB, left side range AL, front range AF, and right side range AR of the body of the host vehicle 2 shown in Fig. 5 are also extended forward of the host vehicle 2 along the future trajectory, and the virtual bottom range vAB, virtual left side range vAL, virtual front range vAF, and virtual right side range vAR are partitioned by the virtual contour line 41. Note that, as described above, the extension distance of the contour line 31 forward of the host vehicle 2 when constructing the virtual contour line 41 can be set to be longer as the vehicle speed Ve of the host vehicle 2 is faster.

[0047] Here, the imaginary contour line 41 corresponds to the extended contour line in the present disclosure. Also, the imaginary bottom surface range vAB, the imaginary left side surface range vAL, the imaginary front surface range vAF, and the imaginary right side surface range vAR correspond to the extended bottom surface range, the extended left side surface range, the extended front surface range, and the extended right side surface range, respectively, in the present disclosure.

[0048] 7, a surrounding image including other vehicles 33, utility poles 35, and buildings 36 is displayed in the image portion of the environmental image excluding the virtual bottom range vAB defined by the virtual contour line 41. This makes it easier for the passenger D to direct his / her gaze forward along the future trajectory that should be observed when traveling at high speed, and also allows the passenger D to obtain a sense of the vehicle based on the mask width.

[0049] Fig. 8 is an example of an environmental image displayed when the vehicle speed Ve of the vehicle 2 exceeds the first threshold value Vth1 and the second threshold value Vth2 (when Ve>Vth1, Vth2). In Fig. 8, the same elements as those in the environmental image shown in Fig. 7 are indicated by the same reference numerals as those in Fig. 7, and the above description of Fig. 7 is used.

[0050] In Fig. 8, since the vehicle speed Ve exceeds the first threshold value Vth1, similarly to Fig. 7, the contour line 31 shown in Fig. 5 is extended forward of the host vehicle 2 along the future trajectory of the host vehicle 2 and displayed as a virtual contour line 41. In addition, this virtual contour line 41 defines a virtual bottom range vAB, a virtual left side range vAL, a virtual front range vAF, and a virtual right side range vAR that extend along the future trajectory.

[0051] 8, since the vehicle speed Ve of the host vehicle 2 also exceeds the second threshold value Vth2, the surrounding image is displayed in the image portion excluding the virtual bottom range vAB, the virtual left side range vAL, and the virtual right side range vAR within the range defined by the virtual contour line 41. This allows the occupant D to be more clearly aware of the vehicle sensation based on the range of the bottom and both sides of the host vehicle 2 where the surrounding image is masked within the environmental image.

[0052] [4. Processing in the environmental display device] Next, the procedure of the operation of the environment display device 1 will be described. 4 is a flow diagram showing the procedure of a display method executed by the processor 20, which is the computer of the environment display device 1, for displaying the surrounding environment of the vehicle 2. This process starts, for example, when the environment display device 1 is powered on while the front camera 3, the projector 7, and the vehicle monitoring device 6 are powered on.

[0053] When the process starts, first, the image acquisition unit 23 starts acquiring an image of the surrounding environment of the vehicle 2 including the blind spot as seen by the passenger D, by using the forward camera 3 (S100). Here, the image processing unit 24 may start generating a moving graphic to be superimposed on the environmental image in a step described later. As described above, the moving graphic is composed of a plurality of dots 32 that move at a speed corresponding to the vehicle speed Ve of the vehicle 2, for example. The vehicle speed Ve is acquired from the vehicle monitoring device 6 at a predetermined time interval, for example, and the moving graphic can be generated so that the speed of the dots 32 changes according to the change in the vehicle speed Ve (so that the moving speed of the dots 32 increases as the vehicle speed Ve increases).

[0054] Next, the image processing unit 24 generates an environmental image including an image of the blind spot seen by the passenger D based on the acquired image of the surrounding environment (S102). The image processing unit 24 also displays the contour line 31 of the vehicle body of the vehicle 2 in the environmental image (S104). The image processing unit 24 also displays a surrounding image, which is an image of the surrounding environment of the vehicle 2, in an image portion of the environmental image excluding the bottom range AB of the vehicle body of the vehicle 2 defined by the contour line 31 (S106). As described above, the image processing unit 24 may superimpose a moving graphic on the portion of the environmental image where the surrounding image is displayed. The same applies to steps S112 and S116 described later.

[0055] Next, the image processing unit 24 judges whether the vehicle speed Ve exceeds the first threshold value Vth1 (S108). If the vehicle speed Ve does not exceed the first threshold value Vth1 (S108, NO), the image processing unit 24 proceeds to step S114, which will be described later. On the other hand, if the vehicle speed Ve exceeds the first threshold value Vth1 (S108, YES), the image processing unit 24 displays a virtual contour line 41, which is an extension of the contour line 31 along the future trajectory of the vehicle 2, superimposed on the environmental image in place of the contour line 31 (S110). In addition, the image processing unit 24 displays a surrounding image in an image portion of the environmental image excluding the virtual bottom range vAB defined by the virtual contour line 41 (S112).

[0056] Next, the image processing unit 24 judges whether the vehicle speed Ve exceeds the second threshold value Vth2 (S114). When the vehicle speed Ve does not exceed the second threshold value Vth2 (S114, NO), the image processing unit 24 proceeds to step S118, which will be described later. On the other hand, when the vehicle speed Ve exceeds the second threshold value Vth2 (S114, YES), the image processing unit 24 displays the surrounding image in an image range excluding the bottom range AB, the left side range AL, and the right side range AR, or in an image portion excluding the virtual bottom range vAB, the virtual left side range vAL, and the virtual right side range vAR, depending on whether the contour line 31 or the virtual contour line 41 is displayed in the environmental image (S116). In step S116 shown in FIG. 4, the left side range AL and the right side range AR are abbreviated as "side ranges", and the virtual left side range vAL and the virtual right side range vAR are abbreviated as "virtual side ranges".

[0057] Next, the image processing unit 24 determines whether the vehicle speed Ve exceeds the first threshold value Vth1 or the second threshold value Vth2 (S118). If the vehicle speed Ve exceeds the first threshold value Vth1 or the second threshold value Vth2 (S118, YES), the image processing unit 24 returns the process to step S108.

[0058] On the other hand, when the vehicle speed Ve does not exceed the first threshold value Vth1 or the second threshold value Vth2 (S118, NO), the image processor 24 judges whether the power supply of the environment display device 1 has been turned off (S120). Then, when the power supply of the environment display device 1 has not been turned off (S120, NO), the image processor 24 returns the process to step S104. On the other hand, when the power supply of the environment display device 1 has been turned off (S120, NO), the image processor 24 ends this process.

[0059] 5. Other embodiments

[0060] The group of graphic elements of the moving graphic is not limited to the group of points shown in the above embodiment. The group of graphic elements of the moving graphic may be a group of lines parallel to the horizon line GL that move from the position of the horizon line GL on the ground plane on which the host vehicle 2 is traveling in the environmental image.

[0061] The display device used by the display control unit 25 of the environmental display device 1 to display the environmental image is not limited to the projector 7 that projects an image onto the inner surface of the passenger compartment of the vehicle 2 as shown in the above-mentioned embodiment, but may be any type of display device.

[0062] For example, the display device may be a head-mounted display that is worn on a person's head instead of the projector 7. For example, the head-mounted display may provide an augmented reality (AV) display in which an image is superimposed on the real world visually recognized by the passenger D. The display control unit 25 may instruct the head-mounted display to display an environmental image such that the position of the outer edge of the range of the bottom, side, and front of the vehicle body of the vehicle 2 as viewed by the passenger D coincides with the display position of the contour line 31.

[0063] When a head-mounted display is used as the display device, the environment display device 1 can be provided inside the housing of the head-mounted display. The environment display device 1 provided inside the housing can obtain information from in-vehicle devices such as a forward camera 3 and a vehicle monitoring device 6 by wireless communication.

[0064] Alternatively, the display device used by the display control unit 25 of the environment display device 1 to display the environment image may be a general-purpose monitor device 11 or a head-up display 17 instead of the projector 7 .

[0065] The contour line 31 and the virtual contour line 41 are not limited to a group of straight lines that define the bottom range AB, etc. and the virtual bottom range vAB, etc. as a rectangular range as in the above-mentioned embodiment, but may be a group of straight lines that define the bottom range AB, etc. and the virtual bottom range vAB, etc. as a rectangular range. Alternatively, the contour line 31 and the virtual contour line 41 may be curved lines that more faithfully represent the shape of the outer edge of the body of the host vehicle 2.

[0066] When the image processing unit 24 extends the contour line 31 along the future trajectory of the vehicle 2 to generate the virtual contour line 41, if the vehicle 2 is equipped with a navigation device (not shown), the future trajectory may be estimated from the route set by the navigation device.

[0067] The present invention is not limited to the configurations of the above-described embodiments, and can be embodied in various forms without departing from the spirit and scope of the present invention.

[0068] [6. Configuration supported by the above embodiment] The above-described embodiment supports the following configurations.

[0069] (Configuration 1) An environmental display device comprising: an image processing unit that generates an environmental image including a blind spot of a passenger of the vehicle from an image of the environment surrounding the vehicle, as seen from the viewpoint of the passenger of the vehicle; and a display control unit that displays the environmental image on a display device, wherein the image processing unit displays a contour line indicating the range of the bottom, sides, and front of the body of the vehicle as seen from the passenger's perspective in the environmental image, and displays the surrounding image, which is an image of the environment surrounding the vehicle, in an image portion of the environmental image other than the inside of the contour line indicating the range of the bottom surface, and does not display the surrounding image in an image portion inside the contour line indicating the range of the bottom surface. According to the environmental display device of configuration 1, by masking the lower part of the vehicle body which is less important for driving and by creating a masked area in front of the vehicle which moves along with the vehicle's movement, the passengers can obtain a sense of security and a sense of the vehicle based on the width of the mask.

[0070] (Configuration 2) When the vehicle speed of the host vehicle exceeds a predetermined first threshold, the image processing unit extends the contour line forward of the host vehicle along a future trajectory of the host vehicle, and displays the surrounding image in an image portion of the environmental image other than an image portion showing at least the extended range of the bottom surface of the host vehicle within the extended range defined by the extended contour line, and does not display the surrounding image in an image portion of the extended bottom surface range. According to the environmental display device of configuration 2, when the vehicle speed exceeds the first threshold, the contour line is extended along the future trajectory of the vehicle, and environmental information in the direction of the extended bottom surface of the vehicle body indicated by the extended contour line is masked, so that the passenger can easily direct their gaze forward along the future trajectory that they should pay attention to when driving at high speed, and can obtain a sense of the vehicle based on the mask width.

[0071] (Configuration 3) The environmental display device according to configuration 2, wherein the distance over which the contour line is extended is longer as the vehicle speed of the host vehicle is higher. According to the environmental display device of configuration 3, the faster the vehicle speed is, the longer the extension distance of the contour line along the vehicle's future trajectory becomes, so that the passenger's line of sight can be guided further forward in accordance with the vehicle speed.

[0072] (Configuration 4) When the vehicle speed of the host vehicle exceeds a predetermined second threshold, the image processing unit displays the surrounding image in an image portion of the environmental image other than the image portion showing the range of the bottom and side of the host vehicle within the range defined by the contour line, and does not display the surrounding image in the image portion showing the range of the bottom and side. According to the environmental display device of configuration 4, when the vehicle speed exceeds the second threshold, environmental information on the sides of the vehicle is also eliminated, so that the passenger can be more clearly aware of the vehicle sensation based on the range of the bottom and both sides of the vehicle where the surrounding image is masked from the environmental image.

[0073] (Configuration 5) An environmental display device as described in configuration 2 or 3, wherein, when the vehicle speed of the vehicle exceeds a predetermined second threshold, the image processing unit displays the surrounding image in an image portion of the environmental image other than the image portion showing the extended bottom and side ranges of the vehicle within the range defined by the extended contour line, and does not display the surrounding image in the image portion showing the extended bottom and side ranges. According to the environmental display device of configuration 5, when the vehicle speed exceeds the first and second thresholds, the extended lateral environmental information is also eliminated, so that the passenger can be more clearly aware of the vehicle sensation based on the range of the bottom and both sides of the vehicle where the surrounding image is masked from the environmental image.

[0074] (Configuration 6) The environmental display device according to any one of configurations 1 to 6, wherein the image processing unit displays an image of wheels in the environmental image at a position of the wheels of the host vehicle as seen by the passenger. According to the environment display device of configuration 6, the wheel positions of the vehicle can be easily grasped, and the passenger's judgment regarding driving behavior can be further supported.

[0075] (Configuration 7) The environmental display device according to any one of configurations 1 to 6, wherein the image processing unit displays a moving graphic, which is a group of fixed or irregular graphic elements that move in accordance with the movement of the surrounding environment flowing from the front to the rear of the vehicle, superimposed on an image portion of the environmental image in which the surrounding image is displayed. According to the environmental display device of configuration 7, the moving graphics are displayed, so that the passenger can also improve the sense of the vehicle speed of the vehicle.

[0076] (Configuration 8) The environmental display device described in Configuration 7, wherein the group of graphic elements of the moving graphic is a group of fixed or irregular points in the environmental image that move from a vanishing point of the image of the surrounding environment across the ground plane on which the vehicle is traveling. According to the environment display device of configuration 8, it is possible to improve the accuracy of the passenger's sense of vehicle speed with respect to the movement of the host vehicle in the forward / backward and left / right directions.

[0077] (Configuration 9) 8. The environmental display device according to claim 7, wherein the group of graphic elements of the movement graphic is a group of lines parallel to the horizon in the environmental image, which move from a position on the horizon along the horizontal plane on which the host vehicle is traveling. According to the environment display device of configuration 9, it is possible to improve the accuracy of the passenger's sense of vehicle speed with respect to the longitudinal movement of the vehicle.

[0078] (Configuration 10) The environmental display device according to any one of configurations 1 to 9, wherein the display device is a projector that displays an image on the inner surface of the passenger compartment of the vehicle. According to the environmental display device of configuration 10, an environmental image is displayed on the interior surface of the vehicle cabin, thereby enhancing the sense of reality of the environmental image for passengers.

[0079] (Configuration 11) The display control unit instructs the display device, which is the projector, to display the environmental image on the inner surface of the vehicle cabin so that the position of the outer edge of the bottom, side, and front areas of the vehicle body as seen by the passenger coincides with the display position of the contour line.An environmental display device as described in Configuration 10. According to the environment display device of configuration 11, the accuracy of the passenger's vehicle body sensation within the range of the vehicle body of the host vehicle centered on the passenger can be further improved.

[0080] (Configuration 12) The environment display device according to any one of configurations 1 to 9, wherein the display device is a head-mounted display worn on a person's head. According to the environment display device of configuration 12, since there is no need to mount a display device inside the vehicle cabin, the configuration of the vehicle can be simplified.

[0081] (Configuration 13) The head-mounted display, which is the display device, provides an augmented reality display in which an image is superimposed on the real world viewed by the occupant, and the display control unit instructs the display device, which is the head-mounted display, to display the environmental image so that the position of the outer edge of the bottom, side, and front areas of the vehicle's body as seen by the occupant coincides with the display position of the contour line.An environmental display device as described in Configuration 12. According to the environment display device of configuration 13, it is possible to further improve the accuracy of the passenger's sense of the range of the vehicle body of the vehicle centered on the passenger (that is, the vehicle body sense).

[0082] (Configuration 14) The environmental display device according to any one of configurations 1 to 9, wherein the display device is a monitor device mounted on the vehicle. According to the environmental display device of configuration 14, since the environmental image can be displayed on the monitor device that is normally provided in the vehicle, the configuration of the vehicle can be simplified.

[0083] (Configuration 15) The environment display device according to any one of configurations 1 to 9, wherein the display device is a head-up display. According to the environmental display device of configuration 15, a passenger in the vehicle can visually recognize the real environment within the passenger's field of vision while also obtaining information from the environmental image.

[0084] (Configuration 16) An environment display system comprising the display device, an imaging device that captures an image of the surrounding environment of the vehicle, and the environment display device according to any one of configurations 1 to 15. According to the environmental display system of configuration 16, by masking the lower part of the vehicle body that is less important for driving and by creating a masked area in front of the vehicle that moves along with the passenger's movement, the passenger can obtain a sense of security and a sense of the vehicle based on the width of the mask. [Explanation of symbols]

[0085] 1...environmental display device, 2...own vehicle, 3...forward camera, 4a...left door mirror, 4b...right door mirror, 5...object detection device, 6...vehicle monitoring device, 7...projector (display device), 8...environmental display system, 10...steering wheel, 11...monitor device, 12...instrument panel, 13...windshield, 14a...left door window, 14b...right door window, 15...rear-view mirror, 16a...left pillar, 16b...right pillar, 17...head-up display, 20...processor, 21 ...Memory, 22...Display program, 23...Image acquisition unit, 24...Image processing unit, 25...Display control unit, 30...Roadside line, 31...Contour line, 32...Dot (moving graphic), 33...Other vehicles, 34...Wheels, 35...Utility pole, 36...Building, 41...Virtual contour line, AB...Bottom range, AF...Front range, AL...Left side range, AR...Right side range, D...Occupant, DA...Display range, GL...Horizon, P...Vanishing point, vAB...Virtual bottom range, vAF...Virtual front range, vAL...Virtual left side range, vAR...Virtual right side range.

Claims

1. an image processing unit that generates an environmental image including a blind spot of a passenger of the vehicle from an image of a surrounding environment of the vehicle; a display control unit that displays the environmental image on a display device; Equipped with The image processing unit includes: A contour line showing the range of the bottom, side, and front of the vehicle body of the vehicle as seen by the passenger is displayed on the environmental image, and a surrounding image, which is an image of the surrounding environment of the vehicle, is displayed in an image portion of the environmental image other than an inside of the contour line indicating the range of the bottom surface, and the surrounding image is not displayed in an image portion inside the contour line indicating the range of the bottom surface. Environmental display device.

2. The image processing unit includes: When a vehicle speed of the host vehicle exceeds a predetermined first threshold, the contour line is extended forward of the host vehicle along a future trajectory of the host vehicle; In the environmental image, the surrounding image is displayed in an image portion other than an image portion showing at least the extended range of the bottom surface of the vehicle within an extended range defined by the extended contour line, and the surrounding image is not displayed in an image portion of the extended bottom surface range. The environment display device according to claim 1 .

3. The distance by which the contour line is extended is longer as the vehicle speed of the host vehicle is higher. The environment display device according to claim 2 .

4. The image processing unit includes: when the vehicle speed of the host vehicle exceeds a predetermined second threshold, the surrounding image is displayed in an image portion of the environmental image other than an image portion showing the range of the bottom surface and the side surface of the host vehicle within a range defined by the contour line, and the surrounding image is not displayed in the image portion showing the range of the bottom surface and the side surface. The environment display device according to claim 1 .

5. The image processing unit includes: when the vehicle speed of the host vehicle exceeds a predetermined second threshold, the surrounding image is displayed in an image portion of the environmental image other than an image portion showing the extended bottom and side ranges of the host vehicle within a range defined by the extended contour line, and the surrounding image is not displayed in the image portion showing the extended bottom and side ranges. The environment display device according to claim 2 .

6. The image processing unit includes: a wheel image is displayed on the environmental image at a position of the wheel of the host vehicle as seen by the passenger; The environment display device according to claim 1 .

7. The image processing unit displays a moving graphic, which is a group of fixed or non-fixed shape graphic elements that move in accordance with the movement of the surrounding environment flowing from the front to the rear of the vehicle, in a superimposed manner on an image portion of the environmental image in which the surrounding image is displayed. The environmental display device according to claim 1 .

8. The group of graphic elements of the moving graphic is a group of fixed or irregular points that move on the ground plane on which the host vehicle runs from a vanishing point of the image of the surrounding environment in the environmental image. The environment display device according to claim 7 .

9. the group of graphic elements of the movement graphic is a group of lines parallel to the horizon, which move from a position of the horizon along the ground on which the host vehicle is traveling in the environmental image; The environment display device according to claim 7 .

10. The display device is a projector that displays an image on the inner surface of the vehicle interior. The environmental display device according to claim 1 .

11. The display control unit instructs the display device, which is the projector, to display the environmental image on an inner surface of the vehicle compartment so that the position of an outer edge of the range of the bottom, side, and front of the vehicle body of the vehicle as seen by the passenger coincides with the display position of the contour line. The environment display device according to claim 10.

12. The display device is a head-mounted display that is worn on a person's head. The environmental display device according to claim 1 .

13. The head mounted display, which is the display device, provides an augmented reality display in which an image is superimposed on a real world visually recognized by the passenger, The display control unit instructs the display device, which is the head mounted display, to display the environmental image so that the position of an outer edge of the range of the bottom surface, side surface, and front surface of the vehicle body of the vehicle as seen by the passenger coincides with the display position of the contour line. The environmental display device according to claim 12 .

14. The display device is a monitor device mounted on the vehicle. The environmental display device according to claim 1 .

15. The display device is a head-up display. The environmental display device according to claim 1 .

16. The display device; An imaging device for acquiring an image of the surrounding environment of the vehicle; An environment display device according to any one of claims 1 to 15, An environmental display system comprising:

Images