Display System

The AR-based display system addresses passenger discomfort by masking parallel vehicles on vehicle windows, allowing external visibility and improved travel experience.

JP7596039B2Active Publication Date: 2024-12-09ALPS ALPINE CO LTD
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
JP2021115638
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-07-13
Publication Date
2024-12-09
Estimated Expiration
2041-07-13

AI Technical Summary

Technical Problem

Passengers in fully automated vehicles experience discomfort due to the fear of being viewed through vehicle windows by other cars, while obstructing external visibility with traditional privacy measures like curtains or films.

Method used

A display system using AR technology superimposes images on vehicle windows to create a virtual view, masking the presence of parallel vehicles and maintaining external visibility.

Benefits of technology

The system effectively eliminates the discomfort of being viewed from outside while ensuring passengers can see outside, enhancing the enjoyment of travel and maintaining situational awareness.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007596039000001
    Figure 0007596039000001
  • Figure 0007596039000002
    Figure 0007596039000002
  • Figure 0007596039000003
    Figure 0007596039000003
Patent Text Reader

Abstract

To provide a display system that can remove unpleasantness caused by being seen from the outside.SOLUTION: A display system 100 of the present invention is equipped in an own vehicle 10 and has a function of displaying a picture in a window 12 of the own vehicle 10. The display system 100 displays animation pictures 300 and 310 in positions on the window 12 which overlap with a vehicle 20 moving side by side with the own vehicle 10 in the vision of passengers 14A and 14B when a vehicle 20 moving side by side with the vehicle 10 is detected, and prevents the passengers 14A and 14B from seeing the vehicle 20. In that way, a comfortable space in a vehicle can be provided.SELECTED DRAWING: Figure 7
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a display system mounted on a mobile object such as an automobile. [Background technology]

[0002] As a display control system for generating a virtual environment inside a vehicle, Patent Document 1 discloses a technology for displaying traffic conditions, weather conditions, and the like on the vehicle window while the vehicle is moving along the current travel route. Patent Document 2 discloses a display system for identifying the driving conditions of other vehicles around the vehicle and making the driver aware of them. For example, an image of the other vehicle is displayed so as to be superimposed on the view seen through the windshield from the driver's seat. Furthermore, Patent Document 3 discloses a communication vehicle display device that, when the source of a transmission is another vehicle visible from the vehicle window, displays the source vehicle in the driver's field of view through the vehicle window. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-515886 A [Patent Document 2] JP 2017-37634 A [Patent Document 3] JP 2010-218568 A Summary of the Invention [Problem to be solved by the invention]

[0004] With advances in autonomous driving technology, the interior of a vehicle is changing from a space for travel to a space for comfort. When we move to fully autonomous driving, it is predicted that passengers will spend time in the vehicle as if they were in their living room or workplace.

[0005] In the case of fully automated driving, it is expected that cars will run at legal speeds, and therefore it is expected that there will be more situations where cars run side by side. For example, as shown in Fig. 1, when there is another car 20 running side by side with the own car 10, the interiors of each car will be clearly visible, and passengers 14A and 14B can see the other car 20 from the car window 12. Even if the passenger 22 of the other car 20 is not actually looking at the passenger, the passengers will still be bothered by the line of sight. In particular, if the passenger is spending the time feeling as if they are in the living room of their home, they will feel uncomfortable as if someone is looking into their house through the window 12.

[0006] On the other hand, it is possible to cover the window 12 with a curtain or film so that the inside cannot be seen from the outside, but this makes it difficult for the passengers 14A and 14B to see outside, which reduces the enjoyment of traveling. Furthermore, even in a fully automated driving system, it is important for the driver to be aware of the external situation.

[0007] An object of the present invention is to provide a display system that can solve such conventional problems and eliminate the discomfort caused by external gazes. [Means for solving the problem]

[0008] The display system of the present invention is mounted on a vehicle such as a moving body, and comprises detection means for detecting a parallel traveling vehicle, display means for displaying an image on a window of the vehicle, and display control means for controlling the display means, wherein when a parallel traveling vehicle is detected by the detection means, the display control means displays an image at a position on the window that overlaps with the parallel traveling vehicle.

[0009] In one aspect, the display control means includes a calculation means for calculating a vehicle size of the parallel running vehicle, and displays an image according to the vehicle size calculated by the calculation means. In one aspect, the display control means includes a detection means for detecting a relative position of the parallel running vehicle with respect to the vehicle, and a detection means for detecting an eye point of a passenger, and the display control means displays the image based on the relationship between the relative position and the eye point. In one aspect, the display control means includes an identification means for identifying a passenger, and a storage means for storing images according to the preferences of a plurality of passengers, and the display control means acquires an image according to the preferences of the identified passenger from the storage means, and displays the acquired image. In one aspect, the display means displays the image in an AR format on an external view seen through a window of the vehicle. In one aspect, the display system further includes an imaging means for capturing an image of the periphery of the vehicle, and the display means displays an image of the periphery of the vehicle captured by the imaging means on the window, and the display control means combines the image with the periphery image displayed on the window. Effect of the Invention

[0010] According to the present invention, when a parallel running vehicle is detected, an image is displayed at a position above the window overlapping with the parallel running vehicle, providing an in-vehicle space where passengers do not feel gazes from the parallel running vehicle, and eliminating discomfort felt by passengers due to gazes from outside. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram for explaining a problem to be solved by the invention. [Diagram 2] FIG. 1 is a diagram showing a configuration of a display system according to an embodiment of the present invention. [Diagram 3] 13 is an example of a management table showing a relationship between user identification and animation data. [Figure 4] 1 is a block diagram showing a configuration of a line of sight removal function of a display system according to an embodiment of the present invention. [Diagram 5] 11 is a diagram showing an example of calculation of the distance and direction between the host vehicle and a vehicle traveling parallel to the host vehicle; FIG. [Figure 6] 11 is a flow diagram illustrating the operation of a line of sight removal function of a display system according to an embodiment of the present invention. [Figure 7] FIG. 1 is a diagram showing an example of AR display of a display system according to an embodiment of the present invention. [Figure 8] 10A to 10C are diagrams illustrating an example of AR display of a display system according to a modified example of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Next, an embodiment of the present invention will be described. The display system of the present invention is mounted on a vehicle such as an automobile, and has a function of displaying images on the vehicle windows, etc. The display system of the present invention uses AR (Augmented Reality) technology to superimpose images onto surrounding vehicles (vehicles traveling parallel to the vehicle) to provide an in-vehicle space in which the presence of people is erased and the driver does not feel the gaze of the vehicles traveling parallel to the vehicle. The AR technology generates a virtual field of view by superimposing virtual visual information on a real landscape. In addition, by synthesizing images only onto the vehicles traveling parallel to the vehicle, a field of view that does not impede understanding of the situation outside the vehicle is ensured. EXAMPLES

[0013] Next, an embodiment of the present invention will be described in detail. Fig. 2 is a block diagram showing the configuration of a display system according to an embodiment of the present invention. The display system 100 includes a plurality of imaging cameras 110, a sensor 120, a vehicle information acquisition unit 130, a communication unit 140, a display unit 150, a storage unit 160, and a control unit 170. Although not shown here, the display system 100 can also include an input unit that receives input from a user, an audio output unit that outputs audio related to the image displayed by the display unit 150, and a media playback unit that plays media such as music and images.

[0014] The imaging camera 110 captures an image of the surroundings of the vehicle and provides the captured image data to the control unit 170. This image data can be displayed by the display unit 150 or can be used to detect surrounding vehicles and calculate the size of surrounding vehicles. The imaging cameras 110 include an in-vehicle imaging camera that captures an image of the interior space of the vehicle in addition to an imaging camera that captures an image of the exterior of the vehicle. The image data captured by the in-vehicle imaging camera is used to detect the presence or absence of a passenger in the vehicle and the passenger's viewpoint (eye point).

[0015] The sensor 120 includes, for example, an infrared sensor, a radar, etc., and detects objects present around the vehicle and the distance to the objects by transmitting electromagnetic waves such as electric waves and receiving the reflected waves. The detection result is provided to the control unit 170. The sensor 120 can also include an illuminance sensor for detecting day / night. The detection result of the illuminance sensor is used to adjust the brightness of the display unit 150, etc.

[0016] The vehicle information acquisition unit 130 acquires information related to the operation of the vehicle. The vehicle information may include, for example, information identifying the level of autonomous driving, as well as other information such as vehicle speed information (vehicle speed pulse), gear position information, parking information, blinker information, and vehicle position information, and this information is acquired via an in-vehicle bus or the like. The vehicle position information may include, for example, position information using a GPS satellite, and position information using an acceleration sensor or an angular velocity sensor. The information acquired by the vehicle information acquisition unit 130 is provided to the control unit 170.

[0017] The communication unit 140 enables wired or wireless communication with electronic devices mounted in the vehicle, enables wireless communication with a server or the like via an external network, and enables vehicle-to-vehicle communication with other vehicles. In one aspect, the communication unit 140 is capable of downloading an image to be displayed by the display unit 150 from a server or the like.

[0018] The display unit 150 has a function of displaying an image on a vehicle window or an image in the vehicle interior space other than the window. As shown in FIG. 1, the vehicle is provided with a glass window 12 attached to the side of the seat, as well as a windshield and a rear glass window. In this embodiment, the display unit 150 displays an image at least on a window (window 12 in the example of FIG. 1) where the line of sight of a parallel vehicle is felt when there is a parallel vehicle. The configuration of the display unit 150 is not particularly limited, but in one embodiment, it includes a projector that projects an image, and projects an image on the glass window 12 to superimpose the projected image on the scenery seen through the window 12. In another embodiment, the display unit 150 is mounted on the window 12 with a display panel such as a liquid crystal display that corresponds to the shape of the window 12. For example, the liquid crystal display has a transparent state that allows the scenery outside the vehicle to be seen through when no image is displayed, and displays an image in a specific area in this state to superimpose the image on the scenery seen through the window.

[0019] The storage unit 160 stores data and application software necessary for the display system 100. The storage unit 160 also stores video data to be displayed by the display unit 150, and in one embodiment, stores animation data that matches the user's preferences. FIG. 3 is an example of a management table that stores the relationship between user identification and animation data. For example, when the display unit 150 displays an image that matches the passenger's preferences, the management table is referenced. The animation data stored in the storage unit 160 may be either a video or a still image, and the animation data that matches the user's preferences may be multiple, or may include multiple images with different image sizes. When images with different image sizes are stored, attribute information that identifies the size is also stored at the same time. Such animation data may be stored in the storage unit 160 by user input, or may be downloaded from a server or the like via the communication unit 140 and stored in the storage unit 160. The data in the management table is updated when the animation data is stored.

[0020] The control unit 170 controls the overall operation of the display system 100. This control is performed by hardware and / or software. In one embodiment, the control unit 170 includes a microcontroller including a ROM / RAM, and a program for controlling the operation of the display system 100 is stored in the ROM / RAM. In this embodiment, the control unit 170 has, as part of its functions, a line of sight removal function for removing the line of sight of a parallel running vehicle when the parallel running vehicle is present. In one embodiment, the control unit 170 may operate the line of sight removal function according to the autonomous driving level from the vehicle information acquisition unit 130. This line of sight removal function will be described below.

[0021] 4 is a block diagram showing the configuration of the line of sight removal function 200 of this embodiment. The line of sight removal function 200 includes a parallel running vehicle detection unit 210, a vehicle size calculation unit 220, a distance / direction detection unit 230, a passenger identification unit 240, an animation data acquisition unit 250, an animation video generation unit 260, an eye point detection unit 270, a display coordinate calculation unit 280, and an animation video output unit 290.

[0022] The parallel running vehicle detection unit 210 detects vehicles running parallel to the own vehicle. A parallel running vehicle is another vehicle that is in such a positional relationship that the own vehicle feels a line of sight from the outside through the window, for example, another vehicle 20 running parallel to the own vehicle 10 as shown in Fig. 1. There are no particular limitations on the method of detecting parallel running vehicles, but the parallel running vehicle detection unit 210 detects other vehicles running parallel to the own vehicle by analyzing image data of the surroundings of the own vehicle captured by the imaging camera 110, or detects other vehicles running parallel to the own vehicle by using a radar of the sensor 120 or the like.

[0023] When a parallel running vehicle is detected by the parallel running vehicle detection unit 210, the vehicle size calculation unit 220 calculates the vehicle size of the parallel running vehicle. For example, the vehicle size calculation unit 220 extracts the vehicle outline by analyzing image data captured by the imaging camera 110, or extracts the vehicle outline from the detection result of the radar of the sensor 110, and calculates a rectangular size that approximates the extracted outline. Alternatively, the vehicle size calculation unit 220 calculates the maximum horizontal and vertical values ​​of the extracted outline. The vehicle size calculated by the vehicle size calculation unit 220 is used to determine the image size when an image is displayed on the window 12.

[0024] When the parallel running vehicle detection unit 210 detects a parallel running vehicle, the distance / direction detection unit 230 detects the distance and direction (angle) to the parallel running vehicle in order to know the relative positional relationship of the parallel running vehicle with respect to the own vehicle. The distance / direction detection unit 230 uses analysis of image data from the imaging camera 110 and the detection result of the sensor 120. In one aspect, the distance / direction detection unit 230 calculates the shortest distance L between the own vehicle and the parallel running vehicle, and the direction from a reference point P1 of the own vehicle to a reference point P2 of the parallel running vehicle. The reference point P2 of the parallel running vehicle can be calculated from the vehicle size calculated by the vehicle size calculation unit 220. For example, when the vehicle size is expressed by a rectangle, the intersection of the diagonal lines of the rectangle can be set as the reference point, or when the vehicle size is expressed by the maximum horizontal and maximum vertical values, the intersection of the horizontal and vertical straight lines can be set as the reference point P2. The reference point P1 of the vehicle is not particularly limited, but may be, for example, the mounting position of the imaging camera 110 or the sensor 120, or the point where the diagonals intersect when the outline of the side of the vehicle is approximated to a rectangle, or the center of the window 12.

[0025] In addition, since the host vehicle and the parallel running vehicle run on the same road surface, it may be assumed that the heights of both reference points are equal if the difference in height between the reference point P1 of the host vehicle and the reference point P2 of the parallel running vehicle is within a certain range. In this case, the distance and direction detected by the distance / direction detection unit 230 can be expressed on a horizontal XY plane including the reference point P1 of the host vehicle 10 and the reference point P2 of the parallel running vehicle 20, as shown in Fig. 5. Fig. 5(A) shows a case where the parallel running vehicle 20 is present right beside the host vehicle 10. In this case, the shortest distance between the host vehicle 10 and the parallel running vehicle 20 is L, and the angle that the straight line connecting the reference point P1 and the reference point P2 forms with the shortest distance L is 0 degrees. Fig. 5(B) shows a case where the parallel running vehicle 20 is present ahead of the host vehicle 10. The shortest distance between the host vehicle 10 and the parallel running vehicle 20 is L, but the angle that the straight line connecting the reference point P1 and the reference point P2 forms with the straight line of the shortest distance L is α. Figure 5 (C) shows a case where a parallel vehicle 20 is present behind the vehicle 10. The shortest distance between the vehicle 10 and the parallel vehicle 20 is L, but the angle formed by the line connecting the reference points P1 and P2 and the line of the shortest distance L is β.

[0026] Of course, if the parallel running vehicle is a vehicle that is taller than the host vehicle, such as a large vehicle or a truck, the heights of reference points P1 and P2 will be different. In that case, distance / direction detection unit 230 may be configured to detect the relative distance and direction of the parallel running vehicle to the host vehicle in three-dimensional space or three-dimensional coordinates.

[0027] The passenger identification unit 240 identifies passengers who board the vehicle. The passenger identification unit 240 performs face recognition based on image data from the imaging camera 110 that captures the vehicle interior space, and identifies the passenger. In this case, facial feature data of the passenger (user) is stored in advance in the memory unit 160, and the passenger is identified by comparing this feature data with the facial image. If there are multiple passengers, the passenger identification unit 240 identifies each passenger, and provides the identification result to the animation data acquisition unit 250.

[0028] The animation data acquisition unit 250 refers to the management table shown in Fig. 3 and acquires animation data corresponding to the preferences of the identified passenger. For example, when the passengers are identified as User A and User B, animation video A and animation video B are acquired. In addition, when multiple animation data of different sizes are stored, the animation data acquisition unit 250 may select animation data corresponding to the vehicle size calculated by the vehicle size calculation unit 220.

[0029] Animation image generating unit 260 processes the size and orientation of the animation image acquired by animation data acquiring unit 250 according to the vehicle size and the distance and direction of the parallel running vehicle. For example, when the parallel running vehicles are in the positional relationship shown in Fig. 5(A), the size of the animation image is determined according to the vehicle size and distance L, and when the parallel running vehicles are in the positional relationship shown in Fig. 5(B) or Fig. 5(C), the size of the animation image is determined taking into consideration the angles α and β.

[0030] The eye point detection unit 270 calculates the position of the passenger's viewpoint based on image data from the imaging camera 110 that captures the vehicle interior space. The calculation of the eye point may be performed simultaneously when the passenger identification unit 240 recognizes the passenger's face.

[0031] The display coordinate calculation unit 280 calculates the display coordinates on the window 12 when the animation image generated by the animation image generation unit 260 is displayed on the window 12, based on the relative positional relationship between the eye point coordinates calculated by the eye point detection unit 270 and the distance / direction detected by the distance / direction detection unit 230. In other words, the display coordinates are calculated so that the animation image displayed on the window 12 overlaps with the parallel running vehicle.

[0032] The animation image output unit 290 outputs the animation image generated by the animation image generation unit 260 and the coordinate position calculated by the display coordinate calculation unit 280 to the display unit 150 , and causes the display unit 150 to display the animation image on the window 12 .

[0033] Next, the operation of the display system 100 of this embodiment will be described with reference to the flow of FIG. 6. First, the parallel running vehicle detection unit 210 detects the presence or absence of a parallel running vehicle (S100), and if there is no parallel running vehicle, the flow is terminated. If a parallel running vehicle is detected, the vehicle size calculation unit 220 calculates the vehicle size of the parallel running vehicle (S110), and further the distance and direction calculation unit 230 calculates the distance and direction to the parallel running vehicle. Next, the animation data acquisition unit 250 acquires animation data according to the vehicle size and the passenger's preferences from the storage unit 160 (S120). Next, the animation image generation unit 260 generates an animation image according to the distance and direction to the parallel running vehicle (S130), the eye point calculation unit 270 calculates the passenger's eye point, and the animation image output unit 290 displays the animation image at a coordinate position on the window based on the positional relationship between the parallel running vehicle and the eye point (S140).

[0034] 7 shows an example of AR display when the line of sight removal function of the display system 100 of this embodiment is implemented. When a parallel vehicle 20 is present to the side of the vehicle 10, an animation image 300 according to the preferences of the passenger 14A is displayed on the window 12 of the vehicle 10 so as to be superimposed on the field of view of the passenger 14A, and an animation image 310 according to the preferences of the passenger 14B is displayed so as to be superimposed on the field of view of the passenger 14B.

[0035] Thus, according to this embodiment, by displaying an animation image in the passenger's field of vision through the window 12, it is possible to eliminate the discomfort caused by the line of sight of vehicles traveling alongside, and provide an in-vehicle space where passengers do not feel the line of sight. In addition, by displaying animations that the passengers prefer, it is possible to increase the entertainment value of the vehicle interior. Furthermore, by displaying the animation image in AR, it is possible to grasp the surrounding situation, such as the degree of traffic congestion outside the vehicle, and to enjoy the scenery and landscapes outside the vehicle.

[0036] Next, a modified example of the display system of this embodiment will be described. The display system of this modified example projects image data captured by the imaging camera 110 onto the window 12, and synthesizes an animation image with this image data. Fig. 8 shows an example in which, when a parallel running vehicle 20 is present directly to the side of the vehicle 10, image data 400 captured by the imaging camera 110 is displayed on the window 12, and an animation image 300 is synthesized and displayed in the center.

[0037] When a parallel running vehicle is detected, the vehicle size is calculated, the distance and direction of the parallel running vehicle is detected, and the passenger is identified, as in the above embodiment. The animation data acquisition unit 250 acquires animation data that matches the preference of either the passenger 14A or the passenger 14B, the animation image generation unit 260 processes the animation image according to the vehicle size and the distance and direction of the parallel running vehicle, the eye point calculation unit 270 calculates the eye point of the passenger, and the display coordinate calculation unit 280 calculates the display coordinates on the window 12 corresponding to the field of view of the passengers 14A and 14B. In the example of FIG. 8, since the parallel running vehicle 20 is located directly to the side of the vehicle 10, the display coordinates of the animation image 300 are determined to be approximately the center of the window 12. In this way, the animation image 300 is synthesized with the image data 400 showing the scenery outside the vehicle projected on the window 12.

[0038] Since the window 12 is a display screen that displays an image captured by the imaging camera 110, the passenger 22 of the parallel traveling vehicle 20 cannot actually see the interior of the vehicle 10, but the discomfort of a stranger constantly appearing on the display screen of the window 12 can be eliminated.

[0039] In the above embodiment, an example was shown in which the display unit 150 displayed an animated image, but the image is not limited to an animated image and may be another image.

[0040] Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to the specific embodiments, and various modifications and changes are possible within the scope of the gist of the invention described in the claims. [Explanation of symbols]

[0041] 10: Car 12: Window 14, 14A, 14B: Passengers 20: Parallel vehicle 22: Passenger 100: Display system 110: Imaging camera 120: Sensor 130: Vehicle information acquisition unit 140: Communication unit 150:Display section 160:Storage section 170: Control unit 200: Line of sight removal function

Claims

1. A display system mounted on a vehicle including a moving body, A detection means for detecting a parallel running vehicle; A display means for displaying an image on a window of the vehicle; A calculation means for calculating a vehicle size of a parallel running vehicle; a position detection means for detecting a relative position of a parallel running vehicle with respect to the vehicle, the position detection means detecting a shortest distance to the parallel running vehicle and a direction from a reference point P1 of the vehicle to a reference point P2 of the parallel running vehicle calculated based on the vehicle size of the parallel running vehicle calculated by the calculation means; A detection means for detecting an eye point of a passenger; A display control means for controlling the display means, When a parallel running vehicle is detected by the detection means, the display control means generates an image according to the distance and direction of the parallel running vehicle detected by the position detection means and according to the vehicle size, and displays the generated image based on the relationship between the position of the parallel running vehicle detected by the position detection means and the eye point position detected by the eye point detection means, at a position on the window overlapping with the parallel running vehicle.

2. A display system as described in claim 1, wherein the calculation means calculates the horizontal maximum value and the vertical maximum value of the vehicle contour, and the position detection means detects the intersection of the horizontal maximum value and the vertical maximum value as reference point P2.

3. A display system as described in claim 1, wherein the calculation means calculates a rectangular size that approximates the outline of the vehicle, and the position detection means detects the intersection of diagonals of the rectangular size as a reference point P2.

4. The display control means includes an identification means for identifying a passenger and a storage means for storing images according to the preferences of a plurality of passengers, 4. The display system according to claim 1, wherein the display control means acquires an image corresponding to the identified passenger's preference from the storage means and displays the acquired image.

5. The display system according to claim 1 , wherein the display unit displays the image in an AR manner on an external view seen through a window of a vehicle.

6. The display system further includes an imaging means for imaging an area around the vehicle; The display means displays an image of the surroundings of the vehicle captured by the imaging means on the window, 6. The display system according to claim 1, wherein the display control means combines the image with a surrounding image displayed on the window.

Citation Information

Patent Citations

  • Communication vehicle display device

    JP2010218568A

  • Interactive vehicle window display system with user identification and vehicle function control

    JP2016539346A

  • Display system

    JP2017037634A

  • Display control device, display system, and display control method

    JP2018169580A

  • Image capture with vehicle object sensor device based on user input

    JP2019091446A