Display device, display method and vehicle

Abstract

The present invention discloses a display device. The device comprises: a display unit provided in a window of a moving body; and a display control unit for controlling display of a virtual object corresponding to the object in the display unit based on a positional relationship between the moving body and the object viewed from the inside of the moving body through the display unit.

Description

Technical Field

[0001] The present disclosure relates to a display device, a display method and a vehicle. Background Art

[0002] Patent document 1 discloses a technology for establishing a correspondence between a photo with geographic information such as scenery and buildings and the location where the photo was taken. According to the technology of patent document 1, a photo with geographic information and information such as the direction where the photo was taken are linked to the location where the photo was taken on the map on an electronic map management server, so that users of display terminals such as mobile phones and personal computers can see the scenery in the direction they are actually looking at.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent No. 5334159. Summary of the invention

[0006] Non-limiting embodiments of the present disclosure contribute to providing a display device, a display method, and a vehicle capable of providing a virtual object corresponding to an object such as a landscape to an occupant of a moving object.

[0007] Solutions to the problem

[0008] A display device according to an embodiment of the present disclosure comprises: a display unit arranged on a moving body; and a display control unit for controlling display of a virtual object corresponding to the object in the display unit based on a positional relationship between the moving body and the object viewed from the inside of the moving body through the display unit.

[0009] A vehicle according to an embodiment of the present disclosure includes the above-mentioned display device.

[0010] A display method according to an embodiment of the present disclosure includes: a step of determining a display position of a virtual object corresponding to the object when it is displayed on the display unit based on a positional relationship between an object viewed from the inside of the moving body through a display unit window provided on the moving body and the moving body; and a step of displaying the virtual object at the display position.

[0011] Further advantages and effects in an embodiment of the present disclosure can be clearly presented according to the specification and the drawings. These advantages and / or effects are provided by several embodiments and the features described in the specification and the drawings, but not necessarily all of them need to be provided in order to obtain one or more of the same features. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a diagram showing a configuration example of a display system according to an embodiment of the present disclosure.

[0013] Figure 2 This is a diagram showing a hardware configuration example of an ECU (Electronic Control Unit) of an in-vehicle device.

[0014] Figure 3 It is a diagram showing a functional configuration example of an ECU of an in-vehicle device.

[0015] Figure 4 It is a diagram showing a hardware configuration example and a functional configuration example of an information processing device of a center server.

[0016] Figure 5 is a cross-sectional view of a display device.

[0017] Figure 6 This is a diagram showing a scenery displayed on a display device provided on a window of a vehicle.

[0018] Figure 7 Yes Figure 6 FIG. 4 is a diagram of the state of the mountain after combining the AR objects.

[0019] Figure 8 This is a diagram showing a scenery displayed on a display device provided on a window of a vehicle.

[0020] Fig. 9 This figure shows a state in which an AR object is combined with an image of a bridge foundation.

[0021] Fig.10 This is a diagram showing an example in which an icon-type AR object is displayed on a street scene.

[0022] Fig.11 It means it is in operation Fig.10 FIG. 1 is an example of an image of a discount code presented when the icon is displayed.

[0023] Fig.12 This is a diagram for explaining an example of a method for determining a display position.

[0024] Fig.13 This is a diagram for explaining an example of a method for determining a display position.

[0025] Fig.14 This is a diagram for explaining an example of a method for determining a display position.

[0026] Fig.15 This is a flowchart for explaining a display method of a display system. DETAILED DESCRIPTION

[0027] In the past, a technology has been disclosed for establishing a correspondence between photos with geographic information such as scenery and buildings and the locations where they were taken. The prior art has the following problem: since the photos are simply superimposed on the scenery, etc., it cannot be said that high-value-added information is provided to the passengers of the moving body. In response to such a problem, according to the display device disclosed in the present invention, virtual objects corresponding to objects such as scenery can be provided to the passengers of the moving body. Below, a suitable embodiment of the present disclosure is described in detail with reference to the accompanying drawings. In addition, in this specification and the accompanying drawings, the same figure marks are given to components with substantially the same functions to omit repeated descriptions.

[0028] (Implementation Method)

[0029] Figure 1 1 is a diagram showing a configuration example of a display system according to an embodiment of the present disclosure. The display system 1 includes: vehicle-mounted devices 30 respectively mounted on a plurality of moving bodies 3; and a central server 5 capable of communicating with the vehicle-mounted devices 30. The moving bodies 3 are, for example, cars, trucks, buses, motorcycles, railway vehicles, and the like. In addition, the moving body 3 is not limited to a vehicle, and may also be an airplane, an amusement ride, and the like. The following describes a case where the moving body 3 is a vehicle.

[0030] The vehicle-mounted device 30 includes: DCM31 (Data Communication Module), ECU (Electronic Control Unit) 32, GPS (Global Positioning System) module 33, ACC (ACCessory) switch 34, sensor 35, camera device 36, display device 14, etc. In addition to these, the vehicle-mounted device 30 also includes, for example, a car navigation device, audio equipment, inverter, motor, auxiliary equipment, etc.

[0031] DCM31 is a communication device that communicates with the central server 5 in both directions through the communication network NW. The communication network NW is, for example, a mobile phone network with many base stations as terminals, a satellite communication network using communication satellites, etc. In addition, DCM31 is connected to ECU32 in a mutually communicative manner through CAN (Controller Area Network) 38 as an in-vehicle network, and sends various information to the external devices of the vehicle according to the request from ECU32, and relays the information sent from the external devices of the vehicle to ECU32. The external devices are, for example, the central server 5, V2X (Vehicle to Everything, vehicle-to-external information exchange) communication equipment, etc. V2X is a communication technology that connects vehicles to various objects. V2X includes: V2V (Vehicle to Vehicle), V2P (Vehicle to Pedestrian), V2I (Vehicle to Infrastructure), V2N (Vehicle to Network), etc.

[0032] The ECU 32 is an electronic control unit that performs various control processes related to predetermined functions in the vehicle, and is, for example, a motor ECU, a hybrid ECU, an engine ECU, etc. The ECU 32 collects, for example, vehicle information and inputs it to the DCM 31 .

[0033] Vehicle information includes vehicle position information, speed information, vehicle status information, camera information, etc. Vehicle position information is information indicating the current position of the vehicle, such as information indicating the latitude and longitude of the vehicle. Vehicle position information is sent from, for example, a car navigation device, a GPS module 33, etc. Speed ​​information is information indicating the current speed of the vehicle sent from a vehicle speed sensor. Vehicle status information includes, for example, a signal indicating whether the ACC switch 34 is in an on state or an off state. In addition, vehicle status information may include the operating state of the wiper, the state of the defogger, the accelerator opening, the amount of brake pedaling, the steering operation amount of the steering wheel, information obtained from ADAS (Advanced Driver-Assistance Systems), etc. ADAS is a system that assists the driver's driving operation in order to improve the convenience of road traffic. Camera information is information indicating the content of an image captured by the camera 36. The camera information includes time information indicating the time when the image was generated.

[0034] The imaging device 36 is a camera having an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), etc. The imaging device 36 includes, for example, an interior imaging device for imaging the interior of the vehicle and an exterior imaging device for imaging the exterior of the vehicle.

[0035] The internal camera device is, for example, arranged at a position where it can capture the faces of passengers in the driver's seat, the front passenger seat, etc., or the faces of passengers in the rear seats, etc. The position is the dashboard of the vehicle, the instrument panel of the driver's seat, the ceiling of the vehicle, etc. The internal camera device is not limited to one, and multiple internal cameras may be arranged in the vehicle. The internal camera device outputs camera information indicating the content of the captured in-vehicle image.

[0036] The external camera device is an omnidirectional camera, a panoramic camera, etc. that captures the scenery around the vehicle. The scenery around the vehicle includes, for example, the scenery in front of the vehicle, the scenery on the side of the vehicle (the driver's door side of the vehicle or the co-driver's door side of the vehicle), the scenery behind the vehicle, etc. The scenery includes, for example: the lane where the vehicle is traveling, objects existing in the lane, the sidewalk facing the lane, objects existing on the sidewalk, etc. Objects existing in the lane include, for example, cars, motorcycles, buses, taxis, buildings, structures (advertisements, road signs, traffic lights, telephone poles, etc.), people, animals, fallen objects, etc. Objects existing on the sidewalk include, for example, pedestrians, animals, bicycles, structures, animals, fallen objects, etc. The external camera device is, for example, arranged at a position where it can capture the scenery outside the vehicle. The position is the front grille, side mirrors, ceiling, rear bumper, etc. The external camera device outputs camera information representing the content of the image outside the vehicle obtained by camera.

[0037] The GPS module 33 receives GPS signals transmitted from satellites and measures the position of the vehicle equipped with the GPS module 33. The GPS module 33 is communicably connected to the ECU 32 via the CAN 38 and transmits vehicle position information to the ECU 32.

[0038] The ACC switch 34 is a switch that turns the accessory power of the vehicle on / off according to the operation of the occupant. For example, the ACC switch 34 turns the accessory power on / off according to the operation of the power switch provided on the instrument panel near the steering wheel of the driver's seat in the vehicle. The power switch is, for example, a push-button switch that operates an ignition device not shown. The output signal of the ACC switch 34 is an example of information indicating the start and stop of the vehicle. Specifically, when the output signal of the ACC switch 34 changes from an off signal to an on signal, it indicates the start of the vehicle, and when the output signal of the ACC switch 34 changes from an on signal to an off signal, it indicates the stop of the vehicle. The ACC switch 34 is connected to the ECU 32 or the like in a communicative manner via the CAN 38, and its status signal (on signal / off signal) is generated to the ECU 32.

[0039] The sensor 35 is a sensor for detecting the voltage applied to the inverter, a sensor for detecting the voltage applied to the motor, a sensor for detecting the vehicle speed, a sensor for detecting the accelerator opening, a sensor for detecting the steering operation amount of the steering wheel, a sensor for detecting the brake operation amount, etc. In addition, the sensor 35 may include, for example, an acceleration sensor for detecting the acceleration of the vehicle, an angular velocity sensor (gyro sensor) for detecting the angular velocity of the vehicle, etc. The detection information output from the sensor 35 is captured by the ECU 32 through the CAN 38.

[0040] The display device 14 is a transparent liquid crystal display, a transparent organic EL (Electroluminescence) display, or the like, which has light transmittance and dimming properties. The display device 14 is, for example, provided at a window of a vehicle. The windows of a vehicle are windshields, side glasses, rear glasses, and the like. In addition to the windows of a vehicle, the display device 14 may also be provided at, for example, windows provided at the boarding and alighting doors of railway vehicles, windows provided near the seats of railway vehicles, cockpit windows of aircraft, cabin windows of aircraft, and the like. A structural example of the display device 14 will be described later. The windows of a vehicle include: windows whose front, rear, left, and right surfaces of the vehicle and the ceiling surface of the vehicle are all made of the display device 14 or glass. In addition, the windows of a vehicle include: windows whose entirety is made of the display device 14 (transparent liquid crystal display), that is, windows that do not use glass (windows made of only a transparent liquid crystal display). In addition, the display device 14 is not limited to the above-mentioned structural example, and may also be a structure in which a light-adjusting film whose transmittance is electronically changed is laminated and bonded on a transparent liquid crystal display or a transparent organic EL display, a transparent micro-LED, or a transparent screen film for imaging a projector image. The film laminated on the glass is arranged within, for example, 90% of the glass shape that can be recognized from inside the vehicle. In this way, it is also possible to take into account the suppression of splashing of glass generated when a person breaks a window to escape when being trapped in the vehicle.

[0041] The center server 5 is a server that collects information from multiple vehicles and distributes the information to the passengers of the multiple vehicles to provide various services, such as car sharing service, authentication key service, trunk delivery service, B2C car sharing service, and advertisement distribution service.

[0042] The center server 5 includes a communication device 51 and an information processing device 52. The communication device 51 is a communication device that performs bidirectional communication with a plurality of vehicles through a communication network NW under the control of the information processing device 52. The information processing device 52 performs various control processes in the center server 5. The information processing device 52 is composed of, for example, a server computer including a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), an auxiliary storage device, an input / output interface, and the like.

[0043] Next, refer to Figure 2 The hardware configuration of the ECU 32 of the in-vehicle device 30 will be described. Figure 2 ECU 32 is a diagram showing an example of the hardware configuration of an ECU of an in-vehicle device. ECU 32 includes an auxiliary storage device 32A, a memory device 32B, a CPU 32C, and an interface device 32D. These devices are connected to each other via a bus 32E.

[0044] The auxiliary storage device 32A is a hard disk drive (HDD), flash memory, etc., which stores files and data required for the processing of the ECU 32. When there is an instruction to start the program, the memory device 32B reads the program from the auxiliary storage device 32A and stores it. The CPU 32C executes the program stored in the memory device 32B and realizes various functions of the ECU 32 according to the program. The interface device 32D is, for example, an interface that connects the CPU 32C to the DCM 31 through the CAN 38, and connects the camera 36, ​​the sensor 35, etc. to the DCM 31 through the CAN 38.

[0045] Next, refer to Figure 3 The function of the ECU 32 of the vehicle-mounted device 30 will be described. Figure 3 It is a diagram showing a functional configuration example of an ECU of an in-vehicle device.

[0046] The memory device 32B includes a display information DB 332 that records display information displayed by the display device 14. The display information is information indicating the content of a virtual object. A specific example of the virtual object will be described later.

[0047] The CPU 32C of the ECU 32 includes a vehicle information transceiver 321 , an image pickup information management unit 323 , a display position determination unit 22 , and a display control unit 26 that performs display processing on the display device 14 .

[0048] The vehicle information transceiver 321 has a function of receiving vehicle information and a function of transmitting the vehicle information to the center server 5 .

[0049] The display position determination unit 22 determines the display position of the display information in the display device 14. As a scenario in which the display position determination unit 22 is used, for example, a case in which the display information is displayed in a correspondence relationship with the line of sight of the passenger in order to display the display information superimposed on the scenery through the display unit in the display device 14 is considered. Next, an example of a method for determining the display position of the display information is described.

[0050] First, the display position determination unit 22 extracts the face of the occupant from the in-vehicle image sent from the internal camera device, for example, and determines the position of the occupant recognizing the screen of the display device 14 based on the position and direction of the face of the occupant in the vehicle and the vehicle position.

[0051] Next, the display position determination unit 22 determines, for example, the position where the screen of the display device 14 is set in the vehicle as the screen position. For example, the screen of the display device 14 is determined when the display device 14 is set in the vehicle, so for the vehicle identification information corresponding to the vehicle, the information indicating the screen position of the display device 14 is associated with it, and the associated information is pre-recorded in the memory device 32B or the like. The vehicle identification information is the vehicle identification number (Vehicle Index Number), the vehicle ID (Identifier), etc. The display position determination unit 22 refers to the memory device 32B when the vehicle-mounted device 30 is started, and thus reads the screen position of the display device 14 using the vehicle identification information. In this way, it is possible to determine that the screen of the display device 14 is set, for example, on a windshield, a side glass, etc.

[0052] In addition, the position where the screen of the display device 14 is set can also be set in more detail. For example, when the display device 14 is set in a part of the windshield, the entire area of ​​the windshield when looking down from the inside of the vehicle toward the front is divided into respective areas from the first quadrant to the fourth quadrant of the orthogonal coordinate, and the identification information of each area is preliminarily associated with the vehicle identification information, and the associated information is preliminarily recorded in the memory device 32B, etc. Thus, the display position determination unit 22 can determine that the display device 14 is arranged, for example, in the area near the upper left part of the windshield, the area near the lower right part of the windshield, etc.

[0053] Next, the display position determination unit 22 determines the display position of the display information in the scenery through the display screen. Specifically, the display position determination unit 22 extracts the building etc. from the two frames of the vehicle exterior image continuously shot, and calculates the distance from the external camera device to the building etc. based on the position difference between the two frames of the building etc. in the vehicle exterior image using the principle of a stereo camera. Next, the display position determination unit 22 determines the position of the building etc. based on the distance from the external camera device to the building etc. and the vehicle position. Then, the display position determination unit 22 determines whether there is display information corresponding to the position of the building etc. in the display information DB 332 by referring to the display information DB 332. If there is such display information, the display position determination unit 22 determines the position of the corresponding building etc. as the display position of the display information.

[0054] In addition, the display position determination unit 22 may be configured to determine the display position of the display information by other methods. For example, the display position determination unit 22 calculates the range of latitude and longitude corresponding to the area of ​​the scenery included in the vehicle exterior image based on the vehicle position and the imaging range of the external imaging device. Then, the display position determination unit 22 determines the display position of the display information by searching the display position of the display information within the calculated range of latitude and longitude from the display information DB 332.

[0055] Finally, the display position determination unit 22 determines the display position of the display information based on the determined current position of the occupant, the screen position, the display position of the display information, etc. The display position determination method will be described in detail later.

[0056] The imaging information management unit 323 creates an imaging information list (imaging information DB 3292 ) by inputting imaging information sent from the imaging device 36 within a predetermined time and embedding time and vehicle position information in the sent imaging information.

[0057] Next, use Figure 4 A hardware configuration example of an information processing device of a center server will be described. Figure 4 1 is a diagram showing a hardware configuration example and a functional configuration example of an information processing device of a center server. The information processing device 52 includes a CPU 16 and a storage unit 520 .

[0058] The CPU 16 includes a communication processing unit 5201 for transmitting and receiving various information to and from each of a plurality of vehicles, an information display object extraction unit 5205, a vehicle identification unit 5212, a command transmission unit 5213, a map matching unit 5214, and a search information generation unit 5215. The storage unit 520 includes a map information DB 520A, a search information DB 520B, an information display object DB 520F, a vehicle information DB 520H, and a camera information DB 520J.

[0059] The information display object extraction unit 5205 extracts information display objects from the image information of the camera device 36 contained in the respective search information of the plurality of vehicles stored in the search information DB 520B based on known image recognition processing. Then, the information display object extraction unit 5205 associates the information display object with the image of the information display object and meta-information such as the position information of the information display object with the identification information after adding inherent identification information to the extracted information display object, and stores the information in the information display object DB 520F. Thus, in addition to information related to pre-registered information display objects such as signboards and digital signboards that display advertising information of a predetermined advertiser, information related to the information display object extracted by the information display object extraction unit 5205 is also registered in the information display object DB 520F. Thus, the information display objects are enriched, and the convenience of the passengers is improved. In addition, the position information of the information display object added as meta-information may be the vehicle position information itself contained in the same search information as the image information as the extraction source, or may be position information after considering the relative position information of the information display object with respect to the vehicle calculated based on the image information. In addition, when the extracted information display object is the same as the information display object already registered in the information display object DB 520F, the information display object extraction unit 5205 does not store the information related to the extracted information display object in the information display object DB 520F. In addition, the processing performed by the information display object extraction unit 5205 can be performed in real time corresponding to the detection information sequentially received from a plurality of vehicles by the communication processing unit 5201, or can be performed periodically for the unprocessed detection information accumulated to a certain extent.

[0060] The vehicle identification unit 5212 identifies the vehicle that has passed through the geographical location or area of ​​the object of collection of the camera information based on the vehicle position information. In addition, in the preparation of three-dimensional highly dynamic map information (dynamic map) used in the automatic driving of the vehicle, it is necessary to obtain the latest image of the place where the vehicle actually travels by photographing. The place as the object of preparation of the dynamic map can be an example of the geographical location or area of ​​the object of collection of the camera information.

[0061] For example, when the vehicle position information transmitted from a plurality of vehicles is input, the vehicle identification unit 5212 compares the vehicle position information with the position or area of ​​the collection target to determine the vehicle that has passed through the position or area. Then, the vehicle information including the vehicle position information determined to have passed through is selected from the vehicle information transmitted from the plurality of vehicle-mounted devices 30, and the vehicle identification information contained in the selected vehicle information is extracted. The vehicle identification unit 5212 that has extracted the vehicle identification information transfers the extracted vehicle identification information to the instruction transmission unit 5213.

[0062] The command transmission unit 5213, which receives the vehicle identification information from the vehicle identification unit 5212, transmits a camera information request command to a vehicle to which the vehicle identification information is assigned, among the vehicle group connected to the center server 5 in a communicable manner via the communication network NW. The camera information distributed in response to the camera information request command is recorded in the storage unit 520 as the camera information DB 520J in correspondence with the data collection target area information.

[0063] The map matching unit 5214 determines the road link where the vehicle is currently located based on the map information DB520A and the vehicle position information. The map information DB520A is composed of GIS (Geographic Information System) data and the like. The GIS data includes: nodes corresponding to intersections, road links connecting nodes, lines and polygons corresponding to ground objects such as buildings or roads, etc. For example, identification information, i.e., link ID, is predefined for each of the multiple road links that constitute the road network contained in the map information DB520A. The map matching unit 5214 determines the link ID of the road link where the vehicle is currently located by referring to the map information DB520A.

[0064] The search information generation unit 5215 generates search information at a predetermined period, the search information including vehicle information sent from the vehicle, time information, and the road link determined by the map matching unit 5214. Then, the search information generation unit 5215 stores the generated search information in the search information DB 520B.

[0065] Next, refer to Figure 5 A configuration example of the display device 14 will be described. Figure 5 : is a cross-sectional view of the display device. The display device 14 is provided on the vehicle, for example, by being attached to the inner side or the outer side of the window of the vehicle. In addition, the configuration method of the display device 14 on the vehicle is not limited thereto. For example, the display device 14 may be fixed to the frame of the vehicle and provided so that the screen of the display device 14 faces the inner side or the outer side of the window of the vehicle. In addition, the display device 14 may be provided in an embedded manner in the window of the vehicle. In addition, the display device 14 may be provided in a manner covering the entire area of ​​the window of the vehicle, or in a manner covering a partial area of ​​the window of the vehicle.

[0066] The display device 14 has a structure in which an electron transmission control film 14a is sandwiched between two transparent OLEDs 14b1 and 14b2, for example. Hereinafter, when the transparent OLED 14b1 and the transparent OLED 14b2 are not distinguished, they are referred to as "transparent OLEDs". The electron transmission control film 14a is an example of a transmissive display unit capable of changing transparency. The transparent OLED 14b1 and the transparent OLED 14b2 are examples of a display information display unit capable of displaying display information.

[0067] The electron transmission control film 14a can control the shade of the scenery seen through the display unit of the vehicle, or the shade of the image displayed by the transparent OLED, for example, by changing the transparency (visible light transmittance). The electron transmission control film 14a can be a film that can uniformly change the transparency of the entire electron transmission control film 14a, or a film that can change the transparency of a part of the electron transmission control film 14a. As a method of changing the transparency of the electron transmission control film 14a, for example, an electrochromic method, a gas-induced chromic method that can perform dimming control faster than the electrochromic method, etc. can be exemplified. In the case of changing the transparency of a part of the electron transmission control film 14a, a local dimming technology or a technology disclosed in non-patent document 1 (non-patent document 1: https: / / www.jst.go.jp / pr / announce / 20171017-3 / index.html) can be used.

[0068] The transparent OLED 14b1 is an example of a transparent display facing the first end surface side of the electron transmission control film 14a. The first end surface side of the electron transmission control film 14a is, for example, the inner side of the window. The transparent OLED 14b2 is an example of a transparent display facing the second end surface side opposite to the first end surface side of the electron transmission control film 14a. The second end surface side of the electron transmission control film 14a is the outer side of the vehicle. In addition, the display device 14 may also include a transparent liquid crystal display instead of the transparent OLED.

[0069] The display device 14 includes two transparent OLEDs, and can display different information on the inside and outside of the window.

[0070] For example, when there is a passenger who likes games in a vehicle in an autonomous driving mode, Figure 5 As shown on the left side of , it is possible to display a picture of a game being played on the transparent OLED 14b1 and to display a character of the game in an enlarged manner on the transparent OLED 14b2 while keeping the transparency of the electron transmission control film 14a low (for example, a visible light transmittance of less than 30%).

[0071] In addition, if there are passengers who like yoga, shadow boxing, etc. in the vehicle during autonomous driving, Figure 5 As shown on the left side of , when the transparency of the electron transmission control film 14a is low, a moving teaching video image can be displayed on the transparent OLED 14b1, and a dynamic image of a person moving in the vehicle can be displayed on the transparent OLED 14b2.

[0072] In addition, it is also possible to display a navigation screen such as a map on the transparent OLED 14b2 and to display an advertising image on the transparent OLED 14b2 while lowering the transparency of the electron transmission control film 14a.

[0073] In addition, if Figure 5 As shown on the right side of the figure, in a state where the transparency of the electronic transmission control film 14a is high (for example, the visible light transmittance is 80% or more), a navigation screen such as a map is displayed on the transparent OLED 14b1 of the display device 14 provided on the windshield. In addition, the transparent OLED 14b2 of the display device 14 does not display display information, so that the navigation screen can be displayed by overlapping the scenery through the windshield.

[0074] In addition, different display information can be displayed on the two transparent OLEDs, or the same or similar display information can be displayed. For example, when there is a passenger who likes sports in the vehicle in automatic driving, the transparency of the electronic transmission control film 14a can be reduced, and only the teaching video image can be displayed on the transparent OLED 14b1, while the teaching video image and the dynamic image of the person in motion can be displayed on the transparent OLED 14b2.

[0075] The configuration example of the display device 14 is not limited to the example shown in the figure. For example, the display device 14 may be configured to include only one of the two transparent OLEDs.

[0076] The transparency of the display device 14 can be changed based on the driving information related to the driving of the mobile body. The driving information related to the driving of the mobile body is, for example, the speed information of the vehicle (vehicle speed), the weather information around the current position of the vehicle, the current time information, the vehicle state information, the traffic information, the information indicating the driving mode of the vehicle, etc. In addition, the transparency of the display device 14 can be changed in stages or continuously. The following is an example of changing the transparency.

[0077] In the case where the transparency is changed based on the vehicle speed, the display control unit 26 uses, for example, table information that establishes a correspondence between the vehicle speed and the transparency. The table information may be table information pre-recorded in the memory device 32B or table information distributed by the center server 5. The display control unit 26 that has input the vehicle speed information refers to the table information, for example, and sets the transparency to the first transparency in the first speed range, and sets the transparency to the second transparency lower than the first transparency in the second speed range higher than the first speed range. The first transparency is, for example, a state where the visible light transmittance is 80%. The second transparency is, for example, a state where the visible light transmittance is 30%. The first speed range is, for example, a speed from 0 km / h to 80 km / h. The second speed range is, for example, a speed of more than 80 km / h. Thus, for example, when the vehicle is traveling in the city in the first speed range in the automatic driving mode, even if it is necessary to avoid an obstacle, the driver can immediately perform an operation to avoid the obstacle because he can recognize the scenery through the display unit. In addition, when the vehicle is traveling in the second speed zone in the automatic driving mode on a bypass road, a highway, etc., there are few scenes to avoid obstacles, etc. Therefore, by shielding the scenery through the display unit, the occupants can concentrate on music appreciation, reading, etc. In addition, the transparency can be changed in stages according to the vehicle speed, or it can be changed continuously according to the vehicle speed. For example, in the first speed zone, the transparency can also be continuously reduced as the vehicle speed increases from 0 km / h to 80 km / h. In addition, in the case of changing the transparency based on the vehicle speed, the display control unit 26 can also be configured to make the transparency of the distant part (the upper part of the window) where the scenery changes less high, and conversely make the transparency of the part (the lower part of the window) where the scenery changes more low.

[0078] In the case of changing the transparency based on the weather, the display control unit 26 can change the transparency by using, for example, weather information distributed through the Internet, wiper operation status information, defogger operation status information, etc. For example, on a sunny day, the overall transparency of the display device 14 is set to about 50%, so that the driver and fellow passengers will not feel glare and the immersion in the displayed image can be improved. In addition, on a cloudy day, since the visibility is reduced compared to a sunny day, the display control unit 26, for example, makes the transparency of the area below the center of the display device 14 high, and makes the transparency of the area above the center of the display device 14 low. As a result, while a part of the area of ​​the display device 14 is shielded, the remaining area is not shielded and the state outside the vehicle can be confirmed. Thus, it is possible to grasp the traffic situation while reducing the glare caused by diffuse reflection of clouds, so that it is possible to perform avoidance operations such as jumping out of pedestrians. And fellow passengers can enjoy the displayed image, etc. In addition, in rainy days, the visibility is lower than in cloudy days, so the display control unit 26 sets the overall visible light transmittance of the display device 14 to a high transparency such as about 80%. As a result, even in a situation where the visibility is poor due to rain, it is easy to grasp traffic lights, intersections, surrounding vehicles, etc., and it is possible to perform avoidance operations such as jumping out of pedestrians. In addition, even in a state where the transparency of the display device 14 provided on the windshield is high, by making the transparency of the display device 14 provided on the side glass, etc. low, it is possible to allow fellow passengers to appreciate the display image, etc.

[0079] When the transparency is changed based on the time, for example, by using table information that establishes a correspondence between each time period such as early morning, daytime, evening, and midnight and a plurality of transparencies that are different for these time periods, the transparency can be changed according to the time. The table information may be table information pre-recorded in the memory device 32B or table information distributed from the center server 5. For example, the display control unit 26 that has input the time information refers to the table information and sets the overall transparency of the display device 14 to about 50% in the early morning and daytime in order to reduce the glare of the driver, etc. In addition, the display control unit 26 sets the overall transparency of the display device 14 to about 80% in order to ensure the field of vision in the evening and midnight.

[0080] In addition, the display control unit 26 may be configured to compare the brightness inside the vehicle with the brightness outside the vehicle and to lower the transparency only when the brightness inside the vehicle is higher than the brightness outside the vehicle. The brightness inside the vehicle is compared with the brightness outside the vehicle by comparing the average brightness level before white balance adjustment of the vehicle exterior camera that photographs the exterior of the vehicle during automatic driving with the average brightness level before white balance adjustment of the interior camera that photographs the interior of the vehicle.

[0081] When the transparency is changed based on the vehicle state information, for example, a plurality of list information is prepared for each type of vehicle state information, and each list information corresponds to a transparency for, for example, the accelerator opening, the brake pedal depression amount, the steering wheel steering operation amount, etc. The list information may be the list information recorded in advance in the memory device 32B, or the list information distributed from the center server 5.

[0082] The display control unit 26, which has input information related to the throttle opening, sets the transparency of the area below the center of the display device 14 to about 80% and sets the transparency of the area above the center of the display device 14 to about 30% when the throttle opening is small. As a result, in a scene where the vehicle is cruising at a constant speed on a highway or the like, since a part of the area of ​​the display device 14 is shielded, the glare of sunlight or the like on the driver can be reduced. In addition, fellow passengers can enjoy images or the like displayed in the area with low transparency.

[0083] The display control unit 26, which has input information related to the accelerator opening, sets the overall transparency of the display device 14 to approximately 80% when the accelerator opening is large, for example. Thus, when the vehicle is traveling up a steep slope with continuous turns such as a mountain road, for example, the display device 14 is not blocked, thereby facilitating safe driving for the driver, and allowing fellow passengers to enjoy the display image displayed superimposed on the scenery.

[0084] The display control unit 26, which has input information related to the amount of brake pedaling, sets the transparency of the area below the center of the display device 14 to about 80% and sets the transparency of the area above the center of the display device 14 to about 30% when, for example, the number of brake pedaling times within a specified time is small or the amount of brake pedaling within a specified time is small. Thus, for example, in a scene where the vehicle is cruising on a highway, since a part of the area of ​​the display device 14 is shielded, the glare of sunlight or the like on the driver can be reduced. In addition, fellow passengers can appreciate the images displayed in the remaining area.

[0085] The display control unit 26, to which the information related to the brake pedaling amount is input, sets the transparency of the entire display device 14 to approximately 80% when, for example, the number of brake pedaling times within a predetermined time is large or the brake pedaling amount within a predetermined time is large. Thus, for example, in a scene where the vehicle is traveling in an urban area with heavy traffic, since the display device 14 is not blocked, it helps the driver to drive safely, and the passengers can enjoy the display image displayed overlapping with the scenery.

[0086] The display control unit 26, which has input information related to the steering operation amount of the steering wheel, sets the transparency of the area below the center of the display device 14 to about 80%, and sets the transparency of the area above the center of the display device 14 to about 30%, for example, when the number of steering operations of the steering wheel within a specified time is small or the steering operation amount of the steering wheel within a specified time is small. Thus, for example, in a scene where the vehicle is cruising on a highway, since a part of the area of ​​the display device 14 is shielded, the glare of sunlight or the like directed to the driver can be reduced. In addition, fellow passengers can appreciate the images displayed in the remaining area.

[0087] The display control unit 26, to which the information related to the steering operation amount of the steering wheel is input, sets the transparency of the entire display device 14 to about 80% when, for example, the number of steering operations of the steering wheel within a predetermined time is large or the steering operation amount of the steering wheel within a predetermined time is large. Thus, for example, in a scene where the vehicle is traveling in an urban area with heavy traffic, since the display device 14 is not blocked, it helps the driver to drive safely, and the passengers can enjoy the display image displayed overlapping with the scenery.

[0088] In the case of changing the transparency based on traffic information, the display control unit 26 may also change the transparency of the display device 14 according to the congestion of the road. Specifically, when the road on which the vehicle is traveling is blocked for about several minutes according to the traffic information distributed when traveling on the expressway, since the vehicle speed is reduced in a short time, the display control unit 26 sets the transparency to about 80% so that fellow passengers can enjoy the scenery through the display unit. On the other hand, when the road on which the vehicle is traveling is blocked for more than tens of minutes, it is forced to drive at a low speed for a relatively long time. In this case, the display control unit 26 sets the transparency to about 30% in order to make the driver of the vehicle happy with the display image, and changes the transparency from about 30% to about 80% when the congestion is relieved.

[0089] When the transparency is changed based on the information indicating the vehicle driving mode, the display control unit 26 can change the transparency, for example, according to whether it is a manual driving mode or an automatic driving mode (including a driving assistance mode, a semi-automatic driving mode, etc.). In addition, when the manual driving mode is selected, the display control unit 26 can also change the transparency according to an eco-driving mode that can drive in a fuel-efficient manner, a sports driving mode that can drive actively, etc. In addition, the display control unit 26 can also be configured to reduce the transparency of all windows when, for example, the vehicle is used as a completely private space through the selection of the passenger.

[0090] In addition, the display device 14 may also be a device that combines an electronic transmission control film 14a with a head-up display. In this case, the information projected from the head-up unit is projected onto the electronic transmission control film 14a provided on the window via a reflector or the like, so that the driver can recognize it as a virtual image. At this time, by changing the transparency of the electronic transmission control film 14a according to the driving state, driving can be assisted in detail. For example, when the vehicle is driving on a snowy road, the display device 14 can clearly display the virtual image overlapping the snowy road by making the transparency of a part of the electronic transmission control film 14a on which the virtual image is projected lower than the transparency of other areas.

[0091] In addition, the head-up display includes: a display formed by pasting a special film close to a semi-transparent mirror on the surface close to the passenger in order to easily recognize the virtual image of the head-up display, or a display formed by sandwiching a special film-like layer as an intermediate layer inside the glass. In particular, the head-up display using a special film close to a semi-transparent mirror almost becomes a reflective mirror when the outside of the car is darker than the inside of the car. That is, when the outside of the car is darker than the inside of the car, the situation inside the car is reflected through the semi-transparent mirror. Therefore, by increasing the brightness of the above-mentioned transparent OLED, although the contrast of the transparent OLED is deteriorated, it is preferable to use it in the following manner, that is, to perform image adjustments such as increasing the black level (black brightness), or to set it as an image that does not include black, and only partially reduce the transmittance of the display part of the head-up display to display the image. In addition to the above-mentioned special film with characteristics close to the semi-transparent mirror, an HOE (Holographic Optical Element) that only diffracts a specific wavelength can be exemplified. In this case, the above-mentioned image adjustment is not required.

[0092] In addition, the display control unit 26 can also change the transparency by using information distributed through V2X communication, for example. In V2X communication, not only vehicle-to-vehicle communication is possible, but also communication between a vehicle and a person with a communication terminal, communication between a vehicle and a roadside device, etc. can be performed. In V2X communication, information indicating the state of traffic signals, traffic control information, traffic obstacle information (information related to ice on the road surface, flooding on the road, fallen objects on the road, etc.), position information of moving objects existing around the vehicle, etc. are provided. By using this information, for example, when a vehicle equipped with the display device 14 turns right, if there is a moving object such as a bicycle or a motorcycle approaching the vehicle from the rear of the vehicle, the display device 14 can also display the moving object in real time. In addition, if the distance between the moving object and the vehicle is less than the set distance, the display control unit 26 can also enable the driver to recognize the moving object by switching the transparency of the electronic transmission control film 14a provided on the side glass, etc. from a low state to a high state. In addition, the display control unit 26 may display a warning message on the display device 14 when it is determined that the collision of the mobile object with the vehicle cannot be avoided. Specifically, the display control unit 26 displays an image of the mobile object approaching the vehicle on the display device 14 in real time by using, for example, camera information obtained from an external camera device, and estimates the speed of the mobile object approaching the vehicle based on the position of the mobile object existing around the vehicle, the amount of movement of the mobile object per unit time, and the like. Furthermore, when it is determined that the collision of the mobile object with the vehicle cannot be avoided based on the estimated speed, the display control unit 26 displays the warning message on the display device 14. In addition, when receiving flood information, the display control unit 26 may estimate the flood location, flood amount, detour route to detour the flood location, and the like by cooperating with the navigation device, and display them on the display device 14.

[0093] In addition, the transparency of the display device 14 may be changed according to the form of the display content of the virtual object, the attributes of the passenger, etc. A specific example of the display content of the virtual object will be described later.

[0094] Next, refer to Figure 6 to Figure 11An example of displaying virtual objects, etc. is described. In addition, virtual objects may include: augmented reality objects (AR objects), virtual reality objects (VR objects), substitute reality (SR objects), mixed reality objects (MR objects), etc. AR (Augmented Reality) is a technology that gives new understanding by overlapping information on objects in real space, etc. VR (Virtual Reality) is a technology that builds a sense of reality (sense of reality) in a virtual space. SR (Substitutional Reality) is a technology that seamlessly replaces information recorded in the past with information currently obtained. MR (Mixed Reality) is a technology that builds a sense of reality based on mixing real space with virtual space. AR objects are objects that are gradually displayed in an image of a landscape or even superimposed on an image of a landscape by rendering an image based on spatial information about a specific object. VR objects are virtualized objects, such as computer graphics images made based on high-definition 3D maps, or object images embedded in videos shot while driving.

[0095] Figure 6 This is a diagram showing a scenery displayed on a display device provided on a window of a vehicle. Figure 6 , an image of a mountain 4 is shown as an example of scenery through the display unit. The scenery through the display unit may be an image of scenery captured by an imaging device or may be actual scenery viewed by the passenger 6 from inside the vehicle through the display unit.

[0096] Figure 7 Yes Figure 6 By combining the AR object 7 deformed into the shape of the sun with the actual mountain 4, the immersion of the passenger 6 in the displayed information can be enhanced.

[0097] Reference Figure 8 and Fig. 9 Another display example of the AR object 7 will be described. Figure 8 This is a diagram showing a scenery displayed on a display device provided on a window of a vehicle. Figure 8 In FIG. 8 , a foundation 8A (bridge) of a bridge under construction is shown as an example of scenery through the display unit. Fig. 9 This is a diagram showing a state where an AR object is combined with an image of a bridge foundation. By combining an AR object 8B of a completed iron bridge with an actual foundation 8A, the sense of immersion of the passenger 6 in the displayed information can be enhanced. In addition, since the unfinished iron bridge can be seen through the display unit, it contributes to town revitalization (regional revitalization) that promotes the revitalization of the region by attracting tourists.

[0098] Fig.10This is a diagram showing an example in which an icon-type AR object is displayed on a street scene. Fig.11 It means operation Fig.10 For example, if the passenger clicks icon 9A, an AR object related to the discount code is displayed. Fig.11 An AR object 9B is shown that is associated with a discount code.

[0099] By displaying such AR objects, the passengers' interest in the scenery through the display unit is greatly enhanced. In addition, since it also greatly contributes to the PR of stores, companies, etc., even if physical advertisements are not posted on actual buildings, the company's products can be promoted to passengers on buses, etc. In addition, if special advertisements are required only during a certain period (such as the Olympic Games), there is no need to set up physical advertisements on the building, but advertisements can be placed in various areas, which can effectively improve purchasing power and attract customers.

[0100] In addition, the AR object 9B is not limited to being related to the discount code, and for example, it can also be an AR object that uses patterns or texts to represent the content of promotional activities implemented in stores, companies, etc., or it can be an AR object that represents the content of word-of-mouth reviews of stores.

[0101] In addition, the AR object, for example, may be newly made by an individual or advertiser even if it is not a content creation company, or may be obtained by customizing an existing AR object. For example, by using a dedicated application installed on a smartphone, the design of the AR object, the display position of the AR object, the language displayed by the AR object, etc. can be freely set. The set AR object is registered in a corresponding relationship with the position of a building, scenery, etc. in the map information (dynamic map, 3D high-definition map, etc.) on the center server 5. Moreover, by downloading the map information to the vehicle, and comparing the position of the map information with the current position of the vehicle by the display control unit 26, when the vehicle enters a specified range from a specific building in the map information, the AR object is combined with the real object corresponding to the position of the vehicle and displayed on the display device 14. Not only AR objects, but VR objects, MR objects, etc. can also be set in the same way. In addition, since AR objects can be freely customized, hot spots can also be created by constructing a unique pattern of scenery that does not exist in other areas. In addition, the display method of VR objects, MR objects, etc. is supplemented below. For example, objects (buildings, roads, etc.) in the space where the vehicle is traveling are set as VR objects (objects close to reality, or objects different from reality) configured in the entire 3D space (virtual 3D space) created virtually. Furthermore, based on the sensor data of the position and motion vector of other moving objects such as people, motorcycles, bicycles, etc. detected in automatic driving, the person is set as character A, the motorcycle is set as character B, and these are set as MR objects in the above-mentioned virtual 3D space. On this basis, the VR objects and MR objects are displayed in conjunction with the actual movement, and the real information is displayed in the virtual space. At this time, it can also be configured that the objects configured in the virtual space close to reality can be provided by the owner of the building in any form, for example.

[0102] In addition, VR objects, AR objects, etc. may also be changed based on driving information related to the driving of the mobile body. Driving information related to the driving of the mobile body includes, for example, the speed of the vehicle (vehicle speed), the weather around the current position of the vehicle, current time information, vehicle status information, traffic information, information indicating the driving mode of the vehicle, etc. Next, a specific example of changing a virtual object based on driving information related to the driving of the mobile body is described.

[0103] In the case of changing virtual objects based on time, for example, by using table information that establishes a correspondence between each time period such as early morning and late night and a plurality of virtual objects that are different according to these time periods, it is possible to change virtual objects according to time. The table information may be table information pre-recorded in the memory device 32B or table information distributed from the center server 5. For example, the display control unit 26 that has input time information from the GPS or the like refers to the table information and, if it is early morning, sets Figure 7 The object of the sun pattern as shown is displayed on the surface of the mountain. In addition, if it is late at night, the display control unit 26 displays an object of the moon pattern near the top of the mountain, for example.

[0104] When the virtual object is changed based on the vehicle speed, the display control unit 26 uses, for example, table information that establishes a correspondence between the vehicle speed and a plurality of virtual objects in different areas. The table information may be table information pre-recorded in the memory device 32B, or may be table information distributed from the center server 5. The display control unit 26 that has received the vehicle speed information refers to the table information, and, for example, if the vehicle speed is in the first speed area below 20 km / h, displays advertisements such as buildings set up in the surrounding area of ​​the vehicle (e.g., within tens of meters from the vehicle) as virtual objects. If the vehicle speed is in the second speed area exceeding 20 km / h, the display control unit 26 displays advertisements such as buildings set up in an area relatively far from the vehicle (e.g., more than tens of meters from the vehicle) as virtual objects. Thus, when the vehicle is traveling at a low speed, the occupants can see the virtual objects near the vehicle. When the vehicle is traveling at a high speed, the virtual objects near the vehicle pass through the window in a short time, so it is difficult for the occupants to see them, but if the virtual objects are in an area relatively far from the vehicle, the occupants can see them.

[0105] In addition, when displaying virtual objects on a windshield, for example, the virtual objects may be displayed during parking, and the virtual objects may be gradually hidden from the objects that require attention during driving, triggered by the release of a braking device such as a side brake, as the vehicle speed, acceleration, etc. increase. This may provide advertisements to the driver, and may prevent accidents caused by overlooking obstacles around a vehicle traveling in an urban area.

[0106] In addition, when a virtual object is displayed on a windshield or the like, when the vehicle is traveling in the second speed zone in the automatic driving mode on a bypass road, a highway, or the like, since there are few scenes to avoid obstacles, etc., a virtual object corresponding to the scenery of an area far from the vehicle is displayed, and when the vehicle speed decreases from the second speed zone to the first speed zone, the object is made non-displayed. Thus, while providing advertisements to the driver, even in scenes where the vehicle speed decreases due to congestion or the entrance and exit of a highway, accidents caused by overlooking obstacles around the moving vehicle can be suppressed.

[0107] In the case of changing the virtual object based on the weather, the display control unit 26 can change the virtual object by using, for example, weather information distributed through the Internet, wiper operation status information, defogger operation status information, etc. For example, it can be considered that since the weather causes people's tastes to change, advertisements for restaurants, etc. can be displayed as virtual objects corresponding to the weather in conjunction with this. For example, a virtual object related to a rice menu can be set for a restaurant advertisement on a sunny day, and a virtual object related to a noodle menu can be set for a restaurant advertisement on a rainy day. In this way, the ability to attract customers, purchasing power, etc. can be efficiently improved without setting up physical advertisements on the building. In addition, for the occupants of the vehicle, the following effect can also be obtained, that is, the effect of increasing the fun of driving or traveling on rainy days or cloudy days.

[0108] When the virtual object is changed based on the vehicle state information, for example, a plurality of list information is prepared for each type of vehicle state information, and each list information corresponds to a virtual object for, for example, the accelerator opening, the brake pedaling amount, the steering operation amount of the steering wheel, etc. The list information may be the list information recorded in advance in the memory device 32B, or the list information distributed from the center server 5.

[0109] For example, since the vehicle is traveling at a constant speed on a highway and the accelerator opening is small, it can be considered that the driver is driving in a relaxed manner. In this case, the display control unit 26 displays the following Fig.10 The same applies to the case where the brake pedaling amount is small, the number of steering wheel operations within a predetermined time is small, or the steering wheel steering operation amount within a predetermined time is small.

[0110] On the other hand, when the throttle opening is large because the vehicle is driving uphill on a steep slope with continuous turns like a mountain road, it can be considered that the driver's tension increases and the fatigue after driving increases. In this case, the display control unit 26 displays the virtual object of the assisted driving on the display device 14 in an overlapping manner with the road signs in the urban area. The virtual object of the assisted driving is, for example, guidance to roadside stations, convenience stores, gas stations, charging stations, etc., which exist near the current position of the vehicle, coupon codes for drinking water, etc. In addition, these objects can also be changed in real time according to the throttle opening. In addition, the same is true for the situation where the brake is stepped on a lot, the number of steering operations of the steering wheel within a specified time is large, or the steering operation amount of the steering wheel within a specified time is large.

[0111] When changing the virtual object based on traffic information, the display control unit 26 may also change the virtual object of the display device 14 according to the congestion of the road. Specifically, when the road where the vehicle is scheduled to pass is congested for several minutes according to the traffic information distributed when traveling on the expressway, it can be considered that the vehicle speed has dropped in a short period of time. In this case, the display control unit 26 displays Figure 7 AR objects such as those shown in the figure are displayed so that fellow passengers can enjoy virtual objects corresponding to the scenery through the display unit. On the other hand, in the case of a traffic jam of more than tens of minutes, it can be considered that the vehicle is forced to drive at a low speed for a relatively long time. In this case, the display control unit 26 displays VR objects that attract the interest of fellow passengers on the side glass, etc. For example, in the case where a building near the current position of the vehicle is being remodeled, a computer graphic image of the building before the reconstruction, a computer graphic image of the building after the reconstruction, etc. can be displayed as a VR object.

[0112] When the virtual object is changed based on the information indicating the vehicle driving mode, the display control unit 26 may change the virtual object according to whether the vehicle is in manual driving mode or automatic driving mode (including driving assistance mode, semi-automatic driving mode, etc.). Figure 7 The AR object shown above displays a VR object of the street when the car is in autonomous driving mode.

[0113] In addition, the display control unit 26 may also change the virtual object according to the attributes of the passenger, for example. For example, in the case where an AR object of a cartoon character popular with children is embedded in a certain location in the map information, the display control unit 26 uses the camera information to recognize the face of the passenger in the vehicle and then determines whether it is a child. And, in the case of a child, the display control unit 26 displays the AR object of the cartoon character on the display device 14 when the vehicle approaches the location where the cartoon character exists on the map.

[0114] In addition, when signs that children do not want to see exist in urban areas, the actual signs can be covered with virtual objects, or replaced with virtual objects of children's favorite characters, etc. Specifically, for example, in the center server 5, information such as advertisements, signs, slogans, etc. that children are not allowed to read is registered in advance with information such as the age, gender, and nationality of the child. In addition, by associating these corresponding list information with coordinates on the map, when there are children corresponding to the registered conditions among the passengers in the vehicle near the location where these advertisements exist, the display control unit 26 covers the actual advertisements with virtual objects to prevent the children from seeing them. In addition, the display control unit 26 can also cooperate with the display position determination unit 22 to analyze the sight line direction of the child who is the target, and when there is a virtual object such as an advertisement that the child does not want to see in front of the sight line, an alternative AR object such as an animated character is displayed instead of the object. In addition, the display control unit 26 can also cooperate with the display position determination unit 22 to display alternative AR objects when these advertisements are close to the child's sight line.

[0115] In addition, the display control unit 26 may be incorporated in the ECU 32 , or may be incorporated in the display device 14 .

[0116] Reference Figure 12 to Figure 14 An example of displaying an AR object or the like by aligning a person's line of sight with a real object will be described.

[0117] Figure 12 to Figure 14 Each of them is a diagram for explaining an example of a method for determining a display position. Figure 12 to Figure 14 , a display device 14 provided on a vehicle window, a passenger u recognizing the display device 14 in the vehicle, and a position p are shown. The position p is a position outside the vehicle that is associated with display information such as an AR object in the scenery seen by the passenger u via the display device 14.

[0118] Fig.12 In the description, for example, a case is described in which a passenger u in the vehicle is located at a position behind a window at the right rear portion of the vehicle and can see a position p at the right front portion outside the vehicle from this position via the display device 14. In this case, the display position determination unit 22 determines the display position of the display information in the display device 14 based on the position of the passenger u, the position of the display device 14, and the position p.

[0119] Hereinafter, the position of the occupant u is referred to as the “occupant position”, the position of the display device 14 is referred to as the “screen position”, and the position p corresponding to the display information is referred to as the “information corresponding position p”.

[0120] exist Fig.12, the display position determination unit 22 determines the intersection point where the imaginary line connecting the information corresponding position p in the scenery and the occupant position intersects the display device 14 as the center point of the display position of the display information d1 associated with the information corresponding position p in the scenery.

[0121] Then, the display position determination unit 22 generates a display instruction for instructing the display device 14 to display the display information d1 at the determined display position, and inputs the instruction to the display control unit 26. At this time, the display position determination unit 22 displays the display information d1 (virtual object) in a form corresponding to the positional relationship between the information corresponding position p in the scenery and the passenger position. In addition, the display position determination unit 22 may be configured to guide the viewpoint through sound and display by outputting a decorative display, a flashing action, an effect sound, etc. for drawing attention to the display information d1 of the display device 14 before displaying the display information d1 in a form corresponding to the positional relationship between the information corresponding position p in the scenery and the passenger position.

[0122] Then, in Fig.13 As shown, when the occupant u moves to a position forward of the window at the right rear part of the vehicle, the display position determination unit 22 determines the intersection of the dotted line connecting the information corresponding position p in the scenery and the occupant position with the display device 14 as the center point of the display position of the display information d2 that establishes a corresponding relationship with the information corresponding position p in the scenery.

[0123] Then, the display position determination unit 22 generates a display instruction for instructing to display the display information d2 (virtual object) at the determined display position, and inputs the instruction to the display control unit 26. The display form of the display information d2 is similar to that of the virtual object. Fig.12 The display form of the display information d1 is different.

[0124] Then, if the vehicle moves forward, the information corresponding position p in the scenery moves relatively to the right rear of the vehicle. In this case, the display position determination unit 22 determines the intersection point where the imaginary line connecting the information corresponding position p in the scenery and the passenger position intersects the display device 14 as the center point of the display position of the display information d3 that establishes a corresponding relationship with the information corresponding position p in the scenery.

[0125] Then, the display position determination unit 22 generates a display instruction for instructing the display device 14 to display the display information d3 (virtual object) at the determined display position, and inputs the instruction to the display control unit 26. Fig.13 The display form of the display information d2 is different.

[0126] In this way, the display position determination unit 22 determines the position that overlaps with the information corresponding position p in the scenery when viewed from the occupant u as the display position of the virtual object in the display device 14 based on the occupant position, the screen position, and the information corresponding position p. Furthermore, the display position determination unit 22 generates a display instruction that instructs to display a virtual object of a form corresponding to the positional relationship between the information corresponding position p in the scenery and the occupant position at the determined display position, and inputs the instruction to the display control unit 26.

[0127] Thus, an AR object or the like can be displayed in a form that seems to exist at the position p corresponding to the real information viewed via the display device 14 , thereby preventing the occupant from feeling uncomfortable with the scenery viewed through the display unit and improving the sense of immersion.

[0128] Fig.15 It is a flowchart for illustrating a display method of a display system. In the display system 1, first, a virtual object is registered (step S1). Specifically, advertisers, individuals, etc. who wish to process an object, etc., create a new virtual object by using a dedicated application installed in a smart phone, mobile terminal, etc., or freely customize a virtual object set along the driving line of the vehicle. In this way, it is possible to set a new virtual object, a processed virtual object, etc. In addition, customization includes, for example: adding a virtualized advertisement to an image obtained by photographing an existing building; depicting the entire building as a fictional building using computer graphics; etc. It is also possible to apply a virtual pattern or a virtual character to the image obtained by photographing an existing building. The customized object is registered in correspondence with the location of a building, scenery, etc., for example, in the map information on the central server 5.

[0129] Next, the display control unit 26 that has downloaded the map information compares the location of the map information with the current location of the vehicle (step S2). When the vehicle does not enter the specified range of a specific building on the distance map (step S3: "No"), the display control unit 26 repeats the processing after step S2. When the vehicle enters the specified range of a specific building on the distance map (step S3: "Yes"), the display control unit 26 combines the AR object with the real building corresponding to the vehicle position, and displays it on the display device 14. In addition, the display control unit 26 displays the VR object in a manner overlapping the real building (step S4).

[0130] In addition, the movement or shape of the virtual object may also change in accordance with the speed, driving position, etc. of the moving body. In addition, the display system 1 may also be configured so that a plurality of virtual objects can be selected by operating the passenger mobile terminal application (settlement application) or touching the icon displayed on the window. In addition, the display device 14 is not limited to having a function of changing transparency, and may not have a function of changing transparency.

[0131] As described above, the display device of the present embodiment is configured to include: a display unit provided in the window of a moving body; and a display control unit, which controls the display of a virtual object corresponding to the object in the display unit based on the positional relationship between the object seen through the display unit from the inside of the moving body and the moving body. With this structure, it is possible to include advertisements or AR objects freely arranged by the user in the scenery seen through the display unit while maintaining the old style of the urban landscape. Thus, in areas where regional revitalization is achieved, areas where the release of advertisements is restricted by strict landscape regulations, etc., it is possible to imagine unique scenery in a short time without damaging the urban landscape, and to build hot spots. In particular, in the case of a city with scarce tourism resources, by setting rare characters that are only displayed in the window of the moving body, it is possible to promote the use of cars, buses, taxis, etc. In addition, by working hard on the scenery through the display unit, it is possible to expect an increase in tourists, thereby improving the ability to attract customers to accommodation facilities, entertainment facilities, etc. In this way, by providing high-value-added information to the passengers of the moving body, it is possible to improve the competitiveness of the region and promote and strengthen employment.

[0132] In addition, the display method of this embodiment includes: a step of determining a display position when a virtual object corresponding to the object is displayed on a display unit set in a window of the moving body based on a positional relationship between the object seen through the display unit from the inside of the moving body and the moving body; and a step of displaying the virtual object at the display position.

[0133] In the above, various embodiments are described with reference to the accompanying drawings, but, of course, the present disclosure is not limited to these examples. As long as one is skilled in the art, it is obvious that various variations or modifications can be thought of within the scope described in the claims, and it should be understood that these variations or modifications also belong to the scope of the technology disclosed in the present disclosure. In addition, the constituent elements in the above embodiments can also be arbitrarily combined within the scope of the disclosed subject matter.

[0134] The specific examples of the present disclosure have been described in detail above, but these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes to the specific examples described above.

[0135] The disclosure of Japanese Patent Application No. 2020-050764 filed on March 23, 2020, including the specification, drawings, and abstract, is incorporated herein by reference in its entirety.

[0136] Industrial Applicability

[0137] An embodiment of the present disclosure is applicable to a display device and a vehicle.

[0138] Description of Reference Numerals

[0139] 1 Display system

[0140] 3 Mobile body

[0141] 4. Mountains

[0142] 5 Central Server

[0143] 6 Crew

[0144] 7 Objects

[0145] 8A Basics

[0146] 8B Object

[0147] 9A Icon

[0148] 9B Object

[0149] 14 Display device

[0150] 14a Electron transmission control film

[0151] 22 Display position determination unit

[0152] 26 Display control unit

[0153] 30 Vehicle Equipment

[0154] 32A Auxiliary storage device

[0155] 32B memory device

[0156] 32D interface device

[0157] 33 GPS module

[0158] 34 ACC switch

[0159] 35 Sensors

[0160] 36 Camera

[0161] 51 Communication equipment

[0162] 52 Information processing device

[0163] 321 Vehicle Information Transceiver Department

[0164] 323 Camera Information Management Department

[0165] 520 Storage

[0166] 5201 Communication Processing Department

[0167] 5205 Information Display Extraction Department

[0168] 5212 Vehicle Identification Department

[0169] 5213 Command sending unit

[0170] 5214 Map Matching Department

[0171] 5215 Exploration Information Generation Department

Claims

1. A display device comprising: A display unit is provided on the mobile body; and A display control unit controls display of a virtual object corresponding to the object in the display unit based on a positional relationship between the object and the moving body that is seen from the inside of the moving body through the display unit. in, In the first speed range, the display control unit causes the display unit to display the virtual object corresponding to the object that is within a predetermined distance from the moving body. The display control unit causes the display unit to display the virtual object corresponding to the object located farther away from the moving body than the predetermined distance when the vehicle speed is in a second speed range faster than the first speed range.

2. The display device according to claim 1, wherein: The virtual object is an augmented reality object.

3. The display device according to claim 2, wherein: The augmented reality object is added to the object that is visible through the display unit and displayed.

4. The display device according to claim 3, wherein: The augmented reality object is overlapped and displayed on the object passing through the display unit.

5. The display device according to claim 3, wherein: The augmented reality object is displayed at a position of the object that is visible through the display unit.

6. The display device according to claim 2, wherein: The augmented reality object is an object processed from an object in the real space.

7. The display device according to claim 2, wherein: The augmented reality object is displayed in a line of sight direction of an occupant of the moving object with respect to the scenery through the display unit.

8. The display device according to claim 1, wherein: The virtual object is a virtual reality object.

9. The display device according to claim 8, wherein: The virtual reality object is an object processed from an object in the real space.

10. The display device according to claim 1, wherein: The virtual object is an object related to the advertisement.

11. The display device according to claim 1, wherein: The virtual object is changed according to the attribute of the occupant of the moving body.

12. The display device according to claim 1, wherein: When the mobile body is traveling in the second speed range in the automatic driving mode, the display control unit causes the display unit to display the virtual object corresponding to the object, and when the vehicle speed decreases from the second speed range to the first speed range, the display control unit causes the virtual object to be non-displayed.

13. A vehicle comprising the display device according to claim 1.

14. A display method comprising the following steps: A step of determining a display position when displaying a virtual object corresponding to the object on the display unit based on a positional relationship between the object and the moving body that is viewed from the inside of the moving body through a display unit provided on the moving body; as well as The step of displaying the virtual object at the display position, Wherein, in the step of displaying the virtual object, In the first speed range, the virtual object corresponding to the object existing within a predetermined distance from the moving body is displayed at the display position, When the vehicle speed is in a second speed range that is faster than the vehicle speed in the first speed range, the virtual object corresponding to the object that is located farther than the predetermined distance from the moving body is displayed at the display position.

Images