Display device, virtual image control method, cockpit system, and transportation vehicle

Abstract

A display device (400) and a virtual image control method, which can be applied to in-vehicle systems and transportation vehicles. The display device (400) can simultaneously display a real image and a virtual image, and implement control of the virtual image by a user touching the real image, thereby realizing a low-delay and high-efficiency interaction solution between the user and a virtual image display device, and achieving the purpose of improving user experience. The display device (400) comprises a window unit (420), a first image generation unit (410), and an image magnification unit (430). The first image generation unit (410) is used for emitting first imaging light to the window unit (420). The window unit (420) is used for reflecting the first imaging light from the image generation unit (410) to the image magnification unit (430), and transmitting to a user the first imaging light reflected from the image magnification unit (430), the first imaging light being used for generating a virtual image. Additionally, the window unit (420) is also used for displaying a real image, so that the user controls the virtual image by touching the real image. The image magnification unit (430) is used for reflecting the first imaging light from the window unit (420) to the window unit (420).

Description

Display device, method of controlling virtual image, cockpit system and vehicle

[0001] This application claims priority to the Chinese patent application No. 202410696198.5, filed on May 30, 2024, and entitled "Display device, method of controlling virtual image, cockpit system and vehicle", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of display, and more particularly, to a display device, a method of controlling a virtual image, a cockpit system and a vehicle. BACKGROUND

[0003] Today, with the design of automobiles focusing on user experience, vehicle display devices are developing rapidly. In order to better realize human-vehicle interaction or improve the driving experience, larger, more and smarter vehicle display devices are applied to many smart cars. At present, virtual image display devices that use optical engine technology to break through the physical space limitations in the car gradually enter people's field of vision. Virtual image display devices are based on unique spatial optical engine design, use light field folding technology principles, and increase the viewing distance through multiple reflections of light, which can achieve the effect of presenting a 40-inch frame at a distance of 3 meters from the human eye. However, compared with traditional display devices, the virtual image generated by the virtual image display device cannot be touched and there are some problems in interaction. Therefore, how to realize human-computer interaction for the virtual image display device is a problem to be solved. SUMMARY

[0004] The present application provides a display device, a method of controlling a virtual image, a cockpit system and a vehicle. The display device provided by the present application can display real images and virtual images at the same time, and can realize interaction between the display device and the user through user touch control, thereby achieving the purpose of improving user experience.

[0005] In a first aspect, an embodiment of the present application provides a display device. The display device comprises a view window unit, a first image generation unit and an image magnification unit, wherein the first image generation unit is configured to emit first imaging light to the view window unit; the view window unit is configured to reflect the first imaging light from the image generation unit to the image magnification unit, and transmit the first imaging light reflected from the image magnification unit to the user, the first imaging light being used to generate a virtual image, the view window unit is also configured to display a real image on the view window unit, so that the user controls the virtual image by touching the real image; and the image magnification unit is configured to reflect the first imaging light from the view window unit to the view window unit.

[0006] Based on the above scheme, the display device provided in the application can enable a user to view a virtual image through a view window unit while displaying a real image, that is, to achieve the effect of simultaneously displaying a real image and a virtual image, so that the user can control the virtual image through the real image displayed by the view window unit, thereby realizing interaction with the display device and improving the user's experience.

[0007] With reference to the first aspect, in some implementations of the first aspect, the view window unit is configured to display the real image by using the second imaging light.

[0008] With reference to the first aspect, in some implementations of the first aspect, the view window unit comprises a display touch module, and the display touch module is configured to display the real image and realize touch control on a real image area.

[0009] With reference to the first aspect, in some implementations of the first aspect, the display touch module comprises one of a transparent display touch module, a projection display touch module, or an illumination display touch module.

[0010] With reference to the first aspect, in some implementations of the first aspect, the display touch module comprises the transparent display touch module, and the transparent display touch module is configured to generate the second imaging light by using image information of the real image, and the second imaging light is configured to display the real image on the view window unit.

[0011] With reference to the first aspect, in some implementations of the first aspect, the display device further comprises an image processing unit, and the image processing unit is configured to input the image information of the real image to the transparent display touch module.

[0012] With reference to the first aspect, in some implementations of the first aspect, the transparent display touch module covers all or part of a surface of the view window unit.

[0013] The transparent display touch module generates a real image and realizes touch control, which not only increases the touch area of the outer surface of the view window unit, but also enables the transparent display touch module to generate different real images according to the application scenario of the display device, thereby increasing the application scenario of the display device.

[0014] With reference to the first aspect, in some implementations of the first aspect, the display touch module comprises the projection display touch module, and the display device further comprises a second image generation unit, the second image generation unit is configured to emit the second imaging light to the view window unit, and the projection display touch module is configured to generate the real image on the view window unit by using the second imaging light emitted by the second image generation unit.

[0015] With reference to the first aspect, in some implementations of the first aspect, the projection display touch module covers part of the surface of the view window unit.

[0016] By means of the projection display touch module, a real image is displayed on the view window unit, and the display device has a touch function. The position of the real image displayed on the view window unit can be changed flexibly, so that the interactivity between the display device and the user is enhanced, and the user experience is improved.

[0017] With reference to the first aspect, in some implementations of the first aspect, the display touch module comprises the illumination display touch module, and the display device further comprises: a light source, configured to emit a light beam to the view window unit; and the illumination display touch module, configured to illuminate the real image on the illumination display touch module by using the light beam emitted by the light source.

[0018] With reference to the first aspect, in some implementations of the first aspect, the illumination display touch module covers part of the surface of the view window unit.

[0019] By means of the illumination display touch module, the display of the real image and the touch function are realized, and the cost of the touch module can be reduced. The scheme is simple and easy to implement.

[0020] With reference to the first aspect, in some implementations of the first aspect, the user touches the real image by using a finger. With reference to the first aspect, in some implementations of the first aspect, when the user touches the real image, the real image changes in at least one of the following manners: the brightness of the real image changes from a first brightness to a second brightness, the brightness value of the second brightness is greater than that of the first brightness; the frame size of the real image changes from a first frame size to a second frame size, the area of the second frame size is greater than that of the first frame size; and the real image displays a bright edge.

[0021] The second aspect provides a method for controlling a virtual image. The method is applied to a display device, and comprises: in response to a touch operation of a user on a real image displayed on the display device, a virtual image displayed by the display device changes from a first virtual image to a second virtual image, the first virtual image is generated by transmitting first imaging light of the display device, and the second virtual image is generated by transmitting second imaging light of the display device.

[0022] With reference to the second aspect, in some implementations of the second aspect, the user touches the real image by using a finger.

[0023] With reference to the second aspect, in some implementations of the second aspect, when the user touches the real image, the real image changes in at least one of the following manners: a brightness of the real image changes from a first brightness to a second brightness, the second brightness having a brightness value greater than that of the first brightness; a frame size of the real image changes from a first frame size to a second frame size, the second frame size having an area greater than that of the first frame size; the real image displays a bright border.

[0024] In a third aspect, an embodiment of the present application provides a cockpit system, which comprises the display device provided by the first aspect and any implementation manner of the first aspect, and a seat for the user to sit on.

[0025] In a fourth aspect, an embodiment of the present application provides a vehicle, which comprises the display device provided by the first aspect and any implementation manner of the first aspect, or the cockpit system provided by the third aspect.

[0026] With reference to the fourth aspect, in some implementations of the fourth aspect, the display device is arranged at at least one of a headrest of a seat of the vehicle, a backrest of the seat of the vehicle, and an instrument panel of the vehicle.

[0027] The beneficial effects brought by the second aspect to the fourth aspect can be referred to the description of the beneficial effects of the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0028] FIG. 1 is a functional schematic diagram of a vehicle 100 according to an embodiment of the present application.

[0029] FIG. 2 is a schematic diagram of an application scenario of an intelligent cockpit display system 200 according to an embodiment of the present application.

[0030] FIG. 3 is a schematic diagram of an application scenario of another intelligent cockpit display system 200 according to an embodiment of the present application.

[0031] FIG. 4 is a structural schematic diagram of a first display device 400 according to an embodiment of the present application.

[0032] FIG. 5 shows two forms of a transparent display touch module covering a window unit according to an embodiment of the present application.

[0033] FIG. 6 is a structural schematic diagram of a first window unit 420 according to an embodiment of the present application.

[0034] FIG. 7 is a structural schematic diagram of a second window unit 420 according to an embodiment of the present application.

[0035] FIG. 8 is a schematic diagram of user touch interaction of the display device 400 in two scenarios of being covered by different transparent display touch modules according to an embodiment of the present application.

[0036] FIG. 9 is a structural schematic diagram of a second display device 900 according to an embodiment of the present application.

[0037] FIG. 10 is a structural schematic diagram of two window units 920 according to an embodiment of the present application.

[0038] FIG. 11 is a structural schematic diagram of a third display device 1100 according to an embodiment of the present application.

[0039] FIG. 12 is a structural schematic diagram of two window units 1120 according to an embodiment of the present application.

[0040] FIG. 13 is a possible side structural perspective view of a display apparatus 30 according to an embodiment of the present application.

[0041] FIG. 14 is a schematic diagram of a cockpit of a vehicle according to an embodiment of the present application.

[0042] FIG. 15 is a circuit schematic diagram of a display device according to an embodiment of the present application.

[0043] FIG. 16 is a possible functional framework schematic diagram of a vehicle according to an embodiment of the present application.

[0044] FIG. 17 is a schematic functional block diagram of a mobile carrier 25 according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the present application will be described below with reference to the accompanying drawings.

[0046] In order to facilitate understanding of the embodiments of the present application, the following explanations are made.

[0047] First, in the textual explanations in the embodiments of the present application shown below or the terms in the accompanying drawings, “first”, “second”, and the like as well as various numerical numbers are only for differentiation for convenience of description, and are not intended to limit the scope of the embodiments of the present application. For example, the first imaging light and the second imaging light are used to differentiate different imaging lights.

[0048] Second, the term “comprise” and any variations thereof in the embodiments of the present application shown below are intended to cover the non-exclusive inclusion, for example, a system, a product or a device comprising a series of units does not have to be limited to only those clearly listed units, but can include other units that are not clearly listed or inherent to these products or devices.

[0049] Third, in the embodiments of the present application, the words such as "exemplarily" or "for example" are used to represent examples, illustrations or descriptions, and the embodiments or design schemes described as "exemplarily" or "for example" should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. The words such as "exemplarily" or "for example" are intended to present relevant concepts in a specific manner and facilitate understanding.

[0050] Fourth, in the embodiments of the present application, the imaging light refers to light carrying an image (or image information) and used for generating an image, which can also be referred to as image light and the like. The light beam refers to light not carrying image information and only used for illumination.

[0051] Fifth, in the drawings of the present application, the thickness, size and shape of each optical element have been slightly exaggerated for the purpose of illustration. Specifically, the shapes of the optical elements shown in the drawings are shown by way of example, and the drawings are drawn only as examples and not strictly according to scale.

[0052] Sixth, unless otherwise defined, all the terms (including technical terms and scientific terms) used in the present application have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0053] It can be understood that the embodiments described in the present application are only a part of the embodiments of the present application, not all the embodiments. Those skilled in the art can know that, with the development of technology and the appearance of new scenes, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0054] Automobiles are developing and innovating in the tide of electrification, networking, intelligence, sharing and the like. FIG. 1 is a functional schematic diagram of a vehicle 100 provided in an embodiment of the present application. The vehicle 100 can include various subsystems, such as an infotainment system 110, a perception system 120, a decision control system 130, a drive system 140 and a computing platform 150. Alternatively, the vehicle 100 can include more or fewer subsystems, and each subsystem can include multiple components. In addition, each subsystem and component of the vehicle 100 can be interconnected by wired or wireless means.

[0055] In some embodiments, the infotainment system 110 can include a communication system 111, an entertainment system 112 and a navigation system 113.

[0056] The communication system 111 can include a wireless communication system that can wirelessly communicate with one or more devices directly or via a communication network. For example, the wireless communication system can use third generation (3G) cellular communication technology, such as code division multiple access (CDMA), or fourth generation (4G) cellular communication technology, such as long time evolution (LTE) communication technology. Or fifth generation (5G) cellular communication technology, such as new radio (NR) communication technology. The wireless communication system can utilize WiFi and wireless local area network (WLAN) communication. In some embodiments, the wireless communication system can utilize infrared links, Bluetooth, or ZigBee to communicate directly with devices. Other wireless protocols, such as various vehicle communication systems, for example, the wireless communication system can include one or more dedicated short range communications (DSRC) devices, which can include public and / or private data communication between vehicles and / or roadside stations.

[0057] The entertainment system 112 can include a center screen, a microphone, and a sound system, based on which a user can listen to the radio or play music in the vehicle, or connect the phone with the vehicle and realize the phone screen projection on the center screen. The center screen can be touchable, and the user can operate it by touching the screen. In some cases, the user's voice signal can be obtained through the microphone, and some control of the vehicle 100 by the user can be realized according to the analysis of the user's voice signal, such as adjusting the temperature in the vehicle, etc. In other cases, music can be played to the user through the sound system.

[0058] The navigation system 113 can include a map service provided by a map provider, thereby providing the vehicle 100 with navigation of the driving route. The navigation system 113 can be used in cooperation with the global positioning system 121 and the inertial measurement unit 122 of the vehicle. The map service provided by the map provider can be a two-dimensional map or a high-definition map.

[0059] The perception system 120 can include several sensors that sense information about the environment surrounding the vehicle 100. For example, the perception system 120 can include a global positioning system 121 (which can be a global position satellite (GPS) system, a Beidou system, or other positioning system), an inertial measurement unit (IMU) 122, a lidar 123, a millimeter wave radar 124, an ultrasonic radar 125, and a camera 126. The perception system 120 can also include sensors that monitor the internal systems of the vehicle 100 (e.g., an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their respective characteristics (location, shape, direction, speed, etc.). Such detection and identification are key functions for the safe operation of the vehicle 100.

[0060] The positioning system 121 can be used to estimate the geographic location of the vehicle 100. The inertial measurement unit 122 is used to sense changes in position and orientation of the vehicle 100 based on inertial acceleration. In some embodiments, the inertial measurement unit 122 can be a combination of an accelerometer and a gyroscope. The lidar 123 can utilize laser light to sense objects in the environment in which the vehicle 100 is located. In some embodiments, the lidar 123 can include one or more laser sources, a laser scanner, and one or more detectors, among other system components. The millimeter wave radar 124 can utilize radio signals to sense objects within the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, the radar 126 can also be used to sense the speed and / or direction of travel of the objects. The ultrasonic radar 125 can utilize ultrasonic signals to sense objects around the vehicle 100. The camera 126 can be used to capture image information of the surrounding environment of the vehicle 100. The camera 126 can include a monocular camera, a binocular camera, a structured light camera, and a panoramic camera, among others, and the image information acquired by the camera 126 can include still images or video stream information.

[0061] The decision control system 130 includes a computing system 131 that makes analytical decisions based on the information acquired by the perception system 120, and the decision control system 130 also includes a vehicle controller 132 that controls the power system of the vehicle 100, as well as a steering system 133, an acceleration pedal 134 (including an acceleration pedal of an electric vehicle or a throttle of a fuel vehicle, which is an exemplary term here) and a braking system 135 of the vehicle 100.

[0062] The computing system 131 can operate to process and analyze the various information acquired by the perception system 120 in order to identify targets, objects, and / or features in the environment surrounding the vehicle 100. The targets can include pedestrians or animals, and the objects and / or features can include traffic signals, road boundaries, and obstacles. The computing system 131 can use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and / or the like. In some embodiments, the computing system 131 can be used to map the environment, track objects, estimate the velocity of objects, and / or the like. The computing system 131 can analyze the acquired information and derive a control strategy for the vehicle.

[0063] The vehicle controller 132 can be used to coordinate the control of the power battery and the drive 141 of the vehicle 100 to improve the power performance of the vehicle 100.

[0064] The steering system 133 can be used to adjust the heading direction of the vehicle 100. For example, the steering system 133 can be a steering wheel system in one embodiment. The accelerator pedal 134 can be used to control the operating speed of the drive 141 and, in turn, the speed of the vehicle 100.

[0065] The braking system 135 can be used to control the deceleration of the vehicle 100. The braking system 135 can use friction to slow the wheels 144. In some embodiments, the braking system 135 can convert the kinetic energy of the wheels 144 into electrical current. The braking system 135 can also take other forms to slow the wheels 144 and, in turn, control the speed of the vehicle 100.

[0066] The drive system 140 can include components that provide motive power for the vehicle 100. In one embodiment, the drive system 140 can include a drive 141, an energy source 142, a transmission 143, and wheels 144. The drive 141 can be an internal combustion engine, an electric motor, an air compression engine, or other types of engine combinations, such as a hybrid engine that includes a gasoline engine and an electric motor, a hybrid engine that includes an internal combustion engine and an air compression engine. The drive 141 converts the energy source 142 into mechanical energy.

[0067] Examples of the energy source 142 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electrical power. The energy source 142 can also provide energy for other systems of the vehicle 100.

[0068] The transmission 143 can transmit the mechanical power from the drive 141 to the wheels 144. The transmission 143 can include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 143 can also include other devices, such as a clutch. The drive shaft can include one or more shafts that can be coupled to one or more wheels 121.

[0069] Some or all of the functionality of the vehicle 100 is controlled by a computing platform 150. The computing platform 150 can include at least one processor 151 that can execute instructions 153 stored in a non-transitory computer readable medium such as a memory 152. In some embodiments, the computing platform 150 can also be a plurality of computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

[0070] The processor 151 can be any conventional processor such as a central process unit (CPU). Alternatively, the processor 151 can also include a graphics process unit (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), or a combination thereof. Although FIG. 1 functionally illustrates the processor, the memory, and other elements of the computer 110 in the same block, those of ordinary skill in the art will appreciate that the processor, the computer, or the memory can actually include multiple processors, computers, or memories that can or can not be stored in the same physical housing. For example, the memory can be a hard drive or other storage medium located in a different housing than that of the computer 110. Thus, references to the processor or the computer will be understood to include references to a collection of processors or computers or memories that can or can not operate in parallel. Rather than using a single processor to perform the steps described herein, some components such as the steering component and the deceleration component can each have their own processor that only performs calculations related to the functionality specific to the component.

[0071] In various aspects described herein, the processor can be located remotely from the vehicle and in wireless communication with the vehicle. In other aspects, some of the processes described herein are performed on a processor disposed within the vehicle while others are performed by a remote processor, including taking the necessary steps to perform a single maneuver.

[0072] In some embodiments, the memory 152 can contain instructions 153 (e.g., program logic) that can be executed by the processor 151 to perform various functions of the vehicle 100. The memory 152 can also contain additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the infotainment system 110, the perception system 120, the decision control system 130, and the drive system 140.

[0073] In addition to instructions 153, memory 152 can store data, such as road maps, route information, the vehicle's position, orientation, speed, and other such vehicle data, and other information. Such information can be used by vehicle 100 and computing platform 150 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0074] Computing platform 150 can control the functions of vehicle 100 based on inputs received from various subsystems, such as drive system 140, perception system 120, and decision control system 130. For example, computing platform 150 can utilize inputs from decision control system 130 in order to control steering system 133 to avoid obstacles detected by perception system 120. In some embodiments, computing platform 150 can be operable to provide control over many aspects of vehicle 100 and its subsystems.

[0075] Optionally, one or more of the above-described components can be installed separately from or associated with vehicle 100. For example, memory 152 can exist partially or entirely separately from vehicle 100. The above-described components can be communicatively coupled together in a wired and / or wireless manner.

[0076] Optionally, the above-described components are just an example, in actual applications, components in each module described above can be added or deleted according to actual needs, and FIG. 1 should not be understood as a limitation on the embodiments of the present application.

[0077] Optionally, vehicle 100 can be configured in a fully or partially autonomous driving mode. For example, vehicle 100 can obtain surrounding environment information of itself through perception system 120, and obtain an autonomous driving strategy based on analysis of the surrounding environment information to realize full autonomous driving, or present the analysis result to a user to realize partial autonomous driving.

[0078] An autonomous vehicle traveling on a roadway, such as vehicle 100 above, can identify objects within its surrounding environment to determine an adjustment to a current speed. The objects can be other vehicles, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently, and based on respective characteristics of the object, such as its current speed, acceleration, spacing from the vehicle, etc., can be used to determine a speed at which the autonomous vehicle is to adjust.

[0079] Optionally, the vehicle 100 or a perception and computing device (e.g., the computing system 131, the computing platform 150) associated with the vehicle 100 can predict the behavior of the identified object based on the characteristics of the identified object and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of the other identified objects, so the behavior of a single identified object can also be predicted by considering all of the identified objects together. The vehicle 100 can adjust its speed based on the predicted behavior of the identified object. In other words, the autonomous vehicle can determine what state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the object. In this process, other factors can also be considered to determine the speed of the vehicle 100, such as the lateral position of the vehicle 100 in the road, the curvature of the road, the proximity of static and dynamic objects, etc.

[0080] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device can also provide instructions to modify the steering angle of the vehicle 100 to cause the autonomous vehicle to follow a given trajectory and / or maintain a safe lateral and longitudinal distance from objects in the vicinity of the autonomous vehicle (e.g., a car in the adjacent lane on the road).

[0081] The vehicle 100 described above can be a car, a truck, a bus, a ship, an airplane, a helicopter, an entertainment vehicle, an amusement park vehicle, a construction device, a trolley, a train, etc., and the embodiments of the present application do not make special limitations.

[0082] It can be understood that the display device, the method of controlling the virtual image, and the cockpit system provided by the embodiments of the present application can be applied to the vehicle as shown in FIG. 1, and the vehicle 100 described above can be a vehicle including the cockpit as shown in FIG. 2 and / or FIG. 3. It should be noted that the display device provided by the present application can be applied to a left-hand drive vehicle or a right-hand drive vehicle. The embodiments of the present application do not limit the type of vehicle.

[0083] FIG. 2 is a schematic diagram of an application scenario of an intelligent cockpit display system 200 to which the embodiments of the present application are applicable. As shown in FIG. 2, the intelligent cockpit display system 200 includes at least one display device 101 and at least one seat 102, and FIG. 2 is exemplarily shown with one display device and one seat, and the display device 101 is arranged on the backrest of the seat 102. The display device 101 can generate an enlarged virtual image at a far distance image plane through the input of an external video signal (which can also be referred to as a signal source), provide a large-format, far-distance visual experience for a viewer, and meet the needs of users in various application scenarios such as leisure and entertainment, business office work, etc.

[0084] It should be noted that the display device 101 can also be installed on the headrest of the seat 102. Alternatively, when the intelligent cockpit display system also includes an instrument panel, the display device 101 can also be installed on the instrument panel, as shown in the cockpit system 300 of FIG. 3. When the display device 101 is installed on the instrument panel 202, the display device 101 can also be designed in a form capable of being accommodated into the instrument panel 202, at this time, the shell of the display device 101 can be designed and shaped according to the shape and color of the instrument panel, achieving perfect unity with the appearance of the instrument panel 202, and achieving the purpose of giving consideration to the beauty of the cockpit system 300. In order to further improve the intelligent effect of the cockpit system, in the system 300, a detector can also be used to detect the posture and position of the user and the like, and the display device 101 can be automatically displayed according to the posture and position of the user and the like. For example, when it is detected that the user is located in front of the instrument panel 202 or the eyes of the user look at the display device 101, the display device 101 can be automatically raised or slid out from the instrument panel 202, and after being automatically adjusted to a suitable position and angle, the display device 101 can display images.

[0085] It should also be noted that in the embodiments of the present application, the display device 101 can be installed on the back of the seat 102, the headrest or the instrument panel 202 before leaving the factory. Alternatively, it can also be installed on the back of the seat 102, the headrest or the instrument panel 202 after leaving the factory by modifying the seat 102, the headrest or the instrument panel 202, and the present application does not make any limitation.

[0086] It can be understood that the intelligent cockpit display systems 200 and 300 shown in FIGS. 2 and 3 are only examples, and the intelligent cockpit system to which the embodiments of the present application are applicable can also include an intelligent steering wheel, a head up display (HUD) and the like. That is, the intelligent cockpit display system to which the embodiments of the present application are applicable is not limited to the intelligent cockpit display systems 200 and 300 shown in FIGS. 2 and 3, but can also be other systems containing the intelligent cockpit display systems 200 and 300 shown in FIGS. 2 and 3 or other systems similar to FIGS. 2 and 3, and the present application does not make any limitation. It should also be noted that the above FIGS. 2 and 3 are only vehicle application scenarios to which the embodiments of the present application are applicable, that is, the display device provided by the present application can be applied to a display device in other display systems, in other words, the application of the display device provided by the embodiments of the present application includes but is not limited to the application in a vehicle display system.

[0087] Currently, there are some interactive solutions of virtual image display. For example, a user can realize voice control interaction through voice recognition, voice synthesis and natural language processing technology, or realize gesture interaction through capturing user gestures by using an image detector (such as a camera) and converting the gestures into machine instructions, or realize eye movement interaction through pupil positioning and analysis by using eye tracking technology and converting the pupil positioning and analysis into machine instructions, and realize out-of-space interaction effect by using a remote control device. Among them, in the voice control interaction and gesture interaction solutions, a central processing unit (CPU) with strong computing power is required, which leads to high power consumption and cost of the display device. The eye movement interaction solution needs to rely on a human eye tracking algorithm with high precision, otherwise it will misjudge and affect the user experience effect. In order to capture the position of the human eye and analyze the human eye, the display device is also configured with a human eye following module, which also increases the cost and power consumption. Although in the pointing remote control interaction solution, the out-of-space touch effect can be realized, the remote control device matched with the display device is usually small in size, which is easy to be lost or damaged during use, resulting in that the display device cannot be used, and thus the user experience is reduced. In order to improve the interaction experience of the virtual image display device, more and more researches will focus on the touch solution. However, in the existing solution using a touch screen, the virtual image display device can only display a real image or a virtual image independently, that is, in such a touch solution, when a touch operation is performed, the virtual image display device functions as a touch panel, and cannot realize real human-computer interaction. Therefore, the present application provides a display device, which reduces the interaction cost and power consumption through touch interaction, and enables a user to control a virtual image through a touch real image while the display device displays the virtual image, so as to improve the user experience.

[0088] The display device provided by the embodiments of the present application is described in detail below.

[0089] FIG. 4 is a structural schematic diagram of a first display device 400 provided by an embodiment of the present application. The display device 400 can be applied to the vehicle 100 shown in FIG. 1, and the vehicle 100 can include the cockpit virtual image display system shown in FIG. 2 and / or FIG. 3. As shown in FIG. 4, the display device 400 is sequentially arranged with an image generation unit 410, a view window unit 420 and an image magnification unit 430 along the transmission direction of the imaging light. The display device 400 can display virtual images and real images simultaneously. Specifically, when the display device 400 displays virtual images and real images, the image generation unit 410 emits first imaging light to the view window unit 420, the first imaging light is reflected by the view window unit 420 to the image magnification unit 430, and then is reflected by the image magnification unit 430 to the view window unit 420 again, and the view window unit 420 transmits the first imaging light to a user, so that the user can view the virtual image formed by the first imaging light through the first imaging light transmitted by the view window unit 420. At the same time, the view window unit 420 displays a real image, so that the user can control the virtual image viewed by the user through the touch of the real image, that is, the user can perform human-computer interaction with the display device 400 through the touch.

[0090] In the display device 400 shown in FIG. 4, the view window unit 420 includes a transparent display touch module, which is arranged on the side of the view window unit 420 viewed by the human eye, that is, on the outer surface of the view window unit 420 close to the user, for displaying a real image and realizing touch control on the real image area. It can be understood that the transparent display touch module generates second image light according to the input image information, and the second image light is used to generate a corresponding real image, so as to be displayed on the view window unit 420. In other words, in the display device 400 shown in FIG. 4, the transparent display touch module in the view window unit 420 is connected with an image processing unit, which is used to input image information of a real image to the transparent display touch module. In an implementable manner, the image processing unit can be a part of the display device 400, and in this case, the display device 400 includes the image processing unit; or in another implementable manner, the image processing unit is an external image processor, for example, when the display device of the present application is applied to a vehicle display, it can be another image processor in the vehicle, that is, the image processor not only provides image information of a real image to the display device 400, but also has other functions, and in this case, the display device 400 does not include the image processing unit.

[0091] It should be noted that the transparent display touch module included in the view window unit 420 is not limited in the present application, for example, the transparent display touch module can be a touch screen based on transparent organic light-emitting diode (OLED) display, or a touch screen based on transparent liquid crystal display (LCD), or a touch screen based on a new transparent display technology developed in the future, etc., which are all within the protection scope of the present application. For example, when the transparent display touch module included in the view window unit 420 is a touch screen based on transparent OLED, the touch screen based on transparent OLED displays a real image based on the second imaging light. At this time, the image processing unit inputs the image information of the real image to the touch screen based on transparent OLED, and the touch screen based on transparent OLED drives the self-luminous pixels to light according to the image information, that is, generates the second imaging light, so that the corresponding real image is displayed on the touch screen based on transparent OLED.

[0092] In some embodiments, the outer surface of the view window unit 420, that is, the surface that can be touched by the user, is all transparent display touch modules, at this time, the size of the transparent display touch module is the same as the size of the view window unit 420. For example, in the schematic diagram of the interactive area covered by the transparent display touch module shown in FIG. 5, (a) in FIG. 5 is a schematic diagram in which the outer surface of the view window unit is all used for interaction. It can be understood that when the view window unit 420 is all covered by the transparent display touch module, the display device 400 shown in FIG. 4 can realize the display effect that the real image is superimposed on the virtual image.

[0093] In other embodiments, only a part of the specific area of the outer surface of the view window unit 420 is used for touch operation by the user. For example, (b) in FIG. 5 is a schematic diagram in which a part of the outer surface of the view window unit is used for interaction. Specifically, if the part of the transparent display touch module used for interaction does not affect the viewing of the long-distance virtual image, it can be kept open, that is, the part of the touch module is kept in a long-light state, and the other part of the transparent display touch module is always transparent, at this time, the real image is always displayed on the part of the area used for interaction.

[0094] It should be noted that the present application does not limit the real image displayed by the display device 400, that is, the content displayed on the transparent display touch module can include but is not limited to various application (application, APP) play icons, real image pictures synchronized with the virtual image, icons for controlling the virtual image picture, buttons, etc. It can be understood that when the display device 400 displays the virtual image and the real image, the user can view the virtual image through the first imaging light transmitted by the transparent display touch module, and the user can perform touch operations on the transparent display touch module as needed. For example, switching APPs to realize switching of virtual image display content; or when the virtual image is displayed as a video animation or a game interface, the sound, brightness, color, etc. of the video can be adjusted through icons, buttons, etc., or game interaction can be performed through icons, buttons, etc.; or when the virtual image is displayed as a real image picture synchronized with the virtual image, such as an office document or a picture, the document or picture can be modified on the real image, and the enlarged virtual image is presented.

[0095] It can be understood that in the present application, whether the entire outer surface of the window unit or part of the area is used for user touch operation, the part used for touch is the transparent display touch module, which can sense whether the user contacts the surface of the module and determine the contact position according to the conductor (such as a human finger), so that the transparent display touch module displaying the real image can recognize the click, long press or sliding operation, etc. of the user based on the contact position of the user, and realize the corresponding function of the operation. For example, when the displayed real image is various APP play icons, the user can select the required application through the click operation; or when the displayed real image is a real image picture synchronized with the virtual image, such as an image, the user can browse the picture through the sliding operation; or when the displayed real image is a real image video or music synchronized with the virtual image, the user can adjust the volume, brightness, etc. through the long press operation. It should be noted that the present application does not limit the user's operation and the content displayed on the transparent display touch module displaying the real image, which can be the above-mentioned real image types and touch operations, or other real images and other ways of touch operation, and can be customized according to the user's needs and use habits, or designed and configured according to the type of vehicle, the use scene of the display device, etc.

[0096] It can also be understood that the area covered by the transparent display touch module shown in FIG. 5 is only illustrative, that is, when the transparent display touch module only covers part of the surface of the window unit 420, the present application does not limit the position of the partial transparent display touch module, which can be distributed on one side of the outer surface of the window unit 420 as shown in FIG. 5, or in other embodiments, the transparent display touch module can also be distributed on the bottom or top of the outer surface of the window unit 420, etc.

[0097] It should be noted that the present application does not limit the conductor used for touch control, which can be a user's finger, a pen with touch control function, a glove with touch control function, etc. In some embodiments, the method for the transparent display touch module to sense the conductor contact module can be direct sensing, i.e., the conductor for touch control directly contacts the transparent display touch module, i.e., sensing is performed by using the transparent display touch module. Illustratively, the transparent display touch module can be designed with a sensing unit (which can also be referred to as a touch control unit), such as a pressure sensor, a capacitive sensor, etc., to sense whether the conductor for touch control contacts the transparent display touch module through the sensor. It can be understood that when the transparent display touch module contains a sensor, the type of the sensor is related to the touch control scheme adopted by the transparent display touch module, which is not limited in the present application. In other embodiments, the sensing method of the transparent display touch module can be indirect sensing, i.e., the conductor for touch control can not contact the transparent display touch module. Illustratively, a camera module can be used to sense the conductor for touch control by capturing the position of the conductor. For example, the camera module can be a fixed-focus camera module, which senses the conductor for touch control after clearly capturing the conductor for touch control. For another example, the camera module can calculate the distance between the conductor for touch control and the transparent display touch module through the captured image, and sense the conductor for touch control when the distance is less than or equal to a threshold value.

[0098] In order to achieve the effect of energy saving, in the present application, the transparent display touch module can be set as a non-long-bright module, at which time the touch module needs to be woken up. It can be understood that the transparent display touch module can be woken up after sensing (direct sensing or indirect sensing) the conductor for touch control. Illustratively, if a finger is used for touch control, when the user's finger contacts the touch module or the user's finger approaches the touch module, the non-long-bright part of the touch module is woken up, thereby entering an interactive mode, so that the user can perform corresponding touch control operations. Correspondingly, when the user does not perform touch control operations within a predetermined period of time (which can be a time preset for the display device when it is manufactured or a time set by the user after it is manufactured, etc., which is not limited), the part of the touch module automatically exits the interactive mode, at which time the part of the touch module for touch control no longer displays a real image.

[0099] It can be understood that the display device 400 can also separately display the real image and the virtual image. When the display device 400 displays the virtual image, the image generating unit 410 is configured to emit the first imaging light to the view window unit 420. The view window unit 420 is configured to reflect the first imaging light from the image generating unit 420 to the image amplifying unit 430, and transmit the first imaging light reflected by the image amplifying unit 430 to the human eye, and the view window unit 420 is also configured to enable the human eye to view the virtual image formed by the first imaging light through the view window unit 420. The image amplifying unit 430 is configured to reflect the imaging light from the view window unit 420 to the view window unit 420. When the display device 400 displays the real image, the image generating unit 410 is closed, the view window unit 420 is configured to generate the real image, and is configured to enable the user to interact with the display device 400 through touch. It should be noted that when the display device 400 displays the real image, in order not to interfere with the displayed real image, the purpose of improving the user interaction experience can be achieved by closing the virtual image screen.

[0100] In order to improve the user experience and enhance the interactive effect, optionally, when the user touches the real image displayed on the view window unit 420, the touched real image can change accordingly, so that the user can determine whether the touched real image is correct according to the change of the real image. The change of the touched real image can include but is not limited to at least one of the following: the brightness of the real image becomes brighter, that is, the first brightness when not touched changes to the second brightness with a larger brightness value after being touched; the size of the real image becomes larger, that is, the first size when not touched changes to the second size with a larger size (size, size area, etc.); or the real image displays a bright edge, that is, the real image without a bright edge when not touched changes to the real image with a bright edge after being touched, and the like.

[0101] It should be noted that the structure of the view window unit 420 is not limited in the present application. It can be understood that when the view window unit 420 included in the display device 400 can realize the splitting of the imaging light, and can be used to generate and display the real image, and also be used for the user to perform the touch operation, all of them are within the protection scope of the present application.

[0102] In an implementable manner, FIG. 6 is a structural schematic diagram of a first window unit 420 provided by an embodiment of the present application. As shown in FIG. 6, the window unit 420 includes a substrate 610, a light splitting film system 620, and a transparent display touch module 630. The substrate 610 includes a first surface 611 and a second surface 612 distributed oppositely, wherein the first surface 611 can be understood as an inner surface of the substrate 610, i.e., located inside the display device 400, that is, the surface of the substrate 610 far away from the user; the second surface 612 can be understood as an outer surface of the substrate 610, i.e., located outside the display device 400, that is, the surface of the substrate 610 close to the user. The light splitting film system 620 is arranged outside the first surface 611 of the substrate 610, and the transparent display touch module 630 is arranged outside the second surface 612 of the substrate 610. Specifically, the light splitting film 620 is used to reflect the first imaging light emitted from the image generation unit 410 to the image magnifying unit 430, and transmit the first imaging light reflected by the image magnifying unit 430 to the substrate 610, which enters the human eye after sequentially transmitting the first surface 611 and the second surface 612 of the substrate 610, so that the human eye can observe the virtual image (which can be a picture, a document, a video, etc.) displayed by the display device 400. At the same time, the transparent display touch module 630 displays images that can be touched by the user, including but not limited to the same real image as the virtual image, touch icons, etc., so that the user can control the virtual image by touching the real image or touch icons on the transparent display touch module 630, thereby realizing interaction with the display device 400.

[0103] In another possible implementation, FIG. 7 is a structural schematic diagram of a second window unit 420 provided by the embodiment of the present application. As shown in FIG. 7, the window unit 420 includes a substrate 710, a light splitting film system 720, a transparent display touch module 730, a polarized film layer 740, and an anti-reflection film 750. The substrate 710 includes a first surface 711 and a second surface 712 distributed oppositely, wherein the first surface 711 and the second surface 712 are respectively an inner surface and an outer surface of the substrate 710, i.e., the first surface 711 is located inside the display device 400, i.e., the surface of the substrate 710 far from the user; and the second surface 712 is located outside the display device 400, i.e., the surface of the substrate 710 close to the user. The light splitting film system 720 is arranged outside the first surface 711 of the substrate 710, the polarized film layer 740 is arranged outside the second surface 712 of the substrate 710, the transparent display touch module 730 is arranged outside the polarized film layer 740, and the anti-reflection film 750 is arranged outside the transparent display touch module 730. Specifically, the light splitting film 720 is configured to reflect the first imaging light emitted from the image generation unit 410 to the image magnifying unit 430, and transmit the first imaging light reflected by the image magnifying unit 430 to the substrate 710, which is sequentially transmitted through the first surface 711, the second surface 712, the polarized film layer 740, the transparent display touch module 730, and the anti-reflection film 750 of the substrate 710, and then enters the human eye, so that the human eye can observe the virtual image (which can be a picture, a document, a video, etc.) displayed by the display device 400. At the same time, the transparent display touch module 730 displays an image that can be touched by the user, including but not limited to the same real image as the virtual image, a touch icon, etc., so that the user controls the virtual image through touching the real image or the touch icon, thereby realizing the interaction with the display device 400. It can be understood that in FIG. 7, when the user performs the touch operation, the surface actually operated by the user is the surface of the anti-reflection film 750, while in the window unit 420 shown in FIG. 6, the surface actually operated by the user is the surface of the transparent display touch module 630. It can also be understood that the anti-reflection film 750 on the window unit 420 can reduce the stray light entering the display device 400, and the polarized film layer 740 has an absorption effect on the stray light entering from the outside of the display device 400, i.e., when the anti-reflection film and the polarized film layer are arranged on the window unit 420, the stray light entering the display device 400 can be reduced, thereby achieving the effect of improving the display quality.

[0104] It should be noted that when the window unit 420 provided by the present application contains a light splitting film system, the light splitting film system is composed of at least one dielectric film. That is, the light splitting film system 620 in FIG. 6 and the light splitting film system 720 in FIG. 7, and the light splitting film system contained in other window units 420 not listed in the embodiments of the present application, contains at least one dielectric film for realizing reflection and transmission of imaging light. Among them, the material of the at least one dielectric film can be silicon oxide such as SiO2, magnesium fluoride such as MgF2, niobium oxide such as Nb2O5, titanium oxide such as TiO2, indium tin oxide ITO, zinc sulfide ZnS, chromium oxide, nickel oxide, aluminum oxide, etc., which is not limited by the present application. At the same time, when the light splitting film system 520 includes multiple layers of dielectric films, whether the materials of the multiple layers of dielectric films are completely the same or not is not limited by the present application, that is, the materials of the multiple layers of dielectric films of the light splitting film system contained in the window unit 420 provided by the present application can be completely the same, or completely different, or partially the same. Exemplarily, for the light splitting film system 620 shown in FIG. 6, if the light splitting film system 620 includes three layers of dielectric films, at this time, the first layer of dielectric film, the second layer of dielectric film and the third layer of dielectric film can be completely the same; or the materials of the first layer of dielectric film, the second layer of dielectric film and the third layer of dielectric film are completely different; or the materials of two layers of dielectric films of the first layer of dielectric film, the second layer of dielectric film and the third layer of dielectric film are the same, for example, the material of the first layer of dielectric film and the material of the third layer of dielectric film are the same, but different from the material of the second layer of dielectric film, etc., which will not be described here.

[0105] It can be understood that in the structure of the window unit shown in FIG. 6 and FIG. 7, the transparent display touch module can be completely covered on the surface of the window unit 420, that is, the surface of the window unit 420 can be used for user touch operation; the transparent display touch module can also be partially covered on the surface of the window unit, that is, only a specific area in the surface of the window unit 420 is used for user touch interaction. Exemplarily, FIG. 8 is a schematic diagram of user touch interaction in two different transparent display touch module coverage scenarios of the display device 400 provided by the embodiments of the present application.

[0106] FIG. 9 is a structural schematic diagram of a second display device 900 provided in an embodiment of the present application. The display device 900 can be applied to the vehicle 100 shown in FIG. 1, and the vehicle 100 can include the virtual image display system for a cockpit as shown in FIG. 2 and / or FIG. 3. As shown in FIG. 9, the display device 900 is sequentially arranged with a first image generation unit 910, a view window unit 920 and an image magnification unit 930 along a transmission direction of the first imaging light. The display device 900 is sequentially arranged with a second image generation unit 940 and the view window unit 920 along a transmission direction of the second imaging light. The display device 400 can display virtual images and real images simultaneously. Specifically, when the display device 400 displays virtual images and real images, the first image generation unit 910 emits the first imaging light to the view window unit 920, the first imaging light is reflected by the view window unit 920, and then reflected by the image magnification unit 930 again to the view window unit 920, and the view window unit 920 transmits the first imaging light to a user, so that the user can view the virtual image formed by the first imaging light through the first imaging light transmitted by the view window unit 920. At the same time, the second image generation unit 940 projects the second imaging light to the view window unit 920, and the second imaging light generates a real image on the view window unit 920, so that the user can control the virtual image viewed by the user through the touch real image, that is, the user can perform human-computer interaction with the display device 900 through the touch.

[0107] In the display device 900 shown in FIG. 9, the view window unit 920 includes a projection display touch module (specifically, refer to the structure of the view window unit 920 in FIG. 10 below), which is arranged on the side of the view window unit that is viewed by the human eye, that is, on the outer surface of the view window unit close to the user. It should be noted that the projection display touch module included in the view window unit 920 is not limited in the present application, for example, it can be an LCD-based projection display touch module, etc., which are all within the protection scope of the present application. It can be understood that in the description of the embodiments of the present application, the name of the projection display touch module is only exemplary, and other names can also be used, for example, it can also be called a projection display touch module, etc., which are not limited in the present application.

[0108] In the display device 900 shown in FIG. 9, the projection display touch module covers part of the outer surface of the view window unit 920, that is, the area of the projection display touch module is smaller than the area of the view window unit 920. However, the position of the projection display touch module is not limited in the present application. For the sake of simplicity of the description, the part of the description can refer to the related description of (b) in FIG. 5 above, which will not be repeated here.

[0109] The same as the display device 400 shown in FIG. 4, in the scheme shown in FIG. 9, the projection display touch module for the part used for touch can perceive whether the user contacts the surface of the module screen according to the conductor (for example, human finger) and determine the contact position, so that the projection display touch module of the display real image can identify the click, long press or sliding operation and the like of the user based on the contact position of the user, and realize the corresponding function of the operation. Exemplarily, when the displayed real image is various APP play icons, the user can select the required application through the click operation; or when the displayed real image is a real image picture synchronized with the virtual image, for example, an image, the user can perform picture browsing through the sliding operation; or when the displayed real image is a real image video or music synchronized with the virtual image, the user can adjust the volume, brightness and the like through the long press operation. It can be understood that the operation of the user and the displayed real image in the scheme are also not limited, and can be customized according to the needs and use habits of the user, or be designed and configured according to the type of vehicle, the use scene of the display device and the like.

[0110] In addition, in the scheme shown in FIG. 9, the touch conductor is also not limited, and the perception of the conductor by the projection display touch module can be direct perception or indirect perception. The awakening and exiting of the projection display touch module can also refer to the description of the display device 400 in FIG. 4, which will not be repeated here.

[0111] It should be noted that the structure of the window unit 920 is not limited in the present application. It can be understood that when the window unit included in the display device can realize the splitting of the imaging light, can be used for displaying the real image, and can be used for touch operation by the user, all of them are within the protection scope of the present application.

[0112] Exemplarily, FIG. 10 shows two possible structures of the view window unit 920. As shown in FIG. 10, in (a) of FIG. 10, the view window unit 920 includes a substrate 1010, a light splitting film system 1020, and a projection display touch module 1030. Compared with the view window unit 420 shown in FIG. 6, in (a) of FIG. 10, the view window unit 920 contains the projection display touch module 1030 instead of the transparent display touch module. Specifically, the light splitting film 1020 is configured to reflect the first imaging light emitted from the first image generating unit 910 to the image magnifying unit 930, and transmit the first imaging light reflected from the image magnifying unit 930 to the substrate 1010. After sequentially transmitting the light splitting film 1020 and the substrate 1010, the first imaging light enters the human eye, so that the human eye can observe the virtual image (which can be a picture, a document, a video, etc.) displayed by the display device 900. Meanwhile, the second imaging light emitted from the second image generating unit 940 sequentially transmits the light splitting film 1020 and the substrate 1010, and generates an image on the projection display touch module 1030, which can be touched by the user, including but not limited to the same real image as the virtual image, a touch icon, etc. Thus, the user can control the virtual image by touching the real image or the touch icon on the projection display touch module 1030, so as to interact with the display device 900.

[0113] In (b) of FIG. 10, the view window unit 920 includes a substrate 1010, a light splitting film system 1020, a projection display touch module 1030, a polarized film layer 1040, and an anti-reflection film 1050. Compared with the view window unit 420 shown in FIG. 7, in (b) of FIG. 10, the view window unit 920 replaces the transparent display touch module with the projection display touch module 1030. Specifically, the light splitting film 1020 is configured to reflect the first imaging light emitted from the first image generating unit 910 to the image magnifying unit 930, and transmit the first imaging light reflected from the image magnifying unit 930 to the substrate 1010. After sequentially transmitting the substrate 1010, the polarized film layer 1040, the projection display touch module 1030, and the anti-reflection film 1050, the first imaging light enters the human eye, so that the human eye can observe the virtual image displayed by the display device 900. Meanwhile, the second imaging light emitted from the second image generating unit 940 sequentially transmits the light splitting film 1020, the substrate 1010, and the polarized film layer 1040, and generates an image on the projection display touch module 1030, which can be touched by the user, including but not limited to the same real image as the virtual image, a touch icon, etc. Thus, the user can control the virtual image by touching the real image or the touch icon on the projection display touch module 1030, so as to interact with the display device 900.

[0114] It should be noted that the touch module shown in FIG. 6 or FIG. 7 is a transparent display touch module, and the projection display touch module included in the view window unit of the display device 900 is not necessarily transparent when displaying a real image. In addition, for the display device 900 to perform touch interaction with the user, the related description in FIG. 8 can also be referred to.

[0115] It should be noted that in some other implementations, when the real image displayed on the display device is generated by projection by the image generation unit, the image generation unit for generating the real image by projection can not be arranged inside the display device. In other words, for the display device that uses projection to perform touch interaction, the image generation unit for generating the real image by projection can be arranged inside the display device, or can be arranged outside the display device, and when arranged outside the display device, the volume of the display device can be reduced. For example, for the display device 900, the second image generation unit 940 can also be arranged outside the display device.

[0116] It can be understood that the projection display touch module in FIG. 9 can also be a non-long-bright module, and at this time the user can perform touch operation on the touch module by wake-up. Specifically, the description in FIG. 4 can be referred to, and details are not repeated here.

[0117] It can be understood that the display device 900 can also display a real image and a virtual image separately. When the display device 900 displays a virtual image, the first image generation unit 910 is used to emit first imaging light to the view window unit 920. The view window unit 920 is used to reflect the first imaging light from the first image generation unit 920 to the image magnification unit 930, and transmit the first imaging light reflected by the image magnification unit 930 to the human eye, and the view window unit 920 is also used for the human eye to view the virtual image formed by the first imaging light through the view window unit 920. The image magnification unit 930 is used to reflect the first imaging light from the view window unit 920 to the view window unit 920. When the display device 900 displays a real image, the first image generation unit 910 is closed, and the second image generation unit 940 projects second imaging light to the view window unit 920, and the second imaging light is used to generate a real image on the view window unit 920. The view window unit 920 is used to display the real image generated by the second imaging light, and is used for the user to perform human-computer interaction with the display device 900 by touch.

[0118] FIG. 11 is a structural schematic diagram of a third display device 1100 provided by an embodiment of the present application. The display device 1100 can be applied to the vehicle 100 shown in FIG. 1, and the vehicle 100 can include the cockpit virtual image display system shown in FIG. 2 and / or FIG. 3. As shown in FIG. 11, the display device 1100 is sequentially arranged along the transmission direction of the imaging light with an image generation unit 1110, a view window unit 1120, an image magnification unit 1130, and a light source 1140. The display device 1100 can simultaneously display a virtual image and a real image. Specifically, when the display device displays a virtual image and a real image, the image generation unit 1110 emits first imaging light to the view window unit 1120, the first imaging light is reflected by the view window unit 1120 to the image magnification unit 1130, and then is reflected by the image magnification unit 1130 to the view window unit 1120 again, and the view window unit 1120 transmits the first imaging light to a user, so that the user can view the virtual image formed by the first imaging light through the first imaging light transmitted by the view window unit 1120. At the same time, the light source 1140 projects an illumination light beam to the view window unit 1120, which illuminates the real image on the view window unit 1120, and the user can interact with the display device 1100 through touch.

[0119] It can be understood that the display device 1100 can also separately display a real image and a virtual image. Specifically, when the display device 1100 displays a virtual image, the image generation unit 1110 is configured to emit first imaging light to the view window unit 1120. The view window unit 1120 is configured to reflect the first imaging light from the image generation unit 1120 to the image magnification unit 1130, and transmit the first imaging light reflected by the image magnification unit 1130 to a human eye, and the view window unit 1120 is also configured to enable the human eye to view the virtual image formed by the first imaging light through the view window unit 1120. The image magnification unit 1130 is configured to reflect the first imaging light from the view window unit 1120 to the view window unit 1120. When the display device 1100 displays a virtual image, the light source 1140 projects a light beam to the view window unit 1120, which is configured to illuminate the image on the view window unit 1120. The view window unit 1120 displays a real image based on the light beam emitted by the light source 1140, and is configured to enable a user to interact with the display device 1100 through touch.

[0120] In the display device 1100 shown in FIG. 11, the view window unit 1120 includes an illumination display touch module (see FIG. 12 for details), which is arranged on the side of the view window unit that is viewed by the human eye, i.e., on the outer surface of the view window unit close to the user. It should be noted that the present application does not limit the illumination display touch module included in the view window unit 1120, which can be an illumination display touch module made of a film, etc., and all are within the protection scope of the present application.

[0121] In the display device 1100 shown in FIG. 11, the illumination display touch module covers part of the outer surface of the view window unit 1120, that is, the area of the illumination display touch module is smaller than the area of the view window unit 1120. It can be understood that the position of the illumination display touch module is not limited in the present application, and will not be described here.

[0122] It can be understood that in the display devices shown in FIG. 9 and FIG. 11, the display touch module covers part of the outer surface of the view window unit, and compared with the illumination display touch module in the display device 1100, the projection display touch module included in the display device 900 in FIG. 9 generates a real image position that changes according to the position of the second image light incident, that is, in the display device 900 shown in FIG. 9, the position and size of the real image can be changed.

[0123] The same as the display device 400 in FIG. 4, in the scheme shown in FIG. 11, the illumination display touch module for touch, which is part of the display device, can sense whether a user contacts the surface of the module curtain according to the conductor (such as a human finger) and determine the contact position, so that the illumination display touch module of the display real image can identify the click, long press or sliding operation and the like performed by the user based on the contact position of the user, and realize the corresponding function of the operation. It should be noted that in the display device 1100 described in FIG. 11, since the real image is generated in advance on the view window unit 1120, for example, the film is formed into a pattern by etching, therefore, the real image displayed in the display device 1100 is a fixed real image. It can be understood that the present scheme is also not limited to the user's operation and the displayed real image, and can be customized according to the user's needs and use habits, or designed and configured according to the type of vehicle, the use scene of the display device and the like.

[0124] In addition, in the scheme shown in FIG. 11, the touch conductor is also not limited, and the perception of the conductor by the illumination display touch module can be direct perception or indirect perception. The wake-up and exit of the illumination display touch module can also refer to the description of the display device 400 in FIG. 4 above, which will not be described here.

[0125] It should be noted that the structure of the view window unit 1120 is not limited in the present application, and it can be understood that when the view window unit included in the display device can realize the splitting of the imaging light, can be used for displaying a real image, and can be used for touch operation by a user, all of them are within the protection scope of the present application.

[0126] Exemplarily, FIG. 12 shows two possible structures of the view window unit 1120. As shown in FIG. 12, in (a) of FIG. 12, the view window unit 1120 includes a substrate 1210, a light splitting film system 1220, and an illumination display touch module 1230. Compared with the view window unit 420 shown in FIG. 6, in (a) of FIG. 12, the view window unit 1120 contains the illumination display touch module 1230 instead of the transparent display touch module. Specifically, the light splitting film 1220 is configured to reflect the first imaging light emitted from the image generation unit 1110 to the image magnifying unit 1130, and transmit the first imaging light reflected from the image magnifying unit 1130 to the substrate 1210, which, after sequentially transmitting the light splitting film 1220 and the substrate 1210, enters the human eye, so that the human eye can observe the virtual image (which can be a picture, a document, a video, etc.) displayed by the display device 1100. Meanwhile, the illumination light beam emitted from the light source 1140, after sequentially transmitting the light splitting film 1220 and the substrate 1210, irradiates on the illumination display touch module 1230, so that the illumination display touch module 1230 displays the image (which includes but is not limited to the same real image as the virtual image, a touch icon, etc.) that can be touched by the user, so that the user realizes the control of the virtual image by touching the real image or the touch icon on the illumination display touch module 1230, thereby realizing the interaction with the display device 1100.

[0127] In (b) of FIG. 12, the view window unit 1120 includes a substrate 1210, a light splitting film system 1220, an illumination display touch module 1230, a polarizing film layer 1240, and an anti-reflection film 1250. Compared with the view window unit 420 shown in FIG. 7, in (b) of FIG. 12, the view window unit 1120 replaces the transparent display touch module with the illumination display touch module 1230. Specifically, the light splitting film 1220 is configured to reflect the first imaging light emitted from the image generation unit 1110 to the image magnifying unit 1130, and transmit the first imaging light reflected from the image magnifying unit 1130 to the substrate 1210, which, after sequentially transmitting the substrate 1210, the polarizing film layer 1240, the illumination display touch module 1230, and the anti-reflection film 1250, enters the human eye, so that the human eye can observe the virtual image displayed by the display device 1100. Meanwhile, the illumination light beam emitted from the light source 1140, after sequentially transmitting the light splitting film 1220, the substrate 1210, and the polarizing film layer 1240, irradiates on the illumination display touch module 1230, so that the illumination display touch module 1230 displays the image (which includes but is not limited to the same real image as the virtual image, a touch icon, etc.) that can be touched by the user, so that the user realizes the control of the virtual image by touching the real image or the touch icon on the illumination display touch module 1230, thereby realizing the interaction with the display device 1100.

[0128] It should be noted that the touch module shown in Figure 6 or Figure 7 above is a transparent display touch module, while the illumination display touch module included in the window unit in the display device 1100 is not transparent when displaying a real image. Furthermore, for the user touch interaction scenario used by the display device 1100, please refer to Figure 8(b) above.

[0129] It should be noted that in some other embodiments, when the real image displayed on the display device is generated by illumination from a light beam, the light source used to emit the light beam to illuminate the real image may not be located inside the display device. In other words, for a display device that uses light beam illumination for touch interaction, it needs to be equipped with an additional light source. This light source can be located inside or outside the display device. When located outside the display device, the size of the display device can be reduced. For example, for display device 1100, the light source 1140 may also be located outside the display device.

[0130] It is understood that the display device 1100 can also display real and virtual images separately. When the display device 1100 displays a virtual image, the image generation unit 1110 emits a first imaging light into the viewing window unit 1120. The viewing window unit 1120 reflects the first imaging light from the image generation unit 1120 to the image magnification unit 1130 and transmits the reflected first imaging light from the image magnification unit 1130 to the human eye. The viewing window unit 1120 is also used by the human eye to view the virtual image formed by the first imaging light through the viewing window unit 1120. The image magnification unit 1130 reflects the first imaging light from the viewing window unit 1120 back to the viewing window unit 1120. When the display device 1100 displays a real image, the image generation unit 1110 is turned off, and the light source 1140 projects a light beam into the viewing window unit 1120, which illuminates the image on the viewing window unit 1120. The window unit 1120 displays a real image based on the light beam emitted from the light source 1140 and is used for human-computer interaction between the user and the display device 1100 via touch.

[0131] It should be noted that Figures 4, 9, and 11 above are merely examples of three display devices provided in the embodiments of this application, but the structure of the display device provided in the embodiments of this application is not limited to the structures shown in Figures 4, 9, and 11. In some other embodiments, the image generation unit or the image magnification unit in the display device provided in this application may be arranged in other positions, that is, the relative positional relationship between the image generation unit, the window unit, and the image magnification unit is not limited in this application. In addition, the display device provided in the embodiments of this application may also include other units, such as a rotation unit, such as a rotation shaft or a rotation gear, which can rotate the display device to change the angle of the display device to accommodate users of different heights.

[0132] It should also be noted that, in the various display devices provided in this application, the image generation unit (including the image unit 410 in FIG4, the first image generation unit 910 and the second image generation unit 940 in FIG9, and the image generation unit 1110 in FIG11) can be an LCD display, a liquid crystal on silicon (LCOS) display, an OLED display, a micro-LED display, a display using miniLED display technology, a digital light processing (DLP) display, or a micro-electro-mechanical system (MEMS) display, etc., and this application does not limit it.

[0133] Optionally, in the various display devices provided in this application, the image magnification unit is a freeform mirror, that is, the image magnification unit 420 in FIG4, the image magnification unit 920 in FIG9, and the image magnification unit 1120 in FIG11 are freeform mirrors.

[0134] Figure 13 is a possible side view of a display device 30 applicable to an embodiment of this application. As shown in Figure 13, the display device 30 includes an image generation unit 310, a window unit 320, an image magnification unit 330, and a first housing 340. The window unit 320 can be any of the above embodiments, such as the window unit 420 in Figure 6. The window unit 320 includes an outer surface 321 and an inner surface 322. One end of the first housing 340 is connected to the upper edge of the window unit 320, and the other end of the first housing 340 is connected to the lower edge of the window unit 320, such that the first housing 340 and the window unit 320 form a closed cavity. The image generation unit 310 and the image magnification unit 330 are disposed within this cavity, and the inner surface 322 of the window unit 320 is located within this cavity. Specifically, when the display device 30 is operating, the image generation unit 310 emits a first imaging light onto the inner surface 322 of the viewing window unit 320. This first imaging light is reflected by the inner surface 322 of the viewing window unit 320 and transmitted to the surface of the image magnification unit 330. After being reflected by the image magnification unit 330, it reaches the inner surface 322 of the viewing window unit 320 again, is transmitted through the inner surface 322, and exits from the outer surface 321 of the viewing window unit 320. When a user views an image through the outer surface 321 of the viewing window unit 320, the image light emitted from the outer surface 321 of the viewing window unit 320 enters the user's eye, allowing the user to see a virtual image located at the image plane. Simultaneously, the user can see a real image displayed on the viewing window unit 320.

[0135] Optionally, the display device 30 also includes a second housing, which is fixed to the periphery of the outer surface 321 of the window unit 320 by means of adhesive or screws. Electronic components, such as buttons, indicator lights, microphones, etc., may also be provided on the second housing to realize different functions of the display device 30.

[0136] It should be noted that the first outer shell 340 can be a single, complete shell. Alternatively, the first outer shell 340 can also be formed by connecting multiple partial shells, for example, by connecting them together through decorative strips.

[0137] It should be noted that Figure 13 is only an example of the display device 400 corresponding to Figure 4 above. That is, for the display devices shown in Figures 9 and 11, as well as other display devices not listed in the embodiments of this application, there can be display devices similar to Figure 13, including the structure and housing of the corresponding display devices, which will not be described in detail here.

[0138] Figure 14 is a schematic diagram of the cockpit of a vehicle according to an embodiment of this application. As shown in Figure 14, the vehicle includes a display device 2000, which is disposed inside the instrument panel. The display device 2000 may be the display device 30 in Figure 13, or it may be a display device that includes the display device 900 and the display device 1100.

[0139] Optionally, in addition to being installed on the dashboard, the display device 2000 provided in this application embodiment can also be installed on other structural equipment of the vehicle, such as the seat back, the inner wall of the vehicle, the control panel, the processing table, etc., and this application does not limit it in this regard.

[0140] Optionally, the operating table can be a resuscitation operating table on an ambulance, a kitchen operating table on a motorhome, etc.; the processing table can be a bar on a motorhome, a dining table in a passenger cabin, an office desk, etc., and this application does not limit it in this regard.

[0141] Figure 15 is a circuit diagram of the display device provided in an embodiment of this application. As shown in Figure 15, the circuit in the display device mainly includes a main processor (host CPU) 1201, an external memory interface 1202, an internal memory 1203, an audio module 1204, a video module 1205, a power supply module 1206, a wireless communication module 1207, an I / O interface 1208, a video interface 1209, a display circuit 12100, and a modulator 1212. The main processor 1201 and its peripheral components, such as the external memory interface 1202, the internal memory 1203, the audio module 1204, the video module 1205, the power supply module 1206, the wireless communication module 1207, the I / O interface 1208, the video interface 1209, and the display circuit 12100, can be connected via a bus. The main processor 1201 can be referred to as a front-end processor.

[0142] Furthermore, the circuit diagrams illustrated in the embodiments of this application do not constitute a specific limitation on the display device. In other embodiments of this application, the display device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0143] The main processor 1201 includes one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units can be independent devices or integrated into one or more processors.

[0144] The main processor 1201 may also include a memory for storing instructions and data. In some embodiments, the memory in the main processor 1201 is a cache memory. This memory can store instructions or data that the main processor 1201 has just used or is recurring. If the main processor 1201 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the main processor 1201, and thus improves the efficiency of the system.

[0145] In some embodiments, the display device may further include multiple input / output (I / O) interfaces 1208 connected to the main processor 1201. Interfaces 1208 may include Inter-Integrated Circuit (I2C) interfaces, Inter-Integrated Circuit Sound (I2S) interfaces, Pulse Code Modulation (PCM) interfaces, Universal Asynchronous Receiver / Transmitter (UART) interfaces, Mobile Industry Processor Interface (MIPI) interfaces, General-Purpose Input / Output (GPIO) interfaces, Subscriber Identity Module (SIM) interfaces, and / or Universal Serial Bus (USB) interfaces, etc. The aforementioned I / O interfaces 1208 can connect to devices such as mice, touchpads, keyboards, cameras, speakers, microphones, etc., and can also connect to physical buttons on the display device (e.g., volume buttons, brightness adjustment buttons, power buttons, etc.).

[0146] The external memory interface 1202 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the display device. The external memory card communicates with the main processor 1201 through the external memory interface 1202 to perform data storage functions.

[0147] Internal memory 1203 can be used to store executable program code, including instructions. Internal memory 1203 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as a call function, time setting function, etc.), etc. The data storage area may store data created during the use of the display device (such as a phone book, world time, etc.). Furthermore, internal memory 1203 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, Universal Flash Storage (UFS), etc. The main processor 1201 executes various functional applications and data processing of the display device by running instructions stored in internal memory 1203 and / or instructions stored in memory located in the main processor 1201.

[0148] The display device can implement audio functions, such as music playback and phone calls, through the audio module 1204 and application processor.

[0149] The audio module 1204 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 1204 can also be used for encoding and decoding audio signals, such as for playback or recording. In some embodiments, the audio module 1204 may be located in the main processor 1201, or some functional modules of the audio module 1204 may be located in the main processor 1201.

[0150] The video interface 1209 can receive externally input audio and video signals, specifically including High Definition Multimedia Interface (HDMI), Digital Visual Interface (DVI), Video Graphics Array (VGA), and DisplayPort (DP). The video interface 1209 can also output video. When the display device is used as an in-vehicle display, the video interface 1209 can receive speed and power signals from peripheral devices, as well as externally input VR video signals. When the display device is in use, the video interface 1209 can receive video signals from an external computer or terminal device.

[0151] The video module 1205 can decode the video input from the video interface 1209, such as performing H.264 decoding. The video module can also encode video captured by the display device, such as performing H.264 encoding on video captured by an external camera. Furthermore, the main processor 1201 can also decode the video input from the video interface 1209 and then output the decoded image signal to the display circuit 12100.

[0152] The display circuit 12100 and modulator 1212 are used to display the corresponding image. In this embodiment, the video interface 1209 receives an externally input video source signal. The video module 1205 performs decoding and / or digitization processing and outputs one or more image signals to the display circuit 12100. The display circuit 12100 drives the modulator 1212 to image the incident polarized light according to the input image signal, and then outputs image light. In addition, the main processor 1201 can also output one or more image signals to the display circuit 12100.

[0153] In this embodiment, the display circuit 12100 and the modulator 1212 are electronic components in the image generation unit described above, and the display circuit 12100 can be referred to as the driving circuit.

[0154] The power module 1206 provides power to the main processor 1201 and the light source 1200 based on the input power (e.g., DC power). The power module 1206 may include a rechargeable battery, which can provide power to the main processor 1201 and the light source 1200. The light emitted by the light source 1200 can be transmitted to the modulator 1212 for imaging, thereby forming an image light signal.

[0155] The wireless communication module 1207 enables the display device to communicate wirelessly with the outside world. It can provide solutions for wireless communication such as Wireless Local Area Networks (WLAN) (e.g., Wireless Fidelity (Wi-Fi)), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR). The wireless communication module 1207 can be one or more devices integrating at least one communication processing module. The wireless communication module 1207 receives electromagnetic waves via an antenna, modulates and filters the electromagnetic wave signal, and sends the processed signal to the main processor 1201. The wireless communication module 1207 can also receive signals to be transmitted from the main processor 1201, modulate and amplify them, and then convert them into electromagnetic waves for radiation via the antenna.

[0156] In addition, the video data decoded by the video module 1205 can be input not only through the video interface 1209, but also wirelessly received through the wireless communication module 1207 or read from external memory. For example, the display device can receive video data from the terminal device or the in-vehicle entertainment system through the vehicle's wireless local area network, and the display device can also read audio and video data stored in external memory.

[0157] The aforementioned display device can be installed on a vehicle. Please refer to Figure 16, which is a schematic diagram of a possible functional framework of a vehicle provided in an embodiment of this application.

[0158] As shown in Figure 16, the functional framework of a vehicle may include various subsystems, such as the sensor system 12, control system 14, one or more peripheral devices 16 (one is shown as an example), power supply 18, computer system 20, and in-vehicle display system 22. Optionally, the vehicle may also include other functional systems, such as an engine system that provides power to the vehicle, etc., which are not limited herein.

[0159] The sensor system 12 may include several detection devices that can sense the measured information and convert the sensed information into electrical signals or other required forms of information output according to a certain rule. As shown in the figure, these detection devices may include a global positioning system (GPS), a vehicle speed sensor, an inertial measurement unit (IMU), a radar unit, a laser rangefinder, a camera device, a wheel speed sensor, a steering sensor, a gear sensor, or other components used for automatic detection, etc., and this application does not limit them.

[0160] The control system 14 may include several components, such as the steering unit, braking unit, lighting system, automatic driving system, map navigation system, network time synchronization system, and obstacle avoidance system shown in the figure. Optionally, the control system 14 may also include components such as a throttle controller and an engine controller for controlling the vehicle's speed; this application is not limiting.

[0161] Peripheral device 16 may include several components, such as the communication system, touch module, user interface, microphone, and speaker shown in the figure. The communication system is used to enable network communication between the vehicle and other devices. In practical applications, the communication system can employ wireless or wired communication technologies to achieve network communication between the vehicle and other devices. The wired communication technology can refer to communication between the vehicle and other devices via network cables or fiber optic cables.

[0162] Power source 18 represents a system that provides electricity or energy to the vehicle, which may include, but is not limited to, rechargeable lithium batteries or lead-acid batteries. In practical applications, one or more battery components in the power source are used to provide electrical energy or power for vehicle startup, and the type and materials of the power source are not limited in this application.

[0163] Several functions of the vehicle are controlled and implemented by the computer system 20. The computer system 20 may include one or more processors 2001 (the illustration shows one processor as an example) and a memory 2002 (also referred to as a storage device). In practical applications, the memory 2002 may be located inside the computer system 20 or outside the computer system 20, for example, as a cache in the vehicle; this application does not limit this. The processor 2001 may include one or more general-purpose processors, such as a graphics processing unit (GPU). The processor 2001 can be used to run relevant programs or instructions corresponding to programs stored in the memory 2002 to implement the corresponding functions of the vehicle.

[0164] The memory 2002 may include volatile memory, such as RAM; it may also include non-volatile memory, such as ROM, flash memory, HDD, or SSD; or it may include a combination of the above types of memory. The memory 2002 can be used to store a set of program code or instructions corresponding to the program code, so that the processor 2001 can call the program code or instructions stored in the memory 2002 to implement the corresponding functions of the vehicle. In this application, the memory 2002 may store a set of program code for vehicle control. The processor 2001 can call this program code to control the safe driving of the vehicle. How to achieve safe driving of the vehicle is detailed below in this application.

[0165] Optionally, in addition to storing program code or instructions, the memory 2002 may also store information such as road maps, driving routes, and sensor data. The computer system 20 can be integrated with other components in the vehicle functional framework diagram, such as sensors in the sensor system and GPS, to realize the vehicle's related functions. For example, the computer system 20 can control the vehicle's direction of travel or speed based on data input from the sensor system 12; this application does not impose limitations on this.

[0166] The in-vehicle display system 22 may include several components, such as a controller and an in-vehicle display. The controller 222 generates images (e.g., images of VR content) according to user instructions and sends the images to the in-vehicle display for display. The in-vehicle display may include an image generation unit, a viewing unit, and an image magnification unit, allowing passengers to view the target image displayed on the in-vehicle display through the viewing unit. Some of the functions of the components in the in-vehicle display system can also be implemented by other subsystems of the vehicle; for example, the controller may also be a component within the control system.

[0167] Figure 16 of this application illustrates four subsystems: sensor system 12, control system 14, computer system 20, and in-vehicle display system 22. These are merely examples and do not constitute a limitation. In practical applications, vehicles can combine various components according to different functions to obtain subsystems with corresponding functions. In practical applications, vehicles may include more or fewer systems or components; this application does not impose any limitations.

[0168] The aforementioned means of transportation can be cars, trucks, buses, ships, airplanes, helicopters, recreational vehicles, trains, etc., and this application does not impose any special limitations on them.

[0169] Figure 17 is a schematic functional block diagram of a mobile carrier 25 provided in an embodiment of this application. The mobile carrier 25 may include a sensing system 120, a display device 130, and a computing platform 150. The sensing system 120 may include one or more sensors for sensing information about the environment surrounding the mobile carrier 25. For example, the sensing system 120 may include a positioning system, which may be a global positioning system (GPS), a BeiDou system or other positioning systems, an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and one or more of a camera device.

[0170] Some or all of the functions of the mobile carrier 25 can be controlled by the computing platform 150. The computing platform 150 may include one or more processors, such as processors 151 to 15n (n is a positive integer). A processor is a circuit with signal processing capabilities. In one implementation, the processor may be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a type of microprocessor), or a digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationship of hardware circuits. The logical relationship of the hardware circuits is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In a reconfigurable hardware circuit, the process of the processor loading a configuration document and implementing the hardware circuit configuration can be understood as the process of the processor loading instructions to implement some or all of the functions of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. In addition, the computing platform 150 may also include a memory for storing instructions. Some or all of the processors 151 to 15n can call and execute the instructions in the memory to achieve the corresponding functions. The display device 130 in the cockpit is a display device suitable for the embodiments of this application, such as the display device 300 in the above embodiments.

[0171] The mobile carrier in this application can include road vehicles, water vehicles, air vehicles, or entertainment equipment. For example, the mobile carrier can be a vehicle, which is a vehicle in a broad sense, and can be a means of transportation (such as commercial vehicles, passenger cars, trains, etc.), amusement equipment, toy vehicles, etc. The embodiments of this application do not specifically limit the type of vehicle. As another example, the mobile carrier can be a means of transportation such as an airplane or a ship.

[0172] Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning as understood by one of ordinary skill in the art to which this disclosure pertains. The above description is merely one embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, or improvements made based on this application should be included within the scope of protection of this application.

Claims

1. A display device, characterized in that, The display device comprises: a window unit, a first image generating unit and an image magnifying unit, wherein the first image generating unit is configured to emit first imaging light to the window unit; the window unit is configured to reflect the first imaging light from the image generating unit to the image magnifying unit, and transmit the first imaging light reflected from the image magnifying unit to a user, the first imaging light being used to generate a virtual image, and the window unit is further configured to display a real image on the window unit, so that the user controls the virtual image by touching the real image; the image magnifying unit is configured to reflect the first imaging light from the window unit to the window unit.

2. The display device according to claim 1, wherein The window unit is particularly configured to display the real image by using second imaging light.

3. The display device according to claim 2, wherein The window unit comprises a display touch module, the display touch module is configured to display the real image and realize touch on a real image area.

4. The display device according to claim 3, wherein The display touch module comprises one of a transparent display touch module, a projection display touch module or an illumination display touch module.

5. The display device according to claim 4, wherein The display touch module comprises the transparent display touch module, the transparent display touch module is configured to generate the second imaging light by using image information of the real image, and the second imaging light is used to display the real image on the window unit.

6. The display device according to claim 5, wherein The display device further comprises an image processing unit, wherein the image processing unit is configured to input the image information of the real image to the transparent display touch module.

7. The display device according to any one of claims 4 to 6, wherein The transparent display touch module covers all or part of the surface of the window unit.

8. The display device according to claim 4, wherein The display touch module comprises the projection display touch module, and the display device further comprises a second image generating unit, the second image generating unit is configured to emit the second imaging light to the window unit; the projection display touch module is configured to generate the real image on the window unit by using the second imaging light emitted by the second image generating unit.

9. The display device according to claim 4 or 8, wherein The projection display touch module covers part of the surface of the window unit.

10. The display device according to claim 4, wherein The display touch module comprises the illumination display touch module, and the display device further comprises a light source, the light source is configured to emit a light beam to the window unit; the illumination display touch module is configured to illuminate the real image on the illumination display touch module by using the light beam emitted by the light source.

11. The display device according to claim 4 or 10, wherein The illumination display touch module covers part of the surface of the window unit.

12. The display device according to any one of claims 1 to 11, wherein The user touches the real image by a finger.

13. The display device according to any one of claims 1 to 12, wherein When the user touches the real image, the real image changes in at least one of the following: the brightness of the real image changes from a first brightness to a second brightness, the brightness value of the second brightness being greater than that of the first brightness; the frame size of the real image changes from a first frame size to a second frame size, the area of the second frame size being greater than that of the first frame size; the real image displays a bright edge.

14. A method of controlling a virtual image, the method comprising: The display device comprises: in response to a touch operation of a user on a real image displayed on the display device, a virtual image displayed by the display device changes from a first virtual image to a second virtual image, the first virtual image being generated by transmitting first imaging light of the display device, and the second virtual image being generated by transmitting second imaging light of the display device.

15. The method of claim 14, wherein, The user touches the real image by a finger.

16. The method according to claim 14 or 15, characterized in that, When the user touches the real image, the real image changes at least one of: a brightness of the real image changes from a first brightness to a second brightness, the second brightness having a brightness value greater than a brightness value of the first brightness; a frame size of the real image changes from a first frame size to a second frame size, the second frame size having an area greater than an area of the first frame size; the real image displays a bright border.

17. A cockpit system characterized by, A seat for a user to sit on, comprising the display device of any one of claims 1 to 13.

18. A vehicle, characterized by A cockpit system comprising the display device of any one of claims 1 to 13 or the seat of claim 17.

19. The vehicle of claim 18, wherein, The display device is arranged at least one of a headrest of a seat of the vehicle, a backrest of a seat of the vehicle, and a dashboard of the vehicle.

Images