Virtual image display device, method, storage medium, program product, and vehicle

Abstract

Embodiments of the present application provide a virtual image display device, method, storage medium, program product and vehicle. The device comprises an image source, an optical window and an imaging assembly. The image source is configured to generate a first light beam; the optical window is configured to reflect the first light beam to obtain a second light beam; and the imaging assembly is configured to process the second light beam to obtain a virtual image. The optical window comprises a touch module configured to receive a touch instruction from a user to interact with content corresponding to the virtual image. In this way, the accuracy and intuitiveness of the interaction can be improved by selecting an interactive object in the virtual image based on hovering touch, and then selecting the interactive object by tapping and entering an option corresponding to the interactive object.

Description

Virtual image display device, method, storage medium, program product and vehicle TECHNICAL FIELD

[0001] The present application relates to the technical field of interaction technology and intelligent vehicle technology, and in particular to a virtual image display device, method, storage medium, program product and vehicle. BACKGROUND

[0002] Virtual image display technology displays the picture to be displayed in the form of a virtual image. Since virtual image display technology is implemented through projection, it is difficult to obtain an interaction interface corresponding to the virtual image display picture, and therefore it is difficult to achieve interaction with the user.

[0003] The current commonly used interaction mode is achieved through eye tracking. However, eye tracking has relatively high requirements for computing power, image sensors and use environment, and is restricted by image accuracy, resulting in low accuracy of eye tracking and poor interaction effect.

[0004] SUMMARY

[0005] The present application discloses a virtual image display device, method, storage medium, program product and vehicle, which can improve the accuracy and intuitiveness of virtual image interaction.

[0006] In a first aspect, an embodiment of the present application provides a virtual image display device. The device includes an image source, an optical window and an imaging assembly. The image source is configured to generate a first light beam. The optical window is configured to reflect the first light beam and obtain a second light beam. The imaging assembly is configured to process the second light beam and obtain a virtual image. The optical window includes a touch control module configured to receive a touch instruction from a user to interact with content corresponding to the virtual image.

[0007] The virtual image display device provided by the embodiment of the present application can know the position in the virtual image that the user intends to point to based on the change of the capacitance value or the change of the light blocking amount of the touch control module. In this way, the touch control module can receive a touch instruction from the user to interact with the content corresponding to the virtual image, and the accuracy and intuitiveness of virtual image interaction can be improved.

[0008] In a possible implementation, the ratio between the area size of the touch control module covering the optical window and the area size of the optical window is less than or equal to a first threshold. In this way, the user can perform touch control in the area of the optical window covered by the touch control module.

[0009] In a possible implementation, the touch control module is located in the lower half of the optical window. In this way, the user can conveniently perform interaction operations.

[0010] In another possible implementation, a ratio between an area size of the touch module and an area size of the optical window is greater than a fifth threshold.

[0011] In a possible implementation, the touch instruction includes a first touch instruction, which is used to represent that the first operation of the user is less than a second threshold and greater than a third threshold from the optical window. In response to the first touch instruction, the image source displays a first identifier, which is used to indicate a first function in the virtual image.

[0012] In this example, the user performs a hovering touch. Based on the first operation of the user, the user enters a stage of virtual image interaction. The virtual image display device displays the first identifier in the virtual image, so that the user knows the position in the virtual image, or facilitates the user to know which interaction he / she is about to perform.

[0013] In this example, the first operation of the user is relatively close to the optical window, but does not touch the optical window. That is, the user performs a hovering touch. The user enters a stage of virtual image interaction.

[0014] The first function, for example, indicates one or more applications. For another example, the first function indicates one or more operations.

[0015] In a possible embodiment, the first identifier indicates one or more applications in the virtual image. Based on the first identifier displayed by the virtual image display device, the user knows the position in the virtual image.

[0016] In another possible embodiment, the first identifier indicates one or more operations in the virtual image. The operation may, for example, include at least one of the following: up, down, left and right movement operation, confirmation operation, return operation, back to main desktop operation, task management operation, volume adjustment operation, brightness adjustment operation, power on / off operation, etc. Based on the first identifier displayed by the virtual image display device, the user knows which interaction he / she is about to perform.

[0017] In a possible embodiment, the position at which the first identifier is displayed is associated with the relative position between the first operation of the user and the touch module.

[0018] In a possible embodiment, the first identifier includes at least one of the following: preset icon, preset text, preset brightness.

[0019] In a possible embodiment, the preset icon includes at least one of the following: up, down, left and right movement icon, confirmation icon, return icon, back to main desktop icon, task management icon, volume adjustment icon, brightness adjustment icon, power on / off icon.

[0020] In a possible embodiment, the touch instruction further includes a second touch instruction, the second touch instruction being used to indicate that the distance between the second operation of the user and the optical window is less than a fourth threshold. In response to the second touch instruction, the image source displays a first interface, the first interface including information corresponding to the first function.

[0021] In this way, the user can select the intended interaction object and enter the option corresponding to the interaction object, and the accuracy and intuitiveness of the interaction can be improved.

[0022] The first interface includes information of the first function, which can be understood as that the content displayed by the first interface corresponds to the first function. That is, when the distance between the second operation of the user and the optical window is less than the fourth threshold, it indicates that the interaction object selected by the user is the first function. This stage indicates that the user has selected the interaction object and entered the option (or content) corresponding to the interaction object. For example, the content displayed by the first interface is the content corresponding to an APP. For another example, the first interface includes content after a certain operation is performed on the virtual image. For an example, the operation is volume adjustment, and the first interface is the virtual image after the volume is adjusted. For another example, the operation is to return to the main desktop, and the first interface is the main desktop.

[0023] Optionally, before the first interface is displayed, the virtual image display device displays an elliptical cursor of a second color at a position corresponding to the first function. The elliptical cursor of the second color indicates that the interaction object selected by the user is the first function.

[0024] In a possible embodiment, the touch module includes at least one of the following: a self-induction type capacitive touch module, a mutual-induction type touch module, an infrared touch module.

[0025] In a possible embodiment, the touch module is integrally formed with the optical window.

[0026] In another possible embodiment, the touch module is fixedly connected with the optical window.

[0027] In a possible implementation, the material of the touch module includes at least one of the following: a semiconductor, a metal, and a polymer.

[0028] In a possible implementation, the virtual image display device further includes an anti-glare layer, the anti-glare layer being used to eliminate stray light.

[0029] In a possible implementation, the anti-glare layer is integrally formed with the touch module.

[0030] In a possible implementation, the anti-glare layer includes at least one of the following: a polarizing device, an anti-reflection film layer, and a matte layer.

[0031] In a possible implementation, the image source includes at least one of the following: a liquid crystal display (LCD), an organic light-emitting diode (OLED), a quantum dot light emitting diode (QLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a liquid crystal on silicon (LCOS), and a digital light procession (DLP).

[0032] In a possible implementation, the imaging assembly includes at least one of the following: a curved mirror, a lens or a lens group, a superlens, a waveguide device, and a diffractive optical device.

[0033] In a second aspect, an embodiment of the present application provides a virtual image display method, including: in response to a first control instruction of a user, displaying a first identifier, the first identifier being used to indicate a first function in the virtual image. The first control instruction is used to indicate that a distance between the first operation of the user and the touch module is less than a second threshold value and greater than a third threshold value.

[0034] In response to the first control instruction of the user, the virtual image display device displays the first identifier, which is used to indicate the first function in the virtual image, so that the user knows the position in the virtual image or knows which interaction the user is about to perform, and the accuracy and intuitiveness of the interaction can be improved.

[0035] In a possible embodiment, in response to a second control instruction of the user, a first interface is displayed, the first interface including information of the first function, and the second control instruction is used to indicate that a distance between the second operation of the user and the touch module is less than a fourth threshold value.

[0036] In this example, the user can select the intended interaction object in the virtual image based on the floating touch, and then select the interaction object by clicking, and enter the option corresponding to the interaction object, so that the accuracy and intuitiveness of the interaction can be improved.

[0037] In a possible embodiment, the first identifier includes at least one of the following: a preset icon, a preset word, and a preset brightness.

[0038] In a possible embodiment, the first identifier is located in a lower half of the virtual image.

[0039] In a possible embodiment, the preset icon comprises at least one of the following: up, down, left, right, back, return to the main desktop, task management, volume adjustment, brightness adjustment, and power on / off.

[0040] In a third aspect, an embodiment of the present application provides a virtual image display device, comprising a processor and a memory; wherein the memory is configured to store program code, and the processor is configured to invoke the program code to execute the method provided in any possible implementation manner of the second aspect.

[0041] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method provided in any possible implementation manner of the second aspect.

[0042] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a computer, causes the computer to execute the method provided in any possible implementation manner of the second aspect.

[0043] In a sixth aspect, an embodiment of the present application provides a vehicle, comprising the virtual image display device provided in any possible implementation manner of the first aspect.

[0044] It can be understood that the method provided in the second aspect, the device provided in the third aspect, the computer readable storage medium provided in the fourth aspect, the computer program product provided in the fifth aspect, or the vehicle provided in the sixth aspect can achieve the beneficial effects of the device provided in the first aspect, which will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0045] The drawings used in the embodiments of the present application are described below.

[0046] FIG. 1 is a schematic diagram of an application scenario of a vehicle according to an embodiment of the present application;

[0047] FIG. 2A is a structural schematic diagram of a virtual image display device;

[0048] FIG. 2B is a schematic diagram of an imaging principle of a virtual image display device;

[0049] FIGS. 3 to 5 are schematic diagrams of virtual image display devices according to embodiments of the present application;

[0050] FIG. 6 is a flowchart of a virtual image display method according to an embodiment of the present application;

[0051] FIG. 7A is a schematic diagram of a first identifier according to an embodiment of the present application;

[0052] FIG. 7B is a schematic diagram of another first identifier according to an embodiment of the present application;

[0053] FIG. 8 is a schematic diagram of a virtual image display according to an embodiment of the present application;

[0054] FIG. 9 is a schematic diagram of a user operation according to an embodiment of the present application;

[0055] FIG. 10 is another schematic diagram of a virtual image display according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] The embodiments of the present application will be described below in conjunction with the accompanying drawings. The terms used in the implementation manner part of the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0057] Please refer to FIG. 1, which is a schematic diagram of an application scenario of a vehicle according to an embodiment of the present application.

[0058] As shown in FIG. 1, the vehicle 1000 in the embodiments of the present application can be a known vehicle such as a car, an airplane, or a ship, and can also be a new vehicle that will appear in the future. The car can be an electric car, a fuel car, or a hybrid car, for example, a pure electric car, a range-extended electric car, a hybrid electric car, a fuel cell car, a new energy car, etc., which is not limited in the present application.

[0059] The vehicle 1000 includes a cabin 200 and a seat installed in the cabin 200. The seat includes a first seat 300 and a second seat 500 for a passenger to sit on. In the embodiments of the present application, the first seat 300 is a front seat arranged in the cabin 200. The second seat 500 is a rear seat arranged behind the first seat 300 for a passenger to sit on. In other embodiments of the present application, the first seat 300 can not be a front seat.

[0060] The first seat 300 includes a seat body 301 and a virtual image display device 10 installed on the seat body 301, which is used to display content to meet the viewing needs of the passenger on the second seat 500. It can be understood that the virtual image display device 10 can also be installed at the co-pilot position (as shown in FIG. 1) of the vehicle 1000, i.e., installed on the instrument panel (IP) of the mobile terminal, which is used to display content to meet the viewing needs of the passenger on the first seat 300. Alternatively, the virtual image display device 10 can also be installed on the roof of the vehicle. The embodiments of the present application do not limit this.

[0061] The virtual image display device is applied to the vehicle as an example for introduction in FIG. 1. The virtual image display device can also be applied to other display scenarios, which is not limited in the present application.

[0062] As shown in FIGS. 2A and 2B, the virtual image display device includes an image source, an optical window, and an imaging assembly. The virtual image display device is characterized by a far distance magnified virtual image display. In the virtual image display device, the image seen by the passenger is a reflected and magnified virtual image of the light emitted by the image source through the optical window and the imaging assembly. The image plane is not on the outer surface of the optical window, but is located at a distance behind the light field screen. Since it is difficult to obtain an interactive interface corresponding to the virtual image display screen, the virtual image display device cannot meet the requirements of the user for the accuracy and intuitiveness of the interaction with the user.

[0063] Therefore, there is an urgent need for a feasible solution to improve the accuracy and intuitiveness of the interaction of the user with the virtual image.

[0064] In view of this, the present application provides a virtual image display device, a method, a storage medium, a program product, and a vehicle to meet the requirements of the user for the accuracy and intuitiveness of the interaction.

[0065] The virtual image display device, the method, the storage medium, the program product, and the vehicle provided by the present application will be described below with reference to the accompanying drawings.

[0066] Referring to FIGS. 3 to 5, the present application provides a schematic diagram of a virtual image display device.

[0067] As shown in FIG. 3, the virtual image display device includes an image source, an optical window, and an imaging assembly. The image source is configured to generate a first light beam. The optical window is configured to reflect the first light beam to obtain a second light beam. The imaging assembly is configured to process the second light beam to obtain a virtual image.

[0068] The optical window includes a touch module (shown in gray in FIG. 3), which is configured to receive a touch instruction from the user to interact with the content corresponding to the virtual image.

[0069] The touch module will be described below.

[0070] In one possible embodiment, the touch module is integrally formed with the optical window. In this example, the product is manufactured by one-time processing, avoiding the detailed processing and splicing of multiple components, thereby greatly saving the production time and cost.

[0071] In one possible embodiment, the ratio between the area size of the touch module covering the optical window and the area size of the optical window is less than or equal to a first threshold value. In an example, the first threshold value can be 50%. In this way, the user can perform touch operation in the area of the optical window covered by the touch module.

[0072] In an example, as shown in FIG. 4, the touch module (shown in gray in FIG. 4) is located in the lower half of the optical window. In this way, the user can conveniently perform the interactive operation.

[0073] For example, the touch module is located at the left half of the optical window. Alternatively, the touch module is located at the right half of the optical window. For example, the touch module is located at the upper half of the optical window. Alternatively, the touch module can also be located at the middle of the optical window, etc. The touch module can also be located at at least two of the upper half, lower half, left half, right half and middle of the optical window, and the present application does not limit this.

[0074] In another possible embodiment, the ratio between the area size of the touch module and the area size of the optical window is greater than a fifth threshold. The fifth threshold can be the same as the first threshold. Alternatively, the fifth threshold is greater than the first threshold, etc. Optionally, the ratio between the area size of the touch module and the area size of the optical window is greater than 50%. In one possible embodiment, the ratio between the area size of the touch module and the area size of the optical window is 100%. In this way, the user can touch the entire area corresponding to the optical window.

[0075] In one possible embodiment, the touch module is a self-induction type capacitive touch module. The self-induction type capacitive touch is a capacitive touch technology that uses a single electrode formed by copper to detect changes in the capacitance of the electrode to ground. When a hand touches, the capacitance of the key to ground increases, thereby realizing key detection. This technology detects changes in the capacitance of the electrode to ground to identify touch events, does not require the use of a stylus, has good accuracy, and is sensitive. For the self-induction type capacitive touch module, when the hand is far away from the self-induction type capacitive touch module, the capacitance to ground is small; when the hand is close to the self-induction type capacitive touch module, the capacitance to ground is large.

[0076] In another possible embodiment, the touch module is a mutual-induction type touch module. Mutual-induction type touch is a touch screen technology that uses electromagnetic induction principles to achieve multi-point touch function. This technology makes horizontal and vertical electrodes on the glass surface, and when a finger touches the screen, it changes the capacitance between the electrodes near the touch point to achieve multi-point touch function. Mutual-induction type touch technology has the characteristics of high precision and high sensitivity. For the mutual-induction type capacitive touch module, when the hand touches, the capacitance of the mutual-induction type capacitive touch module decreases.

[0077] In yet another possible embodiment, the touch module is an infrared touch module. The infrared touch technology is a touch screen technology that uses infrared sensors to detect touches. This technology realizes operations by forming an infrared detection network on the surface of the screen. Any touching object (such as a finger) can change the infrared of the touch point in the infrared detection network, which is then converted into the coordinate position of the touch to realize the response of the operation. The infrared touch technology has high stability and adaptability. For the infrared touch module, the farther the human hand is from the infrared touch module, the smaller the light blocking amount is; the closer the human hand is to the infrared touch module, the larger the light blocking amount is.

[0078] In a possible embodiment, the virtual image display device includes a system on chip (SoC) chip. The touch module is connected to the SoC chip. The SoC chip is connected to the image source. In this way, the SoC chip can control the display of the image source based on the change of the capacitance value corresponding to the touch module.

[0079] In another possible embodiment, the virtual image display device includes an SoC chip and a microcontroller unit (MCU). The touch module is connected to the SoC chip through the MCU. The SoC chip is connected to the image source. In this way, the MCU sends a control signal to the SoC chip based on the change of the capacitance value corresponding to the touch module, to control the display of the image source.

[0080] In a possible embodiment, the material of the touch module includes at least one of the following: indium-tin oxide (ITO), carbon nanotube (CNT), Graphene, silver nanowires (Ag NW), a water-soluble solution of a high-molecular polymer (PEDOT:PSS), and other semiconductors, metals, and polymers.

[0081] The image source, the imaging assembly, and the optical window are introduced as follows.

[0082] In a possible embodiment, the image source includes at least one of the following: a liquid crystal display (LCD), an organic light-emitting diode (OLED), a quantum dot light-emitting diode (QLED), a sub-millimeter light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a liquid crystal on silicon (LCOS), and a digital light processor (DLP).

[0083] In a possible embodiment, the imaging assembly includes at least one of the following: a curved mirror, a lens or a lens group, a superlens, a waveguide device, and a diffractive optical device.

[0084] In a possible embodiment, the material of the substrate of the optical window comprises at least one of the following: an organic material such as polymethyl methacrylate (PMMA), polycarbonate (PC), triallyl cyanurate (TAC), and an inorganic material such as borosilicate crown glass (BK7), silicon dioxide (SiO2), and the like.

[0085] In a possible embodiment, the imaging assembly is attached to the optical window. For example, the imaging assembly is fixedly connected to the optical window by means of glue.

[0086] In a possible embodiment, as shown in FIG. 5, the optical window further comprises a glare-reducing stack (shown in gray in FIG. 5). The glare-reducing stack is configured to eliminate stray light. For example, the glare-reducing stack comprises at least one of a polarizing device, an anti-reflection film layer, and a haze layer. The polarizing device can comprise at least one of a linear polarizer, a quarter-wave plate, and a half-wave plate. The anti-reflection film layer comprises materials with different refractive indexes, for example, at least one of magnesium fluoride, iridium fluoride, silicon oxide, silicon nitride, titanium oxide, niobium oxide, aluminum oxide, tantalum oxide, and hafnium oxide.

[0087] In a possible embodiment, the glare-reducing stack is integrally formed with the optical window. It can be understood that the glare-reducing stack is also integrally formed with the touch module.

[0088] It should be noted that the examples shown in FIGS. 3, 4, and 5 are described by taking the optical window comprising the touch module as an example. In a possible embodiment, the optical window and the touch module can also not be integral. Optionally, the touch module is fixedly connected to the optical window. The fixed connection can be direct connection or indirect connection. For example, the touch module and the optical window are directly connected by means of glue or the like. For another example, the touch module and the optical window are indirectly connected by means of a connecting member or the like. The connecting member can be a screw or a connecting rod, or the like.

[0089] Similarly, the example shown in FIG. 5 is described by taking the optical window comprising the glare-reducing stack as an example. In a possible embodiment, the optical window and the glare-reducing stack can also not be integral. Optionally, the glare-reducing stack is fixedly connected to the optical window. The present solution does not limit this.

[0090] The virtual image display device of the embodiments of the present application is described above, and the virtual image display method of the embodiments of the present application is described in detail below.

[0091] Referring to FIG. 6, a flowchart of a virtual image display method is shown. In combination with the virtual image display device shown in FIGS. 3-5, the method shown in FIG. 6 can include steps 601-603, as follows:

[0092] 601. The virtual image display device displays a virtual image.

[0093] The virtual image display device can be, for example, a light field screen, a head up display (HUD), a projection screen, etc.

[0094] 602. In response to a first control instruction of a user, the virtual image display device displays a first identifier, which indicates a first function in the virtual image.

[0095] The first touch instruction indicates that the first operation of the user is at a distance from the optical window that is less than a second threshold value and greater than a third threshold value. The second threshold value is greater than the third threshold value. For example, the second threshold value can be 50 mm, and the third threshold value can be 10 mm.

[0096] The first operation is performed by the user. For example, the first operation can be the user pointing at the optical window with a finger. Alternatively, the first operation can be the user pointing at the optical window with a capacitive pen. The user can also use other tools, which are not limited by the present application.

[0097] For example, the first operation of the user is relatively close to the optical window, but does not touch the optical window. That is, the user performs a hovering touch. It can be understood that the position in the optical window pointed to by the first operation of the user is provided with a touch module. That is, the user performs the first operation above the area of the optical window provided with the touch module. In this way, based on the change in the capacitance value or the change in the amount of light blocked corresponding to the touch module, the virtual image display device can know that the user intends to point to a position in the virtual image.

[0098] The first identifier is described below.

[0099] The first identifier indicates a first function in the virtual image. In one possible embodiment, the first identifier indicates one or more applications (APPs) in the virtual image. The APPs can be, for example, WeChat, QQ, email, etc. Based on the first identifier displayed by the virtual image display device, the user can know the position in the virtual image that he or she is pointing to. In another possible embodiment, the first identifier indicates one or more operations in the virtual image. The operations can include, for example, at least one of the following: up-down-left-right movement operations, confirmation operations, return operations, return-to-main-desktop operations, task management operations, volume adjustment operations, brightness adjustment operations, power-on / off operations, etc. Based on the first identifier displayed by the virtual image display device, the user can know which interaction he or she is about to perform.

[0100] In a possible embodiment, the position displayed by the first identifier is associated with the relative position between the first operation of the user and the touch module. That is, as the relative position between the first operation of the user and the touch module changes, the position displayed by the first identifier also changes accordingly.

[0101] The association (or correlation) can be a linear correlation or a nonlinear correlation.

[0102] For example, when the first operation of the user corresponds to a position E1 in the touch module, based on the preset correlation (which can also be referred to as a mapping relationship), the position E2 in the virtual image corresponding to the position E1 in the touch module can be obtained, and thus the virtual image display device displays the first identifier at the position E2 in the virtual image. For another example, as the position of the finger of the user moves, when the first operation of the user corresponds to a position E3 in the touch module, the virtual image display device displays the first identifier at a position E4 in the virtual image accordingly.

[0103] In a possible embodiment, the first identifier is a preset icon. In a possible embodiment, the preset icon can be used to represent a position in the virtual image pointed to by the user. For example, the preset icon can be an arrow-shaped cursor, a hand-shaped cursor, a circular cursor, an elliptical cursor, or the like. As shown in FIG. 7A, the preset icon is an elliptical cursor. In another possible embodiment, the preset icon can be used to represent one or more operations. For example, the preset icon includes at least one of the following: an up-down-left-right moving icon, a confirmation icon, a back icon, a back-to-main-desktop icon, a task management icon, a volume adjustment icon, a brightness adjustment icon, a power-on-off icon, and the like. As shown in FIG. 7B, the preset icon includes a left moving icon, a right moving icon, a back-to-main-desktop icon, a volume adjustment icon, and a back icon.

[0104] In another possible embodiment, the first identifier is a preset word. For example, the preset word is “here”, “up-down-left-right moving”, “confirmation”, “back”, “volume up”, “volume down”, or the like.

[0105] In yet another possible embodiment, the first identifier is a preset brightness. For example, the preset brightness is a brightness value not lower than 10 nit.

[0106] In a possible embodiment, the virtual image display device includes a SoC chip. The touch module is connected to the SoC chip. The SoC chip is connected to the image source. In this way, when the user performs the first operation, the capacitance value corresponding to the touch module changes. The SoC chip controls the image source to display the first identifier based on the change in the capacitance value corresponding to the touch module.

[0107] In another possible implementation, the virtual image display device includes an SoC chip and an MCU. The touch module is connected to the SoC chip through the MCU. The SoC chip is connected to the image source. In this way, when the user performs the first operation, the capacitance value corresponding to the touch module changes. The MCU sends a control signal to the SoC chip based on the change of the capacitance value corresponding to the touch module, so that the image source displays the first identifier.

[0108] For example, in response to the first control instruction of the user, the virtual image display device displays an elliptical cursor of a first color. The elliptical cursor of the first color indicates the position pointed to by the user in the virtual image. The first color may be blue or the like.

[0109] For another example, in response to the first control instruction of the user, the virtual image display device displays a left moving icon, a right moving icon, a back to main desktop icon, a volume adjustment icon, and a return icon. In this way, the user can know which interaction he is about to perform.

[0110] For another example, in response to the first control instruction of the user, all regions corresponding to the touch module in the virtual image display device light up. Alternatively, in response to the first control instruction of the user, the region (or position) corresponding to the first operation in the virtual image display device lights up.

[0111] In this example, the user performs a hovering touch. Based on the first operation of the user, the user enters the stage of virtual image interaction. The virtual image display device displays the first identifier in the virtual image, so that the user knows the position he points to in the virtual image, or so that the user knows which interaction he is about to perform.

[0112] 603、In response to the second touch instruction of the user, the virtual image display device displays a first interface including information of a first function.

[0113] The second touch instruction is used to indicate that the distance between the second operation of the user and the optical window is less than a fourth threshold. The fourth threshold may be the same as the third threshold, or the fourth threshold is less than the third threshold, and the present application does not limit this. For example, the fourth threshold is 10 mm.

[0114] The second operation may be the same as the first operation. For example, the second operation is that the user points to the optical window with his finger, or the user points to the optical window with a capacitive pen.

[0115] Alternatively, the second operation is different from the first operation. For example, the second operation is that the user taps the optical window. For example, the user directly contacts the optical window to tap.

[0116] The first interface includes information of the first function. It can be understood that the content displayed by the first interface corresponds to the first function. For example, the content displayed by the first interface is content corresponding to an APP. Specifically, the first interface includes content displayed by opening the APP. For another example, the first interface includes content after performing an operation on the virtual image. For an example, the operation is volume adjustment, and the first interface is the virtual image after the volume is adjusted. For another example, the operation is returning to the home desktop, and the first interface is the home desktop.

[0117] When the distance between the second operation of the user and the optical window is less than the fourth threshold, it indicates that the user selects the first function in the virtual image corresponding to the operation. That is, step 303 indicates that the user has selected the interactive object and entered the option (or content) corresponding to the interactive object.

[0118] Optionally, before displaying the first interface, the virtual image display device displays an elliptical cursor of a second color at a position corresponding to the first function. The elliptical cursor of the second color indicates that the interactive object selected by the user is the first function. The second color may be red, for example. Further, the virtual image display device displays the first interface to indicate entering the content corresponding to the interactive object.

[0119] In response to the first control instruction of the user, the virtual image display device displays a first identifier for indicating a first function in the virtual image, so that the user knows the position in the virtual image pointed by the user or knows which interaction the user is about to perform. In response to the second touch instruction of the user, the virtual image display device displays a first interface including information of the first function. In this way, the user can select the intended interactive object and enter the option corresponding to the interactive object, and the accuracy and intuitiveness of the interaction can be improved.

[0120] The virtual image display method provided by the embodiments of the present application will be described in detail below in combination with Embodiment 1 and Embodiment 3.

[0121] Embodiment 1

[0122] This example takes the ratio between the area size of the touch module covering the optical window in the virtual image display device and the area size of the optical window as an example, and the first identifier is an elliptical cursor.

[0123] As shown in (1) of FIG. 8, the virtual image display device displays a virtual image 1. The content corresponding to the virtual image 1 includes a plurality of APPs, such as an application market, a gallery, settings, a camera, and an email, etc.

[0124] When the distance between the user's finger and the optical window is less than the second threshold value and greater than the third threshold value, as shown in (1) of FIG. 9, the user's finger is at point P1. Accordingly, as shown in (2) of FIG. 8, the virtual image display device displays a blue elliptical cursor (shown in gray in (2) of FIG. 8) in the position corresponding to the camera in the virtual image 1. The blue elliptical cursor indicates the position corresponding to the camera in the virtual image 1 that the user is currently pointing at.

[0125] Optionally, as the user's finger moves, the display position of the above-mentioned blue elliptical cursor also changes. For example, as the user's finger moves, as shown in (2) of FIG. 9, the user's finger is at point P2. Accordingly, as shown in (3) of FIG. 8, the virtual image display device displays a blue elliptical cursor (shown in gray in (3) of FIG. 8) in the position corresponding to the application market in the virtual image 1. The blue elliptical cursor indicates the position corresponding to the application market in the virtual image 1 that the user is currently pointing at.

[0126] When the distance between the user's finger and the optical window is less than the fourth threshold value, as shown in (3) of FIG. 9, the user's finger is at point P3. For example, the user's finger taps point P3 in the optical window. That is, the user selects the application market as the interactive object. Accordingly, the virtual image display device displays a first interface. The first interface includes information corresponding to the above-mentioned application market. For example, as shown in (4) of FIG. 8, the virtual image display device displays interface 100 (or referred to as virtual image 2). The interface 100 includes a plurality of APPs, such as WeChat, QQ, Twitter, and Facebook. Optionally, before displaying the interface 100, the virtual image display device displays a red elliptical cursor (not shown in the figure) in the position corresponding to the application market to indicate that the user selects the application market as the interactive object. Then, the virtual image display device displays the interface 100 to indicate that the interactive object is entered.

[0127] Embodiment 2

[0128] This example takes the ratio between the size of the area covered by the touch module in the virtual image display device and the size of the optical window to be less than 50%, the first identifier to include a left moving icon, a right moving icon, a back to the main desktop icon, a volume adjusting icon, and a back icon as an example for introduction.

[0129] As shown in (1) of FIG. 10, the virtual image display device displays the virtual image 1. The content corresponding to the virtual image 1 includes a plurality of APPs, such as an application market, a gallery, settings, a camera, and an email.

[0130] When the distance between the user's finger and the optical window is less than the second threshold value and greater than the third threshold value, as shown in (1) of FIG. 9, the user's finger is at point P1. Accordingly, as shown in (2) of FIG. 10, the virtual image display device displays a leftward moving icon, a rightward moving icon, a return to main desktop icon, a volume adjustment icon, and a back icon in a region in the lower middle part of the virtual image 1. Optionally, the brightness of the region in which the icons are displayed is higher than that of other regions in the virtual image 1 (not shown in the figure). For example, the virtual image display device displays a blue elliptical cursor (represented by a gray-filled shape in (2) of FIG. 10) in a position in the virtual image 1 corresponding to the gallery. Of course, the elliptical cursor can also be replaced by other cursors or the like.

[0131] As the user's finger moves, the display position of the blue elliptical cursor changes. For example, as the user's finger moves continuously, as shown in (2) of FIG. 9, the user's finger is at point P2. For example, the point P2 corresponds to the leftward moving icon in the virtual image 1. Accordingly, as shown in (3) of FIG. 10, the virtual image display device highlights the leftward moving icon in the virtual image 1 (shown in (3) of FIG. 10 by the leftward moving icon being filled with gray).

[0132] When the distance between the user's finger and the optical window is less than the fourth threshold value, as shown in (3) of FIG. 9, the user's finger is at point P3 (i.e., a position corresponding to the leftward moving icon) in the optical window. That is, the user-selected interactive object is the application market corresponding to the leftward moving icon. Accordingly, as shown in (4) of FIG. 10, the virtual image display device displays an interface 100. The interface 100 includes a plurality of APPs, such as WeChat, QQ, Twitter, and Facebook. Optionally, before the interface 100 is displayed, the virtual image display device displays a red elliptical cursor (not shown in the figure) in a position corresponding to the application market, to indicate that the user-selected interactive object is the application market.

[0133] Embodiment 3

[0134] The difference between this embodiment and Embodiment 2 is that the leftward moving icon, the rightward moving icon, the return to main desktop icon, the volume adjustment icon, and the back icon are provided on the screen of the virtual image display device, for example, directly on the lower edge of the light field screen.

[0135] For example, when the distance between the user's finger and the optical window is less than the second threshold value and greater than the third threshold value, as shown in (1) of FIG. 9, the user's finger is at point P1. Accordingly, the leftward moving icon, the rightward moving icon, the return to main desktop icon, the volume adjustment icon, and the back icon are all lit up.

[0136] When the distance between the user's finger and the optical window is less than the fourth threshold value, as shown in (3) of FIG. 9, the user's finger points at P3 in the optical window. That is, the user can directly select the icon on which the user wants to perform the point operation.

[0137] Alternatively, when the distance between the user's finger and the optical window is less than the second threshold value and greater than the third threshold value, some of the icons, such as the left moving icon, the right moving icon, the back to the main desktop icon, the volume adjusting icon, and the return icon, are lit. For example, the icon corresponding to the position of the user's finger is lit.

[0138] Different icons are lit as the user's finger moves.

[0139] When the distance between the user's finger and the optical window is less than the fourth threshold value, the user's finger points at P3 in the optical window. That is, the user directly performs the point operation on the icon.

[0140] The application also provides a vehicle, which comprises the virtual image display device shown in any one of FIGS. 3-5.

[0141] It should be understood that the vehicle involved in the application can include a vehicle, a robot, a drone, a ship, a ship, or an intelligent terminal or vehicle. Among them, the vehicle is a vehicle in a broad sense, which can be a vehicle (such as a commercial vehicle, a passenger vehicle, a motorcycle, a flying vehicle, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), an agricultural device (such as a mower, a harvester, etc.), etc. For example, the robot can be an automated guided vehicle (AGV), a walkable conversational robot, a service robot, etc.

[0142] It should be understood that the embodiments shown in FIGS. 3-5 are only exemplary descriptions of several possible virtual image display devices provided by the application, and should not be construed as limiting the application. New embodiments obtained by reasonable deformation, supplementation or combination based on the embodiments shown in FIGS. 3-5 are within the protection scope of the application.

[0143] The embodiments of the application also provide a computer readable storage medium, which stores instructions. When the instructions are run on a computer or a processor, the computer or the processor performs one or more steps in any one of the above methods.

[0144] The embodiments of the application also provide a computer program product containing instructions. When the computer program product is run on a computer or a processor, the computer or the processor performs one or more steps in any one of the above methods.

[0145] It should be understood that, in the description of the present application, unless otherwise specified, " / " represents that the objects before and after the " / " are in an "or" relationship, for example, A / B can represent A or B; wherein A, B can be singular or plural. And, in the description of the present application, unless otherwise specified, "multiple" means two or more than two. "At least one of the following" or the like means any combination of the items, including a single item or a combination of multiple items. For example, at least one of a, b, or c can represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c can be singular or plural. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the same items or similar items with basically the same function and role are distinguished by using "first", "second", etc. The skilled in the art can understand that "first", "second", etc. do not limit the quantity and execution order, and "first", "second", etc. also do not limit the difference. At the same time, in the embodiments of the present application, "exemplary" or "for example" means to serve as an example, illustration or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the use of "exemplary" or "for example" is intended to present the relevant concept in a specific manner, for understanding.

[0146] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the division of the unit is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0147] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0148] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in or transmitted by a computer readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media sets. The available medium can be a read-only memory (ROM), or a random access memory (RAM), or a magnetic medium, such as a floppy disk, a hard disk, a magnetic tape, a magnetic disk, or an optical medium, such as a digital versatile disc (DVD), or a semiconductor medium, such as a solid state disk (SSD), etc.

[0149] The above is only a specific implementation of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any change or replacement within the technical scope disclosed by the embodiments of the present application should be covered within the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be subject to the protection scope of the claims.

Claims

1. A virtual image display device, characterized by comprising: The image source, the optical window and the imaging component are included, The image source is configured to generate a first light beam; The optical window is configured to reflect the first light beam to generate a second light beam; The imaging component is configured to process the second light beam to generate a virtual image; The optical window comprises a touch module configured to receive a touch instruction from a user to interact with content corresponding to the virtual image.

2. The virtual image display device of claim 1, wherein A ratio between an area size of the touch module and an area size of the optical window is less than or equal to a first threshold.

3. The virtual image display device according to claim 1 or 2, wherein The touch module is located in a lower half of the optical window.

4. The virtual image display device of any one of claims 1 to 3, wherein The touch instruction comprises a first touch instruction configured to indicate that a distance between a first operation of the user and the optical window is less than a second threshold and greater than a third threshold. In response to the first touch instruction, the image source displays a first identifier configured to indicate a first function in the virtual image.

5. The virtual image display device of any one of claims 1 to 4, wherein The touch instruction further comprises a second touch instruction configured to indicate that a distance between a second operation of the user and the optical window is less than a fourth threshold. In response to the second touch instruction, the image source displays a first interface comprising information corresponding to the first function.

6. The virtual image display device of claim 4 or 5, wherein A displayed position of the first identifier is associated with a relative position between the first operation of the user and the touch module.

7. The virtual image display device of claim 6, wherein The first identifier comprises at least one of a preset icon, a preset text and a preset brightness.

8. The virtual image display device of claim 7, wherein The preset icon comprises at least one of an up-down-left-right moving icon, a confirmation icon, a return icon, a back-to-main desktop icon, a task management icon, a volume adjustment icon, a brightness adjustment icon and a power-on / off icon.

9. The virtual image display device of any one of claims 1, 4-7, wherein, A ratio between an area size of the touch module and an area size of the optical window is greater than a fifth threshold.

10. The virtual image display device of any one of claims 1 to 9, wherein, The touch module comprises at least one of a self-induction capacitive touch module, a mutual-induction capacitive touch module and an infrared touch module.

11. The virtual image display device of any one of claims 1 to 10, wherein, The touch module is integrally formed with the optical window.

12. A virtual image display method characterized by comprising: The method is applied to the virtual image display device of any one of claims 1 to 11, and the method comprises: In response to a first touch instruction of a user, a first identifier is displayed, the first identifier being configured to indicate a first function in the virtual image, the first touch instruction being configured to indicate that a distance between a first operation of the user and the touch module is less than a second threshold and greater than a third threshold.

13. The method of claim 12, wherein, The method further comprises: In response to a second touch instruction of the user, a first interface is displayed, the first interface comprising information of the first function, the second touch instruction being configured to indicate that a distance between a second operation of the user and the touch module is less than a fourth threshold.

14. The method of claim 13, wherein, The first identifier comprises at least one of a preset icon, a preset text and a preset brightness.

15. The method according to any one of claims 12 to 14, characterized in that, The first identifier is located in a lower half of the virtual image.

16. The method according to claim 14 or 15, characterized in that The preset icon comprises at least one of an up-down-left-right moving icon, a confirmation icon, a return icon, a back-to-main desktop icon, a task management icon, a volume adjustment icon, a brightness adjustment icon and a power-on / off icon.

17. A virtual image display device, comprising: The apparatus comprises a processor for causing the apparatus to perform the method of any of claims 12-16 by executing computer programs or computer executable instructions stored in a memory and / or by logic circuitry.

18. A computer-readable storage medium, characterized in that, A computer program is stored, which, when executed by a processor, causes the method of any of claims 12-16 to be implemented.

19. A computer program product comprising instructions which, when executed on a processor, cause the method of any of claims 12-16 to be implemented.

20. A vehicle, characterized by A virtual image display device according to any of claims 1 to 11.

Images