Control method of display module and near-to-eye display device

Abstract

The embodiment of the invention provides a control method of a display module and a near-to-eye display device. The display module is configured to be placed on an optical lens, the display module is configured to output image content, and the method comprises the following steps: under the condition that a main interface is imaged in a preset imaging area on the lens by the display module, detecting the relative position between the display module and each thumbnail on the main interface; when it is detected that the display module is arranged on a thumbnail, the thumbnail is amplified and displayed on the display module.

Description

Technical Field

[0001] This application relates to the field of near-eye displays, and particularly to a control method for a display module and a near-eye display device. Background Art

[0002] Near-eye display refers to imaging at a relatively short imaging distance from the eyes for sensory perception. For example, virtual reality (VR) glasses and augmented reality (AR) glasses both image at a relatively short distance from the eyes. That is to say, near-eye display is a relative concept compared to traditional televisions, monitors, etc. Obviously, the imaging distance from the display panel of a traditional television, monitor, etc. to the eyes is much greater than that of near-eye display devices such as VR glasses or AR glasses.

[0003] Currently, near-eye displays can generally be classified into VR, AR, mixed reality (MR), and extended reality (XR).

[0004] VR, also known as computer-simulated reality. It is an experience created through human-computer interaction and computer-generated three-dimensional simulations. We can interact in the environment by using virtual reality devices, such as headsets, controllers. That is to say, VR is a computer simulation system that can create and experience virtual worlds. It uses a computer to generate a simulated environment and immerses us in this environment.

[0005] AR is a real-time, direct or indirect observation of the physical environment of the real world. It combines what we see in the real environment with digital content generated by computer software to enhance the real environment we are in in some way. The AR system transmits virtual information to headsets or smart glasses in real time through a camera, or through a mobile device, allowing us to clearly view 3D images.

[0006] MR is the integration of the real world and the virtual world to create a new environment and visualization. Among them, physical objects and digital objects coexist and interact in real time. This means that if a new image is placed in the real space, to a certain extent, this new image will interact with real objects in our real environment.

[0007] XR refers to the combination of all real and virtual environments generated by computer technology and wearable devices, as well as human-computer interaction. XR enhances our senses through digitization to fuse the world. In addition, it provides a large number of virtual sensor input levels at different levels for immersive virtual experiences. XR includes the above three emerging technologies, namely VR, AR, and MR. Summary of the Invention

[0008] In the related art, on a near-eye display device, multiple application or file (icon) thumbnails may be simultaneously imaged in a certain area of a display module. How to enable a user to accurately interact with these thumbnails without confusion or errors is a technical problem that urgently needs to be solved. The objective of the embodiments of this application is to provide a control method, device, electronic device, near-eye display device, medium, chip, and computer program product for the display module to independently or semi-independently, to a certain extent, solve the technical problem that if multiple application or file (icon) thumbnails are simultaneously imaged in a certain area of the display module connected to the near-eye display device, it is easy for the user to have confusion or errors when interacting with these thumbnails. The so-called independent solution means providing a hardware configuration solution that can work independently without the cooperation of software, and can achieve the effect that on a near-eye display device, multiple application or file (icon) thumbnails are simultaneously imaged in a certain area of the display module connected to the near-eye display device, and the user can accurately interact with these thumbnails without being prone to confusion or errors. The so-called semi-independent solution means providing a hardware configuration solution that can achieve the effect that on a near-eye display device, multiple application or file (icon) thumbnails are simultaneously imaged in a certain area of the display module connected to the near-eye display device, and the user can accurately interact with these thumbnails without being prone to confusion or errors when working in cooperation with near-eye display software, such as the software for the user to interact with virtual information in AR technology.

[0009] The first aspect of the embodiments of this application provides a control method for a display module. The display module is configured to be placed on an optical lens and output image content. The method includes:

[0010] When the main interface is imaged in the preset imaging area of the display module on the lens, detecting the relative positions between the display module and the respective thumbnails on the main interface;

[0011] When it is detected that the display module is placed above a thumbnail, magnifying and displaying the thumbnail on the display module.

[0012] The second aspect of the embodiments of this application provides a control device for a display module. The display module is configured to be placed on an optical lens and output image content. The device includes:

[0013] A first detection module, configured to detect the relative positions between the display module and each thumbnail on the main interface when the main interface is imaged within a preset imaging area of the display module on the lens.

[0014] A first magnification module, configured to magnify and display the thumbnail on the display module when it is detected that the display module is placed above a thumbnail.

[0015] In a third aspect of the embodiments of the present application, an electronic device is provided, which includes a processor and a memory. The memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the control method of the display module as described in the first aspect are implemented.

[0016] In a fourth aspect of the embodiments of the present application, a near-eye display device is provided, which includes a processor and a memory. The memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the control method of the display module as described in the first aspect are implemented.

[0017] In a fifth aspect of the embodiments of the present application, a readable storage medium is provided. A program or instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the control method of the display module as described in the first aspect are implemented.

[0018] In a sixth aspect of the embodiments of the present application, a chip is provided. The chip includes a processor and a communication interface, and the communication interface is coupled to the processor. The processor is configured to run a program or instruction to implement the steps of the control method of the display module as described in the first aspect.

[0019] In a seventh aspect of the embodiments of the present application, a computer program product is provided. The program product is stored in a storage medium, and the program product is executed by at least one processor to implement the steps of the control method of the display module as described in the first aspect.

[0020] In the embodiments of the present application, when the main interface is imaged by the display module on the connected lens, the relative positions between the display module and each thumbnail on the main interface are detected. When it is detected that the display module is placed above a thumbnail, the thumbnail is magnified. So that the user can confirm whether to select the thumbnail. Thus, to a certain extent, the user can accurately interact with these thumbnails without confusion or error.

[0021] Other technical effects of the present application are described in detail in the following embodiments. Description of the Drawings

[0022] Figure 1It is a schematic connection diagram between a display module and a lens provided by an embodiment of the present application;

[0023] Figure 2 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0024] Figure 3 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0025] Figure 4 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0026] Figure 5 It is a schematic connection diagram of the connection relationship between a display module and an intelligent terminal provided by an embodiment of the present application;

[0027] Figure 6 It is a schematic diagram of the main imaging interface of a display module provided by an embodiment of the present application;

[0028] Figure 7 It is a schematic diagram of the step flow of the control method of the display module provided by an embodiment of the present application;

[0029] Figure 8 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0030] Figure 9 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0031] Figure 10 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0032] Figure 11 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0033] Figure 12 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0034] Figure 13 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0035] Figure 14 Schematic structural diagram of the control device of the display module provided by an embodiment of the present application;

[0036] Figure 15 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0037] Figure 16It is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0038] Figure 17 It is a schematic structural diagram of a near-eye display device provided by an embodiment of the present application. Detailed implementation manners

[0039] Next, the technical solutions in the embodiments of the present application will be clearly described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0040] The terms "first", "second", etc. in the description and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. generally belong to the same category, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and / or" in the description and claims means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0041] The embodiments of the present application provide a control method, device, electronic device, near-eye display device, storage medium, chip, and computer program product for a display module, which can effectively solve the technical problem that if the display module connected to the lens of the glasses forms an image in the blind area of the lens, it will cause the user wearing the glasses to be unable to see or clearly see the virtual information in the blind area. The following will be described with reference to the accompanying drawings. To facilitate the (correct) understanding of the embodiments of the present application, the technical terms that may be involved in the embodiments of the present application will be defined and explained first. Without special explanation in the following text, the semantics of the corresponding technical terms follow the definitions here:

[0042] Computer or computing device: It refers to all electronic devices based on the Turing computability theory, von Neumann architecture, or Harvard architecture. For example, mobile phones, smart watches, single-chip microcomputer systems, etc. are all computers, not limited to the narrow sense of computers such as personal computers (PCs, Personal Computers) mentioned in daily life.

[0043] Application: That is, a computer application program (Computer Program or Application Software), which is written in certain programming languages and runs on a certain target architecture computer system. A computer application program refers to a coded instruction sequence that can be executed by a device with information processing capabilities such as a computer in order to obtain a certain result, or a symbolic instruction sequence or symbolic statement sequence that can be automatically converted into a coded instruction sequence.

[0044] Logical functions: Refers to various functions provided by various electronic devices such as near-eye display devices, computers, or computing devices (with independent computing capabilities). These functions can be user-oriented functions or system (or device)-oriented functions. For example, user-oriented logical functions can be: graphical user interface (Graphical User Interface, GUI) output function, audio output function, main interface or desktop, programs / applications, sub-functions in programs / applications, etc. Among them, the main interface or desktop, applications, etc. can all provide a GUI to the user for the user to interact with. System-oriented logical functions can be: fragmented file sorting, system error diagnosis, etc.

[0045] Foreground running and background running: Foreground running and background running are two relative concepts. Both are used to describe the running modes of applications / programs or processes in an operating system (generally speaking, the operating system refers to the software in a computer that enables applications to communicate with the underlying hardware of the computer). Taking a traditional computer as an example, the foreground running and background running in AR technology are similar to those in a traditional computer. In a traditional computer, when an application is running in the foreground, the GUI of the application usually appears on the display screen, and the user can see the GUI or window of the application, and the user can directly interact or communicate with the application based on the GUI. At this time, the application usually occupies input devices (such as a mouse and a keyboard); the application running in the foreground usually blocks the user interface, and the user needs to wait for the application running in the foreground to complete or interact according to the requirements of the application running in the foreground; it is suitable for tasks that require real-time feedback from the user or user input. In a traditional computer, when an application is running in the background, it usually does not output a GUI on the display screen, that is, it is invisible to the user, and the user usually cannot directly interact or communicate with the application; when the application is running in the background, the user can continue to use the computer to perform other tasks, and the application runs silently in a state invisible to the user, such as processing files, executing system services, etc.; the application running in the background usually does not occupy input devices and does not require the user to directly participate in the execution of the task; it allows the user to continue to perform other tasks while the application is executing, improving the multitasking processing ability of the system.

[0046] Display module: It refers to a display component, display device or display equipment that forms an image based on optical technology. Therefore, the display module is a physical hardware module. In some alternative embodiments, the display module can be connected to the lens and powered on to form an image through the display module. The lens can be, for example, sunglasses commonly seen in daily life, glasses for vision correction, goggles or a mask with a lens, etc., or the lens of a smart glasses with audio, sensing, etc. The display module can be connected to the lens by means of magnetic force, clips or adsorption, so that the display module can move freely on the lens to which it is connected or move within a specified setting area. For example, the display module can move linearly or curvilinearly, rotate axially, etc. on the placed lens.

[0047] Exemplarily, as Figure 1 shown, Figure 1 is a schematic diagram of the connection between a display module and a lens provided by an embodiment of the present application. Please refer to Figure 1 , taking the lens of glasses as an example for illustration, but the present application is not limited to glasses, and it can also be the lens related to head-mounted devices such as helmets, goggles, vision correction, and sports. Among them, glasses 1 can be ordinary glasses, AR glasses, smart glasses, etc. Glasses 1 include a frame 11, a lens 12 embedded in the frame 11, and temple arms 13 movably connected to the frame 11. The lens 12 includes an environment side 24 and an eye side 22. The user's eyes can see the physical environment on the environment side 24 through the lens 12 from the eye side 22. Optionally, the display module 2 is provided with a first magnetic attraction member at one end of the lens 12 facing the environment side 24, and the display module 2 is provided with a second magnetic attraction member on the lens 12 facing the eye side 22. It can be understood that two mutually attracting magnetic attraction members are arranged on both sides of the lens 12 to realize pre-fixing of the display module on the lens. The first magnetic attraction member and the second magnetic attraction member can be mutually attracting magnets. Of course, one of them can be a magnet and the other can be a metal attracted by the magnet, etc., so that the entire display module 2 is adsorbed on the lens 12. At this time, the display module 2 can still slide freely or rotate axially on the lens 12 without detaching from the lens 12. Of course, other adsorption and movement solutions are also feasible here.

[0048] Exemplarily, as Figure 2 and Figure 3 shown, Figure 2 and Figure 3 are both schematic diagrams of an application scenario of a display module provided by an embodiment of the present application. Figure 3 It can be the front view obtained by observing the glasses 1 from the direction where the temple arms 13 of the glasses 1 are located (i.e., the eye side 22). Please refer to Figure 1 , Figure 2 and Figure 3 , Figure 1The direction indicated by the straight arrow a is the direction in which the user's eye line of sight observes the virtual image formed by the display module 2 on the lens 12. The user can observe the end face of the display module 2 from the direction indicated by the arrow a and see the virtual image or graphical interface formed on the lens 12 based on AR technology. That is, from Figure 1 the end face of the display module 2 seen when observing the display module 2 from the direction of the arrow a shown in Figure 1 is the display area 21 of the display module 2. When the display module 2 is in operation, the display area 21 can be in a "lit" state, that is, it can form image content, such as text, interactive interfaces, videos, images, and other related display contents. The area on the lens 12 that is not occupied by the display module 2 still retains the properties of the lens 12 itself. For example, it is the area where the user can see the environment side through the lens 12 from the eye side, such as eye protection, vision correction, etc., which exists as the properties of the lens itself.

[0049] In some alternative embodiments, the central position of the lens 12 is the calibration position, which is the reference position of the display module 2. When the display module 2 is stationary at this calibration position, it forms a main interface or an interface with preset logical functions on the lens 12. The reason for determining the central position of the lens 12 as the calibration position is that the display module 2 can move in more directions and distances in the circumferential direction at the central position, which is convenient for the user to operate the display module 2 and customize the preset imaging area.

[0050] After the user wears the glasses 1, the user can hold (or pinch) the touch part of the display module 2 to drag the display module 2 to move on the lens 12 or rotate the display module 2 to perform an axial rotation. In some embodiments, the display module 2 can be located on the environment side 24, and the light of the display area 21 can pass through the lens and reach the user's eyes on the eye side 22 for imaging. At this time, the touch part can be located on the environment side 24, so the user can perform better interaction. In other embodiments, the display module 2 can be located on the eye side 22. At this time, the light of the display area 21 can directly enter the user's eyes for imaging, and the touch part can be located on the eye side 22, so the user can perform better and more concealed interaction. Exemplarily, as Figure 4 shown, Figure 4 is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application. Please refer to Figure 4 , the touch part of the display module 2 is located on the environment side 24, and the user can hold (or pinch) the touch part of the display module 2 to drag the display module 2 to move on the lens 12 or rotate the display module 2 to perform an axial rotation.

[0051] In some alternative embodiments, the virtual images or graphical interfaces formed by the display area 21 of the display module 2 on the lens 12 may come from the display module 2 itself, or from the glasses 1 communicatively connected to the display module 2. Of course, in other embodiments, they may also come from other electronic devices that are relatively independent of both the glasses 1 and the display module 2, such as smart terminals or computers like mobile phones, tablets, etc. If the glasses 1 are ordinary glasses, the virtual images or graphical interfaces formed by the display module 2 on the lens 12 may come from other electronic devices that are relatively independent of both the glasses 1 and the display module 2, such as smart terminals or computers like mobile phones, tablets, etc.

[0052] Exemplarily, as Figure 5 shown, Figure 5 FIG. is a schematic diagram of the connection relationship between a display module and a smart terminal provided by an embodiment of the present application. Please refer to Figure 5 , the display module 2 is physically connected to the near-eye display device 10, which may be the aforementioned glasses 1. The smart terminal 4 may be a PC, laptop, smartphone, tablet, smart speaker, server, etc. A wired or wireless communication connection is established between the display module 2 and the smart terminal 4. For example, the display module 2 and the smart terminal 4 communicate based on the Wifi protocol or the Bluetooth protocol; or, the display module 2 and the smart terminal 4 communicate via a USB cable based on the USB transmission protocol, and the display module 2 also obtains electrical energy from the smart terminal 4 via the USB cable. Of course, it is not limited to the aforementioned examples. The display module 2 and the smart terminal 4 may also establish a wired or wireless communication connection based on other communication protocols, such as the NFC communication protocol, P2P (peer-to-peer) network, etc. The display module 2 and the near-eye display device 10 together form the "display device" of the smart terminal 4, and the GUI output by the smart terminal 4 is imaged on the near-eye display device 10 through the display module 2.

[0053] In some alternative embodiments, a touch sensor, or a pressure or capacitance sensor, etc. may be provided on the touch portion of the display module 2 to detect whether the touch portion of the display module 2 is touched. In some alternative embodiments, a micro camera is also provided on the touch portion of the display module 2 to collect image / video data from the environment. In some alternative embodiments, a micro microphone is also provided on the touch portion of the display module 2 to collect ambient sound from the environment. In some alternative embodiments, a micro camera is also provided on one side of the display area 21 of the display module 2 to collect the eye image of the user wearing the glasses 1 for tracking the user's eyeballs.

[0054] Trigger condition: It refers to the condition for pre - setting the operation or execution of the trigger logic function. In some optional implementation manners of the embodiments of the present application, certain motion states of the display module are pre - set as the trigger conditions for certain logic functions. For example, when the display module is stationary at a certain set position on the lens to which it is connected, it is pre - set to trigger the output of the main interface function. Then, when it is subsequently monitored that the display module is stationary at this set position, the display module will display the main interface on the lens to which it is connected, and the user can observe the main interface imaged on the lens by observing the display area of the display module. Optionally, there can be at least one application (or file) icon (thumbnail) in the main interface. A certain set position can be the center position of the lens, and of course, it can also be the four corners of the frame or other positions that are easier to remember and identify.

[0055] Pre - set imaging area: It refers to the imaging area pre - set for the display module on the lens to which the display module is connected. In some optional implementation manners, the pre - set imaging area can be a regular area on the lens to which the display module is connected, such as a rectangular area, a circular area, or an elliptical area. In some optional implementation manners, the pre - set imaging area can also be an irregular area on the lens to which the display module is connected. In some optional implementation manners, the pre - set imaging area can also be the entire lens area of the lens to which the display module is connected. In some optional implementation manners, the pre - set imaging area can also be a partial area of the lens to which the display module is connected. The control method of the display module provided by the embodiments of the present application can implement setting the pre - set imaging area on the lens currently connected to the display module. For specific details, please refer to the following description.

[0056] Exemplarily, as Figure 6 shown, Figure 6 is a schematic diagram of the main interface imaged by a display module provided by the embodiments of the present application. Please refer to Figure 6 , in this example, the pre - set imaging area of the display module 2 is the entire lens area of the lens 12. After the user wears the glasses 1, observing from the side of the temple 13 (please refer to Figure 1 and Figure 2 ) towards the lens 12 (i.e., the eye side), the user can observe the main interface imaged by the display module 2 on the lens 12 by observing the display area 21 of the display module 2. That is, in the example of Figure 6 , when the display module 2 is stationary at the illustrated position on the lens 12 (this position is not limited to the position shown in Figure 6 , and the user can set the position where the display module 2 is stationary on the lens 12 at will to trigger the logic function of outputting the main interface), the logic function of outputting the main interface is triggered. In the example of Figure 6 , the pre - set imaging area 121 is the entire lens area of the lens 12, and the pre - set imaging area 121 is divided into multiple sub - areas (for example Figure 6In the six sub-regions shown in [reference], the icons (thumbnails) of the applications or files mapped to each sub-region are imaged. For example, the icon of application F is imaged in sub-region 1211, and the icon of application E is imaged in sub-region 1212. In Figure 6 , a total of six sub-regions are exemplified where the icons of applications A - J are imaged. In some other alternative embodiments, at least one of the icons A - J can be a file icon (thumbnail) such as a video, picture, audio, text, etc. It can be understood that the thumbnail is not visible through the optical lens, that is, the user cannot see the thumbnail on the optical lens after wearing it. The thumbnail can be visible on other terminal devices capable of communicating with the display module, such as a computer, mobile phone, wearable device, etc. (such as Figure 5 the smart terminal 4 shown in [reference]). Therefore, the above-mentioned thumbnail does not interfere with the light transmittance of the optical lens itself, that is, it does not affect the user's normal viewing of the external environment.

[0057] In some alternative embodiments, when the display module 2 is first connected to the lens 12 and the display module 2 is turned on, the display module 2 defaults to imaging the main interface on the lens 12.

[0058] It should be added that, in Figure 6 , for the convenience of illustration, multiple sub-regions are divided in the main interface. In Figure 6 , the grid lines (i.e., the dotted grid on the lens 12) indicating each sub-region are shown. It should not be understood that the grid lines must be present in the main interface imaged by the display module 2 provided in the embodiments of the present application. Obviously, the grid lines are not necessary, that is, the grid lines may not be present in the imaged main interface. That is, after the display module 2 is placed on the lens 12, the grid lines cannot be seen on the body of the lens 12, and the lens 12 still maintains its own properties. Of course, if the lens itself is a holographic lens such as a waveguide, they can be combined with each other for simple prompts, etc. Of course, setting options can also be provided for the user to choose whether to image the grid lines when imaging the main interface. If the user sets to image the grid lines when imaging the main interface, the display module will image the grid lines while imaging the main interface. If the user sets not to image the grid lines when imaging the main interface, the display module will not image the grid lines when imaging the main interface. Of course, the grid lines can also be the graphical interface corresponding to a smart terminal (such as a mobile phone app), which is convenient for the user to divide or customize the grid lines of the preset imaging area 121 through the graphical interface of the terminal. In addition, the line type of the grid lines is not limited to Figure 6 the dotted line shown in [reference], and can also be a solid line, center line, etc.

[0059] To facilitate an overall understanding of the embodiments provided in the present application, on this basis, the technical defects targeted by the embodiments of the present application are exemplarily described below.

[0060] Please continue to refer to Figure 6 , in the central area of the preset imaging area, two thumbnail images [A] and [B] are simultaneously imaged, and in the area near the edge of the spectacle frame 11, three thumbnail images [D1], [D2], and [D3] are simultaneously imaged. Corresponding interaction means need to be provided so that the user can select the thumbnail image to be selected from these (icon) thumbnail images, thereby realizing the interaction between the user and the display module. At the same time, it is necessary to ensure that the user can quickly and accurately select the thumbnail image to be selected without misselection or confusion.

[0061] The embodiments of the present application provide a control method, device, electronic device, near-eye display device, medium, chip, and computer program product for a display module, which can independently or semi-independently solve to a certain extent the technical problem that if multiple application or file (icon) thumbnail images are simultaneously imaged in a certain area of a near-eye display device by a display module connected to the near-eye display device, it is easy for the user to have confusion or errors when interacting with these thumbnail images.

[0062] On this basis, the embodiments of the present application will be specifically described below.

[0063] The embodiments of the present application provide a control method for a display module. The display module is configured to be placed on an optical lens, and the display module is configured to output image content. Exemplarily, as Figures 1 to 6 shown, the display module can be Figures 1 to 6 the display module 2 shown in, and the display module 2 can output image content on the lens 12. The one that executes the control method of the display module can be Figures 1 to 6 at least one of the glasses 1, display module 2, near-eye display device 10, or smart terminal 4 shown in, which can implement controlling the specific information of the GUI imaged by the display module 2 on the glasses 1 (or near-eye display device 10) to which it is connected. As Figure 7 shown,[[]] Figure 7 is a schematic diagram of the step flow of the control method for the display module provided by the embodiments of the present application. The control method for the display module includes:

[0064] S11, when the main interface is imaged in the preset imaging area of the display module on the lens, detecting the relative positions between the display module and the respective thumbnail images on the main interface;

[0065] S12, when it is detected that the display module is placed above a thumbnail image, running the application associated with the thumbnail image and magnifying and displaying the thumbnail image on the display module.

[0066] The lens is the lens on the glasses currently connected to the display module. Exemplarily, it can be the one described in the above embodiments, which will not be elaborated here.

[0067] In the embodiments of the present application, the display module for enhancing augmented reality display imaging on the lens of the glasses can perform some movements on the lens to which it is connected, rather than fixing the display module on the lens. In some alternative embodiments, an inertial sensor or an acceleration sensor (or gyroscope) is built into the display module. Through such sensors, the instantaneous speed and acceleration of the display module can be collected, and thus the movement trajectory of the display module can be determined based on the collected instantaneous speed and acceleration and combined with parameters such as time.

[0068] The operating system usually provides the user with a desktop or main interface function. The user can customize the icons (thumbnails) of some files, applications, or programs on the desktop or main interface. For example, in Figure 6 In the example shown, the display module 2 images the main interface function provided by the display module 2 on the lens 12. The user can customize the icons (thumbnails) of some files, applications, or programs in this main interface.

[0069] Optionally, a fixed reference point is pre-selected on the lens to which the display module is connected as the origin. The display module starts moving from this origin and always monitors its movement trajectory. The position of any point on this movement trajectory relative to the origin can represent the relative position of the display module relative to the connected lens. And the relative positions of the respective thumbnails on the main interface on the lens are determined when the display module outputs the image, that is, the relative positions of the respective thumbnails on the lens are known.

[0070] Exemplarily, as Figure 8 shown, Figure 8 is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application. For example, the point P0 on the lens 12 is the pre-selected origin. The display module 2 moves from the origin along the path l1 to the point P1 and then along the path l2 to the point P2. The movement trajectory l of the display module 2 at any moment t during this movement process can be determined by monitoring the movement parameters (such as speed, acceleration, etc.) of the display module 2. The position relationship of the end point of the movement trajectory at any moment t relative to the origin P0 can represent the relative position of the display module 2 on the lens 12. For example, at the moment t1, the display module 2 moves to the point P1. According to the determined path l1, the position relationship of this point P1 relative to the origin P0 can be determined, that is, the relative position of the display module 2 on the lens 12. Exemplarily, it is represented as indicating that the point P1 is located at the end point of the vector and the starting point of this vector is P0.

[0071] Similarly, when the movement trajectory of the display module 2 during the movement from point P1 to point P2 is l2, it is obvious that the positional relationship of P2 relative to P1 can be expressed as indicating that point P2 is located at the end position of the vector and the starting point of this vector is P1. Based on the vector relationship in mathematics, therefore, there is Obviously, the above vector is actually the displacement of the display module 2 during this process.

[0072] It should be added that in the above example, when describing the path and movement trajectory of the display module 2, a straight-line trajectory (path) is used for illustration. Obviously, the display module 2 can move arbitrarily and is not limited to the form of the straight-line trajectory in the above example, such as a curved trajectory.

[0073] When the display module 2 forms an image of the main interface on the lens 12, the relative positions of the respective thumbnails on the main interface on the lens 12 are known. Based on the known relative positions of the thumbnails on the lens 12 and the monitored relative position of the display module 2 on the lens 12, the relative positional relationship between the display module 2 and the respective thumbnails on the main interface can be determined.

[0074] When it is detected that the display module is placed above a thumbnail, the thumbnail is enlarged and displayed on the display module.

[0075] Exemplarily, in Figure 6 the user moves the display module 2 above the thumbnail [A], then the thumbnail [A] is enlarged and displayed in the display area 21 of the display module 2.

[0076] In the embodiment of the present application, when the display module forms an image of the main interface on the connected lens, the relative positions between the display module and the respective thumbnails on the main interface are detected. When it is detected that the display module is placed above a thumbnail, the thumbnail is enlarged. So that the user can confirm whether to select the thumbnail. Thus, to a certain extent, it enables the user to accurately interact with these thumbnails without confusion or errors.

[0077] In some optional implementation manners, the control method of the display module provided in the embodiment of the present application further includes: when it is detected that the display module is placed above at least one thumbnail, the thumbnail that is the closest to the displayed module among the at least one thumbnail is enlarged and displayed on the display module.

[0078] Exemplarily, in Figure 9Among them, the display module 2 is moved to be placed above the thumbnails [D1], [D2], and [D3]. When determining the distance between the display module 2 and the thumbnail, it is based on the distance between the center point of the thumbnail and the center point of the display area of the display module 2. Obviously, the thumbnail [D1] is closer to the display module 2. Therefore, the thumbnail [D1] is enlarged and displayed within the display area 21 of the display module 2.

[0079] In some optional embodiments, the control method for the display module provided by the embodiments of the present application further includes:

[0080] When there is a thumbnail that is enlarged, detect the contact of the touch part on the display module;

[0081] When it is detected that the touch part is touched in the first touch mode, control the display module to image the graphical interface of the application associated with the enlarged thumbnail within the preset imaging area on the lens; the first touch mode is preset as the trigger condition for the application associated with the enlarged thumbnail

[0082] The first touch mode can be a single click, a double click, a triple click, and so on.

[0083] When a certain thumbnail is enlarged and displayed on the display module, and the user touches the touch part of the display module in the first touch mode, it is determined that the user desires to start (or enter) the application associated with the enlarged thumbnail. Therefore, when a certain thumbnail is enlarged and displayed on the display module, and it is detected that the touch part of the display module is touched in the first touch mode, the application associated with the thumbnail is run in the foreground, and the display module is controlled to image the graphical interface of the application associated with the enlarged thumbnail within the preset imaging area on the lens. When the application associated with the thumbnail is already running in the background, it is switched to the foreground. That is, it realizes assisting the user to select the thumbnail on the main interface to enter the application associated with the thumbnail and use the relevant functions.

[0084] Exemplarily, in the example where the enlarged display is the thumbnail [A] mentioned above, when it is detected that the touch part of the display module 2 is touched in the first touch mode when the enlarged display is the thumbnail [A], the application associated with the thumbnail [A] is run in the foreground, and the display module 2 is controlled to image the graphical interface of the application associated with the enlarged thumbnail [A] within the preset imaging area 121 on the lens. Exemplarily, in the example where the enlarged display is the thumbnail [D1] mentioned above, the application associated with the thumbnail [D1] is run in the foreground, and the display module 2 is controlled to image the graphical interface of the application associated with the enlarged thumbnail [D1] within the preset imaging area 121 on the lens.

[0085] In the embodiments of the present application, when the display module forms an image of the main interface on the connected lens, the relative positions between the display module and each thumbnail on the main interface are detected. When it is detected that the display module is placed above a thumbnail, the thumbnail is enlarged so that the user can confirm whether to select the thumbnail. When the user touches the touch part of the display module according to the first touch mode, it is determined that the user desires to start (or enter) the application associated with the enlarged thumbnail, and then enter the application and display its graphical interface. This realizes assisting the user in selecting the thumbnail on the main interface to enter the application associated with the thumbnail and use related functions, thereby to a certain extent enabling the user to accurately interact with these thumbnails without confusion or errors.

[0086] In some alternative embodiments, the control method of the display module provided in the embodiments of the present application further includes:

[0087] When the main interface is imaged in the preset imaging area of the display module on the lens, monitor the motion state of the display module, detect the contact of the touch part on the display module, and obtain multimedia data;

[0088] When it is monitored that the display module performs axial rotation, or when it is detected that the touch part is touched according to the second touch mode, or when a page turning instruction is recognized in the multimedia data, control the display module to switch the current page of the main interface imaged in the preset imaging area on the lens to the previous page or the next page of the imaged main interface; the axial rotation of the display module, the second touch mode, and the page turning instruction are preset as the trigger conditions for triggering the page turning function.

[0089] When the main interface is imaged in the preset imaging area of the display module on the lens, monitor the motion state of the display module, detect the contact of the touch part on the display module, and obtain multimedia data. For the description of monitoring the motion state of the display module and detecting the touch part on the display module, please refer to the foregoing, and details are not repeated here. The display module can obtain multimedia data through its own camera, microphone, etc., or can obtain multimedia data from the network or from a smart terminal or a near-eye display device communicatively connected thereto.

[0090] Exemplarily, as Figure 10 and Figure 11 shown, Figure 10 and Figure 11 is a schematic diagram of an application scenario of a display module provided in the embodiments of the present application. It can be preset that the axial rotation of the display module 2 in the counterclockwise direction shown in Figure 13 is the trigger condition for triggering the main interface to turn up (or down) a page, and that the axial rotation of the display module 2 in the direction shown in Figure 11The preset clockwise axis rotation shown is the trigger condition for triggering the main interface to flip down (or up). When it is detected that the display module 2 rotates counterclockwise (or clockwise) around the axis, the display module 2 is controlled to update the main interface currently imaged on the lens 12, that is, the display module 2 is controlled to image the previous page (or the next page) of the main interface on the lens 12.

[0091] Exemplarily, the second touch mode may include two different touch methods, such as single click and double click, or double click and triple click. Among them, one touch method of the two different touch methods is the trigger condition for flipping up the page, and one touch method is the trigger condition for flipping down the page. For example, when a single click is detected, the main interface flips up, that is, the display module 2 is controlled to image the previous page of the main interface on the lens 12; when a triple click is detected, the main interface flips down, that is, the display module 2 is controlled to image the next page of the main interface on the lens 12.

[0092] It should be added that the first touch mode and the second touch mode should be two completely different touch modes to ensure the orderly progress of starting applications from the main interface and the main interface page flipping function without errors.

[0093] Exemplarily, the multimedia data is voice data or gesture data. When a page flipping voice command or a target gesture is recognized from it, the main interface flips the page, that is, the display module 2 is controlled to image the previous page (or the next page) of the main interface on the lens 12.

[0094] In some alternative embodiments, setting the detected axis rotation of the display module on the thumbnail as the trigger condition for the main interface zoom function. Setting the clockwise (or counterclockwise) axis rotation of the display module as the trigger condition for magnifying the thumbnail under the display module; setting the counterclockwise (or clockwise) axis rotation of the display module as the trigger condition for zooming the main interface.

[0095] Exemplarily, as Figure 12 shown, in Figure 11 the scene shown, when the display module 2 rotates clockwise around the axis, the thumbnail [J] is magnified; as Figure 12 shown, in Figure 11 the scene shown, when the display module 2 rotates counterclockwise around the axis, the current page of the main interface is zoomed.

[0096] In some alternative embodiments, the control method for the display module provided in the embodiments of the present application further includes:

[0097] When the graphic interface of the application is imaged within the preset imaging area of the display module on the lens, monitoring the motion state of the display module, detecting the contact of the touch part on the display module, and acquiring multimedia data;

[0098] When it is detected that the motion state is the axial rotation of the display module, or when it is detected that the touch part is touched in a target touch mode, or when a target instruction is recognized in the multimedia data, control the display module to update a first target area of the graphical interface of the application imaged in a preset imaging area on the lens; the first target area is a graphical area corresponding to a sub-function in the application, and the axial rotation of the display module, the target touch mode, and the target instruction are preset as trigger conditions for triggering the sub-function of the application.

[0099] Exemplarily, if the currently imaged graphical interface is that of a music player function, the axial rotation of the display module 2 in the counterclockwise direction can be preset as a trigger condition for triggering skipping to the previous (or next) song, and the axial rotation of the display module 2 in the clockwise direction can be preset as a trigger condition for triggering skipping to the next (or previous) song. When it is detected that the display module 2 performs a counterclockwise (or clockwise) axial rotation, perform a song skipping operation on the currently playing song, and control the display module 2 to update the music player interface currently imaged on the lens 12, that is, control the display module 2 to update the song information in the music player interface imaged on the lens 12 to the information of the previous (or next) song.

[0100] Among them, the multimedia data can be image / video data collected by a micro camera built in the display module or voice data collected by a micro microphone, and the eye tracking data is eye image data of the user collected by a micro camera built in the display module.

[0101] The target touch mode can be single click, double click, triple click, etc. The target instruction can be a preset target gesture, target voice, etc. The target eye movement mode can be the eyes moving according to a preset rule. Similarly, the target touch mode and the target instruction are similar to the aforementioned axial rotation of the display module, and they can also be preset as trigger conditions for the sub-functions of the application. When it is detected that the touch part of the display module is touched in a target touch mode, or when a target instruction is recognized from the multimedia data, the trigger condition (preset) is the sub-function of the application of the target touch mode, the target instruction, and the axial rotation of the display module, so as to control the display module to update the area in the graphical interface currently imaged on the lens. The principle is similar to that of the axial rotation of the display module triggering the sub-function of the application, so as to control the display module to update the area in the graphical interface currently imaged on the lens, and will not be elaborated here. Those skilled in the art can understand by referring to the foregoing.

[0102] In some optional embodiments, the control method of the display module further includes:

[0103] When it is detected that the rotation angle of the axis rotation of the display module is not less than a preset first threshold angle β, control the display module to update the graphical interface imaged in the preset imaging area on the lens.

[0104] The first threshold angle β can be flexibly set by those skilled in the art or users according to actual needs. For example, it can be 15°, 20°, 25°, 30°, etc.

[0105] Thus, it is possible to avoid updating the graphical interface imaged on the lens 12 when the display module 2 is slightly driven to perform axis rotation. Therefore, the sub-function of the third logic function can be reduced to a certain extent.

[0106] It should be noted that although the above examples are all described with a monocular (i.e., the display module provided by the embodiment of the present application is only connected to one lens of the glasses) as an example, obviously all the embodiments provided by the present application can also be applied to the binocular (i.e., the display module provided by the embodiment of the present application is connected to both lenses of the glasses) scenario.

[0107] For the connection method of the projection device provided by the embodiment of the present application, the execution subject can be the control device of the display module. In the embodiment of the present application, taking the control device of the display module to execute the control method of the display module as an example, the control device of the display module provided by the embodiment of the present application is described.

[0108] As Figure 14 shown, a schematic structural diagram of a control device of a display module provided by an embodiment of the present application is shown. Please refer to Figure 14 , for the control device 5 of the display module, the display module is configured to be placed on the optical lens, and the display module is configured to output image content. The control device 5 includes:

[0109] The first detection module 51 is used to detect the relative positions between the display module and each thumbnail on the main interface when the main interface is imaged in the preset imaging area on the lens.

[0110] The first magnification module 52 is used to magnify and display the thumbnail on the display module when it is detected that the display module is placed above a thumbnail.

[0111] In some optional embodiments, the device 5 further includes:

[0112] The second magnification module is used to magnify and display the thumbnail closest to the displayed module among the at least one thumbnail on the display module when it is detected that the display module is placed above at least one thumbnail.

[0113] In some alternative embodiments, the device 5 further includes:

[0114] A second detection module, configured to detect contact of the touch part on the display module when there is a magnified thumbnail.

[0115] A first control module, configured to control the display module to image a graphical interface of an application associated with the magnified thumbnail in a preset imaging area on the lens when it is detected that the touch part is touched in a first touch mode; the first touch mode is preset as a trigger condition for the application associated with the magnified thumbnail.

[0116] In some alternative embodiments, the device 5 further includes:

[0117] A first monitoring module, configured to monitor a motion state of the display module when a main interface is imaged in a preset imaging area on the lens.

[0118] A second control module, configured to control the display module to switch the current page of the main interface imaged in the preset imaging area on the lens to the previous page or the next page of the imaged main interface when it is monitored that the display module rotates axially; the axial rotation of the display module is preset as a trigger condition for triggering a page turning function.

[0119] In some alternative embodiments, the device 5 further includes:

[0120] A third detection module, configured to detect contact of the touch part on the display module when a main interface is imaged in a preset imaging area on the lens.

[0121] A third control module, configured to control the display module to switch the current page of the main interface imaged in the preset imaging area on the lens to the previous page or the next page of the imaged main interface when it is detected that the touch part is touched in a second touch mode; the second touch mode is preset as a trigger condition for triggering a page turning function.

[0122] In some alternative embodiments, the device 5 further includes:

[0123] A first acquisition module, configured to acquire multimedia data when a main interface is imaged in a preset imaging area on the lens.

[0124] A fourth control module, configured to control the display module to switch the current page of the main interface imaged in the preset imaging area on the lens to the previous page or the next page of the imaged main interface when a page turning instruction is recognized in the multimedia data; the page turning instruction is preset as a trigger condition for triggering a page turning function.

[0125] In some alternative embodiments, the device 5 further includes:

[0126] A second monitoring module, configured to monitor a motion state of the display module when a graphical interface of an application is imaged within a preset imaging area on the lens.

[0127] A fifth control module, configured to control the display module to update a first target area of the graphical interface of the application imaged within the preset imaging area on the lens when it is monitored that the motion state is axial rotation of the display module; the first target area is a graphical area corresponding to a sub-function in the application, and the axial rotation of the display module is preset as a trigger condition for triggering the sub-function of the application.

[0128] In some alternative embodiments, the device 5 further includes:

[0129] A fourth control module, configured to detect contact of a touch part on the display module when a graphical interface of an application is imaged within the preset imaging area on the lens.

[0130] A sixth control module, configured to control the display module to update a first target area of the graphical interface of the application imaged within the preset imaging area on the lens when it is detected that the touch part is touched in a target touch mode; the first target area is a graphical area corresponding to a sub-function in the application, and the target touch mode is preset as a trigger condition for triggering the sub-function of the application.

[0131] In some alternative embodiments, the device 5 further includes:

[0132] A second acquisition module, configured to acquire multimedia data when a graphical interface of an application is imaged within the preset imaging area on the lens.

[0133] A seventh control module, configured to control the display module to update a first target area of the graphical interface of the application imaged within the preset imaging area on the lens when a target instruction is recognized in the multimedia data; the first target area is a graphical area corresponding to a sub-function in the application, and the target instruction is preset as a trigger condition for triggering the sub-function of the application.

[0134] The control device 5 of the display module in the embodiments of the present application may be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than terminals. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a handheld computer, an in-vehicle electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It may also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0135] The control device 5 of the display module in the embodiments of the present application may be a device with an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems. The embodiments of the present application do not make specific limitations.

[0136] The control device 5 of the display module provided by the embodiments of the present application can implement Figures 1 to 12 each process implemented by the embodiments described above. To avoid repetition, it will not be elaborated here.

[0137] In some alternative embodiments, as Figure 15 shown, the embodiments of the present application further provide an electronic device 130, including a processor 131 and a memory 132. A program or instruction that can run on the processor 131 is stored on the memory 132. When the program or instruction is executed by the processor 131, it implements each step of the embodiments of the above display module control method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0138] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0139] Figure 16 A schematic diagram of the hardware structure of an electronic device for implementing the embodiments of the present application.

[0140] The electronic device 140 includes, but is not limited to, components such as a radio frequency unit 141, a network module 142, an audio output unit 143, an input unit 144, a sensor 145, a display unit 146, a user input unit 147, an interface unit 148, a memory 149, and a processor 1410. Those skilled in the art can understand that the electronic device 140 may further include a power source (such as a battery) for supplying power to each component. The power source can be logically connected to the processor 1410 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 14 The structure of the electronic device shown in Figure 14 does not limit the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0141] Among them, the processor 1410 is used for:

[0142] When the main interface is imaged in the preset imaging area of the lens by the display module, detecting the relative positions between the display module and each thumbnail on the main interface;

[0143] When it is detected that the display module is placed on a thumbnail, running the application associated with the thumbnail and controlling the display module to image the graphical interface of the application in the preset imaging area of the lens.

[0144] It should be understood that in the embodiments of the present application, the input unit 144 may include a graphics processing unit (GPU) 1441 and a microphone 1442. The graphics processing unit 1441 processes the image data of static pictures or videos obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 146 may include a display panel 1461, and the display panel 1461 may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 147 includes at least one of a touch panel 1471 and other input devices 1472. The touch panel 1471 is also called a touch screen. The touch panel 1471 may include two parts: a touch detection device and a touch controller. The other input devices 1472 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0145] The memory 149 can be used to store software programs and various data. The memory 149 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area can store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 149 can include volatile memory or non-volatile memory, or the memory 149 can include both volatile and non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synch link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 149 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0146] The processor 1410 may include one or more processing units; optionally, the processor 1410 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 1410 either.

[0147] For the control method of the display module provided in the embodiments of the present application, the execution subject can be a near-eye display device. In the embodiments of the present application, taking the near-eye display device as an example to execute the control method of the display module, the near-eye display device provided in the embodiments of the present application is described.

[0148] In some alternative embodiments, such as Figure 17As shown in the figure, the embodiment of the present application further provides a near-eye display device 150, including a processor 151 and a memory 152. A program or instruction that can run on the processor 151 is stored on the memory 152. When the program or instruction is executed by the processor 131, each step of the control method embodiment of the above display module is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be elaborated here.

[0149] Each of the above product embodiments can implement each process of the control method embodiment of the above display module by running through its own processor, and the same technical effect can be achieved. To avoid repetition, it will not be elaborated one by one.

[0150] The embodiment of the present application further provides a readable storage medium. A program or instruction is stored on the readable storage medium. When the program or instruction is executed by a processor, each process of the control method embodiment of the above display module is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be elaborated here. Among them, the processor is the processor in the electronic device or electronic system described in the above embodiment. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disk, or optical disc, etc.

[0151] The embodiment of the present application further provides a chip. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run a program or instruction to implement each process of the control method embodiment of the above display module, and the same technical effect can be achieved. To avoid repetition, it will not be elaborated here.

[0152] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0153] The embodiment of the present application provides a computer program product. The program product is stored in a storage medium. The program product is executed by at least one processor to implement each process of the control method embodiment of the above display module, and the same technical effect can be achieved. To avoid repetition, it will not be elaborated here.

[0154] In the embodiments provided by this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0155] In addition, in each embodiment of this application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0156] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application embodiment, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0157] The above is only the embodiments of this application, and does not limit the patent scope of this application. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of this application, those of ordinary skill in the art, as long as they use equivalent structural or equivalent process transformations made by using the specification and drawings of this application embodiment, or directly or indirectly apply them in other related technical fields, without departing from the purpose of this application and the scope protected by the claims, many forms can still be made, and all are equally included in the patent protection scope of this application embodiment.

Claims

1. A control method for a display module, characterized in that, The display module is configured to be placed on an optical lens, and the display module is configured to output image content. The method includes: When the main interface is imaged in a preset imaging area of the display module on the lens, detecting the relative positions between the display module and each thumbnail on the main interface; When it is detected that the display module is placed above a thumbnail, magnifying and displaying the thumbnail on the display module.

2. The method according to claim 1, wherein The method further includes: When it is detected that the display module is placed above at least one thumbnail, magnifying and displaying the thumbnail closest to the display module among the at least one thumbnail on the display module.

3. The method according to claim 1 or 2, characterized in that The method further includes: When there is a magnified thumbnail, detecting the contact of a touch part on the display module; When it is detected that the touch part is touched in a first touch mode, controlling the display module to image the graphical interface of the application associated with the magnified thumbnail in the preset imaging area on the lens; the first touch mode is preset as the trigger condition for the application associated with the magnified thumbnail.

4. The method according to claim 1, characterized in that The method further includes: When the main interface is imaged in a preset imaging area of the display module on the lens, monitoring the motion state of the display module; When it is monitored that the display module performs an axis rotation, controlling the display module to switch the current page of the main interface imaged in the preset imaging area on the lens to the previous page or the next page of the imaged main interface; the axis rotation of the display module is preset as the trigger condition for triggering the page turning function.

5. The method according to claim 1, characterized in that, The method further includes: When the main interface is imaged in a preset imaging area of the display module on the lens, detecting the contact of a touch part on the display module; When it is detected that the touch part is touched in a second touch mode, controlling the display module to switch the current page of the main interface imaged in the preset imaging area on the lens to the previous page or the next page of the imaged main interface; the second touch mode is preset as the trigger condition for triggering the page turning function.

6. The method according to claim 1, characterized in that, The method further includes: When the main interface is imaged in a preset imaging area of the display module on the lens, acquiring multimedia data; When a page turning instruction is recognized in the multimedia data, controlling the display module to switch the current page of the main interface imaged in the preset imaging area on the lens to the previous page or the next page of the imaged main interface; the page turning instruction is preset as the trigger condition for triggering the page turning function.

7. The method according to claim 1, characterized in that The method further includes: When the graphical interface of an application is imaged in a preset imaging area of the display module on the lens, monitoring the motion state of the display module; When it is monitored that the motion state is that the display module performs an axis rotation, controlling the display module to update a first target area of the graphical interface of the application imaged in the preset imaging area on the lens; the first target area is the graphical area corresponding to a sub-function in the application, and the axis rotation of the display module is preset as the trigger condition for triggering the sub-function of the application.

8. The method according to claim 1, wherein The method further includes: When the graphical interface of the application is imaged in the preset imaging area of the display module on the lens, detect the contact of the touch part on the display module; When it is detected that the touch part is touched according to the target touch mode, control the display module to update the first target area of the graphical interface of the application imaged in the preset imaging area on the lens; the first target area is the graphical area corresponding to the sub-function in the application, and the target touch mode is preset as the trigger condition for triggering the sub-function of the application.

9. The method according to claim 1, wherein The method further includes: When the graphical interface of the application is imaged in the preset imaging area of the display module on the lens, obtain multimedia data; When a target instruction is recognized in the multimedia data, control the display module to update the first target area of the graphical interface of the application imaged in the preset imaging area on the lens; the first target area is the graphical area corresponding to the sub-function in the application, and the target instruction is preset as the trigger condition for triggering the sub-function of the application.

10. A near-eye display device, characterized in that, The near-eye display device includes: a processor and a memory, the memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the control method of the display module according to any one of claims 1 to 9 are implemented.

Images