Control method of display module and near-to-eye display device

Abstract

The embodiment of the invention provides a control method of a display module and a near-to-eye display device. The display module is configured to be placed on an optical lens and output image content, and the control method of the display module comprises the following steps: detecting the relative position of the display module on a currently connected lens and the contact of a touch part of the display module; marking a first relative position as a first fitting point under the condition that it is detected that a touch part at the first relative position, static on the lens, of the display module is touched according to a first touch mode in the imaging area setting mode; the first relative position is any position of the display module on the lens, and the first touch mode in the imaging area setting mode is a touch mode used for representing that a user confirms that the user can see the display area of the display module clearly; fitting the at least three first fitting points into a closed area under the condition that the at least three first fitting points are marked; and setting the closed area as a preset imaging area of the display module on the lens.

Description

Technical Field

[0001] This application relates to the field of near-eye displays, and particularly to a control method for a display module and a near-eye display device. Background Art

[0002] Near-eye display refers to imaging at a relatively short imaging distance from the eyes for sensory perception. For example, virtual reality (VR) glasses and augmented reality (AR) glasses both image at a relatively short distance from the eyes. That is to say, near-eye display is a relative concept compared to traditional televisions, monitors, etc. Obviously, the imaging distance from the display panel of a traditional television, monitor, etc. to the eyes is much greater than that of near-eye display devices such as VR glasses or AR glasses.

[0003] Currently, near-eye displays can generally be divided into VR, AR, mixed reality (MR), and extended reality (XR).

[0004] VR, also known as computer-simulated reality. It is an experience created through human-computer interaction and computer-generated three-dimensional simulations. We can interact in the environment by using virtual reality devices, such as headsets, controllers. That is to say, VR is a computer simulation system that can create and experience virtual worlds. It uses a computer to generate a simulated environment and immerses us in this environment.

[0005] AR is a real-time, direct or indirect observation of the physical environment of the real world. It combines what we see in the real environment with digital content generated by computer software to enhance the real environment we are in in some way. The AR system transmits virtual information to headsets or smart glasses in real time through a camera, or through a mobile device, allowing us to clearly view 3D images.

[0006] MR is the integration of the real world and the virtual world to create a new environment and visualization. Among them, physical objects and digital objects coexist and interact in real time. This means that if a new image is placed in the real space, to a certain extent, this new image will interact with real objects in our real environment.

[0007] XR refers to the combination of all real and virtual environments generated by computer technology and wearable devices, as well as human-computer interaction. XR enhances our senses through digitization to integrate the world. In addition, it provides a large number of virtual sensor input levels at different levels for immersive virtual experiences. XR includes the above three emerging technologies, namely VR, AR, and MR. Summary of the Invention

[0008] In related technologies, the size of the lens of certain glasses (such as goggles) may be much larger than the field of view of the human eye, such that a user wearing the glasses cannot see or clearly see some areas on the lens, that is, there may be some areas (i.e., blind spots) on the lens of the glasses that are outside the field of view of the user wearing the glasses; if the display module connected to the lens of the glasses forms an image in the blind spot on the lens, it will cause the user wearing the glasses to be unable to see or clearly see. The purpose of the embodiments of the present application is to provide a control method, device, electronic device, near-eye display device, medium, chip, and computer program product for a display module to independently or semi-independently solve to a certain extent the technical problem that if the display module connected to the lens of the glasses forms an image in the blind spot on the lens, it will cause the user wearing the glasses to be unable to see or clearly see. The so-called independent solution refers to providing a hardware structure solution that can work independently without the cooperation of software, and can achieve the effect that the display module connected to the lens of the glasses in near-eye display technology does not form an image in the blind spot on the lens. The so-called semi-independent solution refers to providing a hardware structure solution that can achieve the effect that the display module connected to the lens of the glasses in near-eye display technology does not form an image in the blind spot on the lens when working in cooperation with near-eye display software - for example, the software for a user to interact with virtual information in AR technology.

[0009] The first aspect of the embodiments of the present application provides a control method for a display module, where the display module is configured to be placed on an optical lens, and the display module is configured to output image content. The method includes:

[0010] Detect the relative position of the display module on the currently connected lens and the contact of the touch part of the display module;

[0011] When it is detected in the imaging area setting mode that the display module is stationary at a first relative position on the lens and the touch part is touched in a first touch mode, mark the first relative position as a first fitting point; the first relative position is an arbitrary position of the display module on the lens, and in the imaging area setting mode, the first touch mode is a touch mode used to indicate that the user confirms that they can clearly see the display area of the display module;

[0012] When at least three first fitting points are marked, fit the at least three first fitting points into a closed area;

[0013] Set the closed area as the preset imaging area of the display module on the lens.

[0014] A second aspect of the embodiments of the present application provides a control device for a display module. The display module is configured to be placed on an optical lens, and the display module is configured to output image content. The device includes:

[0015] A first detection module, configured to detect the relative position of the display module on the currently connected lens and the contact of the touch part of the display module;

[0016] A first marking module, configured to mark the first relative position as a first fitting point when it is detected in the imaging area setting mode that the touch part is touched in a first touch mode at a first relative position where the display module is stationary on the lens; the first relative position is an arbitrary position of the display module on the lens, and in the imaging area setting mode, the first touch mode is a touch mode used to indicate that the user confirms that they can clearly see the display area of the display module;

[0017] A first fitting module, configured to fit the at least three first fitting points into a closed area when at least three first fitting points are marked;

[0018] A first setting module, configured to set the closed area as the preset imaging area of the display module on the lens.

[0019] A third aspect of the embodiments of the present application provides an electronic device, including: a processor and a memory. The memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the control method of the display module as described in the first aspect are implemented.

[0020] A fourth aspect of the embodiments of the present application provides a near-eye display device, including: a processor and a memory. The memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the control method of the display module as described in the first aspect are implemented.

[0021] A fifth aspect of the embodiments of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the control method of the display module as described in the first aspect are implemented.

[0022] A sixth aspect of the embodiments of the present application provides a chip, including a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run a program or instruction to implement the steps of the control method of the display module as described in the first aspect.

[0023] A seventh aspect of the embodiments of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps of the control method of the display module as described in the first aspect.

[0024] In the embodiments of the present application, in the imaging area setting mode, by detecting the relative position of the display module on the currently connected lens and the contact of the touch part of the display module, at least three first fitting points that can be clearly seen by the user are found on the currently connected lens of the display module, and then the at least three found fitting points are fitted into a closed area. Since the boundary points on this closed area (obviously these boundary points include the first fitting points) can all be clearly seen by the user, naturally the user can also clearly see this closed area, and this closed area is set as the preset imaging area of the display module on the lens. After that, the display module will image within the preset imaging area on the lens and will not image within the blind area on the lens, thus effectively solving the technical problem that when the display module connected to the lens of the glasses images within the blind area on the lens, it will cause the user wearing the glasses to be unable to see or clearly see the virtual information within the blind area.

[0025] Other technical effects of the present application are described in detail in the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a schematic connection diagram between a display module and a lens provided by an embodiment of the present application;

[0027] Figure 2 is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0028] Figure 3 is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0029] Figure 4 is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0030] Figure 5 is a schematic diagram of the connection relationship between a display module and a smart terminal provided by an embodiment of the present application;

[0031] Figure 6 is a schematic diagram of the main imaging interface of a display module provided by an embodiment of the present application;

[0032] Figure 7 is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0033] Figure 8 is a schematic diagram of the step flow of the control method of the display module provided by an embodiment of the present application;

[0034] Figure 9 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0035] Figure 10 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0036] Figure 11 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0037] Figure 12 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0038] Figure 13 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0039] Figure 14 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0040] Figure 15 It is a schematic diagram of the step flow of the control method of the display module provided by an embodiment of the present application;

[0041] Figure 16 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0042] Figure 17 It is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application;

[0043] Figure 18 It is a schematic diagram of the step flow of the control method of the display module provided by an embodiment of the present application;

[0044] Figure 19 It is a schematic diagram of the step flow of the control method of the display module provided by an embodiment of the present application;

[0045] Figure 20 It is a schematic diagram of the step flow of the control method of the display module provided by an embodiment of the present application;

[0046] Figure 21 It is a schematic diagram of the step flow of the control method of the display module provided by an embodiment of the present application;

[0047] Figure 22 It is a schematic diagram of the step flow of the control method of the display module provided by an embodiment of the present application;

[0048] Figure 23 It is a schematic diagram of the structure of the control device of the display module provided by an embodiment of the present application;

[0049] Figure 24 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0050] Figure 25 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0051] Figure 26 is a schematic structural diagram of a near-eye display device provided by an embodiment of the present application. Detailed implementation manners

[0052] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0053] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same category, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the related objects before and after.

[0054] An embodiment of the present application provides a control method, device, electronic device, near-eye display device, storage medium, chip and computer program product for a display module, which can effectively solve the technical problem that if the display module connected to the lens of glasses forms an image in the blind area of the lens, it will cause the user wearing the glasses to be unable to see or clearly see the virtual information in the blind area. The following is an explanation in conjunction with the accompanying drawings. To facilitate the (correct) understanding of the embodiments of the present application, the following first defines and explains the technical terms that may be involved in the embodiments of the present application. Without special explanation in the following text, the semantics of the corresponding technical terms follow the definitions here:

[0055] Computer or computing device: refers to all electronic devices based on the Turing computability theory, von Neumann architecture or Harvard architecture. For example, mobile phones, smart watches, single-chip microcomputer systems, etc. are all computers, not limited to the narrow sense of computers such as personal computers (PC, Personal Computer) mentioned in daily life.

[0056] Application: That is, a computer application program (Computer Program or Application Software), which is written in certain programming languages and runs on a certain target architecture computer system. A computer application program refers to a coded instruction sequence that can be executed by a device with information processing capabilities such as a computer to obtain a certain result, or a symbolic instruction sequence or symbolic statement sequence that can be automatically converted into a coded instruction sequence.

[0057] Logical functions: Refer to various functions provided by various electronic devices such as near-eye display devices, computers, or computing devices (with independent computing capabilities). These functions can be user-oriented functions or system (or device)-oriented functions. For example, user-oriented logical functions can be: Graphical User Interface (GUI) output function, audio output function, main interface or desktop, programs / applications, sub-functions in programs / applications, etc. Among them, the main interface or desktop, applications, etc. can all provide a GUI to the user for the user to interact with. System-oriented logical functions can be: fragmented file sorting, system error diagnosis, etc.

[0058] Foreground running and background running: Foreground running and background running are two relative concepts. Both are used to describe the running modes of applications / programs or processes in an operating system (generally speaking, the operating system refers to the software in a computer that enables applications to communicate with the underlying computer hardware). Taking a traditional computer as an example, the foreground running and background running in AR technology are similar to the foreground running and background running characteristics in a traditional computer. In a traditional computer, when an application runs in the foreground, the GUI of the application is usually displayed on the display screen, and the user can see the GUI or window of the application. The user can directly interact or communicate with the application based on the GUI. At this time, the application usually occupies input devices (such as a mouse and keyboard); the application running in the foreground usually blocks the user interface, and the user needs to wait for the application running in the foreground to complete execution or interact according to the requirements of the application running in the foreground; it is suitable for tasks that require real-time feedback from the user or user input. In a traditional computer, when an application runs in the background, the GUI is usually not output on the display screen, that is, it is invisible to the user, and the user usually cannot directly interact or communicate with the application; when the application runs in the background, the user can continue to use the computer to perform other tasks, and the application runs silently in a state invisible to the user, such as processing files, executing system services, etc.; the application running in the background usually does not occupy input devices and does not require the user to directly participate in the execution of tasks; it allows the user to continue to perform other tasks while the application is executing, improving the multitasking processing ability of the system.

[0059] Display module: It refers to a display component, display device, or display equipment that is based on optical technology for imaging. Therefore, the display module is a physical hardware module. In some alternative embodiments, the display module can be connected to the lens and powered on for imaging through the display module. The lens can be, for example, sunglasses commonly seen in daily life, glasses for vision correction, goggles, or a mask with a lens, etc., or the lens of a smart glasses with audio, sensing, etc. The display module can be connected to the lens by means of adsorption such as magnetism, clips, etc., so that the display module can move freely on the lens to which it is connected or move within a designated setting area. For example, the display module can move linearly or curvilinearly, rotate axially, etc. on the placed lens.

[0060] Exemplarily, as Figure 1 shown, Figure 1 is a schematic diagram of the connection between a display module and a lens provided by an embodiment of the present application. Please refer to Figure 1 , taking the lens of glasses as an example for illustration, but the present application is not limited to glasses, and it can also be the lens related to head-mounted devices such as helmets, goggles, vision correction, sports, etc. Among them, glasses 1 can be ordinary glasses, or AR glasses, smart glasses, etc. Glasses 1 include a frame 11, a lens 12 embedded in the frame 11, and temple arms 13 movably connected to the frame 11. The lens 12 includes an environment side 24 and an eye side 22. The user's eyes can see the physical environment on the environment side 24 through the lens 12 from the eye side 22. Optionally, the display module 2 is provided with a first magnetic attracting member at one end of the lens 12 facing the environment side 24, and the display module 2 is provided with a second magnetic attracting member on the lens 12 facing the eye side 22. It can be understood that two mutually attracting magnetic attracting members are arranged on both sides of the lens 12 to realize the pre-fixation of the display module on the lens. The first magnetic attracting member and the second magnetic attracting member can be mutually attracting magnets. Of course, one of them can be a magnet and the other can be a metal attracted by the magnet, etc., so that the whole display module 2 is adsorbed on the lens 12. At this time, the display module 2 can still slide freely or rotate axially on the lens 12 without detaching from the lens 12. Of course, other solutions such as clamping and hanging and moving are also feasible here.

[0061] Exemplarily, as Figure 2 and Figure 3 shown, Figure 2 and Figure 3 are both schematic diagrams of an application scenario of a display module provided by an embodiment of the present application. Figure 3 It can be the front view obtained by observing the glasses 1 from the direction where the temple arms 13 of the glasses 1 are located (i.e., the eye side 22). Please refer to Figure 1 , Figure 2 and Figure 3 . Figure 1The direction indicated by the straight arrow a is the direction in which the user's eye line of sight observes the virtual image formed by the display module 2 on the lens 12. The user can observe the end face of the display module 2 from the direction indicated by the arrow a and see the virtual image or graphical interface formed on the lens 12 based on AR technology. That is, from Figure 1 the end face of the display module 2 seen when observing the display module 2 from the direction of the arrow a shown in is the display area 21 of the display module 2. When the display module 2 is working, the display area 21 can be in a "lit" state, that is, it can form image content, such as text, interactive interfaces, videos, images, and other related display contents. The area on the lens 12 that is not occupied by the display module 2 still retains the properties of the lens 12 itself. For example, as the area that the user can see the environment side through the lens 12 from the eye side, such as eye protection, vision correction, etc., exists as the properties of the lens itself.

[0062] In some alternative embodiments, the central position of the lens 12 is the calibration position, which is the reference position of the display module 2. When the display module 2 is stationary at this calibration position, it forms a main interface or an interface with a preset logical function on the lens 12. The reason for determining the central position of the lens 12 as the calibration position is that the display module 2 can move in more directions and distances in the circumferential direction at the central position, which is convenient for the user to operate the display module 2 and customize the preset imaging area.

[0063] After the user wears the glasses 1, the user can press (or pinch) the touch part of the display module 2 to drag the display module 2 to move on the lens 12 or rotate the display module 2 to perform an axis rotation. In some embodiments, the display module 2 can be located on the environment side 24. The light of the display area 21 can pass through the lens and reach the user's eyes on the eye side 22 for imaging. At this time, the touch part can be located on the environment side 24, so the user can perform better interaction; in some other embodiments, the display module 2 can be located on the eye side 22. At this time, the light of the display area 21 can directly enter the user's eyes for imaging. At this time, the touch part can be located on the eye side 22, so the user can perform better and more concealed interaction. Exemplarily, as Figure 4 shown, Figure 4 is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application. Please refer to Figure 4 , the touch part of the display module 2 is located on the environment side 24. The user can press (or pinch) the touch part of the display module 2 to drag the display module 2 to move on the lens 12 or rotate the display module 2 to perform an axis rotation.

[0064] In some alternative embodiments, the virtual image or graphical interface formed by the display area 21 of the display module 2 on the lens 12 may come from the display module 2 itself or the glasses 1 communicatively connected to the display module 2. Of course, in other embodiments, it may also come from other electronic devices that are relatively independent of both the glasses 1 and the display module 2, such as smart terminals or computers like mobile phones, tablets, etc. If the glasses 1 are ordinary glasses, the virtual image or graphical interface formed by the display module 2 on the lens 12 may come from other electronic devices that are relatively independent of both the glasses 1 and the display module 2, such as smart terminals or computers like mobile phones, tablets, etc.

[0065] Exemplarily, as Figure 5 shown, Figure 5 FIG. is a schematic diagram of the connection relationship between a display module and a smart terminal provided by an embodiment of the present application. Please refer to Figure 5 , the display module 2 is physically connected to the near-eye display device 10, which may be the aforementioned glasses 1. The smart terminal 4 may be a PC, laptop, smartphone, tablet, smart speaker, server, etc. A wired or wireless communication connection is established between the display module 2 and the smart terminal 4. For example, the display module 2 and the smart terminal 4 communicate based on the Wifi protocol or the Bluetooth protocol; or, the display module 2 and the smart terminal 4 communicate based on the USB transmission protocol via a USB cable, and the display module 2 also obtains electrical energy from the smart terminal 4 via the USB cable. Of course, it is not limited to the foregoing examples. The display module 2 and the smart terminal 4 may also establish a wired or wireless communication connection based on other communication protocols, such as the NFC communication protocol, P2P (peer-to-peer) network, etc. The display module 2 and the near-eye display device 10 together form the "display device" of the smart terminal 4, and the GUI output by the smart terminal 4 is imaged on the near-eye display device 10 through the display module 2.

[0066] In some alternative embodiments, a touch sensor, or a pressure or capacitance sensor, etc. may be provided on the touch portion of the display module 2 to detect whether the touch portion of the display module 2 is touched. In some alternative embodiments, a micro camera is also provided on the touch portion of the display module 2 to collect image / video data from the environment. In some alternative embodiments, a micro microphone is also provided on the touch portion of the display module 2 to collect ambient sound from the environment. In some alternative embodiments, a micro camera is also provided on one side of the display area 21 of the display module 2 to collect the eye image of the user wearing the glasses 1 for tracking the user's eyeballs.

[0067] Trigger condition: It refers to the condition for running or executing the pre-set trigger logic function. In some optional implementation manners of the embodiments of the present application, certain motion states of the display module are pre-set as the trigger conditions for certain logic functions. For example, when the display module is stationary at a certain set position on the lens to which it is connected, it is pre-set as the trigger to output the main interface function. Then, when it is subsequently monitored that the display module is stationary at this set position, the display module will image the main interface on the lens to which it is connected, and the user can observe the main interface imaged on the lens by observing the display area of the display module. Optionally, there may be at least one application (or file) icon (thumbnail) in the main interface. A certain set position may be the center position of the lens, or of course, it may also be the four corners of the frame and other positions that are easier to remember and identify.

[0068] Pre-set imaging area: It refers to the imaging area pre-set for the display module on the lens to which the display module is connected. In some optional implementation manners, the pre-set imaging area may be a regular area on the lens to which the display module is connected, such as a rectangular area, a circular area, or an elliptical area. In some optional implementation manners, the pre-set imaging area may also be an irregular area on the lens to which the display module is connected. In some optional implementation manners, the pre-set imaging area may also be the entire lens area of the lens to which the display module is connected. In some optional implementation manners, the pre-set imaging area may also be a partial area of the lens to which the display module is connected. The control method of the display module provided by the embodiments of the present application can implement setting the pre-set imaging area on the lens currently connected to the display module. For specific details, please refer to the description in the following text.

[0069] Exemplarily, as Figure 6 shown, Figure 6 is a schematic diagram of the display module imaging the main interface provided by the embodiments of the present application. Please refer to Figure 6 , in this example, the pre-set imaging area of the display module 2 is the entire lens area of the lens 12. After the user wears the glasses 1 and observes from the side of the temple 13 (please refer to Figure 1 and Figure 2 ) towards the lens 12 (i.e., the eye side), the user can observe the main interface imaged by the display module 2 on the lens 12 by observing the display area 21 of the display module 2. That is, in the example of Figure 6 , when the display module 2 is stationary at the illustrated position on the lens 12 (this position is not limited to the position shown in Figure 6 , and the user can set the logic function of triggering the output of the main interface at any position where the display module 2 is stationary on the lens 12), the logic function of triggering the output of the main interface is triggered. In the example of Figure 6 , the pre-set imaging area 121 is the entire lens area of the lens 12, and the pre-set imaging area 121 is divided into multiple sub-areas (for example Figure 6In each sub-region, an icon (thumbnail) of an application or file mapped to the sub-region is imaged. For example, an icon of application F is imaged in sub-region 1211, and an icon of application E is imaged in sub-region 1212. Figure 6 In the example, the icons of application AI are imaged in six sub-areas. In some other optional implementations, at least one of the icons AI can be a file icon (thumbnail) such as a video, picture, audio, text, etc.

[0070] In some optional implementations, when the display module 2 is connected to the lens 12 for the first time and the display module 2 is turned on, the display module 2 images a main interface on the lens 12 by default.

[0071] It should be noted that in Figure 6 In order to facilitate the explanation, the main interface is divided into multiple sub-areas. Figure 6 The grid lines (i.e., the dotted grids on the lens 12) indicating each sub-area are schematically shown. The grid lines should not be understood as the main interface imaged by the display module 2 provided in the embodiment of the present application must have grid lines. Obviously, the grid lines are not necessary, that is, the grid lines may not be present in the imaged main interface, that is, after the display module 2 is placed on the lens 12, the grid lines cannot be seen on the body of the lens 12, and the lens 12 still maintains the properties of the lens itself. Of course, if the lens itself adopts a holographic lens such as an optical waveguide, it can be combined with each other for simple prompts, etc. Of course, a setting option can also be provided for the user to choose whether to image the grid lines when imaging the main interface. If the user sets the grid lines to be imaged when imaging the main interface, the display module will image the grid lines while imaging the main interface. If the user sets the grid lines not to be imaged when imaging the main interface, the display module will not image the grid lines when imaging the main interface. Of course, the grid lines can also correspond to the graphical interface on the smart terminal (such as a mobile phone app), which is convenient for the user to divide or customize the network lines of the preset imaging area 121 through the graphical interface of the terminal. In addition, the line type of the grid line is not limited to Figure 6 The dotted line shown in , can also be a solid line, a center line, etc.

[0072] In order to facilitate the overall understanding of the embodiments provided by the present application, on this basis, an exemplary description is given below of the technical defects addressed by the embodiments of the present application.

[0073] like Figure 7 As shown, Figure 7FIG. 0 is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application. In this exemplary scenario, a front view of a pair of goggles is schematically shown. In order to enable the goggles to better protect the wearer's eyes as much as possible, manufacturers usually make the lens larger so as to more comprehensively cover the eye area for protection without affecting the wearer's line of sight. Please refer to Figure 7 , after the user wears the exemplary goggles 5, the edge area of the lens 52 closer to the frame 51 is less likely to be clearly seen by the user, or even cannot be seen clearly, that is, some edge areas on the lens 52 are blind spots in the user's field of vision.

[0074] Exemplarily, assume that after user A wears the goggles 5, the area of the lens 52 that can be clearly seen is shown as the area 521 surrounded by the dotted line in the figure (this area is called the visible area of user A) (obviously, there is usually no such dotted line loop on the actual goggles, and this dotted line loop is a virtual boundary for schematically illustrating the boundary between the blind spot and the visible area), and other areas on the lens 52 except the area 521 are blind spots of user A; then the display module 2 can only be clearly seen by user A when it is located within the area 521 and forms an image within the area 521, and the user can also clearly see the image formed by the display module 2 within the area 521 through the display area 21 of the display module 2; and if the display module 2 is in the blind spot on the lens 52 (such as Figure 7 the position shown in), user A cannot even clearly see the display module 2, of course, cannot clearly see the display area 21 of the display module 2, and naturally cannot view the image formed by the display module 2 on the lens 52 by viewing the display area 21 of the display module 2.

[0075] Therefore, in the related art, for some glasses (such as Figure 7 the goggles shown in), due to protection or styling considerations, the size of the lens of the glasses may be much larger than the field of view of the human eye, so that the user wearing the glasses cannot see or clearly see some areas on the lens, that is, there may be some areas (i.e., blind spots) on the lens of the glasses outside the field of vision of the user wearing the glasses; if the display module connected to the lens of the glasses forms an image in the blind spot on the lens, it will cause the user wearing the glasses to be unable to see or clearly see.

[0076] The embodiments of the present application provide a control method, device, electronic device, near-eye display device, medium, chip and computer program product for a display module, which can independently or semi-independently solve to a certain extent the technical problem that if the display module connected to the lens of the glasses forms an image in the blind spot on the lens, the user wearing the glasses will be unable to see or clearly see.

[0077] On this basis, the embodiments of the present application will be specifically described below.

[0078] An embodiment of the present application provides a control method for a display module, where the display module is configured to be placed on an optical lens, and the display module is configured to output image content. Exemplarily, as Figures 1 to 7 shown, the display module may be Figures 1 to 7 the display module 2 shown in Figures 1 to 6 which can output image content on the lens 12 or the lens 52. The one that executes the control method of the display module may be Figures 1 to 7 the glasses 1 in Figure 5 the display module 2 in Figure 8 the near-eye display device 10 in Figure 8 or at least one of the smart terminals 4, which can control the specific information of the GUI imaged by the display module 2 on the glasses 1 (or the near-eye display device 10 or the goggles 5) to which it is connected. As Figure 8 shown, Figure 8 FIG. is a schematic diagram of the step flow of the control method of the display module provided by the embodiment of the present application. To facilitate understanding of the control method of the display module provided by the embodiment of the present application, the step flow shown in Figure 7 will be described below by taking the goggles 5 exemplified in Figure 8 as an example. Please refer to Figure 8 , the control method of the display module includes:

[0079] S11, detecting the relative position of the display module on the currently connected lens and the contact of the touch part of the display module.

[0080] The lens is the lens on the glasses to which the display module is currently connected. Exemplarily, it may be the same as that described in the above embodiment, which will not be elaborated here.

[0081] In the embodiment of the present application, the display module for enhancing the augmented reality display imaging on the lens of the glasses can move on the lens to which it is connected, rather than fixing the display module on the lens. In some alternative embodiments, an inertial sensor or an acceleration sensor (or a gyroscope) is built into the display module, and the instantaneous speed and acceleration of the display module can be collected through such sensors, so as to determine the movement trajectory of the display module according to the collected instantaneous speed and acceleration and combined with parameters such as time.

[0082] Optionally, a fixed reference point is selected in advance on the lens to which the display module is connected as the origin, and the display module starts to move from this origin and always monitors the movement trajectory of the display module. The position of any point on this movement trajectory relative to the origin can be expressed as the relative position of the display module relative to the connected lens.

[0083] Exemplarily, as Figure 9 shown, Figure 9It is a schematic diagram of an application scenario of a display module provided in an embodiment of the present application. For example, point P0 on the lens 52 is a pre-selected origin, and the display module 2 moves from the origin along path l1 to point P1, then moves along path l2 to point P2, then moves along l3 to point P3, and finally moves along l4 to point P4. The motion trajectory l of the display module 2 at any time t in the motion process can be determined by monitoring the motion parameters of the display module 2 (such as speed, acceleration, etc.), and the position relationship of the end point of the motion trajectory at any time t relative to the origin P0 can represent the relative position of the display module 2 on the lens 52. For example, at time t1, the display module 2 moves to point P1. According to the determined path l1, the position relationship of point P1 relative to the origin P0 can be determined, that is, the relative position of the display module 2 on the lens 52, which is exemplarily represented as Indicates that point P1 is located at the vector The end point of the vector The starting point is P0.

[0084] Similarly, when the motion trajectory of the display module 2 from point P1 to point P2 is l2, it is obvious that the position relationship of P2 relative to P1 can be expressed as Indicates that point P2 is located at the vector The end point of the vector The starting point is P1. Based on the mathematical vector relationship, Therefore, there is Accordingly, Obviously, the above vector actually shows the displacement of module 2 during this process.

[0085] It should be noted that in the above examples, a straight line trajectory (path) is used as an example to describe the path and movement trajectory of the display module 2. Obviously, the display module 2 can be moved arbitrarily and is not limited to the form of the straight line trajectory in the above examples, such as a curved trajectory.

[0086] Regarding the detection of contact of the touch portion of the display module, please refer to the relevant description in the previous text, which will not be repeated here.

[0087] S12, in the imaging area setting mode, when it is detected that the display module is stationary at a first relative position on the lens and the touch portion is touched according to a first touch pattern, the first relative position is marked as a first fitting point. The first relative position is an arbitrary position of the display module on the lens. In the imaging area setting mode, the first touch pattern is a touch pattern used to indicate that the user confirms that the display area of ​​the display module can be clearly seen.

[0088] The imaging area setting mode is a logical function provided for users to set a preset imaging area on the lens connected to the display module. In some alternative embodiments, when the display module is first connected to the lens and powered on, it will automatically enter the imaging area setting mode. In some alternative embodiments, an APP for controlling the display module is provided on the smart terminal connected to the display module, and the user can start the imaging area setting mode through the imaging area setting function option in the APP. After the display module enters this mode, it will enter the preset imaging area setting process.

[0089] Exemplarily, Figure 5 the near-eye display device 10 shown in Figure 9 the goggles 5 shown in, when the display module 2 is first connected to the lens 52 and powered on, it automatically enters the imaging area setting mode. The user can also start the imaging area setting mode of the display module 2 through the imaging area setting function option in the APP for controlling the display module on the smart terminal 4.

[0090] When it is detected that the instantaneous speed of the display module is 0, it can be confirmed that the display module is stationary on the lens.

[0091] Exemplarily, in Figure 9 the example in, it can be initially detected that the display module 2 is stationary at point P0, and then it can be detected again that the display module 2 is stationary at point P1. Subsequently, it will be successively detected that the display module is stationary at points P2, P3, and P4. That is, P1, P2, P3, and P4 are all the first relative positions.

[0092] The first touch mode can be a preset touch mode, for example, a single click, double click, or triple click on the touch part of the display module, etc. When it is detected that the touch part of the display module is touched at a certain position on the lens according to the first touch mode, the position where the display module is located is marked as the first fitting point. The fitting point is a boundary point used to fit the boundary of the preset imaging area.

[0093] Exemplarily, in Figure 9 the example in, when it is detected that the display module 2 moves from point P0 to points P1, P2, P3, and P4, and it is also detected that the touch part is touched according to the first touch mode (such as a single click) when it is stationary at points P1, P2, P3, and P4, therefore, points P1, P2, P3, and P4 are all marked as the first fitting points.

[0094] S13. In the case of marking at least three first fitting points, fit the at least three first fitting points into a closed area.

[0095] S14. Set the closed area as the preset imaging area of the display module on the lens.

[0096] Based on common mathematical knowledge, at least three points are required in space to enclose a closed area. Therefore, after marking at least three first fitting points, all the marked first fitting points can be fitted into a closed area. And the closed area fitted according to all the first fitting points is set as the preset imaging area where the display module forms an image on the currently connected lens.

[0097] Fitting means enclosing multiple points into a closed area, and the connection line between any two points does not cross the connection line between any other two points. In some alternative embodiments, at least three first fitting points can be fitted into a closed area of a preset shape. For example, the preset shape can be an ellipse (a circle is a special ellipse), a rectangle, a triangle, etc. In some alternative embodiments, when performing area fitting, between any two first fitting points with the shortest straight-line distance, it can be connected with a curve or a straight line. When connecting with a curve, a smooth curve is preferably used.

[0098] Exemplarily, as Figure 10 and Figure 11 shown, Figure 10 and Figure 11 are both schematic diagrams of an application scenario of a display module provided by an embodiment of the present application. Please refer to Figure 10 , four first fitting points are fitted into an elliptical area, and the elliptical area is set as the preset imaging area 522 of the display module 2 on the lens 52. Please refer to Figure 11 , four first fitting points P1, P2, P3, and P4 are fitted into a quadrilateral area, and the quadrilateral area is set as the preset imaging area 522 of the display module 2 on the lens 52.

[0099] In some alternative embodiments, to facilitate the user to set the preset imaging area, the above steps S11 - S14 can be assisted in setting through an intelligent terminal communicatively connected to the display module (please refer to Figure 5 ), and the following is an exemplary description.

[0100] As Figures 12 to 13 shown, Figures 12 to 13 are both schematic diagrams of an application scenario of a display module provided by an embodiment of the present application. Please refer to Figures 12 to 13, the display module 2 is in communication connection with the smart terminal 6, and the smart terminal 6 has pre-acquired the virtual contour model of the goggles 5. This virtual contour model can be obtained by scanning the goggles 5 with a 3D scanner or from the manufacturer of the goggles 5. After connecting (placing) the display module 2 on the lens 52, when the display module 2 starts the imaging area setting mode, the virtual contour model of the goggles 5 is correspondingly output on the screen of the smart terminal 6, and the relative position of the display module 2 on the goggles 5 is traced on the virtual contour model of the goggles 5. The display module 2 can be moved to different positions on the connected lens 52 for positioning, and each positioning point (i.e., fitting point) is determined by the user wearing the glasses 5 to confirm that they can see clearly. That is, the user places the display module 2 at different positions on the lens 52 for "point aiming" positioning, shows these positioning points on the smart terminal 6, and then fits all the positioning points into a preset imaging area.

[0101] Please refer to Figure 12 , when the display module 2 first enters the imaging area setting mode, the initial position P0 of the display module 2 is shown on the virtual contour model of the goggles 5 on the screen of the smart terminal 6. When the display module 2 moves from P0 to P1, the position P1 is shown on the virtual contour model of the goggles 5 on the screen of the smart terminal 6. If the user confirms that they can see clearly when the display module 2 is at point P1, the position of P1 relative to the origin P0 is recorded. Please refer to Figure 13 , after completing the "point aiming" positioning, all the found positioning points are shown on the virtual contour model of the goggles 5 on the screen of the smart terminal 6, such as Figure 13 P1, P2, P3 and P4 exemplified in Figure 14 , an elliptical closed area fitted is shown on the virtual contour model of the goggles 5 on the screen of the smart terminal 6, and this closed area is set as the preset imaging area 522 for the display module 2 to image on the lens 52.

[0102] In some optional embodiments, when selecting the first relative position, the limit position points on the lens close to the frame are preferably selected, such as the position points in the edge area close to the frame on the lens. If the user confirms that they cannot see clearly at these position points, the user is prompted to gradually move the display module closer to the center position point of the lens, so that the finally fitted preset imaging area is as large as possible to increase the area of the preset imaging area. Obviously, the more the data volume of the calibrated first fitting points, the closer the fitted preset imaging area can be to the user's visible area. Therefore, the number of the first fitting points for fitting the preset imaging area can be flexibly set.

[0103] In an embodiment of the present application, in the imaging area setting mode, by detecting the relative position of the display module on the currently connected lens and the contact of the touch part of the display module, at least three first fitting points that can be clearly seen by the user are found on the lens currently connected to the display module, and then the at least three found fitting points are fitted into a closed area. Since the boundary points on this closed area (obviously these boundary points include the first fitting points) can all be clearly seen by the user, naturally the user can also clearly see this closed area, and this closed area is set as the preset imaging area of the display module on the lens. After that, the display module will image within this preset imaging area on the lens, rather than imaging within the blind area on the lens, thus effectively solving the technical problem that the display module connected to the lens of the glasses imaging within the blind area on the lens will cause the user wearing the glasses to be unable to see or clearly see the virtual information within the blind area.

[0104] An embodiment of the present application provides a control method for a display module, the display module is configured to be placed on an optical lens, and the display module is configured to output image content. Exemplarily, as Figures 1 to 7 shown, the display module can be Figures 1 to 7 the display module 2 shown in, and the display module 2 can output image content on the lens 12 or the lens 52. The one that can execute the control method of the display module can be Figures 1 to 6 the glasses 1 in, Figures 1 to 7 the display module 2 in, Figure 5 the near-eye display device 10 or the smart terminal 4 in at least one of them, which can realize controlling the specific information of the GUI imaged by the display module 2 on the glasses 1 (or the near-eye display device 10 or the goggles 5) it is connected to. As Figure 9 shown, Figure 9 is a schematic diagram of the step flow of the control method of the display module provided by an embodiment of the present application. To facilitate understanding of the control method of the display module provided by an embodiment of the present application, the following will take Figure 7 the goggles 5 exemplified in and Figure 16 the scenario shown in as an example to illustrate Figure 9 the step flow shown in. Please refer to Figure 9 , the control method of this display module includes:

[0105] S21, obtaining a first virtual contour model of the lens and a second virtual contour model of the user's eyes;

[0106] S22, scanning out a contour area on the first virtual contour model that is the same as or similar to the second virtual contour model, and setting the contour area as the preset imaging area of the display module on the lens.

[0107] Please refer to Figure 16For the description of how the smart terminal 6 obtains the first virtual contour model 5' of the lens, please refer to the foregoing text and will not be elaborated here. Similarly, a 3D scanner can also be used to obtain a second virtual contour model of the eyes of user A wearing the goggles 5, and the smart terminal 6 obtains the second virtual contour model from the 3D scanner. The smart terminal 6 overlays and presents the second virtual contour model with the first virtual contour model 5', scans out a contour area 522' on the first virtual contour model 5' that is the same as or similar to the second virtual contour model, and sets the contour area 522' as the preset imaging area 522 on the lens 52 of the display module 2.

[0108] Compared with the foregoing method of setting the preset imaging area, the setting process of this method of setting the preset imaging area is faster, but the difficulty lies in that for each pair of glasses, each user wearing the glasses needs to obtain the virtual contour of the lens of the glasses and the virtual contour of the eyes of the user at the same time. Therefore, in practice, users can flexibly select the setting method of the preset imaging area according to the actual situation.

[0109] After the preset imaging area is set, when the display module is within the preset imaging area on the connected lens, it can image within the preset imaging area on the connected lens. And when the display module is outside the preset imaging area on the connected lens, it will not image on the connected lens. Therefore, in some alternative embodiments, when it is detected that the display module is outside the preset imaging area, the low-power mode is activated.

[0110] Exemplarily, in Figure 11 , after the preset imaging area is set, when the display module 2 moves to an area 522 outside the lens 52, the display module 2 activates the low-power mode. Thereby saving the electric energy of the device and prolonging the battery life and service life of the device. In the low-power mode, the display module 2 can be in a state of not working at all, or in a standby state, or in a state where some modules are working. And when the display module 2 moves back into the area 522 again, the display module 2 turns off the low-power mode and resumes working. In some alternative embodiments, the user can also set the specific states corresponding to the low-power mode for different time periods in the non-preset imaging area (in the example in Figure 11 , that is, the area outside the area 522 on the lens 52), for example, setting the low-power mode from 11 pm to 7 am the next day to a state of not working at all, setting the low-power mode from 7 am to 9 am, from 12 noon to 2 pm, and from 5 pm to 8 pm to a standby state, and setting the remaining time periods to a state where some modules are working (for example, the hardware for monitoring the movement state of the display module is working).

[0111] In some alternative embodiments, on the basis of the foregoing embodiments, the control method of the display module provided in the embodiments of the present application further includes:

[0112] Upon receiving a partitioning instruction, divide the preset imaging area into at least two sub-imaging areas;

[0113] Upon receiving a mapping instruction to image a first graphical interface within a first sub-imaging area, control the display module to image the first graphical interface within the first sub-imaging area; the first sub-imaging area is any one of the at least two sub-imaging areas.

[0114] The user can divide the preset imaging area into multiple sub-imaging areas by themselves, and different images can be presented in different sub-imaging areas, and the images presented in each sub-area can also be set by the user themselves. When the display module receives a partitioning instruction input by the user, the display module responds to the number of areas in the partitioning instruction and divides the preset imaging area into sub-imaging areas of that number of areas (exemplarily, as Figure 17 shown, the preset imaging area 522 is divided into 6 sub-imaging areas). When the display module receives a mapping instruction for a certain sub-imaging area, according to the data mapped in the mapping instruction, it images a graphical interface corresponding to the data mapped in the mapping instruction in that sub-imaging area.

[0115] Exemplarily, as Figure 17 shown, Figure 17 is a schematic diagram of an application scenario of a display module provided by an embodiment of the present application. Please refer to Figure 17 , the user sets a preset imaging area 522 for the display module 2 on the lens 52. The preset imaging area 522 is divided into 6 sub-imaging areas, and graphical interfaces of the data mapped in each sub-imaging area are imaged in each sub-imaging area. Figure 17 The icons A-I shown in

[0116] can be icons (thumbnails) of application programs or files, or other content, such as a video being played, etc.

[0117] Obviously, for a specific pair of glasses, due to the differences in the users wearing them, the blind spots and visible areas are different for each user. And for a specific user, due to the differences in the glasses they wear, the blind spots and visible areas are different for each pair of glasses. That is to say, the method for setting the preset imaging area provided in the foregoing embodiments needs to vary from person to person and from pair of glasses to pair of glasses. In other words, the preset imaging area A set by user A when wearing glasses A may not be suitable for user B when wearing glasses A. User B may not be able to see some areas within the preset imaging area A when wearing glasses A, and user B may need to reset the preset imaging area B when wearing glasses A. Therefore, in some alternative embodiments, for the lens currently connected to the display module, the identity information of the user who sets the preset imaging area can be associated with the preset imaging area set by this user, so that when different users wear glasses later, the display module can perform imaging based on the preset imaging area associated with this user. Therefore, in some alternative embodiments, on the basis of the foregoing embodiments, the control method of the display module provided in the embodiments of the present application further includes:

[0118] Obtain the identity information of the user;

[0119] Establish a mapping rule between the identity information and the preset imaging area and save it.

[0120] The user here is the user who sets the preset imaging area on the lens currently connected to the display module. It can be the user who touches the touch part of the display module according to the first touch mode in the foregoing step S12, or the user corresponding to the second virtual contour model in the foregoing step S21.

[0121] The identity information of the user can be the user's account / password, voiceprint, fingerprint, iris, etc.

[0122] After the user completes the setting of the preset imaging area while wearing a pair of glasses, a mapping rule can be established between the identity information of this user and the preset imaging area set by this user, and this mapping rule is saved. When the user uses this pair of glasses next time, the preset imaging area set by this user on this pair of glasses can be directly obtained according to this mapping rule.

[0123] Exemplarily, for the display module 2, for the preset imaging area A1 set by user A on the goggles 5, the identity information ID-A of user A is mapped to the preset imaging area A1. For the preset imaging area A2 set by user A on the glasses 1, the identity information ID-A of user A is mapped to the preset imaging area A2. For the display module 2, for the preset imaging area B1 set by user B on the goggles 5, the identity information ID-B of user B is mapped to the preset imaging area B1. And these mapping rules are saved.

[0124] As Figure 18 shownFigure 18 It is a schematic diagram of the step flow of the control method for the display module provided by the embodiment of the present application. Please refer to Figure 18 , the embodiment of the present application provides a control method for a display module. On the basis of the foregoing embodiment, it further includes:

[0125] S15, when the imaging area setting mode is exited and then restarted, and it is detected that the display module is stationary at the second relative position on the lens and the touch part is touched according to the first touch mode, mark the second relative position as the second fitting point; the second relative position is any position on the lens different from the first relative position;

[0126] S16, when at least one second fitting point is marked, fit the at least three first fitting points and the at least one second fitting point into a first new closed area, and update and set the first new closed area as the preset imaging area of the display module on the lens; or

[0127] S17, when at least three second fitting points are marked, fit the at least three second fitting points into a second new closed area, and update and set the second new closed area as the preset imaging area of the display module on the lens.

[0128] After the user sets the preset imaging area, the user may feel that the set preset imaging area is not ideal and wants to locally update or completely reset the originally set preset imaging area. Therefore, after setting the preset imaging area, the user can re-enter the imaging area setting mode to update or reset the preset imaging area. That is, when the user re-enters the imaging area setting mode, the user can perform "point aiming" positioning on the lens again to find some positioning points different from the first fitting points (i.e., the second fitting points), add the second fitting points to the first fitting points to re-fit the preset imaging area (i.e., step S16) to update the preset imaging area, or find at least three second fitting points and re-fit a new preset imaging area based on the at least three second fitting points (i.e., step S17) to reset the preset imaging area.

[0129] As Figure 19 shown, Figure 19 It is a schematic diagram of the step flow of the control method for the display module provided by the embodiment of the present application. Please refer to Figure 19 , the embodiment of the present application provides a control method for a display module. On the basis of the foregoing embodiment, it further includes:

[0130] S31, identify the first identity of the first user wearing the first pair of glasses, where the first pair of glasses includes the lens;

[0131] S32. Search for a mapping rule that matches the first identity in the preset mapping set;

[0132] S33. In the case where exactly one mapping rule is found in the mapping set, set the preset imaging area in the found mapping rule as the imaging area where the display module forms an image on the lens; Each mapping rule in the mapping set is a pre-set mapping relationship between a user identity and a preset imaging area, and the mapping rules in the mapping set are all different.

[0133] The first user can be any user.

[0134] In some alternative embodiments, the first identity of the first user wearing the first pair of glasses can be identified by obtaining the account / password and fingerprint information of the user logged in through an intelligent terminal communicatively connected to the display module, or the iris information of the first user can be collected by a camera on the display module to identify the identity information of the first user.

[0135] The display module can be connected to any pair of glasses, so the mapping set is associated with each pair of glasses. For example, mapping set A can be configured for glasses A, and at least one mapping rule for the preset imaging area set when each user wears glasses A is recorded in mapping set A. Mapping set B can be configured for glasses B, and at least one mapping rule for the preset imaging area set when each user wears glasses B is recorded in mapping set B.

[0136] After identifying the first identity of the first user, a mapping rule that matches the first identity can be searched for in the mapping set of the first pair of glasses. In the case where exactly one mapping rule is found in the mapping set of the first pair of glasses, set the preset imaging area in the found mapping rule as the imaging area where the display module forms an image on the lens.

[0137] As Figure 20 shown, Figure 20 is a schematic diagram of the step flow of the control method of the display module provided by the embodiment of the present application. Please refer to Figure 20 , the embodiment of the present application provides a control method of a display module. On the basis of the embodiment shown in Figure 19 it further includes:

[0138] S34. When at least two mapping rules are found in the mapping set, detect selection instructions for the preset imaging areas corresponding to the at least two mapping rules, and set the preset imaging area selected by the selection instruction as the imaging area where the display module images on the lens, or set the first preset imaging area among the at least two mapping rules as the imaging area where the display module images on the lens. The first preset imaging area is the preset imaging area that was last selected in the historical selection record of the first user's selection of the preset imaging area.

[0139] If at least two mapping rules are found in the mapping set of the first pair of glasses, it indicates that the first user has set multiple preset imaging areas on the first pair of glasses. These preset imaging areas can be shown on the smart terminal communicatively connected to the display module for the first user to select, and the preset imaging area selected by the user input's selection instruction is set as the imaging area where the display module images on the lens. Or alternatively, the preset imaging area that was last selected in the historical selection record of the first user's selection of the preset imaging area can be set as the imaging area where the display module images on the lens.

[0140] As Figure 21 shown, Figure 21 is a schematic diagram of the step flow of the control method for the display module provided by the embodiment of the present application. Please refer to Figure 21 , the embodiment of the present application provides a control method for a display module. On the basis of the embodiment shown in Figure 20 , it further includes:

[0141] S35. When no mapping rule is found in the mapping set, start the imaging area setting mode;

[0142] S36. When it is detected in the imaging area setting mode that the touch part is touched in the first touch mode at the third relative position where the display module is stationary on the lens, mark the third relative position as the third fitting point; the third relative position is an arbitrary position of the display module on the lens.

[0143] S37. When at least three third fitting points are marked, fit the at least three third fitting points into a third closed area;

[0144] S38. Set the third closed area as the third preset imaging area of the display module on the lens.

[0145] In the case where no mapping rule is found in the mapping set, it indicates that the first user has not set a preset imaging area on the first pair of glasses. Therefore, the imaging area setting mode is started so that the first user can set a preset imaging area for the display module on the first pair of glasses. The specific process of the first user finding the third fitting point on the first lens and fitting the found third fitting point into the third preset imaging area can refer to the description of setting the preset imaging area in the previous text and will not be elaborated here.

[0146] In some alternative embodiments, based on the embodiment shown above Figure 20 the control method of the display module further includes: establishing a third mapping rule between the first identity and the third preset imaging area, and classifying the third mapping rule into the mapping set.

[0147] Establish a third mapping rule between the first identity of the first user and the third preset imaging area and classify it into the mapping set of the first pair of glasses, so that the first user can quickly determine the preset imaging area corresponding to the first user's use of the first pair of glasses the next time the first user uses the first pair of glasses.

[0148] As Figure 22 shown, Figure 22 is a schematic diagram of the step flow of the control method of the display module provided by the embodiment of the present application. Please refer to Figure 22 , the embodiment of the present application provides a control method of a display module. On the basis of the foregoing embodiments, it further includes:

[0149] S41, in the case where the imaging area setting mode is started again after exiting the imaging area setting mode, and it is detected that the touch part is touched in the first touch mode at the fourth relative position where the display module is stationary on the lens, mark the fourth relative position as the fourth fitting point; the fourth relative position is an arbitrary position of the display module on the lens;

[0150] S42, in the case where at least three fourth fitting points are marked, determine the repetition rate of the at least three fourth fitting points and the at least three first fitting points;

[0151] S43, when it is determined that the repetition rate is not less than the preset threshold, set the preset imaging area as the current imaging area where the display module images on the lens.

[0152] In some alternative embodiments, the user may have forgotten that they have previously set a preset imaging area for the currently worn glasses. When the display module detects that it has entered the preset imaging area setting mode again, and the repetition rate of the relative positions of the detected fourth fitting points and the first fitting points on the lens reaches a preset threshold (e.g., 90%), the setting of the preset imaging area is abandoned, and the preset imaging area fitted based on the first fitting points is directly configured as the current imaging area where the display module forms an image on the lens.

[0153] In some alternative embodiments, based on the embodiment shown above Figure 21 The control method of the display module further includes: when determining that the repetition rate is less than the preset threshold, fitting the at least three fourth fitting points into a fourth closed area, and setting the fourth closed area as the current imaging area where the display module forms an image on the lens.

[0154] It should be noted that although the above examples are all described with a monocular (i.e., the display module provided by the embodiments of the present application is only connected to one lens of the glasses) as an example, obviously, all the embodiments provided by the present application can also be applied to a binocular scenario (i.e., the display module provided by the embodiments of the present application is connected to both lenses of the glasses).

[0155] The control method of the display module provided by the embodiments of the present application may be executed by a control device of the display module. In the embodiments of the present application, taking the control device of the display module executing the control method of the display module as an example, the control device of the display module provided by the embodiments of the present application is described.

[0156] As Figure 23 shown, a schematic structural diagram of a control device of a display module provided by an embodiment of the present application is shown. Please refer to Figure 23 , for the control device 7 of the display module, the display module is configured to be placed on an optical lens, and the display module is configured to output image content. The control device 7 includes:

[0157] A first detection module 71, configured to detect the relative position of the display module on the currently connected lens and the contact of the touch part of the display module;

[0158] A first marking module 72, configured to mark the first relative position as a first fitting point when it is detected in the imaging area setting mode that the touch part is touched in a first touch mode at the first relative position where the display module is stationary on the lens; the first relative position is an arbitrary position of the display module on the lens, and in the imaging area setting mode, the first touch mode is a touch mode used to indicate that the user confirms that they can clearly see the display area of the display module;

[0159] The first fitting module 73 is configured to fit the at least three first fitting points into a closed area when at least three first fitting points are marked.

[0160] The first setting module 74 is configured to set the closed area as the preset imaging area of the display module on the lens.

[0161] In some alternative embodiments, the control device 7 further includes:

[0162] When it is detected that the display module is located outside the preset imaging area, a low-power mode is started.

[0163] In some alternative embodiments, the control device 7 further includes:

[0164] The first division module is configured to divide the preset imaging area into at least two sub-imaging areas when a partitioning instruction is received.

[0165] The first imaging module is configured to control the display module to image the first graphical interface in the first sub-imaging area when a mapping instruction to image the first graphical interface in the first sub-imaging area is received; the first sub-imaging area is any one of the at least two sub-imaging areas.

[0166] In some alternative embodiments, the control device 7 further includes:

[0167] The second marking module is configured to mark the second relative position as a second fitting point when the imaging area setting mode is exited and then restarted, and it is detected that the display module is stationary at the second relative position on the lens and the touch part is touched in the first touch mode; the second relative position is any position on the lens different from the first relative position.

[0168] The second fitting module is configured to fit the at least three first fitting points and the at least one second fitting point into a first new closed area when at least one second fitting point is marked, and update and set the first new closed area as the preset imaging area of the display module on the lens; or fit the at least three second fitting points into a second new closed area when at least three second fitting points are marked, and update and set the second new closed area as the preset imaging area of the display module on the lens.

[0169] In some alternative embodiments, the control device 7 further includes:

[0170] A first recognition module, configured to recognize the first identity of a first user wearing a first pair of glasses, where the first pair of glasses includes the lens;

[0171] A first search module, configured to search for a mapping rule that matches the first identity in a preset mapping set;

[0172] A second setting module, configured to, when exactly one mapping rule is searched from the mapping set, set a preset imaging area in the searched mapping rule as the imaging area where the display module forms an image on the lens; each mapping rule in the mapping set is a pre-set mapping relationship between a user identity and a preset imaging area, and the various mapping rules in the mapping set are different from each other.

[0173] In some alternative embodiments, the control device 7 further includes:

[0174] A third setting module, configured to, when at least two mapping rules are searched from the mapping set, detect a selection instruction for the preset imaging areas in the at least two mapping rules respectively, and set the preset imaging area selected by the selection instruction as the imaging area where the display module forms an image on the lens, or set a first preset imaging area in the at least two mapping rules as the imaging area where the display module forms an image on the lens, where the first preset imaging area is the preset imaging area that was last selected in the historical selection record of the first user's selection of the preset imaging area.

[0175] In some alternative embodiments, the control device 7 further includes:

[0176] A first activation module, configured to activate the imaging area setting mode when no mapping rule is searched from the mapping set;

[0177] A third marking module, configured to, when it is detected in the imaging area setting mode that the touch part is touched in a first touch mode at a third relative position where the display module is stationary on the lens, mark the third relative position as a third fitting point; the third relative position is an arbitrary position of the display module on the lens;

[0178] A third fitting module, configured to, when at least three third fitting points are marked, fit the at least three third fitting points into a third closed area;

[0179] A fourth setting module, configured to set the third closed area as the third preset imaging area of the display module on the lens.

[0180] In some alternative embodiments, the control device 7 further includes:

[0181] A first establishing module, configured to establish a third mapping rule between the first identity and the third preset imaging area, and classify the third mapping rule into the mapping set.

[0182] In some alternative embodiments, the control device 7 further includes:

[0183] A fourth marking module, configured to, when the imaging area setting mode is exited and then restarted, and when it is detected that the touch part is touched in a first touch mode at a fourth relative position where the display module is stationary on the lens, mark the fourth relative position as a fourth fitting point; the fourth relative position is an arbitrary position of the display module on the lens.

[0184] A fourth fitting module, configured to determine the repetition rate between the at least three fourth fitting points and the at least three first fitting points when at least three fourth fitting points are marked.

[0185] A fifth setting module, configured to, when it is determined that the repetition rate is not less than a preset threshold, set the preset imaging area as the current imaging area where the display module images on the lens.

[0186] In some alternative embodiments, the control device 7 further includes:

[0187] A sixth setting module, configured to, when it is determined that the repetition rate is less than the preset threshold, fit the at least three fourth fitting points into a fourth closed area, and set the fourth closed area as the current imaging area where the display module images on the lens.

[0188] In some alternative embodiments, the control device 7 further includes:

[0189] A first obtaining module, configured to obtain a first virtual contour model of the lens and a second virtual contour model of the user's eye.

[0190] A seventh setting module, configured to scan a contour area on the first virtual contour model that is the same as or similar to the second virtual contour model, and set the contour area as the preset imaging area where the display module images on the lens.

[0191] In some alternative embodiments, the control device 7 further includes:

[0192] A second obtaining module, configured to obtain the identity information of the user.

[0193] A second establishing module, configured to establish a mapping rule between the identity information and the preset imaging area and save it.

[0194] The control device 7 of the display module in the embodiments of the present application may be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than terminals. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a handheld computer, an in-vehicle electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., or may also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0195] The control device 7 of the display module in the embodiments of the present application may be a device with an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems. The embodiments of the present application do not make specific limitations.

[0196] The control device 7 of the display module provided by the embodiments of the present application can implement Figures 1 to 22 each process implemented by the embodiments described above. To avoid repetition, details are not described here again.

[0197] In some optional implementation manners, as Figure 24 shown, the embodiments of the present application further provide an electronic device 130, including a processor 131 and a memory 132. A program or instruction that can run on the processor 131 is stored on the memory 132. When the program or instruction is executed by the processor 131, each step of the embodiments of the above display module control method is implemented, and the same technical effects can be achieved. To avoid repetition, details are not described here again.

[0198] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0199] Figure 25 Schematic diagram of the hardware structure of an electronic device for implementing the embodiments of the present application.

[0200] The electronic device 140 includes, but is not limited to, components such as a radio frequency unit 141, a network module 142, an audio output unit 143, an input unit 144, a sensor 145, a display unit 146, a user input unit 147, an interface unit 148, a memory 149, and a processor 1410. Those skilled in the art can understand that the electronic device 140 may further include a power source (such as a battery) for supplying power to each component. The power source can be logically connected to the processor 1410 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 14 The structure of the electronic device shown does not limit the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0201] Among them, the processor 1410 is used for:

[0202] Detect the relative position of the display module on the currently connected lens and the contact of the touch part of the display module;

[0203] When it is detected that the touch part is touched in a first touch mode at a first relative position where the display module is stationary on the lens in the imaging area setting mode, mark the first relative position as a first fitting point; the first relative position is an arbitrary position of the display module on the lens, and in the imaging area setting mode, the first touch mode is a touch mode used to indicate that the user confirms that they can clearly see the display area of the display module;

[0204] When at least three first fitting points are marked, fit the at least three first fitting points into a closed area;

[0205] Set the closed area as the preset imaging area of the display module on the lens.

[0206] It should be understood that in the embodiments of the present application, the input unit 144 may include a Graphics Processing Unit (GPU) 1441 and a microphone 1442. The graphics processor 1441 processes the image data of static pictures or videos obtained by an image capturing device (such as a camera) in the video capture mode or the image capture mode. The display unit 146 may include a display panel 1461, and the display panel 1461 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 147 includes at least one of a touch panel 1471 and other input devices 1472. The touch panel 1471 is also referred to as a touch screen. The touch panel 1471 may include two parts: a touch detection device and a touch controller. The other input devices 1472 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0207] The memory 149 can be used to store software programs and various data. The memory 149 mainly includes a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area can store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 149 may include a volatile memory or a non-volatile memory, or the memory 149 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash memory. The volatile memory may be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 149 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memories.

[0208] The processor 1410 may include one or more processing units; optionally, the processor 1410 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor may not be integrated into the processor 1410 either.

[0209] In the control method of the display module provided by the embodiments of the present application, the execution subject may be an AR glasses. In the embodiments of the present application, taking the near-eye display device executing the control method of the display module as an example, the electronic system provided by the embodiments of the present application is described.

[0210] In some alternative embodiments, as Figure 26 shown, the embodiments of the present application further provide a near-eye display device 150, including a processor 151 and a memory 152. A program or instruction that can run on the processor 151 is stored on the memory 152. When the program or instruction is executed by the processor 131, each step of the embodiment of the above-mentioned control method of the display module is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described in detail here.

[0211] Any of the above product embodiments can implement each process of the embodiment of the above-mentioned control method of the display module through the operation of its own processor, and the same technical effect can be achieved. To avoid repetition, it will not be described one by one.

[0212] The embodiments of the present application further provide a readable storage medium. A program or instruction is stored on the readable storage medium. When the program or instruction is executed by a processor, each process of the embodiment of the above-mentioned control method of the display module is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described in detail here. Among them, the processor is the processor in the electronic device or electronic system described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disk or optical disc, etc.

[0213] The embodiments of the present application further provide a chip. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run a program or instruction to implement each process of the embodiment of the above-mentioned control method of the display module, and the same technical effect can be achieved. To avoid repetition, it will not be described in detail here.

[0214] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0215] An embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement each process of the control method embodiment of the display module as described above, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0216] In the implementation manners provided in the embodiments of the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device implementation manners described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, indirect couplings or communication connections of devices or units, and can be in electrical, mechanical, or other forms.

[0217] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0218] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0219] The above description is only the implementation manner of the embodiments of the present application, and does not thus limit the patent scope of the embodiments of the present application. The above specific implementation manner is merely illustrative rather than restrictive. Under the inspiration of the present application, those of ordinary skill in the art, all equivalent structural or equivalent process transformations made by using the description of the embodiments of the present application and the content of the drawings, or directly or indirectly applied in other related technical fields, without departing from the purpose of the present application and the scope protected by the claims, can still make many forms, and all are equally included in the patent protection scope of the embodiments of the present application.

Claims

1. A control method for a display module, characterized in that The display module is configured to be placed on an optical lens, and the display module is configured to output image content. The method includes: Detecting a relative position of the display module on the currently connected lens and a contact of a touch part of the display module; When it is detected in the imaging area setting mode that the display module is stationary at a first relative position on the lens and the touch part is touched in a first touch mode, marking the first relative position as a first fitting point; the first relative position is an arbitrary position of the display module on the lens, and in the imaging area setting mode, the first touch mode is a touch mode for indicating that the user confirms that they can clearly see a display area of the display module; When at least three first fitting points are marked, fitting the at least three first fitting points into a closed area; Setting the closed area as a preset imaging area of the display module on the lens.

2. The method according to claim 1, wherein After setting the closed area as the preset imaging area of the display module on the lens, the method further includes: When it is detected that the display module is located outside the preset imaging area, starting a low power consumption mode.

3. The method according to claim 1, wherein The method further includes: When a partitioning instruction is received, partitioning the preset imaging area into at least two sub-imaging areas; When a mapping instruction for imaging a first graphical interface in a first sub-imaging area is received, controlling the display module to image the first graphical interface in the first sub-imaging area; the first sub-imaging area is any one of the at least two sub-imaging areas.

4. The method according to claim 1, characterized in that The method further includes: When it is detected in the imaging area setting mode again after exiting the imaging area setting mode and the display module is stationary at a second relative position on the lens and the touch part is touched in the first touch mode, marking the second relative position as a second fitting point; the second relative position is an arbitrary position on the lens different from the first relative position; When at least one second fitting point is marked, fitting the at least three first fitting points and the at least one second fitting point into a first new closed area, and updating and setting the first new closed area as the preset imaging area of the display module on the lens; or when at least three second fitting points are marked, fitting the at least three second fitting points into a second new closed area, and updating and setting the second new closed area as the preset imaging area of the display module on the lens.

5. The method according to claim 1, wherein The method includes: Identifying a first identity of a first user wearing a first pair of glasses, the first pair of glasses including the lens; Searching for a mapping rule matching the first identity in a preset mapping set; When there is exactly one mapping rule found in the mapping set, setting the preset imaging area in the found mapping rule as the imaging area for the display module to image on the lens; each mapping rule in the mapping set is a preset mapping relationship between a user identity and a preset imaging area, and the various mapping rules in the mapping set are different from each other.

6. The method according to claim 5, wherein The method further includes: When at least two mapping rules are searched from the mapping set, detecting selection instructions for respective preset imaging areas in the at least two mapping rules, and setting the preset imaging area selected by the selection instruction as the imaging area where the display module images on the lens, or setting a first preset imaging area in the at least two mapping rules as the imaging area where the display module images on the lens, where the first preset imaging area is the preset imaging area finally selected in the historical selection record of the first user's selection of the preset imaging area.

7. The method according to claim 5, wherein The method further includes: When no mapping rule is searched from the mapping set, starting the imaging area setting mode; When it is detected that the touch part is touched in a first touch mode at a third relative position where the display module is stationary on the lens in the imaging area setting mode, marking the third relative position as a third fitting point; the third relative position is an arbitrary position of the display module on the lens; When at least three third fitting points are marked, fitting the at least three third fitting points into a third closed area; Setting the third closed area as a third preset imaging area where the display module images on the lens.

8. The method according to claim 7, characterized in that The method further includes: Establishing a third mapping rule between the first identity and the third preset imaging area, and classifying the third mapping rule into the mapping set.

9. The method according to claim 1, characterized in that, The method further includes: When the imaging area setting mode is started again after exiting the imaging area setting mode, and when it is detected that the touch part is touched in a first touch mode at a fourth relative position where the display module is stationary on the lens, marking the fourth relative position as a fourth fitting point; the fourth relative position is an arbitrary position of the display module on the lens; When at least three fourth fitting points are marked, determining the repetition rate between the at least three fourth fitting points and the at least three first fitting points; When it is determined that the repetition rate is not less than a preset threshold, setting the preset imaging area as the current imaging area where the display module images on the lens.

10. The method according to claim 9, wherein The method further includes: When it is determined that the repetition rate is less than the preset threshold, fitting the at least three fourth fitting points into a fourth closed area, and setting the fourth closed area as the current imaging area where the display module images on the lens.

11. The method according to claim 1, wherein The method further includes: Obtaining a first virtual contour model of the lens and a second virtual contour model of the user's eye; Scanning a contour area on the first virtual contour model that is the same as or similar to the second virtual contour model, and setting the contour area as a preset imaging area where the display module images on the lens.

12. The method according to claim 11, wherein The method further includes: Obtaining the identity information of the user; Establishing and saving a mapping rule between the identity information and the preset imaging area.

13. A near-eye display device, characterized in that, The near-eye display device includes: a processor and a memory, where the memory stores programs or instructions that can run on the processor, and when the programs or instructions are executed by the processor, the steps of the control method of the display module described in any one of claims 1 to 12 are implemented.

Images