Interface display method and electronic equipment

Abstract

The embodiment of the invention provides an interface display method and electronic equipment. The method comprises the steps that a user interface to be displayed is acquired; determining a light emitting angle of each pixel in the user interface, wherein the light emitting angle of each pixel enables light emitted by the pixel to be focused on a first pupil center of a user; and displaying the user interface according to the light emitting angle of each pixel. According to the embodiment of the invention, the definition of watching the user interface by the user can be improved when the ametropia user and the presbyopia user use the electronic equipment.

Description

Technical Field

[0001] This application relates to the technical field of intelligent terminals, and particularly to an interface display method and an electronic device. Background Art

[0002] An electronic device with a display function (such as a mobile phone) can provide corresponding user interface display methods to adapt to the use of users with refractive errors and presbyopia (such as elderly users). Taking presbyopic users as an example, interface display methods such as bolding and enlarging the fonts in the user interface, reducing the complexity of the user interface, and magnifying the content being gazed at by the user in the user interface based on the user's fixation point can be adopted.

[0003] However, even with the above display methods provided, there may still be a problem that users with refractive errors and presbyopia cannot clearly see the user interface. Summary of the Invention

[0004] This application provides an interface display method and an electronic device, which can improve the clarity of the user's viewing of the user interface when users with refractive errors and presbyopia use the electronic device.

[0005] In a first aspect, an embodiment of this application provides an interface display method, including: obtaining a user interface to be displayed; determining the light-emitting angle of each pixel in the user interface, where the light-emitting angle of each pixel causes the light emitted by the pixel to be focused on the user's first pupil center; and displaying the user interface according to the light-emitting angle of each pixel. In this method, when displaying the user interface, by controlling the light-emitting angle of each pixel, the light emitted by the pixel can be focused on the user's first pupil center, so that an image of the user interface can be obtained on the user's retina, thereby enabling users with refractive errors and presbyopia to view a clear user interface when using the electronic device and improving the clarity of the user's viewing of the user interface.

[0006] In a possible implementation manner, determining the light-emitting angle of each pixel in the user interface includes: for each pixel, determining the light-emitting angle of the pixel according to a first straight line; the first straight line is the straight line where the pixel unit corresponding to the pixel and the user's first pupil center are located.

[0007] In a possible implementation manner, determining the light-emitting angle of the pixel according to the first straight line includes: determining the first coordinate of the user's first pupil center and the second coordinate of the pixel; and determining the light-emitting angle according to the first coordinate and the second coordinate. In this implementation manner, the center of the display screen can be used as the origin of the coordinate system, the plane where the display screen is located can be used as the XY plane, and the direction pointing directly in front of the display screen passing through the origin can be used as the Z axis. Thus, the light-emitting angle of the pixel can be obtained by calculating the angle between the straight line between the first coordinate and the second coordinate and the XY plane, and the angle between the projection of this straight line in the XY plane and the X axis.

[0008] In a possible implementation, displaying a user interface according to the light-emitting angle of each pixel includes: for each pixel, determining a light-emitting pixel component in a pixel unit corresponding to the pixel according to the light-emitting angle of the pixel; controlling the determined pixel component to emit light according to the pixel value of the pixel.

[0009] In a possible implementation, displaying a user interface according to the light-emitting angle of each pixel includes: for each pixel, controlling a pixel unit corresponding to the pixel to emit light according to the pixel value of the pixel; and determining a control voltage of the pixel unit corresponding to the pixel according to the light-emitting angle of the pixel, where the control voltage is used to control the light-emitting angle of the pixel unit.

[0010] In a second aspect, an embodiment of the present application provides an interface display method applied to an electronic device. The method includes: obtaining a first user interface to be displayed; determining the pixel value and light-emitting angle of each pixel of a second user interface according to the first user interface; and displaying the second user interface according to the pixel value and light-emitting angle of each pixel in the second user interface, so that when a user views the second user interface displayed on the electronic device, an image of the first user interface is obtained on the user's retina. In this method, on the condition that an image of the first user interface is displayed on the user's retina, the second user interface corresponding to the first user interface and the light-emitting angle of each pixel in the second user interface are deduced inversely. Thus, when the second user interface is displayed according to the light-emitting angle, although the second user interface is displayed on the electronic device, an image of the first user interface can be obtained on the user's retina, so that ametropic users and presbyopic users can view a clear user interface when using the electronic device, and the clarity of the user interface viewed by the user is improved.

[0011] In a possible implementation, determining the pixel value and light-emitting angle of each pixel of the second user interface includes: for a first pixel in the first user interface, determining at least two second pixels corresponding to the first pixel in the second user interface; using the pixel value of the first pixel as the pixel value of the second pixel, and determining the light-emitting angle of the second pixel. By taking each pixel in the first user interface as the first pixel, at least two second pixels corresponding to each pixel can be obtained, so that the second user interface can be obtained.

[0012] In a possible implementation, determining at least two second pixels corresponding to the first pixel in the second user interface includes: selecting at least two pixels as the second pixels corresponding to the first pixel in a preset area centered on a third pixel in the second user interface, where the position of the third pixel in the second user interface corresponds to the position of the first pixel in the first user interface.

[0013] In a possible implementation, determining the light-emitting angle of the second pixel includes: obtaining the user's diopter, axial length of the eye, and the coordinates of the first pupil center; determining the coordinates of the fourth pixel in the virtual first user interface according to the user's diopter, axial length of the eye, and the coordinates of the first pupil center; the virtual first user interface is the first user interface deduced backwards on the condition that an image of the first user interface is obtained on the user's retina; the fourth pixel is the pixel in the virtual first user interface with the same position as the first pixel; determining the coordinates of the second pixel; and determining the light-emitting angle of the second pixel according to the coordinates of the fourth pixel and the coordinates of the second pixel. The user's diopter can be detected by the electronic device using devices such as a camera, or can be input by the user to the electronic device. The above-mentioned axial length of the user's eye can be detected by the electronic device using devices such as a camera, or can be input by the user to the electronic device, or can also be calculated by the electronic device according to the user's diopter in a preset manner. The coordinates of the user's first pupil center can be detected by the electronic device using devices such as a camera.

[0014] In a possible implementation, determining at least two second pixels corresponding to the first pixel in the second user interface includes: obtaining the user's diopter, axial length of the eye, and the coordinates of the first pupil center; and determining at least two second pixels corresponding to the first pixel in the second user interface according to the user's diopter, axial length of the eye, and the coordinates of the first pupil center.

[0015] In a possible implementation, determining at least two second pixels corresponding to the first pixel in the second user interface according to the user's diopter, axial length of the eye, and the coordinates of the first pupil center includes: determining the coordinates of the fourth pixel in the virtual first user interface according to the user's diopter, axial length of the eye, and the coordinates of the first pupil center; the virtual first user interface is the first user interface deduced backwards on the condition that an image of the first user interface is obtained on the user's retina; the fourth pixel is the pixel in the virtual first user interface with the same position as the first pixel; determining the coordinates of each fifth pixel in a preset area with the third pixel in the second user interface as the center point; the position of the third pixel in the second user interface corresponds to the position of the first pixel in the first user interface; determining the ideal light-emitting angle of the fifth pixel according to the coordinates of the fourth pixel and the coordinates of the fifth pixel; and selecting at least two fifth pixels with relatively small deviation values between the possible light-emitting angle and the ideal light-emitting angle of the fifth pixel as the second pixels.

[0016] In a possible implementation, selecting at least two fifth pixels with relatively small deviation values between the possible light-emitting angle and the ideal light-emitting angle of the fifth pixel as the second pixels includes: for each second pixel, determining the possible light-emitting angle with the smallest deviation value from the ideal light-emitting angle of the second pixel as the light-emitting angle of the second pixel.

[0017] In a possible implementation, determining the coordinates of a fourth pixel in a virtual first user interface according to the diopter of a user, the axial length of the eye, and the coordinates of a first pupil center includes: determining a first distance from the first pupil center to the center of a display screen according to the coordinates of the first pupil center; determining a second distance between the first pupil center and the plane where the virtual first user interface is located according to the diopter of the user, the axial length of the eye, and the first distance; calculating the coordinates of the fourth pixel in the virtual first interface according to the coordinates of the first pupil center, the first distance, and the second distance.

[0018] In a possible implementation, displaying a second user interface according to the light-emitting angle of each pixel in the second user interface includes: for each pixel, controlling a pixel unit corresponding to the pixel to emit light according to the pixel value of the pixel; and determining a control voltage for the pixel unit corresponding to the pixel according to the light-emitting angle of the pixel, where the control voltage is used to control the light-emitting angle of the pixel unit.

[0019] In a possible implementation, displaying a second user interface according to the light-emitting angle of each pixel in the second user interface includes: for each pixel, determining a pixel component that emits light in the pixel unit corresponding to the pixel according to the light-emitting angle of the pixel; controlling the determined pixel component to emit light according to the pixel value of the pixel.

[0020] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory; where one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the processor, cause the electronic device to execute the method according to any one of the first aspect or the second aspect.

[0021] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and when it runs on a computer, it causes the computer to execute the method according to any one of the first aspect or the second aspect. Description of the Drawings

[0022] To more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0023] Figure 1 It is a schematic diagram of the imaging principle of presbyopia provided by an embodiment of the present application;

[0024] Figure 2 It is a schematic diagram of the correction principle of presbyopia provided by an embodiment of the present application;

[0025] Figure 3A A schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0026] Figure 3B A schematic structural diagram of a display panel provided by an embodiment of the present application;

[0027] Figure 3C Another schematic structural diagram of a display panel provided by an embodiment of the present application;

[0028] Figure 3D A schematic diagram of the light-emitting angle provided by an embodiment of the present application;

[0029] Figure 4 A schematic software structure diagram of an electronic device provided by an embodiment of the present application;

[0030] Figure 5 A schematic interface diagram of an interface display method provided by an embodiment of the present application;

[0031] Figure 6 A schematic principle diagram of an interface display method provided by an embodiment of the present application;

[0032] Figure 7A A schematic flow diagram of an interface display method provided by an embodiment of the present application;

[0033] Figure 7B A schematic diagram of a calibration method for scene depth information in binocular camera calibration provided by an embodiment of the present application;

[0034] Figure 7C A schematic interface diagram displayed by an interface display method provided by an embodiment of the present application;

[0035] Figure 8 Another schematic flow diagram of an interface display method provided by an embodiment of the present application;

[0036] Figure 9 A schematic diagram of a method for establishing a coordinate system provided by an embodiment of the present application;

[0037] Figure 10 Another schematic diagram for the principle explanation of an interface display method provided by an embodiment of the present application;

[0038] Figure 11 Another schematic diagram for the principle explanation of an interface display method provided by an embodiment of the present application;

[0039] Figure 12 Another schematic flow diagram of an interface display method provided by an embodiment of the present application;

[0040] Figure 13It is the fourth process schematic diagram of the interface display method provided by the embodiments of the present application;

[0041] Figure 14 It is a schematic diagram for explaining the principle of an interface display method provided by the embodiments of the present application;

[0042] Figure 15 It is another schematic diagram for explaining the principle of the interface display method provided by the embodiments of the present application. Detailed implementation manners

[0043] The terms used in the implementation manner part of the present application are only used to explain the specific embodiments of the present application, rather than aiming to limit the present application.

[0044] The vision problems of users can include refractive errors and presbyopia, etc. Refractive error means that when the eye does not use accommodation, after the parallel light passes through the refractive action of the eye, it cannot form a clear image on the retina, but forms an image in front of or behind the retina. It includes hyperopia, myopia and astigmatism. Presbyopia is the common name of presbyopia, which is a physiological phenomenon. As people age, the accommodation ability (accommodation amplitude) of the eye gradually decreases, resulting in symptoms such as difficulty in seeing near objects for users.

[0045] Taking presbyopia as an example, through Figure 1 and Figure 2 explain the imaging principle and correction principle when the user's vision has problems.

[0046] The imaging principle of presbyopia is, for example, Figure 1 as shown. Due to the significant decline in the eye's accommodation ability, presbyopic users cannot increase the curvature of the lens by the tension of the ciliary muscle when looking at nearby images, so accurate focusing cannot be achieved, and the object is imaged at the back end of the retina. Finally, people see an unfocused and relatively blurred virtual image.

[0047] As Figure 2 shown, the principle of presbyopic glasses in improving presbyopia lies in: through the imaging principle of a convex lens, it equivalently increases the curvature of the cornea and increases the converging imaging, so that the object is imaged on the retina, and presbyopic users can see a clear image.

[0048] It can be understood that for the convenience of principle explanation, in the accompanying drawings of the embodiments of the present application, the user's eyes are equivalently regarded as a convex lens with a certain curvature, and no further explanation will be given below.

[0049] For users with refractive errors and presbyopia, although an electronic device such as a mobile phone can provide corresponding user interface display methods, such as bolding and enlarging the font in the user interface, reducing the complexity of the user interface, and magnifying the content being gazed at by the user in the user interface based on the user's fixation point. However, even with the above display methods provided, without wearing corrective glasses, the user may still have problems seeing the user interface clearly.

[0050] To this end, the embodiments of the present application provide an interface display method and an electronic device. Through different interface display methods, when users with refractive errors and presbyopia use the electronic device without wearing corrective glasses, they can view the user interface relatively clearly.

[0051] Figure 3A FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 3A shown, the electronic device 100 may include: a processor 110, a memory 120, a display screen 130, a camera 140, and a sensor module 150. The sensor module 150 may include a gyroscope sensor 150A, an acceleration sensor 150B, and the like.

[0052] It can be understood that the structure schematically shown in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0053] The processor 110 may include one or more processing units. For example: the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0054] The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0055] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can hold the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated accesses and reduces the waiting time of the processor 110, thus improving the efficiency of the system.

[0056] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0057] The memory 120 can be used to store computer-executable program code, and the executable program code includes instructions. The memory 120 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store the data created during the use of the electronic device 100 (such as audio data, a phone book, etc.). In addition, the memory 120 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the memory 120 and / or the instructions stored in the memory provided in the processor.

[0058] The electronic device 100 realizes the display function through the GPU, the display screen 130, the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 130 and the application processor. The GPU is used to execute mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0059] The display screen 130 is used to display images, videos, etc. In some embodiments, the electronic device 100 may include one or N display screens 130, where N is a positive integer greater than 1.

[0060] The display screen 130 includes a display panel, and pixels are arranged in the display panel. In some embodiments, the light-emitting angle of each pixel in the display panel is adjustable. In another embodiment, each pixel in the display panel includes multiple light-emitting angles.

[0061] As Figure 3B shown, a schematic structural diagram of a display panel with adjustable light-emitting angles of pixels is shown. Among them, each pixel in the display panel is composed of a collimated backlight + a directional diffraction grating structure, so that each pixel includes pixel components at multiple angles. For example Figure 3B shown, a pixel is composed of pixel components with 4 light-emitting angles. Different diffraction gratings are designed on the pixel components with 4 light-emitting angles. When the light of the pixel component hits the corresponding diffraction grating of the pixel component, due to the design of the diffraction grating, the light will be diffracted at a fixed angle. In this way, a pixel has 4 pixel components with controllable light-emitting angles, and one or more pixel components with light-emitting angles can be selected to be lit according to requirements, so that the pixel emits light at the required light-emitting angle.

[0062] In other embodiments, schemes such as liquid crystal lenses and micro-surfaces can also be selected to control the light-emitting angle of the pixel. For example Figure 3C shown, another schematic structural diagram of a display panel with adjustable light-emitting angles of pixels is shown. Among them, each pixel in the display panel is composed of a liquid crystal single pixel and a lens surface. These pixels can also be called corner pixels. A lens layer and a liquid crystal layer are arranged above each corner pixel, and each corner pixel corresponds to a liquid crystal cell. The refractive index n x of the liquid crystal layer above each corner pixel is controlled separately by voltage. When the voltage is increased, the refractive index n x of the liquid crystal layer changes. The light-emitting angle of each corner pixel deflects due to the deviation between the refractive index n0 of the lens layer and the refractive index n x of the liquid crystal layer. Thus, the light-emitting angle of the corner pixel can be deflected from collimated light to other directions. When the refractive index n x of the liquid crystal layer changes with the voltage, the light-emitting angle of the corner pixel can change accordingly. Thus, the refractive index n of the liquid crystal layer is controlled by voltagex The light-emitting angle of the diagonal pixels can be controlled, so that the corner pixels emit light at the required light-emitting angle. The voltage of the liquid crystal layer can be controlled by a transmissive and opaque control circuit above the liquid crystal cell, and the lens layer can be realized by materials such as polyimide (PI).

[0063] It can be understood that the diffraction structure or refraction structure of the above pixels can determine the light-emitting angle of the pixels.

[0064] In some embodiments, the light-emitting angle in the embodiments of the present application may include: the angle between the light emitted by the pixel (or pixel component) and the plane where the display screen is located, and the angle between the projection of the light on the plane where the display screen is located and the X-axis. In some embodiments, the light-emitting angle of the pixel can be represented by the unit spatial vector of the light direction. For example Figure 3D As shown, the angle between the light and the plane where the display screen is located is α, and the angle between the projection of the light on the display screen (XY plane) and the X-axis is θ. The above two angles can constitute the light-emitting angle of the pixel. It should be noted that when the established coordinate system is different, the light-emitting angle of the pixel can also be represented by other angular relationships, as long as the direction of the light in space can be uniquely determined.

[0065] The electronic device 100 can implement the shooting function through the ISP, the camera 140, the video codec, the GPU, the display screen 130, and the application processor, etc.

[0066] The ISP is used to process the data fed back by the camera 140. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 140.

[0067] The camera 140 is used to capture static images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV and other formats. In some embodiments, the electronic device 100 may include one or N cameras 140, where N is a positive integer greater than 1.

[0068] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0069] The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0070] The NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transmission mode between human brain neurons, it can quickly process input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0071] The gyroscope sensor 150A can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 150A. The gyroscope sensor 150A can be used for anti-shake during shooting. Exemplarily, when the shutter is pressed, the gyroscope sensor 150A detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and makes the lens offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 150A can also be used for navigation and somatosensory game scenarios.

[0072] The acceleration sensor 150B can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0073] Figure 4 It is the software structure block diagram of the electronic device 100 in the embodiment of the present invention.

[0074] The layered architecture divides software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer (also known as: the system framework layer), the system libraries and Android Runtime layer, the hardware abstraction layer (HAL), and the kernel layer.

[0075] The application layer may include several applications (hereinafter simply referred to as applications). In the embodiments of the present application, the application layer may include, for example: a settings application, a camera application, and other applications.

[0076] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer, including various components and services to support developers' Android development. In the embodiments of the present application, the application framework layer may include a vision problem support service.

[0077] The system libraries and Android Runtime layer includes system libraries and Android Runtime. The system libraries may include multiple functional modules, such as the surface manager, libc, etc. Android Runtime is responsible for the scheduling and management of the Android system, specifically including the core libraries and the virtual machine. Among them, the core libraries contain two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core libraries of Android; the virtual machine is used to run Android applications developed using the Java language.

[0078] The HAL layer is an interface layer located between the operating system kernel and the hardware circuit. The HAL layer includes, but is not limited to: the camera hardware abstraction layer (Camera HAL), the sensor hardware abstraction layer (Sensor HAL), etc. The camera hardware abstraction layer is used to process the image stream, and the sensor hardware abstraction layer is used to process the data of the sensors.

[0079] The kernel layer is the layer between the hardware and the software. The kernel layer may include: a camera driver, an acceleration sensor driver, a gyroscope sensor driver, etc. The camera driver is used to drive the camera. The acceleration sensor driver is used to drive the acceleration sensor. The gyroscope sensor driver is used to drive the gyroscope sensor.

[0080] Next, the interface rotation control method of the present application will be described in conjunction with the structure of the electronic device shown in FIG. 3 above and Figure 4 shown.

[0081] In some embodiments, the interface display method of the present application embodiment can be provided as a vision problem support service by the operating system of the electronic device, or can be implemented as a vision problem support function provided in a certain application.

[0082] As Figure 5 shown, when the interface display method of the present application embodiment is provided as a vision problem support service by the operating system of the electronic device, the user can open the corresponding setting interface in the setting application provided by the electronic device. As shown in 501, the vision problem support service can be started or closed through user operations. When the vision problem support service is started, the electronic device executes the interface display method of the present application embodiment when displaying the user interface. When the interface display method of the present application embodiment is provided as a vision problem support service in a certain application of the electronic device, the start or close of the service can also be achieved through a similar method.

[0083] In an interface display method provided by the present application embodiment, as Figure 6 shown, when displaying the user interface, by setting the light-emitting angle of each pixel in the display screen, the light emitted by each pixel can be focused on the center of the user's first pupil, so that the user interface displayed on the display screen can be imaged on the user's retina, enabling the user to see a clear image of the user interface. It can be understood that the above-mentioned first pupil center refers to the center of the user's eye pupil. The above-mentioned first pupil center can be the pupil center of the left eye. At this time, the user interface displayed on the display screen can be imaged on the retina of the user's left eye. The above-mentioned first pupil center can also be the pupil center of the right eye. At this time, the user interface displayed on the display screen can be imaged on the retina of the user's right eye.

[0084] FIG. 7 is a schematic flowchart of an interface display method according to an embodiment of the present application. As shown in FIG. 7, the method may include:

[0085] Step 701: Obtain the user interface to be displayed.

[0086] The above-mentioned user interface to be displayed is the user interface that the application hopes the electronic device to display on the display screen. The above-mentioned user interface can be, for example, the desktop of the electronic device provided by the desktop management application, the above-mentioned function setting interface 500 of the setting application, the video playing interface provided by the video playing application, etc.

[0087] Combined with Figure 4 the structure of the electronic device shown, as Figure 8 shown, this step may specifically include: when the first application currently running on the electronic device hopes to display the user interface, send the interface data to the window manager in the framework layer.

[0088] Step 702: Determine the light-emitting angle of each pixel in the user interface. The light-emitting angle of each pixel is such that the light emitted by the pixel unit corresponding to the pixel is focused on the user's first pupil center.

[0089] Optionally, this step may specifically include:

[0090] For each pixel X, determine the light-emitting angle of pixel X according to the first straight line; the first straight line is the straight line where the pixel unit corresponding to pixel X and the user's first pupil center are located.

[0091] Specifically, a coordinate system as shown, for example, Figure 9 can be established. Taking the central pixel unit of the display screen as the origin, the plane where the display screen is located as the XY plane, the horizontal right direction in the user's visible dimension as the positive direction of the X axis, the vertical upward direction as the positive direction of the Y axis, and the direction from the origin to the front of the display screen as the positive direction of the Z axis.

[0092] Optionally, based on the above coordinate system, a camera can be used to capture a user image, so as to determine the first coordinate of the user's first pupil center in the above coordinate system, and determine the second coordinate of each pixel unit corresponding to each pixel in the above coordinate system. Calculate the angle between the first straight line and the XY plane, and the angle between the projection of the first straight line on the XY plane and the X axis, so as to obtain the light-emitting angle of the pixel. It should be noted that the coordinates involved in this embodiment and the following Figure 12 embodiments shown can all be coordinates in this coordinate system.

[0093] Optionally, relevant three-dimensional object detection methods can be used to determine the first coordinate of the user's implementation center in the above preset coordinate system. For example,

[0094] In some embodiments, the first coordinate of the user's first pupil center in the preset coordinate system can be determined by a binocular depth estimation method.

[0095] The principle of the binocular depth estimation method is as follows: Place two parallel cameras with a certain distance between them. Based on the principle that the same spatial point is imaged at different positions on these two cameras respectively, calibrate the position of the spatial point according to the difference in the imaging positions. The above positions can be, for example, the distance from the plane of the parallel cameras and the plane coordinates on the plane of the parallel cameras. Specifically, in the binocular depth estimation method, the depth information of the scene can be estimated first through the two cameras; after obtaining the depth information of the scene, each point in the images captured by the two cameras is combined with the depth information to be converted into a point cloud. When converting the point cloud, relevant technologies can be used to implement, for example, the Pseudo-Lidar algorithm, etc.; then, three-dimensional object detection is performed through methods such as point cloud target detection or point cloud segmentation.

[0096] For exampleFigure 7B The calibration principle of the depth information Z in the binocular depth estimation method is shown. Based on Figure 7B The following relationship can be obtained as shown in the following formula:

[0097]

[0098] From this, the calculation formula for the depth information Z can be deduced as shown in the following formula:

[0099]

[0100] Among them, f is the focal length of the left and right cameras, T is the distance between the binocular cameras, and x l is the horizontal position of the target object in the imaging plane of the left camera, and x r is the horizontal position of the target object in the imaging plane of the right camera.

[0101] Combined with Figure 4 the structure of the electronic device shown, as Figure 8 shown, this step may specifically include: The window manager requests the light-emitting angle of each pixel from the vision problem support service. The vision problem support service obtains the user's image through the camera driver, determines the coordinates of the user's first pupil center based on the user's image, and according to the light-emitting angle of each pixel, sends the light-emitting angle of each pixel to the window manager.

[0102] Step 703: Display the above-mentioned user interface to be displayed according to the light-emitting angle of each pixel.

[0103] In one instance, if each pixel in the display panel realizes the adjustment of the light-emitting angle based on the Figure 3B structure shown, then, in this step, the first diffraction grating with the light-emitting angle closest to the light-emitting angle of the pixel can be obtained from the diffraction gratings of the corresponding pixel units in the display panel according to the light-emitting angle of each pixel, and the pixel component corresponding to the first diffraction grating in the pixel unit is controlled to emit light according to the pixel value of the corresponding pixel. The light-emitting angle of the above-mentioned first diffraction grating being closest to the light-emitting angle of the pixel may mean that the deviation value between the two light-emitting angles is the smallest.

[0104] In another instance, if each pixel in the display panel realizes the adjustment of the light-emitting angle based on the structure shown in 3C, then, in this step, the corresponding corner pixel unit in the display panel can be controlled to emit light according to the pixel value of each pixel, and the voltage of the corresponding pixel unit of the pixel can be controlled according to the light-emitting angle of the pixel, so that the light-emitting angle of the pixel unit is the same as the light-emitting angle of the corresponding pixel.

[0105] For example Figure 7C shown, a schematic diagram of the implementation of the user interface displayed on the display panel is shown. In this figure, each pixel in the display panel is based onFigure 3B Take the structure shown as an example for implementing the light-emitting angle adjustment.

[0106] Combined with Figure 4 the structure of the electronic device shown, such as Figure 8 shown, this step may specifically include: The window manager sends an interface display request of the user interface to the display driver. The interface display request may include the pixel value of each pixel in the user interface and the light-emitting angle of each pixel. The display driver controls each pixel unit in the display to emit light according to the pixel value and the light-emitting angle of the corresponding pixel in the user interface.

[0107] The method shown in FIG. 7 can control each pixel unit in the display screen to emit light according to the light-emitting angle of the corresponding pixel in the user interface, so that the light emitted by each pixel unit can pass through the eye where the user's first pupil center is located and be imaged on the retina, enabling the user to see a clear image of the user interface.

[0108] Moreover, the light-emitting angle of each pixel in the above method is calculated and displayed in real time following the user's pupil position. Therefore, for others other than the user, when viewing the display screen of the electronic device, the image viewed is different from the image viewed by the above user. Therefore, it can also play a certain anti-peeping effect.

[0109] It can be understood that in the method shown in FIG. 7, taking the imaging on one eye of the user as an example, in order to enable the user interface displayed on the display screen of the user device to be imaged on the retinas of the user's two eyes respectively, the electronic device can calculate the light-emitting angle 1 of each pixel unit based on the pupil center of the left eye and calculate the light-emitting angle 2 of each pixel unit based on the pupil center of the right eye according to the above method. Thus, each pixel unit can obtain 2 light-emitting angles, and the electronic device controls each pixel unit to emit light according to the corresponding 2 light-emitting angles respectively. Taking Figure 3B the structure of the display panel shown as an example, the electronic device can control the pixel components corresponding to the 2 light-emitting angles in each pixel unit to emit light according to the pixel value of each pixel unit and the 2 light-emitting angles.

[0110] In another interface display method provided in the embodiments of the present application, such as Figure 10As shown in the figure, assume that there is an electronic device 1 displaying a first user interface at an appropriate position, such as a position at a distance d1 from the user's eyes. It can enable the user to see the clear first user interface under the user's diopter condition. In other words, when the first user interface is displayed on the electronic device X, the first user interface can be imaged on the user's retina. Under this condition, each ray emitted by the pixel unit A of the electronic device X will necessarily be focused on the image a of the pixel unit A on the user's retina after being refracted by the user's eyes. When the user has refractive error or presbyopia, when the above conditions are met, the distance between the electronic device 1 and the user's eyes is relatively far, that is, the value of d1 is relatively large, which is very inconvenient or even impossible for the user to achieve when using this electronic device in daily life. The purpose to be achieved in the embodiments of the present application is: to enable the user to use the electronic device at a position relatively closer to the user's eyes, such as Figure 10 using the electronic device at a position at a distance d2 from the user's eyes in Figure 10 . Although the electronic device displays a second user interface, when the user views the second user interface, what the user actually sees is the clear first user interface. In other words, the second user interface can be imaged on the user's retina as the image of the first user interface.

[0111] Based on this goal, as Figure 11 shown, in the interface display method provided in the embodiments of the present application, when it is necessary to display the first user interface to the user, first reverse-deduce at least two pixel units on the display screen corresponding to the image of each pixel m of the first user interface on the retina when the first user interface is imaged on the user's retina. Set the pixel values of the above at least two pixel units to the pixel value of pixel m, and determine the light-emitting angle of each pixel unit among the above at least two pixel units. Thus, when the above at least two pixel units emit light according to the pixel value and the light-emitting angle, the light can be imaged on the user's retina as the image of pixel m. Based on the above principle, the pixel value of each pixel in the second user interface and the light-emitting angle of each pixel in the second user interface can be obtained according to the pixel value and the light-emitting angle of each pixel unit on the display screen. Thus, it is ensured that when the electronic device displays the second user interface, the image of the first user interface can be obtained on the user's retina, enabling the user to clearly see the first user interface. For example, for the image a corresponding to pixel A of the first user interface on the retina, the pixel units A1, A2, A3, etc. on the display screen and the light-emitting angle of each pixel unit can be reverse-deduced. When the pixel units A1, A2, A3 emit light based on the pixel value of pixel A according to the corresponding light-emitting angles, the light they emit can be imaged on the user's retina as the image a corresponding to pixel A.

[0112] It should be noted that the reason for corresponding at least two pixel units on the display screen in the embodiments of the present application is as follows: The imaging system in the embodiments of the present application is a non-focusing system. Therefore, at least two light rays are required for each image point of the image of the first user interface (corresponding to the image a of pixel A described above) to form a focus. In other words, since the second user interface displayed on the display screen of the electronic device in the embodiments of the present application is different from the first user interface and each pixel has a corresponding light-emitting angle, if it is desired that the image formed on the user's retina is the image of the first user interface, at least two pixel units on the display screen need to emit light according to the light-emitting angle to be focused into one imaging point on the user's retina, and finally form the image of the first user interface.

[0113] Figure 12 It is another flowchart of the interface display method provided by the embodiments of the present application. As Figure 11 shown, the method may include:

[0114] Step 1201: Obtain a first user interface to be displayed.

[0115] The first user interface to be displayed above is the user interface that the application hopes the electronic device to display on the display screen. The above user interface may be, for example, the desktop of the electronic device provided by the desktop management application, the above function setting interface 500 of the setting application, the video playing interface provided by the video playing application, etc. The embodiments of the present application do not make limitations.

[0116] Referring to Figure 13 step 1301 in, this step may specifically include: When the first application currently running on the electronic device hopes to display the first user interface, it sends a display request for the first user interface to the window manager of the framework layer, and the display request may carry the data of the first user interface.

[0117] Step 1202: Determine the pixel value and light-emitting angle of each pixel in the second user interface according to the first user interface to be displayed.

[0118] The purpose of determining the pixel value and light-emitting angle of each pixel in the second user interface is to enable the electronic device to display the second user interface according to the pixel value and light-emitting angle of each pixel in the second user interface, so that when the user views the second user interface displayed by the electronic device, the image of the first user interface is obtained on the user's retina.

[0119] Optionally, in one embodiment, when determining the pixel value and the light-emitting angle of each pixel in the second user interface, for the first pixel in the first user interface, at least two second pixels corresponding to the first pixel can be determined in the second user interface. The pixel value of the first pixel is used as the pixel value of each second pixel, and the light-emitting angle of each second pixel is determined, so as to obtain the pixel value and the light-emitting angle of the second pixel. The pixel value and the light-emitting angle of the second pixel are used such that when the pixel unit corresponding to the second pixel emits light according to the pixel value and the light-emitting angle, when the user views the second pixel displayed on the display screen of the electronic device, an image of the first pixel in the first user interface is obtained on the user's retina. It can be understood that the above first pixel can be any pixel in the first user interface. By performing the above processing on each pixel in the first user interface, the pixel value and the light-emitting angle of each pixel in the second user interface can be obtained. For specific implementation, please refer to Figure 13 and Figure 14 , which will not be elaborated here.

[0120] See Figure 13 In steps 1302 to 1307 in

[0121] Step 1203: Display the second user interface according to the pixel value and the light-emitting angle of each pixel in the second user interface.

[0122] In one embodiment, if each pixel in the display panel realizes the adjustment of the light-emitting angle based on the structure shown in Figure 3B , then in this step, for each pixel, the pixel component that emits light in the pixel unit corresponding to the pixel can be determined according to the light-emitting angle of the pixel, and the determined pixel component is controlled to emit light according to the pixel value of the pixel. Since the pixel component has a fixed light-emitting angle, the pixel unit can emit light according to the pixel value and the light-emitting angle of the corresponding pixel.

[0123] In one embodiment, if each pixel in the display panel realizes the adjustment of the light-emitting angle based on the structure shown in Figure 3C , then in this step, for each pixel, the pixel unit corresponding to the pixel can be controlled to emit light according to the pixel value of the pixel; and, the control voltage of the pixel unit corresponding to the pixel is determined according to the light-emitting angle of the pixel, and the corresponding control voltage is provided for the pixel unit to control the light-emitting angle of the pixel unit.

[0124] Refer to Figure 13 Steps 1308 - 1309 in [reference], this step may specifically include: the window manager sends an interface display request of the user interface to the display driver. The interface display request may include the pixel values of each pixel in the second user interface and the light emission angles of each pixel. The display driver controls each pixel unit in the display to emit light according to the pixel value of the corresponding pixel and emit light at the light emission angle of the corresponding pixel.

[0125] Figure 12 The method shown can control each pixel unit in the display screen to emit light according to the pixel value and light emission angle of the corresponding pixel in the second user interface, so that the second user interface is displayed on the display screen, but an image of the first user interface is obtained on the retina of the eye where the center of the first pupil of the user is located, enabling the user to see a clear image of the first user interface.

[0126] Moreover, the pixel values and light emission angles of each pixel in the above method are calculated and displayed in real time following the position of the user's pupil. Therefore, for others other than the user, when viewing the display screen of the electronic device, the image seen is different from the image seen by the above user. Therefore, it can also achieve a certain anti-peeping effect.

[0127] It should be noted that Figure 12 The coordinates of the pixels and the coordinates of the center of the first pupil in the embodiments of the present application shown are all coordinates in the same coordinate system. This coordinate system may be, for example, Figure 9 the coordinate system shown. It can be understood that there may also be other methods for establishing a coordinate system, as long as the above coordinates are calculated in the same coordinate system.

[0128] Figure 12 In the method shown, taking imaging in one eye of the user as an example, in order to enable the user interface displayed on the display screen of the user device to be imaged on the retinas of both eyes of the user respectively, the electronic device may calculate the light emission angle 1 of each pixel unit based on the pupil center of the left eye and calculate the light emission angle 2 of each pixel unit based on the pupil center of the right eye according to the above method. Thus, each pixel unit can obtain 2 light emission angles, and the electronic device controls each pixel unit to emit light at the corresponding 2 light emission angles respectively. Taking Figure 3B the structure of the display panel shown as an example, the electronic device can control the pixel components corresponding to the 2 light emission angles in each pixel unit to emit light according to the pixel value of each pixel unit and the 2 light emission angles.

[0129] Next, through Figure 14 and Figure 15 an exemplary illustration of the possible implementation of determining at least 2 second pixels corresponding to the first pixel and determining the light emission angles of the second pixels in step 1202 is given.

[0130] In one embodiment, it is assumed that the display panel of the display screen is implemented by the Figure 3C structure shown. At this time, since the light-emitting angle of each pixel unit in the display panel can be adjusted as needed, in this embodiment, at least 2 pixels can be selected as the second pixels corresponding to the first pixel in a preset area centered on the third pixel corresponding to the first pixel in the second user interface.

[0131] Optionally, the above-mentioned third pixel corresponding to the first pixel can be, for example, a pixel in the second user interface that has the same position as the first pixel, or a pixel that has an associated relationship with the position of the first pixel.

[0132] For the convenience of processing, the third pixel and at least 1 pixel adjacent to the third pixel can be selected as the second pixels corresponding to the first pixel. The size of the above-mentioned preset area can be set in advance. For example, it can be a circular area centered on the third pixel, and the radius of the circular area is set in advance. The specific size is not limited in the embodiments of the present application. In some embodiments, the radius of the circular area can be determined based on the proportion of the circular area in the second user interface (for example, 5%).

[0133] For example, assume that the first pixel is a pixel at coordinates (300, 150) in the first user interface. Then, the pixel at coordinates (300, 150) in the second user interface is the above-mentioned third pixel, and the third pixel at (300, 150) in the second user interface and 1 pixel adjacent to it, such as the pixel at coordinates (300, 151), can be selected as the above-mentioned second pixels.

[0134] It can be understood that the resolution of the first user interface in the embodiments of the present application can be lower than the resolution of the second user interface, so that the pixels in the first user interface can correspond to at least 2 pixels in the second user interface; or, in the embodiments of the present application, some pixels in the first user interface can be discarded under the condition of not affecting the imaging effect as much as possible, so that the pixels in the first user interface can correspond to at least 2 pixels in the second user interface.

[0135] For example, when the resolution of the first user interface is lower than the resolution of the second user interface, the above-mentioned third pixel serving as the center point of the preset area can be a pixel in the second user interface that has a corresponding relationship with the first pixel determined based on the difference in resolution. For example, if the resolution of the second user interface is 2 times the resolution of the first user interface, the coordinates of the third pixel corresponding to the first pixel (x, y) can be (2x, 2y), so that each pixel in the first user interface can obtain a corresponding second pixel in the second user interface.

[0136] For example, when the resolution of the first user interface is the same as that of the second user interface, if the pixel at the coordinate (300, 151) in the second user interface has already been used as the second pixel corresponding to (300, 150) in the first user interface, the pixel at the coordinate (300, 151) in the first user interface can be discarded, that is, its corresponding second pixel is not determined in the second user interface.

[0137] When determining the light-emitting angle of each second pixel, the electronic device can obtain the user's diopter, eye axis length, and the coordinates of the user's first pupil center. It can be understood that the diopter, eye axis length, and the coordinates of the user's first pupil center here are parameters based on the same eye. The diopter, eye axis length, and pupil center coordinates of the user's left eye, and the diopter, eye axis length, and pupil center coordinates of the user's right eye can be obtained respectively. Here, the diopter, eye axis length, and pupil center coordinates of one eye are taken as an example.

[0138] Among them, the user's diopter can be measured by the electronic device for the user's eyes, or the electronic device can provide an input interface for the user, and it can be obtained based on the data input by the user in the input interface.

[0139] The user's eye axis length can be measured by the electronic device for the user's eyes, or the electronic device can provide an input interface for the user, and it can be obtained based on the data input by the user in the input interface. Or, the empirical value of the eye axis length corresponding to the diopter can also be used.

[0140] The coordinates of the user's first pupil center can be measured by the electronic device for the user's eyes. For the specific measurement method, please refer to the corresponding description in the embodiment shown in FIG. 7 above, which will not be elaborated here.

[0141] The electronic device can determine the coordinates of the fourth pixel in the virtual first user interface according to the user's diopter, eye axis length, and the coordinates of the first pupil center, and determine the coordinates of the above-mentioned second pixel. The light-emitting angle of the second pixel is determined according to the coordinates of the fourth pixel and the coordinates of the second pixel.

[0142] The virtual first user interface in this embodiment is the first user interface deduced by reverse on the condition that the image of the first user interface is obtained on the user's retina; the fourth pixel is the pixel in the virtual first user interface with the same position as the first pixel. For example, the virtual first user interface can be Figure 13 the interface shown in the figure with a distance of d2 from the first pupil center. This virtual first user interface is the first user interface assumed to be displayed by the electronic device at this position. If the electronic device displays the first user interface at this position, the image of the first user interface can be obtained on the user's retina, but in the embodiments of the present application, the electronic device does not need to display the first user interface at this position. The fourth pixel can be, for example, pixel A in the virtual first user interface.

[0143] Taking Figure 13 as an example, assuming that for the first pixel A0 in the first user interface, its image on the user's retina is the image point a, and its corresponding pixel in the second user interface is Figure 13 the pixel A1 in, then at least 2 second pixels corresponding to the first pixel A0 can be determined within a preset area centered on the pixel A1, such as the pixel A1 and the pixel A2;

[0144] Based on the coordinates of the fourth pixel A at the same position as the first pixel A0 in the virtual first user interface and the coordinates of the pixel A1, the angle between the vector from the fourth pixel A to the pixel A1 and the second user interface (i.e., the plane where the above display screen is located or the XY plane) and the angle between the projection of this vector on the second user interface and the X-axis can be calculated to obtain the light-emitting angle of the pixel A1. The angle between the vector from the fourth pixel A to the pixel A2 and the second user interface and the angle between the projection of this vector on the second user interface and the X-axis are calculated to obtain the light-emitting angle of the pixel A2.

[0145] The calculation method of the coordinates of the above fourth pixel A is described as follows:

[0146] Combined with Figure 13 the coordinate system shown, the electronic device can calculate the first distance d1 between the first pupil center and the central pixel unit of the display screen (corresponding to the origin O of the coordinate system) according to the coordinates of the user's first pupil center. The distance calculation method between the two coordinates is not elaborated in the embodiments of the present application.

[0147] The electronic device can determine the second distance d2 between the first pupil center of the user and the plane where the virtual first user interface is located according to the user's diopter, eye axis length, and the first distance d1. Specifically, according to the diopter of the user's eye, the equivalent curvature radius R of the eye lens can be determined. According to the user's eye axis length, the focal length f of the eye can be obtained. Under the conditions of known R, d1, and f, combined with the optical relationship of the lens and Figure 13 the geometric relationship shown in, the second distance d2 between the first pupil center of the user and the plane where the virtual first user interface is located can be calculated. The specific calculation can be implemented using related technologies and will not be elaborated here.

[0148] After that, the electronic device can calculate the coordinates of the fourth pixel A in the virtual first user interface according to the coordinates of the first pupil center, the first distance d1, and the second distance d2. The specific calculation can be implemented using related technologies and will not be elaborated here.

[0149] The above embodiments illustrate the possible implementation of determining the second pixel and the light-emitting angle of the second pixel under the display panel structure shown in Figure 3C .

[0150] In another embodiment, it is assumed that the display panel of the display screen is implemented by the structure shown, for example, Figure 3B That is, each pixel has multiple pixel components, and the light-emitting angle of each pixel component is fixed.

[0151] In this embodiment, the electronic device can determine at least two second pixels corresponding to the first pixel according to the user's diopter, eye axis length, and the first coordinate of the user's first pupil center in the preset coordinate system. It can be understood that here, the diopter, eye axis length, and the coordinates of the pupil center of the same eye are still taken as examples. For the specific implementation, reference can be made to the foregoing related description, which will not be elaborated here.

[0152] Optionally, in this embodiment, the electronic device can determine the coordinates of the fourth pixel in the virtual first user interface according to the user's diopter, eye axis length, and the coordinates of the first pupil center; determine the coordinates of each fifth pixel in the preset area centered on the third pixel corresponding to the first pixel in the second user interface; determine the ideal light-emitting angle of the fifth pixel according to the coordinates of the fourth pixel and the coordinates of the fifth pixel; select at least two fifth pixels whose possible light-emitting angles are relatively close to the ideal light-emitting angle of the fifth pixel as the second pixels corresponding to the first pixel.

[0153] Since when the display screen of the electronic device is implemented by the Figure 3B shown structure, each pixel includes multiple pixel components, and each pixel component has a fixed light-emitting angle. In other words, each pixel has multiple possible light-emitting angles. In the embodiment of the present application, the possible light-emitting angle and the ideal light-emitting angle of each fifth pixel can be compared, the deviation value between the two can be calculated, and at least two fifth pixels with relatively small deviation values among the fifth pixels are selected as the second pixels corresponding to the first pixel A; furthermore, for each fifth pixel serving as the second pixel, the light-emitting angle corresponding to the pixel component with the smallest deviation value between its light-emitting angle and its ideal light-emitting angle is selected as the final light-emitting angle of the fifth pixel, that is, the light-emitting angle of the determined second pixel.

[0154] The calculation methods of the coordinates of the fourth pixel and the coordinates of each fifth pixel can refer to the foregoing description and related technologies for implementation, which will not be elaborated in the embodiment of the present application.

[0155] As Figure 14 shown, still taking the image a of pixel A0 in the first user interface as an example, the pixel A1 at the same position as pixel A0 in the second user interface can be used as the center point, and the second pixel corresponding to the first pixel A0 is searched in the preset area around pixel A1. Then, the pixel at the same position as the first pixel A0 in the virtual first user interface, that is, the fourth pixel A.

[0156] The electronic device can calculate the ideal light-emitting angle of each pixel within the preset area around the fourth pixel A in the virtual first user interface and each pixel within the preset area around pixel A1 based on the coordinates of each pixel in the preset coordinate system. For example, pixel A2 and pixel A3 in the second user interface correspond to the above-mentioned fifth pixel. By calculating the angle between the vector from the fourth pixel A to pixel A2 and the plane of the second user interface, which is also the plane of the display screen, and the angle between the projection of this vector on the second user interface and the X-axis, the ideal light-emitting vector of pixel A2 can be obtained; by calculating the angle between the vector from the fourth pixel A to pixel A3 and the plane of the second user interface, which is also the plane of the display screen, and the angle between the projection of this vector on the second user interface and the X-axis, the ideal light-emitting vector of pixel A3 can be obtained.

[0157] Since each pixel component of each pixel has a fixed light-emitting angle when the display screen of the electronic device is implemented through the Figure 3B structure shown, in the embodiments of the present application, the possible light-emitting angles and the ideal light-emitting angles of each fifth pixel can be compared, the deviation value between the two can be calculated, and at least 2 fifth pixels with relatively small deviation values among the fifth pixels can be selected as the second pixels corresponding to the first pixel A. At least 2 fifth pixels with relatively small deviation values between the possible light-emitting angle and the ideal light-emitting angle are selected as the second pixels. For example, Figure 15 in, pixel A1, pixel A2, and pixel A3 are all fifth pixels. Compared with the deviation value between the light-emitting angle 33 and its ideal light-emitting angle 30 in pixel A3, the deviation values between the light-emitting angle 13 and its ideal light-emitting angle 10 in pixel A1, and the deviation value between the light-emitting angle 23 and its ideal light-emitting angle 20 in pixel A2 are relatively small. Pixel A1 and pixel A2 can be selected as the second pixels corresponding to the first pixel A0.

[0158] Furthermore, the light-emitting angle 13 with the smallest deviation value from its ideal light-emitting angle 10 in pixel A1 can be used as the final light-emitting angle of pixel A1, that is, the light-emitting angle determined for the second pixel A1; the light-emitting angle 23 with the smallest deviation value from its ideal light-emitting angle 20 in pixel A2 can be used as the final light-emitting angle of pixel A2, that is, the light-emitting angle determined for the second pixel A2.

[0159] Thus, in step 1203, when the display screen displays the second user interface, for the pixel unit corresponding to pixel A1, the pixel component corresponding to the light-emitting angle 13 can be selected to emit light according to the pixel value of pixel A1; for the pixel unit corresponding to pixel A2, the pixel component corresponding to the light-emitting angle 23 can be selected to emit light according to the pixel value of pixel A2.

[0160] Optionally, the resolution of the first user interface in the embodiments of the present application may be lower than that of the second user interface, so that the pixels in the first user interface can correspond to at least 2 pixels in the second user interface; and / or, in the embodiments of the present application, one pixel in the second user interface may correspond to 2 or more pixels in the first user interface. In other words, multiple different pixels in the first user interface may take the same pixel in the second user interface as their corresponding second pixel. At this time, the pixel serving as the second pixel may have 2 or more pixel values and 2 or more light-emitting angles. In step 1203, the 2 or more pixel components of the second pixel may be controlled to emit light respectively according to the pixel values. For example, assume that pixel A1 in the second user interface serves as the second pixel of pixel A01 and pixel A02 in the first user interface respectively. Then, according to pixel A01, the pixel value 1 and the corresponding light-emitting angle 1 of pixel A1 can be determined. According to pixel A02, the pixel value 2 and the corresponding light-emitting angle 2 of pixel A1 can be determined. Then, when the second user interface is displayed, the pixel component corresponding to the light-emitting angle 1 in pixel A1 can be controlled to emit light according to the pixel value 1, and the pixel component corresponding to the light-emitting angle 2 can be controlled to emit light according to the pixel value 2.

[0161] Optionally, when each pixel unit in the display screen has n light-emitting angles, each pixel in the second user interface can correspond to at most n pixels in the first user interface. In other words, it can serve as the second pixel corresponding to at most n first pixels in the first user interface.

[0162] The above embodiments illustrate the possible implementation of determining the second pixel and the light-emitting angle of the second pixel under the Figure 3B display panel structure shown.

[0163] The embodiments of the present application further provide an electronic device, including a processor and a memory. The processor is used to implement the method provided by the embodiments of the present application.

[0164] The embodiments of the present application further provide a computer-readable storage medium, in which a computer program is stored. When it runs on a computer, it causes the computer to execute the method provided by the embodiments of the present application.

[0165] The embodiments of the present application further provide a computer program product, which includes a computer program. When it runs on a computer, it causes the computer to execute the method provided by the embodiments of the present application.

[0166] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent the situations of existing A alone, existing A and B simultaneously, and existing B alone. Where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one of the following" and its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0167] Those of ordinary skill in the art can realize that the various units and algorithm steps described in the embodiments disclosed herein can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0168] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0169] In several embodiments provided by the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (hereinafter referred to as ROM), random access memories (hereinafter referred to as RAM), magnetic disks, or optical discs that can store program codes.

[0170] The above is only the specific implementation manner of the present application. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed in the present application should be covered by the protection scope of the present application. The protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An interface display method, characterized in that, Including: Obtain the user interface to be displayed; Determine the light-emitting angle of each pixel in the user interface, and the light-emitting angle of each pixel is such that the light emitted by the pixel is focused on the center of the user's first pupil; Display the user interface according to the light-emitting angle of each pixel.

2. The method according to claim 1, characterized in that, The determining the light-emitting angle of each pixel in the user interface includes: For each pixel, determine the light-emitting angle of the pixel according to the first straight line; the first straight line is the straight line where the pixel unit corresponding to the pixel and the center of the user's first pupil are located.

3. The method according to claim 2, wherein The determining the light-emitting angle of the pixel according to the first straight line includes: Determine the first coordinate of the center of the user's first pupil and the second coordinate of the pixel; Determine the light-emitting angle according to the first coordinate and the second coordinate.

4. The method according to any one of claims 1 to 3, characterized in that, The displaying the user interface according to the light-emitting angle of each pixel includes: For each pixel, determine the light-emitting pixel component in the pixel unit corresponding to the pixel according to the light-emitting angle of the pixel; Control the determined pixel component to emit light according to the pixel value of the pixel.

5. The method according to any one of claims 1 to 3, characterized in that, The displaying the user interface according to the light-emitting angle of each pixel includes: For each pixel, control the pixel unit corresponding to the pixel to emit light according to the pixel value of the pixel; and determine the control voltage of the pixel unit corresponding to the pixel according to the light-emitting angle of the pixel, and the control voltage is used to control the light-emitting angle of the pixel unit.

6. An interface display method, characterized in that, Applied to an electronic device, the method includes: Obtain the first user interface to be displayed; According to the first user interface, determine the pixel value and the light-emitting angle of each pixel of the second user interface; Display the second user interface according to the pixel value and the light-emitting angle of each pixel in the second user interface, so that when the user views the second user interface displayed by the electronic device, an image of the first user interface is obtained on the retina of the user.

7. The method according to claim 6, characterized in that, The determining the pixel value and the light-emitting angle of each pixel of the second user interface includes: For the first pixel in the first user interface, determine at least two second pixels corresponding to the first pixel in the second user interface; Use the pixel value of the first pixel as the pixel value of the second pixel, and determine the light-emitting angle of the second pixel.

8. The method according to claim 7, characterized in that, The determining at least two second pixels corresponding to the first pixel in the second user interface includes: Select at least two pixels as the second pixels corresponding to the first pixel in a preset area centered on the third pixel in the second user interface, and the position of the third pixel in the second user interface corresponds to the position of the first pixel in the first user interface.

9. The method according to claim 8, characterized in that, The determining the light-emitting angle of the second pixel includes: Obtain the diopter, eye axis length of the user and the coordinates of the center of the first pupil; Determine the coordinates of the fourth pixel in the virtual first user interface based on the diopter, axial length of the user, and the coordinates of the first pupil center; the virtual first user interface is a first user interface deduced backwards on the condition that an image of the first user interface is obtained on the retina of the user; the fourth pixel is a pixel in the virtual first user interface with the same position as the first pixel; Determine the coordinates of the second pixel; Determine the light-emitting angle of the second pixel based on the coordinates of the fourth pixel and the coordinates of the second pixel.

10. The method according to claim 7, characterized in that, The determining at least two second pixels corresponding to the first pixel in the second user interface includes: Obtain the diopter, axial length of the user, and the coordinates of the first pupil center; Determine at least two second pixels corresponding to the first pixel in the second user interface based on the diopter, axial length of the user, and the coordinates of the first pupil center.

11. The method according to claim 10, characterized in that, The determining at least two second pixels corresponding to the first pixel in the second user interface based on the diopter, axial length of the user, and the coordinates of the first pupil center includes: Determine the coordinates of the fourth pixel in the virtual first user interface based on the diopter, axial length of the user, and the coordinates of the first pupil center; the virtual first user interface is a first user interface deduced backwards on the condition that an image of the first user interface is obtained on the retina of the user; the fourth pixel is a pixel in the virtual first user interface with the same position as the first pixel; Determine the coordinates of each fifth pixel in a preset area with the third pixel in the second user interface as the center point; the position of the third pixel in the second user interface corresponds to the position of the first pixel in the first user interface; Determine the ideal light-emitting angle of the fifth pixel based on the coordinates of the fourth pixel and the coordinates of the fifth pixel; Select at least two fifth pixels with relatively small deviation values between the possible light-emitting angle and the ideal light-emitting angle of the fifth pixel as the second pixel.

12. The method according to claim 11, characterized in that, The selecting at least two fifth pixels with relatively small deviation values between the possible light-emitting angle and the ideal light-emitting angle of the fifth pixel as the second pixel includes: For each of the second pixels, determine the possible light-emitting angle with the smallest deviation value from the ideal light-emitting angle of the second pixel as the light-emitting angle of the second pixel.

13. The method according to claim 9 or 11, characterized in that, The determining the coordinates of the fourth pixel in the virtual first user interface based on the diopter, axial length of the user, and the coordinates of the first pupil center includes: Determine the first distance from the first pupil center to the center of the display screen based on the coordinates of the first pupil center; Determine the second distance between the first pupil center and the plane where the virtual first user interface is located based on the diopter, axial length of the user, and the first distance; Calculate the coordinates of the fourth pixel in the virtual first interface based on the coordinates of the first pupil center, the first distance, and the second distance.

14. The method according to any one of claims 6 to 9, characterized in that displaying the second user interface according to the light-emitting angle of each pixel in the second user interface, includes: For each pixel, controlling the pixel unit corresponding to the pixel to emit light according to the pixel value of the pixel; and determining a control voltage for the pixel unit corresponding to the pixel according to the light-emitting angle of the pixel, where the control voltage is used to control the light-emitting angle of the pixel unit.

15. The method according to any one of claims 10 to 13, characterized in that displaying the second user interface according to the light-emitting angle of each pixel in the second user interface, includes: For each pixel, determining a pixel component that emits light in the pixel unit corresponding to the pixel according to the light-emitting angle of the pixel; Controlling the determined pixel component to emit light according to the pixel value of the pixel.

16. An electronic device, characterized in that, including: a processor, a memory; One or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1 to 15.

17. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, and when it runs on a computer, it causes the computer to perform the method according to any one of claims 1 to 15.

Images