Interface display method and electronic device

Abstract

Embodiments of the present application provide an interface display method and an electronic device. The method comprises: obtaining a user interface to be displayed; determining an out-light angle of each pixel in the user interface, the out-light angle of each pixel being such that light emitted by the pixel is focused on a first pupil center of a user; and displaying the user interface according to the out-light angle of each pixel. Embodiments of the present application can improve the clarity of a user interface when viewed by a user with ametropia or presbyopia when using an electronic device.

Description

Technical Field

[0001] This application relates to the field of smart terminal technology, and in particular to an interface display method and an electronic device. Background Technology

[0002] Electronic devices with display capabilities (e.g., mobile phones) can provide corresponding user interface display methods to accommodate users with refractive errors and presbyopia (e.g., elderly users). Taking presbyopic users as an example, display methods such as making the font in the user interface bolder and larger, reducing the complexity of the user interface, and magnifying the content in the user interface based on the user's gaze point can be used.

[0003] However, even with the above display methods, users with refractive errors and presbyopia may still have difficulty seeing the user interface. Summary of the Invention

[0004] This application provides an interface display method and an electronic device that can improve the clarity of the user interface when users with refractive errors and presbyopia use electronic devices.

[0005] In a first aspect, embodiments of this application provide an interface display method, comprising: acquiring a user interface to be displayed; determining the light emission angle of each pixel in the user interface, wherein the light emission angle of each pixel is such that the light emitted by the pixel is focused on the center of the user's first pupil; and displaying the user interface according to the light emission angle of each pixel. In this method, by controlling the light emission angle of each pixel when displaying the user interface, the light emitted by the pixel can be focused on the center of the user's first pupil, thereby enabling an image of the user interface to be obtained on the user's retina. This allows users with refractive errors and presbyopia to view a clear user interface when using electronic devices, improving the clarity of the user's view of the user interface.

[0006] In one possible implementation, determining the light emission angle of each pixel in the user interface includes: for each pixel, determining the light emission angle of the pixel based on a first straight line; the first straight line is the straight line between the pixel unit corresponding to the pixel and the center of the user's first pupil.

[0007] In one possible implementation, determining the light emission angle of a pixel based on a first straight line includes: determining a first coordinate of the center of the user's first pupil and a second coordinate of the pixel; and determining the light emission angle based on the first and second coordinates. In this implementation, the center of the display screen can be used as the origin of the coordinate system, the plane on which the display screen is located can be used as the XY plane, and the direction passing through the origin and pointing directly in front of the display screen can be used as the Z-axis. Therefore, the light emission angle of the pixel can be obtained by calculating the angle between the straight line between the first and second coordinates and the XY plane, and the angle between the projection of this straight line onto the XY plane and the X-axis.

[0008] In one possible implementation, the user interface is displayed according to the light emission angle of each pixel, including: for each pixel, determining the light-emitting pixel component in the pixel unit corresponding to the pixel according to the light emission angle of the pixel; and controlling the determined pixel component to emit light according to the pixel value of the pixel.

[0009] In one possible implementation, displaying the user interface according to the light emission angle of each pixel includes: for each pixel, controlling the corresponding pixel unit to emit light according to the pixel value of the pixel; and determining the control voltage of the corresponding pixel unit according to the light emission angle of the pixel, the control voltage being used to control the light emission angle of the pixel unit.

[0010] Secondly, embodiments of this application provide an interface display method applied to an electronic device. The method includes: acquiring a first user interface to be displayed; determining the pixel value and light emission angle of each pixel of a second user interface based on the first user interface; and displaying the second user interface according to the pixel value and light emission angle of each pixel in the second user interface, so that when a user views the second user interface displayed on the electronic device, an image of the first user interface is obtained on the user's retina. This method uses the condition of displaying an image of the first user interface on the user's retina to deduce the second user interface corresponding to the first user interface and the light emission angle of each pixel in the second user interface. Therefore, when the second user interface is displayed according to the light emission angle, although the electronic device displays the second user interface, the user's retina can still obtain an image of the first user interface. This allows users with refractive errors and presbyopia to view a clear user interface when using the electronic device, improving the clarity of the user's view of the user interface.

[0011] In one possible implementation, determining the pixel value and emission angle of each pixel in the second user interface includes: for a first pixel in the first user interface, determining at least two second pixels corresponding to the first pixel in the second user interface; using the pixel value of the first pixel as the pixel value of the second pixel; and determining the emission angle of the second pixel. By using each pixel in the first user interface as the first pixel, at least two second pixels corresponding to each pixel can be obtained, thereby obtaining the second user interface.

[0012] In one possible implementation, determining at least two second pixels corresponding to the first pixel in the second user interface includes: selecting at least two pixels as second pixels corresponding to the first pixel in a preset area centered on the third pixel in the second user interface, wherein the position of the third pixel in the second user interface corresponds to the position of the first pixel in the first user interface.

[0013] In one possible implementation, determining the light emission angle of the second pixel includes: acquiring the user's refractive error, axial length, and the coordinates of the center of the first pupil; determining the coordinates of the fourth pixel in the virtual first user interface based on the user's refractive error, axial length, and the coordinates of the center of the first pupil; the virtual first user interface is a first user interface derived from an image of the first user interface obtained on the user's retina; the fourth pixel is a pixel in the virtual first user interface that is at the same position as the first pixel; determining the coordinates of the second pixel; and determining the light emission angle of the second pixel based on the coordinates of the fourth pixel and the second pixel. The user's refractive error can be detected by the electronic device using a camera or similar device, or it can be input by the user into the electronic device. The user's axial length can be detected by the electronic device using a camera or similar device, or it can be input by the user into the electronic device, or it can be calculated by the electronic device based on the user's refractive error according to a preset method. The coordinates of the center of the user's first pupil can be detected by the electronic device using a camera or similar device.

[0014] In one possible implementation, determining at least two second pixels corresponding to the first pixel in the second user interface includes: obtaining the user's refractive power, axial length, and coordinates of the center of the first pupil; and determining at least two second pixels corresponding to the first pixel in the second user interface based on the user's refractive power, axial length, and coordinates of the center of the first pupil.

[0015] In one possible implementation, at least two second pixels corresponding to the first pixel are determined in the second user interface based on the user's refractive error, axial length, and the coordinates of the first pupil center. This includes: determining the coordinates of a fourth pixel in the virtual first user interface based on the user's refractive error, axial length, and the coordinates of the first pupil center; the virtual first user interface is a first user interface derived conditioned on obtaining an image of the first user interface on the user's retina; the fourth pixel is a pixel in the virtual first user interface that is at the same position as the first pixel; determining the coordinates of each fifth pixel in a preset area centered on a third pixel in the second user interface; the position of the third pixel in the second user interface corresponds to the position of the first pixel in the first user interface; determining the ideal light emission angle of the fifth pixel based on the coordinates of the fourth pixel and the fifth pixel; and selecting at least two fifth pixels from the fifth pixels whose possible light emission angles deviate relatively from the ideal light emission angle of the fifth pixel as the second pixels.

[0016] In one possible implementation, selecting at least two fifth pixels from the fifth pixels whose possible emission angles have relatively small deviations from the ideal emission angle of the fifth pixel as second pixels includes: for each second pixel, determining the possible emission angle of the second pixel with the smallest deviation from the ideal emission angle of the second pixel as the emission angle of the second pixel.

[0017] In one possible implementation, determining the coordinates of the fourth pixel in the virtual first user interface based on the user's refractive error, axial length, and the coordinates of the first pupil center includes: determining a first distance from the center of the first pupil to the center of the display screen based on the coordinates of the first pupil center; determining a second distance between the center of the first pupil and the plane where the virtual first user interface is located based on the user's refractive error, axial length, and the first distance; and calculating the coordinates of the fourth pixel in the virtual first interface based on the coordinates of the first pupil center, the first distance, and the second distance.

[0018] In one possible implementation, displaying the second user interface according to the light emission angle of each pixel in the second user interface includes: for each pixel, controlling the corresponding pixel unit to emit light according to the pixel value of the pixel; and determining the control voltage of the corresponding pixel unit according to the light emission angle of the pixel, the control voltage being used to control the light emission angle of the pixel unit.

[0019] In one possible implementation, the second user interface is displayed according to the light emission angle of each pixel in the second user interface, including: for each pixel, determining the light-emitting pixel component in the pixel unit corresponding to the pixel according to the light emission angle of the pixel; and controlling the determined pixel component to emit light according to the pixel value of the pixel.

[0020] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory; wherein one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the processor, cause the electronic device to perform the method of either the first aspect or the second aspect.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method of either the first or second aspect. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the imaging principle of presbyopia provided in an embodiment of this application;

[0024] Figure 2 This is a schematic diagram illustrating the correction principle of presbyopia provided in an embodiment of this application;

[0025] Figure 3A A schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0026] Figure 3B This is a schematic diagram of the structure of a display panel provided in an embodiment of this application;

[0027] Figure 3C Another structural schematic diagram of the display panel provided in the embodiments of this application;

[0028] Figure 3D A schematic diagram showing the light emission angle provided in an embodiment of this application;

[0029] Figure 4 A schematic diagram of the software structure of an electronic device provided in an embodiment of this application;

[0030] Figure 5 A schematic diagram of an interface display method provided in an embodiment of this application;

[0031] Figure 6 A schematic diagram illustrating the principle of the interface display method provided in this application embodiment;

[0032] Figure 7A A flowchart illustrating an interface display method provided in an embodiment of this application;

[0033] Figure 7B This is a schematic diagram of a method for calibrating scene depth information in binocular camera calibration provided in an embodiment of this application;

[0034] Figure 7C A schematic diagram of the interface display method provided in the embodiments of this application;

[0035] Figure 8 Another flowchart illustrating the interface display method provided in this application embodiment;

[0036] Figure 9 A schematic diagram illustrating a method for establishing a coordinate system according to an embodiment of this application;

[0037] Figure 10 Another schematic diagram illustrating the principle of the interface display method provided in the embodiments of this application;

[0038] Figure 11 Another schematic diagram illustrating the principle of the interface display method provided in the embodiments of this application;

[0039] Figure 12 Another flowchart illustrating the interface display method provided in this application embodiment;

[0040] Figure 13This is a schematic diagram of the fourth process of the interface display method provided in the embodiments of this application;

[0041] Figure 14 A schematic diagram illustrating the principle of the interface display method provided in the embodiments of this application;

[0042] Figure 15 This is a schematic diagram illustrating another principle of the interface display method provided in the embodiments of this application. Detailed Implementation

[0043] The terminology used in the implementation section of this application is for the purpose of explaining specific embodiments of this application only, and is not intended to limit this application.

[0044] Vision problems for users can include refractive errors and presbyopia. Refractive errors occur when, without accommodation, parallel light rays passing through the eye fail to form a clear image on the retina, instead focusing in front of or behind it. This includes farsightedness, nearsightedness, and astigmatism. Presbyopia, also known as age-related farsightedness, is a physiological phenomenon. As people age, their accommodative ability (amplitude of accommodation) gradually declines, causing symptoms such as difficulty focusing on near objects.

[0045] Taking presbyopia as an example, through Figure 1 and Figure 2 This explains the imaging and correction principles when a user experiences vision problems.

[0046] The imaging principle of presbyopia, for example Figure 1 As shown, due to a significant decrease in the eye's accommodative power, presbyopic users cannot increase the curvature of the lens through ciliary muscle tension when viewing near images. Therefore, they cannot achieve accurate focusing, and the image of the object is formed behind the retina. Ultimately, people see an unfocused, relatively blurry virtual image.

[0047] like Figure 2 As shown, the principle behind reading glasses improving presbyopia is that, through the imaging principle of convex lenses, the curvature of the cornea is effectively increased, and the converging image is increased, so that the object is imaged on the retina, allowing presbyopic users to see a clear image.

[0048] It is understood that, for the sake of easy explanation of the principle, the user's eye is equivalent to a convex lens with a certain curvature in the accompanying drawings of the embodiments of this application, and will not be explained in detail below.

[0049] For users with refractive errors and presbyopia, although electronic devices such as mobile phones can provide corresponding user interface display methods, such as making the font bolder and larger, reducing the complexity of the user interface, and magnifying the content that the user is looking at based on the user's gaze point, even with these display methods, users may still have difficulty seeing the user interface clearly without wearing corrective glasses.

[0050] Therefore, this application provides an interface display method and an electronic device, which enable users with refractive errors and presbyopia to see the user interface more clearly when using the electronic device without wearing corrective glasses, through different interface display methods.

[0051] Figure 3A This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 3A As shown, the electronic device 100 may include: a processor 110, a memory 120, a display screen 130, a camera 140, and a sensor module 150. The sensor module 150 may include a gyroscope sensor 150A, an accelerometer sensor 150B, etc.

[0052] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0053] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0054] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0055] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0056] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0057] The memory 120 can be used to store computer executable program code, which includes instructions. The memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, phonebook, etc.). Furthermore, the memory 120 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the memory 120 and / or instructions stored in memory disposed within the processor.

[0058] Electronic device 100 implements display functions through a GPU, display screen 130, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 130 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0059] The display screen 130 is used to display images, videos, etc. In some embodiments, the electronic device 100 may include one or N display screens 130, where N is a positive integer greater than 1.

[0060] The display screen 130 includes a display panel in which pixels are disposed. In some embodiments, the light emission angle of each pixel in the display panel is adjustable. In another embodiment, each pixel in the display panel includes multiple light emission angles.

[0061] like Figure 3B The diagram illustrates a structural schematic of a display panel with adjustable pixel emission angles. Each pixel in the display panel employs a collimated backlight and a directional diffraction grating structure, thus each pixel includes pixel components at multiple angles. For example... Figure 3B As shown, a pixel consists of pixel components with four light-emitting angles. Different diffraction gratings are designed on the pixel components with four light-emitting angles. When the light of a pixel component hits the diffraction grating corresponding to the pixel component, the light will diffract at a fixed angle due to the design of the diffraction grating. Thus, a pixel has four pixel components with controllable light-emitting angles. One or more pixel components with light-emitting angles can be selected to be lit according to the needs, so that the pixel emits light at the required light-emitting angle.

[0062] In other embodiments, the light emission angle of the pixel can also be controlled using liquid crystal lenses, micro-surfaces, etc., for example. Figure 3C The diagram illustrates another structural schematic of a display panel with adjustable pixel emission angles. In this panel, each pixel consists of a liquid crystal single pixel and a lens surface; these pixels can also be called corner pixels. A lens layer and a liquid crystal layer are disposed above each corner pixel, and each corner pixel corresponds to a liquid crystal cell. The refractive index n of the liquid crystal layer above each corner pixel is shown. x The refractive index n of the liquid crystal layer is controlled solely by voltage. Increasing the voltage will affect the refractive index n of the liquid crystal layer. x The light emission angle of each corner pixel changes due to the refractive index n0 of the lens layer and the refractive index n of the liquid crystal layer. x The light emission angle of the corner pixel is deflected due to the deviation between the two, thus the light emitted from the corner pixel can be deflected from the collimated light to other directions, and the refractive index n of the liquid crystal layer... x As the voltage changes, the light emission angle of the corner pixel can also change, thereby controlling the refractive index n of the liquid crystal layer through voltage control.x This allows for control of the light emission angle of the diagonal pixels, ensuring that the corner pixels emit light at the desired angle. The voltage of the liquid crystal layer can be controlled by a transparency control circuit above the liquid crystal cell, and the lens layer can be made of materials such as polyimide (PI).

[0063] It is understandable that the diffraction or refraction structure of the aforementioned pixels can determine the light emission angle of the pixels.

[0064] In some embodiments, the light emission angle in this application may include: the angle between the light emitted by a pixel (or a pixel component) and the plane of the display screen, and the angle between the projection of the light ray onto the plane of the display screen and the X-axis. In some embodiments, the light emission angle of a pixel may be represented by a unit space vector of the light ray direction. For example... Figure 3D The angle between the ray and the plane of the display screen is α, and the angle between the projection of the ray onto the display screen (XY plane) and the X-axis is θ. These two angles constitute the emission angle of the pixel. It should be noted that when different coordinate systems are established, the emission angle of the pixel can also be represented by other angular relationships, as long as the direction of the ray in space can be uniquely determined.

[0065] Electronic device 100 can perform shooting functions through ISP, camera 140, video codec, GPU, display 130 and application processor.

[0066] The ISP (Image Signal Processor) is used to process data fed back from the camera 140. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimizations on image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be integrated into the camera 140.

[0067] Camera 140 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 140, where N is a positive integer greater than 1.

[0068] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP performs Fourier transforms on the frequency energy.

[0069] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0070] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0071] The gyroscope sensor 150A can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 150A can determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 150A can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 150A detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 150A can also be used in navigation and motion-sensing game scenarios.

[0072] The accelerometer 150B can detect the magnitude of acceleration of electronic device 100 in various directions (typically three axes). When electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of electronic device and applied to applications such as screen orientation switching and pedometers.

[0073] Figure 4 This is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention.

[0074] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer (also known as the system framework layer), the system library and Android runtime layer, the hardware abstraction layer (HAL), and the kernel layer.

[0075] The application layer may include several applications (hereinafter referred to as applications). In this embodiment, the application layer may include, for example, a settings application, a camera application, and other applications.

[0076] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer, including various components and services to support Android development for developers. In this embodiment, the application framework layer may include vision impairment support services.

[0077] The system libraries and Android runtime layer comprise the system libraries and the Android Runtime. The system libraries can include multiple functional modules, such as the surface manager and libc. The Android Runtime is responsible for the scheduling and management of the Android system, specifically including the core libraries and the virtual machine. The core libraries consist of two parts: one part contains the functional functions that Java needs to call, and the other part contains the core Android libraries; the virtual machine is used to run Android applications developed using the Java language.

[0078] The HAL layer is the interface layer located between the operating system kernel and the hardware circuitry. The HAL layer includes, but is not limited to, the Camera HAL and the Sensor HAL. The Camera HAL is used to process image streams, while the Sensor HAL is used to process sensor data.

[0079] The kernel layer is the layer between hardware and software. The kernel layer can include: camera driver, accelerometer driver, gyroscope driver, etc. The camera driver drives the camera. The accelerometer driver drives the accelerometer sensor. The gyroscope driver drives the gyroscope sensor.

[0080] The following, in conjunction with Figure 3 above and Figure 4 The structure of the electronic device shown illustrates the interface rotation control method implemented in this application.

[0081] In some embodiments, the interface display method of this application can be implemented as a vision problem support service provided by the operating system of an electronic device, or as a vision problem support function provided in a certain application.

[0082] like Figure 5 As shown in the embodiments of this application, when the interface display method serves as a vision problem support service provided by the operating system of an electronic device, the user can open the corresponding settings interface in the settings application provided by the electronic device, such as shown in 501. The user can then start or stop the vision problem support service. When the vision problem support service is started, the electronic device executes the interface display method of this application when displaying the user interface. When the interface display method of this application serves as a vision problem support service provided by a certain application in the electronic device, the service can also be started or stopped using a similar method.

[0083] In an interface display method provided in an embodiment of this application, such as Figure 6 As shown, when displaying the user interface, by setting the light emission angle of each pixel on the screen, the light emitted by each pixel can be focused onto the center of the user's first pupil. This allows the user interface displayed on the screen to be imaged on the user's retina, enabling the user to see a clear image of the user interface. It can be understood that the aforementioned first pupil center refers to the center of the user's pupil. This first pupil center can be the center of the left pupil, in which case the user interface displayed on the screen can be imaged on the user's left retina; conversely, it can be the center of the right pupil, in which case the user interface displayed on the screen can be imaged on the user's right retina.

[0084] Figure 7 is a flowchart illustrating an interface display method according to an embodiment of this application. As shown in Figure 7, the method may include:

[0085] Step 701: Obtain the user interface to be displayed.

[0086] The aforementioned user interface to be displayed is the user interface that the application wants the electronic device to display on the screen. For example, the aforementioned user interface may be the desktop of the electronic device provided by the desktop management application, the aforementioned function settings interface 500 of the settings application, the video playback interface provided by the video playback application, etc.

[0087] Combination Figure 4 The electronic device structure shown is as follows: Figure 8 As shown, this step may specifically include: when the first application currently running on the electronic device wants to display the user interface, it sends the interface data to the window manager of the frame layer.

[0088] Step 702: Determine the light emission angle of each pixel in the user interface, such that the light emitted by the pixel unit corresponding to the pixel is focused on the center of the user's first pupil.

[0089] Optionally, this step may specifically include:

[0090] For each pixel X, the light emission angle of pixel X is determined according to the first straight line; the first straight line is the straight line between the pixel unit corresponding to pixel X and the center of the user's first pupil.

[0091] Specifically, it is possible to establish, for example Figure 9 The coordinate system shown has the center pixel unit of the display screen as the origin, the plane where the display screen is located as the XY plane, the horizontal direction to the right in the user's visible dimension is the positive X-axis, the vertical direction upward is the positive Y-axis, and the direction from the origin to the front of the display screen is the positive Z-axis.

[0092] Optionally, based on the above coordinate system, a camera can be used to capture an image of the user, thereby determining the first coordinate of the center of the user's first pupil in the above coordinate system, and determining the second coordinate of the pixel unit corresponding to each pixel in the above coordinate system. The angle between the first straight line and the XY plane, and the angle between the projection of the first straight line in the XY plane and the X-axis are calculated based on the first and second coordinates, thereby obtaining the light emission angle of the pixel. It should be noted that this embodiment and the following... Figure 12 The coordinates involved in the illustrated embodiments can all be coordinates in this coordinate system.

[0093] Optionally, relevant 3D object detection methods can be used to determine the first coordinates of the user's realization center in the aforementioned preset coordinate system. For example,

[0094] In some embodiments, the first coordinates of the user's first pupil center in a preset coordinate system can be determined by a binocular depth estimation method.

[0095] The principle of binocular depth estimation is as follows: two parallel cameras are placed at a certain distance. Based on the principle that the same spatial point is imaged at different positions on the two cameras, the position of the spatial point is determined according to the difference in the imaged positions. These positions can be, for example, the distance from the plane of the parallel cameras and the planar coordinates on the plane of the parallel cameras. Specifically, in binocular depth estimation, the depth information of the scene can first be estimated using the two cameras; after obtaining the depth information, the points in the images captured by the two cameras are combined with the depth information to convert them into a point cloud. During the point cloud conversion, relevant techniques such as the pseudo-radar point cloud (Pseudo-Lidar) algorithm can be used; then, 3D object detection is performed using point cloud target detection or point cloud segmentation methods.

[0096] For example Figure 7B This illustrates the calibration principle of depth information Z in the binocular depth estimation method, based on... Figure 7B The following relationship can be obtained:

[0097]

[0098] Therefore, the formula for calculating depth information Z can be derived as follows:

[0099]

[0100] Where f is the focal length of the left and right cameras, T is the distance between the two cameras, and x is the focal length of the left and right cameras. l It is the lateral position of the target object in the imaging plane of the left camera, x r It is the lateral position of the target object in the imaging plane of the right camera.

[0101] Combination Figure 4 The electronic device structure shown is as follows: Figure 8 As shown, this step may specifically include: the window manager requests the light emission angle of each pixel from the vision support service; the vision support service obtains the user's image through the camera driver, determines the coordinates of the center of the user's first pupil based on the user's image, and sends the light emission angle of each pixel to the window manager accordingly.

[0102] Step 703: Display the user interface to be displayed according to the light emission angle of each pixel.

[0103] In one instance, if each pixel in the display panel is based on Figure 3B The structure shown enables light emission angle adjustment. In this step, based on the light emission angle of each pixel, a first diffraction grating whose emission angle is closest to that of the pixel can be obtained from the diffraction grating of the corresponding pixel unit in the display panel. The pixel component corresponding to the first diffraction grating in that pixel unit is then controlled to emit light according to the pixel value of the corresponding pixel. The closest possible light emission angle between the first diffraction grating and the pixel can be defined as having the smallest deviation between the two emission angles.

[0104] In another example, if each pixel in the display panel adjusts its light emission angle based on the structure shown in 3C, then in this step, the corresponding angular pixel unit in the display panel can be controlled to emit light according to the pixel value of each pixel, and the voltage of the corresponding pixel unit can be controlled according to the light emission angle of the pixel, so that the light emission angle of the pixel unit is the same as the light emission angle of the corresponding pixel.

[0105] For example Figure 7C The diagram illustrates the implementation of the user interface displayed on the display panel, where each pixel in the display panel is based on... Figure 3B The structure shown is used as an example to adjust the light emission angle.

[0106] Combination Figure 4 The electronic device structure shown is as follows: Figure 8 As shown, this step may specifically include: the window manager sending a user interface display request to the display driver. The user interface display request may include the pixel value of each pixel in the user interface and the light emission angle of each pixel. The display driver controls each pixel unit in the display to emit light according to the pixel value and light emission angle of the corresponding pixel in the user interface.

[0107] The method shown in Figure 7 can control each pixel unit in the display screen to emit light according to the light emission angle of the corresponding pixel in the user interface, so that the light emitted by each pixel unit can be imaged on the retina through the eye where the center of the user's first pupil is located, so that the user can see a clear image of the user interface.

[0108] Moreover, in the above method, the light emission angle of each pixel is calculated and displayed in real time according to the position of the user's pupil. Therefore, when other people look at the screen of the electronic device, the image they see is different from the image seen by the user. Thus, it can also play a certain role in preventing peeping.

[0109] As can be understood, the method shown in Figure 7 takes imaging in one of the user's eyes as an example. To ensure that the user interface displayed on the user device's screen can be imaged on the retinas of both eyes, the electronic device can calculate the light emission angle 1 of each pixel unit based on the center of the pupil of the left eye, and the light emission angle 2 of each pixel unit based on the center of the pupil of the right eye. Thus, each pixel unit can obtain two light emission angles, and the electronic device can control each pixel unit to emit light according to the corresponding two light emission angles. Figure 3B Taking the structure of the display panel shown as an example, the electronic device can control the pixel components corresponding to the two light emission angles in each pixel unit to emit light according to the pixel value and two light emission angles of each pixel unit.

[0110] In another interface display method provided in the embodiments of this application, such as Figure 10As shown, suppose an electronic device 1 displays a first user interface at a suitable position, for example, a distance d1 from the user's eyes, so that the user can see the first user interface clearly under the user's refractive error conditions. In other words, when the first user interface is displayed on the electronic device X, the first user interface can be imaged on the user's retina. Under this condition, every ray of light emitted by pixel unit A of the electronic device X, after refraction by the user's eyes, will necessarily focus on the image a of pixel unit A on the user's retina. However, when the user has refractive error or presbyopia, the distance between the electronic device 1 and the user's eyes is greater when the above conditions are met, that is, the value of d1 is larger, which is very inconvenient or even impossible for the user to use the electronic device in daily life. The purpose of this application embodiment is to enable the user to see the first user interface at a position closer to the user's eyes, for example... Figure 10 An electronic device is used at a distance d2 from the user's eyes. Although the electronic device displays a second user interface, when the user views the second user interface, what the user actually sees is a clear first user interface. In other words, the second user interface can be imaged as the first user interface on the user's retina.

[0111] Based on this goal, such as Figure 11 As shown in the embodiment of this application, in the interface display method, when it is necessary to display a first user interface to a user, the method first reverses the process of imaging the first user interface on the user's retina. Each pixel m of the first user interface on the retina corresponds to at least two pixel units on the display screen. The pixel values ​​of these at least two pixel units are set to the pixel value of pixel m. Furthermore, the light emission angle of each of these at least two pixel units is determined. Thus, when these at least two pixel units emit light according to their pixel values ​​and light emission angles, the light can be imaged on the user's retina as the image of pixel m. Based on this principle, the pixel values ​​and light emission angles of each pixel in the second user interface can be obtained according to the pixel values ​​and light emission angles of each pixel unit on the display screen. This ensures that when the electronic device displays the second user interface, the image of the first user interface can be obtained on the user's retina, allowing the user to clearly see the first user interface. For example, for the image a corresponding to pixel A of the first user interface on the retina, the pixel units A1, A2, A3, etc. on the display screen and the light emission angle of each pixel unit can be deduced. When pixel units A1, A2, A3 emit light based on the pixel value of pixel A according to the corresponding light emission angle, the light emitted by them can be imaged on the user's retina as the image a corresponding to pixel A.

[0112] It should be noted that the reason for having at least two pixel units on the display screen in this embodiment is that the imaging system in this embodiment is a non-focusing system. Therefore, each image point of the first user interface (corresponding to image a of pixel A mentioned above) requires at least two light rays to form a focus. In other words, since the second user interface displayed on the screen of the electronic device in this embodiment is different from the first user interface and each pixel has a corresponding light emission angle, if it is desired that the image formed on the user's retina is the image of the first user interface, at least two pixel units on the display screen need to emit light according to the light emission angle to focus on one imaging point on the user's retina, thereby ultimately forming the image of the first user interface.

[0113] Figure 12 This is another flowchart illustrating the interface display method provided in this application embodiment, such as... Figure 11 As shown, the method may include:

[0114] Step 1201: Obtain the first user interface to be displayed.

[0115] The aforementioned first user interface to be displayed is the user interface that the application wants the electronic device to display on the screen. The aforementioned user interface may be, for example, the desktop of the electronic device provided by the desktop management application, the aforementioned function settings interface 500 of the settings application, the video playback interface provided by the video playback application, etc. This application embodiment does not impose any limitations.

[0116] See Figure 13 Step 1301 in the process may specifically include: when the first application currently running on the electronic device wants to display the first user interface, it sends a display request for the first user interface to the window manager of the frame layer, and the display request may carry the data of the first user interface.

[0117] Step 1202: Based on the first user interface to be displayed, determine the pixel value and light emission angle of each pixel in the second user interface.

[0118] The purpose of determining the pixel value and light emission angle of each pixel in the second user interface is to enable the electronic device to display the second user interface according to the pixel value and light emission angle of each pixel in the second user interface, so that when the user views the second user interface displayed by the electronic device, the user's retina receives an image of the first user interface.

[0119] Optionally, in one embodiment, when determining the pixel value and emission angle of each pixel in the second user interface, for a first pixel in the first user interface, at least two second pixels corresponding to the first pixel can be determined in the second user interface. The pixel value of the first pixel is used as the pixel value of each second pixel, and the emission angle of each second pixel is determined, thereby obtaining the pixel value and emission angle of the second pixel. The pixel value and emission angle of the second pixel are used so that when the pixel unit corresponding to the second pixel emits light according to the pixel value and emission angle, when the user views the second pixel displayed on the screen of the electronic device, the user's retina obtains an image of the first pixel in the first user interface. It is understood that the first pixel can be any pixel in the first user interface, and by performing the above processing on each pixel in the first user interface, the pixel value and emission angle of each pixel in the second user interface can be obtained. Please refer to the specific implementation for details. Figure 13 and Figure 14 That will not be elaborated here.

[0120] See Figure 13 In steps 1302-1307, this step may include: the window manager requests the vision problem support service to convert the first user interface; the vision problem support service obtains the user's image through the camera driver, determines the coordinates of the user's first pupil center based on the user's image, and obtains the user's refractive power and axial length, thereby converting the first user interface into a second user interface, and determining the light emission angle of each pixel in the second user interface, and sending the second user interface and the light emission angle of each pixel in the second user interface to the window manager.

[0121] Step 1203: Display the second user interface based on the pixel value and light emission angle of each pixel in the second user interface.

[0122] In one embodiment, if each pixel in the display panel is based on Figure 3B The structure shown allows for adjustment of the light emission angle. Therefore, in this step, for each pixel, the light-emitting pixel component in the corresponding pixel unit can be determined according to the pixel's light emission angle, and the determined pixel component emits light according to the pixel value. Since the pixel component has a fixed light emission angle, the pixel unit can emit light according to the corresponding pixel value and light emission angle.

[0123] In one embodiment, if each pixel in the display panel is based on Figure 3C The structure shown enables the adjustment of the light emission angle. In this step, for each pixel, the light emission of the pixel unit corresponding to the pixel can be controlled according to the pixel value. Furthermore, the control voltage of the pixel unit corresponding to the pixel is determined according to the light emission angle of the pixel, and the corresponding control voltage is provided to the pixel unit to control the light emission angle of the pixel unit.

[0124] See Figure 13 Steps 1308-1309 may specifically include: the window manager sending a user interface display request to the display driver. The user interface display request may include the pixel value of each pixel in the second user interface and the light emission angle of each pixel. The display driver controls each pixel unit in the display to emit light according to the pixel value of the corresponding pixel and to emit light according to the light emission angle of the corresponding pixel.

[0125] Figure 12 The method shown can control each pixel unit in the display screen to emit light according to the pixel value and light emission angle of the corresponding pixel in the second user interface, so that the second user interface is displayed on the display screen, but the retina of the eye where the user's first pupil is located receives the image of the first user interface, so that the user can see a clear image of the first user interface.

[0126] Moreover, in the above method, the pixel value and light emission angle of each pixel are calculated and displayed in real time according to the position of the user's pupil. Therefore, when other people look at the screen of the electronic device, the image they see is different from the image seen by the user. Thus, it can also play a certain role in preventing peeping.

[0127] It should be noted that, Figure 12 The pixel coordinates and the coordinates of the center of the first pupil shown in the embodiments of this application are all coordinates in the same coordinate system, which may be, for example, a coordinate system that is not explicitly defined in the original text. Figure 9 The coordinate system shown is provided. It is understandable that other coordinate systems can be established, as long as the coordinates described above are calculated within the same coordinate system.

[0128] Figure 12 The method described above uses imaging in one of the user's eyes as an example. To ensure that the user interface displayed on the device's screen is imaged on the retinas of both eyes, the electronic device can calculate the emission angle 1 of each pixel unit based on the center of the left pupil and the emission angle 2 of each pixel unit based on the center of the right pupil. Thus, each pixel unit receives two emission angles, and the electronic device controls each pixel unit to emit light according to these two corresponding emission angles. Figure 3B Taking the structure of the display panel shown as an example, the electronic device can control the pixel components corresponding to the two light emission angles in each pixel unit to emit light according to the pixel value and two light emission angles of each pixel unit.

[0129] The following is through Figure 14 and Figure 15 An illustrative description is provided of possible implementations of determining at least two second pixels corresponding to the first pixel and determining the light emission angle of the second pixels in step 1202.

[0130] In one embodiment, assuming the display panel of the screen is through Figure 3C With the structure shown, since the light emission angle of each pixel unit in the display panel can be adjusted as needed, in this embodiment, at least two pixels can be selected as the second pixels corresponding to the first pixel in a preset area centered on the third pixel corresponding to the first pixel in the second user interface.

[0131] Optionally, the third pixel corresponding to the first pixel may be, for example, a pixel in the second user interface that has the same position as the first pixel, or a pixel that has a positional relationship with the first pixel.

[0132] For ease of processing, the third pixel and at least one pixel adjacent to the third pixel can be selected as the second pixel corresponding to the first pixel. The size of the aforementioned preset area can be preset, for example, it can be a circular area centered on the third pixel, and the radius of the circular area can be preset. The specific size is not limited in this embodiment. In some embodiments, the radius of the circular area can be determined based on the proportion (e.g., 5%) of the circular area in the second user interface.

[0133] For example, assuming the first pixel is the pixel at coordinates (300, 150) in the first user interface, then the pixel at coordinates (300, 150) in the second user interface can be the third pixel mentioned above. The third pixel at (300, 150) in the second user interface and its adjacent pixel, such as the pixel at coordinates (300, 151), can be selected as the second pixel mentioned above.

[0134] It is understood that in the embodiments of this application, the resolution of the first user interface may be lower than that of the second user interface, so that a pixel in the first user interface can correspond to at least two pixels in the second user interface; or, in the embodiments of this application, some pixels in the first user interface may be discarded without affecting the imaging effect as much as possible, so that a pixel in the first user interface can correspond to at least two pixels in the second user interface.

[0135] For example, when the resolution of the first user interface is lower than that of the second user interface, the third pixel, which serves as the center point of the preset area, can be a pixel in the second user interface that corresponds to the first pixel, determined based on the difference in resolution. For example, if the resolution of the second user interface is twice that of the first user interface, the coordinates of the third pixel corresponding to the first pixel (x, y) can be (2x, 2y), so that each pixel in the first user interface can obtain a corresponding second pixel in the second user interface.

[0136] For example, if the resolution of the first user interface is the same as the resolution of the second user interface, and the pixel at coordinates (300, 151) in the second user interface has already been used as the second pixel at coordinates (300, 150) in the first user interface, then the pixel at coordinates (300, 151) in the first user interface can be discarded, that is, its corresponding second pixel is not determined in the second user interface.

[0137] When determining the light emission angle of each second pixel, the electronic device can obtain the user's refractive power, axial length, and the coordinates of the user's first pupil center. It can be understood that the refractive power, axial length, and the coordinates of the user's first pupil center are based on the parameters of the same eye. The refractive power, axial length, and pupil center coordinates of the user's left eye, and the refractive power, axial length, and pupil center coordinates of the user's right eye can be obtained separately. Here, we take the refractive power, axial length, and pupil center coordinates of one eye as an example.

[0138] The user's refractive error can be obtained by electronic devices measuring the user's eyes, or by electronic devices providing an input interface for the user, based on the data entered by the user on the input interface.

[0139] The axial length of a user's eye can be measured by an electronic device, or it can be obtained by an electronic device providing an input interface for the user based on the data entered by the user on the input interface, or an empirical value of the axial length corresponding to the refractive power can be used.

[0140] The coordinates of the center of the user's first pupil can be obtained by measuring the user's eyes using an electronic device. For specific measurement methods, please refer to the corresponding description in the embodiment shown in Figure 7 above, which will not be repeated here.

[0141] The electronic device can determine the coordinates of the fourth pixel in the virtual first user interface based on the user's refractive power, axial length, and the coordinates of the center of the first pupil, and also determine the coordinates of the second pixel. Based on the coordinates of the fourth pixel and the second pixel, the light emission angle of the second pixel is determined.

[0142] In this embodiment, the virtual first user interface is a first user interface derived by reversing the image of the first user interface obtained on the user's retina; the fourth pixel is a pixel in the virtual first user interface that is at the same position as the first pixel. For example, the virtual first user interface could be... Figure 13 The interface shown is at a distance d2 from the center of the first pupil. This virtual first user interface is a hypothetical first user interface displayed by the electronic device at this location. If the electronic device displays the first user interface at this location, an image of the first user interface can be obtained on the user's retina. However, in this embodiment, the electronic device does not need to display the first user interface at this location. The fourth pixel can be, for example, pixel A in the virtual first user interface.

[0143] by Figure 13 For example, suppose that for the first pixel A0 in the first user interface, its image on the user's retina is image point a, and its corresponding pixel in the second user interface is... Figure 13 If pixel A1 is in the first pixel A0, then at least two second pixels, such as pixel A1 and pixel A2, can be determined within a preset area centered on pixel A1;

[0144] Based on the coordinates of the fourth pixel A, which is at the same position as the first pixel A0 in the virtual first user interface, and the coordinates of pixel A1, the angle between the vector pointing from the fourth pixel A to pixel A1 and the second user interface (i.e., the plane where the display screen is located or the XY plane), as well as the angle between the projection of this vector in the second user interface and the X-axis, can be calculated to obtain the light emission angle of pixel A1. Similarly, the angle between the vector pointing from the fourth pixel A to pixel A2 and the second user interface, as well as the angle between the projection of this vector in the second user interface and the X-axis, can be calculated to obtain the light emission angle of pixel A2.

[0145] The method for calculating the coordinates of the fourth pixel A is explained below:

[0146] Combination Figure 13 The coordinate system shown allows the electronic device to calculate the first distance d1 between the center of the first pupil and the center pixel unit of the display screen (corresponding to the origin O of the coordinate system) based on the coordinates of the center of the user's first pupil. The method for calculating the distance between the two coordinates is not described in detail in this embodiment.

[0147] The electronic device can determine a second distance d2 between the center of the user's first pupil and the plane containing the virtual first user interface based on the user's refractive error, axial length, and a first distance d1. Specifically, the equivalent radius of curvature R of the eye lens can be determined based on the user's refractive error, and the focal length f of the eye can be obtained based on the user's axial length. Given R, d1, and f, and combining the optical relationships of the lens, ... Figure 13 The geometric relationship shown can be used to calculate the second distance d2 between the center of the user's first pupil and the plane where the virtual first user interface is located. The specific calculation can be achieved using relevant technologies, which will not be elaborated here.

[0148] Then, the electronic device can calculate the coordinates of the fourth pixel A in the virtual first user interface based on the coordinates of the center of the first pupil, the first distance d1, and the second distance d2. The specific calculation can be implemented using relevant technologies, which will not be elaborated here.

[0149] The above embodiments illustrate that in Figure 3C The diagram illustrates a possible implementation for determining the second pixel and its emission angle under the display panel structure shown.

[0150] In another embodiment, it is assumed that the display panel of the display screen is via, for example... Figure 3B The structure shown is implemented such that each pixel has multiple pixel components, and the light emission angle of each pixel component is fixed.

[0151] In this embodiment, the electronic device can determine at least two second pixels corresponding to the first pixel based on the user's refractive error, axial length, and the first coordinate of the user's first pupil center in a preset coordinate system. It is understood that this example still uses the refractive error, axial length, and pupil center coordinates of the same eye. For specific implementation details, please refer to the aforementioned related descriptions, which will not be repeated here.

[0152] Optionally, in this embodiment, the electronic device can determine the coordinates of the fourth pixel in the virtual first user interface based on the user's refractive power, axial length, and the coordinates of the center of the first pupil; determine the coordinates of each fifth pixel in a preset area with the third pixel corresponding to the first pixel in the second user interface as the center point; determine the ideal light emission angle of the fifth pixel based on the coordinates of the fourth pixel and the fifth pixel; and select at least two fifth pixels whose possible light emission angles are relatively close to the ideal light emission angle of the fifth pixel as the second pixels corresponding to the first pixel.

[0153] Because the display screen of electronic devices passes through Figure 3B When the structure shown is implemented, each pixel includes multiple pixel components, and each pixel component has a fixed light emission angle. In other words, each pixel has multiple possible light emission angles. In this embodiment, the possible light emission angles and the ideal light emission angle of each fifth pixel can be compared, and the deviation value between the two can be calculated. At least two fifth pixels with relatively small deviation values ​​are selected as the second pixels corresponding to the first pixel A. Furthermore, for each fifth pixel as the second pixel, the light emission angle corresponding to the pixel component with the smallest deviation value between its light emission angle and its ideal light emission angle is selected as the final light emission angle of the fifth pixel, that is, the determined light emission angle of the second pixel.

[0154] The calculation methods for the coordinates of the fourth pixel and each fifth pixel can be found in the foregoing description and related technical implementations, and will not be repeated in the embodiments of this application.

[0155] like Figure 14 As shown, taking the image a of pixel A0 in the first user interface as an example, we can take pixel A1, which is at the same position as pixel A0 in the second user interface, as the center point, and search for the second pixel corresponding to the first pixel A0 in the preset area around pixel A1. Then, the pixel with the same position as the first pixel A0 in the virtual first user interface is the fourth pixel A.

[0156] The electronic device can calculate the ideal light emission angle of each pixel within the preset coordinate system based on the coordinates of the fourth pixel A in the virtual first user interface and the coordinates of each pixel within the preset area surrounding pixel A1. For example, pixels A2 and A3 in the second user interface correspond to the aforementioned fifth pixel. The ideal light emission vector of pixel A2 can be obtained by calculating the angle between the vector pointing from the fourth pixel A to pixel A2 and the plane of the second user interface (i.e., the plane of the display screen), and the angle between the projection of this vector onto the second user interface and the X-axis. Similarly, the ideal light emission vector of pixel A3 can be obtained by calculating the angle between the vector pointing from the fourth pixel A to pixel A3 and the plane of the second user interface (i.e., the plane of the display screen), and the angle between the projection of this vector onto the second user interface and the X-axis.

[0157] Because the display screen of electronic devices passes through Figure 3B In the structure shown, each pixel component of each pixel has a fixed light emission angle. In this embodiment, the possible light emission angle and the ideal light emission angle of each fifth pixel can be compared to calculate the deviation value between the two. At least two fifth pixels with relatively small deviation values ​​are selected as the second pixels corresponding to the first pixel A. From these, at least two fifth pixels with relatively small deviation values ​​between the possible light emission angle and the ideal light emission angle are selected as the second pixels. For example... Figure 15 In the image, pixels A1, A2, and A3 are all fifth pixels. Compared to the deviation between the light emission angle 33 and the ideal light emission angle 30 in pixel A3, the deviation between the light emission angle 13 and the ideal light emission angle 10 in pixel A1, and the deviation between the light emission angle 23 and the ideal light emission angle 20 in pixel A2 are relatively small. Therefore, pixels A1 and A2 can be selected as the second pixels corresponding to the first pixel A0.

[0158] Therefore, the light emission angle 13 of pixel A1 with the smallest deviation from its ideal light emission angle 10 can be taken as the final light emission angle of pixel A1, which is also the light emission angle determined by the second pixel A1; the light emission angle 23 of pixel A2 with the smallest deviation from its ideal light emission angle 20 can be taken as the final light emission angle of pixel A2, which is also the light emission angle determined by the second pixel A2.

[0159] Therefore, in step 1203, when the display screen shows the second user interface, for the pixel unit corresponding to pixel A1, the pixel component corresponding to the light emission angle 13 can be selected to emit light according to the pixel value of pixel A1, and for the pixel unit corresponding to pixel A2, the pixel component corresponding to the light emission angle 23 can be selected to emit light according to the pixel value of pixel A2.

[0160] Optionally, in this embodiment, the resolution of the first user interface may be lower than the resolution of the second user interface, so that a pixel in the first user interface can correspond to at least two pixels in the second user interface; and / or, in this embodiment, one pixel in the second user interface can correspond to two or more pixels in the first user interface. In other words, multiple different pixels in the first user interface can take the same pixel in the second user interface as their corresponding second pixel. At this time, the pixel as the second pixel can have two or more pixel values ​​and two or more light emission angles. In step 1203, the two or more pixel components of the second pixel can be controlled to emit light according to the pixel values. For example, suppose pixel A1 in the second user interface is the second pixel of pixels A01 and A02 in the first user interface. Then, based on pixel A01, the pixel value 1 and the corresponding light emission angle 1 of pixel A1 can be determined. Based on pixel A02, the pixel value 2 and the corresponding light emission angle 2 of pixel A1 can be determined. When displaying the second user interface, the pixel component corresponding to the light emission angle 1 in pixel A1 can be controlled to emit light according to the pixel value 1, and the pixel component corresponding to the light emission angle 2 can emit light according to the pixel value 2.

[0161] Optionally, when each pixel unit in the display screen has n light emission angles, each pixel in the second user interface can correspond to at most n pixels in the first user interface; in other words, it can be used as the second pixel corresponding to at most n first pixels in the first user interface.

[0162] The above embodiments illustrate that in Figure 3B The diagram illustrates a possible implementation for determining the second pixel and its emission angle under the display panel structure shown.

[0163] This application also provides an electronic device, including a processor and a memory, wherein the processor is used to implement the method provided in this application.

[0164] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method provided in this application.

[0165] This application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to perform the method provided in this application.

[0166] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0167] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0169] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0170] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for displaying an interface, characterized in that, Applied to electronic devices, the method includes: Get the first user interface to be displayed; Based on the first user interface, determine the pixel value and light emission angle of each pixel in the second user interface; The second user interface is displayed according to the pixel value and light emission angle of each pixel in the second user interface, so that when the user views the second user interface displayed by the electronic device, the image of the first user interface is obtained on the user's retina; The determination of the pixel value and light emission angle of each pixel in the second user interface includes: For a first pixel in the first user interface, at least two second pixels corresponding to the first pixel are determined in the second user interface; The pixel value of the first pixel is used as the pixel value of the second pixel, and the light emission angle of the second pixel is determined. The step of determining the light emission angle of the second pixel includes: Obtain the user's refractive error, axial length, and coordinates of the center of the first pupil; The coordinates of the fourth pixel in the virtual first user interface are determined based on the user's refractive error, axial length, and the coordinates of the center of the first pupil; the virtual first user interface is a first user interface derived by inversely constructing an image of the first user interface on the user's retina; the fourth pixel is a pixel in the virtual first user interface that is at the same position as the first pixel. Determine the coordinates of the second pixel; The light emission angle of the second pixel is determined based on the coordinates of the fourth pixel and the coordinates of the second pixel.

2. The method according to claim 1, characterized in that, The step of determining at least two second pixels corresponding to the first pixel in the second user interface includes: In a preset area centered on the third pixel in the second user interface, at least two pixels are selected as the second pixels corresponding to the first pixel, and the position of the third pixel in the second user interface corresponds to the position of the first pixel in the first user interface.

3. The method according to claim 1 or 2, characterized in that, Determining the coordinates of the fourth pixel in the virtual first user interface based on the user's refractive error, axial length, and the coordinates of the center of the first pupil includes: Determine the first distance from the center of the first pupil to the center of the display screen based on the coordinates of the center of the first pupil. The second distance between the center of the first pupil and the plane where the virtual first user interface is located is determined based on the user's refractive error, axial length, and the first distance. The coordinates of the fourth pixel in the virtual first user interface are calculated based on the coordinates of the center of the first pupil, the first distance, and the second distance.

4. The method according to claim 1 or 2, characterized in that, The step of displaying the second user interface according to the light emission angle of each pixel in the second user interface includes: For each pixel, the pixel unit corresponding to the pixel is controlled to emit light according to the pixel value of the pixel; and the control voltage of the pixel unit corresponding to the pixel is determined according to the light emission angle of the pixel, the control voltage being used to control the light emission angle of the pixel unit.

5. A method for displaying an interface, characterized in that, Applied to electronic devices, the method includes: Get the first user interface to be displayed; Based on the first user interface, determine the pixel value and light emission angle of each pixel in the second user interface; The second user interface is displayed according to the pixel value and light emission angle of each pixel in the second user interface, so that when the user views the second user interface displayed by the electronic device, the image of the first user interface is obtained on the user's retina; The determination of the pixel value and light emission angle of each pixel in the second user interface includes: For a first pixel in the first user interface, at least two second pixels corresponding to the first pixel are determined in the second user interface; The pixel value of the first pixel is used as the pixel value of the second pixel, and the light emission angle of the second pixel is determined. The step of determining at least two second pixels corresponding to the first pixel in the second user interface includes: Obtain the user's refractive error, axial length, and coordinates of the center of the first pupil; The coordinates of the fourth pixel in the virtual first user interface are determined based on the user's refractive error, axial length, and the coordinates of the center of the first pupil; the virtual first user interface is a first user interface derived by inversely constructing an image of the first user interface on the user's retina; the fourth pixel is a pixel in the virtual first user interface that is at the same position as the first pixel. Determine the coordinates of each fifth pixel in a preset area centered on the third pixel in the second user interface; the position of the third pixel in the second user interface corresponds to the position of the first pixel in the first user interface; The ideal light emission angle of the fifth pixel is determined based on the coordinates of the fourth pixel and the coordinates of the fifth pixel; Select at least two fifth pixels from the fifth pixel whose possible light emission angle has a relatively small deviation from the ideal light emission angle of the fifth pixel as the second pixel.

6. The method according to claim 5, characterized in that, Selecting at least two fifth pixels from the fifth pixel whose possible light emission angle has a relatively small deviation from the ideal light emission angle of the fifth pixel as the second pixel includes: For each second pixel, the possible light emission angle that has the smallest deviation from the ideal light emission angle of the second pixel among the possible light emission angles of the second pixel is determined as the light emission angle of the second pixel.

7. The method according to claim 5 or 6, characterized in that, Determining the coordinates of the fourth pixel in the virtual first user interface based on the user's refractive error, axial length, and the coordinates of the center of the first pupil includes: Determine the first distance from the center of the first pupil to the center of the display screen based on the coordinates of the center of the first pupil. The second distance between the center of the first pupil and the plane where the virtual first user interface is located is determined based on the user's refractive error, axial length, and the first distance. The coordinates of the fourth pixel in the virtual first user interface are calculated based on the coordinates of the center of the first pupil, the first distance, and the second distance.

8. The method according to claim 5 or 6, characterized in that, The step of displaying the second user interface according to the light emission angle of each pixel in the second user interface includes: For each pixel, the pixel component that emits light in the pixel unit corresponding to the pixel is determined according to the light emission angle of the pixel; The pixel component emits light according to the pixel value determined by the pixel control.

9. An electronic device, characterized in that, include: Processor, memory; One or more computer programs are stored in the memory, the one or more computer programs including instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the method described in any one of claims 1 to 8.

Images