Image display method and device, near-to-eye display equipment and readable storage medium

Abstract

The invention provides an image display method and device, near-to-eye display equipment and a readable storage medium, and relates to the technical field of terminals, the image display method is applied to the near-to-eye display equipment, the near-to-eye display equipment comprises a display screen, and the method comprises the following steps: within a first time period of first frame time, receiving and processing a frame of initial image data to be displayed, the duration of the first frame time is determined based on the number of image transmission frames of the display screen in unit time, the unit time comprises a plurality of frame times, and the first frame time is any one of the plurality of frame times; and controlling the display screen to display an image according to the target image data in a second time period of the first frame time. The process of receiving and processing one frame of initial image data and the process of displaying the target image data obtained after processing are completed in sequence in different time periods within one frame time, so that the display time delay of the image is effectively shortened, and the smear problem is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image display method, apparatus, near-eye display device and readable storage medium. Background Art

[0002] Augmented reality (AR), virtual reality (VR) and extended reality (XR) technologies have been widely used in near-eye display devices. An existing near-eye display device displays images in a manner that: after receiving and processing a frame of image data in a current frame time, the processed image data is cached to a memory, and at the same time, the previous frame of image data stored in the previous frame time of the current frame time is read from the memory and displayed.

[0003] Based on the above image display method, the image processing process and the image display process both occupy the entire frame time, and the image data processed in the current frame time needs to be displayed in the next frame time. The display delay of each frame image is relatively long and is about 1.5 times the length of the frame time. The predicted delay is about 2.5 times the length of the frame time. The longer the predicted delay, the more likely the device's posture prediction is to be inaccurate, which can easily cause the virtual screen to be unstable relative to the background space. In a display mode with three degrees of freedom (Dof) or six degrees of freedom, when the human body drives the near-eye display device to move quickly, the central pixel point of the video area in the display screen of the near-eye display device will produce severe ghosting during the rapid movement of the human body relative to the display screen, thereby affecting the visual effect. Summary of the invention

[0004] The present application provides an image display method, apparatus, near-eye display device and readable storage medium, which to a certain extent solve the technical problems of long display delay and severe ghosting in the existing image display methods.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, the present application provides an image display method, which is applied to a near-eye display device, wherein the near-eye display device includes a display screen, and the method includes: within a first time period of a first frame time, receiving and processing a frame of initial image data to be displayed to obtain target image data, wherein the length of the first frame time is determined based on the number of image transmission frames of the display screen within a unit time, wherein the unit time includes multiple frame times, and the first frame time is any one of the multiple frame times; within a second time period of the first frame time, controlling the display screen to display an image according to the target image data.

[0007] Based on the image display method provided by the present application, the near-eye display device can sequentially complete the process of receiving and processing a frame of initial image data in different time periods within the first frame time, and control the display screen to complete the image display according to the target image data obtained after processing. The process of caching and displaying image data saves one frame time compared to the prior art, thereby reducing the display delay and prediction delay of the near-eye display device. Since the proportion of the second time period for the display screen to display the image in one frame time is reduced, when the device moves quickly with the human body in multi-degree-of-freedom display modes such as 3Dof and 6Dof, the ghosting caused by the rapid movement of the central pixel point of the video area in the display screen relative to the display screen is improved.

[0008] The number of image transmission frames per unit time of the display screen is the frame rate of the display screen, and the duration of a frame time is the duration of a frame of the image that can be presented on the display screen. For example, if the frame rate of the display screen is 60fps (frames per second), it means that the number of image transmission frames per unit time of the display screen is 60, and the unit time includes 60 consecutive frame times, and the duration of each frame time is 1 / 60 second. The display delay of the display screen can be the sum of half the duration of the second period in a frame time and the duration of the first period.

[0009] Exemplarily, the near-eye display device may be a head-mounted near-eye display device such as augmented reality, virtual reality, extended reality, or mixed reality.

[0010] In a possible implementation manner of the first aspect, the duration of the second time period is less than or equal to half the duration of the first frame time.

[0011] In a possible implementation of the first aspect, the method also includes: receiving a trigger instruction of a target display mode of a near-eye display device, the trigger instruction including the degrees of freedom of the target display mode; determining, based on the degrees of freedom, a target drive timing of multiple frame times from a plurality of different pre-stored drive timings; wherein the target drive timing is used to indicate a duration of a first time period and a duration of a second time period within a corresponding frame time, and different drive timings correspond to different durations of the first time period and / or the second time period.

[0012] In this implementation, the degree of freedom of the target display mode is used to indicate whether the image generated by the near-eye display device in the target display mode can be fixed at a certain position in the space. According to the degree of freedom of the target display mode, the display delay, smear and device power consumption requirements of the display screen in the target display mode can be determined, and then the length of the first time period and the length of the second time period corresponding to the corresponding target drive timing can be determined. Among them, the length of the first time period and the length of the second time period are both positively correlated with the display delay, and the length of the second time period is positively correlated with the length of the smear.

[0013] In a possible implementation of the first aspect, if the degree of freedom is 0, the target driving timings of multiple frame times are the same, and the sum of the duration of the first time period and the duration of the second time period corresponding to the target driving timing is equal to the duration of the frame time.

[0014] In this implementation, in the 0DoF display mode, the display delay and smear requirements of the near-eye display device are relatively low, and the time for the device to process image data and the time for displaying the image can occupy the entire frame time, reducing the power consumption of the device. Exemplarily, if the length of a frame time is T, the length of the first period in the frame time corresponding to the target drive timing can be T / 3 and the length of the second period can be 2T / 3; or the length of the first period in the frame time corresponding to the target drive timing can be T / 2 and the length of the second period can be T / 2.

[0015] In a possible implementation of the first aspect, determining a target driving timing of a plurality of frame times from a plurality of pre-stored different driving timings according to the degree of freedom includes:

[0016] If the degree of freedom is greater than 0, a target driving timing of the first frame time is determined from a plurality of pre-stored different driving timings according to the posture information of the near-eye display device in a previous frame time of the first frame time.

[0017] In a possible implementation of the first aspect, determining a target driving timing of the first frame time from a plurality of pre-stored different driving timings according to posture information of the near-eye display device in a previous frame time of the first frame time includes:

[0018] According to the posture information of the near-eye display device in the previous frame time of the first frame time, the displacement of the central pixel point of the video area in the display screen in the previous frame time of the first frame time is determined, and the video area is used to display the target image data; according to the displacement and at least one preset threshold, the target drive timing corresponding to the first frame time is determined from a plurality of different pre-stored drive timings; wherein the sum of the duration of the first time period and the duration of the second time period corresponding to the target drive timing is less than the frame time, and the displacement is negatively correlated with the duration of the first time period and / or the duration of the second time period corresponding to the target drive timing.

[0019] In this implementation, generally, in a display mode with a degree of freedom greater than 0, the display delay requirement for the near-eye display device is high, and it is easy to produce ghosting. The greater the displacement of the near-eye display device in one frame time, the more serious the ghosting. Through sensors such as an inertial measurement unit in the near-eye display device, the posture information of the near-eye display device in the previous frame time of the first frame time can be detected. By determining the displacement of the central pixel point of the video area in the display screen in the previous frame time of the first frame time, the speed at which the user drives the near-eye display device to move is determined, and then the target drive timing of the first frame time is determined. The greater the displacement, the smaller the sum of the duration of the first period and the duration of the second period.

[0020] Among them, in a display mode with a degree of freedom greater than 0, the first frame time corresponding to the target drive timing includes three time periods. The target drive timing is used to instruct the near-eye display device to receive and process a frame of initial image data to be displayed in the first time period of the first frame time to obtain target image data; in the second time period of the first frame time, control the display screen to display an image according to the target image data; in the third time period of the first frame time, do not display the image or process the data, and wait until the next frame time of the first frame time to receive the next initial image data.

[0021] In a possible implementation manner of the first aspect, the color depth of the target image data is greater than the color depth of the initial image data.

[0022] For example, the initial image data may be subjected to display compensation processing by using algorithms such as DeMURA and Gamma to eliminate uneven brightness in the initial image data, so that the output target image data is compatible with human eye observation. For example, the color depth of the initial image data may be RGB8bit, and the color depth of the target image data may be RGB12bit.

[0023] In a possible implementation of the first aspect, the target image data includes multiple sub-frame image data, and the multiple sub-frame image data are determined based on multiple color depths of the target image data; after obtaining the target image data, the method also includes: caching the multiple sub-frame image data within a first time period of a first frame time.

[0024] In the existing image display method, the target image data obtained after processing in the current frame time needs to be displayed in the next frame time. Therefore, the memory needs to cache two consecutive frames of target image data in one frame time, that is, the size of the cache space occupied is twice the size of the target image data. However, in the display method provided by the basic application, the target image data stored in the first period of the first frame time can be displayed in the second period of the first frame time. Therefore, the memory only caches one frame of target image data in one frame time, that is, the size of the cache space occupied is equal to the size of the target image data. Compared with the existing image display method, it can effectively save cache space.

[0025] In a possible implementation of the first aspect, during a second period of the first frame time, a display screen is controlled to display an image according to target image data, including: during the second period of the first frame time, a plurality of sub-frame image data are read in sequence, and according to a preset pulse width dimming signal, the display screen is controlled to display the plurality of sub-frame image data in sequence; the pulse width dimming signal is used to indicate the display duration of each sub-frame image data.

[0026] In a second aspect, the present application provides an image display device, the device comprising: at least one module, wherein the at least one module is used to implement the method in any possible implementation manner of the first aspect.

[0027] In a third aspect, the present application provides a near-eye display device, comprising: a processor and a display screen, the processor being configured to run a computer program stored in a memory to implement a method in any possible implementation of the first aspect.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in any possible implementation manner of the first aspect is implemented.

[0029] In a fifth aspect, the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the method in any possible implementation manner of the first aspect.

[0030] The technical effects of the second to fifth aspects provided in the present application can refer to the technical effects of the various possible implementation methods of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A timing diagram of an existing image display method provided in an embodiment of the present application.

[0032] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0033] Figure 3 A flowchart of an image display method provided in an embodiment of the present application.

[0034] Figure 4 A driving timing and corresponding pulse width dimming signal diagram provided in an embodiment of the present application.

[0035] Figure 5 A flowchart of a method for determining a target drive timing provided in an embodiment of the present application.

[0036] Figure 6 A schematic diagram of a driving timing sequence 1 provided in an embodiment of the present application.

[0037] Figure 7 A schematic diagram of a second driving sequence provided in an embodiment of the present application.

[0038] Figure 8 A schematic diagram of a driving timing sequence three provided in an embodiment of the present application.

[0039] Fig. 9 A schematic diagram of a driving timing sequence 4 provided in an embodiment of the present application.

[0040] Fig.10 A driving timing diagram corresponding to two consecutive frame times provided in an embodiment of the present application.

[0041] Fig.11 A flowchart of another method for determining target drive timing provided by an embodiment of the present application.

[0042] Fig.12 A schematic diagram of the movement of a video area in a display screen provided in an embodiment of the present application.

[0043] Fig.13 A flowchart of another method for determining target drive timing provided by an embodiment of the present application.

[0044] Fig.14 Another driving timing diagram corresponding to two consecutive frame times provided in an embodiment of the present application.

[0045] Fig.15 A schematic block diagram of an image display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings and related embodiments in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in a "or" relationship.

[0047] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0048] In the embodiments of the present application, the words "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0049] The steps involved in the image display method provided in the embodiments of the present application are only examples. Not all steps are mandatory steps, or not all information or messages are mandatory. They can be increased or decreased as needed during use. In the embodiments of the present application, the same step or steps or messages with the same function can be referenced and learned from each other in different embodiments.

[0050] Different display functions or installed applications of near-eye display devices have different display modes, and different display modes correspond to different degrees of freedom. For example, display modes such as air projection mode, auxiliary screen mode, and head-following mode have 0 DoF. In such display modes, the image generated by the near-eye display device can rotate with the movement of the human head, and the coordinates of the central pixel point of the video area in the display screen remain unchanged, that is, the central coordinates of the image displayed in the video area in the display screen remain unchanged; the degrees of freedom corresponding to the body-following mode can be 2 DoF, 3 DoF or 6 DoF. In such display modes, the near-eye display device can detect the spatial displacement of the device in multiple dimensions driven by the human body. During the movement of the near-eye display device, the coordinates of the central pixel point of the video area in the display screen will move rapidly relative to the display screen. Correspondingly, the central coordinates of the image displayed in the video area will move rapidly relative to the display screen as the near-eye display device moves rapidly.

[0051] See also Figure 1 As shown is a timing diagram of an existing image display method, a near-eye display device generally receives and processes a frame of image data within a current frame time, caches the processed image data to a memory, and reads a previous frame of image data processed and stored in a previous frame time from the memory and displays it.

[0052] Among them, the duration of a frame time is determined according to the frame rate when the near-eye display device displays the picture. The frame rate is the number of image transmission frames of the display screen per unit time, and the unit time includes multiple continuous frame times. The duration of a frame time is the ratio between the length of the unit time and the number of image transmission frames per unit time. The frame rate of the display screen of the near-eye display device can be greater than or equal to 60fps. For example, if the frame rate of the display screen is 72fps, that is, the number of image transmission frames per second of the display screen is 72, then the unit time includes 72 continuous frame times, and the duration of each frame time is 1 / 72 seconds.

[0053] based on Figure 1In the existing image display method shown, the near-eye display device needs to simultaneously perform the operations of receiving, processing, caching a frame of image data and reading and displaying the previous frame of image data within one frame time, wherein the process of receiving, processing and caching a frame of image data and the process of reading and displaying the previous frame of image data all occupy the entire frame time, and the image data processed within the current frame time is the image data displayed within the next frame time of the current frame time, and the display delay of each frame of image data is relatively long and is about 1.5 times of a frame time. The luminous duty cycle of the display screen of the near-eye display device is about 100%. In the 3Dof or 6Dof display mode, when the human body drives the near-eye display device to move quickly, the longer luminous duty cycle of the display screen causes the display screen of the near-eye display device to switch to the corresponding screen as the human body moves quickly. Severe smearing occurs, thereby affecting the visual effect. Among them, the luminous duty cycle of the display screen can be the ratio of the total time from the first pixel point in a frame of image data to the display completion of the last pixel point in a frame of image data to a frame time.

[0054] In addition, based on the above-mentioned image display mode, the memory needs to store two consecutive frames of image data at the same time, and the size of the cache space occupied is about twice the size of one frame of image data, and the occupied cache space is relatively large.

[0055] In order to solve the above technical problems, the embodiments of the present application provide an image display method, device, near-eye display device and readable storage medium. The near-eye display device can complete the reception and processing of a frame of initial image data in a first period of time within a first frame time, and control the display screen to complete the image display process according to the target image data obtained after processing in a second period of time within the first frame time, while reducing the proportion of the device processing image data in a frame time and the proportion of the image display time in a frame time, so that the target image data processed in the first frame time can be displayed in the first frame time, thereby reducing the image display delay and the device prediction delay. Because the proportion of the second period of image display on the display screen is reduced, when the device moves rapidly with the human body in multi-degree-of-freedom display modes such as 3Dof and 6Dof, the coordinates of the central pixel point of the video area in the display screen relative to the smear caused by the rapid movement of the display screen are effectively improved.

[0056] In an embodiment of the present application, the near-eye display device may be a head-mounted electronic device such as augmented reality (AR), virtual reality (VR), extended reality (XR), or mixed reality (MR).

[0057] Figure 2is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. The near-eye display device involved in the embodiment of the present application can be Figure 2 The electronic device 200 shown is implemented.

[0058] like Figure 2 As shown, the electronic device 200 may include at least one processor 201, a communication bus 202, a memory 203, and at least one communication interface 204. Figure 2 The components of the electronic device 200 are described in detail:

[0059] The processor 201 is the control center of the electronic device 200, which can be a processing unit or a general term for multiple processing units. For example, the processor 201 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), a display processing module (display processing unit, DPU), an image signal processor (image signal processor, ISP, controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, central processing unit (central processing unit, CPU), application specific integrated circuit (application specific integrated circuit, ASIC), field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, other general-purpose processors, and / or neural-network processing unit (neural-network processing unit, NPU), etc. Among them, the general-purpose processor can be a microprocessor, or can be any conventional processor, etc. Different processing units can be independent devices, or they can be integrated in one or more processors.

[0060] The processor 201 can execute various functions of the electronic device 200 by running or executing software programs stored in the memory 203 and calling image data stored in the memory 203. For example, Figures 3 to 14 In the illustrated embodiment, the actions of the near-eye display device can be executed by the processor calling the image data in the memory.

[0061] As an embodiment, the processor 201 may include one or more CPUs, such as Figure 2CPU0 and CPU1 are shown in the figure.

[0062] As an embodiment, the electronic device 200 may include multiple processors, such as Figure 2 201 and processor 205 shown in FIG. Each of these processors may be a single-CPU processor or a multi-CPU processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0063] The communication bus 202 may include a path to transmit information between the above components. The communication bus 202 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0064] The memory 203 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 203 may exist independently and be connected to the processor 201 via the communication bus 202. The memory 203 may also be integrated with the processor 201. Among them, the memory 203 is used to store the software program and image data that execute the solution provided in the embodiment of the present application, and is controlled and executed by the processor 201.

[0065] The communication interface 204 is used to communicate with other devices or communication networks, such as Ethernet, RAN, wireless local area networks (WLAN), etc. The communication interface 204 may include a receiving unit to implement a receiving function, and a sending unit to implement a sending function.

[0066] As an embodiment, the electronic device 200 may further include an output device 206 and an input device 207 .

[0067] Among them, the output device 206 communicates with the processor 201 and can display information in a variety of ways. Exemplarily, the output device 206 can be a display, and the display is used to display images, videos, etc., for example, for displaying the target image data in the embodiment of the present application. The display may include a display screen and a pixel circuit. The display screen may adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (AMOLED), a flexible light emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc. The pixel circuit may include a column drive circuit and a row drive circuit for controlling the color and brightness of each pixel in the display screen.

[0068] In some embodiments, the electronic device 200 may include 1 or N displays, where N is a positive integer greater than 1.

[0069] The input device 207 communicates with the processor 201 and can receive input in a variety of ways. For example, the input device 207 can be a mouse, a keyboard, a touch screen device, or a sensor device. Among them, the sensor device can be an inertial measurement unit, and the inertial measurement unit (IMU) can measure the three-axis attitude angle (or angular rate) and acceleration of the electronic device 200 in three-dimensional space to determine the attitude of the electronic device 200 based on the measured data.

[0070] It should be noted that Figure 2The device structure shown does not constitute a limitation on the electronic device 200. The electronic device 200 may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components, which is not limited in the embodiments of the present application.

[0071] The image display method provided in the embodiment of the present application will be exemplarily described below with reference to the accompanying drawings and specific application scenarios.

[0072] In one possible implementation, see Figure 3 The flowchart of an image display method shown is applied to the above-mentioned near-eye display device, mainly involving a process in which the near-eye display device processes the received initial image data and controls the display screen to display the image within a frame time, and the method includes steps S310 to S320. Each step is described in detail below.

[0073] S310, within the first period of the first frame time, receiving and processing a frame of initial image data to be displayed to obtain target image data, the duration of the first frame time is determined based on the number of image transmission frames of the display screen within a unit time, the unit time includes multiple frame times, and the first frame time is any one of the multiple frame times.

[0074] Among them, the number of image transmission frames of the display screen per unit time is the frame rate of the display screen. Exemplarily, the frame rate of the display screen of the near-eye display device can be greater than or equal to 60fps, that is, the number of image transmission frames of the image per second can be greater than or equal to 60. A unit time includes multiple consecutive frame times, and the length of a frame time is the ratio between the length of the unit time and the number of image transmission frames per unit time. Exemplarily, if the frame rate of the display screen is 72fps, that is, the number of image transmission frames of the display screen in 1 second is 72, then the length of a frame time is 1 / 72 seconds.

[0075] In one example, the initial image data may be image data corresponding to an image of a virtual space scene and / or a real space scene acquired by the near-eye display device through an image acquisition device. The image acquisition device may be a camera installed in the near-eye display device, or a camera connected to the near-eye display device via a wired or wireless connection.

[0076] In another example, the initial image data may also be image data sent to the near-eye display device by other electronic devices through wired or wireless transmission. Exemplarily, the other electronic devices may be mobile phones, tablet computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), desktop computers, and other electronic devices.

[0077] In one embodiment, in order to improve the color richness and brightness of the displayed image, the near-eye display device may perform display compensation processing on the received initial image data through a digital image processing algorithm to obtain target image data.

[0078] Exemplarily, the digital image processing algorithm may include but is not limited to compensation algorithms such as DeMURA and Gamma.

[0079] The color depth of the target image data may be greater than the color depth of the initial image data. Color depth is also called bit depth. If the color depth of the target image data is RGB (red green blue) n bits (binary digit, bit), it means that the number of binary digits occupied by the pixel value of each pixel point in the target image indicated by the target image data is n, and n can be an integer greater than or equal to 3. The larger the n value, the higher the color richness and more uniform the brightness of the image.

[0080] For example, the color depth of the initial image data may be RGB3 bits, and the color depth of the corresponding target image data may be RGB5 bits; or the color depth of the initial image data may be RGB8 bits, and the color depth of the corresponding target image data may be RGB12 bits; or the color depth of the initial image data may be RGB8 bits, and the color depth of the corresponding target image data may be RGB16 bits. The embodiments of the present application do not impose any limitation on the specific color depths of the initial image data and the target image data.

[0081] In an embodiment of the present application, the target image data may include multiple sub-frame image data, and the multiple sub-frame image data are determined according to multiple color depths of the target image data. Specifically, according to the color depth of the target image data, the pixel value of each pixel in the image can be converted into an n-bit binary number, and different pixel values ​​can be converted into different n-bit binary numbers. Among them, the n values ​​in the n-bit binary number correspond to n bits, and the values ​​located at the same bit in the binary numbers corresponding to all the pixels in the image can constitute a sub-frame image data. Accordingly, the RGBnbit target image data may include n sub-frame image data corresponding to n bits, and each sub-frame image data includes the binary values ​​of all the pixels in the image at the corresponding bit.

[0082] For example, if the color depth of the initial image data is RGB3bit, the color depth of the target image data obtained after processing the initial image data can be RGB5bit, and the pixel value of each pixel in the target image indicated by the target image data can be converted into a 5-bit binary number. For example, if the pixel value of a pixel is 31, the pixel value can be converted into a 5-bit binary number 11111; if the pixel value of another pixel is 16, the pixel value can be converted into a 5-bit binary number 10000.

[0083] Optionally, in step S310, see Figure 4 According to the driving timing shown in , after obtaining the target image data, multiple sub-frame image data in the target image data can be cached in the memory within the first period of the first frame time. Exemplarily, the near-eye display device can process the initial image data line by line and pixel by pixel, and store each sub-frame image data in the target image data obtained after processing in the memory line by line and pixel by pixel.

[0084] S320, controlling the display screen to display an image according to the target image data in a second period of the first frame time.

[0085] In one implementation, during the second period of the first frame time, multiple sub-frame image data can be read from the memory in sequence, and according to a preset pulse width dimming signal, the display screen is controlled to sequentially display the multiple sub-frame image data, wherein the pulse width dimming signal is used to indicate whether each sub-frame image data is displayed and the display duration of each sub-frame image data. The target image data stored in the first period of the first frame time can be read and displayed in the second period of the first frame time. Therefore, the memory only caches one frame of target image data in one frame time, that is, the size of the occupied cache space is equal to the size of the target image data. Compared with the existing image display method that requires a cache space twice the size of the target image data in one frame time, the cache space can be effectively saved.

[0086] As an example but not a limitation, the pixel circuit in the near-eye display device can read and display the n sub-frame image data corresponding to the n bits in the target image data in sequence from high bit to low bit. Specifically, for each sub-frame image data, the binary value corresponding to each pixel point can be read from the memory row by row and pixel by pixel, and the binary value corresponding to each pixel point is 0 or 1. If the binary value corresponding to the pixel point is 1, the pixel point on the display screen is controlled to be in a lit state, and if the binary value corresponding to the pixel point is 0, the pixel point on the display screen is controlled to be in an unlit state.

[0087] In one embodiment, during the image display process, the near-eye display device may control the display screen to display n sub-frame image data in the target image data according to the display brightness level of the display screen set by the user.

[0088] Specifically, the n sub-frame image data may correspond to different display durations, and any one or more of the n sub-frame image data may be displayed in sequence, so that the display screen has different brightness levels. Therefore, the corresponding pulse width dimming signal may be determined according to the display brightness level of the display screen set by the user, and the pulse width dimming signal may be used to control whether to display multiple sub-frame image data in the target image data and the display duration, so that the display screen reaches the brightness level set by the user.

[0089] For example, assuming that the color depth of the target image data is RGB5bit, the target image data includes 5 sub-frame image data, and the near-eye display device reads and displays the 5 sub-frame image data corresponding to the 5 bits in the target image data in sequence from high bit to low bit in the second period of the first frame time. If the display brightness level of the preset display screen is the highest, refer to Figure 4 A driving timing and a corresponding pulse width dimming signal diagram are shown, the driving timing is used to indicate the duration of the first period and the second period within a frame time, the corresponding pulse width dimming signal is high level in the second period, and the near-eye device can read and display 5 sub-frame image data from high bit to low bit in the second period according to the pulse width dimming signal, and the higher the bit, the longer the display duration of the corresponding sub-frame image data. If the preset display brightness level of the display screen is the lowest, then within the duration corresponding to the data reading and image display process of the first 4 sub-frame image data in the second period, the pulse width dimming signal is low level; within the duration corresponding to the data reading and image display process of the 5th sub-frame image data corresponding to the lowest bit, the pulse width signal is high level, and the duration of the high level of the pulse width dimming signal in the second period is equal to the display duration of the sub-frame image data corresponding to the lowest bit.

[0090] Optionally, the duration of the second period in the first frame time can be less than or equal to half of the duration of the first frame time, which can effectively reduce the display delay of the near-eye display device and alleviate the ghosting problem. At the same time, the duration of the second period is positively correlated with the display delay of the near-eye display device. The shorter the duration of displaying the target image data in the second period of the first frame time, the more significant the ghosting alleviation effect of the near-eye display device.

[0091] Based on the image display method provided by the embodiment of the present application, the processing of a frame of initial image data and the caching process of target image data, as well as the process of reading multiple sub-frame image data in the target image data line by line and pixel by pixel and displaying the image can be completed in different time periods within the first frame time, effectively shortening the time for data processing, caching, reading and displaying images, reducing the image display delay of the near-eye display device and alleviating the smear phenomenon. In addition, the target image data stored in the first time period can be read and displayed in the second time period, which can effectively save the cache space occupied by the storage of image data.

[0092] Generally, near-eye display devices have different display modes. In different display modes, near-eye display devices have different requirements for image display delay, and different requirements for the first and second time periods in each frame time. For example, in 0Dof display modes such as static viewing, air projection mode, auxiliary screen mode, and head-following mode, the coordinates of the central pixel point of the video area in the display screen remain unchanged, that is, the central coordinates of the image displayed in the video area in the display screen remain unchanged, and the display delay has a low impact on the image display effect and is not prone to ghosting. For another example, in a multi-degree-of-freedom display mode such as the body-following mode, in the process of the human head driving the near-eye display device to move quickly, the coordinates of the central pixel point of the video area in the display screen will move rapidly relative to the display screen as the device moves, and the display delay has a large impact on the image display effect and is prone to ghosting. Therefore, the near-eye display device is required to have a low image display delay and ghosting length in a multi-degree-of-freedom display mode.

[0093] As can be seen from the above, the corresponding driving timing can be matched for each frame time in the image display process based on different display modes, and then according to the driving timing, the image display method in the above steps S310 to S320 performs data processing, caching, reading and display operations on the acquired initial image data in each frame time. The driving timing can be used to indicate the duration of the first time period and the second time period in the corresponding frame time during the image display of the near-eye display device in the corresponding display mode.

[0094] Figure 5 A flowchart of a method for determining a target drive timing provided in an embodiment of the present application mainly relates to a process of determining the duration of a first period and a second period in each frame time during image display in a target display mode by a near-eye display device, and the method includes steps S510 to S520. Each step is described in detail below.

[0095] S510, receiving a trigger instruction of a target display mode of a near-eye display device, where the trigger instruction includes degrees of freedom of the target display mode.

[0096] Specifically, the near-eye display device may have one or more different display modes, and each display mode may be displayed according to the corresponding degree of freedom. In one example, one or more application software may be installed in the near-eye display device, and different application software corresponds to different display modes. When the user starts one of the target application software among the multiple application software, a trigger instruction corresponding to the target display mode of the target application software may be triggered, and the processor in the near-eye display device may display the image in the target display mode according to the degree of freedom corresponding to the target display mode in the trigger instruction.

[0097] The degrees of freedom of the target display mode can be used to indicate the degrees of freedom direction in which the displayed content can move when the near-eye display device moves in the target display mode. The target display mode can have 0DoF or multiple degrees of freedom, and the multiple degrees of freedom can be 2DoF, 3DoF, 6DoF or others. The degrees of freedom direction corresponding to each degree of freedom can be any one or more of the rolling direction, pitch direction, heading direction, vertical direction, left-right direction and front-back direction. Among them, the rolling direction is the direction in which the near-eye display device rotates around the X-axis in the preset spatial coordinate system, the pitch direction is the direction in which the near-eye display device rotates around the Y-axis in the preset spatial coordinate system, the heading direction is the direction in which the near-eye display device rotates around the Z-axis in the preset spatial coordinate system, the vertical direction is the direction in which the near-eye display device translates along the Z-axis in the preset spatial coordinate system, the left-right direction is the direction in which the near-eye display device translates along the Y-axis in the preset spatial coordinate system, and the front-back direction is the direction in which the near-eye display device translates along the X-axis in the preset spatial coordinate system.

[0098] For example, the target display mode of the projection application software can be a 0Dof aerial projection mode, that is, when the near-eye display device moves, the coordinates of the central pixel point of the video area in the display screen will not move with the movement of the device. For another example, the target display mode of the film and television software can be a 3Dof VR viewing mode, and the corresponding degrees of freedom directions can include rolling direction, pitching direction and heading direction, so that when the near-eye display device moves in the three degrees of freedom directions of rolling, pitching and heading, the coordinates of the central pixel point of the video area in the display screen will move relative to the display screen as the device moves.

[0099] S520, according to the degree of freedom, determine the target driving timing of multiple frame times from multiple different pre-stored driving timings, the target driving timing is used to indicate the duration of the first time period and the duration of the second time period in the corresponding frame time, and the duration of the first time period and / or the duration of the second time period corresponding to different driving timings are different.

[0100] It should be noted that the operating duration of the near-eye display device in the target display mode includes multiple continuous unit times, and each unit time includes multiple continuous frame times. When the target display mode has 0 Dof, the display content will move with the movement of the device, and the display delay has a lower impact on the display effect of the image and is not prone to smearing. Therefore, for the target display mode of 0 Dof, the processor can determine a target drive timing corresponding to the target display mode before receiving the first initial image data, and the target drive timing corresponding to all frame times within the operating duration range in the target display mode is the same.

[0101] However, when the degree of freedom of the target display mode is greater than 0, when the human body drives the near-eye display device to move, the display delay has a greater impact on the image display effect and is prone to produce ghosting. In addition, the speed of the near-eye display device's movement has different requirements for image display delay and the degree of impact on ghosting. Therefore, for the target display mode with a degree of freedom greater than 0 Dof, the processor can adjust the target drive timing of the next frame time according to the movement speed or displacement of the near-eye display device in one frame time.

[0102] Therefore, the above-mentioned multiple different pre-stored driving timings can be based on the data processing performance of the processor of the near-eye display device, and different display modes and different displacements of the near-eye display device in one frame time can be pre-set accordingly. Among them, the duration of the first time period in the corresponding frame time indicated by the driving timing should be adapted to the data processing performance of the processor. Specifically, the shorter the duration of the first time period, the higher the data processing performance requirements of the processor, and the greater the power consumption of the processor.

[0103] In one embodiment, the multiple driving timings pre-stored in the near-eye display device may include one or more different driving timings corresponding to 0DoF.

[0104] For example, assuming that the duration of a frame time is T, on this basis, the pre-stored multiple driving timings include two different driving timings corresponding to 0DoF. The duration of the first period in a frame time indicated by one of the driving timings may be 2T / 3, and the duration of the second period is T / 3. A schematic diagram of another driving timing is shown in FIG. Figure 6 As shown, Figure 6 The length of the first period in a frame time indicated by the driving timing 1 shown in may be T / 2, and the length of the second period is T / 2.

[0105] It should be noted that the embodiment of the present application does not impose any limitation on the specific duration of the first time period and the second time period indicated by the driving timing corresponding to 0DoF in the above example.

[0106] In another embodiment, the near-eye display device may further include a plurality of different drive timings corresponding to the multiple degrees of freedom. The plurality of drive timings correspond to different movement ranges of the near-eye display device within a frame time, and the displacement of the near-eye display device within a frame time is negatively correlated with the duration of the first time period and / or the duration of the second time period corresponding to the drive timing, that is, the greater the displacement, the shorter the total duration of the first time period and the second time period within a frame time indicated by the corresponding drive timing.

[0107] In addition, the total duration of the first time period and the second time period within a frame time indicated by the driving timing corresponding to the multiple degrees of freedom is less than the duration of a frame time, that is, one frame time may include three time periods, which are the first time period, the second time period and the third time period. The driving timing can be used to instruct the near-eye display device to cache the target image data after receiving and processing a frame of initial image data to be displayed within the first time period of the first frame time; read the target image data line by line and frame by frame within the second time period of the first frame time and control the display screen to display the image according to the target image data; and do not display or process the image data within the third time period of the first frame time, and wait until the next frame time of the first frame time to receive the next initial image data to be processed.

[0108] For example, assuming that the length of a frame time is T, on this basis, the pre-stored multiple driving timings include three different driving timings corresponding to multiple degrees of freedom, and the three different driving timings are driving timing 2, driving timing 3 and driving timing 4. The schematic diagram of driving timing 2 is as follows: Figure 7 As shown, the displacement range corresponding to the driving timing sequence 2 is the range in which the displacement of the near-eye display device within one frame time is less than or equal to the first threshold value. The duration of the first period within one frame time indicated by the driving timing sequence 2 can be 5T / 12, the duration of the second period can be 5T / 12, and the duration of the third period is T / 6. The timing diagram of the driving timing sequence 3 is shown in FIG. Figure 8 As shown, the displacement range corresponding to the driving timing sequence 3 is the range in which the displacement of the near-eye display device within one frame time is greater than the first threshold and less than the second threshold. The duration of the first period within one frame time indicated by the driving timing sequence 3 may be T / 3, the duration of the second period may be T / 3, and the duration of the third period is T / 3. The timing diagram of the driving timing sequence 4 is shown in FIG. Fig. 9 As shown, the displacement range corresponding to the driving timing sequence 4 is a range in which the displacement of the near-eye display device within one frame time is greater than or equal to the second threshold value, and the duration of the first period within one frame time indicated by the driving timing sequence 4 may be T / 4, the duration of the second period may be T / 4, and the duration of the third period may be T / 2. The first threshold value is less than the second threshold value.

[0109] It should be noted that the embodiments of the present application do not impose any limitation on the number of thresholds corresponding to different movement ranges in the above examples, the value of each threshold, and the specific duration of the first time period and the second time period indicated by the driving timing corresponding to the display mode with multiple degrees of freedom.

[0110] In one implementation, after the processor of the near-eye display device receives a trigger request for a target display mode, if the degree of freedom of the target display mode in the trigger request is 0, it can be determined that during the process of the near-eye display device displaying an image in the target display mode, the target drive timings corresponding to all frame times within the operating duration range are the same, and the sum of the duration of the first time period and the duration of the second time period corresponding to the target drive timing is equal to the duration of the frame time.

[0111] As an example but not limitation, if the target display mode has 0 Dof, it is determined that during the operation of the near-eye display device in the target display mode, the target drive timing corresponding to all frame times can be Figure 6 On this basis, if the preset pulse width dimming signal indicates that all sub-frame image data of each target image data is to be displayed, and the duration of each frame time is T, the driving timing of the near-eye display device corresponding to two consecutive frame times can be as follows: Fig.10 As shown, wherein, within any two consecutive frame times (i.e., the first frame time and the second frame time, the second frame time being the next frame time of the first frame time) of all frame times within the operating duration range of the target display mode, the target drive timing is used to indicate that when the near-eye display device is operating in the target display mode of 0 DoF, within the first period of the first frame time being T / 2, after receiving and processing a frame of initial image data to be displayed, the obtained target image data is cached, and within the second period of the first frame time being T / 2, all sub-frame image data in the target image data are read line by line and frame by frame in sequence, and the display screen is controlled to display the image. After the first frame time ends, no image data is stored in the memory. After the second frame time is T / 2 in the first period of the second frame time being T / 2, after receiving and processing another frame of initial image data to be displayed, another obtained target image data is cached, and within the second period of the second frame time being T / 2, all sub-frame image data in the target image data cached in the first period of the second frame time are read line by line and frame by frame in sequence, and the display screen is controlled to display the image.

[0112] In another implementation, after receiving a trigger request for a target display mode, if the target display mode in the trigger request has multiple degrees of freedom, the near-eye display device may determine the target drive timing for the first frame time from a plurality of pre-stored different drive timings according to the posture information of the near-eye display device in the previous frame time during the process of displaying an image in the target display mode. Fig.11 As shown in the method flow chart, the target driving timing can be determined by the method in steps S1110 to S1120. Each step is described in detail below.

[0113] S1110, determining a displacement of a central pixel point of a video area in a display screen within a frame time preceding the first frame time according to posture information of a near-eye display device within a frame time preceding the first frame time.

[0114] The posture information is used to indicate the position and posture of the near-eye display device in space, and may specifically include the three-axis acceleration and three-axis angular velocity of the device. For example, the posture information of the device may be detected in real time by an inertial measurement unit or other sensor device provided in the near-eye display device.

[0115] The video area in the video screen of the near-eye display device is used to display the target image data. When the target display mode of the near-eye display device has multiple degrees of freedom, the size of the video area in the display screen can remain unchanged during the process of the human body driving the near-eye display device to move, but the central pixel point of the video area in the display screen will move with the movement of the device, and accordingly, the central pixel point of the target image data displayed in the video area will also move. Therefore, in an embodiment of the present application, by detecting the posture information of the near-eye display device in the previous frame time of the first frame time, the spatial movement direction and spatial displacement of the near-eye display device in the previous frame time of the first frame time in the world coordinate system can be first determined, and then according to the mapping relationship between the world coordinate system and the display screen coordinate system, the spatial movement direction and the spatial displacement, the displacement of the central pixel point of the video area in the display screen in the previous frame time of the first frame time, and the movement direction of the central pixel point of the video area in the display screen corresponding to the first frame time relative to the central pixel point of the video area in the display screen corresponding to the previous frame time of the first frame time are determined.

[0116] Specifically, the displacement of the central pixel point of the video area in the display screen in the frame time before the first frame time may be: the number of pixels moved by the central pixel point of the video area in the display screen corresponding to the first frame time relative to the central pixel point of the video area in the display screen corresponding to the frame time before the first frame time in the display screen coordinate system. The video area in the display screen in the first frame time is used to display the target image data cached in the first time period of the first frame time, and the video area in the display screen in the frame time before the first frame time is used to display the target image data cached in the first time period of the frame time before the first frame time.

[0117] For example, see Fig.12 Schematic diagram of the movement of a video area in a display screen shown in FIG. 1 , assuming that, during the process of displaying an image in a multi-degree-of-freedom target display mode, the near-eye display device moves rapidly with the human body in the previous frame time of the first frame time, and the video area in the display screen corresponding to the previous frame time of the first frame time is Fig.12 In the first video area shown in , in the display screen coordinate system, the coordinates of the first central pixel point of the first video area can be expressed as (x1, y1). Then, according to the posture information of the near-eye display device in the previous frame time of the first frame time, the displacement and moving direction of the central pixel point of the video area in the display screen in the previous frame time of the first frame time can be determined, and then the position of the central pixel point of the video area in the display screen corresponding to the first frame time can be determined, wherein the video area in the display screen corresponding to the first frame time is Fig.12 In the second video area shown in FIG, in the display screen coordinate system, the coordinates of the second central pixel point of the second video area can be expressed as (x2, y2), then the displacement of the second central pixel point relative to the first central pixel point can be expressed as The size of the first video area is the same as the size of the second video area.

[0118] S1120, based on the displacement amount and at least one preset threshold, determine the target driving timing corresponding to the first frame time from multiple different pre-stored driving timings, wherein the sum of the duration of the first time period and the duration of the second time period corresponding to the target driving timing is less than the frame time, and the displacement amount is negatively correlated with the duration of the first time period and / or the duration of the second time period corresponding to the target driving timing.

[0119] Specifically, by comparing the displacement of the near-eye display device in the previous frame time of the first frame time with one or more preset thresholds corresponding to multiple different drive timings corresponding to multiple degrees of freedom, the displacement range to which the displacement belongs can be determined, and then the target drive timing of the first frame time is determined to be the preset drive timing corresponding to the displacement range to which the displacement belongs. In addition, the larger the displacement of the previous frame time of the first frame time, the smaller the duration of the first time period corresponding to the target drive timing corresponding to the first frame time and / or the smaller the duration of the second time period.

[0120] In one example, it is assumed that the pre-stored multiple driving timings include three different driving timings corresponding to multiple degrees of freedom, and the three different driving timings are respectively Figures 7 to 9 The drive timing 2, drive timing 3 and drive timing 4 shown in . Fig.13 , if the displacement of the central pixel point of the video area in the display screen in the previous frame time of the first frame time is less than or equal to the first threshold, then the target driving timing of the first frame time can be determined as Figure 7 If the displacement of the central pixel point of the video area in the display screen in the previous frame time of the first frame time is greater than the first threshold and less than the second threshold, the target driving timing of the first frame time can be determined as Figure 8 If the displacement of the central pixel point of the video area in the display screen in the previous frame time of the first frame time is greater than or equal to the second threshold, the target driving timing of the first frame time can be determined as Fig. 9 The driving timing sequence 4 shown in FIG.

[0121] It should be noted that when the near-eye display device is operating in a display mode with multiple degrees of freedom, based on the different requirements for image display delay and the different impacts on ghosting caused by different moving speeds of the near-eye display device, the processor can repeatedly execute the methods in steps S1110 to S1120 above, and can determine the target drive timing for each frame time except the first frame time in the operating time, that is, determine the target drive timing for the current frame time by the displacement of the previous frame time, and then adjust the target drive timing for each frame time in real time, thereby reducing the display delay and predicted delay of the image, improving the problem of ghosting, and enhancing the visual display effect of the near-eye display device.

[0122] By way of example and not limitation, see Fig.13 The flowchart of determining the target drive timing corresponding to the first frame time is shown in FIG. Fig.14Another driving timing diagram corresponding to two consecutive frame times is shown. Assume that the target display mode has 3Dof, the preset pulse width dimming signal is used to indicate that all sub-frame image data in each target image data are displayed in sequence, the preset threshold includes a first threshold and a second threshold, and the first threshold is 10, the second threshold is 20, Fig.14 The two consecutive time frames shown in are the first frame time and the second frame time, wherein the second frame time is the next frame time of the first frame time, and the durations of the first frame time and the second frame time are both T. If the displacement of the central pixel point of the video area in the display screen in the previous frame time of the first frame time is 15, and the displacement is greater than the first threshold and less than the second threshold, then the target driving timing of the first frame time can be determined as Figure 8 The preset driving timing three shown in the figure, that is, when the near-eye display device is operating in the target display mode of 3Dof, it can receive and process a frame of initial image data to be displayed within a first time period of T / 3 of the first frame time, and then cache the obtained target image data; within a second time period of T / 3 of the first frame time, read all sub-frame image data in the target image data line by line and frame by frame in sequence and control the display screen to display the image; and within a third time period of T / 3 of the first frame time, neither process the image data nor display the image, wait until the second frame time receives new initial image data, and then display the image according to the target driving timing corresponding to the second frame time.

[0123] If it is detected that the displacement of the central pixel point of the video area in the display screen in the previous frame time (i.e., the first frame time) of the second frame time is 30, and the displacement is greater than the second threshold, then the target driving timing of the second frame time can be determined as Fig. 9 The preset driving timing four shown in the figure, that is, when the near-eye display device is operating in the target display mode of 3Dof, it can receive and process a new frame of initial image data to be displayed within a first time period of T / 4 of the second frame time, and then cache the obtained new target image data; within a second time period of T / 4 of the second frame time, read all sub-frame image data in the new target image data line by line and frame by frame in sequence and control the display screen to display the image; and within a third time period of T / 2 of the second frame time, neither process the image data nor display the image, wait until the next frame time of the second frame time to receive a new initial image data, and then display the image according to the target driving timing corresponding to the next frame time of the second frame time.

[0124] In another implementation, after receiving a trigger request for a target display mode, if the target display mode in the trigger request has multiple degrees of freedom, and the first frame time is the first time frame when the near-eye display device is running in the target display mode, that is, there is no previous frame time of the first frame time, then any one of the pre-stored multiple different drive timings that satisfies the condition that the duration of the first time period and the duration of the second time period are less than the duration of one frame time is determined as the target drive timing of the first frame time. And starting from the next frame time of the first frame time, the method in the above steps S1110 to S1120 can be used to determine the target drive timing of each frame time.

[0125] Based on the image display method provided by the embodiment of the present application, the corresponding target drive timing can be matched for different display modes of the near-eye display device, so that the near-eye display device can complete the processing and storage process of the initial image data and the process of reading the target image data obtained after processing and displaying the image in different time periods of the current time, while reducing the proportion of the time for the device to process the image data in one frame time and the proportion of the time for image display in one frame time, so that the image data processed in the first frame time can be displayed in the first frame time, thereby reducing the display delay of the image, reducing the cache space occupied by the target image data and improving the problem of ghosting. In addition, in multi-degree-of-freedom target display modes such as 3Dof and 6Dof, the corresponding target drive timing can be matched for each frame time according to the speed of movement of the near-eye display device during the image display process, further improving the image display effect when the device is running in the multi-degree-of-freedom target display mode.

[0126] The above describes the method embodiment provided by the present application, and the following describes the device embodiment provided by the present application. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment, so the content not described in detail can be referred to the method embodiment above, and for the sake of brevity, it will not be repeated here.

[0127] Fig.15 is a schematic block diagram of an image display device provided in an embodiment of the present application, and the device can be deployed in the cloud server in the aforementioned embodiment. Fig.15 As shown, the image display device 1500 may include a processing unit 1510 and a display unit 1520. The processing unit 1510 may implement corresponding processing functions.

[0128] Optionally, the image display device 1500 may further include a storage unit, which may be used to store instructions and image data. The processing unit 1510 may read the instructions and image data in the storage unit so that the device implements the aforementioned method embodiment.

[0129] Specifically, the processing unit 1510 is used to receive and process a frame of initial image data to be displayed in a first period of the first frame time to obtain target image data, the duration of the first frame time is determined based on the number of image transmission frames of the display screen in a unit time, the unit time includes multiple frame times, and the first frame time is any one of the multiple frame times. The display unit 1520 is used to control the display screen to display images according to the target image data in a second period of the first frame time.

[0130] Optionally, the duration of the second time period is less than or equal to half the duration of the first frame time.

[0131] Optionally, the processing unit 1510 is also used to receive a trigger instruction for a target display mode of the near-eye display device, the trigger instruction including the degrees of freedom of the target display mode; based on the degrees of freedom, determining a target drive timing for a plurality of frame times from a plurality of different pre-stored drive timings; wherein the target drive timing is used to indicate the duration of a first time period and a duration of a second time period within the corresponding frame time, and different drive timings correspond to different durations of the first time period and / or the second time period.

[0132] Optionally, if the degree of freedom is 0, the target driving timings of multiple frame times are the same, and the sum of the duration of the first time period and the duration of the second time period corresponding to the target driving timing is equal to the duration of the frame time.

[0133] Optionally, if the degree of freedom is greater than 0, the processing unit 1510 is further used to determine a target driving timing for the first frame time from a plurality of pre-stored different driving timings based on posture information of the near-eye display device in a previous frame time of the first frame time.

[0134] Optionally, the processing unit 1510 determines a target driving timing of the first frame time from a plurality of pre-stored different driving timings based on the posture information of the near-eye display device in the previous frame time of the first frame time, specifically including: the processing unit 1510 determines the displacement of a central pixel point of a video area in the display screen in the previous frame time of the first frame time based on the posture information of the near-eye display device in the previous frame time of the first frame time, the video area being used to display target image data; determining a target driving timing corresponding to the first frame time from a plurality of pre-stored different driving timings based on the displacement and at least one preset threshold; wherein the sum of the duration of a first time period and the duration of a second time period corresponding to the target driving timing is less than the frame time, and the displacement is negatively correlated with the duration of the first time period and / or the duration of the second time period corresponding to the target driving timing.

[0135] Optionally, the color depth of the target image data is greater than the color depth of the initial image data.

[0136] Optionally, the target image data includes multiple sub-frame image data, and the multiple sub-frame image data are determined according to multiple color depths of the target image data. After obtaining the target image data, the processing unit 1510 is further configured to cache the multiple sub-frame image data within a first period of the first frame time.

[0137] Optionally, the display unit 1520 controls the display screen to display an image according to the target image data during the second time period of the first frame time, specifically including: the display unit 1520 reads multiple sub-frame image data in sequence during the second time period of the first frame time, and controls the display screen to display multiple sub-frame image data in sequence according to a preset pulse width dimming signal; the pulse width dimming signal is used to indicate the display duration of each sub-frame image data.

[0138] It should be understood that the description of the device embodiment can refer to the above-mentioned description of the image display method embodiment, and its implementation principle and technical effect are similar to those of the above-mentioned method embodiment, which will not be repeated here.

[0139] Based on the image display methods provided in the above embodiments, the embodiments of the present application also provide the following contents:

[0140] An embodiment of the present application provides a computer program product, which includes a program. When the program is executed by an electronic device, the electronic device performs the image display method shown in the above embodiments.

[0141] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the image display method shown in the above embodiments is implemented.

[0142] An embodiment of the present application provides a chip, which includes a memory and a processor. The processor executes a computer program stored in the memory to control the above-mentioned electronic device to execute the image display method shown in the above-mentioned embodiments.

[0143] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0144] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (doubledatarate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synchlink DRAM, SLDRAM), and direct rambus RAM (DR RAM).

[0145] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described or recorded in detail in a certain embodiment, refer to the relevant description of other embodiments.

[0146] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0147] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0148] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0149] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0150] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to a large-screen device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0151] Finally, it should be noted that the above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image display method, characterized in that: Applied to a near-eye display device, the near-eye display device includes a display screen, and the method includes: In a first period of a first frame time, a frame of initial image data to be displayed is received and processed to obtain target image data, wherein the duration of the first frame time is determined based on the number of image transmission frames of the display screen in a unit time, wherein the unit time includes a plurality of frame times, and the first frame time is any one of the plurality of frame times; In a second period of the first frame time, the display screen is controlled to display an image according to the target image data.

2. The method according to claim 1, characterized in that: The duration of the second time period is less than or equal to half of the duration of the first frame time.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: receiving a trigger instruction of a target display mode of the near-eye display device, wherein the trigger instruction includes degrees of freedom of the target display mode; Determining, according to the degree of freedom, a target driving timing of the plurality of frame times from a plurality of different pre-stored driving timings; The target driving timing is used to indicate the duration of the first period and the duration of the second period in the corresponding frame time, and different driving timings correspond to different durations of the first period and / or the second period.

4. The method according to claim 3, characterized in that If the degree of freedom is 0, the target driving timings of the multiple frame times are the same, and the sum of the duration of the first period and the duration of the second period corresponding to the target driving timing is equal to the duration of the frame time.

5. The method according to claim 3, characterized in that: Determining the target driving timing of the plurality of frame times from a plurality of pre-stored different driving timings according to the degree of freedom includes: If the degree of freedom is greater than 0, a target driving timing of the first frame time is determined from a plurality of pre-stored different driving timings according to the posture information of the near-eye display device in a previous frame time of the first frame time.

6. The method according to claim 5, characterized in that The step of determining a target driving timing of the first frame time from a plurality of pre-stored different driving timings according to the posture information of the near-eye display device in a previous frame time of the first frame time includes: Determine, according to the posture information of the near-eye display device in a frame time previous to the first frame time, a displacement of a central pixel point of a video area in the display screen in a frame time previous to the first frame time, the video area being used to display target image data; Determining a target driving timing of the first frame time from a plurality of pre-stored different driving timings according to the displacement and at least one preset threshold; The sum of the duration of the first period and the duration of the second period corresponding to the target driving timing is less than the frame time, and the displacement is negatively correlated with the duration of the first period and / or the duration of the second period corresponding to the target driving timing.

7. The method according to any one of claims 1 to 6, characterized in that The color depth of the target image data is greater than the color depth of the original image data.

8. The method according to any one of claims 1 to 7, characterized in that The target image data includes a plurality of sub-frame image data, and the plurality of sub-frame image data are determined according to a plurality of color depths of the target image data; After obtaining the target image data, the method further includes: The plurality of sub-frame image data are buffered within a first period of the first frame time.

9. The method according to claim 8, characterized in that The step of controlling the display screen to display an image according to the target image data during the second period of the first frame time includes: In a second period of the first frame time, the plurality of sub-frame image data are read sequentially, and according to a preset pulse width dimming signal, the display screen is controlled to sequentially display the plurality of sub-frame image data; The pulse width dimming signal is used to indicate the display duration of each sub-frame image data.

10. An image display device, characterized in that: include: At least one module, wherein the at least one module is used to implement the method according to any one of claims 1 to 9.

11. A near-eye display device, characterized in that: include: A processor and a display screen, wherein the processor is used to run a computer program stored in a memory to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images