A full-link low-latency display system, method, device, and medium

Abstract

The application provides a full-link low-delay display system, method, device and medium, and belongs to the technical field of computer display. The system comprises a direct connection interface module of video memory, an AI frame synchronization engine and a photonic matrix driving module, the AI frame synchronization engine is connected with the direct connection interface module of video memory and the photonic matrix driving module respectively; the direct connection interface module of video memory is used for establishing a data channel with a graphics processor and directly reading frame buffer data in the video memory of the graphics processor; the AI frame synchronization engine is used for reading frame data in the frame buffer data, predicting the generation time of a next frame image based on an LSTM model, and outputting a synchronization control signal; and the photonic matrix driving module is used for driving a display panel to display at a preset pixel response time in response to the synchronization control signal, and controlling the end-to-end display delay to be within 0.5 milliseconds.

Description

Technical Field

[0001] This invention belongs to the field of computer display technology, and more specifically relates to a low-latency display system, method, device and medium across the entire chain. Background Technology

[0002] In today's display technology field, high refresh rates and low latency have become core indicators for measuring the performance of modern high-end monitors. High refresh rates enable smoother image presentation, while low latency ensures instantaneous response between displayed content and input signals; both are crucial for enhancing the user's visual experience. However, current technological solutions face many inherent and insurmountable shortcomings in achieving these two key performance indicators.

[0003] Traditional display architectures suffer from severe latency issues during data transmission and processing. Specifically, each frame rendered by the graphics processing unit (GPU) requires up to six processing stages, including copying to system memory, video interface encoding, display scaling processor (Scaler), and timing controller (TCON). This cumbersome process inevitably generates significant cumulative latency, exceeding 2ms in tests. This undoubtedly becomes a major bottleneck for improving display performance, severely impacting the immediacy and smoothness of image display.

[0004] Existing synchronization technologies also have significant limitations. Take variable refresh rate (VRR) technologies like G-SYNC or FreeSync as examples. While they have successfully solved the screen tearing problem, these technologies are essentially passive synchronization methods. They cannot accurately predict the specific timing of frame generation, and when the frame rate fluctuates, they still produce a synchronization delay of more than 1ms, accompanied by screen jitter. This significantly degrades the visual experience for users watching dynamic scenes or playing high frame rate games.

[0005] Furthermore, the trade-off between pixel response time and motion blur has long plagued the development of display technology. To achieve a 1ms GTG (gray-to-gray response time), some technologies employ overdrive. However, while accelerating pixel response, this technology is highly prone to overshoot and artifacts, severely impacting image quality. While anti-aliasing technologies like ULMB can mitigate motion blur to some extent, they come at the cost of screen brightness, failing to deliver an ideal user experience. In conclusion, existing display technologies still face numerous unresolved issues in balancing high refresh rates and low latency, urgently requiring a novel technological solution to overcome these bottlenecks. Summary of the Invention

[0006] To address the above problems, the present invention aims to provide a low-latency display system, method, device, and medium that significantly reduces end-to-end system latency from GPU rendering to screen text output through direct memory access, AI frame time prediction, and photon matrix driving technology.

[0007] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, embodiments of this application provide an end-to-end low-latency display system. The system is disposed in a display panel, and the display panel is provided with a graphics processor. The system includes: a video memory direct connection interface module, an AI frame synchronization engine, and a photon matrix driving module. The AI ​​frame synchronization engine is connected to the video memory direct connection interface module and the photon matrix driving module, respectively. The video memory direct connection interface module is used to establish a data channel with the graphics processor and directly read the frame buffer data in the video memory of the graphics processor; The AI ​​frame synchronization engine is used to read frame data from the frame buffer, predict the generation time of the next frame image based on the LSTM model, and output a synchronization control signal. The photon matrix driving module is used to respond to the synchronization control signal, drive the display panel to display with a preset pixel response time, and control the end-to-end display delay to within 0.5 milliseconds.

[0008] In an optional implementation, the video memory direct connection module communicates with the graphics processor based on the DisplayPort 2.1 UHBR20 protocol to obtain the pointer address of the frame buffer data with a transmission bandwidth of 80Gbps.

[0009] In an optional implementation, the AI ​​frame synchronization engine is specifically used for: Read the frame data in the frame buffer, extract the historical frame rendering timestamps and GPU real-time load rate, and obtain user input device operation commands in real time; The historical frame rendering timestamps are arranged in chronological order to form a historical frame rendering timestamp sequence. The difference between adjacent timestamps is calculated to obtain the historical frame rendering time interval sequence. The min-max normalization method is used to normalize the historical frame rendering time interval sequence to generate a time feature vector. The real-time GPU load rate is normalized to generate a load feature vector; The user input device operation command is one-hot encoded to generate a vector of length n. The encoded vector is then weighted to obtain a weighted feature vector. The pre-time feature vector, load feature vector, and weighted feature vector are integrated into an input vector, which is then sequentially input into the LSTM model according to a set time step t; the input vector at each time step i is denoted as X. i ; At each time step i, the LSTM model calculates the input vector X. i and the output h of the hidden layer at the previous time step i-1 Through the forgetting gate f i The decision is made from the cell state C at the previous moment. i-1 The discarded information; the forget gate f i Through formula f i =σ(W f •[h i-1 ,X i ]+b f The calculation yields σ, where σ is the Sigmoid function, and W... f This is the weight matrix of the forget gate, b f It is the bias vector, [h i-1 ,X i This indicates that the output of the hidden layer at the previous time step and the current input are concatenated. Meanwhile, the LSTM model uses input gate i i and candidate cell state Z i Determines the input information to be added to the cell state; input gate i i =σ(W i •[h i-1 ,X i ]+b i Candidate cell state Z i =tanh(W c •[h i-1 ,X i ]+b c ), where W i W c This is the corresponding weight matrix, b i b c It is the bias vector, tanh is the hyperbolic tangent function, and cell state C i Through formula C i =f i •C i-1 +i i •Z i Update; Through output gate o i =σ(W o •[h i-1 ,X i ]+b o The information output to the hidden layer from the cell state is determined; the hidden layer outputs h. i =oi •tanh(C i ), where W o It is the weight matrix of the output gate, b o It is the bias vector; After processing the input vectors at all time steps, the hidden layer output h at the last time step. t It contains comprehensive feature information of the input feature data in the time series; h t The input is fed into a fully connected layer, which then passes the weight matrix W. fc and bias vector b fc Perform a linear transformation using the formula y pred =W fc •h t +b fc The predicted value y for the generation time of the next frame image is obtained. pred .

[0010] In one optional implementation, the LSTM model has a hidden layer dimension of 32 and a time step of 8.

[0011] In one optional implementation, the photonic matrix driving module includes an indium gallium zinc oxide thin-film transistor backplane and a microlens array; The mobility of the backplane of the indium gallium zinc oxide thin-film transistor is not less than The switching ratio is not less than ; The microlens array is disposed on the light-emitting side of the display panel, and the numerical aperture of the microlens array is not less than 0.3, and the radius of curvature of a single lens is 5μm±0.5μm.

[0012] In one optional implementation, the display panel is a quantum dot light-emitting diode panel; The photon matrix driving module is configured as follows: The response time for a driving pixel to complete a grayscale-to-grayscale switch is no greater than 0.1 milliseconds; By working in conjunction with a flicker backlight, the dynamic image response time is reduced to no more than 0.5 milliseconds.

[0013] In an optional implementation, the system further includes a scaling processor chip, wherein the output of the memory direct connection module is configured to bypass the scaling processor chip and be directly coupled to the input of the AI ​​frame synchronization engine to transmit frame buffer data to the AI ​​frame synchronization engine.

[0014] Secondly, embodiments of this application also provide an end-to-end low-latency display method, including: The frame buffer data in the graphics processor's video memory can be directly obtained through the video memory direct connection interface module; Feature extraction is performed on the frame buffer data, using frame rendering time, GPU load rate and user input commands as input features. The LSTM model is used to analyze the acquired frame data and system status to predict the generation time of the next frame image. A synchronization control signal is generated based on the generation time of the next frame image; According to the synchronization control signal, an indium gallium zinc oxide thin film transistor backplane is used to drive the quantum dot light-emitting diode panel, and a microlens array is used to focus the emitted light to drive the display panel to display images.

[0015] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the end-to-end low-latency display method as described in any of the above.

[0016] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the end-to-end low-latency display method as described in any of the above claims.

[0017] As can be seen from the above technical solutions, the present invention has the following advantages: The end-to-end low-latency display system provided in this application is installed in a display panel containing a graphics processor. It obtains the frame buffer data pointer address at high speed through a direct memory interface module based on a specific protocol, bypassing the scaling processor chip and directly transmitting it to the AI ​​frame synchronization engine. The AI ​​frame synchronization engine uses an LSTM model to accurately predict the generation time of the next frame image through a complex data processing flow and outputs a synchronization control signal. The photon matrix driving module responds to the signal and drives the quantum dot light-emitting diode panel to display quickly with components of specific performance, controlling the end-to-end display latency to within 0.5 milliseconds. This achieves a high-quality display effect with low latency and fast response, improving display performance and user experience.

[0018] The direct memory interface module in this application communicates with the graphics processor based on the DisplayPort 2.1 UHBR20 protocol, obtaining the frame buffer data pointer address with a high bandwidth of 80Gbps. It bypasses the scaling processor chip and directly transmits data to the AI ​​frame synchronization engine, significantly reducing data transmission steps and effectively lowering latency, laying the foundation for achieving end-to-end low-latency display.

[0019] The AI ​​frame synchronization engine of this application performs in-depth processing on multi-dimensional data such as historical frame rendering timestamps, GPU real-time load rate, and user input device operation commands. It uses an LSTM model to accurately predict the generation time of the next frame image. The setting of a hidden layer dimension of 32 and a time step of 8 further optimizes the prediction accuracy, thereby providing an accurate basis for display synchronization.

[0020] This application outputs a synchronization control signal based on the prediction results of the LSTM model, enabling the photon matrix driving module to respond precisely and drive the display panel to display at a preset pixel response time, thereby achieving precise synchronization between the display process and image generation, and improving the stability and smoothness of the image.

[0021] The photon matrix driving module of this application adopts an indium gallium zinc oxide thin film transistor backplane and a microlens array. The former has high mobility and high on / off ratio, while the latter has a specific numerical aperture and radius of curvature. Combined with a quantum dot light-emitting diode panel, it achieves a pixel grayscale-to-grayscale response time of no more than 0.1 milliseconds and a dynamic image response time of no more than 0.5 milliseconds, which significantly improves the display quality and reduces dynamic blur.

[0022] This application optimizes the collaborative processes of data transmission, frame generation prediction, synchronization control, and display panel driving, controlling the end-to-end display latency to within 0.5 milliseconds, comprehensively improving the performance of the display system and bringing users a smoother and clearer visual experience. Attached Figure Description

[0023] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of the structure of the end-to-end low-latency display system provided in this application.

[0025] Figure 2 A flowchart illustrating the end-to-end low-latency display method provided in this application.

[0026] Figure 3 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0027] The various embodiments of this disclosure will be described more fully in the detailed system architecture and functions of the end-to-end low-latency display system described below. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0028] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a particular feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of the foregoing.

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Please see Figure 1 The diagram shows a schematic of a low-latency end-to-end display system in a specific embodiment. The system is housed within a display panel, which is a quantum dot light-emitting diode panel and contains a graphics processor. The system includes a direct memory interface module, an AI frame synchronization engine, and a photon matrix driver module. The AI ​​frame synchronization engine is connected to both the direct memory interface module and the photon matrix driver module.

[0031] The direct memory interface module is used to establish a data channel with the graphics processor (GPU) and directly read the frame buffer data in the GPU's video memory. Specifically, the direct memory interface module communicates with the GPU based on the DisplayPort 2.1 UHBR20 protocol, using an 80Gbps transmission bandwidth to obtain the pointer address of the frame buffer data.

[0032] In addition, the system also includes a traditional scaling processor chip. The output of the video memory direct connection module is configured to bypass the scaling processor chip and be directly coupled to the input of the AI ​​frame synchronization engine, so as to transmit the frame buffer data to the AI ​​frame synchronization engine.

[0033] For example, the direct memory connection module is based on the DisplayPort 2.1 UHBR20 (80Gbps) protocol and uses a custom ASIC to implement the DP2.1 UHBR20 receiving function, negotiating a direct connection channel with graphics processors (GPUs) such as NVIDIA / AMD. This module allows the monitor to directly read frame buffer data in the GPU's video memory, bypassing the traditional data copy path and effectively reducing signal input latency from 1.5ms to <0.05ms. Specifically, it communicates with the graphics processor based on this protocol, obtaining the pointer address of the frame buffer data with an 80Gbps transmission bandwidth.

[0034] The AI ​​frame synchronization engine is used to read frame data from the frame buffer, predict the generation time of the next frame image based on the LSTM model, and output a synchronization control signal.

[0035] In a specific implementation, the AI ​​frame synchronization engine is used for: The system reads frame data from the frame buffer, extracts historical frame rendering timestamps and GPU real-time load rates, and obtains user input device operation commands in real time.

[0036] The historical frame rendering timestamps are arranged in chronological order to form a historical frame rendering timestamp sequence. The difference between adjacent timestamps is calculated to obtain the historical frame rendering time interval sequence. The min-max normalization method is used to normalize the historical frame rendering time interval sequence to generate a time feature vector.

[0037] The real-time GPU load rate is normalized to generate a load feature vector.

[0038] The user input device operation command is one-hot encoded to generate a vector of length n. The encoded vector is then weighted to obtain a weighted feature vector.

[0039] The pre-time feature vector, load feature vector, and weighted feature vector are integrated into an input vector, which is then sequentially input into the LSTM model according to a set time step t; the input vector at each time step i is denoted as X. i The time step can be set to 8.

[0040] At each time step i, the LSTM model calculates the input vector X. i and the output h of the hidden layer at the previous time step i-1 Through the forgetting gate fi The decision is made from the cell state C at the previous moment. i-1 The information is discarded in the LSTM model. The hidden layer dimension is 32. The forget gate f i Through formula f i =σ(W f •[h i-1 ,X i ]+b f The calculation yields σ, where σ is the Sigmoid function, and W... f This is the weight matrix of the forget gate, b f It is the bias vector, [h i-1 ,X i This indicates that the output of the hidden layer at the previous time step and the current input are concatenated.

[0041] Meanwhile, the LSTM model uses input gate i i and candidate cell state Z i Determines the input information to be added to the cell state; input gate i i =σ(W i •[h i-1 ,X i ]+b i Candidate cell state Z i =tanh(W c •[h i-1 ,X i ]+b c ), where W i W c This is the corresponding weight matrix, b i b c It is the bias vector, tanh is the hyperbolic tangent function, and cell state C i Through formula C i =f i •C i-1 +i i •Z i Update.

[0042] Through output gate o i =σ(W o •[h i-1 ,X i ]+b o The information output to the hidden layer from the cell state is determined; the hidden layer outputs h. i =o i •tanh(C i ), where W o It is the weight matrix of the output gate, b o It is the bias vector.

[0043] After processing the input vectors at all time steps, the hidden layer output h at the last time step. t It contains comprehensive feature information of the input feature data in the time series; h t The input is fed into a fully connected layer, which then passes the weight matrix W. fc and bias vector b fc Perform a linear transformation using the formula y pred =W fc •h t +b fc The predicted value y for the generation time of the next frame image is obtained. pred .

[0044] For example, the AI ​​frame synchronization engine incorporates a dedicated ASIC chip, including an NPU core, to run a lightweight LSTM neural network model. This model takes the rendering time of consecutive frames, GPU load rate, and user input commands as input (input_dim≥3). Specifically, in implementation, the model extracts historical frame rendering timestamps and real-time GPU load rate from the frame buffer data, and acquires user input device operation commands in real time. The historical frame rendering timestamps are arranged into a sequence in chronological order, and the difference between adjacent timestamps is calculated to obtain a rendering time interval sequence. This sequence is then normalized using a min-max normalization method to generate a time feature vector. The real-time GPU load rate is normalized to generate a load feature vector. User input device operation commands are one-hot encoded to generate a vector of length n, which is then weighted to obtain a weighted feature vector. These vectors are integrated as the input vector and sequentially input into the LSTM model according to a set time step t (e.g., t=8). The LSTM model has a hidden layer dimension of 32 and can accurately predict the generation time of the next frame with a prediction accuracy ≥92.7%. Prediction is performed every 0.1ms, changing the traditional passive synchronization to active synchronization and reducing the synchronization latency to <0.1ms.

[0045] The photon matrix driving module is used to respond to the synchronization control signal, drive the display panel to display with a preset pixel response time, and control the end-to-end display delay to within 0.5 milliseconds.

[0046] In a specific implementation, the photonic matrix driving module includes an indium gallium zinc oxide thin-film transistor backplane (IGZOTFT backplane) and a microlens array (MLA); the mobility of the indium gallium zinc oxide thin-film transistor backplane is not less than... The switching ratio is not less than The microlens array is disposed on the light-emitting side of the display panel, and the numerical aperture of the microlens array is not less than 0.3, and the radius of curvature of a single lens is 5μm±0.5μm.

[0047] The photon matrix driving module is configured as follows: The response time for a driving pixel to complete a grayscale-to-grayscale switch is no greater than 0.1 milliseconds; By working in conjunction with a flicker backlight, the dynamic image response time is reduced to no more than 0.5 milliseconds.

[0048] For example, the photon matrix driving module includes modules with high mobility (≥ The backplane includes a driving circuit for an indium gallium zinc oxide thin-film transistor (IGZO TFT) backplane and a microlens array (MLA) integrated above the panel. The IGZO TFT backplane has a mobility of not less than [missing information]. The switching ratio is not less than This achieves a gray-to-gray (GTG) response time of 0.1ms. The microlens array is positioned on the light-emitting side of the display panel, with a numerical aperture of not less than 0.3 and a radius of curvature of 5μm ± 0.5μm for each individual lens, suppressing dynamic blur (MPRT) to within <0.5ms. For example, when using a Samsung QD-OLED panel, a matching Sharp IGZOTFT backplane is used, and MLA (radius of curvature 5μm, NA=0.3) is fabricated on the encapsulation glass using a nanoimprint lithography process.

[0049] In this embodiment, a novel "direct memory connection - AI prediction - photon drive" mode is pioneered in system architecture, bypassing traditional Scaler chips, optimizing data flow, and reducing system latency. Regarding protocols, the DP interface protocol for transmitting frame data pointers is modified to reduce transmission volume, improve transmission efficiency, and lower latency. At the algorithm level, an LSTM model is applied to display synchronization to accurately predict frame generation time, achieving "frame-to-screen" synchronization and enhancing display synchronization and stability. In terms of materials and optical technology, the IGZO TFT backplane and MLA microlenses work together to balance high-speed pixel response and low dynamic blur, improving image quality and providing users with a smooth and clear display experience from all angles.

[0050] like Figure 2 As shown, the following are embodiments of the end-to-end low-latency display method provided in this disclosure. This method and the end-to-end low-latency display system of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the end-to-end low-latency display method, please refer to the embodiments of the end-to-end low-latency display system described above.

[0051] A low-latency display method across the entire link includes the following steps: S1: Directly obtain frame buffer data from the graphics processor's memory through the direct memory connection module.

[0052] In a specific implementation, the direct memory interface module operates based on the DisplayPort 2.1 UHBR20 protocol. Upon system startup, the module negotiates with the graphics processing unit (GPU) to complete link training and configuration, establishing a high-speed and stable data channel.

[0053] After the link is established, the module sends a data request command to the GPU, and the GPU responds with a pointer address to the frame buffer data. The module receives the pointer with a bandwidth of 80Gbps and reads the frame buffer data directly from the GPU memory based on the pointer, bypassing the traditional data copy path and significantly reducing data acquisition latency.

[0054] S2: Extract features from the frame buffer data, using frame rendering time, GPU load rate, and user input commands as input features. Analyze the acquired frame data and system status using an LSTM model to predict the generation time of the next frame image.

[0055] In a specific implementation, historical frame rendering timestamps are first extracted from the frame buffer data, organized into a sequence, and the difference between adjacent timestamps is calculated to obtain a time interval sequence. The minimum-maximum normalization method is used to map the time interval values ​​to the [0,1] interval to generate a time feature vector.

[0056] The GPU load rate is acquired in real time and normalized to the [0,1] interval (0-100%) to form a load feature vector. At the same time, user input device operation commands are captured, one-hot encoded, and weighted according to pre-determined weight coefficients to obtain a weighted feature vector.

[0057] The three feature vectors mentioned above are integrated into an input vector, which is then input into an LSTM model with a hidden layer dimension of 32 and a time step size of 8. The model is trained on a large amount of data. At each time step, based on the input vector and the hidden layer output from the previous time step, the cell state and hidden layer output are updated through mechanisms such as the forget gate, input gate, and candidate cell states. After processing all time steps, the hidden layer output of the last time step is linearly transformed through a fully connected layer to obtain the predicted time of the next frame image generation, which is then converted to the actual time through inverse normalization.

[0058] The prediction process for the generation time of the next frame image is as follows: The pre-time feature vector, load feature vector, and weighted feature vector are integrated into an input vector, which is then sequentially input into the LSTM model according to a set time step t; the input vector at each time step i is denoted as X. i ; At each time step i, the LSTM model calculates the input vector X. i and the output h of the hidden layer at the previous time step i-1 Through the forgetting gate f i The decision is made from the cell state C at the previous moment. i-1 The discarded information; the forget gate f i Through formula f i =σ(W f •[h i-1 ,Xi ]+b f The calculation yields σ, where σ is the Sigmoid function, and W... f This is the weight matrix of the forget gate, b f It is the bias vector, [h i-1 ,X i This indicates that the output of the hidden layer at the previous time step and the current input are concatenated. Meanwhile, the LSTM model uses input gate i i and candidate cell state Z i Determines the input information to be added to the cell state; input gate i i =σ(W i •[h i-1 ,X i ]+b i Candidate cell state Z i =tanh(W c •[h i-1 ,X i ]+b c ), where W i W c This is the corresponding weight matrix, b i b c It is the bias vector, tanh is the hyperbolic tangent function, and cell state C i Through formula C i =f i •C i-1 +i i •Z i Update; Through output gate o i =σ(W o •[h i-1 ,X i ]+b o The information output to the hidden layer from the cell state is determined; the hidden layer outputs h. i =o i •tanh(C i ), where W o It is the weight matrix of the output gate, b o It is the bias vector; After processing the input vectors at all time steps, the hidden layer output h at the last time step. t It contains comprehensive feature information of the input feature data in the time series; h t The input is fed into a fully connected layer, which then passes the weight matrix W. fc and bias vector b fc Perform a linear transformation using the formula y pred =W fc •h t+b fc The predicted value y for the generation time of the next frame image is obtained. pred .

[0059] S3: Generate a synchronization control signal based on the generation time of the next frame image.

[0060] In a specific implementation, a timing advance, such as 0.1ms, is set based on the predicted generation time of the next frame image. 0.1ms before the generation time, the system generates a pre-charge control signal and sends it to the backplane of the indium gallium zinc oxide thin-film transistor in the photonic matrix driving module to prepare for pixel driving.

[0061] When the predicted generation time is reached, the system generates precise full-frame pixel drive control signals, including information such as the color and brightness of each pixel, to accurately control the display state of the quantum dot light-emitting diode panel pixels.

[0062] S4: According to the synchronization control signal, an indium gallium zinc oxide thin film transistor backplane is used to drive the quantum dot light-emitting diode panel, and a microlens array is used to focus the emitted light to drive the display panel to display images.

[0063] In a specific implementation, the indium gallium zinc oxide thin-film transistor backplane receives a pre-charge control signal and pre-charges the pixel electrodes using its high mobility and high on / off ratio characteristics. Then, the full-frame pixel drive control signal arrives, and the backplane completes pixel voltage writing at a speed of 0.1ms, driving the quantum dot light-emitting diode panel to emit light.

[0064] The microlens array is located on the light-emitting side of the quantum dot LED panel, with a numerical aperture of not less than 0.3 and a radius of curvature of 5μm ± 0.5μm for each lens. Light emitted from the panel is focused by the microlens array, improving light emission efficiency, reducing scattering and retention, suppressing motion blur, and enhancing image clarity, thus providing users with a superior visual experience.

[0065] The end-to-end low-latency display method provided in this embodiment firstly obtains frame buffer data from the graphics processor's memory directly through a direct memory interface module, bypassing the traditional data copy path and significantly reducing signal input latency, providing a fast data foundation for subsequent processing. Secondly, using frame rendering time, GPU load rate, and user input commands as input features, an LSTM model is used for deep analysis to predict the next frame image generation time. This innovative algorithm application achieves accurate prediction, transforming traditional passive synchronization into active synchronization, effectively reducing synchronization latency, making display and image generation more time-matched, and enhancing display stability and smoothness. Furthermore, based on the predicted synchronization control signal, an indium gallium zinc oxide thin-film transistor backplane drives a quantum dot light-emitting diode panel, and a microlens array focuses the emitted light, achieving not only rapid pixel response and ensuring extremely short grayscale switching time, but also effectively suppressing motion blur, improving display quality, and providing users with a clearer, smoother, and lag-free visual experience, comprehensively improving the performance of the display system.

[0066] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0067] The end-to-end low-latency display method provided in this application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0068] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0069] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0070] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0071] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0072] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0073] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0074] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0075] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0076] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0077] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0078] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0079] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0080] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0081] The aforementioned electronic device realizes the end-to-end low-latency display method of this application by using a series of steps such as directly acquiring data through a direct connection interface of video memory, predicting frame generation time using an LSTM model, and driving the display panel with a specific backplane and microlenses. This achieves the beneficial effects of reducing signal input delay and synchronization delay, realizing fast pixel response, suppressing dynamic blur, and thus bringing users a smooth, clear and lag-free visual experience, and comprehensively improving the performance of the display system.

[0082] The storage medium provided in this application stores a program product capable of implementing a low-latency display method across the entire chain.

[0083] End-to-end low-latency display methods include: The frame buffer data in the graphics processor's video memory can be directly obtained through the video memory direct connection interface module; Feature extraction is performed on the frame buffer data, using frame rendering time, GPU load rate and user input commands as input features. The LSTM model is used to analyze the acquired frame data and system status to predict the generation time of the next frame image. A synchronization control signal is generated based on the generation time of the next frame image; According to the synchronization control signal, an indium gallium zinc oxide thin film transistor backplane is used to drive the quantum dot light-emitting diode panel, and a microlens array is used to focus the emitted light to drive the display panel to display images.

[0084] In some possible implementations, the end-to-end low-latency display method of this disclosure can be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0085] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0086] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A full-link low-latency display system, the system being provided in a display panel in which a graphic processor is provided, characterized by, The system includes: a video memory direct connection interface module, an AI frame synchronization engine, and a photon matrix driving module, wherein the AI ​​frame synchronization engine is connected to the video memory direct connection interface module and the photon matrix driving module, respectively. The video memory direct connection interface module is used to establish a data channel with the graphics processor and directly read the frame buffer data in the video memory of the graphics processor; The AI ​​frame synchronization engine is used to read frame data from the frame buffer, predict the generation time of the next frame image based on the LSTM model, and output a synchronization control signal. The photon matrix driving module is used to respond to the synchronization control signal, drive the display panel to display with a preset pixel response time, and control the end-to-end display delay to within 0.5 milliseconds; The AI ​​frame synchronization engine is specifically used for: Read the frame data in the frame buffer, extract the historical frame rendering timestamps and GPU real-time load rate, and obtain user input device operation commands in real time; The historical frame rendering timestamps are arranged in chronological order to form a historical frame rendering timestamp sequence. The difference between adjacent timestamps is calculated to obtain the historical frame rendering time interval sequence. The min-max normalization method is used to normalize the historical frame rendering time interval sequence to generate a time feature vector. The real-time GPU load rate is normalized to generate a load feature vector; The user input device operation command is one-hot encoded to generate a vector of length n. The encoded vector is then weighted to obtain a weighted feature vector. The pre-time feature vector, the load feature vector and the weighted feature vector are integrated into an input vector, and the input vector is sequentially input into the LSTM model according to a set time step t; the input vector of each time step i is denoted as X i ; At each time step i, the LSTM model calculates the input vector X. i and the output h of the hidden layer at the previous time step i 1. Through the forgetting gate f i The decision is made from the cell state C at the previous moment. i The discarded information in 1; the forget gate f i Through formula f i =σ(W f •[h i 1,X i ]+b f The calculation yields σ, where σ is the Sigmoid function, and W... f This is the weight matrix of the forget gate, b f It is the bias vector, [h i 1,X i This indicates that the output of the hidden layer at the previous time step and the current input are concatenated. Meanwhile, the LSTM model uses input gate i i and candidate cell state Z i Determines the input information to be added to the cell state; input gate i i =σ(W i •[h i 1,X i ]+b i Candidate cell state Z i =tanh(W c •[h i 1,X i ]+b c ), where W i W c It is the corresponding weight matrix, b i b c It is the bias vector, tanh is the hyperbolic tangent function, and cell state C i Through formula C i =f i •C i 1+i i •Z i Update; Through output gate o i =σ(W o •[h i 1,X i ]+b o The information output to the hidden layer from the cell state is determined; the hidden layer outputs h. i =o i •tanh(C i ), where W o It is the weight matrix of the output gate, b o It is the bias vector; After processing the input vectors at all time steps, the hidden layer output h at the last time step. t It contains comprehensive feature information of the input feature data in the time series; h t The input is fed into a fully connected layer, which then passes the weight matrix W. fc and bias vector b fc Perform a linear transformation using the formula y pred =W fc •h t +b fc The predicted value y for the generation time of the next frame image is obtained. pred .

2. The full-link low-latency display system of claim 1, wherein, The video memory direct connection module communicates with the graphics processor based on the DisplayPort 2.1 UHBR20 protocol, and obtains the pointer address of the frame buffer data with a transmission bandwidth of 80Gbps.

3. The full-link low-latency display system of claim 1, wherein, The LSTM model has a hidden layer dimension of 32 and a time step of 8.

4. The full-link low-latency display system of claim 1, wherein, The photonic matrix driving module includes an indium gallium zinc oxide thin-film transistor backplane and a microlens array; The mobility of the indium gallium zinc oxide thin film transistor backplane is not less than 10cm2 / Vs , and the on-off ratio is not less than 105 . The microlens array is disposed on the light-emitting side of the display panel, and the numerical aperture of the microlens array is not less than 0.3, and the radius of curvature of a single lens is 5μm±0.5μm.

5. The full-link low-latency display system of claim 4, wherein, The display panel is a quantum dot light-emitting diode panel; The photon matrix driving module is configured as follows: The response time for a driving pixel to complete a grayscale-to-grayscale switch is no greater than 0.1 milliseconds; By working in conjunction with a flicker backlight, the dynamic image response time is reduced to no more than 0.5 milliseconds.

6. The full-link low-latency display system of claim 1, wherein, The system also includes a scaling processor chip, and the output of the video memory direct connection module is configured to bypass the scaling processor chip and be directly coupled to the input of the AI ​​frame synchronization engine to transmit frame buffer data to the AI ​​frame synchronization engine.

7. A full-link low-latency display method, characterized by, The method employs the end-to-end low-latency display system as described in any one of claims 1 to 6; The method includes: The frame buffer data in the graphics processor's video memory can be directly obtained through the video memory direct connection interface module; Feature extraction is performed on the frame buffer data, using frame rendering time, GPU load rate and user input commands as input features. The LSTM model is used to analyze the acquired frame data and system status to predict the generation time of the next frame image. A synchronization control signal is generated based on the generation time of the next frame image; According to the synchronization control signal, an indium gallium zinc oxide thin film transistor backplane is used to drive the quantum dot light-emitting diode panel, and a microlens array is used to focus the emitted light to drive the display panel to display images.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the end-to-end low-latency display method as described in claim 7.

9. A storage medium having stored thereon a computer program, characterized in that When the computer program is executed by the processor, it implements the steps of the end-to-end low-latency display method as described in claim 7.

Images