Chip and display system

Abstract

The application discloses a chip and a display system. The chip has a display transmission path, comprising a first processor and a second processor. The first processor is configured to: in response to receiving a first frame data input, send a frame acquisition start signal and a reference time length to the second processor; in response to the received frame data reaching a frame trigger data amount, send a frame trigger signal to the second processor. The second processor is configured to: in response to receiving the frame acquisition start signal and the reference time length, set a first frame synchronization time; in response to receiving the frame trigger signal, perform preparation work before display processing; in response to reaching the first frame synchronization time, generate a first frame synchronization signal to read the to-be-displayed data written by the first processor from the memory for display processing, and obtain display data for sending to a display module for output display. The frame trigger data amount is less than the data amount of the first image frame, and the first frame synchronization time is later than the receiving time of the frame trigger signal.

Description

Technical Field

[0001] This application relates to the field of display technology, and more specifically, to a chip and a display system. Background Technology

[0002] In recent years, large-screen entertainment media terminal devices have become increasingly common, with users increasingly favoring them as mobile displays. In some scenarios, users choose to use the display screen of a tablet device as a monitor. The tablet's System-on-Chip (SoC) can be used as a data relay for the display, meaning the SoC receives data, caches data, and then displays the data. However, this solution suffers from significant display link latency, resulting in a poor user experience. Summary of the Invention

[0003] In view of the above, this application provides the following technical solution:

[0004] The first aspect of this application provides a chip having a display transmission path, including a first processor and a second processor;

[0005] The first processor is used to receive data to be displayed input from an external device and store it in memory;

[0006] The second processor is used to read the data to be displayed from memory and process it into display data;

[0007] The first processor is configured to: in response to receiving the first frame data input, send a frame acquisition start signal and a reference duration to the second processor; in response to the received frame data amount reaching the frame trigger data amount, send a frame trigger signal to the second processor;

[0008] The second processor is configured to: set a first frame synchronization time in response to receiving the frame acquisition start signal and the reference duration; perform preparatory work before display processing in response to receiving the frame trigger signal; and generate a first frame synchronization signal in response to reaching the first frame synchronization time, so as to read the data to be displayed written by the first processor from the memory for display processing, and obtain display data for sending to the display module for output display.

[0009] The frame trigger data volume is less than the data volume of the first image frame, and the first frame synchronization time is later than the frame trigger signal reception time.

[0010] In one possible implementation, determining the reference duration includes:

[0011] The sum of the frame trigger duration and the frame processing preparation duration is determined as the reference duration;

[0012] Wherein, the frame trigger duration represents the time it takes for the first processor to write the frame trigger data corresponding to the amount of data to memory, the frame processing preparation duration represents the time it takes for the second processor to perform preparatory work before processing the data to be displayed, and the reference duration represents the time from when the first processor starts receiving frame data to when the second processor first starts processing the data to be displayed.

[0013] In one possible implementation, determining the frame trigger duration includes:

[0014] The frame trigger duration is determined based on the frame trigger data volume and the speed at which the frame data is written to memory. The frame trigger duration represents the time it takes for the first processor to write the frame trigger data volume to memory.

[0015] The amount of received frame data reaches the frame trigger data amount determination, including:

[0016] If the cumulative duration from the acquisition of the start signal from the frame reaches the frame trigger duration, it is determined that the number of received frames has reached the frame trigger data amount.

[0017] In one possible implementation, determining the first frame synchronization time includes:

[0018] The first frame synchronization time is determined based on the frame acquisition start signal and the reference duration, such that the difference between the first frame synchronization time and the time corresponding to the frame acquisition start signal is the reference duration.

[0019] In one possible implementation, both the first processor and the second processor are configured to process image frame data using a line processing method.

[0020] In one possible implementation, the configuration of the frame trigger data amount satisfies the following condition: at any given time, the amount of data to be displayed written by the first processor to the memory is greater than the amount of data to be displayed read by the second processor from the memory, including:

[0021] At any given time, there exists a frame data unit in memory that the first processor has already processed, and the frame data unit that the first processor is currently writing into memory is different from the frame data unit that the second processor is currently reading from memory; the frame data unit is the smallest data unit that the first processor and the second processor access the memory.

[0022] In one possible implementation, the configuration of the frame trigger data amount includes:

[0023] The amount of frame trigger data is determined based on the access bandwidth ratio of the first processor and the second processor, as well as the processing speed of the first processor and the second processor.

[0024] In one possible implementation, determining the frame processing preparation time includes any one of the following:

[0025] The frame processing preparation time is determined based on the preparation time required by the second processor before performing the display processing on the data to be displayed, according to historical measurements.

[0026] Based on the pre-defined mapping table of different scenarios and frame processing preparation time, and the scenario in which the second processor is currently located, the corresponding frame processing preparation time is determined.

[0027] In one possible implementation, the first processor is an image signal processor and the second processor is a data processor.

[0028] A second aspect of this application provides a display system, including a first device and a second device, wherein the first device has a display module and a system chip;

[0029] The second device is connected to the first device via a display input interface and is used to send data to be displayed to the first device. The data to be displayed is processed by the display transmission path within the system chip of the first device and then displayed and output by the display module.

[0030] The system chip includes: a first processor and a second processor;

[0031] The first processor is used to receive data to be displayed input from an external device and store it in memory;

[0032] The second processor is used to read the data to be displayed from memory and process it into display data;

[0033] The first processor is configured to: in response to receiving the first frame data input, send a frame acquisition start signal and a reference duration to the second processor; in response to the received frame data amount reaching the frame trigger data amount, send a frame trigger signal to the second processor;

[0034] The second processor is configured to: set a first frame synchronization time in response to receiving the frame acquisition start signal and the reference duration; perform preparatory work before display processing in response to receiving the frame trigger signal; generate a first frame synchronization signal in response to reaching the first frame synchronization time, so as to read the data to be displayed written by the first processor from the memory for display processing; and obtain display data for sending to the display module for output display.

[0035] The frame trigger data volume is less than the data volume of the first image frame, and the first frame synchronization time is later than the frame trigger signal reception time. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of the structure of a chip disclosed in an embodiment of this application;

[0038] Figure 2 This is a schematic diagram of the structure of a display system disclosed in an embodiment of this application;

[0039] Figure 3 This is a schematic diagram illustrating the data exchange between the ISP and DPU in a frame-based manner as disclosed in the embodiments of this application.

[0040] Figure 4 This is a schematic diagram illustrating the data exchange between the ISP and DPU according to the data volume method disclosed in the embodiments of this application;

[0041] Figure 5 This is a timing diagram of the conventional scheme disclosed in the embodiments of this application;

[0042] Figure 6 This is a timing diagram illustrating the implementation process of the scheme disclosed in the embodiments of this application;

[0043] Figure 7 This is a schematic diagram illustrating the implementation process of the display control scheme disclosed in the embodiments of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] Figure 1 This is a schematic diagram of the structure of a chip disclosed in an embodiment of this application. Figure 1 The chip shown is a chip with a display transmission path. This chip can be integrated into devices such as tablets and smart displays to process data to be displayed input from external devices via a DPIN (DisplayPort Input) interface. The chip includes a first processor 10 and a second processor 20, both of which are connected to memory 30 via an internal bus, forming a configuration as shown below. Figure 2The display transmission path shown is "External device (second device) → DPIN interface → first processor → memory → second processor → display module". The memory can be DDR (Double Data Rate SDRAM). Figure 1 The diagram only shows the connection between two processors and memory; in actual applications, there is also a communication connection between the first and second processors. Similarly, Figure 2 This only shows the sequential connection relationship of each structure or module in the display transmission path of the image data stream. In reality, there is also a communication connection between the first processor and the second processor to achieve the necessary data interaction.

[0046] The first processor receives data to be displayed from an external device and stores it in memory. The first processor may be an ISP (Image Signal Processing) processor, which has functions such as image data acquisition, format conversion, and memory writing. In this application, the first processor has different operating modes. In the first operating mode, the first processor receives image data acquired from the image sensor integrated into the device itself and processes the image data. In the second operating mode, the first processor receives data to be displayed from an external device via a DPIN interface and processes the data to obtain content suitable for display on this device. For ease of understanding, the following description will use an ISP as the first processor.

[0047] This application primarily addresses the implementation scheme for the aforementioned second working mode. Specifically, the ISP connects to external devices via a DPIN interface. When an external device (such as a game console or laptop) is connected, the ISP receives the data to be displayed and, based on the AXI bus, writes the received data to a pre-allocated buffer in memory via its internally integrated write DMA (or WDMA) module. The buffer size can be configured according to the maximum supported resolution and frame rate.

[0048] The second processor is used to read the data to be displayed from memory and process it into display data. The second processor can be a DPU (Data Processing Unit) processor, which has display processing functions such as layer compositing, scaling, and color space conversion. The DPU integrates a read DMA (or RDMA) module, which can read the data to be displayed from memory and send the processed display data to the display module (such as an LCD or OLED panel) for output display. For ease of understanding, the following description will use a DPU as the second processor.

[0049] The first processor is configured to: in response to receiving the first frame data input, send a frame acquisition start signal (SOF) and a reference duration T to the second processor. sync When an external device connects and begins outputting image data, the ISP first detects the start position of the first frame and generates a frame start signal (SOF). The SOF is a hardware pulse signal sent to the DPU via an interrupt. Simultaneously, the ISP calculates the reference duration T based on the current input signal's resolution, frame rate, and the preset frame trigger data amount. vsync T vsync This represents the total time from when the ISP starts receiving the first frame of data to when the DPU first begins processing the data to be displayed. The specific calculation method will be detailed in subsequent embodiments. The ISP will use T... vsync The value is passed to the DPU, and the DPU can configure the first frame synchronization time after reading the value.

[0050] The first processor is also configured to send a frame trigger signal (also known as the frame intermediate signal HF) to the second processor in response to the received frame data amount reaching the frame trigger data amount. The frame trigger data amount is a preset value that is less than the data amount of the first image frame, and its determination method will be described later. During the data reception process, the ISP can count the amount of data written to memory through an internal counter. When the accumulated data amount reaches the frame trigger data amount, the frame trigger signal HF is generated. HF is also sent to the DPU via a hardware signal or interrupt to trigger the DPU to start the preparatory work before display processing. In this embodiment, the setting of the frame trigger data amount must satisfy the following: the first frame synchronization time (first Vsync transmission time) is later than the reception time of the frame trigger signal HF, i.e., T vsync (The duration between sof and first Vsync) > T hf (Frame trigger duration, i.e. the time required for the ISP to write the frame trigger data) ensures that the DPU has enough time to complete the preparation work before reading the data.

[0051] To facilitate understanding, the mechanism by which the ISP triggers the synchronization signal is explained as follows:

[0052] ISPs typically provide image data in frames. The ISP firmware provides a frame completion interrupt signal. This application has the capability to output an interrupt signal based on a specified data amount (corresponding to the frame trigger data amount), and can implement a programmable mechanism for triggering synchronization signals using a specified number of lines or a specified time. For example, a timer mechanism can be used to implement triggering at a specified time, and other firmware methods can achieve similar effects. Of course, the above implementation requires that both the first processor and the second processor be configured to process image frame data in a line-based processing mode.

[0053] During normal ISP acquisition, a complete ISP frame cycle begins at SOF and ends at EOF. ISP HWs typically lack the capability to transmit the intermediate frame signal (HF) at a specified stage. This solution employs a high-precision timer to transmit the HF signal, and this timer is programmable. At SOF, the timer's runtime is set, and then it is started. When the timer expires, an interrupt signal is triggered. This interrupt signal becomes the HF signal, replacing the original EOF signal. The interval between SOF and HF signals is T. hf ,like Figure 2 As shown.

[0054] In traditional implementations, the ISP and DPU exchange data in frames, such as Figure 3 As shown, the DPU HW reads N-1 frames of image data, and the ISP HW writes N frames of image data. In this way, the ISP HW and the DPU HW cannot access the same frame of data at the same time. This exchange method has a 1-frame delay, and this delay cannot be avoided.

[0055] To reduce access latency for the ISP HW / DPU HW, this application employs HF mode for interaction. Since the ISP typically lacks the capability to output interrupt signals with a specified number of lines, HF signal triggering can be achieved by monitoring the amount of received frame data, as described above using a Timer. This satisfies the scene's accuracy requirements and allows for programmable HF trigger positions. Specifically, utilizing the Timer's programmability and software driver integration, the interrupt service routine immediately starts Timer Start upon the appearance of the SOF signal, ensuring real-time performance. Furthermore, the interval between DPU HW / ISP HW data access can be flexibly adjusted according to actual conditions, with the interval adjustable to a maximum extent, such as configurable to 2ms. In HF mode, the ISP and DPU exchange data based on data volume, such as... Figure 4 As shown.

[0056] Of course, from a timing perspective, in order to reduce the latency of the display transmission path in the chip, in addition to the ISP sending the HF signal in advance, it is also necessary to combine the adjustment of the vertical synchronization signal Vsync of the DPU, that is, to adjust the timing of DPU First Vsync. Figure 5 This is a timing diagram of the conventional solution disclosed in the embodiments of this application, combined with... Figure 5 As shown, in traditional implementations, without using frame advance transmission, the Sync feature between the ISP and DPU is achieved. The DPU receives the SOF signal and T signal sent by the ISP. vsync After generating the two parameters, the DPU generates the first synchronization signal, First V-Sync, based on the parameter information. Where T... vsync =Tframe +T sf_hwc T represents the duration from the Start of Frame (SOF) signal to the DPU issuing the First Vsync signal; frame T represents the duration of a frame interval. sf_hwc This indicates the time required for the DPU to perform preparation work before it can perform display processing.

[0057] Figure 6 This is a timing diagram illustrating the implementation process of the scheme disclosed in the embodiments of this application. (Combined with...) Figure 6 As shown, the design of this application involves sending a frame trigger signal HF in advance to cause the DPU to generate a Vsync signal after the ISP generates a frame SOF. During implementation, it is necessary to ensure that T... sf_hwc The intervals are sufficient, ensuring ample time for the software; due to the ISP sending frame data in advance, T is significantly reduced. vsync Time interval, and satisfying T hf <=T frame / 2;T vsync =T hf +T sf_hwc In applications, the timing of HF signal generation, DPUFrist V-sync timing, and the overlapping area of ​​ISP_W / DPU_R can all be adjusted according to actual conditions. In ISP_W, W represents write, and ISP_W characterizes the period during which the ISP writes data to be displayed into memory; in DPU_R, R represents read, and DPU_R characterizes the period during which the DPU reads data to be displayed from memory. The overlapping area of ​​the two indicates that the ISP's write operation and the DPU's read operation are performed synchronously during this period, thus saving processing time.

[0058] The second processor is configured to: in response to receiving the start signal (SOF) and reference duration of a frame, set the first frame synchronization time. The DPU integrates a V-sync generation module, which generates the vertical synchronization signal Vsync. When the DPU receives the SOF signal, it records the SOF time (denoted as T). sof ), and according to the received T vsync Calculate the synchronization time T of the first frame first_vsync =T sof +T vsync The DPU writes this moment into the register of the V-sync generation module. When the counter value matches the register value, the first frame synchronization signal (first Vsync) is generated. The first frame synchronization signal is a critical timing node, marking that the DPU can begin reading data from memory and performing display processing.

[0059] The second processor is also configured to perform pre-display processing preparations in response to receiving a frame trigger signal HF. Upon receiving the HF signal, the DPU immediately initiates the pre-display processing preparations, which corresponds to stage T. sf_hwc The preparation work includes, but is not limited to: configuring the RDMA read address and read data format parameters; configuring the scaling engine, color matrix, gamma correction and other processing modules according to the physical characteristics of the display module (such as resolution and refresh rate); handshaking with the display module to ensure timing matching; and completing the initialization operation of layer compositing. These preparations usually require a fixed amount of time overhead. By starting them in advance after HF triggering, the additional delay caused by waiting for Vsync to arrive before starting the configuration can be avoided.

[0060] The second processor is also configured to: generate a first-frame synchronization signal `firstVsync` in response to the arrival of the first-frame synchronization time, to read the data to be displayed written by the first processor from memory for display processing (commit), and obtain display data for sending to the display module for output display. When the counter inside the DPU reaches T... first_vsync At this time, the first frame Vsync signal is generated. The ISP has already written at least a portion of the frame trigger data to be displayed into memory. The DPU reads the data from this memory area via the RDMA module, and after processing by the corresponding modules, finally generates display data that conforms to the display module interface specification. The display data is then sent to the display module for output display via the display interface.

[0061] It should be noted that the core of this application's solution lies in optimizing the first frame synchronization time. After the first frame synchronization signal is generated, the DPU's V-sync generation module will automatically generate subsequent Vsync signals at fixed intervals, without relying on the SOF or T sent by the ISP again. vsync Dynamic adjustments can be made, and this control logic is simple and easy to implement.

[0062] This embodiment configures the first processor to send a frame acquisition start signal and reference duration to the second processor when the first frame data is input, and to send a frame trigger signal when a portion of the frame data (frame trigger data amount) is received, so that the second processor can generate the first frame synchronization signal during the first frame synchronization time. By setting the frame trigger data amount to be less than the data amount of the first image frame, the DPU can start preparation work as soon as the ISP completes the writing of part of the first frame data, which greatly shortens the latency time compared to the traditional implementation of buffering the entire frame before reading.

[0063] In one embodiment, determining the reference duration includes: setting the frame trigger duration ( Figure 6 The time period between SOF and FirstVsync (as described in the text) and the frame processing preparation time (T) sf_hwc The sum of these values ​​is used as the reference duration.

[0064] Wherein, the frame trigger duration represents the time it takes for the first processor to write the frame trigger data corresponding to the amount of data to memory, the frame processing preparation duration represents the time it takes for the second processor to perform preparatory work before processing the data to be displayed, and the reference duration represents the time from when the first processor starts receiving frame data to when the second processor first starts processing the data to be displayed.

[0065] In other words, the formula for calculating the reference duration can be expressed as T vsync = T hf +T sf_hwc T hf T is the frame trigger duration. sf_hwc Preparation time for frame processing.

[0066] Frame trigger duration T hf This is the time required for the ISP to write the frame trigger data from the moment it receives the SOF. Its physical meaning is that the DPU can only begin reading from memory normally after the ISP has written at least the frame trigger data amount, and this read operation does not conflict with the ISP's write operation. hf The size is directly proportional to the amount of frame trigger data and inversely proportional to the ISP's write bandwidth.

[0067] Frame processing preparation time T sf_hwc This is the time required for the DPU to complete all preparatory work before display processing, from receiving the HF signal. This time includes the initialization of various modules within the DPU, register configuration, bus handshake, and other operations. sf_hwc The size of this stage is related to factors such as the processing power of the DPU, the current system load, and the complexity of the software algorithm. In the Android system, this stage corresponds to the configuration process of SurfaceFlinger (SF) and Hardware Composer (HWC), including the parsing of layer information, the decision of the compositing method, and the allocation of hardware resources.

[0068] Reference duration T vsync Essentially, it's the sum of the time it takes for the ISP to write some data and for the DPU to complete its preparation work. From a timing perspective, the ISP in T... sof Data writing begins at time T hf After sending an HF signal, the DPU immediately starts preparation work upon receiving the HF signal, and after T... sf_hwc Then preparation is completed. Therefore, the earliest time that the DPU completes preparation is T. sof +T hf +T sf_hwc T vsync Setting it to this value ensures that the DPU will immediately enter the data reading phase once it is ready, without any additional waiting.

[0069] During implementation, the reference duration can be configured and debugged. sf_hwc Parameter tuning strategies may include: using statistical strategies to obtain T sf_hwc The relationship is related to CPU performance load, which can be determined using benchmark measurements and multi-condition testing. Benchmark measurements can include inserting timestamps into the DPU software processing logic and measuring the following time periods: the time it takes for RDMA to read one frame of data (depending on bandwidth and frame size) Tread; software processing time (such as the execution time of algorithms like composition and scaling) Tsw; T... sf_hwc = Tread + Tsw. Multi-condition testing can include measuring T under different scenarios. sf_hwc A parameter table is created. Key test conditions include: resolution from low to high (e.g., 720P, 1080P, 4K); CPU load: simulating different system loads (e.g., background task execution); core allocation: DPU tasks running on different CPU cores (e.g., large cores, small cores). Specifically, the parameter table construction process covers multi-dimensional test combinations: in terms of resolution, it covers common display specifications from 720P and 1080P to 4K to adapt to different data processing volumes; in terms of CPU load, it simulates a gradient load model from light to heavy load by injecting background audio playback, network transmission, and other tasks; in terms of core allocation, it binds the DPU control thread to processing cores with different processing performance (large cores, medium cores, small cores, etc.) to determine the impact of scheduling strategies on software processing time.

[0070] This embodiment clarifies that the reference duration consists of two parts: frame trigger duration and frame processing preparation duration, defining the total path time overhead from when the ISP starts receiving data to when the DPU starts processing. By setting the reference duration to the sum of the frame trigger duration and the frame processing preparation duration, it ensures that the ISP has written a sufficient amount of data before the DPU starts reading data, and that the DPU itself has completed all preparation work, avoiding data conflicts and processing delays. This timing configuration method based on actual hardware processing capabilities has higher accuracy and adaptability compared to traditional fixed-delay schemes, providing a unified theoretical framework for display optimization at different resolutions and frame rates.

[0071] In one embodiment, determining the frame trigger duration may include: determining the frame trigger duration based on the frame trigger data amount and the speed of writing the frame data into memory, wherein the frame trigger duration characterizes the time taken for the first processor to write the frame trigger data amount into memory.

[0072] The frame trigger duration, also known as the timing of HF signal generation, can be configured, adjusted, and optimized in practical applications. In implementation, the frame trigger duration can be understood as the ratio of the amount of frame trigger data to the speed at which the ISP writes the frame data into memory.

[0073] In one implementation, T hf Parameter tuning strategies may include:

[0074] 1. Currently T hf The minimum unit is 100 lines of image frames, with a valid range of [100, 1080] to ensure that the DPU and ISP do not access the same line; for example, 4K resolution is 2160, 8330us / 2160 * 100 = 384us.

[0075] 2. Clearly define the bandwidth ratio between DPU RDMA and ISP WDMA access, ensuring sufficient bandwidth in both cases. If the ratio is less than 1.0, theoretically there are no access conflicts, and 100 lines of data are sufficient. To increase security redundancy, the interval can be set to 200 lines. If the ratio is greater than or equal to 1.0, theoretically there are access conflicts, and the interval can be increased to 500 lines. In practical applications, relevant tests can be conducted to test system stability. If conflicts occur, gradually increase the interval. hf (For example, add 100 lines each time) until there are no conflicts.

[0076] In applications, the configuration of frame trigger data volume should be considered in addition to the access bandwidth ratio, taking into account the processing speeds of the ISP and DPU. Theoretically, the ISP's processing speed needs to be greater than the DPU's processing speed to ensure the stability of the final display module's output content.

[0077] Therefore, configuring the frame trigger data amount may include: determining the frame trigger data amount based on the access bandwidth ratio of the first processor and the second processor, and the processing speed of the first processor and the second processor.

[0078] The processing speed of the first processor refers to the effective data rate (in bytes per second) at which the first processor writes the data to be displayed into memory via write DMA; the processing speed of the second processor refers to the effective data rate (in bytes per second) at which the second processor actually reads the data to be displayed from memory via read DMA. The processing speed can be calculated using theoretical bus parameters during system initialization, or it can be obtained through actual measurement during runtime.

[0079] Determining that the amount of received frame data has reached the frame trigger data amount may include: if the cumulative duration from acquiring the start signal (SOF) of the frame reaches the frame trigger duration, then determining that the number of received frames has reached the frame trigger data amount.

[0080] This implementation provides a trigger-based judgment method that does not require real-time data collection. In the implementation, a high-precision timer can be configured internally within the ISP. When the SOF signal arrives, this timer is started. The timer starts counting from 0, and when the count value reaches T... hf, it is determined that the ISP has written data of a size corresponding to the configured frame trigger data volume, and an HF signal is generated at this time.

[0081] Of course, in implementation, the determination that the amount of received frame data reaches the frame trigger data volume can also be implemented in other ways, for example, the amount of written data can be determined by counting the number of rows written by the ISP. Specifically, for progressively scanned images, the amount of data can be expressed as that corresponding to N rows of pixels. For example, under 4K resolution, the number of pixels in one row is 3840. If the pixel format is RGB888 (3 bytes per pixel), the data volume of one row is 3840×3=11520 bytes. If the frame trigger data volume is configured as 200 rows, then the frame trigger data volume = 200×11520=2,304,000 bytes.

[0082] Overall, the configuration of the frame trigger data volume meets the following condition: at any time, the data volume of to-be-displayed data written by the first processor into the memory is greater than the data volume of to-be-displayed data read by the second processor from the memory. Specifically, it is required to ensure that at any time, there are processed frame data units that have been processed by the first processor in the memory, and the frame data unit currently being written into the memory by the first processor is different from the frame data unit currently being read from the memory by the second processor; the frame data unit is the minimum data unit for the first processor and the second processor to access the memory.

[0083] The minimum data unit refers to the minimum data granularity commonly adopted by the first processor ISP and the second processor DPU when accessing the memory. Under a row processing architecture, the minimum data unit can be one row of pixel data of an image; it can also be an integer multiple of rows agreed by both parties in the row processing pipeline (such as 2 rows, 3 rows), as long as both the ISP write DMA and the DPU read DMA use the minimum data unit as the address increment step, and both HF triggering and Tsync calculation count based on this unit.

[0084] Taking "row" as an example, when the ISP writes the N-th row, the DPU reads the M-th row (M<N). The frame data units accessed by the two parties are different, and there is the M-th row data that has been completely written by the ISP in the memory for the DPU to read.

[0085] This embodiment provides an implementation method for determining the frame trigger duration. Based on the ratio of the frame trigger data volume to the writing speed, the time required for the ISP to write partial data can be accurately quantified. The accuracy of the frame trigger duration can ensure that the HF signal is sent only after the ISP actually writes enough data, which provides a reliable data basis for the subsequent processing of the DPU and avoids display abnormalities caused by insufficient data.

[0086] In one embodiment, determining the first frame synchronization time includes: determining the first frame synchronization time based on the frame acquisition start signal and the reference duration, such that the difference between the first frame synchronization time and the time corresponding to the frame acquisition start signal is the reference duration.

[0087] First frame synchronization time T first_vsync The calculation formula is:

[0088] T first_vsync =T sof +T vsync

[0089] Among them, T sof T is the time when the frame acquires the start signal (SOF). vsync For reference duration.

[0090] When the ISP detects the start of the first frame input from an external device, it generates a precise SOF hardware signal. The DPU receives this signal via a dedicated interrupt pin or a shared interrupt line and records the current system timestamp, T, in the interrupt service routine. sof .

[0091] DPU obtains T sof and T vsync Then, T can be calculated. first_vsync The DPU typically contains a V-sync generation module, which includes a register. The DPU will then... first_vsync The value is converted to the corresponding counter value and written to the register. The counter of the V-sync generation module increments based on the system clock. When the count value equals the data stored in the register, the module outputs the first frame of the Vsync signal.

[0092] Traditional DPU V-sync generation is typically a periodic, self-running process. Its initial phase is determined by system reset or software configuration, independent of the timing of the external input signal, and the timing of the first V-sync can be arbitrary. However, in this embodiment, the timing of the first frame V-sync is jointly determined by the SOF timing of the external input signal and the reference duration, achieving hard synchronization between the DPU V-sync and the external signal source. This synchronization mechanism ensures that when the DPU reads data, the ISP has already written a sufficient amount of data (frame trigger data). After the first frame V-sync is generated, the DPU's V-sync generation module switches to periodic mode.

[0093] Switching to periodic mode means that after the first frame synchronization signal is generated by the DPU based on the frame acquisition start signal and reference duration provided by the ISP, the V-sync generation module inside the DPU automatically generates each subsequent V-sync signal according to the frame period (T_frame) corresponding to the external input signal, using the V-sync time of the first frame as a reference. In other words, the ISP and DPU only complete timing negotiation once in the first frame, and thereafter the DPU's V-sync generation module runs autonomously at a fixed period.

[0094] This embodiment introduces the implementation of determining the first frame synchronization time, which makes the generation of the first frame Vsync precise and controllable. By setting the first frame synchronization time to the sum of the SOF time and the reference duration, it is ensured that the ISP has completed the writing of the frame trigger data before the DPU starts reading data, and the DPU itself has completed all preparation work, thus realizing the effective and seamless connection between the ISP and the DPU accessing memory.

[0095] In one implementation, determining the frame processing preparation time may include: determining the frame processing preparation time based on historical measurements of the preparation time required by the second processor before performing the display processing on the data to be displayed.

[0096] Frame processing preparation time T sf_hwc This is the total time required for the DPU to go from receiving the HF signal to being ready to read data. In the Android display system, this stage corresponds to the processing flow of SF and HWC, and mainly includes:

[0097] Layer collection: SF collects all layer information to be composited (such as position, size, format, buffer handle).

[0098] Compositing strategy decision: HWC determines the compositing method (hardware overlay or GPU compositing) based on hardware capabilities (such as the number of overlay layers and scaling capabilities).

[0099] Register configuration: Configure the parameters of each hardware module of the DPU (such as RDMA, scaler, mixer, color processor).

[0100] Buffer mapping: Mapping the buffer of data to be displayed in memory to the address space of the DPU.

[0101] Handshake and Synchronization: Perform a timing handshake with the display module to ensure accurate data output when Vsync arrives.

[0102] T sf_hwc The length of a file is affected by a variety of factors, including CPU performance, system load, layer complexity, DPU hardware capabilities, and resolution.

[0103] The frame processing preparation time is determined based on the preparation time required by the second processor before performing the display processing on the data to be displayed, according to historical measurements. This is achieved by adjusting T in the actual operating environment. sf_hwc Perform measurements to obtain typical values ​​(such as averages) or maximum values, which can be used as a reference for subsequent configurations.

[0104] Alternatively, the determination of the frame processing preparation time includes: determining the corresponding frame processing preparation time based on a pre-calibrated mapping table of different scenarios and frame processing preparation times, and the scenario in which the second processor is currently located.

[0105] This implementation allows for the pre-measurement of T under various possible scenarios. sf_hwc Establish scene parameters to T sf_hwc The mapping table. During system runtime, the DPU looks up the corresponding T value in the table based on the currently detected scene parameters. sf_hwc The scene parameters can include: resolution, refresh rate, CPU performance status, system load, layer complexity, etc.

[0106] This embodiment provides two practical methods for determining the frame processing preparation time, implementing T sf_hwc The accuracy of the determination is crucial. Methods based on historical measurements are simple and easy to implement; methods based on mapping tables offer the advantage of rapid response. Both methods ensure that the DPU has sufficient time to complete pre-display preparations, avoiding display delays or errors due to insufficient preparation.

[0107] Figure 2 This is a schematic diagram of the structure of a display system disclosed in an embodiment of this application. See also... Figure 2 As shown, the display system includes a first device 21 and a second device 22. The first device 21 has a display module 211 and a system chip 212. The second device 22 is connected to the first device 21 via a display input interface 213 (DPIN) and is used to send data to be displayed to the first device. The data to be displayed is processed by the display transmission path in the system chip of the first device and then displayed by the display module.

[0108] The system chip 212 includes a first processor 10 and a second processor 20. The first processor 10 is used to receive data to be displayed input from an external device and store it in memory 30; the second processor 20 is used to read the data to be displayed from memory and process it into display data.

[0109] The first processor 10 is configured to: in response to receiving a first frame data input, send a frame acquisition start signal (SOF) and a reference duration (T) to the second processor. vsyncWhen the amount of received frame data reaches the frame trigger data amount, a frame trigger signal (hf) is sent to the second processor.

[0110] The second processor 20 is configured to: set the first frame synchronization time (first Vsync transmission time) in response to receiving the frame acquisition start signal (SOF) and the reference duration; and perform preparatory work (corresponding to T) before display processing in response to receiving the frame trigger signal (HF). sf_hwc (Interval); in response to the arrival of the first frame synchronization time, a first frame synchronization signal (first Vsync) is generated to read the data to be displayed written by the first processor from the memory for display processing; and display data is obtained for sending to the display module for output display.

[0111] The frame trigger data volume is less than the data volume of the first image frame, and the synchronization time of the first frame is later than the reception time of the frame trigger signal (HF).

[0112] In the display system described in this embodiment, the first device configures the first processor to send a frame acquisition start signal and reference duration to the second processor when the first frame data is input, and sends a frame trigger signal when a portion of the frame data (frame trigger data amount) is received, so that the second processor can generate the first frame synchronization signal during the first frame synchronization time. By setting the frame trigger data amount to be less than the data amount of the first image frame, the DPU starts preparation work as soon as the ISP completes the writing of part of the first frame data. Compared with the traditional implementation of buffering the entire frame and then reading it, the latency time is greatly shortened, and the overall latency time of the first device transmitting the content to be displayed to the first device for output display is shortened.

[0113] Figure 7 This is a schematic diagram illustrating the implementation process of the display control scheme disclosed in the embodiments of this application. (In conjunction with...) Figure 7 The solution implements a bidirectional synchronous signal transmission mechanism between the first processor (ISP) and the second processor (DPU). Specifically, the solution adds a bidirectional synchronous signal transmission channel between the ISP and the underlying driver of the traditional Camera Preview system, serving as the software foundation for the low-latency solution. The transmitted information is shown in the table below:

[0114]

[0115] Combination Figure 7 Taking an Android tablet as an example, the implementation of a display control scheme can include:

[0116] S1. A large-screen Android tablet computer can be powered on normally and complete the boot process.

[0117] S2. Wait for the user to insert a USB Type-C DPIN peripheral (such as a mobile phone, handheld game console, etc. as a video source).

[0118] S3. After inserting the DPIN device, the PD Plug In event notifies the DPIN IC driver. The driver then initializes the DPIN IC, including power-on and power-off operations. Simultaneously, it sends DPIN status information (resolution, frame rate, etc.) to the DPIN service and triggers the launch of the DPIN app.

[0119] S4. Open the Camera with the DPIN App and preview normally.

[0120] S5. DPIN application opens Display and proceeds with normal rendering.

[0121] The S4 process (Camera processing part) can be further broken down as follows:

[0122] The S4.1 DPIN application requests the creation of an Android Camera context, enabling control of the ISP HW through the Android Camera software path.

[0123] The S4.2 ISP underlying software initializes the ISP HW and requires the DPU to provide the SF / HWC estimated time (Tsf_hwc) (estimated using historical data, such as the mean and maximum).

[0124] S4.3 When the ISP HW starts to acquire DPIN image data normally, it completes the Stream On operation.

[0125] The above steps S4 (including S4.1-S4.3) reuse the SoC's Camera image processing path to perform corresponding image processing; step S5 reuses the SoC's display path to perform corresponding display processing.

[0126] When the S4.4 ISP underlying software receives the SOF signal, it calculates Tsync based on the situation and sends the SOF start time and Tsync to the DPU underlying software.

[0127] After receiving the SOF signal, S4.5 configures the timer according to the preset Tef parameters and starts the high-precision timer, waiting for the timer to complete counting.

[0128] After the S4.6 timer completes its count, it triggers the HF signal. The HF signal serves as the frame complete signal, triggering the SF / HWC operation by the DPU.

[0129] S4.7 ISP HW completes frame data acquisition, and the EOF signal appears.

[0130] S4.8 continues processing for one frame, then jumps to S4.4.

[0131] The S5 process (DPU processing part) can be further broken down as follows:

[0132] The S5.1 DPIN application requests the creation of a Display context, enabling control of the ISP HW through the Android Display software path.

[0133] The S5.2 DPU underlying software initializes the DPU HW and provides the estimated time Tsf_hwc for SF / HWC required by the ISP.

[0134] S5.3 DPU HW waits for the ISP SOF signal and Tsync, and then jumps to S5.4 after the ISP SOF signal appears.

[0135] When the S5.4 DPU HW underlying software receives the SOF signal, it configures the Vsync time point based on the Tsync and SOF time points.

[0136] S5.5 DPU HW waits for the ISP HF signal to appear. If the HF signal is received, it jumps to S5.6 after it appears.

[0137] After the S5.6 DPU HW obtains the image data, it begins to perform SF / HWC operations.

[0138] After S5.7 generates DPU HW Vsync, it jumps to S5.8.

[0139] In S5.8, after Vsync, the software completes the commit, submits the frame data, and displays a completion message.

[0140] S5.9 continues processing for one frame, then jumps to S5.5.

[0141] The above solution differs from the traditional Android display chain in the following ways:

[0142] HF signal triggering mechanism: Traditional solutions rely on EOF to trigger subsequent processing. This solution uses a timer to simulate HF, triggering the DPU to prepare before data writing is complete, thus enabling simultaneous writing and reading.

[0143] SF / HWC startup timing: Traditional solutions require waiting for Vsync to start SF / HWC, while this solution starts immediately after HF arrives, hiding the software processing time in the data transmission process.

[0144] Vsync generation logic: Traditional solutions use Vsync with a fixed period or synchronized with the GPU. This solution dynamically adjusts the Vsync of the first frame based on the SOF of the ISP, and then enters the periodic mode after establishing the optimal phase difference.

[0145] Two-way parameter interaction: In traditional solutions, there is no direct timing interaction between the ISP and the DPU. This solution achieves collaborative optimization through the interaction between Tsync and Tsf_hwc.

[0146] This application's solution achieves precise coordination between ISP writing and DPU reading through the transmission of three types of information: SOF time point, Tsync, and Tsf_hwc; it achieves half-frame-level data triggering by triggering the HF signal through a Timer; and it shortens the software processing latency by advancing the SF / HWC operation to after HF reception rather than after Vsync arrives, thus effectively reducing the overall latency of the display link.

[0147] Any of the chips described in the above embodiments includes a processor and a memory. In the above embodiments, each function that the processor can implement is stored as a program module in the memory, and the processor executes the program module stored in the memory to implement the corresponding function.

[0148] The processor contains a kernel, which retrieves the corresponding program modules from memory. One or more kernels can be configured, and the processing of backtracking data can be achieved by adjusting kernel parameters.

[0149] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0150] In an exemplary embodiment, a computer-readable storage medium is also provided, which can be directly loaded into the internal memory of a computer and contains software code. The computer program, after being loaded and executed by the computer, can realize the above-mentioned chip functions.

[0151] In an exemplary embodiment, a computer program product is also provided, which can be directly loaded into the internal memory of a computer and contains software code. After being loaded and executed by the computer, the computer program can realize the functions shown in any of the chip embodiments described above.

[0152] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0153] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0154] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0155] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A chip having a display transmission path, comprising a first processor and a second processor; The first processor is used to receive data to be displayed input from an external device and store it in memory; The second processor is used to read the data to be displayed from memory and process it into display data; The first processor is configured to: in response to receiving the first frame data input, send a frame acquisition start signal and a reference duration to the second processor; In response to the received frame data amount reaching the frame trigger data amount, a frame trigger signal is sent to the second processor; The second processor is configured to: set the first frame synchronization time in response to receiving the frame acquisition start signal and the reference duration; and perform preparatory work before display processing in response to receiving the frame trigger signal. In response to the arrival of the first frame synchronization time, a first frame synchronization signal is generated to read the data to be displayed written by the first processor from the memory for display processing, and to obtain display data for sending to the display module for output display; The frame trigger data volume is less than the data volume of the first image frame, and the first frame synchronization time is later than the frame trigger signal reception time.

2. The chip according to claim 1, wherein determining the reference duration includes: The sum of the frame trigger duration and the frame processing preparation duration is determined as the reference duration; Wherein, the frame trigger duration represents the time it takes for the first processor to write the frame trigger data corresponding to the amount of data to memory, the frame processing preparation duration represents the time it takes for the second processor to perform preparatory work before processing the data to be displayed, and the reference duration represents the time from when the first processor starts receiving frame data to when the second processor first starts processing the data to be displayed.

3. The chip according to claim 2, wherein determining the frame trigger duration includes: The frame trigger duration is determined based on the frame trigger data volume and the speed at which the frame data is written to memory. The frame trigger duration represents the time it takes for the first processor to write the frame trigger data volume to memory. The determination that the received frame data volume reaches the frame trigger data volume includes: If the cumulative duration from the acquisition of the start signal from the frame reaches the frame trigger duration, it is determined that the number of received frames has reached the frame trigger data amount.

4. The chip according to claim 1, wherein determining the first frame synchronization time includes: The first frame synchronization time is determined based on the frame acquisition start signal and the reference duration, such that the difference between the first frame synchronization time and the time corresponding to the frame acquisition start signal is the reference duration.

5. The chip according to claim 1, wherein both the first processor and the second processor are configured to process image frame data using a line processing method.

6. The chip according to claim 5, wherein, The configuration of the frame trigger data volume satisfies the following condition: at any given time, the amount of data to be displayed written by the first processor to the memory is greater than the amount of data to be displayed read by the second processor from the memory, including: At any given time, there exists a frame data unit in memory that the first processor has already processed, and the frame data unit that the first processor is currently writing into memory is different from the frame data unit that the second processor is currently reading from memory; the frame data unit is the smallest data unit that the first processor and the second processor access the memory.

7. The chip according to claim 6, wherein the configuration of the frame trigger data amount includes: The amount of frame trigger data is determined based on the access bandwidth ratio of the first processor and the second processor, as well as the processing speed of the first processor and the second processor.

8. The chip according to claim 2, wherein the determination of the frame processing preparation time includes any one of the following: The frame processing preparation time is determined based on the preparation time required by the second processor before performing the display processing on the data to be displayed, according to historical measurements. Based on the pre-defined mapping table of different scenarios and frame processing preparation time, and the scenario in which the second processor is currently located, the corresponding frame processing preparation time is determined.

9. The chip according to claim 1, wherein the first processor is an image signal processor and the second processor is a data processor.

10. A display system, comprising a first device and a second device, wherein the first device has a display module and a system chip; The second device is connected to the first device via a display input interface and is used to send data to be displayed to the first device. The data to be displayed is processed by the display transmission path within the system chip of the first device and then displayed and output by the display module. The system chip includes: a first processor and a second processor; The first processor is used to receive data to be displayed input from an external device and store it in memory; The second processor is used to read the data to be displayed from memory and process it into display data; The first processor is configured to: in response to receiving the first frame data input, send a frame acquisition start signal and a reference duration to the second processor; in response to the received frame data amount reaching the frame trigger data amount, send a frame trigger signal to the second processor; The second processor is configured to: set a first frame synchronization time in response to receiving the frame acquisition start signal and the reference duration; perform preparatory work before display processing in response to receiving the frame trigger signal; generate a first frame synchronization signal in response to reaching the first frame synchronization time, so as to read the data to be displayed written by the first processor from the memory for display processing; and obtain display data for sending to the display module for output display. The frame trigger data volume is less than the data volume of the first image frame, and the first frame synchronization time is later than the frame trigger signal reception time.

Images