Terminal equipment and display methods

Abstract

This application provides a terminal device and display method, applied in the field of terminal technology, to solve the problem of high power consumption during terminal device display. The terminal device includes a first processing unit, a second processing unit, and a display screen. The first processing unit is used to determine whether the current scene is a scene controlled by the second processing unit; the first processing unit is also used to generate processing data if the current scene is a scene controlled by the second processing unit; the second processing unit is used to synchronously process data from the first processing unit and generate first display driving data based on the processed data; the display screen is used to display according to the first display driving data; the first processing unit is also used to generate second display driving data if the current scene is not a scene controlled by the second processing unit; the display screen is used to display according to the second display driving data. This application is applied in the display process of a terminal device.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a terminal device and display method. Background Technology

[0002] With the continuous development and advancement of terminal technology, the types and functions of terminal devices are becoming increasingly diverse. While improving the performance of terminal devices, there is also a strong emphasis on reducing their power consumption. During the use of terminal devices, display power consumption has become a prominent aspect of power consumption issues. Therefore, how to reduce the display power consumption of terminal devices without affecting the display effect has become an urgent technical problem to be solved. Summary of the Invention

[0003] This application provides a terminal device and display method that reduces the display power consumption of the terminal device and improves the battery life of the terminal device without affecting the display effect.

[0004] To achieve the above-mentioned technical objectives, the embodiments of this application provide the following technical solutions:

[0005] In a first aspect, a terminal device is provided, comprising a first processing unit, a second processing unit, and a display screen; the first processing unit is configured to determine whether the current scene is a scene controlled by the second processing unit; the first processing unit is further configured to generate processing data if the current scene is a scene controlled by the second processing unit; the second processing unit is configured to synchronously process data from the first processing unit and generate first display driving data based on the processing data; the display screen is configured to display data based on the first display driving data; the first processing unit is further configured to generate second display driving data if the current scene is not a scene controlled by the second processing unit; the display screen is configured to display data based on the second display driving data; wherein the first processing unit is a high-power processing unit, and the second processing unit is a low-power processing unit.

[0006] Based on the above scheme, in a scenario controlled by the second processing unit, the terminal device first processes the data using a high-power first processing unit (such as an AP), and then the low-power second processing unit (such as a CP) synchronously processes the data, generating display driver data and sending it to the display. That is, the high-power first processing unit processes the data required for display, and the low-power second processing unit sends the processed data to the display screen for display. This distributes part of the display load from the high-power first processing unit to the low-power second processing unit, reducing the display load on the high-power first processing unit and increasing the probability of it going into sleep mode. Furthermore, the low-power second processing unit has the ability to take over display interaction in application scenarios, broadening its application scenarios. Moreover, the low-power second processing unit has lower power consumption, thereby reducing the overall power consumption of the terminal device and improving its battery life.

[0007] According to the first aspect, in one possible implementation, the first display driving data is image data generated by the second processing unit for driving the display screen; the second display driving data is image data generated by the first processing unit for driving the display screen.

[0008] According to the first aspect, in one possible implementation, the display screen is an electronic ink screen, and the processing data includes preprocessed data, which is first graphic data generated by the first processing unit; the second processing unit is further configured to generate first data to be displayed based on the preprocessed data; the second processing unit is further configured to perform a conversion operation on the first data to be displayed to generate first display driving data.

[0009] According to the first aspect, in one possible implementation, the first data to be displayed is a combination of preprocessed data to obtain a first primitive control.

[0010] In some examples, the second processing unit combines preprocessed data according to a preset method to generate the first data to be displayed.

[0011] In other examples, the second processing unit generates first display data based on the preprocessed data; the second processing unit reads pixel data from the first display data in parallel; the second processing unit converts the pixel data into grayscale data; and the second processing unit generates first display driving data based on the grayscale data.

[0012] Based on this design, the high-power first processing unit draws and generates preprocessed data, while the low-power second processing unit combines and transforms the preprocessed data to generate the first display driving data. The low-power second processing unit handles the subsequent display load, reducing the display load on the high-power first processing unit and effectively lowering the overall power consumption of the terminal device.

[0013] According to the first aspect, in one possible implementation, the display screen is an electronic ink screen, and the processed data includes conversion data, which is a second image data generated by the first processing unit after performing a conversion operation on the first image data generated by the first processing unit; the second processing unit is further configured to generate second data to be displayed based on the conversion data; the second processing unit is further configured to determine the second data to be displayed as the first display driving data.

[0014] According to the first aspect, in one possible implementation, the second data to be displayed is a combination of transformed data to obtain a second primitive control.

[0015] In some examples, the second processing unit combines and transforms data according to a preset method to generate second data to be displayed.

[0016] Based on this design, a high-power first processing unit draws and generates the conversion data, while a low-power second processing unit combines the preprocessed data to generate the second data to be displayed. The display operation is completed by the low-power second processing unit, which speeds up data processing and reduces the power consumption of the terminal device.

[0017] According to the first aspect, in one possible implementation, the first processing unit is further configured to classify the image data in the currently displayed image; the first processing unit is further configured to draw and generate processing data based on the classified image data.

[0018] In some examples, image data includes constant pattern primitives, enumerable pattern primitives, and composite pattern primitives. Primitives are the most basic data units displayed by the terminal device and can be understood as the effective data actually displayed in the image.

[0019] Based on this design, the high-power first processing unit does not need to draw and process the entire image. It only needs to draw and generate the processing data corresponding to each piece of image data based on the image data contained in the image. While not affecting the image display effect, the data size is greatly reduced and memory storage is optimized so that the display load can be executed on the smaller processing data in the subsequent stages. This can effectively reduce the amount of data calculation and memory usage, thereby reducing the overall power consumption of the terminal device during display and improving the battery life of the terminal device.

[0020] According to the first aspect, in one possible implementation, the conversion operation includes: grayscale processing or timing processing.

[0021] According to the first aspect, in one possible implementation, the display screen is a non-e-ink screen, and the processing data includes preprocessed data, which is first graphic data generated by the first processing unit; the second processing unit is further configured to combine the preprocessed data according to a preset method to generate first display driving data.

[0022] Based on this design, when the terminal device's display is a non-e-ink screen and the current scenario is controlled by a low-power second processing unit, the high-power first processing unit generates smaller amounts of processed data, and the low-power second processing unit combines the processed data to generate the first display driving data, effectively reducing the computational load on the display. Furthermore, the low-power second processing unit has the ability to take over display interaction in application scenarios, broadening its application scenarios and reducing the terminal device's display power consumption, thus improving its battery life.

[0023] According to the first aspect, in one possible implementation, the second processing unit is further configured to take over the display control from the first processing unit after synchronously processing data from the first processing unit, if the display control belongs to the first processing unit.

[0024] According to the first aspect, in one possible implementation, the display control right is the right to control the display subsystem.

[0025] It should be understood that display control refers to the right to control the display subsystem, responsible for retrieving image data from memory and sending the processed image data to the display screen. At any given time, only one processing unit in the terminal device can exercise display control.

[0026] Based on this design, in the scenario controlled by the second processing unit, if the display control belongs to the first processing unit, the display control belonging to the high-power first processing unit will be switched to the low-power second processing unit. The low-power second processing unit takes over the display control, can process the data synchronized from the high-power first processing unit, and send the processed data to the display screen for display. This can reduce the power consumption of the terminal device and improve the battery life of the terminal device.

[0027] According to the first aspect, in one possible implementation, the second processing unit controls a scenario including a reading / browsing scenario or a screen-on standby scenario.

[0028] Secondly, a display method is provided, applied to a terminal device including a first processing unit, a second processing unit, and a display screen. The method includes: determining whether the current scene is a scene controlled by the second processing unit. If the current scene is a scene controlled by the second processing unit, after generating processing data through the first processing unit, the second processing unit synchronously processes the data from the first processing unit and generates first display driving data based on the processing data; displaying the image based on the first display driving data; the first display driving data is image data generated by the second processing unit and used to drive the display screen. If the current scene is not a scene controlled by the second processing unit, second display driving data is generated through the first processing unit; displaying the image based on the second display driving data; the second display driving data is image data generated by the first processing unit and used to drive the display screen.

[0029] According to the second aspect, in one possible implementation, the first processing unit is a high-power processing unit and the second processing unit is a low-power processing unit.

[0030] According to the second aspect, in one possible implementation, the second processing unit controls scenarios including reading and browsing scenarios or screen-on standby scenarios.

[0031] In some examples, the terminal device can determine whether the current scenario is controlled by the second processing unit based on the application being used by the user.

[0032] According to the second aspect, in one possible implementation, the display screen is an electronic ink screen, and the processing data includes preprocessed data, which is first graphic data generated by the first processing unit; the second processing unit generates first display driving data based on the processed data, including: generating first data to be displayed based on the preprocessed data by the second processing unit; and performing a conversion operation on the first data to be displayed to generate the first display driving data.

[0033] According to the second aspect, in one possible implementation, the first data to be displayed is a combination of preprocessed data to obtain the first primitive control.

[0034] According to the second aspect, in one possible implementation, the display screen is an electronic ink screen, and the data processing includes conversion data, which is a second image data generated by the first processing unit after performing a conversion operation on the first image data generated by the first processing unit; the second processing unit generates first display driving data based on the processing data, including: generating second data to be displayed based on the conversion data by the second processing unit; and determining the second data to be displayed as the first display driving data.

[0035] According to the second aspect, in one possible implementation, the second data to be displayed is a combination of transformed data to obtain a second primitive control.

[0036] According to the second aspect, in one possible implementation, the conversion operation includes: grayscale processing or timing processing.

[0037] According to the second aspect, in one possible implementation, generating processing data through the first processing unit includes: classifying the image data in the currently displayed image through the first processing unit; and drawing and generating processing data based on the classified image data.

[0038] According to the second aspect, in one possible implementation, the display screen is a non-e-ink screen, and the processing data includes preprocessed data, which is first graphic data generated by the first processing unit; the second processing unit generates first display driving data based on the processing data, including: the second processing unit combines the preprocessed data according to a preset method to generate the first display driving data.

[0039] According to the second aspect, in one possible implementation, after the second processing unit synchronously processes data from the first processing unit, the method further includes: if display control belongs to the first processing unit, then the second processing unit takes over display control from the first processing unit.

[0040] According to the second aspect, in one possible implementation, the display control right is the right to control the display subsystem.

[0041] Thirdly, a terminal device is provided, comprising: a processor and a memory, the memory being coupled to the processor, the memory being used to store computer-readable instructions, wherein when the processor reads the computer-readable instructions from the memory, the terminal device causes the terminal device to execute the method of the second aspect or any embodiment of the second aspect.

[0042] Fourthly, a chip system is provided, including at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor, wherein the at least one processor executes the instructions, and wherein the at least one processor performs the method of the second aspect or any embodiment of the second aspect.

[0043] Fifthly, a computer-readable storage medium is provided, the computer-readable storage medium including a computer program that, when run on an electronic device, causes the electronic device to perform the method of the second aspect or any embodiment of the second aspect.

[0044] Sixthly, a computer program product is provided that, when run on a computer, causes the computer to perform the method of the second aspect or any of the embodiments of the second aspect.

[0045] The technical effects corresponding to any implementation method of aspects two through six, as well as any aspect, can be found in the first aspect and the technical effects corresponding to any implementation method of the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0046] Figure 1 This application provides a schematic diagram illustrating the usage time of a terminal device in different scenarios.

[0047] Figure 2 This is a schematic diagram of the display framework process of a terminal device provided in an embodiment of this application;

[0048] Figure 3 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application;

[0049] Figure 4 This application provides a schematic diagram of the system architecture of a terminal device.

[0050] Figure 5 This is a schematic diagram of the structure of a data collaborative management module provided in an embodiment of this application;

[0051] Figure 6 This is a schematic diagram of the structure of a state collaborative management module provided in an embodiment of this application;

[0052] Figure 7 A schematic diagram of the structure of a LiteTCON lightweight image processing module provided in an embodiment of this application;

[0053] Figure 8 A schematic flowchart illustrating a display method provided in an embodiment of this application;

[0054] Figure 9 A schematic diagram of an interface provided for an embodiment of this application;

[0055] Figure 10 This is a schematic diagram illustrating the generation of display driver data provided in an embodiment of this application;

[0056] Figure 11 A schematic diagram of memory optimization provided in an embodiment of this application;

[0057] Figure 12 A schematic diagram illustrating a display effect provided in an embodiment of this application;

[0058] Figure 13 A schematic diagram illustrating the benefit effect provided in an embodiment of this application;

[0059] Figure 14 A schematic diagram illustrating yet another benefit effect provided by an embodiment of this application;

[0060] Figure 15 A flowchart illustrating yet another display method provided in an embodiment of this application;

[0061] Figure 16 This is a schematic diagram of the structure of another terminal device provided in an embodiment of this application;

[0062] Figure 17 This is a schematic diagram of the structure of another terminal device provided in an embodiment of this application;

[0063] Figure 18 This is a schematic diagram of a chip system provided in an embodiment of this application. Detailed Implementation

[0064] In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can mean A or B. "And / or" in this application is merely a description of the relationship between the related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. A and B can be singular or plural.

[0065] In the description of this application, unless otherwise stated, "multiple" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can mean: a, b, c, a and b, a and c, b and c, a and b and c, where a, b, and c can be single or multiple.

[0066] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0067] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0068] The features, structures, or characteristics in this application can be combined in any suitable manner in one or more embodiments. In the various embodiments of this application, the sequence number of each process does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0069] Some optional features in the embodiments of this application can be implemented independently without relying on other features in certain scenarios to solve the corresponding technical problems and achieve the corresponding effects. In other scenarios, they can be combined with other features according to needs.

[0070] In this application, unless otherwise specified, the same or similar parts between the various embodiments can be referred to each other. In the various embodiments of this application, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments are consistent and can be mutually referenced. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships. The implementation methods of this application do not constitute a limitation on the scope of protection of this application.

[0071] Furthermore, the network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0072] To better understand the technical solution of this application, the technical terms and related concepts that may be involved in the embodiments of this application will be introduced below.

[0073] 1. Heterogeneous Collaborative Display Architecture

[0074] Heterogeneous collaborative display architecture is an architectural model for controlling device displays. In some scenarios, heterogeneous collaborative display architecture can refer to a device including different processing units (or computing modules, computing units, or processing cores, etc.). For example, processing units can include, but are not limited to, application processors (APs), coprocessors (CPs), and graphics processing units (GPUs). These processing units can deploy different operating systems (OSs). For instance, the AP can run Android®, HarmonyOS®, iOS®, etc.; the CP can run embedded real-time operating systems such as Huawei LiteOS, FreeRTOS®, etc.; and the GPU can run binary operating systems, etc.

[0075] In some embodiments, when a device employing a heterogeneous collaborative display architecture provides display functions, different processing units need to cooperate with each other, such as communicating, synchronizing data or states, etc. The execution logic of this mutual cooperation to complete the display function can be called heterogeneous collaborative display.

[0076] 2. Electronic Ink Screen

[0077] E-ink screens generally refer to ink screens, which utilize electrophoretic display (EDP) technology to display interfaces. The display effect of e-ink screens is very close to that of traditional paper, hence they are also known as "electronic paper."

[0078] An e-ink screen consists of two substrates. Each substrate has several light-emitting units on its surface, each unit representing a pixel. Numerous tiny e-ink particles are attached to the substrate surface. Each e-ink particle is composed of several liquid microcapsules, with each pixel corresponding to at least one microcapsule. These microcapsules contain positively and negatively charged particles, each with a different color (e.g., white for positively charged particles and black for negatively charged particles). When a voltage level is applied to the substrate, the positively and negatively charged particles within the microcapsules are attracted and repelled, respectively, reaching the opposite ends of the microcapsules, thus causing each pixel to appear black or white.

[0079] E-ink screens do not require backlighting. Ambient light shines on the e-ink screen, which then reflects the ambient light back to the user's eyes. This method simulates the properties of ink and paper, ensuring that text appears comfortable and clear under any light source.

[0080] 3. Lightweight timing controller (LiteTCON)

[0081] A lightweight timing controller is a timing controller used to control the display on an e-ink screen. In typical terminal devices (such as mobile phones), when switching images, the operating system (OS) renders and composites the image, then directly displays the composite image on the screen to complete the image switch. However, due to the physical characteristics of e-ink screens, each pixel changes according to its corresponding voltage level. Therefore, the image composited by the OS in an e-ink screen needs to undergo strict timing processing by a lightweight timing controller before it can be displayed correctly on the e-ink screen.

[0082] 4. CP Full Control Display Mode

[0083] The CP full-control display mode is a mode in which the display interaction of the terminal device is controlled by a low-power processing unit. For example, because the AP has very strong processing power and high computing power, the AP usually handles the display interaction of the terminal device (such as displaying reading content in the reading area of ​​the display page in a reading display scenario, and displaying the next page in response to user operation). The CP, on the other hand, is a low-energy-efficiency general-purpose processing unit with low computing power, and can only perform simple calculation tasks (such as displaying time and battery level in the status bar of the display page in a reading display scenario). In the CP full-control display mode, the CP can achieve the same display effect as the AP, handling the display interaction of the terminal device (e.g., displaying reading content in the reading area of ​​the display page in a reading display scenario, displaying the next page in response to user operation, and displaying time and battery level in the status bar of the display page).

[0084] 5. Display control

[0085] The display subsystem (DSS) is a collective term for display-related hardware. It is responsible for retrieving image data from memory and sending the processed image data to the display screen. The DSS can be controlled by the operating system in the processing unit. At any given time, only one processing unit can control the DSS. This right of the processing unit to control the DSS can be called display control.

[0086] For example, consider a device employing a heterogeneous architecture, where the processing units include an application processing unit (AP) and a processing unit (CP). In some embodiments, when the device is in high-performance mode, the display switch (DSS) can be controlled by the AP. When the device switches from high-performance mode to low-power mode, the DSS can switch from AP control to CP control, with the CP taking over display control. The process of switching the processing unit's control over the DSS between the two processing units can be referred to as display control switching.

[0087] Currently, there is a wide variety of terminal devices, and e-ink screens can be applied to terminal devices with interface display functions, such as mobile phones and tablets. Taking a terminal device using an e-ink screen as an example, for instance... Figure 1 The diagram shown illustrates the usage time of a terminal device in different scenarios according to an embodiment of this application.

[0088] Understandable Figure 1 The scenarios described refer to those when users are using the terminal device, and do not include scenarios where users are not using the e-ink screen terminal device and are in lock-screen standby mode.

[0089] like Figure 1As shown, when using e-ink screen terminal devices, the usage time in reading and browsing scenarios (such as reading documents and browsing notes) is longer than that in other scenarios (such as entertainment and social networking). The usage time in reading and browsing scenarios accounts for more than 50%. Correspondingly, the display power consumption of e-ink screen terminal devices is mainly concentrated in reading and browsing scenarios; that is, the display power consumption of e-ink screen terminal devices in reading and browsing scenarios is higher than in other scenarios.

[0090] In related technologies, e-ink display terminal devices typically employ a heterogeneous collaborative display architecture. When implementing display service functions, the display workload, such as image drawing, rendering, and compositing, is usually uniformly handled by the application processing unit (AP) in the system (e.g., the AP handles the display interaction of a specific application running on the e-ink display terminal device). The processing unit (CP) is only responsible for simple display tasks (such as displaying simple controls like a clock), limiting its application scenarios and making it unable to handle the display interaction of a specific application running on the e-ink display terminal device. However, while the AP has strong computing power and can provide good display effects, its high power consumption leads to high overall system power consumption when the e-ink display terminal device is displaying.

[0091] It is understood that, in this application embodiment, the operations related to image display are referred to as display load. For example, Table 1 shows a comparison table of display performance of different brands of e-ink screen terminal devices provided in this application embodiment.

[0092] Table 1

[0093]

[0094] According to the information shown in Table 1, the chip energy efficiency should be sorted from high to low as follows: chip 2 > chip 3 > chip 1.

[0095] According to the information in Table 1, the display load of Brand 1 is deployed on the AP, and the computing power and energy efficiency of the chip 1 used by Brand 1 are low. Therefore, Brand 1 has low power consumption and long battery life, but poor display quality (clear reading, no ghosting). Moreover, Brand 1 has an independent operating ecosystem, which is single and closed. Therefore, Brand 1's functions are relatively limited, it cannot quickly install applications from other systems, and its operation method differs from that of mobile platforms, resulting in a high learning curve for users.

[0096] As shown in Table 1, the display load of both Brand 2 and Brand 3 is deployed on the AP, and both Chip 2 and Chip 3 have high computing power. Therefore, in terms of display effect, Brand 2 and Brand 3 have the same display effect, both providing high-definition reading and page-turning functions, and there is no ghosting interference problem when switching images on the e-ink screen. However, Chip 2 and Chip 3 have different design processes. Chip 2 has a superior design process, so Chip 2 has higher energy efficiency than Chip 3, and Chip 2 has an advantage in base power consumption. That is, in terms of overall system power consumption, Brand 2's overall system power consumption is better than Brand 3's. Moreover, both Brand 2 and Brand 3 are compatible with the Android system. Therefore, Brand 2 and Brand 3 can use the Android system, install various applications, and operate in the same way as the Android system, with a low user learning cost.

[0097] Having understood the display performance of e-ink screen terminal devices in related technologies, the following describes the display load processing flow when e-ink screen terminal devices display images.

[0098] For example, Figure 2 This illustration shows a schematic diagram of the display framework of an electronic ink screen terminal device according to an embodiment of this application. Figure 2 As shown, the terminal device adopts a heterogeneous collaborative display architecture, in which the processing units include a high-performance processing unit (AP) and a low-power processing unit (CP). The terminal device system deploys all UI display loads to the AP.

[0099] like Figure 2 As shown, the system of a terminal device may include an application layer, a system layer, a driver layer, and a hardware layer.

[0100] It is understood that in this embodiment of the application, all operations performed by the terminal device that require power consumption of the terminal device are referred to as load.

[0101] In this embodiment of the application, the application layer includes system applications (APPs) that come with the terminal device and / or third-party applications.

[0102] System applications, also known as embedded applications, are applications that function as part of a terminal device. Third-party applications, also known as downloadable applications, are applications that can provide their own Internet Protocol Multimedia Subsystem (IMS) connectivity. These downloadable applications can be pre-installed on the terminal device or downloaded and installed by the user.

[0103] In this embodiment of the application, the application layer is used to determine the display content of the terminal device's UI. For example... Figure 2 As shown, the application layer includes reading applications, work applications, note-taking applications, etc. If a reading application is launched, the terminal device's UI will display the corresponding reading content (such as novel documents, chapters, novel titles, time, etc.).

[0104] In this embodiment, the system layer includes a UI display load, which includes UI drawing, UI rendering, compositor compositing, grayscale cropping, and TCON image processing.

[0105] In this embodiment of the application, the system layer is used to draw and render the display content of the terminal device UI determined by the application layer.

[0106] Understandably, because each pixel on an e-ink screen displays black or white based on its corresponding voltage level, e-ink screens, compared to other types of displays, also require grayscale clipping and timing controller (TCON) image processing.

[0107] In this embodiment, UI drawing is mainly used to complete image position calculations and drawing operations for each layer in the image. Specifically, UI drawing can be performed by handling UI events, animation, measurement, layout, and drawing. When a UI element, such as a view (control), is manipulated by the user in a specific way, the terminal device generates an event corresponding to that UI element, referred to as a UI event. Animation can be used to implement UI animation effects. Measurement and layout can be used to determine the size and position of each control included in the layer that needs to be drawn, and drawing is used to complete the drawing of the controls, etc.

[0108] In this embodiment, UI rendering is primarily used to complete the rendering operations of each layer in the image. Specifically, UI rendering can be completed through vertical synchronization (VSYNC) signals, updating the rendering cache, etc. In some embodiments, the VSYNC signal can be used to trigger the drawing, rendering, and compositing process of the next frame of the image. Updating the rendering cache can be used to store the rendered layers in the cache. Optionally, the related calculations in UI rendering can be completed by a central processing unit (CPU) or a graphics processing unit (GPU).

[0109] In this embodiment of the application, the composer is mainly used to combine one or more rendered layers into a single frame image.

[0110] In the embodiments of this application, grayscale cropping is mainly used to perform grayscale processing and cropping operations on a synthesized frame image to obtain an image to be displayed.

[0111] In this embodiment, TCON image processing is mainly used to perform timing and image processing on the image to be displayed, obtain display driving data, and transmit the display driving data to the display screen.

[0112] Display driver data, also known as timing control data, can include the waveforms and timing sequences required to display pixels on an e-ink screen.

[0113] As is understandable, e-ink display devices utilize electrophoretic display technology to achieve interface display. By inputting different voltage levels, they attract and repel positively and negatively charged particles within liquid microcapsules, thereby changing the pixel's display color from white to black, thus displaying different images. Therefore, e-ink display devices need to perform timing and image processing on the image to be displayed, converting it into display driving data. The e-ink screen applies different voltages according to the waveform and timing in the display driving data, controlling the display color of each corresponding pixel, thereby driving the screen to display different content.

[0114] In the embodiments of this application, the driver layer includes general memory management, inter-process communication (IPC), shared memory, and mobile industry processor interface (MIPI).

[0115] In this embodiment, the general-purpose memory manager can provide a universal memory management interface to manage different types of memory. Inter-process communication (IPC) is used for transmitting or exchanging information between different processes. Shared memory is used for sharing data between different processes. The mobile industry processor interface is the interface between the processor and the display screen. The mobile industry processor interface is mainly used to send display driver data generated by the system layer to the display screen (such as an e-ink screen), and the display screen displays the image according to the display driver.

[0116] In this embodiment, the hardware layer includes an AP, double data rate SDRAM (which can also be described as DDR memory), a display, and a CP.

[0117] In this embodiment, the AP (Application Processor) is a high-power core, serving as the main central processing unit (CPU). It connects to peripheral devices (such as DDR memory and a display screen) and is primarily responsible for the operation of the e-ink screen terminal system and applications. For example, in response to a reading application, the AP is responsible for drawing, rendering, compositing, grayscale cropping, and timing processing of the text portion of the displayed content to obtain display driving data, which is then transmitted to the display screen via MIPI. DDR memory can be used to store computational data and data exchanged with external storage devices such as hard drives. The display screen displays the corresponding image based on the display driving data transmitted via MIPI. The CP (Processing Unit) is a low-power core, typically a microcontroller (MCU). It connects to various sensors (such as accelerometers) and is primarily responsible for handling simple computational tasks and tasks with high real-time requirements. For example, in response to a reading application, the CP generates a time-corresponding image in the displayed content.

[0118] Since terminal devices often need to perform UI display workloads, as mentioned above, these workloads are deployed on high-power APs, resulting in high power consumption and poor battery life for the terminal devices.

[0119] As can be seen, although the e-ink screen terminal devices in the aforementioned related technologies adopt a heterogeneous collaborative display architecture, due to the high computing power of the AP, the terminal device deploys all the display load of the user interface (UI) to the AP, which runs various loads to complete the display interaction. Although the AP has strong computing power, it needs to run at high frequency for a long time to complete all display-related loads, resulting in relatively high power consumption for the terminal device. For example, if the frame rate of the terminal device's user interface (UI) is 60 Hz, the AP needs to execute all display-related loads within 16.6 milliseconds, resulting in high system power consumption. Moreover, the low-power CP cores in the heterogeneous collaborative display architecture are idle and not fully utilized, limiting application scenarios and causing resource waste.

[0120] Based on this, this application provides a display method applied to a terminal device including a first processing unit and a second processing unit. The method includes: the terminal device determining whether the current scenario is controlled by the second processing unit; if so, the first processing unit generates processing data; the second processing unit synchronously processes the data from the first processing unit, determines first display driving data based on the processing data, and then transmits the first display driving data to the display screen. Otherwise, the first processing unit processes and generates second display driving data and sends it to the display. In the technical solution provided by this application, when the terminal is in a scenario controlled by the second processing unit, the first processing unit first processes the data to obtain processing data, then the second processing unit synchronously processes the data, generates display driving data based on the processing data, and performs a display sending operation, which is then processed and displayed by the second processing unit. The first processing unit can be a high-power processing unit, such as an AP, and the second processing unit can be a low-power processing unit, such as a CP. In this way, the terminal device deploys part of the UI display load from the AP to the CP, reducing the UI display load on the AP and increasing the probability of the AP going into sleep mode. Moreover, the CP is a low-power core, with lower execution power consumption than the AP, thereby reducing the power consumption of the terminal device and improving its battery life.

[0121] The technical solutions provided in this application embodiment can be applied to terminal device 100 or to a system containing terminal device 100.

[0122] For example, Figure 3 A schematic diagram of the terminal device 100 is shown. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a sensor module 130, and an electronic ink screen 140, etc. The sensor module 130 may include a pressure sensor 130A and a touch sensor 130K, etc.

[0123] The structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0124] Processor 110 may include multiple different processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), coprocessor (CP), image signal processor (ISP), controller, memory, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. These different processing units may be independent devices or integrated into one or more processors.

[0125] In some embodiments of this application, the processor 110 may include one or more high-power processing units, such as AP, and one or more low-power processing units, such as CP. In some examples, if the power consumption of a processing unit is greater than or equal to a power consumption threshold, then the processing unit is a high-power processing unit. If the power consumption of a processing unit is less than the power consumption threshold, then the processing unit is a low-power processing unit. Optionally, the power consumption threshold can be set by the developer according to actual needs. The controller can generate operation control signals based on the instruction opcode and timing signals to control instruction fetching and execution.

[0126] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0127] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0128] Terminal device 100 implements display functions through a GPU, an e-ink screen 140, an AP, and a CP. The GPU is a microprocessor for image processing, connected to the e-ink screen 140 and the AP. The GPU performs mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information. The AP and CP collaboratively process images. The e-ink screen 140 is used to display images. In some embodiments, terminal device 100 may include one or N e-ink screens 140, where N is a positive integer greater than 1. In some embodiments of this application, the display screen can be used to display an image synthesized from one or more layers. The digital signal processor (DSP) processes digital signals, including digital image signals and other digital signals. For example, when terminal device 100 selects a frequency point, the DSP performs a Fourier transform on the frequency point energy.

[0129] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to perform data storage functions.

[0130] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of terminal device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area.

[0131] The program storage area can store the operating system, at least one application program required for a function (such as image display function), etc. The data storage area can store data created during the use of the terminal device 100, etc. In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0132] The sensor module 130 may include a pressure sensor 130A and a touch sensor 130K, also known as a "touch panel". The pressure sensor 130A is used to sense pressure signals and can convert the pressure signals into electrical signals.

[0133] In some embodiments, a pressure sensor 130A may be disposed on the e-ink screen 140. When force is applied to the pressure sensor 130A, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the e-ink screen 140, the terminal device 100 detects the intensity of the touch operation based on the pressure sensor 130A. The terminal device 100 may also calculate the touch position based on the detection signal from the pressure sensor 130A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities may correspond to different operation commands.

[0134] Touch sensor 130K, also known as a "touch panel," can be located on e-ink screen 140. The touch sensor 130K and e-ink screen 140 together form a touchscreen, also known as a "touch display." Touch sensor 130K detects touch operations applied to or near it. For example, a user can touch e-ink screen 140 using an active capacitive pen, passive capacitive pen, or electromagnetic pen to input touch operations. Touch sensor 130K can detect these touch operations input by the user using an active capacitive pen, passive capacitive pen, or electromagnetic pen on e-ink screen 140.

[0135] Touch sensor 130K can also transmit detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided via e-ink screen 140. In other embodiments, touch sensor 130K may also be disposed on the surface of terminal device 100, in a different location than e-ink screen 140.

[0136] For example, in this embodiment of the application, the touch sensor 130K can be used to detect the refresh operation of the user interface by the user's instruction to refresh the user interface through the touch screen, and transmit the detected refresh operation to the processor 110 so that the processor 110 controls the refresh of the content displayed on the e-ink screen 140.

[0137] In other embodiments of this application, the terminal device 100 may include a ratio Figure 3 The illustrated components may include more or fewer components, or some components may be combined, some components may be separated, or different component arrangements may be made. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. This application does not limit the structure or form of the terminal device 100.

[0138] For example, taking the processing unit installed in terminal device 100 as including AP and CP as an example, Figure 4 A schematic diagram of the system architecture of a terminal device 100 provided in an embodiment of this application is shown.

[0139] like Figure 4 As shown, the system of terminal device 100 may include an application layer, a system layer, a driver layer, and a hardware layer.

[0140] In this embodiment, the application layer includes system applications and / or third-party applications that come with the terminal device. The application layer is used to determine the display content of the terminal device's UI. A detailed description of the application layer can be found above.

[0141] In this embodiment, the system layer includes UI display load, touch interaction load, AP heterogeneous network service module, and CP heterogeneous network service module.

[0142] Specifically, the UI display workload includes UI drawing, UI rendering, compositor compositing, grayscale clipping, and TCON image processing. The UI display workload is deployed on the AP. A detailed description of the UI display workload can be found above. The touch interaction workload is used to detect and respond to user touch operations. The AP heterogeneous network service module is deployed on the AP and includes a data collaboration management module and a state collaboration management module. The CP heterogeneous network service module is deployed on the CP and includes a data collaboration management module, a state collaboration management module, and a LiteTCON lightweight image processing module.

[0143] In this embodiment, the terminal device determines whether the current scenario is a CP control scenario. If it is a CP control scenario, the display processing is performed collaboratively by the AP heterogeneous network service module and the CP heterogeneous network service module. If it is not a CP control scenario, the display processing is performed by the AP. See the above for details on the implementation method. Figure 2 A detailed description.

[0144] Compared to the system architecture in related technologies, the system layer of the system architecture in this application embodiment adds an AP heterogeneous network service module and a CP heterogeneous network service module. The functions and structures of each module in the newly added AP heterogeneous network service module and CP heterogeneous network service module are described in detail below.

[0145] In this embodiment, the data collaboration management module manages the data collaboration between the AP and CP, and determines the image to be displayed on the screen via MIPI. For example, the data collaboration management module is primarily responsible for operations such as data protocol, data update, and data destruction between the AP and CP.

[0146] For example, such as Figure 5 The diagram shown is a structural schematic of a data collaboration management module provided in an embodiment of this application. The AP-side data collaboration management module (which can also be described as AP data collaboration management module 500) includes a data drawing interface 501, a data management module 502, and a data synchronization module 503. The CP-side data collaboration management module (which can also be described as CP data collaboration management module 504) includes a data display interface 505, a data management module 502, and a data synchronization module 503.

[0147] The data drawing interface 501 on the AP side includes a constant pattern primitive drawing unit 5011, an enumerable pattern primitive drawing unit 5012, and a combined pattern primitive drawing unit 5013.

[0148] The constant pattern primitive drawing unit 5011 is used to draw the corresponding constant pattern primitive data according to the constant pattern primitive in the currently displayed image; the enumerable pattern primitive drawing unit 5012 is used to draw the corresponding enumerable pattern primitive data according to the enumerable pattern primitive in the currently displayed image; and the combined pattern primitive drawing unit 5013 is used to draw the corresponding combined pattern primitive data according to the combined pattern primitive in the currently displayed image.

[0149] The data drawing interface 501 is used to draw the generated processing data. This processing data includes preprocessed data or operational data.

[0150] Specifically, the data drawing interface 501 is also used to integrate the graphic data drawn by each graphic element drawing unit (constant mode graphic element drawing unit 5011, enumerable mode graphic element drawing unit 5012 and combined mode graphic element drawing unit 5013) into preprocessed data.

[0151] Alternatively, after obtaining the preprocessed data, the data drawing interface 501 can also perform transformation operations on the preprocessed data to obtain transformed data.

[0152] In this embodiment, whether the data drawing interface 501 generates processing data depends on the usage scenario of the terminal device 100. When the terminal device 100 is currently in a CP control scenario (such as a reading / browsing scenario or a screen-on standby scenario), the terminal device 100 performs CP and AP display interactions, and the data drawing interface 501 generates processing data.

[0153] For example, when terminal device 100 is in a social scene, display control belongs to the AP. When terminal device 100 switches from a social scene to a reading / browsing scene, processing data is generated only according to the drawing interface 501.

[0154] For example, if both the previous and current scenarios are CP-controlled scenarios, since the display control has already been switched from AP to CP in the first scenario, the display control will still be controlled by CP in the second scenario until the terminal usage scenario is no longer a CP-controlled scenario, at which point the display control will be switched from CP to AP and the AP will perform the display control.

[0155] The data display interface 505 on the CP side includes a constant mode primitive control unit 5051, an enumerable mode primitive control unit 5052, and a composite mode primitive control unit 5053.

[0156] Among them, the constant pattern primitive control unit 5051 generates a constant pattern primitive control based on the constant pattern primitive data in the processed data; the enumerable pattern primitive control unit 5052 generates an enumerable pattern primitive control based on the enumerable pattern primitive data in the processed data; and the composite pattern primitive control unit 5053 generates a composite pattern primitive control based on the composite pattern primitive data in the processed data.

[0157] The constant pattern primitive controls include a first constant pattern primitive control and a second constant pattern primitive control. The enumerable pattern primitive controls include a first enumerable pattern primitive control and a second enumerable pattern primitive control. The composite pattern primitive controls include a first composite pattern primitive control and a second composite pattern primitive control.

[0158] Optionally, the first constant mode primitive control, the first enumerable mode primitive control, and the first composite mode primitive control are the first primitive controls generated by each primitive control unit in the data display interface 505 based on preprocessed data.

[0159] Optionally, the second constant mode primitive control, the second enumerable mode primitive control, and the second combined mode primitive control are second primitive controls generated by each primitive control unit in the data display interface 505 based on the converted data.

[0160] The data display interface 505 is also used to generate data to be displayed based on the graphic element control. This data to be displayed includes either first data to be displayed or second data to be displayed.

[0161] Specifically, the data display interface 505 is also used to integrate the first primitive control generated by each primitive control unit into the first data to be displayed.

[0162] Alternatively, the data display interface 505 is also used to integrate the second primitive controls generated by each primitive control unit into a second set of data to be displayed.

[0163] The data management module 502 on the AP side includes a graphic element management unit 5021 and a graphic element distribution unit 5022. The graphic element management unit 5021 manages the processed data (preprocessed data or transformed data) generated by the data drawing interface. The graphic element distribution unit 5022 distributes the processed data (preprocessed data or transformed data) generated by the data drawing interface to the data synchronization module, so that the AP side can synchronize the processed data (preprocessed data or transformed data) to the CP side.

[0164] The data management module 502 on the CP side includes a graphic element management unit 5021 and a graphic element update unit 5023. The graphic element management unit 5021 manages the graphic element controls (first graphic element control or second graphic element control) generated by each unit in the data display interface 505. The graphic element update unit 5023 updates each graphic element control (first graphic element control or second graphic element control).

[0165] The data synchronization module 503 is used to synchronize data between the AP side and the CP side.

[0166] It is understood that the embodiments of this application do not limit the specific implementation method of data coordination and interaction between AP and CP.

[0167] In this way, the data collaboration management module on the AP side and the data collaboration management module on the CP side manage and synchronize the data between the AP and the CP through shared memory.

[0168] Understandable. Figure 5 The data collaboration management module included in the terminal device 100 is only one possible division method. In actual applications, the data collaboration management module in the terminal device 100 may include more or fewer modules, or there may be other module division methods. This application does not limit this.

[0169] In this embodiment, the state coordination management module manages the coordination state of the AP and CP, and also determines the processor currently controlling the display on the terminal device 100. For example, the state coordination management module determines whether the processor currently controlled by the terminal device 100 to take over and control the display screen is the AP or the CP, and then coordinates the states of the AP and CP. The state coordination management module is mainly responsible for operations such as enabling, disabling, synchronizing, and updating the coordination state between the AP and CP.

[0170] For example, such as Figure 6 The diagram shown is a structural schematic of a state coordination management module provided in an embodiment of this application. The state coordination management module on the AP side (which can also be described as AP state coordination management module 600) and the state coordination management module on the CP side (which can also be described as CP state coordination management module 601) include an enable state interface 602, a coordination state management module 603, and a coordination state synchronization module 604.

[0171] Specifically, the enable state interface 602 is responsible for enabling and disabling the heterogeneous collaborative display function, and is the entry point for UI heterogeneous collaborative display.

[0172] For example, when the enabled state interface 602 is turned on, the AP and CP can interact collaboratively. The AP switches display control to the CP, which then takes over display control and sends output to the display screen. When the enabled state interface 602 is turned off, the AP and CP do not interact collaboratively. Display control belongs to the AP, which is responsible for display processing and sending output to the display screen.

[0173] Specifically, the collaboration status management module 603 is responsible for managing and updating the collaboration status between the AP and CP. The collaboration status includes collaboration status enabled, data rendering status, data transmission status, and collaboration status disabled. Collaboration status enabled indicates that the AP and / or CP are in an enabled collaboration state; data rendering status indicates that the AP and / or CP are rendering and displaying data; data transmission status indicates that the AP and / or CP are transmitting data; and collaboration status disabled indicates that the AP and / or CP are in a disabled collaboration state.

[0174] Specifically, the collaborative state synchronization module 604 is responsible for synchronizing the states of the AP and CP. For example, the collaborative state synchronization module 604 on the AP side can synchronize the states with the collaborative state synchronization module 604 on the CP side through the IPC.

[0175] Understandable. Figure 6The state coordination management module included in the terminal device 100 is only one possible division method. In actual applications, the state coordination management module in the terminal device 100 may include more or fewer modules, or there may be other module division methods. This application does not limit this.

[0176] In this embodiment, the LiteTCON lightweight image processing module is deployed on the CP (Content Processing Unit). The LiteTCON lightweight image processing module has a lightweight timing controller (not shown in the figure) for providing lightweight e-ink screen image processing capabilities on the CP side. Specifically, the LiteTCON lightweight image processing module processes the data to be displayed generated by the CP-side data collaboration management module and transmits the processed display driver data to the display screen via MIPI, so that the display screen displays the corresponding image based on the processed display driver data.

[0177] For example, such as Figure 7 The diagram shown is a structural schematic of a LiteTCON lightweight image processing module provided in an embodiment of this application. The LiteTCON lightweight image processing module 700 includes an image input module 701, a processing mode selection module 702, a pass-through processing module 703, an image display module 704, a high-definition processing module 705, a parallel reading module 706, a pixel conversion module 707, and a parallel processing module 708.

[0178] In this embodiment, the image input module 701 is used to receive the data to be displayed generated from the CP-side data collaborative management module and send it to the processing mode selection module 702.

[0179] Specifically, the data received by the image input module 701 can be either the first data to be displayed or the second data to be displayed.

[0180] In this embodiment, the processing mode selection module 702 is used to determine the processing mode based on the data to be displayed transmitted by the image input module 701, and transmit the data to be displayed to the corresponding processing module.

[0181] Specifically, if the transmitted data is the first data to be displayed, the processing mode is determined to be high-definition mode; if the transmitted data is the second data to be displayed, the processing mode is determined to be pass-through mode.

[0182] In this embodiment, the pass-through processing module 703 is a processing module corresponding to the pass-through mode, used to determine the second data to be displayed as display driving data and transmit the display driving data to the image sending and display module 704.

[0183] In this embodiment, the image sending module 704 is used to send display driving data processed by different processing modes to the display screen.

[0184] Specifically, the image display module 704 is connected to the display screen via MIPI. When the processing mode is pass-through mode, it transmits the display driver data sent by the pass-through processing module 703 to the display screen via MIPI. Alternatively, when the processing mode is high-definition mode, it transmits the display driver data sent by the parallel processing module 708 to the display screen via MIPI.

[0185] In this embodiment, the high-definition processing module 705 is a processing module corresponding to the high-definition mode, used to transmit the first data to be displayed to the parallel reading module 706.

[0186] In this embodiment, the parallel reading module 706 is used to read pixel data in the first data to be displayed in parallel and transmit the pixel data to the pixel conversion module 707.

[0187] Understandably, the AP (Application Processor) typically reads and writes pixel data from memory one pixel at a time, processes that pixel data, and then writes it back to the original memory. However, the CP (Content Processor) has significantly lower computing power than the AP. If the CP uses the same read / write method as the AP, the data read / write time would be too long, hindering fast data processing. Therefore, to overcome the slow data read / write problem on the CP side, this embodiment employs a parallel data reading method on the CP side. That is, it reads a large block of pixel data, processes that large block, and then writes the processed large block. For example, it reads an entire row of pixel data, processes that entire row, and then writes the processed entire row.

[0188] In this way, data processing speed is accelerated through parallel processing.

[0189] In this embodiment, the pixel conversion module 707 is used to perform image processing on pixel data, convert the pixel data into grayscale data, and transmit the grayscale data to the parallel processing module 708.

[0190] In this embodiment, the parallel processing module 708 is used to generate display driving data based on grayscale data and transmit the display driving data to the image display module 704.

[0191] It is understandable that when the CP side transmits data, the data displayed on the screen is display driver data. In this embodiment, the CP side performs corresponding operations based on the different types of data transmitted by the AP side. In pass-through mode, the CP side directly transmits the second data to be displayed as display driver data to the screen. In high-definition mode, the CP side performs image processing operations such as conversion on the first data to be displayed to obtain display driver data, and then transmits the display driver data to the screen.

[0192] Understandable. Figure 7 The LiteTCON lightweight image processing module included in the terminal device 100 is only one possible division method. In actual applications, the LiteTCON lightweight image processing module in the terminal device 100 may include more or fewer modules, or there may be other module division methods. This application does not limit this.

[0193] In this way, after the system layer processes the interaction between the AP heterogeneous network service module and the CP heterogeneous network service module, it can determine whether the display control sovereignty lies with the AP or the CP, which enriches the display function of the CP. When the display control sovereignty lies with the CP, the CP is responsible for sending the display, which can reduce the overall power consumption of the system.

[0194] In this embodiment, the AP-side driver layer includes a general memory management system, IPC, shared memory, and MIPI.

[0195] For example, the general memory management interface of the AP-side driver layer can provide a general memory management interface to manage different types of memory. Inter-process communication is used to transfer or exchange information between different processes. Shared memory is used for sharing data between different processes (such as sharing display data between AP and CP). MIPI is the interface between AP and display screen, used to send the first display data generated by the AP-side system layer to the display screen so that the display screen can display the first display data.

[0196] In this embodiment, the CP-side driver layer includes shared memory and MIPI.

[0197] For example, shared memory is used for sharing data (such as display data (first display data, second display data), pixel data, etc.) between the AP and CP. MIPI connects to the display screen and is used to send the second display data generated by the CP-side system layer to the display screen for display.

[0198] In this embodiment, the hardware layer includes an access point (AP), double data rate SDRAM (also described as DDR memory), a display screen, and a content provider (CP). The AP and CP work together to determine the ownership of display control. If display control belongs to the AP, the AP generates an image and displays it. If display control belongs to the CP, the AP first generates an image, the CP synchronously acquires the image generated by the AP, and then the CP displays the image obtained from the AP.

[0199] Understandable Figure 4The system architecture of the terminal device 100 described herein is only one possible division method. In practical applications, the system architecture of the terminal device 100 may include more or fewer modules, or there may be other module division methods. This application does not limit this.

[0200] It is understood that in the embodiments of this application, the terminal device may execute some or all of the steps in the embodiments of this application. These steps or operations are merely examples, and the embodiments of this application may also execute other operations or variations thereof. Furthermore, the steps may be executed in different orders as presented in the embodiments of this application, and it is not necessary to execute all the operations in the embodiments of this application.

[0201] For example, the technical solutions involved in the following embodiments can all be implemented in the terminal device 100 described above. The display method provided by the embodiments of this application will be described in detail below with reference to the accompanying drawings and application scenarios.

[0202] The technical solutions involved in the following embodiments are described with the first processing unit being AP and the second processing unit being CP.

[0203] For example, Figure 8 The diagram shows a flowchart of a display method provided in an embodiment of this application, which includes the following steps S801-S805:

[0204] S801, The terminal device determines whether the current situation is a CP control scenario.

[0205] If yes, proceed to step S802; otherwise, proceed to step S805.

[0206] In this embodiment, the CP control scenario refers to a scenario where the display is controlled by the CP. For example, the CP control scenario could be a reading / browsing scenario or a screen-on standby scenario. That is, a scenario where the user is using the terminal device to read text or images, or a scenario where the user is not operating the terminal device while the screen is on. Examples include scenarios where the user is using the terminal device to read documents, novels, notes, or browse web pages.

[0207] In one possible implementation, the terminal device can determine whether the current scenario is a CP control scenario based on the application used by the user. For example, if the terminal device launches a reading application and displays reading content in response to a user's action, then the current scenario can be determined to be a CP control scenario. This application does not limit the specific implementation method by which the terminal device determines whether the current scenario is a CP control scenario.

[0208] It is understood that the embodiments of this application are based on Figure 1Based on experimental data, the reading and browsing scenario is pre-configured as a CP-controlled scenario, with the CP taking over display control in this scenario. Furthermore, in the screen-on standby scenario, the content displayed on the terminal device remains unchanged, and the CP can also take over display control in this scenario. In other words, the low-power processing unit (CP) takes over display control in both the reading and browsing scenarios and the screen-on standby scenario, effectively reducing system power consumption. In the future, with upgrades and changes to the terminal device, other scenarios can also be pre-configured as CP-controlled scenarios.

[0209] S802, the AP of the terminal device generates and processes data.

[0210] In this embodiment of the application, the processed data includes preprocessed data or transformed data, and the preprocessed data is image data that can be displayed on the CP side.

[0211] The preprocessed data is the image data obtained by the AP after performing preprocessing operations on the currently displayed image. The transformed data is the image data obtained by the AP after transforming the preprocessed data.

[0212] The image data includes constant pattern images, enumerable pattern images, and combined pattern images. Images are the most basic data units displayed on the terminal device. For example, if the terminal device displays the number "13", then the images of "13" are "1" and "3".

[0213] Among them, constant mode primitives are primitives that remain unchanged during display, such as the Bluetooth icon in the status bar of a terminal device's display interface or each character in the document content displayed on the display interface (each character is a primitive). Enumerable mode primitives are images that change but have limited changes during display, such as each character representing time in a terminal device's display interface. Composite mode primitives are primitives composed of a combination of scene mode primitives and enumerable mode primitives.

[0214] In some embodiments of this application, the AP of the terminal device performs preprocessing operations on the currently displayed image to obtain preprocessed data.

[0215] The preprocessing operations include classifying graph data and drawing graph data.

[0216] For example, the AP classifies the metadata of each image element in the currently displayed image according to the metadata category, and draws and generates the corresponding preprocessed data based on the classified metadata.

[0217] In some other embodiments of this application, the AP of the terminal device first performs preprocessing operations on the currently displayed image to obtain preprocessed data; then performs conversion operations on the preprocessed data to obtain converted data.

[0218] The conversion operations include image conversion processing operations such as grayscale processing and temporal processing.

[0219] For example, after the AP performs preprocessing operations on the currently displayed image to obtain preprocessed data, the AP can also read the pixel data in the preprocessed data, convert the pixel data into grayscale data, and generate conversion data based on the grayscale data.

[0220] The grayscale data can be any of the three modes: black, white, or gray.

[0221] In one possible implementation, the AP reads a pixel from the preprocessed data, determines the corresponding grayscale data from a preset grayscale conversion table based on the pixel data, and performs time-series processing on the grayscale data to generate converted data.

[0222] It's understandable that when the AP generates and processes data, if it only preprocesses the currently displayed image and synchronizes the resulting preprocessed data to the CP side, then the data sent from the AP to the CP side hasn't undergone conversion processing. Therefore, after receiving the preprocessed data from the AP, the CP side still needs to perform conversion processing before sending the converted data for display. Alternatively, if the AP generates and processes data, performs preprocessing on the currently displayed image and then converts the preprocessed data, subsequently synchronizing the converted data to the CP side, then the data sent from the AP to the CP side has already undergone conversion processing. Therefore, after receiving the converted data from the AP, the CP side doesn't need to perform further conversion processing and can directly combine the converted data sent by the AP according to a preset method before sending it for display.

[0223] In this embodiment, the AP does not need to draw all currently displayed images; it only needs to draw and generate preprocessed data corresponding to each image data element based on the image data elements contained in the currently displayed image.

[0224] For example, the currently displayed image is as follows: Figure 9As shown, the currently displayed image 900 includes a signal icon 901 and a time display 902. After determining the currently displayed image 900, the AP first classifies the metadata of each element in the image 900 according to its element type, resulting in two types of metadata: constant pattern metadata (signal icon) and enumerable pattern metadata ("0", "8", ":"). Then, based on the classified metadata, the AP draws the corresponding preprocessed data. For example, based on the signal icon, it draws constant pattern metadata related to the signal icon, such as Bluetooth icon and alarm clock icon. Based on "0", "8", and ":", it draws enumerable pattern metadata related to "0", "8", and ":", including "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", and ":". Integrating the metadata from different categories, the AP draws the generated related metadata to obtain the preprocessed data.

[0225] In this way, the preprocessed data after AP preprocessing only includes relevant map data. Only the preprocessed data needs to be transmitted and stored, which greatly reduces the data size and optimizes memory storage.

[0226] In this embodiment, an AP heterogeneous network service module is deployed on the AP, which includes a data collaboration management module and a state collaboration management module. A CP heterogeneous network service module is deployed on the CP, which includes a data collaboration management module, a state collaboration management module, and a LiteTCON lightweight image processing module.

[0227] The AP-side data collaboration management module interacts with the CP-side data collaboration management module, managing data collaboration between the AP and CP. The AP-side state collaboration management module interacts with the CP-side state collaboration management module, managing state collaboration between the AP and CP. The CP-side LiteTCON lightweight image processing module provides lightweight e-ink screen image processing capabilities, enabling the CP to take over display control and complete display interactions.

[0228] In step S802, after the terminal device determines that the current scene is a CP control scene, the data collaboration management module on the AP side of the terminal device generates processing data based on the currently displayed image.

[0229] Specifically, the data collaboration management module on the AP side includes a data drawing interface, a data management module, and a data synchronization module.

[0230] The data drawing interface includes a constant pattern primitive drawing unit, an enumerable pattern primitive drawing unit, and a combined pattern primitive drawing unit. The constant pattern primitive drawing unit is used to draw corresponding constant pattern primitive data based on the constant pattern primitives in the currently displayed image; the enumerable pattern primitive drawing unit is used to draw corresponding enumerable pattern primitive data based on the enumerable pattern primitives in the currently displayed image; and the combined pattern primitive drawing unit is used to draw corresponding combined pattern primitive data based on the combined pattern primitives in the currently displayed image.

[0231] The data drawing interface is also used to integrate the graphic data drawn by each graphic element drawing unit into preprocessed data.

[0232] The data plotting interface is also used to perform transformation operations on preprocessed data to obtain transformed data.

[0233] The data management module includes a primitive management unit and a primitive distribution unit. The primitive management unit manages the processed data (preprocessed or transformed data) generated by the data drawing interface. The primitive distribution unit distributes the processed data (preprocessed or transformed data) generated by the data drawing interface to the data synchronization module, so that the AP side can synchronize the processed data (preprocessed or transformed data) to the CP side.

[0234] The data synchronization module is used to synchronize the processed data (preprocessed data or transformed data) generated on the AP side to the CP side.

[0235] In this embodiment, after generating processing data, the data collaboration management module on the AP side saves the processing data in shared content so that the data collaboration management module on the CP side can retrieve the processing data from the shared memory.

[0236] S803, the terminal device will process data and synchronize it from the AP to the CP.

[0237] In this embodiment, after generating processing data, the AP sends the processing data to the CP. Correspondingly, the CP receives the processing data from the AP.

[0238] Specifically, the data collaboration management module on the CP side includes a data display interface, a data management module, and a data synchronization module.

[0239] The data synchronization module is used to receive processing data sent from the AP side. That is, the data synchronization module on the AP side sends processing data to the data synchronization module on the CP side, thereby synchronizing the processing data from the AP to the CP.

[0240] In this embodiment, the data synchronization methods between the AP-side data synchronization module and the CP-side data synchronization module include, but are not limited to, shared memory interaction.

[0241] In this embodiment of the application, after receiving the processed data from the AP, the CP returns response information to the AP so that the AP can determine that the CP has received the processed data.

[0242] In some embodiments of this application, after the terminal device synchronizes the processing data from AP to CP, if the display control belongs to AP, the terminal device first switches the display control from AP to CP, and the CP takes over the display control; after the CP takes over the display control, S804 is executed.

[0243] For example, if the previous scenario was not a CP-controlled scenario, and the current scenario is a CP-controlled scenario, when the terminal device was displaying in the previous scenario, the display was controlled by the AP, and the display control belonged to the AP. In the current scenario, since the current scenario is a CP-controlled scenario, the terminal device switches the display control from the AP to the CP, and the CP takes over the display control.

[0244] In this embodiment of the application, the terminal device switches the display control of the AP to the CP according to the synchronization and coordination state.

[0245] In this embodiment, after receiving the response information sent by the CP, the AP enables the heterogeneous collaborative display function, changes the collaborative state, and synchronizes the changed collaborative state to the CP, thus transferring the display control from the AP to the CP. Correspondingly, the CP receives and synchronizes the changed collaborative state sent by the AP, and takes over the display control according to the synchronized collaborative state.

[0246] Specifically, both the AP-side state coordination management module and the CP-side state coordination module include an enable state interface, a coordination state management module, and a coordination state synchronization module.

[0247] The enable state interface is responsible for enabling and disabling the heterogeneous collaborative display function, serving as the entry point for UI heterogeneous collaborative display. The collaborative state management module is responsible for managing and updating the collaborative state between AP and CP. The collaborative state synchronization module is responsible for synchronizing the states of AP and CP.

[0248] For example, after receiving the response information sent by the CP, the AP enables the state interface of its state coordination module. The AP-side coordination state management module responds to this by changing the coordination state to "coordination enabled." The AP-side coordination state synchronization module then synchronizes the changed coordination state (coordination enabled) to the coordination state synchronization module of the CP-side state coordination module. Correspondingly, the CP-side coordination state management module changes the coordination state to "coordination enabled" and enables the CP-side state interface. At this point, the CP takes over display control.

[0249] Optionally, when the terminal device exits the CP control scenario, the CP side disables the heterogeneous collaborative display function, changes the collaborative state, and synchronizes the changed collaborative state to the AP, switching the display control from the CP to the AP. Correspondingly, the AP receives and synchronizes the changed collaborative state sent by the CP, and takes over the display control based on the synchronized collaborative state.

[0250] It should be understood that the CP is a low-power processing unit. Taking over the display control by the CP can reduce the AP's running time, increase the probability of the AP going into sleep mode, and greatly save power consumption.

[0251] It is understood that this application does not limit the execution order of the terminal device synchronizing processing data from the AP to the CP when display control belongs to the AP, and the terminal device switching display control from the AP to the CP. These can be executed synchronously or sequentially. When executed sequentially, this application does not limit the order.

[0252] In some other embodiments of this application, after the terminal synchronizes the processing data from AP to CP, if the control right belongs to CP, S804 can be executed directly.

[0253] For example, if both the previous and current scenarios are CP-controlled scenarios, since the display control has already been switched from AP to CP in the first scenario, the display control will still be controlled by CP in the second scenario until the terminal usage scenario is no longer a CP-controlled scenario, at which point the display control will be switched from CP to AP and the AP will perform the display control.

[0254] S804: The CP of the terminal device determines the display driver data based on the processed data and transmits the display driver data to the display screen of the terminal device.

[0255] Display driver data, also known as timing control data, can include the waveforms and timing sequences required for displaying pixels on the terminal device.

[0256] In some embodiments of this application, if the processing data synchronized by the CP is preprocessed data, the CP of the terminal device determines the display driving data based on the processing data, including: the CP generating first data to be displayed based on the preprocessed data; the CP reading pixel data in the first data to be displayed in parallel; the CP converting the pixel data into grayscale data; and the CP generating display driving data based on the grayscale data.

[0257] In one possible implementation, after the CP sends preprocessed data to the AP side, it combines the preprocessed data according to a preset method to generate the first data to be displayed.

[0258] For example, based on the above S802 and Figure 9For example, the preprocessed data includes constant pattern graph metadata: signal icons, enumerable pattern graph metadata: "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", and ":". The CP side determines the graph metadata that needs to be displayed in the preprocessed data and encapsulates and combines the graph metadata to be displayed to generate the first data to be displayed. That is, the CP side encapsulates and combines the signal icons of the constant pattern graph metadata in the preprocessed data, as well as the enumerable pattern graph metadata "0", "8", and ":", to generate the first data to be displayed.

[0259] Specifically, after the data synchronization module of the data collaboration management module on the CP side synchronizes the preprocessed data generated on the AP side, the data display interface of the data collaboration management module on the CP side draws and generates the first data to be displayed based on the preprocessed data.

[0260] The data display interface on the CP side includes constant pattern primitive control units, enumerable pattern primitive control units, and composite pattern primitive control units.

[0261] Among them, the constant pattern primitive control unit generates the first constant pattern primitive control based on the constant pattern primitive data in the preprocessed data; the enumerable pattern primitive control unit generates the first enumerable pattern primitive control based on the enumerable pattern primitive data in the preprocessed data; and the composite pattern primitive control unit generates the first composite pattern primitive control based on the composite pattern primitive data in the preprocessed data.

[0262] The data display interface is also used to integrate the first primitive control generated by each primitive control unit (constant mode primitive control unit, enumerable mode primitive control unit, and composite mode primitive control unit) into the first data to be displayed.

[0263] The data management module on the CP side includes a primitive management unit and a primitive update unit. The primitive management unit manages the first primitive controls generated by each unit in the data display interface (the first primitive controls include a first constant mode primitive control, a first enumerable mode primitive control, and a first combined mode primitive control). The primitive update unit updates each first primitive control.

[0264] In one possible implementation, the CP reads pixel data from the first set of data to be displayed in parallel via direct memory access (DMA).

[0265] In one possible implementation, after the terminal device CP side reads the pixel data in the first pixel to be displayed in parallel, it converts the pixel data into grayscale data according to a preset grayscale conversion table.

[0266] The preset grayscale conversion table can be determined based on experimental tests, historical data, or experience.

[0267] Specifically, after reading the pixel data of the pixels to be displayed in parallel, the grayscale data corresponding to the RGBA value of the pixel data is determined from a preset grayscale conversion table based on the red, green, and blue alpha (RGBA) values ​​of the pixel data. Thus, using the RGBA value of the pixel data as an index, the corresponding grayscale data is determined by looking up the table, converting the pixel data into grayscale data.

[0268] In one possible implementation, the terminal device CP converts pixel data into grayscale data, and then performs a conversion operation based on the currently displayed image and the grayscale data to generate display driving data.

[0269] The conversion operations include image conversion processing operations such as grayscale processing and temporal processing.

[0270] For example, such as Figure 10 The diagram illustrates a method for generating display driving data according to an embodiment of this application. S1 in the diagram represents the currently displayed image data of the terminal device interface (which can be understood as the first frame image). In the second frame image processing cycle, the CP side performs a conversion operation based on the currently displayed image data S1, grayscale data, and the screen refresh rate of the terminal device to generate display driving data S2 (which can be understood as the second frame image). The grayscale data is obtained from the raw AP processing data read in parallel from memory by the CP. Subsequently, the terminal device's display screen shows the display driving data S2. After the terminal device displays the display driving data S2, the display driving data S2 becomes the currently displayed image data corresponding to the next processing cycle. That is, in the third frame image processing cycle, the CP side performs a conversion operation based on the display driving data S2, grayscale data, and the screen refresh rate of the terminal device to generate display driving data S3 (which can be understood as the third frame image). Following the above method, the CP side ultimately generates n frames of display driving data S2, S3, ..., Sn+1, and sends these n frames of display driving data to the display screen frame by frame for display. Thus, display driver data to be displayed is generated based on the currently displayed image data, grayscale data, and the screen refresh rate of the terminal device.

[0271] In this embodiment, after the CP side generates the first data to be displayed, it transmits the first data to be displayed to the LiteTCON lightweight image processing module, and the image input module transmits the first data to be displayed to the processing mode selection module.

[0272] In this embodiment, the LiteTCON lightweight image processing module on the CP side of the terminal device has a lightweight timing controller. Therefore, the CP side has image conversion capability. In this way, the conversion operation is completed on the low-power core, reducing the amount of computation and memory usage for timing control in the conversion operation.

[0273] The LiteTCON lightweight image processing module's processing mode selection module determines the current processing mode as high-definition mode based on the first data to be displayed, and then transmits the first data to be displayed to the high-definition processing module.

[0274] The high-definition processing module of the LiteTCON lightweight image processing module is the processing module corresponding to the high-definition mode, which is used to transmit the first data to be displayed to the parallel reading module.

[0275] The parallel reading module of the LiteTCON lightweight image processing module is used to read pixel data in the first set of data to be displayed in parallel and transfer the pixel data to the pixel conversion module.

[0276] The pixel conversion module of the LiteTCON lightweight image processing module is used to process pixel data, convert pixel data into grayscale data, and transmit the grayscale data to the parallel processing module.

[0277] The parallel processing module of the LiteTCON lightweight image processing module is used to generate display driving data based on grayscale data and transmit the display driving data to the image display module.

[0278] In some other embodiments of this application, if the processing data synchronized by the CP is conversion data, the CP of the terminal device determines the display driving data based on the processing data, including: the CP generating second data to be displayed based on the conversion data; and the CP determining the second data to be displayed as the display driving data.

[0279] In one possible implementation, after the CP sends the conversion data to the AP side, it combines the conversion data according to a preset method to generate the second data to be displayed.

[0280] Specifically, after the data synchronization module of the data collaboration management module on the CP side synchronizes the conversion data generated on the AP side, the data display interface of the data collaboration management module on the CP side draws and generates the second data to be displayed based on the conversion data.

[0281] The constant pattern primitive control unit in the data display interface on the CP side generates a second constant pattern primitive control based on the constant pattern primitive data in the conversion data; the enumerable pattern primitive control unit generates a second enumerable pattern primitive control based on the enumerable pattern primitive data in the conversion data; and the composite pattern primitive control unit generates a second composite pattern primitive control based on the composite pattern primitive data in the conversion data.

[0282] The data display interface is also used to integrate the second primitive controls generated by each primitive control unit (constant mode primitive control unit, enumerable mode primitive control unit, and composite mode primitive control unit) into the second data to be displayed.

[0283] The data management module on the CP side includes a primitive management unit and a primitive update unit. The primitive management unit manages the second primitive controls generated by each unit in the data display interface (the second primitive controls include a second constant mode primitive control, a second enumerable mode primitive control, and a second combined mode primitive control). The primitive update unit updates each second primitive control.

[0284] It is understandable that the data synchronized from the AP side by the CP side is already converted data. Therefore, the CP side does not need to convert the converted data again. The CP side can directly send the second data to be displayed after the converted data is encapsulated and combined according to the preset method.

[0285] In this embodiment, after the CP side generates the second data to be displayed, it transmits the second data to be displayed to the image input module of the LiteTCON lightweight image processing module, and the image input module transmits the second data to be displayed to the processing mode selection module.

[0286] The LiteTCON lightweight image processing module's processing mode selection module determines the current processing mode as pass-through mode based on the second data to be displayed, and then transmits the second data to be displayed to the pass-through processing module.

[0287] The pass-through processing module of the LiteTCON lightweight image processing module determines the second data to be displayed as display driver data and transmits the display driver data to the image sending module.

[0288] For example, such as Figure 11 The diagram shown is a memory optimization illustration provided in an embodiment of this application. Figure 11 The screen resolution of the terminal device is 1920 pixels. 1421, then processing one frame of an image requires processing 1920 data units. 1421 32 / 8 = 10.4M.

[0289] Figure 11 The 1100 in the figure refers to the image data transmitted from the AP to the display screen when the display control belongs to the AP side in the related technology, and the data size is also 1920. 1421 32 / 8 = 10.4M. Furthermore, 1100 is stored discretely in memory in a non-contiguous manner. However, the actual valid image data (e.g., pixels) on the display only accounts for a small portion of 1100. For example, 1101 contains the actual valid image data ( Figure 11 (represented by dashed boxes), data size is 276. 1421 4 = 1.5M.

[0290] Figure 11 In the technical solution provided in this application embodiment, 1102 refers to the image data transmitted from the CP side to the display screen when the display control belongs to the CP side. In this application embodiment, the AP side first determines the valid image data in the image and only processes the valid image data. The actual data size processed in this application embodiment is 276. 1421 4 = 1.5M. Moreover, the 1102 uses a contiguous storage method, storing data continuously in memory, which speeds up memory processing, reduces memory usage, and optimizes memory storage performance.

[0291] In some embodiments of this application, display driving data is transmitted to the display screen of the terminal device, including: the CP side transmits display driving data to the display screen of the terminal device via MIPI.

[0292] In this embodiment, the image delivery module of the LiteTCON lightweight image processing module receives display driving data sent by the parallel processing module, or the image delivery module receives display driving data sent by the pass-through processing module. The image delivery module transmits the received display driving data to the display screen via MIPI.

[0293] In this embodiment of the application, after receiving the display driver data, the display screen displays an image according to the display driver data.

[0294] S805, the terminal equipment uses AP for display control.

[0295] In this embodiment, if the terminal device determines that the current scene is not a CP-controlled scene, then the display control belongs to the AP, and the AP performs the display control. In one possible implementation, the terminal device uses the AP for image processing and image display. For example, the AP, through the above... Figure 2 The processing flow shown executes various UI display tasks, such as drawing, rendering, compositing, grayscale clipping, and TCON image processing, to obtain display driver data. The AP side transmits the display driver data to the terminal device's display screen via MIPI, and the display screen shows the image based on the display driver data.

[0296] Based on the above technical solution, after determining that the current scenario is controlled by a CP, the terminal device generates preprocessed data on the AP side and synchronizes the preprocessed data to the CP side. The CP side then completes the display interaction based on the preprocessed data. In other words, in a CP-controlled scenario, the AP processes the data required for display, and the CP sends the processed data to the display screen for display. By sharing the AP's computing power, the CP side uses the preprocessed data generated by the AP to deploy the UI display load (such as display sending operations) from the AP to the CP, reducing the UI display load deployed on the AP. Furthermore, the deployment of a lightweight timing controller on the CP enables the low-power CP to take over display interaction in application scenarios, broadening the application scenarios of the CP. Since the CP core consumes less power than the AP, after the CP takes over display control, it can increase the probability of the AP going into sleep mode, thus reducing the power consumption of the terminal device and improving its battery life.

[0297] For example, Figure 12 A schematic diagram illustrating a display effect provided by an embodiment of this application is shown. For example... Figure 12 The first interface 1201 displays the image shown by the terminal device when control is held by the AP side. The second interface 1202 displays the image shown by the terminal device when control is held by the CP side. It can be seen that there is no difference in display quality between the two interfaces; the content is clear and smooth with no ghosting during page turns.

[0298] For example, Figure 13 A schematic diagram illustrating a benefit effect provided by an embodiment of this application is shown. It can be understood that... Figure 13 This explanation uses a reading scenario where the terminal device turns a page once per minute as an example. In this scenario, the technical solution provided in this application embodiment can reduce the power consumption of the terminal device by approximately 20 milliamps.

[0299] like Figure 13 As shown in (a), when the display control belongs to the AP and the AP processes the display, the current consumption of the terminal device is 107.05 mA. For example, the current consumption of the terminal device is 107.05 mA in the time between 1 minute and 2 minutes.

[0300] like Figure 13 As shown in Figure (b), the current power consumption of the terminal device corresponding to the technical solution provided in this application embodiment is as follows: when the display control right belongs to the CP and the CP takes over the display, the current power consumption of the terminal device is as follows: for example, within 3 to 4 minutes, the current power consumption of the terminal device is 82.35 mA, which is reduced by about 20 mA.

[0301] For example, Figure 14 A schematic diagram illustrating another benefit effect provided by an embodiment of this application is shown. It can be understood that... Figure 14This explanation uses a terminal device in a screen-on standby scenario as an example. In this scenario, the technical solution provided in this application embodiment can reduce the power consumption of the terminal device by approximately 10 milliamps.

[0302] like Figure 14 As shown in (a), when the display control belongs to the AP and the AP processes the display, the current consumption of the terminal device is 93.37 mA.

[0303] like Figure 14 As shown in Figure (b), the current power consumption of the terminal device corresponding to the technical solution provided in the embodiment of this application is as follows: when the display control right belongs to the CP and the CP takes over the display, the current power consumption of the terminal device is as follows: for example, within 2 to 3 minutes, the current power consumption of the terminal device is 82.87 mA, which is reduced by about 10 mA.

[0304] It is understood that the above description uses an e-ink screen terminal device as an example, and the technical solutions provided in this application can also be applied to terminal devices where the display screen is not an e-ink screen. When the display screen of the terminal device is not an e-ink screen, there is no need to perform grayscale processing and timing processing, etc.

[0305] For example, a non-e-ink display screen can be any product or component with display functionality, such as a liquid crystal display (LCD), an organic light-emitting diode display (OLED) (which can also be described as an OLED screen), an active-matrix organic light-emitting diode screen (AMOLED), a low-temperature polycrystalline oxide screen (LTPO), or a flexible display screen.

[0306] It should be understood that images displayed on non-e-ink screens only require drawing, rendering, and compositing operations before being displayed, and the composite image is directly sent to the display. Unlike e-ink screens, which require grayscale processing and timing processing of the composite image before sending it to the display, non-e-ink screens do not need to do so.

[0307] For example, such as Figure 15 The diagram shown is a flowchart of another display method provided in an embodiment of this application. The method includes the following steps S1501-S1505:

[0308] S1501, The terminal device determines whether the current situation is a CP control scenario.

[0309] If yes, proceed to step S1502; otherwise, proceed to step S1505.

[0310] The specific implementation method for the terminal device to determine whether the current situation is a CP control scenario is described in S801 above, and will not be repeated here.

[0311] S1502, The AP of the terminal device generates preprocessed data.

[0312] In this embodiment of the application, when the terminal device is currently in a CP control scenario, the AP only needs to perform preprocessing operations on the currently displayed image to obtain preprocessed data.

[0313] In this embodiment, the AP of the terminal device is equipped with an AP heterogeneous network service module, which includes a data collaboration management module and a status collaboration management module. The CP is equipped with a CP heterogeneous network service module, which includes a data collaboration management module and a status collaboration management module.

[0314] The specific implementation of the AP generating preprocessed data in the terminal device is described in S802 above, and will not be repeated here.

[0315] S1503, The terminal device synchronizes the preprocessed data from the AP to the CP.

[0316] In this embodiment, the terminal device synchronizes preprocessed data from the AP to the CP so that the CP can display the data based on the preprocessed data.

[0317] The specific implementation of how the terminal device synchronizes preprocessed data from AP to CP is described in S803 above, and will not be repeated here.

[0318] In some embodiments of this application, after the terminal device synchronizes the processing data from AP to CP, if the display control belongs to AP, the terminal device first switches the display control from AP to CP, and then executes S1504.

[0319] In some other embodiments of this application, after the terminal device synchronizes the processing data from the AP to the CP, if the control right belongs to the CP, then S1504 is executed directly.

[0320] S1504, the CP of the terminal device determines the display driver data based on the preprocessed data and transmits the display driver data to the display screen of the terminal device.

[0321] In one possible implementation, after the CP synchronizes the preprocessed data sent by the AP side, it combines the preprocessed data according to a preset method to generate display driver data. The CP then transmits the display driver data to the display screen of the terminal device via MIPI.

[0322] S1505, The terminal equipment uses an AP for display control.

[0323] For details on how terminal devices use AP for display control, please refer to S805 above, which will not be repeated here.

[0324] In this way, the terminal device can be displayed by the CP and AP working together, and some UI display load can be deployed to the CP, which enables the CP to take over the display interaction in the application scenario. The display is completed by the low-power core CP, which reduces the overall power consumption of the terminal device when displaying while ensuring the same effect on the CP side and the AP side, and improves the battery life of the terminal device.

[0325] The above primarily describes the solutions provided by the embodiments of this application from a methodological perspective. It is understood that, in order to achieve the above functions, the terminal device includes hardware structures and / or software modules corresponding to the execution of each function. Based on the units and algorithm steps of the various examples described in the embodiments disclosed in this application, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by a computer driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solutions of the embodiments of this application.

[0326] This application provides embodiments that can divide a terminal device into functional modules based on the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional module. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0327] like Figure 16 The diagram shown illustrates the structure of another terminal device provided in this application embodiment. This terminal device 1600 can be used to implement the methods described in the above method embodiments. For example, the terminal device 1600 may specifically include: a processing module 1601, an acquisition module 1602, and a display module 1603.

[0328] The processing module 1601 is used to execute commands that support the terminal device 1600. Figures 4 to 15 The processing function of any item in it.

[0329] The acquisition module 1602 is used to execute the execution of the terminal device 1600. Figures 4 to 15This allows you to retrieve any item from the list, such as the currently displayed image.

[0330] The display module 1603 can be used to display screens, etc., according to the display driver. And / or, the display module 1603 is also used to support the terminal device 1600 in performing other display operations performed by the terminal device in the embodiments of this application.

[0331] Optional, Figure 16 The terminal device 1600 shown may also include a communication module ( Figure 16 (Not shown in the image), this communication module is used to support the terminal device 1600 in performing the steps of communication between the terminal device and other devices in the embodiments of this application.

[0332] Optional, Figure 16 The terminal device 1600 shown may also include a storage module ( Figure 16 (Not shown in the image), this storage module stores programs or instructions. When the processing module 1601 executes the program or instructions, it causes... Figure 16 The terminal device 1600 shown can execute the methods shown in the above method embodiments.

[0333] Figure 16 The technical effects of the terminal device 1600 shown can be referred to the technical effects of the method described in the above method embodiments, and will not be repeated here. Figure 16 The processing module 1601 involved in the terminal device 1600 shown can be implemented by a processor or processor-related circuit components, and can be a processor or a processing module. The communication module can be implemented by a transceiver or transceiver-related circuit components, and can be a transceiver or a transceiver module. The display module 1603 can be implemented by display screen-related components.

[0334] like Figure 17 The diagram shown is a structural schematic of another terminal device provided in this application embodiment. This terminal device 1700 can be used to implement the methods described in the above method embodiments. For example, the terminal device 1700 may specifically include: a first processing unit 1701, a second processing unit 1702, and a display screen 1703.

[0335] In this embodiment of the application, the first processing unit 1701 is used to determine whether the current scene is a scene controlled by the second processing unit.

[0336] In this embodiment of the application, the first processing unit 1701 is further configured to generate processing data if the current scene is a scene controlled by the second processing unit; the first processing unit 1701 is further configured to switch the display control from the first processing unit 1701 to the second processing unit 1702; the second processing unit 1702 is configured to generate first display driving data according to the processing data; and the display screen 1703 is configured to display according to the first display driving data.

[0337] In this embodiment of the application, the first processing unit 1701 is further configured to generate second display driving data if the current scene is not a scene controlled by the second processing unit; the display screen 1703 is further configured to display according to the second display driving data.

[0338] In this embodiment of the application, the first processing unit 1701 is a high-power processing unit, and the second processing unit 1702 is a low-power processing unit.

[0339] In this embodiment of the application, if the display screen is an electronic ink screen, the processing data includes preprocessed data, and the preprocessed data is the first graphic data generated by the first processing unit 1701. Then the second processing unit 1702 is further used to generate first data to be displayed based on the preprocessed data. The second processing unit 1702 is further used to perform a conversion operation on the first data to be displayed to generate first display driving data.

[0340] In this embodiment of the application, the first data to be displayed is a combination of preprocessed data to obtain the first graphic element control.

[0341] In this embodiment of the application, if the display screen is an electronic ink screen, the processing data includes conversion data. The conversion data is the second image data generated after the first processing unit 1701 performs a conversion operation on the first image data generated by the first processing unit. Then, the second processing unit 1702 is further used to generate second data to be displayed based on the conversion data. The second processing unit 1702 is further used to determine the second data to be displayed as the first display driving data.

[0342] In this embodiment of the application, the second data to be displayed is a combination of converted data to obtain a second graphic element control.

[0343] In this embodiment of the application, the first processing unit 1701 is further configured to classify the image data in the currently displayed image; the first processing unit 1701 is further configured to draw and generate processing data based on the classified image data.

[0344] In this embodiment of the application, the conversion operation includes: grayscale processing or timing processing.

[0345] In this embodiment of the application, if the display screen is not an electronic ink screen, the processing data includes preprocessed data, and the preprocessed data is the first image data generated by the first processing unit 1701. Then the second processing unit 1702 is also used to combine the preprocessed data according to a preset method to generate the first display driving data.

[0346] In this embodiment of the application, the second processing unit 1702 is further configured to synchronously process data from the first processing unit 1701 before the display control is switched from the first processing unit 1701 to the second processing unit 1702.

[0347] In this embodiment, the second processing unit controls either a reading / browsing scenario or a screen-on standby scenario.

[0348] This application also provides a chip system, such as... Figure 18 As shown, the chip system includes at least one processor 1801 and at least one interface circuit 1802. The processor 1801 and the interface circuit 1802 are interconnected via lines. For example, the interface circuit 1802 can be used to receive signals from other devices. As another example, the interface circuit 1802 can be used to send signals to other devices (e.g., the processor 1801). Exemplarily, the interface circuit 1802 can read instructions stored in memory and send those instructions to the processor 1801. When the instructions are executed by the processor 1801, the terminal device can perform the various steps executed by the terminal device in the above embodiments. Of course, the chip system may also include other discrete components, and this application embodiment does not specifically limit this.

[0349] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0350] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application does not specifically limit the type of memory or the arrangement of the memory and processor.

[0351] For example, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0352] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0353] This application also provides a computer storage medium storing computer instructions, which, when executed on a terminal device, cause the terminal device to perform the methods described in the above-described method embodiments.

[0354] This application provides a computer program product, which includes a computer program or instructions that, when run on a computer, cause the computer to perform the methods described in the above-described method embodiments.

[0355] In addition, this application also provides an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the apparatus to perform the methods in the above-described method embodiments.

[0356] In this embodiment, the terminal device, computer storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0357] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0358] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The embodiments can be combined with or referenced to each other without conflict. The apparatus embodiments described above are merely illustrative; for example, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0359] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0360] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0361] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0362] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A terminal device, comprising a first processing unit, a second processing unit, and a display screen, characterized in that, The first processing unit is used to determine whether the current scene is a scene controlled by the second processing unit; The first processing unit is further configured to generate processing data if the current scene is a scene controlled by the second processing unit; the second processing unit is configured to synchronize the processing data from the first processing unit and generate first display driving data based on the processing data; The display screen is used to display according to the first display driving data; the first display driving data is image data generated by the second processing unit and used to drive the display screen to display; the second processing unit is further used to take over the display control from the first processing unit if the current display control belongs to the first processing unit after synchronizing the processing data from the first processing unit; the display control is the right to control the display subsystem; The first processing unit is further configured to generate second display driving data if the current scene is not a scene controlled by the second processing unit; the display screen is configured to display according to the second display driving data; The second display driving data is image data generated by the first processing unit and used to drive the display screen to display the image. The first processing unit is a high-power processing unit, and the second processing unit is a low-power processing unit.

2. The terminal device according to claim 1, characterized in that, The display screen is an electronic ink screen, and the processing data includes preprocessed data, which is the first image data generated by the first processing unit. The second processing unit is further configured to generate first data to be displayed based on the preprocessed data; The first data to be displayed is a first primitive control obtained by combining the preprocessed data; The second processing unit is further configured to perform a conversion operation on the first data to be displayed to generate the first display driving data.

3. The terminal device according to claim 1, characterized in that, The display screen is an electronic ink screen, and the processing data includes conversion data, which is the second image data generated by the first processing unit after performing a conversion operation on the first image data generated by the first processing unit. The second processing unit is further configured to generate second data to be displayed based on the converted data; the second data to be displayed is a second graphic element control obtained by combining the converted data; The second processing unit is further configured to determine the second data to be displayed as the first display driving data.

4. The terminal device according to any one of claims 1-3, characterized in that, The first processing unit is also used to classify the image data in the currently displayed image; The first processing unit is further configured to generate the processed data based on the classified image data.

5. The terminal device according to any one of claims 2-3, characterized in that, The conversion operations include grayscale processing or time-series processing.

6. The terminal device according to claim 1, characterized in that, The display screen is a non-e-ink screen, and the processing data includes preprocessed data, which is the first image data generated by the first processing unit. The second processing unit is further configured to combine the preprocessed data according to a preset method to generate the first display driving data.

7. The terminal device according to any one of claims 1-3, characterized in that, The second processing unit controls scenarios including reading / browsing scenarios or screen-on standby scenarios.

8. A display method, characterized in that, Applied to a terminal device including a first processing unit, a second processing unit, and a display screen, the method includes: Determine whether the current scene is controlled by the second processing unit; If the current scene is controlled by the second processing unit, then after the processing data is generated by the first processing unit, the second processing unit synchronizes the processing data from the first processing unit and generates first display driving data based on the processing data; the display is performed based on the first display driving data; the first display driving data is image data generated by the second processing unit and used to drive the display screen to display. If the current scene is not controlled by the second processing unit, then the first processing unit generates second display driving data; and displays the image data according to the second display driving data. The second display driving data is image data generated by the first processing unit to drive the display screen. Wherein, the first processing unit is a high-power processing unit, and the second processing unit is a low-power processing unit; After the second processing unit synchronizes the processed data from the first processing unit, the method further includes: If the current display control belongs to the first processing unit, then the second processing unit takes over the display control from the first processing unit; the display control is the right to control the display subsystem.

9. The method according to claim 8, characterized in that, The display screen is an electronic ink screen, and the processing data includes preprocessed data, which is the first image data generated by the first processing unit. The second processing unit generates first display driving data based on the processed data, including: The second processing unit generates first data to be displayed based on the preprocessed data; the first data to be displayed is a first primitive control obtained by combining the preprocessed data. The first display driver data is generated by performing a conversion operation on the first data to be displayed.

10. The method according to claim 8, characterized in that, The display screen is an electronic ink screen, and the processing data includes conversion data, which is the second image data generated by the first processing unit after performing a conversion operation on the first image data generated by the first processing unit. The second processing unit generates first display driving data based on the processed data, including: The second processing unit generates second data to be displayed based on the converted data; the second data to be displayed is a second graphic element control obtained by combining the converted data. The second data to be displayed is determined to be the first display driver data.

11. The method according to any one of claims 8-10, characterized in that, The generation of processing data through the first processing unit includes: The first processing unit categorizes the image data in the currently displayed image; The processed data is generated based on the categorized image and data metadata.

12. The method according to any one of claims 9-10, characterized in that, The conversion operations include grayscale processing or time-series processing.

13. The method according to claim 8, characterized in that, The display screen is a non-e-ink screen, and the processing data includes preprocessed data, which is the first image data generated by the first processing unit. The second processing unit generates first display driving data based on the processed data, including: The second processing unit combines the preprocessed data according to a preset method to generate the first display driving data.

14. The method according to any one of claims 8-10, characterized in that, The second processing unit controls scenarios including reading / browsing scenarios or screen-on standby scenarios.

15. A chip system, characterized in that, It includes at least one processor and at least one interface circuit, the at least one interface circuit being used to perform transceiver functions and send instructions to the at least one processor, the at least one processor executing the instructions, and the at least one processor performing the method as described in any one of claims 8-14.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 8-14.

17. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 8-14.

Images