Display method and apparatus
By maintaining display bandwidth when the terminal device detects that the application has opened the camera, the problem of screen flickering or stuttering caused by insufficient display bandwidth is solved, and the smoothness of camera use is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-19
- Publication Date
- 2026-03-13
AI Technical Summary
When a terminal device detects that an application has activated its camera, insufficient display bandwidth can cause screen flickering or stuttering.
When a terminal device detects that an application has activated its camera, it maintains the primary display bandwidth to avoid reducing the display bandwidth to the secondary display bandwidth, thereby reducing screen flickering or stuttering.
By maintaining display bandwidth, screen flickering or stuttering caused by insufficient display bandwidth is reduced, thus improving the smoothness of camera usage.
Smart Images

Figure CN119728900B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of terminal technology, and in particular to a display method and apparatus. Background Technology
[0002] With the widespread adoption and development of terminal devices, people's functional needs for these devices are becoming increasingly diverse. For example, many terminal devices support the use of cameras in certain applications; social applications, for instance, can use cameras to take photos, record videos, scan QR codes, and make video calls. However, when users use the camera in an application, the terminal device may experience screen flickering or lag. Summary of the Invention
[0003] This application provides a display method and apparatus. When a terminal device detects that a first application has turned on its camera, the terminal device maintains a first display bandwidth to reduce the screen flickering or stuttering caused by the terminal device reducing the first display bandwidth to a second display bandwidth after detecting that the application has turned on its camera.
[0004] In a first aspect, embodiments of this application provide a display method, the method comprising: at a first moment, a terminal device receiving a first operation to open a first application; in response to the first operation, the terminal device displaying a first interface;
[0005] The terminal device receives a second operation for turning on the camera in a first application; in response to the second operation, the terminal device displays a second interface, which is the interface displayed by the first application when the camera is turned on; the terminal device receives a third operation for turning off the camera in the first application; at a second moment, in response to the third operation, the terminal device displays a first interface, and between the first moment and the second moment, the display bandwidth of the terminal device is a first value; at a third moment, the terminal device receives a fourth operation for opening the second application; in response to the fourth operation, the terminal device displays a third interface; at a fourth moment, the terminal device receives a fifth operation for turning on the camera in the second application, and between the third moment and the fourth moment, the display bandwidth of the terminal device is the first value; at a fifth moment, in response to the fifth operation, the terminal device displays a fourth interface, which is the interface displayed by the second application when the camera is turned on, and between the fourth moment and the fifth moment, the display bandwidth of the terminal device is the first value; the terminal device receives a sixth operation for turning off the camera in the second application; at a sixth moment, in response to the sixth operation, the terminal device displays a third interface, and between the fifth moment and the sixth moment, the display bandwidth of the terminal device is a second value, where the first value is greater than the second value.
[0006] In this way, when the terminal device detects that the first application has turned on the camera, the terminal device can maintain the first display bandwidth, so as to reduce the screen flickering or stuttering caused by the terminal device reducing the first display bandwidth to the second display bandwidth after detecting that the application has turned on the camera.
[0007] Wherein, the first value can be the value of the first display bandwidth described in the embodiments of this application, and the second value can be the value of the second display bandwidth described in the embodiments of this application.
[0008] In one possible implementation, after responding to the second operation, the method further includes: when the terminal device determines that the first application is a preset application and the camera is on, the terminal device sets the display bandwidth of the terminal device to a first value, wherein the preset application is an application for which the display bandwidth required by the terminal device to run the preset application is greater than a second value, and / or the preset application is an application for which the number of flashes of the terminal device when running the preset application is greater than a first threshold; after responding to the fifth operation, the method further includes: when the terminal device determines that the second application is not a preset application and the camera is on, the terminal device sets the display bandwidth of the terminal device to the second value.
[0009] In this way, when the terminal device detects that the application meets the preset requirements and the camera is turned on, it will set the display bandwidth to the first value to improve the smoothness of camera usage.
[0010] In one possible implementation, the terminal device is configured with: a first application, a camera hardware abstraction layer (HAL), and a hardware synthesis processor. When the terminal device determines that the first application is a preset application and the camera is on, the terminal device sets the display bandwidth of the terminal device to a first value. The method further includes: the first application sending a first message to the camera HAL, the first message including: a message indicating that the camera is on and the package name of the first application; the camera HAL sending the first message to the hardware synthesis processor; when the hardware synthesis processor determines that the first application is a preset application and the camera is on based on the first message, the hardware synthesis processor sets the display bandwidth of the terminal device to the first value.
[0011] In this way, the display synthesizer can set the display bandwidth for a preset application when the camera is on, so as to reduce the screen flickering or stuttering caused by the terminal device reducing the first display bandwidth to the second display bandwidth after detecting that the application has turned on the camera.
[0012] In one possible implementation, the terminal device is further configured with a display synthesizer, a data processor (DPU), and a liquid crystal display (LCD). After the hardware synthesizer sets the display bandwidth of the terminal device to a first value, the method further includes: the hardware synthesizer sending the first value to the DPU; the first application sending N layers to the DPU via the camera HAL, the display synthesizer, and the hardware synthesizer; when the DPU determines that the sum of the display bandwidths of the N layers is less than the first value, the DPU synthesizes the N layers into a target image and sends the target image to the LCD.
[0013] In this way, after setting the display bandwidth to the first value, the terminal device can perform composite processing and display of N layers according to the set display bandwidth, reducing screen flickering or stuttering caused by some layers being composited in the GPU and some in the DPU.
[0014] In one possible implementation, after responding to the third operation, the method further includes: the first application sending a second message to the camera HAL, the second message including: a message indicating that the camera is in a closed state, and the package name of the first application; the camera HAL sending the second message to the hardware synthesis processor; and when the hardware synthesis processor determines that the camera is in a closed state based on the second message, the hardware synthesis processor setting the display bandwidth of the terminal device to a first value.
[0015] In this way, when the terminal device detects that the camera is on or off, it can simultaneously transmit the camera status and application package name to the hardware synthesis processor, enabling the hardware synthesis processor to set the display bandwidth appropriately.
[0016] In one possible implementation, when the terminal device determines that the second application is not a preset application and the camera is on, the terminal device sets the display bandwidth of the terminal device to a second value, including: the second application sending a third message to the camera HAL, the third message including: a message indicating that the camera is on and the package name of the second application; the camera HAL sending the third message to the hardware synthesis processor; when the hardware synthesis processor determines based on the third message that the second application is not a preset application and the camera is on, the hardware synthesis processor sets the display bandwidth of the terminal device to the second value.
[0017] In this way, when the terminal device detects that the camera is on or off, it can simultaneously transmit the camera status and application package name to the hardware synthesis processor, enabling the hardware synthesis processor to set the display bandwidth appropriately.
[0018] In one possible implementation, before the terminal device receives the first operation to open the first application, the method further includes: the terminal device receiving a seventh operation to power on or restart; at the seventh moment, in response to the seventh operation, the terminal device displays a fifth interface, which is the desktop of the terminal device; the seventh moment is before the first moment, and between the seventh moment and the first moment, the display bandwidth of the terminal device is a first value.
[0019] In this way, the terminal device can set the display bandwidth to the first value when it detects that the device is powered on or restarted, ensuring smooth display.
[0020] Secondly, embodiments of this application provide a display method, the method comprising: at an eighth moment, a terminal device receiving a first operation to open a first application; in response to the first operation, the terminal device displaying a first interface;
[0021] The terminal device receives a second operation to enable the camera in the first application; at the ninth moment, in response to the second operation, the terminal device sets its display bandwidth to a second value. Between the eighth and ninth moments, the display bandwidth of the terminal device is a first value, which is greater than the second value; at the tenth moment, when the terminal device acquires N layers and determines the existence of a preset layer based on the layer names of the N layers, the terminal device sets its display bandwidth to the first value, where N is an integer greater than or equal to 1. The terminal device displays a second interface based on the first value of the display bandwidth. The second interface is the interface displayed by the first application when the camera is enabled. Between the ninth and tenth moments, the display bandwidth of the terminal device is the second value; the terminal device receives a third operation to disable the camera in the first application; at the eleventh moment, in response to the third operation, the terminal device displays the first interface. Between the tenth and eleventh moments, the display bandwidth of the terminal device is always the first value.
[0022] In this way, the terminal device can set the display bandwidth to the first value based on the layer name received and the existence of the preset layer, thereby reducing screen stuttering caused by insufficient display bandwidth and improving the smoothness of the display.
[0023] In one possible implementation, the terminal device is configured with: a first application, a camera hardware abstraction layer (HAL), a display compositor, and a hardware compositing processor. When the terminal device acquires N layers and determines, based on the layer names of the N layers, that a preset layer exists, the terminal device sets its display bandwidth to a first value. This includes: the first application sending the N layers to the hardware compositing processor via the camera HAL and the display compositor, with each of the N layers carrying the layer name corresponding to any one of the N layers; and when the hardware compositing processor determines, based on the layer names of the N layers, that a preset layer exists, the hardware compositing processor sets the terminal device's display bandwidth to the first value.
[0024] In this way, the hardware compositing processor can improve the smoothness of the image by filtering the layers and providing higher display bandwidth to preset layers with higher display bandwidth requirements.
[0025] In one possible implementation, the terminal device is further configured with a DPU and an LCD. After the hardware synthesis processor sets the display bandwidth of the terminal device to a first value, the method further includes: the hardware synthesis processor sending the first value to the DPU; the hardware synthesizer sending N layers to the DPU; when the DPU determines that the sum of the display bandwidths of the N layers is less than the first value, the DPU synthesizes the N layers into a target image and sends the target image to the LCD.
[0026] In one possible implementation, before the terminal device receives the first operation of opening the first application, the method further includes: the terminal device receiving a seventh operation of powering on or restarting; at the seventh moment, in response to the seventh operation, the terminal device displays a fifth interface, which is the desktop of the terminal device; the seventh moment is before the eighth moment, and between the seventh moment and the eighth moment, the display bandwidth of the terminal device is always the first value.
[0027] Thirdly, embodiments of this application provide a display device, which includes a display unit and a processing unit. The display unit is used to process the display of data in the display device, and the processing unit is used to process the data processing in the display device.
[0028] In one possible implementation, the display device may further include a storage unit, which may include one or more memories, which may be devices in one or more devices or circuits used to store programs or data.
[0029] Fourthly, embodiments of this application provide a terminal device, including a processor and a memory, wherein the memory is used to store code instructions; the processor is used to run the code instructions, causing the terminal device to perform the method described in the first aspect or any implementation thereof, or to perform the method described in the second aspect or any implementation thereof.
[0030] Fifthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed, cause a computer to perform the method described in the first aspect or any implementation thereof, or to perform the method described in the second aspect or any implementation thereof.
[0031] A sixth aspect is a computer program product comprising a computer program that, when run, causes a computer to perform the method described in the first aspect or any implementation thereof, or to perform the method described in the second aspect or any implementation thereof.
[0032] It should be understood that the third to sixth aspects of this application correspond to the technical solutions of the first to second aspects of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description
[0033] Figure 1 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application;
[0034] Figure 2 This is a schematic diagram of the software structure of a terminal device provided in an embodiment of this application;
[0035] Figure 3 A scenario diagram provided for an embodiment of this application;
[0036] Figure 4 This is another scenario illustration provided for an embodiment of this application;
[0037] Figure 5 This is another scenario illustration provided by an embodiment of the present application;
[0038] Figure 6 A schematic diagram illustrating a process for setting display bandwidth, provided in an embodiment of this application;
[0039] Figure 7 A schematic flowchart illustrating a display method provided in an embodiment of this application;
[0040] Figure 8 A flowchart illustrating another display method provided in an embodiment of this application;
[0041] Figure 9 A flowchart illustrating yet another display method provided in an embodiment of this application;
[0042] Figure 10 A schematic diagram of an interface for activating a camera in a first application, provided as an embodiment of this application;
[0043] Figure 11 This application provides a schematic diagram of an interface for activating a camera in a second application.
[0044] Figure 12 This is a schematic diagram of the structure of a display device provided in an embodiment of this application;
[0045] Figure 13This is a schematic diagram of the hardware structure of another terminal device provided in an embodiment of this application. Detailed Implementation
[0046] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. For example, the first value and the second value are only used to distinguish different values and do not limit their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.
[0047] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.
[0048] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.
[0049] The terminal devices described in the embodiments of this application can also be referred to as terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Terminal devices can be mobile phones with touchscreens, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on. The embodiments of this application do not limit the specific technologies or device forms used in the terminal devices.
[0050] Therefore, in order to better understand the embodiments of this application, the structure of the terminal device of the embodiments of this application will be described below. For example, Figure 1 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application.
[0051] The terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, an indicator 192, a camera 193, and a display screen 194, etc.
[0052] The sensor module 180 may include one or more of the following: pressure sensor, gyroscope sensor, barometric pressure sensor, magnetic sensor, accelerometer, distance sensor, proximity sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor, or bone conduction sensor, etc., which are not specifically limited in this embodiment.
[0053] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the terminal device. In other embodiments of this application, the terminal device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
[0054] The processor 110 may include one or more processing units. These processing units may be independent devices or integrated within one or more processors. The processor 110 may also include memory for storing instructions and data.
[0055] USB port 130 is a USB standard compliant interface, which can be a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge terminal devices, and can also be used for data transfer between terminal devices and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.
[0056] The charging management module 140 receives charging input from the charger. The charger can be a wireless charger or a wired charger. The power management module 141 connects the charging management module 140 to the processor 110.
[0057] The wireless communication function of the terminal device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.
[0058] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Antennas in terminal equipment can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization.
[0059] The mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G on terminal devices. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation.
[0060] The wireless communication module 160 can provide solutions for wireless communication applications on terminal devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), etc.
[0061] The terminal device implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering.
[0062] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the terminal device may include one or N display screens 194, where N is a positive integer greater than 1.
[0063] Terminal devices can achieve shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.
[0064] Camera 193 is used to capture still images or videos. In some embodiments, the terminal device may include one or N cameras 193, where N is a positive integer greater than 1.
[0065] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.
[0066] Internal memory 121 can be used to store executable program code, including instructions. Internal memory 121 may include a program storage area and a data storage area.
[0067] The terminal device can implement audio functions such as music playback and recording through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, and an application processor.
[0068] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The speaker 170A, also called a "loudspeaker," is used to convert audio electrical signals into sound signals. The terminal device can listen to music or hands-free calls through the speaker 170A. The receiver 170B, also called a "handpiece," is used to convert audio electrical signals into sound signals. When the terminal device answers a phone call or voice message, it can listen to the voice by bringing the receiver 170B close to the ear. The headphone jack 170D is used to connect wired headphones. The microphone 170C, also called a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. In this embodiment, the terminal device may have one microphone 170C.
[0069] A pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor may be located on the display screen 194. A gyroscope sensor can be used to determine the motion posture of the terminal device. A barometric pressure sensor is used to measure air pressure. A magnetic sensor includes a Hall effect sensor. An accelerometer can detect the magnitude of acceleration of the terminal device in various directions (typically three axes). A distance sensor is used to measure distance. A proximity sensor may include, for example, a light-emitting diode (LED) and a photodetector, such as a photodiode. An ambient light sensor is used to sense ambient light intensity. A fingerprint sensor is used to collect fingerprints. A temperature sensor is used to detect temperature. A touch sensor, also known as a "touch device," is used. A bone conduction sensor can acquire vibration signals.
[0070] A touch sensor can be set on the display screen 194. The touch sensor and the display screen 194 together form a touch screen, or "touch screen". A capacitive sensing node grid (hereinafter referred to as capacitive sensor) can be set inside the touch screen.
[0071] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. The terminal device can receive button input and generate key signal inputs related to user settings and function control of the terminal device. Indicator 192 can be an indicator light, used to indicate charging status, power level changes, messages, missed calls, notifications, etc.
[0072] The software system of terminal devices can adopt layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture, etc., which will not be elaborated here.
[0073] For example, Figure 2 This is a schematic diagram of the software structure of a terminal device provided in an embodiment of this application. For example... Figure 2As shown, the layered architecture divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into multiple layers, from top to bottom: the application (APP) layer, the application framework layer, the hardware abstraction layer (HAL), and the kernel layer, etc. This application does not impose any limitations on these layers.
[0074] The application layer may include a series of application packages. The application layer may include one or more of the following: camera, desktop, map, Bluetooth, or preset applications, etc., which are not limited in this embodiment.
[0075] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes several predefined interfaces. It may include one or more of the following: a render engine, a graphical user interface (GUI), or a surface flinger (SF).
[0076] The main function of the display compositor is to receive graphics data buffers from multiple sources, composite them into layers, and then send them to the display device. Simultaneously, the display compositor generates vertical synchronization (Vsync) signals, such as SF-VSync and APP-VSync signals. The compositor flow control, signal simulator, and connection manager work together to notify the application and the display compositor itself of the VSync signals. Specifically, Vsync-APP triggers the application's rendering process for image frames. Vsync-SF triggers the surface flinger's layer compositing process for image frames. The hardware Vsync signal (Vsync-HW) triggers the screen display refresh process.
[0077] The rendering engine can utilize the GPU to perform functions such as rendering and compositing of layers.
[0078] In possible implementations, the application framework layer may also include one or more of the following: window manager, content provider, resource manager, view system, or notification manager, etc.
[0079] The window manager is used to manage window applications. For example, the window manager can be used to request foreground windows, obtain screen size, determine if there is a status bar, lock the screen, touch the screen, drag the screen, capture the screen, etc.
[0080] The display compositor is used to switch frame rates and composite layers.
[0081] Content providers store and retrieve data, making that data accessible to applications. Data can include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc. View systems include visual controls, such as controls for displaying text and controls for displaying images. View systems are used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon can include views for displaying text and views for displaying images. Resource managers provide applications with various resources, such as localized strings, icons, images, layout files, video files, etc.
[0082] The purpose of the Hardware Abstraction Layer (HAL) is to abstract hardware, providing a unified interface for upper-layer applications to query hardware devices, or to provide data storage services for upper-layer applications. The HAL can include one or more of the following modules: a hardware editor (hardware composer, HWC), or a camera HAL module.
[0083] The hardware editor primarily provides hardware support for the display compositor, as well as support for layer compositing and display.
[0084] The Camera HAL connects the underlying camera hardware to the framework APIs located in android.hardware. The Camera subsystem mainly contains various implementations of camera pipeline components, and the Camera HAL provides the interface for using these components.
[0085] The kernel layer is the layer between hardware and software. It drives the hardware to function. The kernel layer can include one or more of the following: a data processing unit (DPU), display driver, camera driver, or sensor driver, etc.
[0086] The DPU can perform composite processing on layers and send the composited image to the LCD display.
[0087] The hardware layer may include one or more of the following: liquid crystal display (LCD), graphics processing unit (GPU), or central processing unit (CPU), etc.
[0088] GPUs can perform layer rendering and layer compositing.
[0089] In this application embodiment, no specific limitations are made on the software layers involved in the software architecture, the modules contained in the layers, and the functions of the modules.
[0090] The display method provided in this application embodiment can be applied to... Figures 3-5 For any of the scenarios described in the corresponding embodiments, such as the terminal device displaying a preview interface in a short video application in full screen and displaying any video in a video application in a floating window (see...). Figure 3 (Corresponding embodiments), the terminal device displays any interface in the video application in split-screen mode, as well as the interface when sharing the content displayed by the camera to other devices (see...). Figure 4 (Corresponding embodiments) The terminal device displays the desktop in full screen and the video call interface of the social application in a floating window (see... Figure 5 (Corresponding implementation examples).
[0091] For example, Figure 3 This is a schematic diagram of a scenario provided for an embodiment of this application. Figure 3 In the corresponding embodiments, a foldable screen phone is used as an example for illustration. This example does not constitute a limitation on the embodiments of this application.
[0092] When a user turns on the camera in a short video app or opens a video in a video app, the terminal device can... Figure 3 The interface shown in 'a' is the same as... Figure 3 Switching back and forth between the interfaces shown by b in the example. Figure 3The interface shown as 'a' in the diagram can be a short video shooting interface 301. Interface 301 can include content displayed after the camera is turned on in the short video application. Interface 301 can display one or more of the following: a preview image captured when the camera is turned on (e.g., showing a girl), a control at the top of the interface for selecting music, a shooting control at the bottom of the interface, a control for selecting shooting effects, a control for opening the photo album, and functional controls within the dashed frame 303. The functional controls within the dashed frame 303 can include one or more of the following: a control for using the segmented shooting function (displayed as "Segmented Shooting"), a control for using the quick shooting function (displayed as "Quick Shooting"), a control for opening a shooting template (displayed as "Template"), or a control for starting live video streaming (displayed as "Live Stream").
[0093] The upper layer of the short video application may display a floating window 302, which may display a video content from the video application, or other content from the video application or other applications. This application embodiment does not limit this.
[0094] In the terminal device by Figure 3 The interface shown in 'a' will switch to... Figure 3 During the process shown in b of the interface, the color of the functional controls in the dashed box 303 gradually darkens as the terminal device changes from... Figure 3 The interface shown in b in the image switches to... Figure 3 During the process shown in the interface (a), the color of the functional controls in the dashed box 303 gradually lightens, which can be understood as the terminal device... Figure 3 Screen flickering or lag may occur in the corresponding scenarios.
[0095] For example, Figure 4 This is a schematic diagram of another scenario provided for an embodiment of this application.
[0096] When a user opens a video application or shares the camera on device 400 with other devices, the terminal device can... Figure 4 The interface shown in 'a' is the same as... Figure 4 Switching back and forth between the interfaces shown by b in the example. Figure 4The interface shown in Figure 'a' can display two windows: window 401 on the left half of the terminal device's screen and window 402 on the right half. Window 401 can be any interface in a video application, while window 402 is the interface displayed when the terminal device turns on its camera and shares it with other devices. Window 402 can display a prompt message, a pause control, and a control for disconnecting. The prompt message can be displayed as "Rear camera in use, connected to 'Device 200'", which can be understood as the terminal device sharing its camera with Device 400. In this scenario, Device 400 can display the content captured by the terminal device's camera on its own screen. The background of window 402 can be a portion of the wallpaper on the terminal device, and this portion of the wallpaper can be semi-transparent.
[0097] like Figure 4 As shown in b, the content displayed in window 403 can be the same as that displayed in window 401. However, the background of window 404 is not part of the wallpaper on the terminal device, unlike that of window 402.
[0098] In the terminal device by Figure 4 The interface shown in 'a' will switch to... Figure 4 The interface shown in b, or by Figure 4 The interface shown in b in the image switches to... Figure 4 During the process shown in the image (a), the background of the right half of the terminal device's screen continuously changes, which can be understood as the terminal device... Figure 4 Screen flickering or lag may occur in the corresponding scenarios.
[0099] In a possible implementation, when the left half of the terminal device's screen displays other application interfaces and the right half displays any interface displayed after accessing the camera, the terminal device will also generate a similar... Figure 4 Similar screen-scratching situations exist in corresponding embodiments, but the specific content displayed on the terminal device is not limited in this application embodiment.
[0100] For example, Figure 5 This is a schematic diagram of another scenario provided for an embodiment of this application.
[0101] When a user activates the video call function in a social application, the terminal device can... Figure 5 The interface shown in 'a' is the same as... Figure 5 Switching back and forth between the interfaces shown by b in the example. Figure 5The interface shown in 'a' can display the desktop in full screen. The desktop can display at least one application icon, such as the icon for a file management application (as shown within the dashed box 501). A line segment can be displayed below the file management application icon. This line segment indicates that the file management application can be displayed as a card on the desktop. For example, in response to a user's long-press operation on the file management application icon, the terminal device can set a card for the file management application.
[0102] A floating window 502 can be displayed on the top layer of the desktop interface. The floating window 502 can display the video call interface of the social application. For example, the video call interface can include: the video content obtained by the terminal device based on the camera (such as the video content displayed as a girl), and the video content obtained by the camera of the other device when the terminal device establishes a call connection with the other device (such as the video content displayed as a boy).
[0103] Understandably, in response to a user's trigger action on the desktop interface, the terminal device can flash the line segment within the dashed box 501. For example, when the terminal device... Figure 5 The interface shown in 'a' will switch to... Figure 5 During the process shown in b of the interface, the line segment below the file management application icon in the dashed box 501 is removed, and the terminal device is switched from... Figure 5 The interface shown in b in the image switches to... Figure 5 During the process shown in the interface (as shown in 'a'), the line segment below the file management application icon can be interpreted as the terminal device... Figure 5 In the corresponding scenarios, screen flickering or lag may occur, and the terminal device may overheat.
[0104] In possible implementations, when the terminal device displays other interfaces in full screen and the floating window displays the interface of other applications after the camera is turned on, there may also be screen flickering or lag in the other interfaces. In this application embodiment, the content displayed in full screen by the terminal device and the content displayed in the floating window are not limited.
[0105] Combination Figures 3-5In a corresponding embodiment, in response to a user triggering the activation of the camera on the terminal device, the terminal device may experience screen flickering. The reason for this flickering could be that when the terminal device detects the user activating the camera, it may set the display bandwidth to a secondary display bandwidth (e.g., 4700Mbps). When the required display bandwidth for a particular scenario is high, some layers may be composited using the GPU while others are composited using the DPU. If an image is composited using both the GPU and DPU over a period of time, these different compositing methods can cause screen flickering or stuttering.
[0106] Display bandwidth can be understood as the amount of display data transmitted per unit time. Display bandwidth can be related to parameters such as the horizontal resolution, vertical resolution, and maximum refresh rate of the displayed image. For example, display bandwidth can be the product of the horizontal resolution, vertical resolution, and maximum refresh rate. The specific calculation method for display bandwidth is not limited in this embodiment.
[0107] Under normal circumstances, combined Figure 6 The corresponding embodiments illustrate the display bandwidth settings and the reasons for screen flickering on terminal devices. Figure 6 This is a schematic diagram illustrating a process for setting display bandwidth, provided in an embodiment of this application. Figure 6 In a corresponding embodiment, the terminal device may include modules such as a target application, a camera HAL, a rendering engine, a display compositor, a hardware compositing processor, a DPU, and an LCD.
[0108] like Figure 6 As shown, the method for setting the display bandwidth is as follows:
[0109] S601. In response to a user's power-on or restart operation, the camera HAL initializes the camera service.
[0110] S602, in response to user power-on or restart operations, the hardware synthesis processor is initialized.
[0111] It is understood that, in response to the user's power-on or restart operation, the camera HAL can execute S701, and the hardware synthesis processor can sequentially execute the steps shown in S702-S703. In this embodiment of the application, the order relationship between S701 and S702 is not specifically limited.
[0112] The S603 hardware synthesis processor is configured with a first display bandwidth and a second display bandwidth.
[0113] The first display bandwidth can be the display bandwidth set when the camera is off, and the second display bandwidth can be the display bandwidth set when the camera is on. The first display bandwidth is greater than the second display bandwidth. For example, the first display bandwidth can be set to a value such as 8800Mbps, and the second display bandwidth can be set to a value such as 4700Mbps.
[0114] It is understandable that the total bandwidth of a terminal device can include: bandwidth used during calls, bandwidth used for taking photos, display bandwidth, and other bandwidth, with the latter accounting for a relatively small proportion of the total bandwidth. To ensure the smooth operation of the photo-taking function, the terminal device can set a smaller display bandwidth when the camera is detected as being on, allowing more bandwidth to be allocated to the photo-taking function.
[0115] For example, the terminal device may have an LCD configuration file, which may be set in the kernel layer. The LCD configuration file may include a first display bandwidth and a second display bandwidth. The hardware synthesis processing and the camera HAL can determine the first display bandwidth that can be set when the camera is off and the second display bandwidth that can be set when the camera is on by parsing the LCD configuration file.
[0116] S604, the camera HAL sends a message to the hardware synthesis processor to indicate that the camera is off.
[0117] It is understandable that the camera HAL can execute the steps shown in S604 after S601. The camera HAL can execute the steps shown in S604 after confirming that the camera is off, so that the hardware synthesis processor can set the default value for the display bandwidth.
[0118] S605, the hardware synthesis processor sets the display bandwidth to the first display bandwidth.
[0119] Adaptably, the hardware synthesis processor can synchronize the primary display bandwidth to the DPU.
[0120] S606. In response to the operation of turning on the camera, the target application sends a message to the camera HAL indicating that the camera is turned on.
[0121] The operation for turning on the camera can be: taking a photo, starting video recording, scanning a QR code, starting a video call, initiating face verification, or sharing the camera with other devices. This operation for turning on the camera can be one that is imperceptible to the user but requires the target application to turn on the camera for image acquisition.
[0122] S607, the camera HAL sends a message to the hardware synthesis processor to indicate that the camera is on.
[0123] S608, the hardware synthesis processor sets the display bandwidth to the second display bandwidth.
[0124] Accordingly, when the hardware synthesis processor receives a message indicating that the camera is on, it can utilize the display bandwidth in S603 to configure a second display bandwidth and synchronize the second display bandwidth to the DPU.
[0125] S609. The target application sends N layers to the display compositor.
[0126] These N layers can be the layers requested by the target application after the camera in the target application is turned on.
[0127] N layers can carry layer information corresponding to each of the N layers. The layer information for any given layer can include one or more of the following: the layer's horizontal resolution, vertical resolution, maximum refresh rate, or layer name. The horizontal resolution, vertical resolution, and maximum refresh rate of the layer can be used to calculate the display bandwidth of the current layer.
[0128] S610, the display compositor sends N layers to the hardware compositing processor.
[0129] exist Figure 6 In the corresponding embodiment, N is an integer greater than 1.
[0130] S611, the hardware synthesis processor sends N layers to the DPU.
[0131] Based on the display bandwidth of N layers, the S612 and DPU determine which M layers out of the N layers will be processed by the GPU and set labels for the M layers.
[0132] The DPU can sort N layers according to their stacking relationship, for example, by ordering the layers from the topmost to the bottommost. It then calculates the display bandwidth of each of the N layers using their layer information. Using the second display bandwidth and the sorted N layers, the DPU determines that the sum of the display bandwidths of the NM layers is less than the second display bandwidth. The remaining M layers require layer compositing processing by the GPU, and therefore the DPU can assign labels to these M layers.
[0133] The S613 DPU sends M layers to the hardware synthesis processor.
[0134] S614, The hardware compositing processor sends M layers to the display compositing unit.
[0135] S615, Display compositor sends M layers to the rendering engine.
[0136] S616, the rendering engine uses the GPU to render M layers, creates the first layer, and fills the first layer with the data from the rendered M layers.
[0137] For example, when the rendering engine receives M layers, it can call a method based on the label to make OpenGL ES process the M layers. OpenGL ES then calls a method to make the GPU process the M layers. The GPU can then render the M layers, create the first layer, and fill the first layer with the data of the rendered M layers. The GPU can then return the first layer to the rendering engine via OpenGL ES.
[0138] The first layer can also carry layer information. The types of information carried in the layer information of the first layer are shown in step S609 and will not be repeated here.
[0139] S617, The rendering engine sends the first layer to the reality compositor.
[0140] S618, The display synthesizer sends the first layer to the hardware synthesis processor.
[0141] S619, the hardware synthesis processor sends the first layer to the DPU.
[0142] The S620 and DPU determine that the display bandwidth of N-M+1 layers meets the second display bandwidth requirement.
[0143] The DPU can obtain the first layer and NM layers, totaling N-M+1 layers. Based on the layer information corresponding to each of the N-M+1 layers, it reconfirms whether the sum of the display bandwidths of the N-M+1 layers meets the second display bandwidth. If it is determined that the sum of the display bandwidths of the N-M+1 layers is less than the second display bandwidth, the step shown in S621 is executed.
[0144] S621 and DPU perform layer compositing on N-M+1 layers to obtain the first image, and then send the first image to the LCD.
[0145] The appropriate LCD can display the first image.
[0146] S622, In response to the user's action of turning off the camera, the target application sends a message to the camera HAL to indicate that the camera is off.
[0147] S623, the camera HAL sends a message to the hardware synthesis processor to indicate that the camera is off.
[0148] S624, the hardware synthesis processor sets the display bandwidth to the first display bandwidth.
[0149] Accordingly, when the hardware synthesis processor receives a message indicating that the camera is off, it can utilize the display bandwidth in S603 to configure a first display bandwidth and synchronize the first display bandwidth to the DPU.
[0150] Combination Figure 6 In some implementations, some layers are composited using the GPU while others are composited using the DPU. Therefore, when an image is composited using both the GPU and the DPU within a certain period of time, the different composite methods can cause screen flickering or lag.
[0151] For example, combining Figure 3 In a corresponding embodiment, the layer containing the dashed box 303 can be a layer composited using the GPU; combined with Figure 4 In a corresponding embodiment, the layer containing window 402 or window 404 can be a layer composited by the GPU; combined with Figure 5 In a corresponding embodiment, the layer containing the line segments in the dashed box 501 can be a layer composited by the GPU.
[0152] In view of this, this application provides a display method in which the display bandwidth can be set to a first display bandwidth when the terminal device detects that the device is powered on. When the terminal device detects that a preset application has turned on the camera, the terminal device can maintain the first display bandwidth to reduce the screen flickering or stuttering caused by reducing the first display bandwidth to a second display bandwidth when the application turns on the camera.
[0153] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be implemented independently or in combination with each other. The same or similar concepts or processes may not be described again in some embodiments.
[0154] This application provides two methods for setting display bandwidth, such as determining display bandwidth by filtering by application package name (see...). Figure 7 (Corresponding embodiments), or determine the display bandwidth by filtering by layer name (see...) Figure 8 (Corresponding implementation examples).
[0155] For example, Figure 7 This is a schematic flowchart illustrating a display method provided in an embodiment of this application. Figure 7In a corresponding embodiment, the terminal device may include modules such as a target application, a camera HAL, a display synthesizer, a hardware synthesis processor, a DPU, and an LCD.
[0156] like Figure 7 As shown, the display method may include the following steps:
[0157] S701, in response to user power-on or restart operations, the camera HAL initializes camera services.
[0158] S702, in response to user power-on or restart operations, hardware synthesis processor initialization.
[0159] The S703 hardware synthesis processor is configured with a first display bandwidth and a second display bandwidth.
[0160] S704, the camera HAL sends a message to the hardware synthesis processor to indicate that the camera is off.
[0161] S705, the hardware synthesis processor sets the display bandwidth to the first display bandwidth.
[0162] The steps shown in S701-S705 can be found in the descriptions of the steps shown in S601-S605, and will not be repeated here.
[0163] S706, In response to an operation to turn on the camera, the target application sends a first message to the camera HAL.
[0164] The first message may include: a message indicating that the camera is on, and the package name of the target application.
[0165] S707, the camera HAL sends the first message to the hardware synthesis processor.
[0166] S708, the hardware synthesis processor determines whether the target application meets the preset application requirements.
[0167] Preset applications can be applications that consume a lot of display bandwidth during runtime, and / or applications that frequently experience screen flickering or stuttering. For example, a terminal device can set applications that consume more display bandwidth than a second display bandwidth during runtime as preset applications, and / or, a terminal device can set applications that experience more than a preset number of screen flickering (or stuttering) occurrences within a certain time threshold as preset applications.
[0168] When the camera is on, if the hardware synthesis processor determines that the target application meets the preset application criteria (or the package name of the target application belongs to the preset application's package name), the hardware synthesis processor executes the step shown in S709; or, if the camera is on, if the hardware synthesis processor determines that the target application does not meet the preset application criteria (or the package name of the target application does not belong to the preset application's package name), the hardware synthesis processor executes the step shown in S710. The package name of the preset application can be a preset setting in the terminal device.
[0169] S709, the hardware synthesis processor sets the display bandwidth to the first display bandwidth.
[0170] Adaptably, the hardware synthesis processor can synchronize the primary display bandwidth to the DPU.
[0171] S710, the hardware synthesis processor sets the display bandwidth to the second display bandwidth.
[0172] Adaptably, the hardware synthesis processor can synchronize the second display bandwidth to the DPU.
[0173] Understandably, when the hardware synthesizer in S709 sets the display bandwidth to the first display bandwidth, the terminal device can execute the steps shown in S711-S715 when N layers are detected.
[0174] S711, The target application sends N layers to the display compositor.
[0175] exist Figure 7 In the corresponding embodiment, N is an integer greater than 1.
[0176] S712, the display compositor sends N layers to the hardware compositing processor.
[0177] S713, the hardware synthesis processor sends N layers to the DPU.
[0178] S714, DPU determines that the display bandwidth of N layers meets the first display bandwidth requirement.
[0179] The DPU can obtain the layer information corresponding to N layers respectively. When it is determined that the sum of the display bandwidths of the N layers is less than the first display bandwidth, the step shown in S715 is executed.
[0180] The S715 and DPU perform layer compositing on N layers to obtain a second image, and then send the second image to the LCD.
[0181] The adapted LCD can display a second image.
[0182] S716, In response to the user's action of turning off the camera, the target application sends a second message to the camera HAL.
[0183] The second message may include: a message indicating that the camera is off, and the package name of the target application; alternatively, the second message may only include a message indicating that the camera is off.
[0184] S717, the camera HAL sends a second message to the hardware synthesis processor.
[0185] S718, the hardware synthesis processor sets the display bandwidth to the first display bandwidth.
[0186] Adaptably, the hardware synthesis processor can synchronize the primary display bandwidth to the DPU.
[0187] Based on this, the hardware synthesis processor can set the display bandwidth to the primary display bandwidth when it detects that a preset application has activated its camera. This reduces screen flickering or stuttering caused by insufficient display bandwidth, improving the smoothness of the displayed image. Furthermore, by setting the primary display bandwidth, the terminal device can also reduce GPU usage when display bandwidth is insufficient, thus saving GPU power consumption.
[0188] Understandably, in combination Figures 3-5 In any scenario described in the corresponding embodiments, the terminal device can maintain a high display bandwidth when it detects that the camera of the preset application is turned on. In this way, the terminal device can not only reduce screen stuttering or flickering in the preset application, but also reduce screen stuttering or flickering in other applications when the terminal device is turned on at the same time, thus improving the smoothness of device use.
[0189] For example, Figure 8 This is a flowchart illustrating another display method provided in an embodiment of this application. Figure 8 In a corresponding embodiment, the terminal device may include modules such as a target application, a camera HAL, a display synthesizer, a hardware synthesis processor, a DPU, and an LCD.
[0190] like Figure 8 As shown, the display method may include the following steps:
[0191] S801, in response to user power-on or restart operations, the camera HAL initializes camera services.
[0192] S802, in response to user power-on or restart operations, initializes the hardware synthesis processor.
[0193] The S803 hardware synthesis processor is configured with a first display bandwidth and a second display bandwidth.
[0194] S804, the camera HAL sends a message to the hardware synthesis processor to indicate that the camera is off.
[0195] S805, the hardware synthesis processor sets the display bandwidth to the first display bandwidth.
[0196] S806. In response to the operation of turning on the camera, the target application sends a message to the camera HAL indicating that the camera is on.
[0197] S807, the camera HAL sends a message to the hardware synthesis processor to indicate that the camera is on.
[0198] S808, the hardware synthesis processor sets the display bandwidth to the second display bandwidth.
[0199] S809, The target application sends N layers to the display compositor.
[0200] S810, the display compositor sends N layers to the hardware compositing processor.
[0201] The steps shown in S801-S810 can be found in [reference]. Figure 6 The steps shown in S601-S610 in the corresponding embodiments will not be repeated here.
[0202] S811: When the hardware synthesis processor determines that there is a layer among N layers whose layer name satisfies the preset layer name, it sets the display bandwidth to the first display bandwidth.
[0203] exist Figure 8 In the corresponding embodiment, N is an integer greater than or equal to 1.
[0204] The default layer name can be a layer whose display bandwidth is greater than the third display bandwidth when the application it belongs to is running, and / or a layer that frequently experiences screen flickering or lag. The third display bandwidth is less than the second display bandwidth; for example, the third display unit can be a value such as 2500Mbps or 3000Mbps.
[0205] Adaptably, the hardware synthesizer can synchronize the first display bandwidth to the DPU after performing the steps shown in S811.
[0206] In a possible implementation, when there is no layer among the N layers whose name matches the preset layer name, the terminal device can maintain the second display bandwidth. S812, the hardware synthesis processor sends N layers to the DPU.
[0207] S813 and DPU determine that the display bandwidth of N layers meets the first display bandwidth requirement.
[0208] The DPU can obtain the layer information corresponding to each of the N layers. When it is determined that the sum of the display bandwidths of the N layers is less than the first display bandwidth, the step shown in S814 is executed.
[0209] The S814 and DPU perform layer compositing on N layers to obtain a third image, and then send the third image to the LCD.
[0210] The adapted LCD can display a third image.
[0211] S815, In response to the user's action of turning off the camera, the target application sends a message to the camera HAL to indicate that the camera is off.
[0212] S816, the camera HAL sends a message to the hardware synthesis processor to indicate that the camera is off.
[0213] S817, the hardware synthesis processor sets the display bandwidth to the first display bandwidth.
[0214] Adaptably, the hardware synthesis processor can synchronize the primary display bandwidth to the DPU.
[0215] Based on this, when the hardware synthesis processor receives a layer that meets the preset layer name (or can be understood as the existence of a preset layer), it can set the display bandwidth to the first display bandwidth to reduce screen stuttering caused by insufficient display bandwidth and improve the smoothness of the display.
[0216] In possible implementations, the terminal device can also simultaneously adopt... Figure 7 as well as Figure 8 The method described in the corresponding embodiment determines the display bandwidth using both the application package name and the layer name. This can be understood as... Figure 8 The preset layers described in the text can be Figure 7 The preset layers in the application can also be used as a way to determine whether an application is a preset application.
[0217] It is understood that the interface provided in the embodiments of this application is only an example and does not constitute a limitation on the embodiments of this application.
[0218] exist Figure 7 Based on the corresponding embodiments, taking the first application and the second application as examples, the display method provided in this application is illustrated. See [link to relevant documentation]. Figures 9-11 Corresponding implementation examples.
[0219] For example, Figure 9 This is a flowchart illustrating yet another display method provided in an embodiment of this application. Figure 9In the corresponding embodiments, the first application can be a short video application and the second application can be a shopping application as an example for illustration. This example does not constitute a limitation on the embodiments of this application.
[0220] like Figure 9 As shown, the display method may include the following steps:
[0221] S901, At the first moment, the terminal device receives the first operation of opening the first application.
[0222] The first application is a short video application, and the first application satisfies the preset application described in the embodiments of this application.
[0223] The first operation to open the first application can be to open the first application when it is closed, or it can be to open the first application from the background multitasking interface, or it can be to open the first application by swiping left or right at the bottom of the interface. This application embodiment does not limit this.
[0224] S902, In response to the first operation, the terminal device displays the first interface.
[0225] Figure 10 This application provides a schematic diagram of an interface for activating a camera in a first application. The first interface can be... Figure 10 The interface shown in 'a'.
[0226] S903, The terminal device receives a second operation for turning on the camera in the first application.
[0227] like Figure 10 The interface shown in Figure 'a' may include: short video content, a control 1001 for shooting short videos, etc. The second operation for turning the camera on and off can be an operation on control 1001.
[0228] S904. In response to the second operation, the terminal device displays the second interface.
[0229] The second interface is the interface displayed by the first application when the camera is turned on. For example, the second interface could be... Figure 10 The interface shown in b can include: a control 1002 for turning off short video recording based on the content captured by the camera.
[0230] S905, The terminal device receives a third operation for turning off the camera in the first application.
[0231] The third operation can be an operation targeting control 1002, or an operation targeting... Figure 10 The gestures shown in b indicate the actions of swiping right or left on the interface.
[0232] S906, at the second moment, in response to the third operation, the terminal device displays the first interface.
[0233] Between the first moment and the second moment, the display bandwidth of the terminal device is always a first value, which can be the first display bandwidth described in the embodiments of this application.
[0234] S907, at the third moment, the terminal device receives the fourth operation of opening the second application.
[0235] The second application is a shopping application, which does not meet the preset application described in the embodiments of this application.
[0236] S908, in response to the fourth operation, the terminal device displays the third interface.
[0237] Figure 11 This is a schematic diagram of an interface for activating a camera in a second application, provided as an embodiment of this application. The third interface can be... Figure 11 The interface shown in 'a' may include: a control 1101 for scanning QR codes, a control 1102 for taking photos, etc.
[0238] S909, Fourth moment, the terminal device receives a fifth operation for turning on the camera in the second application.
[0239] The fifth operation can be a click operation on control 1101 or a click operation on control 1102.
[0240] Between the third and fourth time points, the display bandwidth of the terminal device is always the first value.
[0241] S910, Fifth Moment, in response to the Fifth Operation, the terminal device displays the Fourth Interface.
[0242] The fourth interface is the interface displayed by the second application when the camera is turned on. Between the fourth and fifth moments, the display bandwidth of the terminal device is always the first value. For example, the fourth interface can be... Figure 11 The interface shown in b can display a control 1103 for exiting the current interface.
[0243] S911, The terminal device receives a sixth operation for turning off the camera in the second application.
[0244] The sixth operation can be a click operation on control 1103, or an operation on... Figure 11 The left or right swipe operation is shown in b in the interface.
[0245] S912, at the sixth moment, in response to the sixth operation, the terminal device displays the third interface.
[0246] Between the fifth and sixth moments, the display bandwidth of the terminal device is the second value, and the first value is greater than the second value.
[0247] It is understandable that when the terminal device is using the first application, it can set the display bandwidth to a first value, and when it is using the second application, it can set the display bandwidth to a second value when it detects that the camera is turned on.
[0248] In a possible implementation, at the eighth moment, the terminal device receives a first operation to open the first application; in response to the first operation, the terminal device displays a first interface; the terminal device receives a second operation to turn on the camera in the first application; at the ninth moment, in response to the second operation, the terminal device sets its display bandwidth to a second value, and between the eighth and ninth moments, the terminal device's display bandwidth is a first value, which is greater than the second value; at the tenth moment, when the terminal device obtains N layers and determines the existence of a preset layer based on the layer names of the N layers, the terminal device sets its display bandwidth to the first value, where N is an integer greater than or equal to 1, and the terminal device displays a second interface based on the first value of the display bandwidth. The second interface is the interface displayed by the first application when the camera is turned on, and between the ninth and tenth moments, the terminal device's display bandwidth is the second value; the terminal device receives a third operation to turn off the camera in the first application; at the eleventh moment, in response to the third operation, the terminal device displays the first interface, and between the tenth and eleventh moments, the terminal device's display bandwidth is always the first value.
[0249] Combination Figure 10 as well as Figure 8 In a corresponding embodiment, the terminal device can set the display bandwidth to a second value when it detects that the camera is turned on, and determine to set the display bandwidth to a first value when it detects that a preset layer is among the N layers, and process the N layers based on the first value.
[0250] The above combination Figures 4-11 The methods provided in the embodiments of this application have been described. The apparatus for executing the above methods, provided in the embodiments of this application, is described below. Figure 12 As shown, Figure 12 This is a schematic diagram of the structure of a display device provided in an embodiment of this application. The display device may be a terminal device in the embodiment of this application, or a chip or chip system within a terminal device.
[0251] like Figure 12As shown, the display device 1200 can be used in communication equipment, circuits, hardware components, or chips. The display device 1200 includes a display unit 1201 and a processing unit 1202. The display unit 1201 supports the display steps performed by the display method; the processing unit 1202 supports the information processing steps performed by the display device 1200.
[0252] In one possible implementation, the display device 1200 may further include a communication unit 1203, which is used to support the display device 1200 in performing steps such as receiving or sending messages.
[0253] The display devices described in the embodiments of this application may include all of the following: Figure 12 The units described in the corresponding embodiments.
[0254] Specifically, the processing unit 1202 can be integrated with the display unit 1201, and the processing unit 1202 and the display unit 1201 may communicate with each other.
[0255] In one possible implementation, the display device 1200 may further include a storage unit 1204. The storage unit 1204 may include one or more memories, which may be devices in one or more devices or circuits used to store programs or data.
[0256] The storage unit 1204 can exist independently and be connected to the processing unit 1202 via a communication bus. Alternatively, the storage unit 1204 can be integrated with the processing unit 1202.
[0257] Taking the display device 1200 as an example, which may be a chip or chip system of the terminal device in the embodiments of this application, the storage unit 1204 may store computer-executable instructions of the terminal device's methods, so that the processing unit 1202 executes the methods of the terminal device in the above embodiments. The storage unit 1204 may be a register, cache, or random access memory (RAM), etc., and the storage unit 1204 may be integrated with the processing unit 1202. The storage unit 1204 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, and the storage unit 1204 may be independent of the processing unit 1202.
[0258] In one possible implementation, the display device 1200 may further include a communication unit 1203. The communication unit 1203 supports interaction between the display device 1200 and other devices. For example, when the display device 1200 is a terminal device, the communication unit 1203 may be a communication interface or interface circuit. When the display device 1200 is a chip or chip system within a terminal device, the communication unit 1203 may be a communication interface. For example, the communication interface may be an input / output interface, pins, or circuits.
[0259] The apparatus in this embodiment can be used to execute the steps performed in the above method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.
[0260] Figure 13 This is a schematic diagram of the hardware structure of another terminal device provided in an embodiment of this application.
[0261] The terminal device includes a processor 1301, a communication line 1304, and at least one communication interface. Figure 13 (The example is illustrated using communication interface 1303).
[0262] The processor 1301 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.
[0263] Communication line 1304 may include circuitry for transmitting information between the aforementioned components.
[0264] Communication interface 1303 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, wireless local area networks (WLAN), etc.
[0265] The terminal device may also include a memory 1302.
[0266] The memory 1302 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via communication line 1304. The memory may also be integrated with the processor.
[0267] The memory 1302 stores computer execution instructions for implementing the scheme of this application, and the processor 1301 controls the execution. The processor 1301 executes the computer execution instructions stored in the memory 1302 to implement the method provided in the embodiments of this application.
[0268] It is possible that the computer execution instructions in the embodiments of this application may also be referred to as application code, and the embodiments of this application do not specifically limit this.
[0269] In a specific implementation, as one example, the processor 1301 may include one or more CPUs, for example... Figure 13 CPU0 and CPU1 in the CPU.
[0270] In a specific implementation, as one example, the terminal device may include multiple processors, for example... Figure 13 Processors 1301 and 1305 are mentioned. Each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).
[0271] In the above embodiments, the instructions stored in the memory for execution by the processor can be implemented in the form of a computer program product. This computer program product can be pre-written into the memory, or it can be downloaded and installed into the memory as software.
[0272] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. For example, available media may include magnetic media (e.g., floppy disks, hard disks, or magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).
[0273] This application also provides a computer-readable storage medium. The methods described in the above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. The computer-readable medium may include computer storage media and communication media, and may also include any medium capable of transferring a computer program from one place to another. The storage medium can be any target medium accessible by a computer.
[0274] As one possible design, computer-readable media may include compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM, or other optical disc storage; computer-readable media may also include disk storage or other disk storage devices. Furthermore, any connecting cable may also be appropriately referred to as computer-readable media. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. As used herein, disks and optical discs include optical discs (CD), laser discs, optical discs, digital versatile discs (DVD), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers.
[0275] The above combinations should also be included within the scope of computer-readable media. The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
[0276] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.
Claims
1. A display method characterized by comprising: The method comprises: At a first time, a terminal device receives a first operation of opening a first application; In response to the first operation, the terminal device displays a first interface; The terminal device receives a second operation of opening a camera in the first application; In response to the second operation, the terminal device displays a second interface, which is an interface displayed by the first application when the camera is opened; The terminal device receives a third operation of closing the camera in the first application; At a second time, in response to the third operation, the terminal device displays the first interface, and the display bandwidth of the terminal device is a first value between the first time and the second time; At a third time, the terminal device receives a fourth operation of opening a second application; In response to the fourth operation, the terminal device displays a third interface; At a fourth time, the terminal device receives a fifth operation of opening a camera in the second application, and the display bandwidth of the terminal device is the first value between the third time and the fourth time; At a fifth time, in response to the fifth operation, the terminal device displays a fourth interface, which is an interface displayed by the second application when the camera is opened, and the display bandwidth of the terminal device is the first value between the fourth time and the fifth time; The terminal device receives a sixth operation of closing the camera in the second application; At a sixth time, in response to the sixth operation, the terminal device displays the third interface, and the display bandwidth of the terminal device is a second value between the fifth time and the sixth time, the first value being greater than the second value; After the response to the second operation, the method further comprises: when the terminal device determines that the first application is a preset application and the state of the camera is opened, the terminal device sets the display bandwidth of the terminal device to the first value, the preset application being an application whose required display bandwidth when the preset application is run on the terminal device is greater than the second value, and / or the preset application being an application whose frequency of flashing of the terminal device when the preset application is run is greater than a first threshold value. After the response to the fifth operation, the method further comprises: when the terminal device determines that the second application is not the preset application and the state of the camera is opened, the terminal device sets the display bandwidth of the terminal device to the second value.
2. The method of claim 1, wherein, The terminal device is configured with a first application, a camera hardware abstraction layer (HAL), and a hardware composition processor, and when the terminal device determines that the first application is a preset application and the state of the camera is opened, the terminal device sets the display bandwidth of the terminal device to the first value, and the method further comprises: The first application sends a first message to the camera HAL, the first message comprising a message for indicating that the state of the camera is opened and a package name of the first application; The camera HAL sends the first message to the hardware composition processor; In a case where the hardware composition processor determines, based on the first message, that the first application is the preset application and the state of the camera is turned on, the hardware composition processor sets a display bandwidth of the terminal device as the first value.
3. The method of claim 2, wherein, The terminal device is further configured with a display composition processor, a data processor (DPU), and a liquid crystal display (LCD). After the hardware composition processor sets the display bandwidth of the terminal device as the first value, the method further includes: The hardware composition processor sends the first value to the DPU; The first application sends N layers of images to the DPU via the camera HAL, the display composition processor, and the hardware composition processor; In a case where the DPU determines that a sum of display bandwidths of the N layers of images is less than the first value, the DPU composes the N layers of images into a target image, and sends the target image to the LCD.
4. The method of claim 2, wherein, After responding to the third operation, the method further includes: The first application sends a second message to the camera HAL, the second message including a message indicating that the state of the camera is turned off and a package name of the first application; The camera HAL sends the second message to the hardware composition processor; In a case where the hardware composition processor determines, based on the second message, that the state of the camera is turned off, the hardware composition processor sets a display bandwidth of the terminal device as the first value.
5. The method of claim 2, wherein, In a case where the terminal device determines that the second application is not the preset application and the state of the camera is turned on, the terminal device sets a display bandwidth of the terminal device as a second value, including: The second application sends a third message to the camera HAL, the third message including a message indicating that the state of the camera is turned on and a package name of the second application; The camera HAL sends the third message to the hardware composition processor; In a case where the hardware composition processor determines, based on the third message, that the second application is not the preset application and the state of the camera is turned on, the hardware composition processor sets a display bandwidth of the terminal device as the second value.
6. The method according to any one of claims 1 to 5, characterized in that, Before the terminal device receives a first operation of opening the first application, the method further includes: The terminal device receives a seventh operation of starting up or restarting; At a seventh time, in response to the seventh operation, the terminal device displays a fifth interface, the fifth interface being a desktop of the terminal device, the seventh time being before the first time, and the display bandwidth of the terminal device being the first value between the seventh time and the first time.
7. A display method characterized by comprising: The method includes: At an eighth time, the terminal device receives a first operation of opening the first application; In response to the first operation, the terminal device displays a first interface; The terminal device receives a second operation of turning on the camera in the first application; In response to the second operation, the terminal device sets the display bandwidth of the terminal device to a second value, and the display bandwidth of the terminal device is the first value between the eighth time and the ninth time, and the first value is greater than the second value; In response to the third operation, the terminal device displays the first interface, and the display bandwidth of the terminal device is the first value between the tenth time and the eleventh time. The terminal device receives a third operation for closing the camera of the first application; In response to the third operation, the terminal device displays the first interface, and the display bandwidth of the terminal device is the first value between the tenth time and the eleventh time.
8. The method of claim 7, wherein, The terminal device is configured with a first application, a camera hardware abstraction layer (HAL), a display compositor, and a hardware composition processor, When the terminal device obtains N layers and determines that there is a preset layer based on the layer names of the N layers, the terminal device sets the display bandwidth of the terminal device to the first value, including: The first application sends the N layers to the hardware composition processor via the camera HAL and the display compositor, and the N layers carry the layer names corresponding to any layer in the N layers; When the hardware composition processor determines that there is the preset layer based on the layer names of the N layers, the hardware composition processor sets the display bandwidth of the terminal device to the first value.
9. The method of claim 8, wherein, The terminal device is further configured with a DPU and an LCD, and after the hardware composition processor sets the display bandwidth of the terminal device to the first value, the method further includes: The hardware composition processor sends the first value to the DPU; The hardware composition processor sends the N layers to the DPU; When the DPU determines that the sum of the display bandwidths of the N layers is less than the first value, the DPU composes the N layers into a target image and sends the target image to the LCD.
10. The method according to any one of claims 7-9, characterized in that, Before the terminal device receives a first operation of opening the first application, the method further includes: The terminal device receives a seventh operation of starting up or restarting; In response to the seventh operation, the terminal device displays a fifth interface at a seventh time, the fifth interface is a desktop of the terminal device, the seventh time is before the eighth time, and the display bandwidth of the terminal device is the first value between the seventh time and the eighth time.
11. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program, so that the terminal device executes the method in any one of claims 1-6 or executes the method in any one of claims 7-10.
12. A computer-readable storage medium, the computer-readable storage medium storing a computer program, characterized in that, The computer program, which is executed by the processor, causes the computer to perform the method according to any one of claims 1-6, or to perform the method according to any one of claims 7-10.
Citation Information
Patent Citations
Method and device for dynamic bandwidth adaptation of display layer
CN110610688A
Camera starting method and related apparatus
CN113366819A