Concurrent processing method for image generation and screen dimming, and terminal device

Abstract

The present application relates to the technical field of terminals, and discloses a concurrent processing method for image generation and screen dimming, and a terminal device. The method comprises: when a terminal device detects an operation input by a user to play back a dynamic image on a display screen of the terminal device, and if the terminal device adjusts the brightness of the display screen at the same time, a display task set received from an image compositor by a hardware compositor of the terminal device comprises a dimming task and a sending and displaying task, and the hardware compositor sends the sending and displaying task to a first thread for execution, and sends the dimming task to a second thread for execution; and in this way, the first thread and the second thread respectively execute respective tasks, and once the sending and displaying task has been executed in the first thread, an image is sent to the display screen for display. The problem that when the terminal device detects the operation input by the user to play back the dynamic image on the display screen of the terminal device, and if the terminal device adjusts the brightness of the display screen at the same time in the process of playing back the dynamic image, the display picture stuttering is solved.

Description

Processing method and terminal device for concurrently generating image and dimming screen

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on December 29, 2023, with application number 202311865422.0 and invention name “Processing method and terminal device for generating images and screen dimming concurrently”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a processing method and terminal device for concurrently generating an image and dimming a screen. Background Art

[0003] With the development of terminal technology, the interaction between various terminals (such as mobile phones) and users is becoming more and more diverse. Among them, most of the interactions come from the display screen on the terminal, and the images displayed on the display screen usually need to go through processes such as drawing, rendering and synthesis.

[0004] Among them, the application process of the terminal device is responsible for drawing and rendering various visual elements in the display screen, and the image synthesis system of the terminal device is responsible for synthesizing the different layers where various visual elements in the display screen are located and displaying them. The synthesis and display process is carried out frame by frame. After synthesizing a frame of image, the image synthesis system sends the frame of image to the display screen for display, and then performs the synthesis operation of the next frame of image. If the frame of image is not completed and displayed within the specified time, the synthesis of the next frame of image cannot be started, resulting in frame loss.

[0005] In some cases, the image synthesis system will perform other tasks while sending the image for display, resulting in the failure to complete the display within the specified time, which in turn causes the next frame of image to be unable to be synthesized, causing the display screen to freeze. Summary of the Invention

[0006] The embodiment of the present application provides a method and terminal device for concurrently generating an image and dimming a screen, which solves the problem of frame loss on the display screen during the process of the terminal device playing a first dynamic image and adjusting the brightness of the display screen.

[0007] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0008] In a first aspect, a method for concurrently generating an image and dimming a screen is provided, the method comprising:

[0009] When the terminal device detects the user input operation and plays a dynamic image on the display screen of the terminal device, if the terminal device is adjusting the brightness of the display screen at the same time, the display task set received by the hardware synthesizer of the terminal device from the image synthesizer will include a dimming task and a display sending task. The hardware synthesizer sends the display sending task to the first thread for execution, and sends the dimming task to the second thread for execution. In this way, the first thread and the second thread execute their respective tasks, and after the first thread completes the display sending task, the image is sent to the display screen for display.

[0010] In this implementation, when the display task set includes a display transmission task and a dimming task, the hardware synthesizer, upon receiving the display task set, sends the dimming task and the display transmission task in the display task set to different threads for execution. Since each thread runs independently, after the first thread completes the display transmission task, it can send the image to the display screen for display without having to wait for the second thread to complete the dimming task, and promptly notify the image synthesizer to perform the synthesis operation for the next frame of the image. This avoids the problem of frame loss on the display screen due to execution timeout when the hardware synthesizer simultaneously executes the display transmission task and the dimming task, resulting in failure to complete the display transmission within the specified time. It also solves the problem of display freeze when the terminal device detects user input and plays a dynamic image on the terminal device's display screen, and the terminal device simultaneously adjusts the brightness of the display screen during the playback of the dynamic image.

[0011] In one possible implementation, the image compositor may include a SurfaceFlinger process, and the hardware compositor may include a Composer process. The SurfaceFlinger process is used to synthesize content rendered by various applications to be displayed on the display screen. Each application renders on a Surface, and the Surface corresponds to a layer. The SurfaceFlinger process performs a compositing operation on the rendered Surface layer. In addition, the SurfaceFlinger process generates a display task based on the compositing operation and saves the display task in a display task set.

[0012] In a possible implementation, the SurfaceFlinger process may also receive a dimming task sent by the dimming process, and save the dimming task in a display task set.

[0013] In a possible implementation, the terminal device adjusting the brightness of the display screen includes a user manually triggering the adjustment of the brightness of the display screen and the terminal device automatically triggering the adjustment of the brightness of the display screen.

[0014] The user actively adjusting the display brightness may involve pulling down the display to open the terminal device's control center and adjusting the display brightness by sliding the dimming control in the control center. The terminal device automatically triggering the display brightness adjustment may involve adjusting the display brightness when an ambient light sensor deployed on the terminal device detects changes in ambient light.

[0015] In a possible implementation, the user's operation includes: starting an application, sliding a display page, playing a video, etc.

[0016] In a possible implementation of the first aspect, the image synthesizer adds the display sending task and the dimming task to the display task set in the first display sending cycle.

[0017] In one possible implementation, the first display transmission cycle is represented by the time after the current Vsync-SF signal arrives at the image synthesizer and before the next Vsync-SF signal arrives at the image synthesizer. During this time, the image synthesizer adds the display transmission task and the dimming task to the display task set.

[0018] In this implementation, the image synthesizer adds the display sending task and the dimming task to the display task set within a display sending cycle, so that the hardware synthesizer can execute the display sending task in the display task set within the display sending cycle.

[0019] In a possible implementation of the first aspect, a first thread in a hardware synthesizer of the terminal device executes a display task in a display task set in a first display transmission cycle.

[0020] In a possible implementation, the first thread may be a main thread of a hardware synthesizer.

[0021] In one possible implementation of the first aspect, before the second thread of the hardware synthesizer executes the dimming task, it raises its thread priority so that the second thread is preferentially invoked by the processor. This reduces the time it takes for the terminal device to respond to display brightness changes, enhancing the user experience.

[0022] In a possible implementation, the second thread includes a background thread, the background thread includes a task queue, the dimming task is stored in the task queue, and the background thread cyclically executes the tasks in the task queue.

[0023] In a possible implementation of the first aspect, after executing the dimming task, the second thread of the hardware synthesizer lowers the priority of the thread.

[0024] In a possible implementation of the first aspect, the priority of the second thread called by the processor is increased, including: the priority of the second thread called by the processor is adjusted from a first value to a second value; the priority of the second thread called by the processor is lowered, including: the priority of the second thread called by the processor is adjusted from the second value to the first value.

[0025] In a possible implementation, the priority of the second thread is adjusted to the priority before the dimming task is executed.

[0026] In a possible implementation, the priority of the second thread is adjusted to the default priority of the thread.

[0027] In a possible implementation of the first aspect, after the first thread in the hardware compositor executes the display sending task in the display task set, the hardware compositor returns a resource unlocking signal to the image compositor in the first display sending cycle.

[0028] In a possible implementation, the resource unlocking signal is used to notify the image synthesizer to synthesize the next frame of image.

[0029] In this implementation, the hardware synthesizer returns a resource unlock signal to the image synthesizer within a display transmission cycle, so that the image synthesizer can perform image synthesis operations when the next display transmission cycle arrives, thereby avoiding frame loss.

[0030] In a possible implementation of the first aspect, after the hardware compositor receives the display task set from the image compositor, the terminal device determines that the display task set includes the dimming task, and distributes the dimming task to the second thread.

[0031] In one possible implementation, a function call can be used to call the dimming parameters in the display task set. If the return result is a specific value, it can be determined that the dimming parameters exist in the display task. If the return result is empty, it is determined that the dimming task does not exist in the display task set.

[0032] In this implementation, the communication thread in the hardware compositor is used to determine whether there is a dimming task in the display task set and distribute the dimming task.

[0033] In a possible implementation manner of the first aspect, after the terminal device determines that the display task set includes the dimming task, the terminal device establishes a second thread.

[0034] In a possible implementation of the first aspect, the second thread is a background thread.

[0035] In a second aspect, a terminal device is provided, comprising: a processor and a memory; the memory is used to store computer execution instructions, and when the terminal device is running, the processor executes the computer execution instructions stored in the memory to enable the terminal device to execute a method as described in any one of the above-mentioned first aspects.

[0036] In a third aspect, a chip system is provided, including a processor for supporting a terminal device in implementing the functions described in the first aspect. In one possible design, the device also includes a memory for storing program instructions and data necessary for the terminal device. When the device is a chip system, it can be composed solely of a chip or include a chip and other discrete components.

[0037] In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the computer-readable storage medium is run on a computer, the computer can execute any one of the methods in the first aspect.

[0038] Among them, the technical effects brought about by any design method in the second to fourth aspects can refer to the technical effects brought about by different design methods in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] FIG1 is a schematic diagram of a display interface of a terminal device;

[0040] FIG2 is a schematic diagram of an image sending and displaying process;

[0041] FIG3 is a schematic diagram of a scenario applicable to the method for concurrently processing image generation and screen dimming provided in an embodiment of the present application;

[0042] FIG4 is a schematic diagram of another scenario applicable to the method for concurrently generating an image and dimming a screen provided in an embodiment of the present application;

[0043] FIG5 is a flow chart of a method for concurrently generating an image and dimming a screen provided in an embodiment of the present application;

[0044] FIG6 is a schematic diagram of a scenario in which the method for concurrently generating an image and dimming a screen provided in an embodiment of the present application is applicable;

[0045] FIG7 is a schematic diagram of another scenario applicable to the method for concurrently generating an image and dimming a screen provided in an embodiment of the present application;

[0046] FIG8 is an interactive diagram of a method for concurrently processing image generation and screen dimming provided by an embodiment of the present application;

[0047] FIG9 is a flowchart of a method for concurrently generating an image and dimming a screen provided in an embodiment of the present application;

[0048] FIG10 is a schematic structural diagram of a terminal device provided in an embodiment of the present application;

[0049] FIG11 is a schematic structural diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: the situation where A exists alone, A and B exist at the same time, and B exists alone, wherein A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are a kind of "or" relationship.

[0051] References to "one embodiment" or "some embodiments" etc. described in this specification mean that the specific features, structures or characteristics described in conjunction with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in another way. The term "connected" includes direct and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0052] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0053] An operating system (OS) is a set of interrelated system software programs that manage and control terminal device operations, utilize and run hardware and software resources, and provide public services to organize user interactions. It is a bridge connecting terminal device hardware and software.

[0054] The image synthesis system (also called the compositor) is an important component of the operating system. It is responsible for managing and scheduling the terminal device's central processing unit (CPU), graphics processing unit (GPU), display driver and other hardware devices, as well as software resources such as the graphics library and media library to synthesize surfaces rendered by different applications (APP) and display them on the display.

[0055] As shown in Figure 1, a schematic diagram of the display interface of a terminal device includes visual elements such as icons, wallpapers, and widgets in the picture displayed on the display screen of the terminal device. Each icon, wallpaper, widget, and other visual elements comes from a corresponding application and is rendered on the Surface. The image synthesis system synthesizes them into a frame of image, and the frame of image is displayed on the display screen.

[0056] For example, if the image displayed on the display screen of the terminal device in FIG1 is to be displayed, the synthesis and display process includes: the APP process that needs to display the visual element on the display screen requests a Vsync signal from the Vsync thread in the image synthesis process (SurfaceFlinger). When the Vsync signal reaches the APP process, the APP process renders on the Surface to obtain a rendered Surface. A buffer is requested from the cache queue, and the rendered Surface is stored in the buffer. When the Vsync signal reaches the SurfaceFlinger process, the SurfaceFlinger process takes the buffer from the cache queue, reads the rendered Surface on the buffer, and performs a synthesis operation based on the layer where the rendered Surface is located to determine the synthesis data of the rendered Surface (display position, display range, display layer, transparency, etc. in the picture displayed on the display screen). Furthermore, the synthesis thread is also responsible for sending the rendered Surface and synthesis data to the hardware synthesizer (HWComposer, HWC), referred to as the Composer process in the embodiment of this application. The Composer process provides hardware synthesis support for the SurfaceFlinger process and sends the synthesized image to the display screen for display through the display driver.

[0057] Among them, the Vsync signal is used to synchronize the display transmission cycle with the refresh cycle (frame rate) of the display screen. The display transmission cycle of the embodiment of the present application represents the time interval between two consecutive Vsync signals arriving at the SurfaceFlinger process. The period at which the Vsync thread generates the Vsync signal is related to the frame rate of the display screen refresh. The frame rate refers to the number of frames that refresh the displayed image in 1 second, which can also be understood as the number of times the graphics processor in the terminal device refreshes the screen per second. A high frame rate can produce a smoother and more realistic picture. The more frames per second, the smoother the displayed picture will be. For example, a frame rate of 60 Hz means that 60 frames of images are refreshed in 1 second, that is, one frame of image is refreshed every 16.6 milliseconds. Accordingly, the period at which the Vsync thread generates the Vsync signal is 16.6 milliseconds. For example, a frame rate of 90 Hz means that 90 frames of image are refreshed in 1 second, that is, one frame of image is refreshed every 11.1 milliseconds. Accordingly, the period at which the Vsync thread generates the Vsync signal is 11.1 milliseconds. Currently, the refresh frame rate of the display screen also includes specifications such as 120HZ and 144HZ, which will not be introduced one by one here.

[0058] The Vsync signals generated by the Vsync thread include the Vsync_APP signal, the Vsync_SF signal, and the HW_Vsync signal. The Vsync thread generates the Vsync_APP signal and sends it to the APP process, which then renders on the Surface. The Vsync thread generates the Vsync_SF signal and sends it to the SurfaceFlinger process. When the Vsync_SF signal arrives, the SurfaceFlinger process obtains the rendered Surface and performs synthesis operations on the rendered Surface. The Vsync thread generates the HW_Vsync signal and sends it to the display driver of the terminal device. When the HW_Vsync signal arrives, the display driver refreshes the displayed image.

[0059] When the Vsync_SF signal reaches the SurfaceFlinger process, the SurfaceFlinger process and the Composer process execute the image synthesis and display process.

[0060] In an interactive example, the Vsync thread periodically sends a Vsync-SF signal to the SurfaceFlinger process. After receiving the Vsync-SF signal, the SurfaceFlinger process first takes out the buffer from the data cache queue, reads the rendered Surface stored on the buffer, and then performs a synthesis operation on the rendered Surface to determine the layer positions of these rendered Surfaces in the multiple layers included in the display screen, as well as synthesis parameters such as the display range and display position in the corresponding layers. According to the layer position of each rendered Surface in the display screen, as well as the display range and display position in the corresponding layers, a display sending task is generated, and the display sending task is saved in the display task set (Display Commad). Finally, it is sent to the Composer process through cross-process communication. After receiving the display task set, the Composer process's communication thread extracts the display delivery task from the display task set and sends it to the Composer process's main thread for execution. When the display delivery task is sent to the Composer process's main thread for execution, the Composer process releases the fence resource and returns a resource unlock signal to the SurfaceFlinger process, which notifies the SurfaceFlinger process that it can perform the next frame of image synthesis. After completing the display delivery task, the Composer process's main thread displays the synthesized image through the display driver.

[0061] The SurfaceFlinger process waits for the resource unlock signal returned by the Composer process until it receives the resource unlock signal. After receiving the resource unlock signal, when the next Vsync-SF signal arrives, the SurfaceFlinger process performs the synthesis operation of the next frame of the image to be displayed. If the SurfaceFlinger process has not received the resource unlock signal returned by the Composer process when the next Vsync-SF signal arrives, the SurfaceFlinger process does not perform the synthesis operation of the next frame of the image to be displayed.

[0062] In some embodiments, a terminal device with a display brightness adjustment function can detect changes in ambient light through an ambient light sensor and then control the display brightness according to the ambient light intensity. For example, when the user is outdoors in a well-lit area, the phone controls the display brightness to increase so that the user can see the content displayed on the display clearly. When the user is indoors in a low-light area, the phone controls the display brightness to decrease to avoid excessive brightness and increased eye fatigue. In addition, the display brightness can also be adjusted based on manual triggering by the user. For example, the user can adjust the display brightness by sliding up and down, or left and right in the dimming control in the drop-down control center.

[0063] In a possible application scenario, if the user uses the terminal device to play a video, start an application, slide to switch the display screen, etc., the terminal device receives a request from the user to adjust the display brightness, or the terminal device automatically adjusts the display brightness according to changes in ambient light. The dimming process (LightsService) responsible for responding to the display brightness adjustment request generates a dimming task and sends it to the SurfaceFlinger process. The APP responsible for displaying the screen on the display stores the rendered Surface in the cache queue. The SurfaceFlinger process takes the rendered Surface from the cache queue for synthesis, generates a display task based on the synthesized data, and saves the display task and the dimming task in a display task set. It is sent to the Composer process through cross-process communication. The communication thread of the Composer process, according to the system's task execution logic, determines that there is a dimming task in the display task set and executes the dimming task first. After the dimming task is completed, the display task is sent to the main thread for execution. Then, the communication thread of the Composer process sends the display task to the main thread for execution and returns the resource unlock signal to the SurfaceFlinger process. Therefore, if the communication thread takes too long to execute the dimming task, it will not be able to return the resource unlock signal before the next Vsync-SF signal reaches the SurfaceFlinger process, causing the SurfaceFlinger process to not perform the synthesis operation of the next frame of the image, resulting in frame loss and freezes on the display screen.

[0064] As shown in Figure 2, a schematic diagram of the image display process is provided. A detailed analysis shows that in the current image synthesis system, when the Composer process executes a display task set including a dimming task, an execution timeout occurs (all display tasks are not completed within one display cycle), causing the display screen to drop frames. It should be noted that the following Vsync-SF-1 to Vsync-SF-5 signals represent the Vsync thread periodically sending Vsync-SF signals to the SurfaceFlinger process continuously. The following examples are no longer limited to one by one.

[0065] 1. The Vsync-SF-1 signal arrives at the SurfaceFlinger process, and the synthesis thread of the SurfaceFlinger process starts to perform the synthesis operation for the first frame of image to be displayed and generates a display task; the dimming process sends the dimming task to the SurfaceFlinger process through cross-process communication, and the communication thread packages the dimming task and the display task together and saves them in the display task set, and sends the display task set to the Composer process through cross-process communication.

[0066] The Composer process's communication thread receives the display task set and prioritizes the dimming task within it. After the dimming task is completed, the display delivery task is passed to the main thread for execution. After passing the display delivery task to the main thread, the communication thread returns a resource unlock signal to the SurfaceFlinger process, indicating that the SurfaceFlinger process can proceed with compositing the next frame. After the Composer process's main thread completes the display delivery task, it sends the first frame of image to be displayed to the display driver.

[0067] 2. The Vsync-SF-2 signal arrives at the SurfaceFlinger process. The SurfaceFlinger process has received the resource unlock signal returned by the Composer process. The synthesis thread performs the synthesis operation for the second frame image to be displayed, and sends the display task corresponding to the synthesis operation to the communication thread. The dimming process sends the dimming task to the SurfaceFlinger process through cross-process communication. The communication thread packages the dimming task and the display task together and saves them in the display task set, and sends the display task set to the Composer process through cross-process communication.

[0068] The communication thread of the Composer process receives the display task set and prioritizes executing the dimming task in the display task set. At this time, the dimming task execution times out. The reason for the execution timeout here may be: when executing the dimming task and interacting with the hardware (backlight driver), the hardware is occupied by other processes, and it is necessary to wait for the other process to release the occupation before continuing to execute the dimming task.

[0069] At this time, the first frame image is still displayed on the screen.

[0070] 3. The Vsync-SF-3 signal reaches the SurfaceFlinger process. Since the SurfaceFlinger process has not yet received the resource unlock signal returned by the Composer process, the synthesis thread of the SurfaceFlinger process does not perform the synthesis operation.

[0071] The Composer process's communication thread sends the display task to the main thread for execution. After sending the display task to the main thread for execution, the communication thread returns a resource unlock signal to the SurfaceFlinger process. The Composer process's main thread completes the display task and sends the second frame of image to be displayed to the display driver.

[0072] 4. The Vsync-SF-4 signal arrives at the SurfaceFlinger process. The synthesis thread of the SurfaceFlinger process performs the synthesis operation for the fourth frame image to be displayed and generates a display task. The dimming process sends the dimming task to the SurfaceFlinger process through cross-process communication. The communication thread packages the dimming task and the display task together and saves them in the display task set, and sends the display task set to the Composer process through cross-process communication.

[0073] The Composer process's communication thread sends the display task to the main thread for execution. After placing the display task in the main thread for execution, the communication thread returns a resource unlock signal to the SurfaceFlinger process. The Composer process's main thread executes the display task and sends the fourth frame of image to be displayed to the display driver.

[0074] 5. The Vsync-SF-5 signal arrives at the SurfaceFlinger process, and the SurfaceFlinger process continues to perform the synthesis operation.

[0075] Vsync-SF-n indicates that the subsequent Vsync-SF signal arrives at the SurfaceFlinger process, where n is a positive integer, indicating the nth Vsync-SF signal.

[0076] In order to intuitively demonstrate the actual impact of execution timeout of the Composer process when executing a display task set including a dimming task in the current image synthesis system, a scenario diagram is shown in Figure 3. Figures 3 (a) to 3 (d) show schematic diagrams of a user manually adjusting the display brightness while watching a video, causing the display screen to freeze.

[0077] In conjunction with the interaction process shown in Figure 2 above, during the first display transmission cycle (after the Vsync-SF-1 signal reaches the SurfaceFlinger process and before the Vsync-SF-2 signal reaches the SurfaceFlinger process), the SurfaceFlinger process synthesizes the first frame of image to be displayed in the video. The Composer process performs the display transmission task, returns the resource unlock signal to the SurfaceFlinger process, and after completing the display transmission task, sends the first frame of image to be displayed to the display driver. As shown in Figure 3(a), the display screen displays the first frame of image.

[0078] During the second display transmission cycle (after the Vsync-SF-2 signal arrives at the SurfaceFlinger process and before the Vsync-SF-3 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process performs the synthesis operation of the second frame image to be displayed. The Composer process times out when executing the display transmission task and does not return the resource unlock signal. As shown in Figure 3(b), the display screen displays the first frame image.

[0079] During the third display cycle (after the Vsync-SF-3 signal arrives at the SurfaceFlinger process and before the Vsync-SF-4 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process does not perform the synthesis operation of the third frame image to be displayed. The Composer process performs the display task of the second frame image, returns the resource unlocking signal to the SurfaceFlinger process, and after completing the display task, sends the second frame image to the display driver. As shown in Figure 3 (c), the display screen displays the second frame image.

[0080] During the fourth display cycle (between the time the Vsync-SF-4 signal reaches the SurfaceFlinger process and the time the Vsync-SF-5 signal reaches the SurfaceFlinger process), the SurfaceFlinger process synthesizes the fourth frame of image to be displayed, the Composer process performs the display task of the second frame of image, returns the resource unlock signal to the SurfaceFlinger process, and after completing the display task, sends the fourth frame of image to the display driver. As shown in Figure 3(d), the display screen displays the fourth frame of image.

[0081] In addition, as shown in another scenario diagram in FIG4, FIG4 (a) to FIG4 (d) show a schematic diagram of a display screen that is stuck when the user clicks the camera application during the automatic dimming process. It can be seen that when synthesizing an image, the Surface rendered by all APPs displayed on the entire display screen is synthesized. The specific process is similar to the example in FIG3 above and will not be described separately here. It can be seen that in the example scenarios in FIG3 and FIG4 above, two frames of the same image (the first frame image) are displayed on the display screen, and then the second frame image and the fourth frame image are displayed. Due to the lack of the third frame image, when the second frame image changes to the fourth frame image, the picture changes greatly. When the user watches the display screen, the picture will feel stuck and not smooth, which affects the user's viewing experience.

[0082] In addition, it should be noted that the above examples are only examples of certain usage scenarios in which the display screen may freeze due to problems with the image synthesis system's display transmission process. In actual applications, there are many usage scenarios similar to the above examples, and the embodiments of this application will no longer give examples one by one. For the sake of convenience, the embodiments of this application are all illustrated with four display transmission cycles. In fact, the display transmission cycle is continuous, and the image synthesis system continuously performs synthesis and display transmission operations. In actual applications, there may also be multiple display transmission cycles with consecutive execution timeouts, resulting in severe screen freezes and affecting the user experience.

[0083] An embodiment of the present application provides a processing method for concurrently generating an image and dimming the screen. The Composer process obtains a display task set in which the SurfaceFlinger process packages the dimming task and the display sending task together, and sends the dimming task in the display task set to a thread different from the display sending task (for example, a background thread) for execution. The two threads independently execute their respective tasks. The main thread executes the display sending task, and the background thread executes the dimming task. After the main thread completes the display sending task, the Composer process does not have to wait for the background thread to execute the dimming task, and returns a resource unlocking signal to the SurfaceFlinger process. After receiving the resource unlocking signal, the SurfaceFlinger process prepares for the synthesis operation of the next frame of the image to be displayed. In this way, since the display sending task and the dimming task are executed in different threads respectively, and the Composer process does not have to wait for the background thread to execute the dimming task after the main thread completes the display sending task, it can return the resource unlocking signal within one display sending cycle, thus solving the above-mentioned frame loss problem.

[0084] Furthermore, the terminal device used in the embodiment of the present application can improve the stability and smoothness of the display screen when the display screen triggers a dimming operation during the process of changes in the display screen image (for example, application startup, page switching, video playback, etc.).

[0085] Furthermore, the embodiment of the present application optimizes the processing mechanism of the display delivery process in the Composer process, avoiding changes to the task packaging mechanism of the system's native SurfaceFlinger process (saving dimming tasks and display delivery tasks in a display task set), and has the advantages of simple implementation and easy operation.

[0086] As shown in Figure 5, a flow chart of a method for concurrently processing image generation and screen dimming applicable to an embodiment of the present application is provided. An image synthesis system that applies the method for concurrently processing image generation and screen dimming provided in an embodiment of the present application is analyzed in detail, and the interaction process when the Composer process executes a display task set including a dimming task.

[0087] 1. The Vsync-SF-1 signal arrives at the SurfaceFlinger process, and the synthesis thread of the SurfaceFlinger process starts to perform the synthesis operation for the first frame of image to be displayed and generates a display task; the dimming process sends the dimming task to the SurfaceFlinger process through cross-process communication, and the communication thread packages the dimming task and the display task together in the display task set, and sends the display task set to the Composer process through cross-process communication.

[0088] The communication thread of the Composer process receives the display task set, sends the dimming task to the background thread of the Composer process for execution, and sends the display task to the main thread of the Composer process for execution. After sending the display task to the main thread of the Composer process for execution, it returns a resource unlock signal to the SurfaceFlinger process. After the main thread completes the display task of the first frame of image, it sends the first frame of image to the display driver.

[0089] 2. The Vsync-SF-2 signal arrives at the SurfaceFlinger process, and the synthesis thread of the SurfaceFlinger process starts to perform the synthesis operation for the second frame image to be displayed and generates a display task; the dimming process sends the dimming task to the SurfaceFlinger process through cross-process communication, and the communication thread packages the dimming task and the display task together and saves them in the display task set, and sends the display task set to the Composer process through cross-process communication.

[0090] The communication thread of the Composer process receives the display task set, and the communication thread of the Composer process sends the dimming task to the background thread of the Composer process for execution, and sends the display task to the main thread of the Composer process for execution. After sending the display task to the main thread of the Composer process for execution, it returns a resource unlock signal to the SurfaceFlinger process. After the main thread completes the display task of the second frame of image, it sends the second frame of image to the display driver.

[0091] 3. The Vsync-SF-3 signal arrives at the SurfaceFlinger process, and the synthesis thread of the SurfaceFlinger process starts to perform the synthesis operation for the third frame image to be displayed and generates a display task; the dimming process sends the dimming task to the SurfaceFlinger process through cross-process communication, and the communication thread packages the dimming task and the display task together and saves them in the display task set, and sends the display task set to the Composer process through cross-process communication.

[0092] The communication thread of the Composer process receives the display task set, and the communication thread of the Composer process sends the dimming task to the background thread of the Composer process for execution, and sends the display task to the main thread of the Composer process for execution. After sending the display task to the main thread of the Composer process for execution, it returns a resource unlock signal to the SurfaceFlinger process. After the main thread completes the display task of the third frame of image, it sends the third frame of image to the display driver.

[0093] 4. The Vsync-SF-4 signal arrives at the SurfaceFlinger process, and the synthesis thread of the SurfaceFlinger process starts to perform the synthesis operation for the fourth frame image to be displayed and generates a display task; the dimming process sends the dimming task to the SurfaceFlinger process through cross-process communication, and the communication thread packages the dimming task and the display task together and saves them in the display task set, and sends the display task set to the Composer process through cross-process communication.

[0094] The communication thread of the Composer process receives the display task set, and the communication thread of the Composer process sends the dimming task to the background thread of the Composer process for execution, and sends the display task to the main thread of the Composer process for execution. After sending the display task to the main thread of the Composer process for execution, it returns a resource unlock signal to the SurfaceFlinger process. After the main thread completes the display task of the fourth frame of image, it sends the fourth frame of image to the display driver.

[0095] 5. The Vsync-SF-5 signal arrives at the SurfaceFlinger process, and the SurfaceFlinger process continues to perform the synthesis operation.

[0096] Vsync-SF-n indicates that the subsequent Vsync-SF signal arrives at the SurfaceFlinger process, where n is a positive integer, indicating the nth Vsync-SF signal.

[0097] In order to intuitively demonstrate the image synthesis system using the method provided in the embodiment of the present application, its Composer process optimizes the smoothness of the display screen when executing a display task set including a dimming task. A scene diagram is shown in Figure 6. Figures 6 (a) to 6 (d) show a schematic diagram of a user manually adjusting the brightness of the display screen while watching a video, which causes the display screen to freeze. Combined with the interaction process shown in Figure 5 above, in the first display transmission cycle (after the Vsync-SF-1 signal reaches the SurfaceFlinger process and before the Vsync-SF-2 signal reaches the SurfaceFlinger process), the SurfaceFlinger process synthesizes the first frame of image to be displayed included in the video. The Composer process executes the display transmission task and returns the resource unlocking signal to the SurfaceFlinger process. After completing the display transmission task, the first frame of image to be displayed is sent to the display driver. As shown in Figure 6 (a), the display screen displays the first frame of image.

[0098] During the second display transmission cycle (after the Vsync-SF-2 signal arrives at the SurfaceFlinger process and before the Vsync-SF-3 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process synthesizes the second frame of image to be displayed included in the video. The Composer process performs the display transmission task and returns the resource unlock signal to the SurfaceFlinger process. After completing the display transmission task, the second frame of image to be displayed is sent to the display driver. As shown in Figure 6(b), the display screen displays the second frame of image.

[0099] During the third display cycle (after the Vsync-SF-3 signal arrives at the SurfaceFlinger process and before the Vsync-SF-4 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process synthesizes the third frame of the video to be displayed. The Composer process performs the display task and returns the resource unlock signal to the SurfaceFlinger process. After completing the display task, it sends the third frame of the video to the display driver. As shown in Figure 6(c), the display screen displays the third frame of the video.

[0100] During the fourth display transmission cycle (after the Vsync-SF-4 signal arrives at the SurfaceFlinger process and before the Vsync-SF-5 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process synthesizes the fourth frame of image to be displayed in the video. The Composer process performs the display transmission task and returns the resource unlock signal to the SurfaceFlinger process. After completing the display transmission task, the fourth frame of image to be displayed is sent to the display driver. As shown in Figure 6(d), the display screen displays the fourth frame of image.

[0101] In addition, as shown in another scenario diagram in Figure 7, Figures 7 (a) to 7 (d) show a schematic diagram of the display screen displaying a picture when the display screen is automatically dimming and the user clicks on the camera application. The specific process is similar to the example in Figure 6 above and will not be described separately here.

[0102] It can be seen that in the examples of Figures 6 and 7, the display screen displays the first to fourth frames of images in sequence during the process of continuously refreshing the screen. Since the display screen continuously displays adjacent frames, the change range of the picture is small, and the user feels that the picture is smooth when actually watching the display screen.

[0103] It can be known that the above examples are only some application scenarios of the embodiments of the present application. In actual applications, when the display screen displays a dynamic picture, if the display screen adjusts the brightness, the method provided by the embodiments of the present application is applicable. The changes in the picture displayed on the display screen include, for example: the user slides the screen, pulls down the display control center, plays a video, and other operations.

[0104] The following is a detailed description of a method for concurrently generating an image and dimming the screen provided by an embodiment of the present application. As shown in FIG8 , an interaction diagram of a method for concurrently generating an image and dimming the screen is shown. The interaction process includes: the synthesis thread of the compositor (SurfaceFlinger), the communication thread, the main thread of the hardware compositor (Composer process), the background thread, and the communication thread. Among them:

[0105] S801: A synthesis thread of a synthesizer receives a vertical synchronization signal.

[0106] Exemplarily, the vertical synchronization signal is the Vsync_SF signal in the above example.

[0107] S802: The synthesis thread of the synthesizer performs a synthesis operation.

[0108] The synthesis thread synthesizes the Surface processes rendered by each APP to determine the synthetic data such as the display layer, display position, and display range of the rendered Surface in the image displayed on the display screen.

[0109] S803: The synthesis thread sends the display task to the communication thread of the synthesizer.

[0110] The synthesis thread generates display tasks based on the rendered Surface and synthetic data.

[0111] S804: The communication thread of the synthesizer sends the display task set to the communication thread of the hardware synthesizer.

[0112] Exemplarily, the display task set includes display sending tasks and dimming tasks sent by the dimming process.

[0113] Here, the communication thread may not receive a dimming task in every display transmission cycle. This depends on whether the terminal device triggers a dimming request. Whether there is a dimming task does not affect the delivery of the display task set.

[0114] S805: The communication thread of the hardware synthesizer determines whether there is a dimming task in the display task set.

[0115] The communication thread of the hardware synthesizer determines whether there is a brightness parameter (nits) for adjusting the brightness of the display screen in the display task set by calling a function. If the returned numerical result is a specific value, it is confirmed that there is a dimming task. If the returned numerical result is empty, it is confirmed that there is no dimming task.

[0116] S806: The communication thread of the hardware synthesizer determines that a dimming task exists in the display task set, and sends the dimming task to the background thread.

[0117] Exemplarily, after setting a task type identifier for the dimming task, the dimming task and the task type identifier are stored in a task queue of a background thread to wait for execution.

[0118] S807. The communication thread of the hardware synthesizer sends the display sending task in the display task set to the main thread for execution.

[0119] S808: The communication thread of the hardware synthesizer sends a resource unlocking signal to the synthesis thread of the synthesizer.

[0120] S809. The main thread of the hardware synthesizer executes the display sending task in the display task set.

[0121] S810: The background thread extracts a task to be executed from the task queue and determines whether the task to be executed is a dimming task.

[0122] Exemplarily, the background thread cyclically extracts tasks to be executed in the task queue. Before executing the task to be executed, the task type identifier carried by the task to be executed is confirmed, and whether the task to be executed is a dimming task is determined through the task type identifier. For example, by setting an enumeration class, a numerical value is used to represent the task type of the task in the task queue. For example, the dimming task is set to 1, the query task is set to 2, and so on. When confirming whether it is a dimming task, the correspondence between the numerical value and the task type is used to determine whether the task to be executed is a dimming task.

[0123] S811. When the background thread determines that the task to be executed is a dimming task, it increases the priority of the background thread and executes the task.

[0124] It's well known that the higher the thread priority, the greater the probability that the tasks in that thread will be executed by the CPU. For example, in the computer programming language Java, thread priorities range from 1 to 10, with threads with priority 1 having a low probability of being executed by the CPU, and threads with priority 10 having a high probability of being executed by the CPU. Thread priorities are set by calling a function. In addition to Java, thread priorities can also be set using other computer programming languages, which are not listed here.

[0125] For example, if the dimming task is automatically triggered by the terminal device based on the change in ambient light intensity detected by the ambient light sensor, the priority can be set to level 7, for example. If the dimming task is triggered by the user manually controlling the screen, which has a higher real-time requirement, the priority can be set to level 9, for example. The specific setting can be flexibly adjusted according to actual needs and the overall scheduling of the system, and this application does not impose specific restrictions.

[0126] S812. After the background thread completes the dimming task, it restores the priority of the background thread and returns to S812 to continue extracting tasks to be executed in the task queue.

[0127] In one embodiment, after the dimming task is completed, the priority of the background thread is restored to the priority before the dimming task was executed by calling a function method. For example, before the dimming task is executed, the priority of the background thread is level 5. After the dimming task is completed, the priority of the background thread is restored to level 5. The process then returns to S810 to continue executing subsequent tasks in the task queue.

[0128] S813: When the background thread determines that the task to be executed is a non-dimming task type, it executes the task.

[0129] S814. After completing the task, the background thread returns to S810 to continue extracting the tasks to be executed in the task queue.

[0130] Exemplarily, a flowchart of a method for concurrently generating an image and dimming a screen as shown in FIG9 includes: a first process (Composer process) and a second process (SurfaceFlinger process).

[0131] After completing the synthesis operation of the image data rendered by each APP, the second process saves the display sending task in the display task set, and sends the display task set to the first process through cross-process communication (Binder). After receiving the display task set, the first process uses the function to call the target parameters (for example, brightness parameters) of each task in the display task set. If the returned numerical result is empty, it is determined that there is no dimming task in the display task set, and the display sending task in the display task set continues to be executed. After the display sending task in the display task set is executed, the resource unlocking signal is returned to the SurfaceFlinger process through cross-process communication (Binder). If the returned numerical result is a specific numerical value, it is determined that there is a dimming task in the display task set. After setting the task type identifier for the dimming task, the dimming task is sent to the task queue of the background thread of the first process to wait for execution, and then the function is continued to be called to traverse the display task set to determine whether there is a dimming task, until there is no dimming task in the display task set, and the display sending task of the display task set is continued to be executed.

[0132] The background thread loops through the tasks in the task queue. When executing the current pending task, it determines whether it is a dimming task based on the task type identifier. If the task type identifier indicates that the current pending task is a non-dimming task, it waits for the CPU to execute it according to the current thread's priority. After the CPU completes execution, the background thread continues to execute the next task. If the task type identifier indicates that the current pending task is a dimming task, the thread priority is increased and the CPU waits for execution. After the CPU executes the dimming task, the thread priority is restored. The background thread continues to execute the next task until the background thread is terminated or destroyed.

[0133] The terminal device in the embodiments of the present application can be a portable computer (such as a mobile phone), a tablet computer, a laptop computer, a personal computer (PC), a wearable terminal device (such as a smart watch), an augmented reality (AR) virtual reality (VR) device, a car computer, a smart TV, and other devices with a display screen that can adjust the display brightness. The following embodiments do not impose any special restrictions on the specific form of the terminal device.

[0134] For example, FIG10 shows a schematic diagram of the structure of a terminal device 200. As shown in FIG10 , it shows a schematic diagram of the structure of a terminal device 200. The terminal device 200 may include a processor 210, an external memory interface 220, an internal memory 221, an audio module 230, a display 240, a communication module 250, a power module 260, an input device 270, a sensor module 280, etc. The sensor module 280 may include an ambient light sensor, a touch sensor, etc.

[0135] It should be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the terminal device 200. In other embodiments of the present application, the terminal device 200 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0136] The processor 210 may include one or more processing units. For example, the processor 210 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0137] The controller may be the nerve center and command center of the terminal device 200. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0138] The application processor can run the operating system of the terminal device 200, which is used to manage the hardware and software resources of the terminal device 200. For example, it manages and configures memory, determines the priority of system resource supply and demand, controls input and output devices, operates the network, manages the file system, and manages drivers. The operating system also provides an interface for users to interact with the system. Various software, such as drivers and application programs, can be installed within the operating system.

[0139] Processor 210 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 210 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 210. If processor 210 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 210 latency, and thus improves system efficiency.

[0140] In some embodiments, the processor 210 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0141] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present application is merely an illustrative illustration and does not constitute a structural limitation on the terminal device 200. In other embodiments of the present application, the terminal device 100 may also adopt a different interface connection method from the above embodiment, or a combination of multiple interface connection methods.

[0142] The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 200. The external memory card communicates with the processor 210 via the external memory interface 220 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0143] The internal memory 221 can be used to store one or more computer programs, which include instructions. The processor 210 can execute the above instructions stored in the internal memory 221, so that the terminal device 200 executes the application operation method provided in some embodiments of the present application, as well as various applications and data management. The internal memory 221 may include a code storage area and a data storage area. Among them, the data storage area can store data created during the use of the terminal device 200, etc. In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage components, flash memory components, universal flash storage (UFS), etc. In some embodiments, the processor 210 can execute the application operation method provided in the embodiments of the present application, as well as other applications and data management by executing the instructions stored in the internal memory 221 and / or the instructions stored in the memory provided in the processor 210.

[0144] The terminal device 200 can implement audio functions through the audio module 230, speakers, microphones, and application processors. For example, music playback and recording. The audio module 230 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 230 can also be used to encode and decode audio signals. In some embodiments, the audio module 230 can be provided in the processor 210, or some functional modules of the audio module 230 can be provided in the processor 210.

[0145] A loudspeaker, also known as a "horn", is used to convert audio electrical signals into sound signals.

[0146] A microphone, also known as a "microphone" or "microphone," is used to convert sound signals into electrical signals. Users can put their mouths close to the microphone and speak, inputting the sound signal into the microphone.

[0147] The communication function of the terminal device 200 can be realized through antenna 1, antenna 2 and communication module 250, etc.

[0148] The communication module 250 can provide wireless communication solutions including cellular, Wi-Fi, Bluetooth (BT), wireless data transmission modules (e.g., 433MHz, 868MHz, 915MHz) applied to the terminal device 200. The communication module 250 can be one or more devices integrating at least one communication processing module. The communication module 250 receives electromagnetic waves via antenna 1 or antenna 2, filters and frequency modulates the electromagnetic wave signals, and sends the processed signals to the processor 210. The communication module 250 can also receive the signal to be transmitted from the processor 210, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through antenna 1 or antenna 2.

[0149] The terminal device 200 implements the display function through a GPU, a display screen 240, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 240 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs that execute program instructions to generate or change display information. In an embodiment of the present application, the GPU is used to render the graphics data (mainly model data and texture images) that needs to be rendered provided by the APP to obtain a rendered image frame.

[0150] The display screen 240 is used to display images, videos, etc. The display screen 240 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 200 may include 1 or N display screens 240, where N is a positive integer greater than 1. In the embodiment of the present application, the display screen 240 can be used to display a UI and receive user operations on the UI.

[0151] In some embodiments, the display screen 240 is provided with a pressure sensor, a touch sensor, etc. The pressure sensor is used to sense pressure signals and can convert pressure signals into electrical signals. When a touch operation is applied to the display screen 240, the terminal device 200 detects the intensity of the touch operation based on the pressure sensor. The terminal device 200 can also calculate the position of the touch based on the detection signal of the pressure sensor. The touch sensor, also known as a "touch panel", can form a touch screen, also known as a "touch screen", with the display screen 240. The touch sensor is used to detect touch operations applied thereto or nearby. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can also be provided through the display screen 240.

[0152] In the embodiments of the present application, the display screen can be any type of display screen, such as a flat screen, a curved screen, a foldable screen, a special-shaped screen, etc. It can be a touch screen or a non-touch screen, and the display screen needs to be able to adjust the brightness. If the display screen is a touch screen, the brightness adjustment function can be achieved by touching the dimming control on the screen, for example, by pulling down the control center and sliding up and down in the dimming control of the control center to adjust the display screen brightness. If the display screen is a non-touch screen, the brightness adjustment function can be adjusted by input devices such as a keyboard and a mouse.

[0153] The power module 260 can be used to supply power to various components included in the terminal device 200. In some embodiments, the power module 260 can be a battery, such as a rechargeable battery.

[0154] Input device 270 may include a keyboard, a mouse, etc. The keyboard is used to input English letters, numbers, punctuation marks, etc. into terminal device 200, thereby issuing commands to terminal device 200 and inputting data. The mouse is an indicator that indicates the vertical and horizontal coordinates of the terminal device 200 display system and is used to input commands to terminal device 200. Input device 270 may be connected to terminal device 200 via a wired connection, for example, via a GPIO interface, a USB interface, etc. Input device 270 may also be connected to terminal device 200 wirelessly, for example, via Bluetooth, infrared, etc.

[0155] The sensor module 280 of the terminal device 200 also includes an ambient light sensor, which is used to adjust the brightness of the display screen according to the ambient light. In an embodiment of the present application, when the ambient light sensor detects a change in the ambient light, the brightness of the display screen is automatically adjusted according to the intensity of the ambient light.

[0156] In the embodiment of the present application, the operating system of the terminal device 200 may be the same as that in the above example. System, or other display process is the same as or similar to the process in the above example system, System, etc.

[0157] An embodiment of the present application provides a terminal device, which may include: a memory and one or more processors, wherein the memory is configured to store computer program code, the computer program code comprising computer instructions. When the processor executes the computer instructions, the terminal device may perform the functions or steps performed by the mobile phone in the above-described method embodiment. The structure of the terminal device may refer to the structure of the terminal device shown in FIG10 .

[0158] An embodiment of the present application also provides a chip system (for example, a system on a chip (SoC)). As shown in Figure 11, the chip system includes at least one processor 1101 and at least one interface circuit 1102. The processor 1101 and the interface circuit 1102 can be interconnected via lines. For example, the interface circuit 1102 can be used to receive signals from other devices (for example, a memory of a terminal device). For another example, the interface circuit 1102 can be used to send signals to other devices (for example, the processor 1101 or a camera of a terminal device). Exemplarily, the interface circuit 1102 can read an instruction stored in the memory and send the instruction to the processor 1101. When the instruction is executed by the processor 1101, the terminal device can execute the various steps in the above embodiments. Of course, the chip system can also include other discrete components, which is not specifically limited in the embodiment of the present application.

[0159] An embodiment of the present application further provides a computer-readable storage medium comprising computer instructions. When the computer instructions are executed on a terminal device, the terminal device executes the various functions or steps executed by the terminal device 200 in the above-described method embodiment.

[0160] The present application also provides a computer program product, which, when executed on a terminal device, enables the computer to execute the functions or steps executed by the terminal device 200 in the above method embodiment. For example, the computer may be the above terminal device 200.

[0161] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0162] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0163] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0164] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0165] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0166] The above content is only a specific embodiment of this application, but the scope of protection of this application is not limited to this. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A processing method for concurrent generation of images and screen dimming, applied to a terminal device, the terminal device including a display screen, characterized in that, The method includes: In response to a first operation of the user, the terminal device plays a first dynamic image; During the process of playing the first dynamic image, the terminal device adjusts the brightness of the display screen; During the process of the terminal device playing the first dynamic image and adjusting the brightness of the display screen, the hardware synthesizer of the terminal device receives a display task set from the image synthesizer. A first thread in the hardware synthesizer executes the display task for sending the display task in the task set, and a second thread in the hardware synthesizer executes the dimming task in the display task set; wherein, the display task for sending is used to send a first image to the display screen for display, and the first image is a frame image in the first dynamic image; the dimming task is used to adjust the brightness of the display screen.

2. The method according to claim 1, characterized in that, The method further includes: The image synthesizer adds the display task for sending and the dimming task to the display task set in a first display sending cycle.

3. The method according to claim 2, wherein The first thread in the hardware synthesizer executing the display task for sending in the display task set includes: The first thread in the hardware synthesizer executes the display task for sending in the display task set in the first display sending cycle.

4. The method according to any one of claims 1 to 3, characterized in that, Before the second thread in the hardware synthesizer executes the dimming task in the display task set, the method further includes: The priority of the second thread being called by the processor is increased.

5. The method according to claim 4, wherein After the second thread in the hardware synthesizer executes the dimming task in the display task set, the method further includes: The priority of the second thread being called by the processor is decreased.

6. The method according to claim 5, wherein The priority of the second thread being called by the processor being increased includes: The priority of the second thread being called by the processor is adjusted from a first value to a second value; The priority of the second thread being called by the processor being decreased includes: The priority of the second thread being called by the processor is adjusted from the second value to the first value.

7. The method according to claim 2 or 3, characterized in that, After the first thread in the hardware synthesizer executes the display task for sending in the display task set, the method further includes: The hardware synthesizer returns a resource unlocking signal to the image synthesizer in the first display sending cycle.

8. The method according to any one of claims 1-7, characterized in that, After the hardware synthesizer of the terminal device receives a display task set from the image synthesizer, the method further includes: The terminal device determines that the display task set includes the dimming task, and distributes the dimming task to the second thread.

9. The method according to claim 8, wherein After the terminal device determines that the display task set includes the dimming task, the method further includes: The terminal device starts the second thread.

10. The method according to claim 9, wherein The second thread is a background thread.

11. A terminal device, characterized in that, The terminal device includes a memory, a display screen, and one or more processors; the memory, the display screen are coupled to the processor; computer program code is stored in the memory, and the computer program code includes computer instructions. When the computer instructions are executed by the processor, the terminal device executes the method according to any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, Including computer instructions, when the computer instructions run on the terminal device, the terminal device executes the method according to any one of claims 1-10.

Images