Processing method and terminal equipment for concurrence of image generation and screen dimming

Abstract

The invention discloses a concurrent image generation and screen dimming processing method and terminal equipment, and relates to the technical field of terminals, the method comprises the following steps: when the terminal equipment detects that a dynamic image is played on a display screen of the terminal equipment by an operation input by a user, if the terminal equipment adjusts the brightness of the display screen at the same time, the brightness of the display screen is adjusted; if a display task set received by a hardware synthesizer of the terminal equipment from an image synthesizer comprises a dimming task and a display sending task, the hardware synthesizer sends the display sending task to a first thread to be executed and sends the dimming task to a second thread to be executed, so that the first thread and the second thread execute respective tasks respectively, and the display task set is displayed in a display mode. And after the first thread executes the display sending task, sending the image to a display screen for displaying. The problem that when the terminal equipment detects that the operation input by the user plays the dynamic image on the display screen of the terminal equipment, if the terminal equipment adjusts the brightness of the display screen at the same time in the playing process of the dynamic image, the display image is not smooth is solved.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and in particular, to a processing method and a terminal device for concurrent generation of images and screen dimming. Background Art

[0002] With the development of terminal technologies, the interaction between various terminals (such as mobile phones) and users has become increasingly rich. Among them, most interactions come from the display screen on the terminal, and the images displayed on the display screen usually go through processes such as drawing, rendering, and composition.

[0003] Among them, the application process of the terminal device is responsible for drawing and rendering various visual elements in the display screen, and the image synthesis system of the terminal device is responsible for synthesizing different layers where various visual elements in the display screen are located and sending them for display. The synthesis and display process is carried out frame by frame. After the image synthesis system synthesizes a frame of image, it sends the frame of image to the display screen for display, and then performs the synthesis operation of the next frame of image. If the frame of image fails to complete the display within the specified time, the synthesis of the next frame of image cannot start, resulting in image frame loss.

[0004] In some cases, when the image synthesis system sends an image for display, it will perform other tasks, resulting in the failure to complete the display within the specified time, and further causing the inability to synthesize the next frame of image, resulting in the problem of screen jitter in the display screen. Summary of the Invention

[0005] Embodiments of this application provide a processing method and a terminal device for concurrent generation of images and screen dimming. During the process of the terminal device playing a first dynamic image and adjusting the brightness of the display screen, the problem of frame loss in the display screen of the display is solved.

[0006] To achieve the above objective, the embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, a processing method for concurrent generation of images and screen dimming is provided, and the method includes:

[0008] When the terminal device plays a dynamic image on the display screen of the terminal device after detecting an operation input by the user, if the terminal device is simultaneously adjusting the brightness of the display screen, the display task set received by the hardware synthesizer of the terminal device from the image synthesizer will include a dimming task and a display task. The hardware synthesizer sends the display task to a first thread for execution, and sends the dimming task to a second thread for execution. In this way, the first thread and the second thread execute their respective tasks. After the first thread finishes executing the display task, the image is sent to the display screen for display.

[0009] In this implementation, when the display task set includes a display task and a dimming task, when the hardware synthesizer receives the display task set, it sends the dimming task and the display task in the display task set into different threads for execution. Since each thread runs independently, after the first thread finishes executing the display task, it does not have to wait for the second thread to finish executing the dimming task to send the image to the display screen for display, and can promptly notify the image synthesizer to perform the synthesis operation of the next frame of the image. This avoids the problem of frame loss on the display screen caused by failure to complete the display within the specified time due to execution timeout when the hardware synthesizer simultaneously executes the display task and the dimming task, and solves the problem of display screen jitter that occurs when the terminal device detects a user input operation to play a dynamic image on the display screen of the terminal device and the terminal device simultaneously adjusts the brightness of the display screen during the playback of the dynamic image.

[0010] In a possible implementation, the image synthesizer may include a SurfaceFlinger process, and the hardware synthesizer may include a Composer process. The SurfaceFlinger process is used to synthesize the content rendered by each application program that is about to be displayed on the display screen. Among them, each application program renders on a Surface, and a layer corresponds to the Surface. The SurfaceFlinger process performs a synthesis operation on the layers of the rendered Surface. In addition, the SurfaceFlinger process generates a display task according to the synthesis operation and saves the display task in the display task set.

[0011] In a possible implementation, the SurfaceFlinger process may also receive a dimming task sent by the dimming process and save the dimming task in the display task set.

[0012] In a possible implementation, the terminal device adjusts the brightness of the display screen, including the user manually triggering the adjustment of the display screen brightness and the terminal device automatically triggering the adjustment of the display screen brightness.

[0013] Among them, the user actively adjusting the brightness of the display screen may be that the user opens the control center of the terminal device through a pull-down operation on the display screen and adjusts the brightness of the display screen by sliding the dimming control in the control center. The terminal device automatically triggering the adjustment of the display screen brightness may be that the ambient light sensor deployed on the terminal device detects a change in ambient light and adjusts the brightness of the display screen.

[0014] In a possible implementation, the user's operations include: starting an application program, swiping the display page, playing a video, etc.

[0015] In a possible implementation of the first aspect, the image synthesizer adds the display task and the dimming task to the display task set in the first display cycle.

[0016] In a possible implementation, the first display sending period is represented as the time from when the current Vsync - SF signal arrives at the image synthesizer until the next Vsync - SF signal arrives at the image synthesizer. During this time, the image synthesizer adds the display sending task and the dimming task to the display task set.

[0017] In this implementation, the image synthesizer adding the display sending task and the dimming task to the display task set within one display sending period enables the hardware synthesizer to execute the display sending task in the display task set during this display sending period.

[0018] In a possible implementation of the first aspect, the first thread in the hardware synthesizer of the terminal device executes the display sending task in the display task set during the first display sending period.

[0019] In a possible implementation, the first thread can be the main thread of the hardware synthesizer.

[0020] In a possible implementation of the first aspect, before the second thread of the hardware synthesizer executes the dimming task, it raises the priority of the thread so that the second thread can be preferentially called by the processor. In this way, the terminal device can reduce the time to respond to the brightness of the display screen and enhance the user experience.

[0021] In a possible implementation, the second thread includes a background thread. The background thread includes a task queue, and the dimming task is saved in the task queue. The background thread loops to execute the tasks in the task queue.

[0022] In a possible implementation of the first aspect, after the second thread of the hardware synthesizer executes the dimming task, it lowers the priority of the thread.

[0023] In a possible implementation of the first aspect, raising the priority of the second thread being called by the processor includes: the priority of the second thread being called by the processor is adjusted from a first value to a second value; lowering the priority of the second thread being called by the processor includes: the priority of the second thread being called by the processor is adjusted from the second value to the first value.

[0024] In a possible implementation, the priority of the second thread is adjusted to the priority before executing the dimming task.

[0025] In a possible implementation, the priority of the second thread is adjusted to the default priority of the thread.

[0026] In a possible implementation of the first aspect, after the first thread in the hardware synthesizer executes the display sending task in the display task set, the hardware synthesizer returns a resource unlocking signal to the image synthesizer during the first display sending period.

[0027] In a possible implementation, the resource unlocking signal is used to notify the image synthesizer to synthesize the next frame of image.

[0028] In this implementation, the hardware synthesizer returns the resource unlocking signal to the image synthesizer within a display cycle, which enables the image synthesizer to perform image synthesis operations when the next display cycle arrives, thereby avoiding frame drops.

[0029] In a possible implementation of the first aspect, after the hardware synthesizer receives the display task set from the image synthesizer, the terminal device determines that the display task set includes the dimming task and distributes the dimming task to the second thread.

[0030] In a possible implementation, the dimming parameters in the display task set can be called using a function call method. If the return result is a specific value, it can be determined that there are dimming parameters in the display task. If the return result is empty, it is determined that there is no dimming task in the display task set.

[0031] In this implementation, the communication thread in the hardware synthesizer is used to determine whether there is a dimming task in the display task set and distribute the dimming task.

[0032] In a possible implementation of the first aspect, after the terminal device determines that the display task set includes a dimming task, the terminal device creates a second thread.

[0033] In a possible implementation of the first aspect, the second thread is a background thread.

[0034] In a second aspect, a terminal device is provided, including: a processor and a memory; the memory is used to store computer execution instructions. When the terminal device runs, the processor executes the computer execution instructions stored in the memory, so that the terminal device executes the method described in any one of the above first aspects.

[0035] In a third aspect, a chip system is provided, including a processor for supporting the terminal device to implement the functions involved in the above first aspect. In a possible design, the device further includes a memory for storing necessary program instructions and data of the terminal device. When the device is a chip system, it can be composed of chips or include chips and other discrete devices.

[0036] In a fourth aspect, a computer-readable storage medium is provided, in which instructions are stored. When it runs on a computer, the computer can execute the method described in any one of the above first aspects.

[0037] Among them, for the technical effects brought by any one of the design manners in the second to fourth aspects, reference may be made to the technical effects brought by different design manners in the first aspect, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of a display interface of a terminal device;

[0039] Figure 2 It is a schematic diagram of an image transmission and display process;

[0040] Figure 3 It is a schematic diagram of a scenario applicable to the processing method for concurrent generation of an image and screen dimming provided by an embodiment of the present application;

[0041] Figure 4 It is another schematic diagram of a scenario applicable to the processing method for concurrent generation of an image and screen dimming provided by an embodiment of the present application;

[0042] Figure 5 It is a schematic flowchart of the processing method for concurrent generation of an image and screen dimming provided by an embodiment of the present application;

[0043] Figure 6 It is a schematic diagram of a scenario applicable to the processing method for concurrent generation of an image and screen dimming provided by an embodiment of the present application;

[0044] Figure 7 It is another schematic diagram of a scenario applicable to the processing method for concurrent generation of an image and screen dimming provided by an embodiment of the present application;

[0045] Figure 8 It is an interaction diagram of the processing method for concurrent generation of an image and screen dimming provided by an embodiment of the present application;

[0046] Figure 9 It is a flowchart of the processing method for concurrent generation of an image and screen dimming provided by an embodiment of the present application;

[0047] Figure 10 It is a schematic diagram of the structure of a terminal device provided by an embodiment of the present application;

[0048] Figure 11 It is a schematic diagram of the structure of a chip system provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] In the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular forms "a", "the", "above-mentioned", "this" and "this one" are also intended to include expressions such as "one or more", unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of the present application, "at least one" and "one or more" mean one or more than two (including two). The term "and / or" is used to describe the association relationship of associated objects and indicates that three relationships may exist; for example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B may be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0050] The reference to "one embodiment" or "some embodiments" etc. described in this specification means that a specific feature, structure or characteristic described in connection with that embodiment is included in one or more embodiments of the present application. Thus, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprise", "include", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise stated. "First" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features.

[0051] In the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0052] The operating system (OS) is a set of interrelated system software programs that manage and control the operations, applications and running of a terminal device, utilize hardware and software resources, and provide public services to organize user interactions. It is a bridge connecting the hardware and software of the terminal device.

[0053] The image composition system (also known as the compositor) is an important part of the operating system. It is responsible for managing and scheduling hardware devices such as the central processing unit (CPU), graphics processing unit (GPU), and display driver of the terminal device, as well as software resources such as the graphics library and media library. It composes the surfaces (Surfaces) rendered by different applications (APPs) and displays them on the display screen.

[0054] As Figure 1 shown in the schematic diagram of the display interface of a terminal device, in the picture displayed on the display screen of the terminal device, it includes visual elements such as icons, wallpapers, and widgets. Among them, each visual element such as an icon, wallpaper, and widget comes from the corresponding application rendering on the Surface, and is composed by the image composition system in a frame of image and displayed on the display screen.

[0055] Exemplarily, if you want to display the image shown on the display screen of the above Figure 1 terminal device, the composition and display process includes: the APP process that needs to display visual elements on the display screen requests a Vsync signal from the Vsync thread in the image composition process (SurfaceFlinger). When the Vsync signal arrives at the APP process, the APP process renders on the Surface to obtain the rendered Surface. And apply for a buffer in the buffer queue, store the rendered Surface in the buffer. When the Vsync signal arrives at the SurfaceFlinger process, the SurfaceFlinger process takes out the buffer from the buffer queue, reads the rendered Surface on the buffer, and performs a composition operation according to the layer where the rendered Surface is located to determine the composition data of the rendered Surface (the display position, display range, display layer, transparency, etc. in the picture displayed on the display screen). Further, the composition thread is also responsible for sending the rendered Surface and the composition data to the hardware compositor (HWComposer, HWC), which is simply called the Composer process in the embodiments of this application. The Composer process provides hardware composition support for the SurfaceFlinger process and sends the composed image to the display screen for display through the display driver.

[0056] Among them, the Vsync signal is used to synchronize the display period with the refresh period (frame rate) of the display screen. The display period in the embodiments of this application represents the time interval between two consecutive Vsync signals arriving at the SurfaceFlinger process. The period of the Vsync signal generated by the Vsync thread is related to the frame rate of the display screen refresh. The frame rate refers to the number of frames of the displayed image refreshed in 1 second, and can also be understood as the number of times the graphics processor in the terminal device refreshes the screen per second. A high frame rate can obtain a smoother and more realistic picture. The more frames per second, the smoother the displayed picture. Exemplarily, a frame rate of 60 Hertz (HZ) means that 60 frames of images are refreshed in 1 second, that is, one frame is refreshed every 16.6 milliseconds. Correspondingly, the period of the Vsync signal generated by the Vsync thread is 16.6 milliseconds. Exemplarily, a frame rate of 90 HZ means that 90 frames of images are refreshed in 1 second, that is, one frame is refreshed every 11.1 milliseconds. Correspondingly, the period of the Vsync signal generated by the Vsync thread is 11.1 milliseconds. Currently, there are also specifications such as 120 HZ and 144 HZ for the refresh frame rate of the display screen, which will not be introduced one by one here.

[0057] The Vsync signal generated by the Vsync thread includes the Vsync_APP signal, the Vsync_SF signal, and the HW_Vsync signal. The Vsync thread generates the Vsync_APP signal and sends the Vsync_APP signal to the APP process, and the APP process performs rendering on the Surface. The Vsync thread generates the Vsync_SF signal and sends the Vsync_SF signal to the SurfaceFlinger process. When the Vsync_SF signal arrives, the SurfaceFlinger process obtains the rendered Surface and performs the composition operation of the rendered Surface. The Vsync thread generates the HW_Vsync signal and sends the HW_Vsync signal to the display driver of the terminal device. When the HW_Vsync signal arrives, the display driver refreshes the displayed image.

[0058] When the Vsync_SF signal arrives at the SurfaceFlinger process, the SurfaceFlinger process and the Composer process execute the image composition and display process.

[0059] In an interaction example, the Vsync thread periodically sends a Vsync-SF signal to the SurfaceFlinger process. After receiving the Vsync-SF signal, the SurfaceFlinger process first retrieves a buffer from the data cache queue, reads the rendered Surface stored on the buffer, and then performs a composition operation on the rendered Surface to determine composition parameters such as the layer position of these rendered Surfaces in multiple layers included in the display screen, as well as the display range and display position in the corresponding layer. Based on the layer position of each rendered Surface in the display screen and composition parameters such as the display range and display position in the corresponding layer, a display task is generated and saved in the Display Command set, and finally sent to the Composer process through inter-process communication. After receiving the Display Command set, the communication thread of the Composer process extracts the display task in the Display Command set and sends the display task to the main thread of the Composer process for execution. When sending the display task to the main thread of the Composer process for execution, the Composer process releases the Fence resource and returns a resource unlock signal to the SurfaceFlinger process, which is used to notify the SurfaceFlinger process that it can perform the composition operation of the next frame of image. After the main thread of the Composer process finishes executing the display task, the composed image is sent for display through the display driver.

[0060] The SurfaceFlinger process waits for the resource unlock signal returned by the Composer process until it receives the resource unlock signal. After receiving the resource unlock signal, when the next Vsync-SF signal arrives, the SurfaceFlinger process performs the composition operation of the next frame of image to be displayed. If, when the next Vsync-SF signal arrives at the SurfaceFlinger process, the SurfaceFlinger process has not received the resource unlock signal returned by the Composer process, the SurfaceFlinger process does not perform the composition operation of the next frame of image to be displayed.

[0061] In some embodiments, a terminal device with the function of adjusting the brightness of a display screen can detect changes in ambient light through a set ambient light sensor, and then control the brightness of the display screen according to the ambient light intensity. For example, when the user is in an outdoor environment with sufficient ambient light, the mobile phone controls the brightness of the display screen to be increased so that the user can clearly see the content displayed on the display screen. When the user is in an indoor environment with weak light intensity, the mobile phone controls the brightness of the display screen to be decreased to avoid the display screen being too bright and increasing the user's eye fatigue. In addition, the brightness of the display screen can also be adjusted according to the user's manual trigger. For example, the user can adjust the brightness of the display screen by swiping up and down or left and right in the dimming control in the drop-down control center.

[0062] In a possible application scenario, when the user uses the terminal device to play a video, start an application, slide and switch the display screen, etc., and the terminal device receives a request from the user to adjust the brightness of the display screen, or the terminal device automatically adjusts the brightness of the display screen according to changes in ambient light, the dimming process (LightsService) responsible for responding to the request to adjust the brightness of the display screen generates a dimming task and sends it to the SurfaceFlinger process. The APP responsible for displaying the screen on the display screen stores the rendered Surface in the cache queue. The SurfaceFlinger process takes out the rendered Surface from the cache queue for synthesis operations, generates a display task according to the synthesis data, and saves the display task and the dimming task in a display task set, and sends them to the Composer process through inter-process communication. After the communication thread of the Composer process determines that there is a dimming task in the display task set according to the system's task execution logic, it preferentially executes the dimming task. After the dimming task is executed, it sends the display task to the main thread for execution. Then, after the communication thread of the Composer process sends the display task to the main thread for execution, it returns the resource unlock signal to the SurfaceFlinger process. Therefore, if the execution time of the communication thread for executing the dimming task is too long, it will cause the resource unlock signal not to be returned before the next Vsync-SF signal reaches the SurfaceFlinger process, resulting in the SurfaceFlinger process not performing the synthesis operation of the next frame of the image, causing frame drops and lags in the display screen.

[0063] Such as Figure 2As shown in the figure, it is a schematic diagram of an image transmission and display process. In a detailed analysis of the current image synthesis system, when the Composer process executes a display task set including a dimming task, an execution timeout occurs (all display tasks cannot be completed within a transmission and display cycle), resulting in frame drops on the display screen. It should be noted that the following Vsync-SF-1 to Vsync-SF-5 signals represent that the Vsync thread periodically and continuously sends Vsync-SF signals to the SurfaceFlinger process. The following examples will not be limited one by one.

[0064] 1. When the Vsync-SF-1 signal arrives at the SurfaceFlinger process, the composition thread of the SurfaceFlinger process starts to execute the composition operation for the first frame of the image to be displayed and generates a display task; the dimming process sends the dimming task into the SurfaceFlinger process through cross-process communication. The communication thread packages the dimming task and the display task together and stores them in the display task set, and sends the display task set into the Composer process through cross-process communication.

[0065] The communication thread of the Composer process receives the display task set, preferentially executes the dimming task in the display task set. After the dimming task is completed, it sends the display task into the main thread for execution. After the communication thread puts the display task into the main thread for execution, it returns a resource unlock signal to the SurfaceFlinger process, indicating that the SurfaceFlinger process can perform the composition operation for the next frame of the image. After the main thread of the Composer process finishes executing the display task, it sends the first frame of the image to be displayed to the display driver.

[0066] 2. When the Vsync-SF-2 signal arrives at the SurfaceFlinger process, the SurfaceFlinger process has received the resource unlock signal returned by the Composer process. The composition thread executes the composition operation for the second frame of the image to be displayed and sends the display task corresponding to the composition operation into the communication thread. The dimming process sends the dimming task into the SurfaceFlinger process through cross-process communication. The communication thread packages the dimming task and the display task together and stores them in the display task set, and sends the display task set into the Composer process through cross-process communication.

[0067] The communication thread of the Composer process receives the display task set and preferentially executes the dimming task in the display task set. At this time, the dimming task execution times out. The reason for this execution timeout may be that when interacting with the hardware (backlight driver) during the execution of the dimming task, the hardware is occupied by other processes and needs to wait for other processes to release the occupation before the dimming task can continue to be executed.

[0068] At this time, the first frame of image is still displayed on the display screen.

[0069] 3. The Vsync-SF-3 signal arrives at the SurfaceFlinger process. Since the SurfaceFlinger process has not received the resource unlock signal returned by the Composer process at this time, the composition thread of the SurfaceFlinger process does not perform the composition operation.

[0070] The communication thread of the Composer process sends the display task into the main thread for execution. After the communication thread puts the display task into the main thread for execution, it returns the resource unlock signal to the SurfaceFlinger process. The main thread of the Composer process finishes executing the display task and sends the second frame of image to be displayed to the display driver.

[0071] 4. The Vsync-SF-4 signal arrives at the SurfaceFlinger process. The composition thread of the SurfaceFlinger process performs the composition operation for the fourth frame of image to be displayed and generates a display task. The dimming process sends the dimming task into the SurfaceFlinger process through cross-process communication. The communication thread packages the dimming task and the display task together and stores them in the display task set, and sends the display task set into the Composer process through cross-process communication.

[0072] The communication thread of the Composer process sends the display task into the main thread for execution. After the communication thread puts the display task into the main thread for execution, it returns the resource unlock signal to the SurfaceFlinger process. The main thread of the Composer process executes the display task and sends the fourth frame of image to be displayed to the display driver.

[0073] 5. The Vsync-SF-5 signal arrives at the SurfaceFlinger process, and the SurfaceFlinger process continues to perform the composition operation.

[0074] Vsync-SF-n represents that subsequent Vsync-SF signals arrive at the SurfaceFlinger process, where n is a positive integer representing the nth Vsync-SF signal.

[0075] To visually show the actual impact caused by the Composer process timing out when executing the display task set including the dimming task in the current image composition system, such as Figure 3 shown in a schematic diagram of a scenario, in Figure 3 (a) to Figure 3Figure (d) shows a schematic diagram of the screen freeze that occurs when the user manually adjusts the display brightness while watching a video.

[0076] Combined with the above Figure 2 In the first display cycle (after the Vsync-SF-1 signal arrives at the SurfaceFlinger process and before the Vsync-SF-2 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process composes the first frame image to be displayed included in the video. The Composer process executes the display task, returns the resource unlock signal to the SurfaceFlinger process, and sends the first frame image to be displayed to the display driver after completing the display task. As Figure 3 shown in Figure (a), the display screen displays the first frame image.

[0077] In the second display cycle (after the Vsync-SF-2 signal arrives at the SurfaceFlinger process and before the Vsync-SF-3 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process performs the composition operation of the second frame image to be displayed. The Composer process times out while executing the display task and does not return the resource unlock signal. As Figure 3 shown in Figure (b), the display screen displays the first frame image.

[0078] In the third display cycle (after the Vsync-SF-3 signal arrives at the SurfaceFlinger process and before the Vsync-SF-4 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process does not perform the composition operation of the third frame image to be displayed. The Composer process executes the display task of the second frame image, returns the resource unlock signal to the SurfaceFlinger process, and sends the second frame image to the display driver after completing the display task. As Figure 3 shown in Figure (c), the display screen displays the second frame image.

[0079] In the fourth display cycle (after the Vsync-SF-4 signal arrives at the SurfaceFlinger process and before the Vsync-SF-5 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process performs the composition operation of the fourth frame image to be displayed. The Composer process executes the display task of the second frame image, returns the resource unlock signal to the SurfaceFlinger process, and sends the fourth frame image to the display driver after completing the display task. As Figure 3 shown in Figure (d), the display screen displays the fourth frame image.

[0080] In addition, as shown in Figure 4 another schematic diagram of a scenario, during the automatic dimming of the display screen as shown in Figure 4 (a) to Figure 4 (d) of Figure 3 shows a schematic diagram of the display screen freezing when the user clicks on the camera application during the automatic dimming process. It can be known that when synthesizing the image, the Surfaces of all APPs rendered on the entire display screen are synthesized. The specific process is similar to the above Figure 3 example and will not be described separately here. It can be seen that in the above Figure 4 example scenarios, two identical images (the first frame image) are displayed on the display screen, and then the second frame image and the fourth frame image are displayed. Due to the lack of the third frame image, when changing from the second frame image to the fourth frame image, the change amplitude of the screen is relatively large. When the user views the display screen, the screen will feel stuck and not smooth, affecting the user's viewing experience.

[0081] In addition, it should be noted that the above examples are only used to illustrate that in some usage scenarios, due to problems in the display sending process of the image synthesis system, the display screen freezes. In actual applications, there are many usage scenarios similar to the above examples, and the embodiments of the present application will not list them one by one. For the convenience of description, the embodiments of the present application are all illustrated with four display sending cycles. In fact, the display sending cycles are continuous, and the image synthesis system continuously performs synthesis and display sending operations. In actual applications, there may also be problems where multiple display sending cycles continuously experience execution timeouts, resulting in serious screen freezing and affecting the use experience.

[0082] The embodiments of the present application provide a processing method for concurrent generation of images and screen dimming. The Composer process obtains a display task set in which the SurfaceFlinger process packages the dimming task and the display sending task together, and sends the dimming task in the display task set to a thread different from the display sending task (for example, a background thread) for execution. The two threads independently execute their respective tasks. The main thread executes the display sending task, and the background thread executes the dimming task. After the main thread finishes executing the display sending task, the Composer process does not have to wait for the background thread to execute the dimming task, but returns a resource unlocking signal to the SurfaceFlinger process. After receiving the resource unlocking signal, the SurfaceFlinger process prepares for the synthesis operation of the next frame of the image to be displayed. In this way, since the display sending task and the dimming task are executed in different threads respectively, and the Composer process does not have to wait for the background thread to execute the dimming task after the main thread finishes executing the display sending task, the resource unlocking signal can be returned within one display sending cycle, solving the above problem of frame loss.

[0083] Further, the terminal device to which the embodiments of the present application are applied can improve the stability and smoothness of the display screen when the display screen performs a dimming operation during the process of screen change (such as application startup, page switching, video playback, etc.).

[0084] Further, the embodiments of the present application optimize the processing mechanism of the display sending process in the Composer process, avoiding modifying the task packaging mechanism of the system's native SurfaceFlinger process (saving the dimming task and the display sending task in a display task set), and having the advantages of simple implementation and easy operation.

[0085] As Figure 5 shown in the flowchart of a processing method applicable to the concurrent generation of images and screen dimming provided by the embodiments of the present application, a detailed analysis is made of the interaction process of the image synthesis system applying the processing method for concurrent generation of images and screen dimming provided by the embodiments of the present application when the Composer process executes a display task set including a dimming task.

[0086] 1. When the Vsync-SF-1 signal arrives at the SurfaceFlinger process, the synthesis thread of the SurfaceFlinger process starts to execute the synthesis operation for the first frame of the image to be displayed and generates a display sending task; the dimming process sends the dimming task into the SurfaceFlinger process through cross-process communication, and the communication thread packages the dimming task and the display sending task together and saves them in the display task set, and sends the display task set into the Composer process through cross-process communication.

[0087] The communication thread of the Composer process receives the display task set. The communication thread of the Composer process sends the dimming task into the background thread of the Composer process for execution, sends the display sending task into the main thread of the Composer process for execution, and after sending the display sending task into the main thread of the Composer process, returns a resource unlocking signal to the SurfaceFlinger process. After the main thread finishes executing the display sending task of the first frame of the image, it sends the first frame of the image into the display driver.

[0088] 2. When the Vsync-SF-2 signal arrives at the SurfaceFlinger process, the synthesis thread of the SurfaceFlinger process starts to execute the synthesis operation for the second frame of the image to be displayed and generates a display sending task; the dimming process sends the dimming task into the SurfaceFlinger process through cross-process communication, and the communication thread packages the dimming task and the display sending task together and saves them in the display task set, and sends the display task set into the Composer process through cross-process communication.

[0089] The communication thread of the Composer process receives the display task set. The communication thread of the Composer process sends the dimming task to the background thread of the Composer process for execution, sends the display task to the main thread of the Composer process for execution, and after sending the display task to the main thread of the Composer process, returns a resource unlock signal to the SurfaceFlinger process. After the main thread finishes executing the display task of the second frame image, it sends the second frame image to the display driver.

[0090] 3. The Vsync-SF-3 signal arrives at the SurfaceFlinger process. The composition thread of the SurfaceFlinger process starts to execute the composition operation for the third frame image to be displayed and generates a display task. The dimming process sends the dimming task to the SurfaceFlinger process through inter-process communication. The communication thread packs the dimming task and the display task together and stores them in the display task set, and sends the display task set to the Composer process through inter-process communication.

[0091] The communication thread of the Composer process receives the display task set. The communication thread of the Composer process sends the dimming task to the background thread of the Composer process for execution, sends the display task to the main thread of the Composer process for execution, and after sending the display task to the main thread of the Composer process, returns a resource unlock signal to the SurfaceFlinger process. After the main thread finishes executing the display task of the third frame image, it sends the third frame image to the display driver.

[0092] 4. The Vsync-SF-4 signal arrives at the SurfaceFlinger process. The composition thread of the SurfaceFlinger process starts to execute the composition operation for the fourth frame image to be displayed and generates a display task. The dimming process sends the dimming task to the SurfaceFlinger process through inter-process communication. The communication thread packs the dimming task and the display task together and stores them in the display task set, and sends the display task set to the Composer process through inter-process communication.

[0093] The communication thread of the Composer process receives the display task set. The communication thread of the Composer process sends the dimming task to the background thread of the Composer process for execution, sends the display task to the main thread of the Composer process for execution, and after sending the display task to the main thread of the Composer process, returns a resource unlock signal to the SurfaceFlinger process. After the main thread finishes executing the display task of the fourth frame image, it sends the fourth frame image to the display driver.

[0094] 5. The Vsync-SF-5 signal arrives at the SurfaceFlinger process, and the SurfaceFlinger process continues to perform the composition operation.

[0095] Vsync-SF-n indicates that the subsequent Vsync-SF signal arrives at the SurfaceFlinger process, where n is a positive integer representing the nth Vsync-SF signal.

[0096] To intuitively show the optimization of the display frame smoothness by the Composer process when the image composition system applying the method provided in the embodiments of the present application executes a display task set including a dimming task, such as Figure 6 shown in a schematic diagram of a scenario. In Figure 6 (a) to Figure 6 (d) shown are schematic diagrams of the display screen image freezing when the user manually adjusts the display screen brightness while watching a video. Combining the above Figure 5 shown interaction process, in the first display cycle (after the Vsync-SF-1 signal arrives at the SurfaceFlinger process and before the Vsync-SF-2 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process composes the first frame of the image to be displayed included in the video. The Composer process executes the display task, returns the resource unlock signal to the SurfaceFlinger process, and after completing the display task, sends the first frame of the image to be displayed to the display driver. As Figure 6 (a) shown, the display screen displays the first frame of the image.

[0097] In the second display cycle (after the Vsync-SF-2 signal arrives at the SurfaceFlinger process and before the Vsync-SF-3 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process composes the second frame of the image to be displayed included in the video. The Composer process executes the display task, returns the resource unlock signal to the SurfaceFlinger process, and after completing the display task, sends the second frame of the image to be displayed to the display driver. As Figure 6 (b) shown, the display screen displays the second frame of the image.

[0098] In the third display cycle (after the Vsync-SF-3 signal arrives at the SurfaceFlinger process and before the Vsync-SF-4 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process composes the third frame image to be displayed included in the video. The Composer process executes the display task, returns the resource unlock signal to the SurfaceFlinger process, and after completing the display task, sends the third frame image to be displayed to the display driver. As shown in (c) of Figure 6 , the display screen displays the third frame image.

[0099] In the fourth display cycle (after the Vsync-SF-4 signal arrives at the SurfaceFlinger process and before the Vsync-SF-5 signal arrives at the SurfaceFlinger process), the SurfaceFlinger process composes the fourth frame image to be displayed included in the video. The Composer process executes the display task, returns the resource unlock signal to the SurfaceFlinger process, and after completing the display task, sends the fourth frame image to be displayed to the display driver. As shown in (d) of Figure 6 , the display screen displays the fourth frame image.

[0100] In addition, as shown in another schematic diagram of a scenario in Figure 7 , during the automatic dimming process of the display screen, when the user clicks on the camera application, the schematic diagram of the display screen showing the picture is as shown in (a) to (d) of Figure 7 . The specific process is similar to the above Figure 7 example and will not be described separately here. Figure 6

[0101] It can be seen that in the examples of Figure 6 and Figure 7 , during the continuous refreshing process of the display screen, the first frame to the fourth frame images are sequentially displayed on the display screen. Since the display screen continuously displays adjacent frame pictures, the change amplitude of the pictures is small. When the user actually views the display screen, the pictures feel smooth.

[0102] It can be known that the above examples are only some application scenarios of the embodiments of the present application. In actual applications, when the display screen displays dynamic pictures, if the display screen adjusts the brightness, the method provided by the embodiments of the present application is applicable. The operations that cause the pictures displayed on the display screen to change include, for example: the user swiping the screen, pulling down the display control center, playing a video, etc.

[0103] Figure 8 The following will describe in detail a method for processing the concurrency of generating images and screen dimming provided by the embodiments of the present application, as shown in Figure 8Interaction diagram of a processing method for concurrent generation of images and screen dimming. The interaction process includes: the composition thread, communication thread of the synthesizer (SurfaceFlinger), the main thread, background thread, and communication thread of the hardware synthesizer (Composer process). Among them:

[0104] S801. The composition thread of the synthesizer receives a vertical synchronization signal.

[0105] Exemplarily, the vertical synchronization signal is the Vsync_SF signal in the above example.

[0106] S802. The composition thread of the synthesizer performs a composition operation.

[0107] The composition thread performs a composition operation on the Surface processes rendered by each APP, and determines composition data such as the display layer, display position, and display range of the rendered Surface in the image displayed on the display screen.

[0108] S803. The composition thread sends the display task to the communication thread of the synthesizer.

[0109] The composition thread generates a display task based on the rendered Surface and the composition data.

[0110] S804. The communication thread of the synthesizer sends the display task set to the communication thread of the hardware synthesizer.

[0111] Exemplarily, the display task set includes the display task and the dimming task sent by the dimming process.

[0112] Here, the communication thread does not receive the dimming task in every display cycle, which depends on whether the terminal device triggers a dimming request. Whether there is a dimming task does not affect the distribution of the display task set.

[0113] S805. The communication thread of the hardware synthesizer determines whether there is a dimming task in the display task set.

[0114] The communication thread of the hardware synthesizer determines whether there is a brightness parameter (nit) for adjusting the brightness of the display screen in the display task set by calling a function. If the returned numerical result is a specific value, it is confirmed that there is a dimming task. If the returned numerical result is empty, it is confirmed that there is no dimming task.

[0115] S806. When the communication thread of the hardware synthesizer determines that there is a dimming task in the display task set, it sends the dimming task to the background thread.

[0116] Exemplarily, after setting a task type identifier for the dimming task, the dimming task and the task type identifier are stored in the task queue of the background thread for waiting to be executed.

[0117] S807. The communication thread of the hardware synthesizer sends the display tasks in the display task set to the main thread for execution.

[0118] S808. The communication thread of the hardware synthesizer sends a resource unlocking signal to the synthesis thread of the synthesizer.

[0119] S809. The main thread of the hardware synthesizer executes the display tasks in the display task set.

[0120] S810. The background thread extracts the tasks to be executed in the task queue and determines whether the task to be executed is a dimming task.

[0121] Exemplarily, the background thread repeatedly extracts the tasks to be executed in the task queue. Before executing the task to be executed, it confirms the task type identifier carried by the task to be executed, and determines whether the task to be executed is a dimming task through the task type identifier. For example, by setting an enumeration class, using numerical values to represent the task types of the tasks in the task queue. For example, the dimming task is set to 1, the inquiry task is set to 2, and so on. When confirming whether it is a dimming task, it determines whether the task to be executed is a dimming task through the corresponding relationship between the numerical value and the task type.

[0122] S811. When the background thread determines that the task to be executed is a dimming task, it raises the priority of the background thread and executes the task.

[0123] It can be known that the higher the priority of the thread, the greater the probability that the tasks in the thread will be executed by the CPU. Exemplarily, in the computer programming language Java, the priority of the thread has 1 to 10 levels. The thread with a priority of 1 has a low probability of being executed by the CPU, and the thread with a priority of 10 has a high probability of being executed by the CPU. The priority of the thread is set by calling a function method. In addition to Java, other computer programming languages can also be used to set the priority of the thread, which will not be exemplified one by one here.

[0124] Exemplarily, if the dimming task is automatically triggered by the terminal device according to the change in the ambient light intensity detected by the ambient light sensor, the priority can be set to level 7, for example. If the dimming task is triggered by the user manually controlling the screen and has a high requirement for real-time performance, the priority can be set to level 9, for example. Specifically, it is flexibly set according to actual requirements and the overall system scheduling, and this application does not make specific limitations.

[0125] S812. After the background thread finishes executing the dimming task, it restores the priority of the background thread and returns to S812 to continue extracting the tasks to be executed in the task queue.

[0126] In one embodiment, after the dimming task is completed, the priority of the background thread is restored to the priority before the dimming task is executed by calling a function method. For example, before the dimming task is executed, the priority of the background thread is at level 5. Then, after the dimming task is completed, the priority of the background thread is restored to level 5. And it returns to S810 to continue executing the subsequent tasks in the task queue.

[0127] S813. When the background thread determines that the task to be executed is a non-dimming task type, it executes the task.

[0128] S814. After the background thread finishes executing the task, it returns to S810 to continue extracting the tasks to be executed in the task queue.

[0129] Exemplarily, as Figure 9 shown in the flowchart of a processing method for concurrent generation of images and screen dimming, which includes: a first process (Composer process) and a second process (SurfaceFlinger process).

[0130] After the second process completes the synthesis operation of the image data rendered by each APP, it saves the display task in the display task set and sends the display task set to the first process through inter-process communication (Binder). After receiving the display task set, the first process uses a function to call the target parameters (such as brightness parameters) of each task in the display task set. If the returned numerical result is empty, it determines that there is no dimming task in the display task set and continues to execute the display tasks in the display task set. After executing the display tasks in the display task set, it returns the resource unlock signal to the SurfaceFlinger process through inter-process communication (Binder). If the returned numerical result is a specific value, it determines that there is a dimming task in the display task set. After setting the task type identifier for the dimming task, it sends the dimming task to the task queue of the background thread in the first process to wait for execution, and then continues to call the function to traverse the display task set to determine whether there is a dimming task until there is no dimming task in the display task set, and then continues to execute the display tasks in the display task set.

[0131] The background thread loops through and executes the tasks in the task queue. When executing the current task to be executed, it determines whether it is a dimming task based on the task type identifier of the current task to be executed. If the task type identifier indicates that the current task to be executed is a non-dimming task, it waits for the CPU to execute according to the priority of the current thread. After the CPU finishes executing, the background thread continues to execute the next task. If the task type identifier indicates that the current task to be executed is a dimming task, it increases the priority of the thread and waits for the CPU to execute. After the CPU executes the dimming task, it restores the thread priority. The background thread continues to execute the next task until the background thread is aborted or destroyed.

[0132] The terminal device in the embodiments of the present application may be a portable computer (such as a mobile phone), a tablet computer, a laptop computer, a personal computer (PC), a wearable terminal device (such as a smart watch), an augmented reality (AR) virtual reality (VR) device, a vehicle-mounted computer, a smart TV, etc., which have a display screen for adjusting the display brightness. The following embodiments do not impose special restrictions on the specific form of the terminal device.

[0133] Exemplarily, Figure 10 A schematic structural diagram of a terminal device 200 is shown. As Figure 10 shown, it shows a schematic structural diagram of a terminal device 200. The terminal device 200 may include a processor 210, an external memory interface 220, an internal memory 221, an audio module 230, a display screen 240, a communication module 250, a power module 260, an input device 270, a sensor module 280, etc. Among them, the sensor module 280 may include an ambient light sensor, a touch sensor, etc.

[0134] It can be understood that the structure schematically shown in the embodiments of the present application does not constitute a specific limitation on the terminal device 200. In other embodiments of the present application, the terminal device 200 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0135] The processor 210 may include one or more processing units. For example: the processor 210 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0136] Among them, the controller may be the nerve center and command center of the terminal device 200. The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching instructions and executing instructions.

[0137] An operating system of the terminal device 200 can run on the application processor, which is used to manage the hardware and software resources of the terminal device 200. For example, it manages and configures the memory, determines the priority order of system resource supply and demand, controls input and output devices, operates the network, manages the file system, manages drivers, etc. The operating system can also be used to provide an operation interface for users to interact with the system. Among them, various software can be installed in the operating system, such as drivers, application programs, etc.

[0138] A memory can also be set in the processor 210 for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory can save the instructions or data that the processor 210 has just used or recycled. If the processor 210 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated access and reduces the waiting time of the processor 210, thus improving the efficiency of the system.

[0139] In some embodiments, the processor 210 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0140] It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only for illustrative purposes and does not constitute a structural limitation on the terminal device 200. In other embodiments of the present application, the terminal device 100 can also adopt different interface connection methods or a combination of multiple interface connection methods in the above embodiments.

[0141] The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to implement the storage capacity expansion of the terminal device 200. The external memory card communicates with the processor 210 through the external memory interface 220 to implement the data storage function. For example, files such as music and videos are saved in the external memory card.

[0142] The internal memory 221 can be used to store one or more computer programs, and the one or more computer programs include instructions. The processor 210 can run the above instructions stored in the internal memory 221, so that the terminal device 200 executes the application running method provided in some embodiments of this application, as well as various applications and data management, etc. The internal memory 221 can include a code storage area and a data storage area. Among them, the data storage area can store data created during the use of the terminal device 200. In addition, the internal memory 221 can include high-speed random access memory, and can also include non-volatile memory, such as one or more disk storage components, flash memory components, universal flash storage (UFS), etc. In some embodiments, the processor 210 can run the instructions stored in the internal memory 221, and / or the instructions stored in the memory provided in the processor 210, to make the terminal device 200 execute the application running method provided in the embodiments of this application, as well as other applications and data management.

[0143] The terminal device 200 can implement audio functions through the audio module 230, speakers, microphones, and application processors, etc. For example, music playback, recording, etc. The audio module 230 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal. The audio module 230 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 230 can be set in the processor 210, or some functional modules of the audio module 230 can be set in the processor 210.

[0144] The speaker, also known as the "horn", is used to convert an audio electrical signal into a sound signal.

[0145] The microphone, also known as the "microphone", "transmitter", is used to convert a sound signal into an electrical signal. The user can make a sound close to the microphone with the mouth to input the sound signal into the microphone.

[0146] The communication function of the terminal device 200 can be implemented through antenna 1, antenna 2, and communication module 250, etc.

[0147] The communication module 250 may provide wireless communication solutions applied to the terminal device 200, including cellular, Wi-Fi, Bluetooth (BT), wireless data transmission modules (such as 433 MHz, 868 MHz, 915 MHz), etc. The communication module 250 may be one or more devices integrating at least one communication processing module. The communication module 250 receives electromagnetic waves via antenna 1 or antenna 2, filters and frequency-modulates the electromagnetic wave signals, and sends the processed signals to the processor 210. The communication module 250 may also receive the signals to be sent from the processor 210, frequency-modulate and amplify them, and convert them into electromagnetic waves through antenna 1 or antenna 2 for radiation.

[0148] The terminal device 200 realizes the display function through the GPU, the display screen 240, and the application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 240 and the application processor. The GPU is used to execute mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or change display information. In the embodiments of the present application, the GPU is used to render the graphics data (mainly model data and texture images) provided by the APP that needs to be rendered, and obtain the rendered image frames.

[0149] The display screen 240 is used to display images, videos, etc. The display screen 240 includes a display panel. The display panel may adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 200 may include 1 or N display screens 240, where N is a positive integer greater than 1. In the embodiments of the present application, the display screen 240 may be used to display the UI and receive user operations on the UI.

[0150] In some embodiments, a pressure sensor, a touch sensor, etc. are provided on the display screen 240. The pressure sensor is used to sense pressure signals and can convert the pressure signals into electrical signals. When a touch operation acts on the display screen 240, the terminal device 200 detects the intensity of the touch operation according to the pressure sensor. The terminal device 200 can also calculate the position of the touch according to the detection signal of the pressure sensor. The touch sensor, also called a "touch panel", can form a touch screen, also called a "touch control screen", with the display screen 240. The touch sensor is used to detect touch operations acting on it or nearby. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can also be provided through the display screen 240.

[0151] In the embodiments of the present application, the display screen can be any type of display screen, for example, a straight screen, a curved screen, a foldable screen, a special-shaped screen, etc. It can be a touch screen or a non-touch screen, and the display screen needs to be able to adjust the brightness. If the display screen is a touch screen, the brightness adjustment function can be achieved by touching the dimming control on the screen. For example, by pulling down the control center and in the dimming control of the control center, the brightness of the display screen is adjusted by sliding up and down. If the display screen is a non-touch screen, the brightness adjustment function can be adjusted through input devices such as a keyboard and a mouse.

[0152] The power module 260 can be used to supply power to each component included in the terminal device 200. In some embodiments, the power module 260 can be a battery, such as a rechargeable battery.

[0153] The input device 270 can include a keyboard, a mouse, etc. The keyboard is used to input letters, numbers, punctuation marks, etc. into the terminal device 200, so as to send commands to the terminal device 200 and input data, etc. The mouse is an indicator for positioning the horizontal and vertical coordinates of the display system of the terminal device 200 and is used to input instructions to the terminal device 200. Among them, the input device 270 can be connected to the terminal device 200 in a wired connection manner. For example, the input device 270 is connected to the terminal device 200 through a GPIO interface, a USB interface, etc. The input device 270 can also be connected to the terminal device 200 wirelessly. For example, the input device 220 is connected to the terminal device 200 through Bluetooth, infrared, etc.

[0154] The sensor module 280 of the terminal device 200 further includes an ambient light sensor. The ambient light sensor is used to adjust the brightness of the display screen according to the ambient light. In the embodiments of the present application, when the ambient light sensor detects a change in the ambient light, the brightness of the display screen is automatically adjusted according to the intensity of the ambient light.

[0155] In the embodiments of the present application, the operating system of the above terminal device 200 can be the same as that in the above examples The system, or other display sending processes, are the same as or similar to the process in the above example system systems, etc.

[0156] Embodiments of the present application provide a terminal device, which may include: a memory and one or more processors. Among them, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the terminal device can execute each function or step executed by the mobile phone in the above method embodiments. The structure of the terminal device can refer to Figure 10 the structure of the terminal device shown.

[0157] Embodiments of the present application also provide a chip system (for example, a system on a chip (SoC)), as Figure 11 shown, the chip system includes at least one processor 1101 and at least one interface circuit 1102. The processor 1101 and the interface circuit 1102 can be interconnected through a line. For example, the interface circuit 1102 can be used to receive signals from other devices (such as the memory of the terminal device). Again, for example, the interface circuit 1102 can be used to send signals to other devices (such as the processor 1101 or the camera of the terminal device). Exemplarily, the interface circuit 1102 can read the instructions stored in the memory and send the instructions to the processor 1101. When the instructions are executed by the processor 1101, the terminal device can execute each step in the above embodiments. Of course, the chip system can also include other discrete devices, and embodiments of the present application do not make specific limitations on this.

[0158] Embodiments of the present application also provide a computer-readable storage medium, including computer instructions. When the computer instructions run on the terminal device, the terminal device is enabled to execute each function or step executed by the terminal device 200 in the above method embodiments.

[0159] Embodiments of the present application also provide a computer program product. When the computer program product runs on the terminal device, the computer is enabled to execute each function or step executed by the terminal device 200 in the above method embodiments. For example, the computer can be the above terminal device 200.

[0160] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and conciseness of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0161] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0162] The units described as separate components may or may not be physically separated. The components displayed as units can be one physical unit or multiple physical units, that is, they can be located in one place, or they can be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0163] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0164] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0165] The above content is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A processing method for concurrent generation of images and screen dimming, applied to a terminal device, the terminal device including a display screen, characterized in that The method includes: In response to a first operation of the user, the terminal device plays a first dynamic image; During the playing of the first dynamic image, the terminal device adjusts the brightness of the display screen; During the terminal device playing the first dynamic image and adjusting the brightness of the display screen, the hardware synthesizer of the terminal device receives a display task set from the image synthesizer. A first thread in the hardware synthesizer executes the display task for sending the image in the display task set, and a second thread in the hardware synthesizer executes the dimming task in the display task set; wherein, the display task for sending the image is used to send a first image to the display screen for display, and the first image is a frame image in the first dynamic image; the dimming task is used to adjust the brightness of the display screen.

2. The method according to claim 1, wherein The method further includes: The image synthesizer adds the display task for sending the image and the dimming task to the display task set in a first display cycle for sending the image.

3. The method according to claim 2, wherein The first thread in the hardware synthesizer executing the display task for sending the image in the display task set includes: The first thread in the hardware synthesizer executes the display task for sending the image in the display task set in the first display cycle for sending the image.

4. The method according to any one of claims 1 to 3, characterized in that, Before the second thread in the hardware synthesizer executes the dimming task in the display task set, the method further includes: The priority of the second thread being called by the processor is increased.

5. The method according to claim 4, wherein After the second thread in the hardware synthesizer executes the dimming task in the display task set, the method further includes: The priority of the second thread being called by the processor is decreased.

6. The method according to claim 5, wherein The priority of the second thread being called by the processor being increased includes: The priority of the second thread being called by the processor is adjusted from a first value to a second value; The priority of the second thread being called by the processor being decreased includes: The priority of the second thread being called by the processor is adjusted from the second value to the first value.

7. The method according to claim 2 or 3, characterized in that After the first thread in the hardware synthesizer executes the display task for sending the image in the display task set, the method further includes: The hardware synthesizer returns a resource unlock signal to the image synthesizer in the first display cycle for sending the image.

8. The method according to any one of claims 1-7, characterized in that, After the hardware synthesizer of the terminal device receives the display task set from the image synthesizer, the method further includes: The terminal device determines that the display task set includes the dimming task, and distributes the dimming task to the second thread.

9. The method according to claim 8, wherein After the terminal device determines that the display task set includes the dimming task, the method further includes: The terminal device starts the second thread.

10. The method according to claim 9, characterized in that The second thread is a background thread.

11. A terminal device, characterized in that, The terminal device includes a memory, a display screen, and one or more processors; the memory, the display screen are coupled to the processor; computer program code is stored in the memory, and the computer program code includes computer instructions. When the computer instructions are executed by the processor, the terminal device executes the method according to any one of claims 1-10.

12. A computer-readable storage medium, characterized in that, Including computer instructions, when the computer instructions run on the terminal device, the terminal device executes the method according to any one of claims 1-10.

Images