Layer synthesis method and device, electronic equipment and storage medium

Abstract

The invention discloses a layer synthesis method and device, electronic equipment and a storage medium, and belongs to the technical field of image synthesis. The method comprises the following steps: acquiring layer data of a to-be-displayed interface of a target application program; determining a target synthesis thread corresponding to the to-be-displayed interface from a plurality of parallel synthesis threads; and synthesizing the layer data through the target synthesis thread to obtain the to-be-displayed interface of the target application program. The frame dropping risk can be reduced, the frame rate can be improved, and the display effect can be improved.

Description

Technical Field

[0001] This application belongs to the technical field of image processing, and particularly relates to a method, apparatus, electronic device, and storage medium for layer composition. Background Art

[0002] The display interface of an application (App) is obtained by a compositor synthesizing its layer data. When displaying the display interfaces of multiple applications, the compositor needs to sequentially synthesize the layers of multiple applications in order. The layer composition efficiency of this method is relatively low. If the overall composition time of multiple applications exceeds the frame composition time (VSync period) allowed by the system, frame drops will occur. Additionally, if there is an abnormality in the layer composition of one application, it will cause other applications ranked behind it to be unable to perform layer composition, resulting in frame drops and affecting the display effect. Summary of the Invention

[0003] This application aims to at least solve one of the technical problems existing in the prior art. For this purpose, this application provides a method, apparatus, electronic device, and storage medium for layer composition, which can reduce the risk of frame drops, increase the frame rate, and thus improve the display effect.

[0004] In a first aspect, this application provides a method for layer composition, which includes:

[0005] Obtain the layer data of the to-be-displayed interface of the target application;

[0006] Determine the target composition thread corresponding to the to-be-displayed interface from multiple parallel composition threads;

[0007] Synthesize the layer data through the target composition thread to obtain the to-be-displayed interface of the target application.

[0008] According to the layer composition method of this application, by setting multiple parallel composition threads, the target composition thread corresponding to the to-be-displayed interface of the target application is determined from multiple parallel composition threads, enabling the target application to perform layer composition through the target composition thread. Multiple applications can perform layer composition through different parallel composition threads, improving the layer composition efficiency of multiple applications, reducing the risk of frame drops, increasing the frame rate, and thus improving the display effect.

[0009] According to an embodiment of this application, the determining the target composition thread corresponding to the to-be-displayed interface from multiple parallel composition threads includes:

[0010] Determine the priority of the to-be-displayed interface;

[0011] Determine the target synthesis thread corresponding to the interface to be displayed from multiple parallel synthesis threads according to the priority of the interface to be displayed.

[0012] According to an embodiment of the present application, determining the priority of the interface to be displayed includes:

[0013] Determine the priority of the target application, and use the priority of the target application as the priority of the interface to be displayed.

[0014] According to an embodiment of the present application, the priority of the target application includes high priority or low priority;

[0015] Determining the priority of the target application includes:

[0016] Detect whether the target application meets the high-priority conditions;

[0017] If so, determine that the priority of the target application is high priority;

[0018] If not, determine that the priority of the target application is low priority.

[0019] According to an embodiment of the present application, the high-priority conditions include at least one of the following: the target application is the application pointed to by the user input; the target application is located in the foreground area in the extended reality space; the application type of the target application belongs to the target application type; the target application belongs to the preset high-priority application.

[0020] According to an embodiment of the present application, determining the priority of the interface to be displayed includes:

[0021] Determine the application scenario of the interface to be displayed according to the layer data of the interface to be displayed;

[0022] Determine the priority of the interface to be displayed according to the application scenario.

[0023] According to an embodiment of the present application, the priorities of the multiple synthesis threads are different, and the priority of the target synthesis thread corresponds to the priority of the interface to be displayed.

[0024] According to an embodiment of the present application, the multiple synthesis threads include high-priority synthesis threads and low-priority synthesis threads, the high-priority synthesis threads run on the large CPU cores, and the low-priority synthesis threads run on the small CPU cores.

[0025] In a second aspect, the present application provides a layer synthesis device, including:

[0026] An acquisition module, configured to acquire layer data of a to-be-displayed interface of a target application;

[0027] A determination module, configured to determine a target synthesis thread corresponding to the to-be-displayed interface from multiple parallel synthesis threads;

[0028] A synthesis module, configured to synthesize the layer data through the target synthesis thread to obtain the to-be-displayed interface of the target application.

[0029] According to the layer synthesis device of the present application, by setting multiple parallel synthesis threads, a target synthesis thread corresponding to the to-be-displayed interface of the target application is determined from the multiple parallel synthesis threads, so that the target application performs layer synthesis through the target synthesis thread. Multiple applications can perform layer synthesis through different parallel synthesis threads, improving the layer synthesis efficiency of multiple applications, reducing the risk of frame drops, increasing the frame rate, and thus improving the display effect.

[0030] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the layer synthesis method described in the first aspect above is implemented.

[0031] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the layer synthesis method described in the first aspect above is implemented.

[0032] In a fifth aspect, the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the layer synthesis method described in the first aspect above is implemented.

[0033] One or more of the above technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0034] By setting multiple parallel synthesis threads, a target synthesis thread corresponding to the to-be-displayed interface of the target application is determined from the multiple parallel synthesis threads, so that the target application performs layer synthesis through the target synthesis thread. Multiple applications can perform layer synthesis through different parallel synthesis threads, improving the layer synthesis efficiency of multiple applications, reducing the risk of frame drops, increasing the frame rate, and thus improving the display effect.

[0035] The additional aspects and advantages of the present application will be partially given in the following description, partially become apparent from the following description, or be understood through the practice of the present application. Description of the Drawings

[0036] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the description of the embodiments in conjunction with the following drawings, where:

[0037] Figure 1 is a schematic flowchart of a layer synthesis method provided by an embodiment of the present application;

[0038] Figure 2 is a schematic diagram of the principle of a layer synthesis method provided by an embodiment of the present application;

[0039] Figure 3 is a schematic diagram of an application scenario of a layer synthesis method provided by an embodiment of the present application;

[0040] Figure 4 is a schematic structural diagram of an interaction device provided by an embodiment of the present application;

[0041] Figure 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0042] Figure 6 is a schematic diagram of the hardware of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0043] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application fall within the protection scope of the present application.

[0044] The terms "first", "second", etc. in the description and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the description and claims means at least one of the connected objects, and the character " / " generally means an "or" relationship between the associated objects before and after.

[0045] Next, in conjunction with the accompanying drawings, the layer synthesis method, device, electronic device, and storage medium provided by the embodiments of the present application will be described in detail through specific embodiments and their application scenarios.

[0046] Among them, the layer synthesis method can be applied to a terminal, and can be specifically executed by hardware or software in the terminal.

[0047] Figure 1Schematic flowchart of a layer composition method provided by an embodiment of the present application.

[0048] The layer composition method provided by an embodiment of the present application. The execution subject of this layer composition method can be an electronic device or a functional module or functional entity in the electronic device that can implement this layer composition method. Hereinafter, taking the electronic device as the execution subject as an example, the layer composition method provided by an embodiment of the present application will be described.

[0049] As Figure 1 shown, this layer composition method includes: step 110, step 120, and step 130.

[0050] Step 110: Obtain the layer data of the to-be-displayed interface of the target application.

[0051] The to-be-displayed interface of the target application refers to the interface that the target application needs to display on the electronic device. The to-be-displayed interface includes at least one layer, such as a navigation bar and a status bar, etc. Each layer can correspond to a layer buffer. The target application draws each layer of the to-be-displayed interface to the corresponding layer buffer by calling the GPU (Graphics Processing Unit), that is, saves the layer data of each layer of the to-be-displayed interface to the corresponding layer buffer.

[0052] In step 110, the layer data of the to-be-displayed interface of the target application can be obtained from all the layer buffers corresponding to the target application. The layer data of the to-be-displayed interface of the target application can also be obtained by other means, which is not specifically limited here.

[0053] As Figure 2 shown, the to-be-displayed interface of the target application (App) includes two layers. The target application draws the two layers of the to-be-displayed interface to the two layer buffers Buffer1 and Buffer2 respectively by calling the GPU, that is, the layer data of the two layers of the to-be-displayed interface are respectively saved in Buffer1 and Buffer2. The layer data of the to-be-displayed interface of the target application can be obtained from the two layer buffers Buffer1 and Buffer2.

[0054] In the case where the electronic device needs to display the to-be-displayed interfaces of multiple applications, each application calls the GPU to draw the layers of its to-be-displayed interface to the corresponding layer buffer. By sequentially taking each application among the multiple applications as the target application, the layer data of the to-be-displayed interface of the target application can be obtained from the layer buffer corresponding to the target application. Among them, the multiple applications are independent of each other.

[0055] Step 120: Determine the target composition thread corresponding to the interface to be displayed from multiple parallel composition threads.

[0056] In this embodiment, the compositor can set multiple composition threads, and the multiple composition threads run in parallel. After obtaining the layer data of the interface to be displayed, first determine the target composition thread corresponding to the target application from the multiple parallel composition threads, that is, select one composition thread from the multiple parallel composition threads as the composition thread for the interface to be displayed of the target application.

[0057] There are various ways to determine the target composition thread corresponding to the interface to be displayed of the target application.

[0058] For example, randomly select one composition thread from the multiple parallel composition threads as the target composition thread corresponding to the interface to be displayed of the target application. For another example, sort the multiple parallel composition threads and assign a composition thread to the interface to be displayed of the target application in sequence as the target composition thread. For example, after obtaining the layer data of the interface to be displayed of the first application, use the first composition thread as the target composition thread corresponding to the interface to be displayed of the first application; after obtaining the layer data of the interface to be displayed of the second application, use the second composition thread as the target composition thread corresponding to the interface to be displayed of the second application.

[0059] In some embodiments, determining the target composition thread corresponding to the interface to be displayed from the multiple parallel composition threads in step 120 includes:

[0060] Determine the priority of the interface to be displayed;

[0061] According to the priority of the interface to be displayed, determine the target composition thread corresponding to the interface to be displayed from the multiple parallel composition threads.

[0062] This embodiment can divide multiple priorities for the interface to be displayed. The priorities of the interfaces to be displayed of different applications can be the same or different, and the priorities of different interfaces to be displayed in the target application can be the same or different.

[0063] Pre-set the corresponding relationship between multiple composition threads and multiple priorities, so that each priority corresponds to at least one composition thread. After determining the priority of the interface to be displayed of the target application, according to the corresponding relationship between the multiple composition threads and the multiple priorities, use the composition thread corresponding to the priority of the interface to be displayed of the target application as the target composition thread.

[0064] In some embodiments, priorities are set for the composition threads, and the priorities of multiple composition threads are different. The priority of the target composition thread corresponds to the priority of the interface to be displayed of the target application, that is, a composition thread whose priority corresponds to the priority of the interface to be displayed of the target application is selected from multiple composition threads as the target composition thread.

[0065] For example, the priority of the interface to be displayed can be divided into high priority and low priority, and multiple composition threads can include high-priority composition threads and low-priority composition threads. When the priority of the interface to be displayed of the target application is high priority, it is determined that the target composition thread corresponding to the interface to be displayed of the target application is a high-priority composition thread; when the priority of the interface to be displayed of the target application is low priority, it is determined that the target composition thread corresponding to the interface to be displayed of the target application is a low-priority composition thread.

[0066] There are various ways to determine the priority of the interface to be displayed.

[0067] In some embodiments, the priority of the interface to be displayed of the target application can be determined by the priority of the target application. The steps of determining the priority of the interface to be displayed include:

[0068] Determine the priority of the target application and use the priority of the target application as the priority of the interface to be displayed.

[0069] In this embodiment, the priorities of different interfaces to be displayed in the target application are the same. After determining the priority of the target application, the priority of the interface to be displayed of the target application can be determined, that is, the priorities of different interfaces to be displayed of the target application are the same as the priority of the target application.

[0070] Among them, there are various ways to determine the priority of the target application.

[0071] In some embodiments, the priority of the target application can be determined by setting priority conditions. For example, if an application divides multiple priorities, multiple priority conditions are set, and the multiple priority conditions correspond one-to-one to the multiple priorities divided by the application. Detect which priority condition the target application meets, and use the priority corresponding to the priority condition that the target application meets as the priority of the target application.

[0072] For example, the priority of the target application includes high priority or low priority, that is, the application divides two priorities (high priority and low priority). The steps of determining the priority of the target application include:

[0073] Detect whether the target application meets the high-priority condition;

[0074] If so, determine that the priority of the target application is high priority;

[0075] If not, determine that the priority of the target application is low priority.

[0076] Among them, high-priority conditions can be set in advance, and it is detected whether the target application meets the high-priority conditions. If the target application meets the high-priority conditions, determine that the priority of the target application is high priority, and determine that the priority of the interface to be displayed of the target application is high priority; if the target application does not meet the high-priority conditions, determine that the priority of the target application is low priority, and determine that the priority of the interface to be displayed of the target application is low priority.

[0077] Alternatively, low-priority conditions can be set in advance, and it is detected whether the target application meets the low-priority conditions. If the target application meets the low-priority conditions, determine that the priority of the target application is low priority, and determine that the priority of the interface to be displayed of the target application is low priority; if the target application does not meet the low-priority conditions, determine that the priority of the target application is high priority, and determine that the priority of the interface to be displayed of the target application is high priority.

[0078] Among them, the high-priority conditions can include at least one of the following: the target application is the application pointed to by the user input; the target application is located in the foreground area in the extended reality space; the application type of the target application belongs to the target application type; the target application belongs to the preset high-priority application. The high-priority conditions can also include other conditions, which are not specifically limited here.

[0079] In the case where the high-priority condition includes that the target application is the application pointed to by the user input, it is detected whether the target application is the application pointed to by the user input. Among them, the user input refers to the input based on the electronic device by the user. In the case where the electronic device is an extended reality (XR) device, the user input can include at least one of gaze input, gesture input, voice input, handle input, and head-mounted device input, etc. Among them, the gaze input can include the gaze information of the user's eyes collected by the image acquisition device (such as a camera) in the electronic device; the gesture input can include the hand information of the user collected by the image acquisition device (such as a camera) in the electronic device; the voice input can include the user voice information input by the voice acquisition device in the electronic device; the handle input can include the operation information triggered by the user's operation on the handle in the electronic device; the head-mounted device input can include the operation information triggered by the user's operation on the head-mounted device in the electronic device.

[0080] An XR device can simulate an extended reality space, which can be a simulation environment of the real world, a semi-simulated and semi-virtual virtual scene, or a purely virtual virtual scene. Multiple applications can be located in the extended reality space. User input can point to one or more applications in the extended reality space. For example, the user input includes a gaze input, and the application corresponding to the gaze point position of the gaze input is the application pointed to by the user input. The application pointed to by the user input is the application that the user is currently interacting with. If the target application is the application pointed to by the user input, it indicates that the user currently needs to interact with the target application, and the priority of the target application can be determined as a high priority.

[0081] When the high-priority condition includes that the target application is located in the near view area of the extended reality space, detect whether the target application is located in the near view area of the extended reality space. Among them, the extended reality space can include a near view area and a far view area. The near view area is close to the user's field of view, and the far view area is far from the user's field of view. The application can be located in the near view area or the far view area of the extended reality space. Since the near view area is close to the user's field of view, the user may pay more attention to the applications in the near view area. If the target application is located in the near view area of the extended reality space, it indicates that the user may pay more attention to the target application, and the priority of the target application can be determined as a high priority.

[0082] When the high-priority condition includes that the application type of the target application belongs to the target application type, detect whether the application type of the target application is the target application type. Among them, the target application type can be preset, and the target application type can refer to application types with high real-time frame rate requirements, such as video applications and game applications, etc. Since the applications of the target application type have high real-time frame rate requirements, if the application type of the target application is the target application type, it indicates that the target application has high real-time frame rate requirements, and the priority of the target application can be determined as a high priority.

[0083] In the case where the high - priority condition includes that the target application belongs to a preset high - priority application, detect whether the target application belongs to the preset high - priority application. The list of high - priority applications can be set in advance, and this list of high - priority applications can be customized by the user according to their own preferences. Each application has a unique identification code, and the list of high - priority applications can include the identification codes of at least one application. For example, the list of high - priority applications can include the identification code of a certain instant messaging application and the identification code of a certain game application, etc. It can be detected whether the target application belongs to the preset high - priority application by determining whether the identification code of the target application is in the list of high - priority applications. If the identification code of the target application is in the list of high - priority applications, it is determined that the target application belongs to the preset high - priority application, and the priority of the target application can be determined as high - priority; if the identification code of the target application is not in the list of high - priority applications, it is determined that the target application does not belong to the preset high - priority application, and the priority of the target application can be determined as low - priority.

[0084] In the case where the high - priority condition includes multiple conditions, detect whether the target application meets each condition in the high - priority conditions. If the target application meets each condition in the high - priority conditions, the priority of the target application can be determined as high - priority; if the target application does not meet any one of the high - priority conditions, the priority of the target application can be determined as low - priority.

[0085] For example, the high - priority conditions include that the target application is the application pointed to by the user input, and the target application is in the scenic area in the extended reality space. Detect whether the target application is the application pointed to by the user input, and whether the target application is in the scenic area in the extended reality space. If the target application is the application pointed to by the user input and the target application is in the scenic area in the extended reality space, the priority of the target application is determined as high - priority; if the target application is not the application pointed to by the user input, or the target application is in the far - scenic area in the extended reality space, the priority of the target application is determined as low - priority.

[0086] In some embodiments, the priority of the display interface of the target application can be determined by the application scenario of the display interface of the target application. Determining the priority of the display interface includes:

[0087] Determine the application scenario of the display interface according to the layer data of the display interface;

[0088] Determine the priority of the display interface according to the application scenario.

[0089] In this embodiment, the priorities of different to-be-displayed interfaces in the target application can be different. The priorities can be set for multiple application scenarios in the target application in advance, and the priorities of different application scenarios can be the same or different. After determining the application scenario of the to-be-displayed interface of the target application, the priority of this application scenario is used as the priority of the to-be-displayed interface of the target application.

[0090] For example, the target application is a combat game, and the application scenarios of the combat game include a game character selection scenario, a battle scenario, etc. Among them, the game character selection scenario refers to the scenario of selecting a game character before the battle, and the battle scenario refers to the scenario where the game character is in battle. The priority corresponding to the game character selection scenario can be a low priority, and the priority corresponding to the battle scenario can be a high priority. When the to-be-displayed interface is a game character selection interface, it is determined that the application scenario of the to-be-displayed interface is the game character selection scenario, and then it is determined that the priority of the to-be-displayed interface is a low priority. When the to-be-displayed interface is a battle interface, it is determined that the application scenario of the to-be-displayed interface is the battle scenario, and then it is determined that the priority of the to-be-displayed interface is a high priority.

[0091] Step 130: Synthesize the layer data through the target synthesis thread to obtain the to-be-displayed interface of the target application.

[0092] Combined Figure 2 As shown, after determining the target synthesis thread, the layer data of the to-be-displayed interface is sent to the SurfaceFlinger (SF) process of the synthesizer. There are multiple synthesis threads (SurfaceFlinger RenderThread) in SF. In the target synthesis thread among the multiple synthesis threads, SF calls the GPU to synthesize the layer data of the to-be-displayed interface, that is, synthesizes the layer data in buffer1 and buffer2 into the display buffer Display Buffer to obtain the to-be-displayed interface of the target application, so as to display the to-be-displayed interface of the target application on the electronic device.

[0093] In the case where the electronic device needs to display the to-be-displayed interfaces of multiple applications, the target synthesis threads corresponding to the to-be-displayed interfaces of the multiple applications are sequentially determined from the parallel multiple synthesis threads, and the layer data of each application is synthesized through the target synthesis threads corresponding to the to-be-displayed interfaces of each application to obtain the to-be-displayed interfaces of each application. If the target synthesis threads corresponding to the to-be-displayed interfaces of multiple applications are the same, then the layer data of the multiple applications is sequentially synthesized through this target synthesis thread.

[0094] Such as Figure 3As shown, there are two parallel synthesis threads in the synthesizer, namely the first synthesis thread and the second synthesis thread. The electronic device needs to display the to-be-displayed interfaces of three applications (i.e., the first application, the second application, and the third application). When the vertical synchronization signal VSync arrives, the target synthesis threads corresponding to the to-be-displayed interfaces of the first application, the second application, and the third application are determined from the parallel first synthesis thread and second synthesis thread in sequence. When it is determined that the target synthesis thread corresponding to the to-be-displayed interface of the first application is the first synthesis thread, the first application is assigned to the first synthesis thread, that is, the first synthesis thread synthesizes the layer data of the first application to obtain the to-be-displayed interface of the first application, so as to display the to-be-displayed interface of the first application on the electronic device. When it is determined that the target synthesis thread corresponding to the to-be-displayed interface of the second application is the second synthesis thread, the second application is assigned to the second synthesis thread, that is, the second synthesis thread synthesizes the layer data of the second application to obtain the to-be-displayed interface of the second application, so as to display the to-be-displayed interface of the second application on the electronic device. When it is determined that the target synthesis thread corresponding to the to-be-displayed interface of the third application is the first synthesis thread, the third application is assigned to the first synthesis thread, that is, the first synthesis thread synthesizes the layer data of the third application to obtain the to-be-displayed interface of the third application, so as to display the to-be-displayed interface of the third application on the electronic device.

[0095] It should be noted that the synthesis thread assigned to the second application is different from those assigned to the first application and the third application. Therefore, the second application can synthesize the layer data simultaneously with the first application and the third application. That is, during the process of the first application synthesizing the layer data, the second application can start synthesizing the layer data without waiting for the first application to complete the synthesis of the layer data. After the first application completes the synthesis of the layer data, the third application can start synthesizing the layer data without waiting for the second application to complete the synthesis of the layer data. This embodiment can improve the layer synthesis efficiency of multiple applications, avoid the frame drop problem caused by multiple applications being unable to complete synthesis within one VSync cycle, and improve the frame rate.

[0096] In some embodiments, the multiple synthesis threads include a high-priority synthesis thread and a low-priority synthesis thread. The high-priority synthesis thread runs on the CPU big core, and the low-priority synthesis thread runs on the CPU small core.

[0097] In this embodiment, the CPU of the electronic device is a multi-core design. The large core of the CPU has high frequency and strong performance, but relatively high power consumption; the small core of the CPU has low frequency and weak performance, but relatively low power consumption. In the related art, when the electronic device needs to display the to-be-displayed interfaces of multiple application programs, only one synthesis thread sequentially synthesizes the layer data of the to-be-displayed interfaces of multiple application programs, and this synthesis thread occupies the large core of the CPU, resulting in a large load on the large core of the CPU and high power consumption.

[0098] In this embodiment, the high-priority synthesis thread runs on the large core of the CPU, and the low-priority synthesis thread runs on the small core of the CPU, so as to allocate the high-priority synthesis thread to the to-be-displayed interface with high priority, so that the synthesis of the layer data of the to-be-displayed interface with high priority is executed on the large core of the CPU, and the low-priority synthesis thread is allocated to the to-be-displayed interface with low priority, so that the layer data of the to-be-displayed interface with low priority is executed on the small core of the CPU. While meeting the synthesis requirements of the to-be-displayed interfaces of multiple application programs, the CPU parallelism of the synthesis task is improved, the load on the large core of the CPU is reduced, and the power consumption is reduced.

[0099] According to the layer synthesis method provided by the embodiment of the present application, by setting multiple parallel synthesis threads, the target synthesis thread corresponding to the to-be-displayed interface of the target application program is determined from the multiple parallel synthesis threads, so that the target application program performs layer synthesis through the target synthesis thread. Multiple application programs can perform layer synthesis through different parallel synthesis threads, improving the layer synthesis efficiency of multiple application programs, reducing the risk of frame drops, increasing the frame rate, and improving the display effect. Moreover, the high-priority synthesis threads among the multiple synthesis threads run on the large core of the CPU, and the low-priority synthesis threads run on the small core of the CPU, improving the CPU parallelism of the synthesis task, reducing the load on the large core of the CPU, and reducing the power consumption.

[0100] For the layer synthesis method provided by the embodiment of the present application, the execution subject may be a layer synthesis device. In the embodiment of the present application, taking the layer synthesis device executing the layer synthesis method as an example, the layer synthesis device provided by the embodiment of the present application is described.

[0101] The embodiment of the present application also provides a layer synthesis device.

[0102] As Figure 4 shown, the layer synthesis device includes: an acquisition module 401, a determination module 402, and a synthesis module 403.

[0103] The acquisition module 401 is configured to acquire the layer data of the to-be-displayed interface of the target application program;

[0104] The determination module 402 is configured to determine the target synthesis thread corresponding to the to-be-displayed interface from the multiple parallel synthesis threads;

[0105] The synthesis module 403 is configured to synthesize the layer data through the target synthesis thread to obtain the interface to be displayed of the target application program.

[0106] According to the layer synthesis device provided by the embodiments of the present application, by setting a plurality of parallel synthesis threads, a target synthesis thread corresponding to the interface to be displayed of the target application program is determined from the plurality of parallel synthesis threads, so that the target application program performs layer synthesis through the target synthesis thread. Multiple application programs can perform layer synthesis through different parallel synthesis threads, improving the layer synthesis efficiency of multiple application programs, reducing the risk of frame drops, increasing the frame rate, and improving the display effect.

[0107] In some embodiments, the determination module 402 is further configured to:

[0108] Determine the priority of the interface to be displayed;

[0109] According to the priority of the interface to be displayed, determine the target synthesis thread corresponding to the interface to be displayed from the plurality of parallel synthesis threads.

[0110] In some embodiments, the determination module 402 is further configured to:

[0111] Determine the priority of the target application program and use the priority of the target application program as the priority of the interface to be displayed.

[0112] In some embodiments, the priority of the target application program includes a high priority or a low priority; the determination module 402 is further configured to:

[0113] Detect whether the target application program meets the high-priority conditions;

[0114] If so, determine that the priority of the target application program is high priority;

[0115] If not, determine that the priority of the target application program is low priority.

[0116] In some embodiments, the high-priority conditions include at least one of the following: the target application program is the application program pointed to by the user input; the target application program is located in the foreground area in the extended reality space; the application type of the target application program belongs to the target application type; the target application program belongs to the preset high-priority application programs.

[0117] In some embodiments, the determination module 402 is further configured to:

[0118] According to the layer data of the interface to be displayed, determine the application scenario of the interface to be displayed;

[0119] Determine the priority of the to-be-displayed interface according to the application scenario.

[0120] In some embodiments, the priorities of multiple synthesis threads are different, and the priority of the target synthesis thread corresponds to the priority of the to-be-displayed interface.

[0121] In some embodiments, the multiple synthesis threads include high-priority synthesis threads and low-priority synthesis threads. The high-priority synthesis threads run on the large cores of the CPU, and the low-priority synthesis threads run on the small cores of the CPU.

[0122] The interaction device in the embodiments of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than terminals. Exemplarily, the electronic device can be an augmented reality (AR) / virtual reality (VR) device or an XR device, which is not specifically limited in the embodiments of the present application.

[0123] The interaction device in the embodiments of the present application can be a device with an operating system. The operating system can be the Microsoft (Windows) operating system, the Android operating system, the IOS operating system, or other possible operating systems, which is not specifically limited in the embodiments of the present application.

[0124] The layer synthesis device provided in the embodiments of the present application can implement each process of the method embodiments. To avoid repetition, it will not be elaborated here.

[0125] In some embodiments, as Figure 5 shown, the embodiments of the present application further provide an electronic device 800, including a processor 801, a memory 802, and a computer program stored on the memory 802 and executable on the processor 801. When the program is executed by the processor 801, it implements each process of the above interaction method embodiments and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0126] Figure 6 It is a schematic diagram of the hardware structure of an electronic device for implementing the embodiments of the present application.

[0127] The electronic device 900 includes but is not limited to: a radio frequency unit 901, a network module 902, an audio output unit 903, an input unit 904, a sensor 905, a display unit 906, a user input unit 907, an interface unit 908, a memory 909, and a processor 910, etc.

[0128] Those skilled in the art can understand that the electronic device 900 may further include a power source (such as a battery) for powering each component. The power source can be logically connected to the processor 910 through a power management system, so as to manage functions such as charging, discharging, and power consumption management through the power management system. Figure 6 The structure of the electronic device shown in Figure 6 does not limit the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0129] Among them, the processor 910 is used to obtain the layer data of the to-be-displayed interface of the target application; determine the target composition thread corresponding to the to-be-displayed interface from a plurality of parallel composition threads; and compose the layer data through the target composition thread to obtain the to-be-displayed interface of the target application.

[0130] According to the electronic device provided by the embodiments of the present application, by setting a plurality of parallel composition threads, and determining the target composition thread corresponding to the to-be-displayed interface of the target application from the plurality of parallel composition threads, the target application composes layers through the target composition thread, and multiple applications can compose layers through different parallel composition threads, improving the layer composition efficiency of multiple applications, reducing the risk of frame drops, and improving the display effect.

[0131] In some embodiments, the processor 910 is further used to:

[0132] Determine the priority of the to-be-displayed interface;

[0133] According to the priority of the to-be-displayed interface, determine the target composition thread corresponding to the to-be-displayed interface from a plurality of parallel composition threads.

[0134] In some embodiments, the processor 910 is further used to:

[0135] Determine the priority of the target application and use the priority of the target application as the priority of the to-be-displayed interface.

[0136] In some embodiments, the priority of the target application includes a high priority or a low priority; the processor 910 is further used to:

[0137] Detect whether the target application meets the high-priority condition;

[0138] If so, determine that the priority of the target application is high priority;

[0139] If not, determine that the priority of the target application is low priority.

[0140] In some embodiments, the high-priority condition includes at least one of the following: the target application is the application pointed to by the user input; the target application is located in the foreground area in the extended reality space; the application type of the target application belongs to the target application type; the target application belongs to a preset high-priority application.

[0141] In some embodiments, the processor 910 is further configured to:

[0142] Determine the application scenario of the interface to be displayed according to the layer data of the interface to be displayed;

[0143] Determine the priority of the interface to be displayed according to the application scenario.

[0144] In some embodiments, the priorities of the multiple synthesis threads are different, and the priority of the target synthesis thread corresponds to the priority of the interface to be displayed.

[0145] In some embodiments, the multiple synthesis threads include high-priority synthesis threads and low-priority synthesis threads. The high-priority synthesis threads run on the large cores of the CPU, and the low-priority synthesis threads run on the small cores of the CPU.

[0146] It should be understood that in the embodiments of the present application, the input unit 904 may include a Graphics Processing Unit (GPU) 9041 and a microphone 9042. The graphics processor 9041 processes the image data of the static pictures or videos obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 906 may include a display panel 9061, and the display panel 9061 may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 907 includes at least one of a touch panel 9071 and other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include two parts: a touch detection device and a touch controller. The other input devices 9072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, power-on keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0147] The memory 909 can be used to store software programs and various data. The memory 909 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 909 may include volatile memory or non-volatile memory, or the memory 909 may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synch link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 909 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memory.

[0148] The processor 910 may include one or more processing units; the processor 910 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 910 either.

[0149] The embodiments of the present application also provide a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements each process of the above-mentioned embodiment of the layer composition method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0150] Among them, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs.

[0151] An embodiment of the present application also provides a computer program product, including a computer program, which when executed by a processor implements the above-mentioned layer composition method.

[0152] Among them, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs.

[0153] An embodiment of the present application also provides a computer program product, including a computer program, which when executed by a processor implements the above-mentioned layer composition method.

[0154] Among them, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs.

[0155] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above-mentioned layer composition method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0156] It should be understood that the chip mentioned in the embodiments of the present application can also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0157] It should be noted that in this text, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, the features described with reference to certain examples may be combined in other examples.

[0158] From the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present application.

[0159] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Those of ordinary skill in the art, under the inspiration of the present application and without departing from the spirit and scope protected by the claims of the present application, can also make many forms, all of which fall within the protection scope of the present application.

[0160] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0161] Although embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present application. The scope of the present application is defined by the claims and their equivalents.

Claims

1. A method for layer composition, characterized in that, the method includes: obtaining layer data of a to-be-displayed interface of a target application; determining a target composition thread corresponding to the to-be-displayed interface from multiple parallel composition threads; composing the layer data through the target composition thread to obtain the to-be-displayed interface of the target application.

2. The layer composition method according to claim 1, characterized in that, the determining a target composition thread corresponding to the to-be-displayed interface from multiple parallel composition threads includes: determining the priority of the to-be-displayed interface; determining a target composition thread corresponding to the to-be-displayed interface from multiple parallel composition threads according to the priority of the to-be-displayed interface.

3. The layer composition method according to claim 2, characterized in that, the determining the priority of the to-be-displayed interface includes: determining the priority of the target application and using the priority of the target application as the priority of the to-be-displayed interface.

4. The layer composition method according to claim 3, characterized in that, the priority of the target application includes high priority or low priority; the determining the priority of the target application includes: detecting whether the target application meets the high-priority condition; if so, determining the priority of the target application as high priority; if not, determining the priority of the target application as low priority.

5. The layer composition method according to claim 4, characterized in that, the high-priority condition includes at least one of the following: the target application is the application pointed to by the user input; the target application is in the foreground area in the extended reality space; the application type of the target application belongs to the target application type; the target application belongs to the preset high-priority applications.

6. The layer composition method according to claim 2, characterized in that, the determining the priority of the to-be-displayed interface includes: determining the application scenario of the to-be-displayed interface according to the layer data of the to-be-displayed interface; determining the priority of the to-be-displayed interface according to the application scenario.

7. The layer composition method according to claim 2, characterized in that, the priorities of the multiple composition threads are different, and the priority of the target composition thread corresponds to the priority of the to-be-displayed interface.

8. The layer composition method according to claim 7, characterized in that, the multiple composition threads include high-priority composition threads and low-priority composition threads, the high-priority composition threads run on the CPU big cores, and the low-priority composition threads run on the CPU small cores.

9. A layer composition device, characterized in that, it includes: an obtaining module for obtaining layer data of a to-be-displayed interface of a target application; a determining module for determining a target composition thread corresponding to the to-be-displayed interface from multiple parallel composition threads; a composing module for composing the layer data through the target composition thread to obtain the to-be-displayed interface of the target application.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, when the processor executes the program, the layer composition method described in any one of claims 1-8 is implemented.

11. A non-transitory computer-readable storage medium, having a computer program stored thereon, wherein, when the computer program is executed by a processor, the layer composition method described in any one of claims 1-8 is implemented.

12. A computer program product, comprising a computer program, wherein, when the computer program is executed by a processor, the layer composition method described in any one of claims 1-8 is implemented.

Images