Operation frame rate control method, electronic device, chip system and readable storage medium

Abstract

The present application discloses a method for controlling an operating frame rate, an electronic device, a chip system, and a readable storage medium, and belongs to the field of terminal technology. The method includes: at a first moment, displaying a first application interface of a first application, and when displaying the first application interface, the operating frame rate of the first application is a first operating frame rate, and the first operating frame rate is determined based on the first interface name of the first application interface; at a second moment, obtaining a second interface name of a second application interface of a second application; determining the operating frame rate of the second application according to the second interface name, and obtaining a second operating frame rate; controlling the second application to display a second application interface based on the second operating frame rate, and the second application interface and the first application interface are displayed on the same display interface. The present application can reduce the operating load of the application by controlling each application to draw frames according to its own operating frame rate, and can minimize unnecessary waste of resources.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an operating frame rate control method, an electronic device, a chip system and a readable storage medium. Background Art

[0002] The running frame rate refers to the number of frames generated by the application per second, which determines the actual speed of content generation. During the running of the application, the application draws frames based on the running frame rate, that is, renders the application interface. In some scenarios, the electronic device may display multiple application interfaces at the same time. In this case, the electronic device usually controls each foreground application to draw frames according to the highest running frame rate. This is likely to cause unnecessary waste of resources. Summary of the invention

[0003] The present application provides a method for controlling an operating frame rate, an electronic device, a chip system, and a readable storage medium, which can solve the problem in the related art that when multiple application interfaces are displayed simultaneously, the electronic device controls each foreground application to draw frames at the highest operating frame rate, thereby causing a waste of resources. The technical solution is as follows:

[0004] In a first aspect, a method for controlling an operating frame rate is provided, the method comprising:

[0005] At a first moment, a first application interface of a first application is displayed. When the first application interface is displayed, the running frame rate of the first application is a first running frame rate, and the first running frame rate is determined based on the first interface name of the first application interface; at a second moment, a second interface name of a second application interface of a second application is obtained; according to the second interface name, the running frame rate of the second application is determined to obtain a second running frame rate; the second application is controlled to display a second application interface based on the second running frame rate, and the second application interface and the first application interface are displayed on the same display interface.

[0006] Exemplarily, the first moment is moment t1, and the second moment is moment t3.

[0007] In this way, the running frame rate corresponding to each application is determined according to the interface name of each application interface, so as to control each application to draw frames according to its own running frame rate, rather than drawing frames according to a unified maximum running frame rate. This can reduce the running load of some applications with a lower running frame rate and minimize unnecessary waste of resources.

[0008] As an example of the present application, an electronic device includes a layer management module; at a second moment, obtaining a second interface name of a second application interface of a second application, including: at the second moment, the second application sends a first message to the layer management module, the first message includes the interface name of the second application interface; the layer management module obtains the interface name of the second application interface from the first message.

[0009] In this way, when the second application needs to draw a frame, it sends a first message to the layer management module, and carries the interface name of the second application interface to be drawn in the first message, so that the layer management module determines the corresponding second operating frame rate according to the interface name of the second application interface.

[0010] As an example of the present application, the running frame rate of the second application is determined according to the second interface name to obtain the second running frame rate, including: the layer management module queries the corresponding interface scene type from the first mapping relationship according to the second interface name, and the first mapping relationship is used to represent the mapping relationship between different interface names and corresponding interface scene types; the layer management module determines the second running frame rate according to the queried interface scene type.

[0011] In this way, by querying the pre-configured first mapping relationship to determine the interface scene type corresponding to the second application interface, and then determining the second operating frame rate according to the interface scene type, the speed of determining the second operating frame rate can be improved.

[0012] As an example of the present application, the layer management module determines the second operating frame rate according to the queried interface scene type, including: the layer management module queries the corresponding operating frame rate from the second mapping relationship according to the queried interface scene type to obtain the second operating frame rate, and the second mapping relationship is used to represent the mapping relationship between different interface scene types and corresponding operating frame rates.

[0013] In this way, by querying the pre-configured second mapping relationship to determine the second running frame rate, the speed of determining the second running frame rate can be improved.

[0014] As an example of the present application, the first mapping relationship is established through a first intelligent recognition model, and the first intelligent recognition model can determine the corresponding operating frame rate based on any interface scene type.

[0015] In this way, by establishing the first mapping relationship through the first intelligent recognition model, the accuracy of establishing the first mapping relationship can be improved.

[0016] As an example of the present application, the layer management module determines the second operating frame rate according to the queried interface scene type, including: the layer management module calls the first intelligent recognition model, and the first intelligent recognition model can determine the corresponding operating frame rate based on any interface scene type; the layer management module determines the second operating frame rate based on the queried interface scene type through the first intelligent recognition model.

[0017] In this way, by determining the second operating frame rate based on the queried interface scene type through the first intelligent recognition model, the accuracy of determining the second operating frame rate can be improved.

[0018] As an example of the present application, the first intelligent recognition model is obtained by training the initial intelligent recognition model based on the first training sample set. Each training sample data in the first training sample set includes a training running frame rate, a training interface scene type and a frame loss rate. The frame loss rate in each training sample data is less than or equal to the first frame loss rate threshold.

[0019] In this way, the first intelligent recognition model is trained when the frame loss rate is relatively low, so that after the running frame rate is subsequently determined by the first intelligent recognition model, the frame loss rate of the drawing frame is relatively low at the determined running frame rate.

[0020] As an example of the present application, controlling the second application to display a second application interface based on a second operating frame rate includes: a layer management module sending a drawing frame signal to the second application at a time interval corresponding to the second operating frame rate, the drawing frame signal being used to indicate rendering of the second application interface; after receiving the drawing frame signal, the second application renders the second application interface; when a Vsync signal arrives, the layer management module displays the second application interface through the display screen of the electronic device.

[0021] In this way, the layer management module sends a frame drawing signal to the second application program at a time interval corresponding to the second operating frame rate, thereby controlling the second application program to draw frames at the second operating frame rate as the operating frame rate.

[0022] As an example of the present application, when a Vsync signal arrives, a layer management module displays a second application interface through a display screen of an electronic device, including: when a Vsync signal arrives, the layer management module obtains interface data of the first application interface from a first cache queue, and obtains interface data of the second application interface from a second cache queue, the first cache queue being used to cache interface data of an application interface rendered by a first application, and the second cache queue being used to cache interface data of an application interface rendered by a second application; the layer management module synthesizes the interface data of the first application interface and the interface data of the second application interface to obtain target interface data to be displayed; the layer management module sends the target interface data to the display screen to display the first application interface and the second application interface based on the target interface data through the display screen.

[0023] Thus, when the Vsync signal arrives, the frame data is obtained from the cache queues corresponding to each application and synthesized so that the electronic device can simultaneously display the application interfaces of multiple applications in one display interface, such as the first application interface and the second application interface.

[0024] As an example of the present application, after the layer management module obtains the interface data of the first application interface from the first cache queue, and obtains the interface data of the second application interface from the second cache queue, it also includes: the layer management module determines whether to release the first cache queue according to the first running frame rate, and determines whether to release the second cache queue according to the second running frame rate.

[0025] In this way, the layer management module releases the cache queue corresponding to each application according to the running frame rate of each application, instead of releasing the cache queue immediately after each acquisition of the drawing frame data, which can alleviate the pressure of Surfaceflinger releasing the cache queue.

[0026] In a second aspect, an electronic device is provided, comprising: one or more processors, and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions so that the electronic device executes a method as described in any one of the first aspects.

[0027] In a third aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes a method as described in any one of the first aspects.

[0028] According to a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes instructions, and when the instructions are executed on an electronic device, the electronic device executes the method as described in any one of the first aspects.

[0029] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the method described in the first aspect.

[0030] The technical effects obtained by the above-mentioned second, third, fourth and fifth aspects are similar to the technical effects obtained by the corresponding technical means in the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a schematic diagram of a display interface according to an exemplary embodiment;

[0032] Figure 2 is a schematic diagram showing a layer overlay relationship according to an exemplary embodiment;

[0033] Figure 3 is a schematic diagram showing an interface display process according to an exemplary embodiment;

[0034] Figure 4 is a schematic diagram of a display interface according to another exemplary embodiment;

[0035] Figure 5 is a schematic diagram of a display interface according to another exemplary embodiment;

[0036] Figure 6 is a schematic diagram showing a frame drawing rule of an application according to an exemplary embodiment;

[0037] Figure 7 is a schematic diagram of the architecture of a software system according to an exemplary embodiment;

[0038] Figure 8 is a flow chart of a method for controlling an operating frame rate according to an exemplary embodiment;

[0039] Fig. 9 is a schematic diagram showing an application scenario according to an exemplary embodiment;

[0040] Fig.10 is a flow chart of a method for controlling an operating frame rate according to another exemplary embodiment;

[0041] Fig.11 The diagram is a schematic structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0042] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0043] It should be understood that the "multiple" mentioned in this application refers to two or more. In the description of this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of this application, the words "first" and "second" are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not limit them to be different.

[0044] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0045] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0046] Before introducing the operating frame rate control method provided in the embodiment of the present application, a brief introduction to the nouns and terms involved in the embodiment of the present application is first given.

[0047] (1) Surface: It can be understood as a transparent film or a superimposed transparent sheet. Each layer can contain different image elements. Each application may correspond to one or more layers. The display interface of an electronic device can be composed of one or more layers, or in other words, a frame of image displayed on the display screen can be composed of one or more layers. Among them, multiple layers may belong to the same application or different applications, that is, the application interface of one application can be displayed in one display interface, or the application interfaces of multiple applications can be displayed simultaneously.

[0048] For example, see Figure 1 The display interface of the mobile phone can be composed of three layers: a status bar 11 at the top, a navigation bar 12 at the bottom, and an application interface 13 in the middle. The application interface 13 is the application interface of the application currently running in the foreground (referred to as: foreground application).

[0049] Among them, the status bar 11 at the top is the layer corresponding to the status bar application. The navigation bar 12 at the bottom is the layer corresponding to the navigation bar application. The application interface 13 is the layer corresponding to the foreground application, for example, the application interface 13 is the WeChat application interface of the WeChat application currently running in the foreground. That is, at this time, one display interface includes the application interface 11 of the status bar application, the application interface 12 of the navigation bar application, and the WeChat application interface 13 of the WeChat application.

[0050] It should be understood that the display interface of the electronic device can also be composed of other layers. Figure 1 Take this as an example to illustrate.

[0051] Each layer has corresponding layer properties, which may include information such as the layer's position, size, and level.

[0052] The position information is used to indicate the display position of the layer on the display screen. For example, the position information may include the pixel coordinates of each endpoint of the layer on the display screen, such as the position information of the layer includes the pixel coordinates of the four endpoints of the upper left, lower left, upper right and lower right.

[0053] Size information is used to indicate the size of the layer.

[0054] The level information is used to indicate the stacking order of layers in the vertical direction of the display screen plane. That is, the level information of each layer included in a frame of image describes the up-and-down stacking relationship between the multiple layers included in this frame of image. The larger the level information, the closer the corresponding layer is to the front in the vertical direction of the display screen plane.

[0055] For example, see Figure 2 , Figure 2This is a schematic diagram of the order of multiple layers on the Z axis provided by an embodiment of the present application. In the order from top to bottom when displayed on the display screen, it includes surface0, surface1 and surface2. Surface2 partially covers surface1, and surface1 partially covers surface0, that is, surface2 is superimposed on surface1, and surface1 is superimposed on surface0. The layer information of surface0, surface1 and surface2 is represented as: 0, 1, 2 respectively.

[0056] (2) Surfaceflinger (SF for short): is a system service that is mainly used to implement functions such as layer creation, control, and management.

[0057] In the display system of an electronic device, layer synthesis can be achieved through Surfaceflinger. For example, after the application is started, Surfaceflinger can create a layer for the application so that the application can render based on the layer, or draw frames to obtain the layer to be displayed. In addition, Surfaceflinger also allocates a corresponding cache queue for the application to cache the drawing frame data of the layer rendered by the application. During the operation of the electronic device, Surfaceflinger can obtain the layers to be displayed of each foreground application in the electronic device, and then perform layer synthesis to obtain the image data corresponding to the display interface to be displayed.

[0058] When compositing layers, Surfaceflinger can be synthesized through the hardware composer (HWC) or through the HWC and the graphics processing unit (GPU).

[0059] (3) GPU synthesis: GPU is a general-purpose image processing device. In addition to being used for layer synthesis, GPU can also be used to complete other graphics processing tasks. GPU synthesis refers to a layer synthesis method that uses the GPU to synthesize layers. In some layer synthesis strategies, when HWC synthesis cannot meet the synthesis conditions, the layers that do not meet the conditions can be synthesized by GPU, and the synthesized data can be used as the input data of HWC synthesis for secondary synthesis.

[0060] (4) HWC synthesis: refers to the layer synthesis method of synthesizing layers through HWC. HWC synthesis is hardware synthesis, that is, there is a separate piece of hardware in the electronic device to handle the synthesis of layers, and the synthesis result is directly sent to the display screen for display (i.e., sent for display). Its advantage is that the highly customized process and results can produce higher synthesis efficiency. Compared with GPU synthesis, HWC synthesis power consumption is relatively low. For example, the comparison of the synthesis method of HWC synthesis and GPU synthesis can be shown in the following Table 1:

[0061] Table 1

[0062]

[0063] As can be seen from Table 1, compared with GPU synthesis, HWC synthesis has the advantages of low power consumption, high performance, and fast synthesis speed.

[0064] (6) Original frame rate: The inherent frame rate when the content is created, such as the frame rate when shooting a video or designing a game. The original frame rate is usually fixed and determined by the content creator.

[0065] (7) Operating frame rate: refers to the speed at which an electronic device actually renders and displays images. Its unit is frames per second (FPS). The operating frame rate is a dynamic value that is affected by factors such as hardware performance and software optimization, and may not be equal to the original frame rate of the application. A higher operating frame rate can provide smoother images and better responsiveness.

[0066] (8) Screen refresh rate: the refresh rate of the display screen of an electronic device. The screen refresh rate refers to the number of times the display screen refreshes the image per second, measured in Hz (Hertz). For example, a 60Hz screen refresh rate means that the display screen can refresh the image 60 times in 1 second. The higher the screen refresh rate, the lower the image delay and the smoother the displayed image, but the power consumption will also be higher.

[0067] In some embodiments, the screen refresh rate can be pre-set, for example, the user can trigger the electronic device to set the screen refresh rate to 120Hz; in other embodiments, the screen refresh rate can also be adaptively adjusted according to the running frame rate, for example, when the running frame rate is 60FPS, the electronic device adaptively adjusts the screen refresh rate to 60Hz, and when the running frame rate is 120FPS, the electronic device adaptively adjusts the screen refresh rate to 120Hz. Generally, the running frame rate should match or be close to the screen refresh rate to obtain the best display effect. For example, on a display screen with a screen refresh rate of 120Hz, if 120FPS can be stably run, a very smooth experience can be obtained.

[0068] (9) Vertical synchronization (Vsync) signal: used to synchronize the screen refresh rate and operating frame rate to avoid screen tearing effect.

[0069] Generally speaking, in a display system, the Vsync signal can simultaneously trigger the application rendering process, the layer synthesis process, and the hardware display sending process. When the display screen completes the display of a frame of interface data, the hardware side triggers the software side to execute the next display sending process according to the Vsync signal cycle. Correspondingly, the software side simultaneously triggers the application rendering process, the layer synthesis process, and the hardware display sending process.

[0070] For example, Figure 3 FIG. 1 is a schematic diagram of a display process based on a Vsync signal according to an exemplary embodiment. Figure 3 As shown, the display system can execute the application rendering process of the i-th frame image in the Nth Vsync cycle, execute the layer synthesis process of the i-th frame image in the N+1th Vsync cycle, and execute the hardware display sending process of the i-th frame image in the N+2th Vsync cycle. In addition, when the N+1th Vsync signal arrives, that is, at the beginning of the N+1th Vsync cycle, the application rendering process of the i+1th frame image can also be started, the layer synthesis process of the i+1th frame image can be executed in the N+2th Vsync cycle, and the hardware display sending process of the i+1th frame image can be executed in the N+3rd Vsync cycle. In addition, when the N+2th Vsync signal arrives, that is, at the beginning of the N+2th Vsync cycle, the application rendering process of the i+2th frame image can also be started, the layer synthesis process of the i+2th frame image can be executed in the N+3rd Vsync cycle, and the hardware display sending process of the i+2th frame image can be executed in the N+4th Vsync cycle.

[0071] After the display is sent, the hardware side displays the interface based on the image data sent by the software side. In some scenarios, the display interface displayed by the hardware side includes the application interfaces of multiple applications. Figure 1 In addition to the application scenarios shown, the following exemplary application scenarios may also be included but are not limited to:

[0072] Scenario 1: See Figure 4 The display screen of the electronic device displays a game application interface S1 of the game application and a WeChat application interface S2 of the WeChat application. The WeChat application interface S2 is displayed in the form of a floating window superimposed on the game application interface S1.

[0073] Scenario 2: See Figure 5The display screen of the electronic device displays application interfaces of multiple applications, including an application interface S3 of the Tik Tok application displayed in an upper and lower split screen (it can also be displayed in a left and right split screen) and an application interface S4 of the gallery application, as well as an application interface S5 of the status bar application displayed at the bottom.

[0074] It should be noted that the embodiments of the present application only illustrate the case where the same display interface includes multiple application interfaces through the above-mentioned exemplary application scenarios, and do not limit the application scenarios. In another example, the same display interface including multiple application interfaces may also involve other application scenarios, such as a complex scenario that may include application interfaces of more application programs. In addition, the method provided in this embodiment can also be applied to continuous frame refresh animation scenarios implemented on the application side, such as list sliding scenarios, etc., which are not limited in the embodiments of the present application.

[0075] Typically, when an electronic device displays multiple application interfaces in one display interface, the running frame rates of these applications are uniformly adjusted based on the maximum frame rate, which refers to the maximum value of all running frame rates of these applications. Figure 5 In the application scenario shown, assuming that the running frame rate of the game application is 120FPS, the running frame rate of the WeChat application is 60FPS, and the running frame rate of the status bar application is 30FPS, then Figure 5 In the display interface process shown, Surfaceflinger controls each application to draw frames at a running frame rate of 120FPS, that is, controls the game application to draw frames at a running frame rate of 120FPS, controls the WeChat application to draw frames at a running frame rate of 120FPS, and controls the status bar application to draw frames at a running frame rate of 120FPS.

[0076] However, the WeChat application and the status bar application do not actually need to draw frames at a running frame rate of 120FPS. If the running frame rate of each application is controlled according to this implementation method, the WeChat application and the status bar application will be overloaded, resulting in unnecessary waste of resources. That is, according to the above implementation method, it is easy to cause the application with a lower running frame rate to have a large load, and there is a problem of waste of resources. In addition, in some complex business scenarios, such as when there are many application interfaces displayed in the display interface, due to the complexity of the business, some applications may not be able to draw frames based on the maximum frame rate, which can easily cause freeze problems.

[0077] To this end, an embodiment of the present application provides a method for controlling an operating frame rate, which can control each application to draw frames according to the operating frame rate corresponding to the interface name according to the interface name of the application interface to be displayed by each application. For example, the electronic device can determine the operating frame rate of each application according to the interface name of the application interface of each application through Surfaceflinger, and control each application to draw frames according to the determined operating frame rate.

[0078] For example, see Figure 6 , the display screen of the electronic device simultaneously displays the application interface of APP1, the application interface of APP2 and the application interface of APP3, and the electronic device controls each application to draw frames according to their corresponding running frame rates, such as APP1 draws frames at a running frame rate of 30FPS, APP2 draws frames at a running frame rate of 60FPS, and APP3 draws frames at a running frame rate of 120FPS.

[0079] In this way, it is avoided that each application is controlled to draw frames according to the maximum frame rate when multiple application interfaces are displayed at the same time, thereby reducing the load of the application and unnecessary waste of resources. In addition, each application can complete the frame drawing according to its corresponding running frame rate, and the problem of freeze caused by the inability to complete the frame drawing can be avoided as much as possible.

[0080] For ease of understanding, the software system of the electronic device involved in the embodiment of the present application is first introduced. The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system with a layered architecture as an example to exemplify the software system of the electronic device.

[0081] Figure 7 is a block diagram of a software system of an electronic device provided in an embodiment of the present application. Figure 7 The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer (App layer), application framework layer (FWK layer), hardware abstract layer (HAL layer), and kernel layer.

[0082] The application layer can include a series of application packages. Figure 7As shown, the application package may include games, status bar, WeChat, video, vibrato, gallery, calendar, call, map, music, short message and other applications. It is used to interact directly with the user. These applications can be system built-in applications or non-system level applications. And these applications can have icons and application interfaces, or have application interfaces without icons, or have neither icons nor application interfaces.

[0083] The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some predefined functions. As an example of the present application, the application framework layer may include a layer management module, etc. The layer management module is used to implement functions such as layer creation, control, and management. For example, the layer management module may be Surfaceflinger. Surfaceflinger is a system service, which may be called a layer deliverer in some embodiments.

[0084] That is, in the display system of the electronic device, the synthesis of layers can be achieved through Surfaceflinger. During the operation of the electronic device, Surfaceflinger can obtain the layers to be displayed by various applications running on the electronic device, such as obtaining the drawing frame data of each layer in the layer to be synthesized from the cache queue, and then decide whether to synthesize through HWC, or through HWC and GPU synthesis, and synthesize the obtained drawing frame data according to the corresponding layer synthesis strategy based on the decision result to obtain a frame of image data to be displayed, and then this frame of image data can be sent for display.

[0085] The hardware abstraction layer is located between the application framework layer and the kernel layer, and its purpose is to abstract the hardware. The hardware abstraction layer hides the hardware interface details of a specific platform and provides a virtual hardware platform for the software system, making it hardware-independent and portable on multiple platforms. As an example of this application, the hardware abstraction layer includes HWC, which is responsible for managing and organizing layers from different applications and deciding how to best combine them, that is, responsible for layer synthesis and display. HWC can use hardware acceleration for layer synthesis.

[0086] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver. These drivers can drive the display screen, front / rear camera, audio player, sensor, etc.

[0087] In other embodiments, the system of the electronic device may further include a system library layer, which is located between the framework layer and the kernel layer, and is not limited here. The system library may include multiple functional modules. For example, the system library may include a surface manager, a media library, a three-dimensional graphics engine (e.g., OpenGLES), and a 2D graphics engine. The surface manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static graphics files, etc. The three-dimensional graphics engine is used to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0088] also, Figure 7 The relationship between the software system and the hardware layer is also shown. The hardware layer may include physical devices such as a GPU and a display screen.

[0089] exist Figure 7 Based on the electronic device shown, the following describes the operating frame rate control method provided by the embodiment of the present application. Figure 8 , Figure 8 is a flowchart of a method for controlling a frame rate according to an exemplary embodiment. As an example but not a limitation, the method can be applied to Figure 7 The electronic device shown can be implemented by multiple modules interacting with each other. The method may include some or all of the following contents:

[0090] S801: The first application sends a message message1 to Surfaceflinger, where the message message1 carries the interface name of the first application interface to be displayed.

[0091] Exemplarily, after the user triggers a first user operation, the first application sends a message message1 to Surfaceflinger, wherein the first user operation is used to trigger the electronic device to display a first application interface of the first application.

[0092] The first application may be any application running in the foreground of the electronic device, and the first application interface is the application interface currently displayed by the first application. For example, the first application is a game application in the electronic device, and the first application interface to be displayed may be a game application interface, such as an interface of a game in progress.

[0093] As an example, when the first application needs to display the first application interface, for example, when the interface content changes, a message messag1 can be sent to Surfaceflinger, and the message messag1 carries the interface name of the first application interface, for example, the interface name of the first application interface is name1. Message messag1 is used to request Surfaceflinger to trigger the first application to draw a frame.

[0094] S802: Surfaceflinger determines the interface scene type of the first application interface according to the interface name in message message1.

[0095] The interface scene type of the first application interface refers to the type of scene corresponding to the interface content of the first application interface. As an example and not a limitation, the interface scene type of the first application interface can be but is not limited to video, news, text, video call or game.

[0096] As an example of the present application, a mapping relationship 1 between an interface name and an interface scene type is stored in an electronic device. Thus, after Surfaceflinger receives the message messag1, based on the interface name name1 of the first application interface carried in the message messag1, a specific implementation of determining the interface scene type of the first application interface may include: Surfaceflinger queries the interface scene type corresponding to the interface name name1 from the mapping relationship 1, and uses the queried interface scene type as the interface scene type of the first application interface.

[0097] It should be noted that the implementation method for determining the interface scene type provided in the embodiment of the present application is only exemplary. In another embodiment, it can also be determined in other ways. For example, Surfaceflinger can input the interface name name1 into a pre-trained classification model, and output the corresponding interface scene type through the classification model, wherein the classification model can determine the corresponding interface scene type based on any interface name, and the embodiment of the present application does not limit this.

[0098] S803: Surfaceflinger determines a corresponding first operating frame rate according to the interface scene type of the first application interface.

[0099] In one example, a mapping relationship 2 between interface scene types and operating frame rates may be stored in the electronic device. Thus, after determining the interface scene type corresponding to the interface name name1, Surfaceflinger may query the operating frame rate corresponding to the determined interface scene type from the mapping relationship 2, and use the queried operating frame rate as the first operating frame rate.

[0100] Exemplarily, the mapping relationship 2 between the interface scene type and the running frame rate can be as shown in Table 2.

[0101] Table 2

[0102]

[0103] Exemplarily, if the interface scene type is a game, by querying mapping relationship 2, it can be determined that the corresponding operating frame rate is 120 Hz, and Surfaceflinger determines that the first operating frame rate is 120 Hz.

[0104] It should be noted that the embodiment of the present application is only described by taking the mapping relationship 2 represented by the form of Table 2 as an example. In another example, the mapping relationship 2 can also be represented by other forms, such as each operating frame rate corresponds to an interface scene type set, and the interface scene type set includes at least one interface scene type. Exemplarily, the interface scene type set corresponding to the operating frame rate of 120FPS is {game, news, ...}, which is not limited in the embodiment of the present application.

[0105] It is worth mentioning that the above-mentioned mapping relationship 2 between the static configuration interface scene type and the running frame rate enables when Surfaceflinger needs to determine the running frame rate in real time, it can be determined directly by querying the mapping relationship 2, thereby improving the timeliness of determining the running frame rate.

[0106] As an example but not limitation, mapping relationship 2 can be established through a first intelligent recognition model, which is a pre-trained model that can determine the corresponding operating frame rate based on any interface scene type. For example, in implementation, the electronic device can obtain multiple different interface scene types, and then input each of the multiple different interface scene types into the first intelligent recognition model, and output the operating frame rate corresponding to each interface scene type through the first intelligent recognition model. The electronic device establishes a mapping relationship between each interface scene type and the corresponding operating frame rate to obtain mapping relationship 2.

[0107] It is worth mentioning that by establishing the mapping relationship 2 through the pre-trained first intelligent recognition model, the reliability of establishing the mapping relationship can be improved.

[0108] In another example, Surfaceflinger can also determine the first operating frame rate based on the queried interface scene type through the first intelligent recognition model. For example, Surfaceflinger can input the queried interface scene type into the first intelligent recognition model for recognition processing, and the first intelligent recognition model outputs the first operating frame rate. Exemplarily, if the queried interface scene type is "game", the queried interface scene type is input into the first intelligent recognition model for recognition processing, and the first intelligent recognition model outputs 120, so the first operating frame rate can be determined to be 120FPS.

[0109] It is worth mentioning that, by determining the first operating frame rate through the first intelligent recognition model, the accuracy of determining the first operating frame rate can be improved.

[0110] In one example, the first intelligent recognition model can be obtained by training an initial intelligent recognition model based on a first training sample set, and each training sample data in the first training sample set includes a training running frame rate, a training interface scene type, and a frame loss rate, and the frame loss rate in each training sample data is less than or equal to a first frame loss rate threshold.

[0111] The first frame loss rate threshold can be set according to requirements. For example, the first frame loss rate threshold can be 2%.

[0112] During the training process, the electronic device first obtains the first training sample set. In the process of obtaining the first training sample set, a certain training running frame rate can be used to draw frames for the training samples of each training interface scene type, and then the frame loss rate of the training sample is detected. If the frame loss rate is less than or equal to the first frame loss rate threshold, the training interface scene type, training running frame rate and frame loss rate corresponding to the training sample are obtained as a set of training sample data. If the frame loss rate is greater than the first frame loss rate threshold, the training running frame rate corresponding to the training sample can be reduced, and then the frame loss rate of the training sample at the adjusted training running frame rate is continued to be detected. If the frame loss rate at this time is still greater than the first frame loss rate threshold, the training running frame rate corresponding to the training sample is continued to be reduced, and the frame loss rate of the training sample at the training running frame rate after the adjustment is continued to be detected until the frame loss rate is less than or equal to the first frame loss rate threshold, and the training interface scene type corresponding to the training sample, the last adjusted training running frame rate and the last detected frame loss rate are obtained as a set of training sample data.

[0113] As an example, the frame loss rate can be determined based on the single frame drawing time. If the single frame drawing time exceeds the time interval corresponding to the training running frame rate, it is determined that the frame is lost. Otherwise, if the single frame drawing time exceeds the time interval corresponding to the training running frame rate, it is determined that the frame is lost. The number of frame losses within a period of time can be counted to determine the frame loss rate.

[0114] Exemplarily, the electronic device may draw frames at a running frame rate of 120FPS for training samples of news interface scene types, and then detect the frame loss rate, such as rate1. If the frame loss rate is less than or equal to 2%, the news interface scene type, 120FPS, and frame loss rate rate1 are used as a set of training sample data. For another example, the electronic device may draw frames at a running frame rate of 120FPS for training samples of video interface scene types, and then detect the frame loss rate, such as rate2. If the frame loss rate is greater than 2%, the electronic device may redraw frames at a running frame rate of 60FPS for training samples of video interface scene types, and then detect the frame loss rate, such as rate3. If the frame loss rate is less than or equal to 2%, the video interface scene type, 60FPS, and frame loss rate rate3 are used as a set of training sample data.

[0115] In this way, multiple groups of training sample data can be obtained, and the electronic device thereby determines a first training sample set, which includes the multiple groups of training sample data.

[0116] After obtaining the first training sample set, the first training sample set can be input into the initial intelligent recognition model for iterative training. When the training end conditions are met, such as the number of iterative training reaches the training number threshold (which can be set according to needs), or the recognition performance of the trained initial intelligent recognition model meets the requirements, the electronic device determines the trained initial intelligent recognition model as the first intelligent recognition model, that is, the trained initial intelligent recognition model can determine the corresponding operating frame rate based on any interface scene type.

[0117] It is worth mentioning that the first intelligent recognition model is trained under the condition of a low frame loss rate, so that in the application process, after the running frame rate is determined by the first intelligent recognition model, the frame loss rate of the drawing frame is low under the determined running frame rate.

[0118] It should be noted that S802 to S803 are optional operations, and the embodiment of the present application is only described by taking the mapping relationship 1 and the mapping relationship 2 in the electronic device as an example. In another example, Surfaceflinger can also determine the first running frame rate based on the interface name by other means. For example, in one example, a mapping relationship 3 between the interface name and the running frame rate can also be established in the electronic device. In this way, after Surfaceflinger receives the message message1, the running frame rate corresponding to the interface name name1 can be queried from the mapping relationship 3, and the queried running frame rate is determined as the first running frame rate. In another example, Surfaceflinger can also determine the first running frame rate based on the interface name name1 through a pre-trained second intelligent recognition model, and the second intelligent recognition model is a model that can determine the corresponding running frame rate based on any interface name. Among them, the second intelligent recognition model can be obtained by training the initial intelligent recognition model based on the second training sample set, and each training sample data in the second training sample set includes a training running frame rate, a training interface name and a frame loss rate. Similarly, the frame loss rate in each training sample data is less than or equal to the first frame loss rate threshold. For the specific training process, please refer to the training process of the first intelligent recognition model.

[0119] S804: If Surfaceflinger determines that the drawing frame timing of the first application has arrived according to the first running frame rate, a drawing frame signal Vsync-APP1 is sent to the first application.

[0120] The Vsync signal is triggered by the hardware side according to the screen refresh rate, that is, the Vsync signal period corresponds to the screen refresh rate. For example, when the screen refresh rate is 120Hz, the Vsync signal period is 8.3ms, and when the screen refresh rate is 60Hz, the Vsync signal period is 16.6ms.

[0121] The frame drawing signal Vsync-APP1 is generated based on the Vsync signal, and is used to trigger the first application to draw a frame.

[0122] After Surfaceflinger determines the first operating frame rate, it can determine whether the frame drawing timing of the first application is reached based on the first operating frame rate and the screen refresh rate. If the frame drawing timing of the first application is reached, Surfaceflinger sends a frame drawing signal Vsync-APP1 to the first application to trigger the first application to draw the first application interface.

[0123] For example, when the first operating frame rate is 120FPS, if the screen refresh rate is 120Hz, Surfaceflinger determines the frame drawing timing of the first application whenever it receives a Vsync signal, and therefore sends a frame drawing signal Vsync-APP1 to the first application to trigger the first application to draw the first application interface. For another example, when the first operating frame rate is 60FPS, if the screen refresh rate is 120Hz, Surfaceflinger does not send a frame drawing signal Vsync-APP1 to the first application every time it receives a Vsync signal, but sends a frame drawing signal Vsync-APP1 to the first application every time it receives a Vsync signal. For example, if Surfaceflinger sends a drawing frame signal Vsync-APP1 to the first application when the Mth Vsync signal arrives to trigger the first application to draw the first application interface, when the M+1th Vsync signal arrives, Surfaceflinger does not send the drawing frame signal Vsync-APP1 to the first application. When the M+2th Vsync signal arrives, Surfaceflinger determines that the drawing frame timing has arrived for the first application, so it sends the drawing frame signal Vsync-APP1 to the first application at the M+2th Vsync signal to trigger the first application to draw the first application interface.

[0124] S805: If the frame drawing signal Vsync-APP1 is received, the first application performs frame drawing to obtain frame drawing data.

[0125] In one example, after the first application is started, it can send a layer creation request to Surfaceflinger. Accordingly, after receiving the layer creation request of the first application, Surfaceflinger can create a layer for the first application, such as creating layer 1. Afterwards, Surfaceflinger can send the ID of layer 1 (such as ID1) to the first application, so that when the first application needs to draw a frame, it can identify layer 1 based on ID1 and then perform a frame drawing operation based on layer 1.

[0126] In addition, in addition to creating layer 1 for the first application, Surfaceflinger can also allocate a cache queue for the first application, such as allocating cache queue 1, which is used to cache drawing frame data. The number of cache queues 1 can be multiple, for example, the number of cache queues 1 allocated to the first application is 4, and the 4 cache queues 1 can be used cyclically.

[0127] After the first application draws a frame, the frame data of the drawn layer is stored in the cache queue 1.

[0128] S806: Surfaceflinger obtains drawing frame data.

[0129] After Surfaceflinger instructs the first application to draw a frame, when the Vsync signal is received again, the drawing frame data is obtained from cache queue 1.

[0130] As an example but not a limitation, after Surfaceflinger obtains the drawing frame data from the cache queue 1, the cache queue 1 may be released so that the released cache queue 1 can be used for the next drawing frame of the first application.

[0131] S807: Surfaceflinger sends drawing frame data to HWC.

[0132] S808: The HWC performs synthesis processing on the drawing frame data to obtain image data to be displayed.

[0133] In one possible implementation, Surfaceflinger can send relevant information of the drawing frame data to HWC to request HWC to determine the layer synthesis strategy, wherein the relevant information may include but is not limited to the number of layers, the amount of drawing frame data, and other information. HWC can determine the layer synthesis strategy based on the relevant information and its own synthesis capability, and then send the layer synthesis strategy to Surfaceflinger. Usually, when the number of layers is small, the layer synthesis strategy is HWC synthesis, that is, GPU synthesis is not required, which can reduce the power consumption of GPU synthesis. When the number of layers is large, due to the limited synthesis capability of HWC, the layer synthesis strategy is generally GPU synthesis and HWC synthesis.

[0134] As an example, when Surfaceflinger determines that the layer synthesis strategy is HWC synthesis according to the layer synthesis strategy, the drawing frame data is sent to the HWC for synthesis processing by the HWC.

[0135] S809: The HWC sends the drawing frame data to the display screen to display the first application interface through the display screen.

[0136] As an example, Surfaceflinger instructs the first application to draw frames when the Mth Vsync signal arrives, synthesizes the drawing frame data of the first application through HWC when the M+1th Vsync signal arrives, and sends the synthesized image data to the display through HWC when the M+2th Vsync signal arrives, so that the display displays the first application interface.

[0137] S810: The display screen displays the first application interface.

[0138] In this way, the display of the first application interface is realized.

[0139] When displaying the first application interface, the user may also trigger the electronic device to simultaneously display the second application interface of the second application program. For the specific display process, please refer to the following steps.

[0140] S811: After the user triggers the second user operation, the second application sends a message message2 to Surfaceflinger, where the message message2 carries the interface name of the second application interface to be displayed.

[0141] The second user operation is used to trigger the electronic device to display a second application interface of the second application, and the second application interface is to be displayed in the same display interface as the first application interface.

[0142] That is, during the process of displaying the first application interface, when the user wants the electronic device to simultaneously display the second application interface of the second application, a second user operation can be triggered.

[0143] As an example but not limitation, the second user operation may be a split-screen display operation; or the second user operation may also be a multi-window overlay display operation. For example, the second user operation may be a swipe down operation on a notification message received by the electronic device, etc. The embodiments of the present application are not limited to this.

[0144] The second application may be an application other than the first application in the electronic device, and the second application interface is an application interface to be displayed by the second application. For example, the second application may be an application such as WeChat, address book, short message, etc. When the second application is a WeChat application, the second application interface is a WeChat application interface, such as a WeChat chat interface.

[0145] As an example, when the second application needs to display the second application interface, it can send a message messag2 to Surfaceflinger, and the message messag2 carries the interface name of the second application interface, for example, the interface name of the second application interface is name2. Message Messag2 is used to request Surfaceflinger to trigger the second application to draw a frame.

[0146] S812: Surfaceflinger determines the second running frame rate according to the interface name in the message message2.

[0147] In an example, Surfaceflinger can determine the corresponding interface scene type by querying mapping relationship 1 according to the interface name name2, and determine the second running frame rate by querying mapping relationship 2 according to the scene interface type.

[0148] In another example, Surfaceflinger can determine the corresponding interface scene type according to the interface name name2 by querying the mapping relationship 1, and determine the second running frame rate based on the queried interface scene type through the first intelligent recognition model.

[0149] In another example, Surfaceflinger may determine the corresponding running frame rate according to the interface name name2 by querying the mapping relationship 3, thereby obtaining the second running frame rate.

[0150] In another example, Surfaceflinger can determine the second operating frame rate based on the interface name name2 through a second intelligent recognition model.

[0151] For the specific implementation of the above-mentioned implementation methods, please refer to S802 to S803, that is, the specific implementation of Surfaceflinger determining the second running frame rate according to the interface name name2 in the message message2 can be referred to the specific implementation of Surfaceflinger determining the first running frame rate according to the interface name name1 in the message message1.

[0152] S813: If Surfaceflinger determines that the drawing frame timing of the second application has arrived according to the second running frame rate, a drawing frame signal Vsync-APP2 is sent to the second application.

[0153] The frame drawing signal Vsync-APP2 is generated based on the Vsync signal, and is used to trigger the second application to draw frames.

[0154] After Surfaceflinger determines the second operating frame rate, it can trigger the second application to draw the second application interface according to the time interval corresponding to the second operating frame rate. For example, when the second operating frame rate is 120FPS, if the screen refresh rate is 120Hz, Surfaceflinger determines the frame drawing timing of the second application every time it receives the Vsync signal, so it sends the frame drawing signal Vsync-APP2 to the second application to trigger the second application to draw the second application interface. For another example, when the second running frame rate is 30FPS, if the screen refresh rate is 120Hz, Surfaceflinger determines the frame drawing timing of the second application when receiving the M+kth Vsync signal, so it sends the frame drawing signal Vsync-APP2 to the second application to trigger the second application to draw the frame. The second application does not send the frame drawing signal Vsync-APP2 to the second application when the M+k+1th Vsync signal arrives, nor does it send the frame drawing signal Vsync-APP2 to the second application when the M+k+2th Vsync signal is received, nor does it send the frame drawing signal Vsync-APP2 to the second application when the M+k+3th Vsync signal is received, but sends the frame drawing signal Vsync-APP2 to the second application when the M+k+4th Vsync signal is received to trigger the second application to draw the second application interface. Wherein, k is an integer greater than 1.

[0155] S814: If the frame drawing signal Vsync-APP2 is received, the second application performs frame drawing to obtain frame drawing data.

[0156] In one example, after the second application is started, it can send a layer creation request to Surfaceflinger. Accordingly, after receiving the layer creation request of the second application, Surfaceflinger can create a layer for the second application, such as creating layer 2. Afterwards, Surfaceflinger can send the ID of layer 2 (such as ID2) to the second application, so that when the second application needs to draw a frame, it can identify layer 2 based on ID2 and then perform a frame drawing operation based on layer 2.

[0157] Similarly, in addition to creating layer 2 for the second application, Surfaceflinger can also allocate cache queues for the second application, such as Cache Queue 2, which is used to cache frame data. There can be multiple cache queues 2, and the number of cache queues 2 can be the same as the number of cache queues 1, or different from the number of cache queues 1. For example, the number of cache queues 2 allocated by Surfaceflinger to the second application is also 4, and the 4 cache queues 2 can be used cyclically.

[0158] It should be noted that, in the process of Surfaceflinger controlling the second application to draw frames according to the second running frame rate, Surfaceflinger also controls the first application to draw frames according to the first running frame rate, that is, the running frame rates of the first application and the second application are different. Figure 6 , the first application is APP3, and the second application is APP2. As an example, Surfaceflinger controls the first application to draw a frame every 8.3ms, and controls the second application to draw a frame every 16.6ms.

[0159] S815: Surfaceflinger obtains drawing frame data of the second application interface.

[0160] After Surfaceflinger instructs the second application to draw a frame, when the Vsync signal is received again, the drawing frame data of the second application can be obtained from cache queue 2. In addition, since the application interface of the first application needs to be displayed at the same time, Surfaceflinger also obtains the drawing frame data of the first application interface from cache queue 1.

[0161] In the case where there are multiple application interfaces to be displayed, after Surfaceflinger obtains the drawing frame data from the cache queue corresponding to each application, it can decide whether to release the cache queue of each application according to the running frame rate of each application. As an example, when the running frame rates of multiple applications are different, after Surfaceflinger obtains the drawing frame data from the cache queue corresponding to each application, for any application among the multiple applications, if the running frame rate of the application is equal to the screen refresh rate, then each time the drawing frame data is obtained from the cache queue of the application, the cache queue of the application is released, otherwise, if the running frame rate of the application is not equal to the screen refresh rate, the screen refresh rate is divided by the running frame rate of the application to obtain a first value, and after Surfaceflinger obtains the first value of drawing frame data from the cache queue of the application, the cache queue of the application is released.

[0162] For example, the first running frame rate of the first application is 120FPS, the second running frame rate of the second application is 60FPS, and the screen refresh rate is 120Hz. Surfaceflinger releases cache queue 1 each time it obtains drawing frame data from cache queue 1, but Surfaceflinger does not release cache queue 2 each time it obtains drawing frame data from cache queue 2. Instead, it releases cache queue 2 only after obtaining drawing frame data from cache queue 2 twice. In this way, the pressure on Surfaceflinger to release the cache queue can be alleviated.

[0163] S816: Surfaceflinger sends the acquired drawing frame data to HWC.

[0164] S817: The HWC synthesizes the drawing frame data to obtain image data to be displayed.

[0165] In a possible implementation, Surfaceflinger may synthesize the acquired drawing frame data through HWC, that is, synthesize the drawing frame data of the first application and the drawing frame data of the second application through HWC to obtain image data to be displayed.

[0166] It should be noted that the embodiment of the present application is described by taking Surfaceflinge synthesizing the acquired drawing frame data through HWC as an example. In another example, if the synthesis process cannot be completed only through HWC, the synthesis can be performed through the GPU first, and then the synthesized data is sent to the HWC, and the final image data to be displayed is synthesized through the HWC, which is not limited in the embodiment of the present application.

[0167] S818: The HWC sends image data to the display screen.

[0168] As an example, Surfaceflinger instructs the second application to draw frames when the M+kth Vsync signal arrives, synthesizes the drawing frame data of each application through HWC when the M+k+1th Vsync signal arrives, and sends the synthesized image data to the display screen through HWC when the M+k+2th Vsync signal arrives, so that the display screen displays the first application interface and the second application interface.

[0169] S819: The display screen displays the first application interface and the second application interface based on the image data.

[0170] For example, see Figure 4 The display screen displays the first application interface S1 and the second application interface S2 in superposition.

[0171] In an embodiment of the present application, when the application interfaces of multiple applications are displayed simultaneously in an electronic device, the operating frame rate corresponding to each application is determined according to the interface name of each application interface, so as to control each application to draw frames according to its own operating frame rate, rather than drawing frames according to a unified maximum operating frame rate. This can reduce the operating load of some applications that have a lower operating frame rate and minimize unnecessary waste of resources.

[0172] It should be noted that the embodiment of the present application is only described by taking the display interface of the display screen including the first application interface and the second application interface as an example. In another example, the display interface of the display screen may also include application interfaces of more application programs. In this case, the electronic device can control other application programs to draw frames through Surfaceflinger according to the above implementation method, that is, the implementation principle is similar, and the embodiment of the present application does not limit this.

[0173] When there are multiple application interfaces to be displayed on the same display interface, whenever Surfaceflinger receives a first message (such as message message1 or message message2) sent by any application to request a trigger for drawing frames, Surfaceflinger determines the running frame rate of any application based on the interface name carried in the first message, and controls any application to draw frames according to the determined running frame rate, avoiding unnecessary waste of resources.

[0174] For ease of understanding, let's combine Fig. 9 The application scenario shown here introduces the running frame rate control method provided by the embodiment of the present application. Fig.10 , the method may include some or all of the following:

[0175] S1001: At time t1, the electronic device displays a game application interface of the game application. When the game application interface is displayed, the running frame rate of the game application is 120FPS.

[0176] As an example, a user triggers the electronic device to display the game application interface in full screen, such as Fig. 9 In (a), the electronic device displays the game application interface S1 in full horizontal screen.

[0177] Among them, the specific implementation of the game application interface that displays the game application can be found in the previous article Figure 8 S801 to S810 in the illustrated embodiment.

[0178] S1002: At time t2, after the electronic device receives the WeChat notification, it displays the WeChat notification at the top of the display screen.

[0179] Time t2 is later than time t1, that is, time t2 is after time t1.

[0180] For example, see Fig. 9 In (b), during the process of displaying the game application interface S1, the electronic device receives a WeChat notification 12 and displays the WeChat notification 12 at the top of the display screen.

[0181] S1003: At time t3, after receiving the user's pull-down operation on the WeChat notification, the electronic device determines the interface name of the WeChat application interface of the WeChat application program.

[0182] Time T3 is later than time t2, that is, time t3 is after time t2.

[0183] See also Fig. 9 In (b), when the user wants to overlay the WeChat application interface on the game application interface S1, the user can pull down the WeChat notification 12 from the top of the display screen. Accordingly, in response to the user's pull-down operation on the WeChat notification 12, the electronic device determines the interface name of the WeChat application interface to be displayed.

[0184] As an example of this application, the electronic device determines the interface name of the WeChat application interface through Surfaceflinger. For its specific implementation, please refer to Figure 8 S811 in the embodiment.

[0185] S1004: The electronic device determines that the running frame rate of the WeChat application is 60FPS based on the interface name of the WeChat application interface.

[0186] The electronic device determines that the running frame rate of the WeChat application is 60FPS according to the interface name of the WeChat application interface. For specific implementation, please refer to Figure 8 S812 in the embodiment.

[0187] S1005: The electronic device controls the WeChat application to draw frames at a running frame rate of 60FPS.

[0188] For its specific implementation, please refer to Figure 8 S813 to S815 in the illustrated embodiment.

[0189] S1006: At time t4, the electronic device displays the game application interface and the WeChat application interface. At this time, the running frame rate of the game application is 120FPS, and the running frame rate of the WeChat application is 60FPS.

[0190] Time t4 is later than time t3, that is, time t4 is after time t3.

[0191] See also Fig. 9In (c), the electronic device displays the game application interface and the WeChat application interface at the same time. In this process, the electronic device controls the game application and the WeChat application to draw frames according to their respective corresponding running frame rates. For its specific implementation, please refer to Figure 8 S816 to S819 in the illustrated embodiment.

[0192] In an embodiment of the present application, at a first moment, a first application interface of a first application is displayed, and when the first application interface is displayed, the running frame rate of the first application is a first running frame rate, and the first running frame rate is determined based on the first interface name of the first application interface. At a second moment, a second interface name of a second application interface of a second application is obtained, and a second running frame rate is determined based on the second application interface name, and the second application is controlled to display a second application interface based on the second running frame rate, and the second application interface and the first application interface are displayed on the same display interface. In this way, when the application interfaces of multiple applications are displayed in one display interface, by controlling each application to draw frames according to its corresponding running frame rate, the running load of the application with a lower running frame rate can be reduced, and unnecessary waste of resources can be minimized.

[0193] It should be noted that the embodiment of the present application is described by taking the example of an electronic device superimposing and displaying the application interface of a WeChat application in the application interface of a game application. In another example, the application interfaces of more applications can also be displayed in one display interface, and the display principle is similar to the above display principle.

[0194] The electronic devices involved in the embodiments of the present application may be mobile phones, sports cameras (GoPro), digital cameras, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, vehicle-mounted equipment, ultra-mobile personal computers (ultra-mobile personal computers, UMPC), netbooks, cellular phones, personal digital assistants (personal digital assistants, PDA), augmented reality (augmented reality, AR) \ virtual reality (virtual reality, VR) devices, etc., and the embodiments of the present application are not limited to this.

[0195] Fig.11 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.11The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0196] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0197] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0198] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0199] In some embodiments, the processor 110 may include one or more interfaces, such as an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0200] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193 and the wireless communication module 160. In some other embodiments, the power management module 141 can also be set in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0201] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0202] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0203] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is an integer greater than 1.

[0204] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0205] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0206] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created by the electronic device 100 during use (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0207] The electronic device 100 can implement audio functions, such as music playback, recording, etc., through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D and the application processor.

[0208] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect a touch operation acting on or near it. The touch sensor 180K can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194.

[0209] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0210] An embodiment of the present application also provides a chip system, which is applied to an electronic device. The chip system includes one or more processors, and the one or more processors are used to call computer instructions to enable the electronic device to execute the methods described in the above embodiments.

[0211] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (such as: coaxial cable, optical fiber, data subscriber line (Digital Subscriber Line, DSL)) or wireless (such as: infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a digital versatile disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)).

[0212] The above are optional embodiments provided for the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the technical scope disclosed in the present application shall be included in the protection scope of the present application.

Claims

1. A method for controlling an operating frame rate, characterized in that: Applied to an electronic device, the electronic device includes a layer management module, and the method includes: At a first moment, a first application interface of a first application is displayed, and when the first application interface is displayed, the running frame rate of the first application is a first running frame rate, and the first running frame rate is determined based on a first interface name of the first application interface; At a second moment, the second application sends a first message to the layer management module, where the first message includes an interface name of a second application interface of the second application; The layer management module obtains the interface name of the second application interface from the first message; The layer management module searches for a corresponding interface scene type from a first mapping relationship according to the interface name of the second application interface, where the first mapping relationship is used to represent a mapping relationship between different interface names and corresponding interface scene types; The layer management module determines a second running frame rate according to the queried interface scene type; When the i-th Vsync signal arrives, the layer management module sends a drawing frame signal to the second application according to the time interval corresponding to the second operating frame rate, wherein the drawing frame signal is used to instruct rendering of the second application interface, the Vsync signal is triggered based on the screen refresh rate, and i is an integer greater than or equal to 1; After receiving the drawing frame signal, the second application renders the second application interface; When the (i+1)th Vsync signal arrives, the layer management module obtains the interface data of the first application interface from the first cache queue, and obtains the interface data of the second application interface from the second cache queue, the first cache queue is used to cache the interface data of the application interface rendered by the first application, and the second cache queue is used to cache the interface data of the application interface rendered by the second application; The layer management module synthesizes the interface data of the first application interface and the interface data of the second application interface to obtain target interface data to be displayed; When the (i+2)th Vsync signal arrives, the layer management module sends the target interface data to the display screen, so as to display the first application interface and the second application interface based on the target interface data through the display screen, and the second application interface and the first application interface are displayed on the same display interface; The layer management module determines whether to release the first cache queue according to the first operating frame rate, and determines whether to release the second cache queue according to the second operating frame rate.

2. The method according to claim 1, characterized in that The layer management module determines the second running frame rate according to the queried interface scene type, including: The layer management module queries the corresponding operating frame rate from the second mapping relationship according to the queried interface scene type to obtain the second operating frame rate. The second mapping relationship is used to represent the mapping relationship between different interface scene types and corresponding operating frame rates.

3. The method according to claim 2, characterized in that The first mapping relationship is established through a first intelligent recognition model, and the first intelligent recognition model can determine the corresponding running frame rate based on any interface scene type.

4. The method according to claim 1, characterized in that The layer management module determines the second running frame rate according to the queried interface scene type, including: The layer management module calls a first intelligent recognition model, and the first intelligent recognition model can determine a corresponding running frame rate based on any interface scene type; The layer management module determines the second operating frame rate based on the queried interface scene type through the first intelligent recognition model.

5. The method according to claim 3 or 4, characterized in that The first intelligent recognition model is obtained by training an initial intelligent recognition model based on a first training sample set. Each training sample data in the first training sample set includes a training running frame rate, a training interface scene type and a frame loss rate. The frame loss rate in each training sample data is less than or equal to a first frame loss rate threshold.

6. An electronic device, characterized in that: The electronic device includes: one or more processors, and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method as described in any one of claims 1-5.

7. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 5.

Images