Processing method and device

Abstract

The invention discloses a processing method and device, and relates to the technical field of application management and resource scheduling, and the method comprises the steps: determining to execute a current frame drawing operation, and monitoring the execution state of the drawing operation; and in response to the execution state characterizing that an overtime event of the current frame drawing operation will occur, the target hardware resource capability is increased to accelerate the drawing operation on the current frame, and the target hardware resource is used for the frame drawing operation.

Description

Technical Field

[0001] The present application relates to the technical field of application management and resource scheduling, and in particular to a processing method and device. Background Art

[0002] Electronic devices like mobile phones and tablets often have a variety of apps installed, including games, videos, communications, and office applications, to meet users' daily and work needs. When these apps run on these devices and perform related operations (such as 3D rendering), they often experience slow performance and lag due to insufficient system resources, impacting both application performance and user experience. Summary of the Invention

[0003] To this end, this application discloses the following technical solutions:

[0004] A processing method comprising:

[0005] Determining to execute a current frame drawing operation, and monitoring an execution status of the drawing operation;

[0006] In response to the execution state indicating that a timeout event will occur for the current frame drawing operation, increasing target hardware resource capabilities to accelerate the drawing operation for the current frame;

[0007] The target hardware resources are used for frame drawing operations.

[0008] Optionally, determining to execute the current frame drawing operation and monitoring the execution status of the drawing operation include:

[0009] At a first moment of starting to execute a drawing operation of a current frame, monitoring the drawing operation is started simultaneously, and an execution state of the drawing operation is acquired at a second moment of the drawing operation, where the second moment is later than the first moment;

[0010] The second time is configured differently for current frames of different applications or different types of current frames of the same application.

[0011] Optionally, increasing the target hardware resource capability to accelerate the drawing operation of the current frame includes:

[0012] Determine a first hardware resource of a first thread and a second hardware resource corresponding to a second thread of an application corresponding to a current frame, and increase resource capabilities of the first hardware resource and the second hardware resource;

[0013] The first thread and the second thread are created in response to the application corresponding to the current frame being started, the first thread is used to initiate the drawing operation based on the first hardware resource, and the second thread is used to execute the drawing operation based on the second hardware resource.

[0014] Optionally, determining to execute the current frame drawing operation and monitoring the execution status of the drawing operation include:

[0015] obtaining a frame drawing configuration parameter of the current application through a third thread, where the third thread is created when the current application is started;

[0016] Feedback information of the second thread's drawing operation on the current frame is obtained through the third thread, and whether the timeout event will occur is determined based on the frame configuration parameters and the feedback information.

[0017] Optionally, obtaining the frame drawing configuration parameters of the current application through a third thread includes:

[0018] The third thread obtains the frame drawing configuration parameters of the current application from the policy service module based on intra-system communication, and the third thread determines the second time according to the frame drawing configuration parameters;

[0019] Different applications have different frame drawing configuration parameters and different second moments.

[0020] Optionally, the processing method further includes:

[0021] Controlling the third thread to be in a first state at the first moment;

[0022] At the second moment, the third thread is controlled to be in a second state, so as to determine an execution state of the drawing operation based on the third thread in the second state; wherein power consumption of the first state is less than power consumption of the second state.

[0023] Optionally, increasing the target hardware resource capability includes:

[0024] Determine the target frequency increase strategy for the application corresponding to the current frame;

[0025] increasing the operating frequency of the target hardware resource from a first frequency to a second frequency greater than the first frequency according to a target frequency increase strategy; different applications correspond to corresponding frequency increase strategies;

[0026] Determining a frequency boost duration based on an execution state of subsequent frame drawing operations of the current frame after the frequency boost and a target frequency boost exit condition corresponding to the current application; different applications correspond to corresponding frequency boost exit conditions;

[0027] During the frequency boosting duration, the operating frequency of the target hardware is maintained at the second frequency after the frequency boosting.

[0028] Optionally, determining the frequency boost duration according to execution status information of subsequent frame drawing operations of the current frame after the frequency boost and a target frequency boost exit condition corresponding to the current application includes:

[0029] The frame drawing configuration parameters of the current application are obtained from the policy service module, and a target frequency boost exit condition is determined based on the frame drawing configuration parameters to determine the frequency boost duration. Different applications have different frame drawing configuration parameters, and thus different frequency boost durations are determined.

[0030] Optionally, the processing logic of the processing method is integrated into an operating system of the electronic device;

[0031] The first thread and the second thread respectively run on the first core of the corresponding target hardware resource, and the third thread runs on the second core of the corresponding target hardware resource; the computing power and power consumption of the first core are respectively higher than the computing power and power consumption of the second core.

[0032] A processing device, comprising:

[0033] A monitoring module, configured to determine whether to execute a current frame drawing operation and monitor the execution status of the drawing operation;

[0034] an acceleration module, configured to increase target hardware resource capabilities in response to the execution state indicating that a timeout event will occur in the current frame drawing operation, so as to accelerate the drawing operation of the current frame;

[0035] The target hardware resources are used for frame drawing operations.

[0036] An electronic device, comprising:

[0037] a memory for storing at least one set of computer instructions;

[0038] A processor is configured to implement any one of the processing methods provided above by executing the instruction set stored in the memory.

[0039] A storage medium carries one or more computer instruction sets, which, when executed by an electronic device, can enable the electronic device to implement any one of the processing methods provided above. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0041] Figure 1 It is a flow chart of the processing method provided by this application;

[0042] Figure 2 This is a flowchart of the execution status of the monitoring frame drawing operation provided by this application;

[0043] Figure 3 This is an example provided by the present application of a third thread obtaining the frame drawing configuration parameters of the current application;

[0044] Figure 4 This is a flowchart of increasing target hardware resource capabilities provided by this application;

[0045] Figure 5 It is a structural diagram of the processing device provided by this application;

[0046] Figure 6 This is a structural diagram of the electronic device provided in this application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] The present application provides a processing method and apparatus for predicting jamming during an application's frame drawing operation and intervening in advance to avoid or mitigate the effects of jamming. The processing method provided by the present application can be applied to electronic devices in a variety of general-purpose or specialized computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, and the like.

[0049] See also Figure 1 As shown in the method flow chart, the processing method provided in the embodiment of the present application may include at least the following steps 101 to 102, and these steps are described in detail below.

[0050] Step 101: Determine to execute a current frame drawing operation, and monitor the execution status of the drawing operation.

[0051] The frame drawing operation may be, but is not limited to, a drawing operation on an interface frame of an application during the running of the application, or a drawing operation on a desired image in a drawing software / tool, such as 3D drawing.

[0052] A frame drawing operation is used to draw one frame of image.

[0053] In this step, specifically, at the first moment of starting to execute the current frame drawing operation, that is, when starting to execute the current frame drawing operation, the drawing operation can be monitored simultaneously, and the execution status of the drawing operation can be obtained at the second moment of the drawing operation.

[0054] The second moment is later than the first moment, and may be earlier or later than the completion moment of the current frame drawing, depending on the actual duration of the current frame drawing operation.

[0055] Optionally, the second moment configured for current frames of different applications or different types of current frames of the same application is different.

[0056] Typically, frame drawing operations of different applications (such as games, videos, instant messaging, or office applications) have different requirements for system resources. The same application also has different requirements for system resources for drawing operations of different types of frames. To address this situation, the embodiments of the present application configure different second moments for current frames of different applications, or different types of current frames of the same application, to adapt to different applications or different types of frames of the same application to obtain the execution status of the current frame drawing operation at appropriate times, so as to effectively predict whether a timeout event of the current frame drawing operation will occur based on the acquired execution status information, and further support early intervention when a timeout event of the current frame drawing operation is predicted to occur, so as to avoid the occurrence of lag or mitigate the impact of lag.

[0057] The different applications may include but are not limited to games, videos, instant messaging, office applications, etc. Different frame types may include but are not limited to different frame types divided based on different scenes corresponding to the image frames or different features of the image frames.

[0058] Different scenarios corresponding to image frames may include, but are not limited to, application scenarios or refresh rate scenarios corresponding to image frames, such as different application scenarios such as player guides, information configuration, and equipment trading included in a game, or refresh rate scenarios in which different frames in a game run at different refresh rates. Different features of image frames may include, but are not limited to, attribute features such as image frame color and clarity, as well as image compression methods.

[0059] For example, based on different scenarios such as player strategies, information configuration, equipment trading contained in the same game, or the same game running at different refresh rates, different frame types are divided from the perspective of corresponding scenarios; or, based on the different clarity corresponding to different video clips and dramas (such as ultra-high definition, high definition, standard definition, etc.), different frame types are divided from the perspective of clarity; or, based on different compression methods of image frames, different frame types are divided from the perspective of compression methods (such as I frames, P frames and B frames), etc.

[0060] In this embodiment, the execution status of the drawing operation is obtained at the second moment of the current frame drawing operation in order to predict whether a timeout event of the current frame drawing operation will occur based on the execution status, thereby realizing freeze prediction. Based on this, the duration threshold corresponding to the drawing operation timeout event of the image frames of different applications or different types of image frames in the same application can be estimated in advance according to the application types of different applications and / or the types of different frames in the same application, that is, the duration threshold corresponding to the drawing operation timeout event at the application level or the frame level can be estimated. For example, the frame drawing operation timeout event of high-power gaming, video and other applications corresponds to a smaller duration threshold, and the frame drawing operation timeout event of low-power office applications corresponds to a larger duration threshold. In video applications, the frame drawing operation timeout event of ultra-high-definition video frames corresponds to a smaller duration threshold, and the frame drawing operation timeout event of standard-definition video frames corresponds to a larger duration threshold, etc.

[0061] If the duration of the frame drawing operation exceeds the corresponding duration threshold, it means that the frame drawing operation has timed out. If the duration does not exceed the corresponding duration threshold, it means that the frame drawing operation has not timed out.

[0062] On this basis, the second moment corresponding to each image frame in the application can be determined according to the duration threshold at the application level or frame level. Specifically, taking the current frame as an example, the second moment can be the sum of the first moment and the duration threshold, but is not limited to this. It can also be a certain value that is lower than the sum and the absolute value of the difference with the sum is less than the set threshold.

[0063] When monitoring a frame drawing operation (such as a current frame drawing operation) to obtain the execution status of the frame drawing operation at the second moment of the frame drawing operation, the frame drawing operation may be monitored in a timing manner or a polling manner, but is not limited to the timing manner.

[0064] In the timing mode, a timer for monitoring the drawing operation of each frame can be configured based on the second moment corresponding to each frame. Specifically, the timing duration of the timer can be configured to be the duration represented by the second moment (i.e., the duration elapsed from the first moment to the second moment). On this basis, the timer corresponding to each frame can be started at the beginning of each frame drawing, that is, at the first moment of each frame drawing, and the execution status of the frame drawing operation can be obtained when the timing is completed. For example, the execution status of the current frame drawing operation can be obtained when the timing is completed. The execution status is the execution status of the current frame drawing operation obtained at the second moment of the current frame drawing operation, thereby realizing the execution status monitoring of the frame drawing operation. If the current frame drawing operation is completed within the timing duration of the timer, the timer for the current frame drawing operation can be canceled when the current frame drawing operation is completed.

[0065] In the polling method, a polling time corresponding to the polling process for monitoring the drawing operation of each frame can be configured based on the second time corresponding to each frame. Specifically, the polling time can be configured as the second time. On this basis, the execution status of the drawing operation of each frame (i.e., the execution status corresponding to the second time of the drawing operation of the frame) can be obtained by polling the drawing operation of each frame at the corresponding polling time, thereby realizing the execution status monitoring of each drawing operation of the frame, such as the drawing operation of the current frame.

[0066] The execution status of the current frame drawing operation obtained through execution status monitoring may include, but is not limited to, the drawing progress / ratio of the current frame corresponding to the second moment, or whether a frame switching event of the drawn frame occurs or has occurred (i.e., whether the drawing has been switched or has been switched to the drawing of the next frame corresponding to the current frame) at the second moment, etc.

[0067] After obtaining the execution status of the current frame drawing operation, it may be further pre-determined whether the execution status indicates that a timeout event of the current frame drawing operation will occur.

[0068] Optionally, it may be specifically determined based on the execution state of the current frame drawing operation whether the current frame drawing operation is completed at the second moment, and based on this, it may be further determined whether a timeout event of the current frame drawing operation will occur.

[0069] If the execution status indicates that all progress in drawing the current frame has been completed at the second moment, or frame switching is in progress at the second moment to switch to drawing the next frame corresponding to the current frame, or switching to drawing the next frame corresponding to the current frame has already occurred before the second moment (i.e., between the first moment and the second moment), then the current frame drawing operation has been completed, and a timeout event for the current frame drawing operation will not occur. Otherwise, if the execution status indicates that only a certain percentage of the current frame drawing operation has been completed and further execution is required after the second moment, i.e., the current frame drawing operation is not completed at the second moment, then it can be predicted that a timeout event for the current frame drawing operation will occur.

[0070] Step 102: In response to the execution state indicating that a timeout event will occur for the current frame drawing operation, target hardware resource capabilities are increased to accelerate the drawing operation of the current frame.

[0071] The target hardware resources are used for frame drawing operations.

[0072] In response to the execution state indicating that a timeout event will occur for the current frame drawing operation, this embodiment increases the capabilities of the target hardware resources used for the frame drawing operation. By increasing the capabilities of the target hardware resources used for the frame drawing operation, hardware acceleration is enabled to increase the drawing frequency or rate, thereby avoiding or mitigating the effects of lag.

[0073] For the current frame drawing operation, since the capability of the target hardware resource for the frame drawing operation has been increased at the second moment, the drawing frequency or rate of the unfinished portion of the current frame drawing operation at the second moment can be increased based on the increased capability of the target hardware resource, thereby accelerating the unfinished portion of the current frame drawing operation. Accordingly, a timeout event of the current frame drawing operation can be avoided, or at least the timeout duration can be reduced.

[0074] The target hardware resources may include, but are not limited to, CPUs (Central Processing Units) and GPUs (Graphics Processing Units) used for frame drawing operations. Increasing the capabilities of target hardware resources may include, but are not limited to, increasing the frequency of CPUs, GPUs, and other hardware, or increasing the number of CPU cores and GPU cores allocated for the current frame drawing operation. The embodiments of this application primarily illustrate the solution by increasing the frequency of CPUs, GPUs, and other hardware.

[0075] In summary, the processing method provided in the embodiment of the present application monitors the execution status of the current frame drawing operation and, in response to the execution status indicating that a timeout event of the current frame drawing operation will occur, increases the capacity of the target hardware resources used for the frame drawing operation. This enables, during the frame drawing process, prediction of a timeout event of the frame drawing operation based on the execution status of the frame drawing operation, and correspondingly achieves jam prediction. When a timeout event of the current frame drawing operation is predicted to occur, early intervention is performed by increasing the capacity of the target hardware resources, thereby accelerating the drawing operation of the current frame and achieving the effect of avoiding jamming or mitigating the impact of jamming.

[0076] In an optional embodiment, in step 102 of the method provided in the present application, increasing the target hardware resource capability can be further implemented as: determining the first hardware resource of the first thread of the application corresponding to the current frame and the second hardware resource corresponding to the second thread, and increasing the resource capability of the first hardware resource and the second hardware resource.

[0077] The first thread and the second thread are created in response to the application corresponding to the current frame being started, the first thread is used to initiate the drawing operation based on the first hardware resource, and the second thread is used to execute the drawing operation based on the second hardware resource.

[0078] Optionally, the first thread is the main thread of the application corresponding to the current frame, and the second thread is the drawing thread of the application corresponding to the current frame. Taking the current frame as an interface frame in a game application as an example, the first thread and the second thread can be the main thread and the drawing thread of the game application respectively.

[0079] These two threads can be created by the operating system when it detects the startup of an application. The main thread, i.e., the first thread, can be used to respond to user operations and initiate the drawing operations, such as initiating drawing operations on game interface frames, etc. The drawing thread is used to execute the drawing operations. The two threads cooperate with each other to complete the frame drawing of the application.

[0080] The first hardware resource corresponding to the first thread is the hardware resource used to run the first thread, such as the main thread, and may include but is not limited to the CPU. The second hardware resource corresponding to the second thread is the hardware resource used to run the second thread, such as the drawing thread, and may include but is not limited to the GPU.

[0081] When increasing the capabilities of the target hardware resources, this embodiment specifically increases the resource capabilities of the first hardware resources and the second hardware resources, such as increasing the frequencies of the CPU and GPU, so as to improve the processing capabilities of each thread and accelerate the drawing operation of the current frame.

[0082] The application corresponding to the current frame may also be referred to as the current application.

[0083] This embodiment, when predicting a timeout event for the current frame drawing operation, determines the first hardware resources corresponding to the first thread and the second hardware resources corresponding to the second thread of the current application, and increases the capabilities of the first hardware resources and the second hardware resources. This effectively improves the processing capabilities of the first thread and the second thread used to implement frame drawing, thereby accelerating the drawing operation of the current frame and correspondingly avoiding or mitigating the impact of lag.

[0084] In an alternative embodiment, see Figure 2 Step 101 of the method provided in the present application, i.e., determining to execute the current frame drawing operation and monitoring the execution status of the drawing operation, can be further implemented as the following steps 201-202.

[0085] Step 201: Obtain frame drawing configuration parameters of the current application through a third thread, where the third thread is created when the current application is started.

[0086] The current application is the application corresponding to the current frame, that is, the application to which the current frame belongs.

[0087] The third thread is a monitoring thread, which can be created by the operating system when it detects that the current application is started. The third thread is used to monitor at least the second thread of the application, namely the drawing thread, for example, the main thread and the drawing thread of the application, or only the drawing thread, and perform related processing (such as hardware frequency increase) based on the monitoring.

[0088] The frame drawing configuration parameters of the current application may be obtained through the third thread, but is not limited to, when the third thread is started.

[0089] Optionally, the third thread may obtain the frame drawing configuration parameters of the current application from the policy service module based on intra-system communication, and the third thread may determine the second moment of the current frame drawing operation according to the frame drawing configuration parameters.

[0090] Different applications have different frame drawing configuration parameters and different second moments.

[0091] Optionally, the intra-system communication may be IPC (InterProcess Communication), and the policy service module may be implemented in the form of, but not limited to, a policy center app (application).

[0092] The frame drawing configuration parameters of the current application can be application-level or frame-level. For application-level parameters, different applications can correspond to different frame drawing configuration parameters. For frame-level parameters, different types of frames in the same application can correspond to different frame drawing configuration parameters.

[0093] If the frame drawing configuration parameters of the current application are application-level frame drawing configuration parameters, this step may specifically obtain the frame drawing configuration parameters corresponding to the current application for use in monitoring and related processing of the current frame drawing operation. If the frame drawing configuration parameters of the current application are frame-level frame drawing configuration parameters, this step may extract the frame drawing configuration parameters corresponding to the frame type of the current frame from the frame drawing configuration parameters of each type of frame corresponding to the current application for use in monitoring and related processing of the current frame drawing operation.

[0094] The frame drawing configuration parameters may include, but are not limited to, frequency increase start condition parameters and frequency increase exit condition parameters. The frequency increase start condition parameters are used to construct a frequency increase start condition, triggering a frequency increase on the target hardware resource when the current frame drawing operation meets the frequency increase start condition; the frequency increase exit condition parameters are used to construct a frequency increase exit condition, ending the frequency increase on the target hardware resource when the frame drawing operation meets the frequency increase exit condition.

[0095] Among them, the frequency increase start condition parameters may include but are not limited to the duration threshold corresponding to the timeout event of the frame drawing operation. Once it is predicted that the time consumption of a certain frame drawing operation will exceed the duration threshold corresponding to it in the frequency increase start condition parameters, it indicates that a timeout event will occur in the frame drawing operation, which correspondingly indicates that the frame drawing operation meets the frequency increase start condition and needs to start the frequency increase. The duration threshold here can be an application-level duration threshold, or it can also be a frame-level duration threshold. If it is an application-level duration threshold, the same application corresponds to one duration threshold. If it is a frame-level duration threshold, the same application can correspond to multiple duration thresholds, and different duration thresholds correspond to different types of frames of the same application.

[0096] The frequency boost exit condition parameters may include, but are not limited to, a set frequency boost duration limit, an upper limit on the number of consecutive frames after the current frame under normal conditions, and an upper limit on the duration of a normal drawing event. The normal consecutive frame drawing means that no timeout event occurs in the drawing operation of any of the consecutive frames, i.e., the drawing operation time of any frame does not exceed the upper limit on the duration of a normal drawing event.

[0097] Similar to the frequency increase start condition parameter, the frequency increase exit condition parameter may also be an application-level parameter or a frame-level parameter.

[0098] The third thread determines the second time based on the frame drawing configuration parameters. Specifically, the third thread may perform a sum operation on the first time and the duration threshold corresponding to the timeout event of the corresponding frame drawing operation in the frequency increase start condition parameters, and use the resulting sum as the value of the second time, or use a value that is lower than the sum and whose absolute difference with the sum is less than the threshold as the value of the second time. In implementation, the first time may be optionally set directly to 0, and the value of the second time may correspond to the duration threshold.

[0099] After determining the second moment, the timing duration of a timer required for monitoring the drawing operation of each frame, or the polling time of a polling process required for monitoring the drawing operation of each frame can be further configured based on the second moment, so as to monitor the execution status of the drawing operation of each frame in a timing manner or a polling manner.

[0100] Optionally, the third thread may specifically call a corresponding parameter acquisition function through IPC to obtain the frame drawing configuration parameters of the current application from a policy service module such as the policy center app.

[0101] See also Figure 3, provides an example of a third thread obtaining the frame drawing configuration parameters of the current application from the policy center app. In this example, the third thread (Jank monitor thread) calls the getJankParameters function based on IPC, and obtains the frame drawing configuration parameters of the current application from the policy center app based on this function.

[0102] The input parameter of the getJankParameters function includes the identifier of the current application, which can be, but is not limited to, the pid (ProcessIdentifier) ​​of the current application. The policy center app queries the frame drawing configuration parameters of the current application based on the identifier and feeds them back to the third thread.

[0103] Optionally, the information fed back by the policy center app to the third thread is a string containing multiple key-value pairs in the form of "key:value".

[0104] Each key-value pair represents a series of frame drawing configuration parameters for the current application. The key in each key-value pair represents the parameter name of the frame drawing configuration parameter, and the value represents the parameter value. If the policy center app cannot find the frame drawing configuration parameters for the current application, it can optionally return an empty string to the third thread.

[0105] Step 202: Obtain feedback information of the second thread's drawing operation on the current frame through the third thread, and determine whether the timeout event will occur based on the frame configuration parameters and the feedback information.

[0106] The feedback information of the second thread on the drawing operation of the current frame may include, but is not limited to, the drawing progress / proportion of the current frame at the current moment (i.e., the second moment), whether a frame switch event of the drawn frame occurs or has occurred at the current moment (i.e., whether the drawing has been switched or has been switched to the drawing of the next frame corresponding to the current frame), etc.

[0107] After receiving the aforementioned feedback information from the second thread, a third thread, such as a jank monitor thread, may determine whether a timeout event will occur for the current frame drawing operation based on the frame configuration parameters corresponding to the current frame drawing operation and the feedback information. Optionally, the third thread may determine whether a timeout event will occur based on frequency increase initiation condition parameters in the frame configuration parameters, such as a duration threshold corresponding to a timeout event for the frame drawing operation, and the execution status of the current frame drawing operation as indicated by the feedback information (e.g., whether the current frame drawing operation has completed, or is incomplete and requires continued drawing).

[0108] If the feedback information indicates that the current frame drawing operation has been completed, the time consumed by the current frame drawing operation does not exceed the time threshold, and accordingly, a timeout event for the current frame drawing operation will not occur. Conversely, if the feedback information indicates that the current frame drawing operation is not completed and needs to be continued, it can be predicted that the time consumed by the current frame drawing operation will exceed the time threshold, and accordingly, it can be predicted that a timeout event for the current frame drawing operation will occur.

[0109] This embodiment creates a third thread to monitor the execution status of frame drawing operations based on relevant parameters, which can effectively predict frame drawing operation timeout events and support early intervention in impending frame drawing operation timeout events, thereby effectively avoiding the occurrence of lag or mitigating the impact of lag.

[0110] In an optional embodiment, the processing method provided in this application may further include the following steps 11)-12):

[0111] 11) At the first moment, the third thread is controlled to be in a first state.

[0112] The first state may be a low power consumption state indicating that the power consumption of the third thread is lower than a first set threshold, and may specifically be, but not limited to, a sleep / hibernation state, for example, a state in which the third thread is controlled by an operating system.

[0113] The first moment is the moment when the current frame drawing operation starts to be executed.

[0114] In this step, the third thread may be controlled to be in a first state such as sleep / hibernation when the current frame drawing operation starts to be executed.

[0115] In implementation, optionally, the first thread, such as the main thread of the current application, may control the third thread to be in a first state such as sleep / hibernation at the first moment of starting to execute the current frame drawing operation.

[0116] 12) At the second moment, controlling the third thread to be in a second state, so as to determine an execution state of the drawing operation based on the third thread in the second state.

[0117] The power consumption in the first state is less than that in the second state.

[0118] Optionally, the second state may be a high power consumption state indicating that the power consumption of the third thread is higher than a second set threshold, and specifically may be a normal working state after the third thread is awakened from a sleep / hibernation state.

[0119] The first set threshold and the second set threshold may be the same or different, which is not limited. In the case where the two are different, the second set threshold is higher than the first set threshold.

[0120] At the second moment of the current frame drawing operation, the third thread may be awakened by the operating system, for example, to determine the execution state of the current frame drawing operation based on the awakened third thread.

[0121] After being awakened, the third thread may first obtain feedback information from the second thread regarding the current frame drawing operation, such as the drawing progress / ratio of the current frame at the current moment (i.e., the second moment), whether a frame switch event for the drawn frame has occurred or has occurred at the current moment, etc. Based on this information, the third thread may further determine whether a timeout event will occur for the current frame drawing operation based on the frame configuration parameters corresponding to the current frame drawing operation and the feedback information, thereby monitoring the execution status of the current frame drawing operation and predicting whether a timeout event will occur for the current frame drawing operation.

[0122] The specific implementation process of the third thread determining whether a timeout event will occur for the current frame drawing operation based on the frame configuration parameters corresponding to the current frame drawing operation and the feedback information can be found in the description of the previous embodiment and will not be repeated here.

[0123] This embodiment controls the third thread to be in a first low-power state such as sleep / hibernation at the first moment of starting to execute the current frame drawing operation, controls the third thread to be in a second high-power state such as normal working state at the second moment of the current frame drawing operation, and determines the execution state of the current frame drawing operation based on the third thread in the second state. On the one hand, this embodiment can monitor the execution state of the current frame drawing operation based on the third thread, thereby effectively predicting whether a timeout event of the current frame drawing operation will occur, supporting early intervention before the frame drawing timeout event occurs, thereby achieving the effect of avoiding or mitigating the impact of jamming. On the other hand, it can also reduce the power consumption of the third thread, save resource expenditure, reduce resource waste, and achieve power consumption optimization during application operation.

[0124] In an alternative embodiment, participating Figure 4 As shown in the flowchart, in step 102 of the processing method provided in this application, increasing the target hardware resource capability can be further implemented as the following steps 401-404.

[0125] Step 401: Determine a target frequency increase strategy for an application corresponding to a current frame.

[0126] Among them, different applications correspond to corresponding frequency increase strategies.

[0127] Different applications usually have different characteristics such as application type, application scenarios (such as video playback, document editing, etc.), power consumption, etc., which often lead to different requirements for system resources for different applications. It also easily leads to different requirements for increasing hardware resource capabilities when increasing hardware resource capabilities to accelerate frame drawing operations for different applications.

[0128] Based on this, during implementation, different frequency boosting strategies can be configured for different applications in advance according to the types of different applications, corresponding application scenarios (such as video playback, document editing, etc.), power consumption and other characteristics, so that the frequency boosting strategy configured for each application matches the characteristics of the corresponding application, thereby meeting the system resource usage requirements of different applications.

[0129] The frequency boost policy configured for each application may include, but is not limited to, the frequency value to be increased, the magnitude / ratio to be increased, and the number of gears (frequency is divided into multiple gears) when increasing the frequency of the target hardware resources used by the application, such as the CPU or GPU. If a frequency boost policy is not configured for an application, the application uses the default frequency boost policy provided by the system.

[0130] On this basis, in response to the execution status of the current frame drawing operation indicating that a timeout event will occur for the current frame drawing operation, this embodiment first determines the target frequency increase policy of the application corresponding to the current frame, so as to increase the frequency of the target hardware resource based on the target frequency increase policy of the application corresponding to the current frame.

[0131] In implementation, each application's frequency boosting policy information (e.g., the number of frequency boost levels) can be included as part of the application's frame drawing configuration parameters, but is not limited to being included in the frame drawing configuration parameters. When the third thread is launched, it can obtain the frame drawing configuration parameters of the current application to which the current frame belongs from the policy service module via intra-system communication. Subsequently, when it is necessary to increase the target hardware resource capacity for the current frame drawing operation, the target frequency boosting policy for the application corresponding to the current frame can be extracted from the current application's frame drawing configuration parameters for use.

[0132] In actual applications, frame-level frequency boosting strategies can also be set for different types of frames in the same application as needed. In this implementation, this step requires extracting the target frequency boosting strategy corresponding to the frame type of the current frame from the frame-level frequency boosting strategies corresponding to the current application.

[0133] Step 402: Increase the operating frequency of the target hardware resource from a first frequency to a second frequency greater than the first frequency according to a target frequency increase strategy.

[0134] After obtaining the target frequency increase strategy of the application corresponding to the current frame, the second frequency to which the operating frequency of the target hardware resource needs to be increased can be further determined based on the target frequency increase strategy. For example, based on the current first frequency of the target hardware resources such as the CPU and GPU and the frequency increase amplitude / ratio or the number of increase levels indicated in the target frequency increase strategy, it is determined to which frequency the operating frequency of the target hardware resources such as the CPU and GPU needs to be increased.

[0135] The second frequency is greater than the first frequency.

[0136] On this basis, the operating frequency of the CPU and GPU target hardware resources is further increased from the current first frequency to the second frequency to increase the capacity of the target hardware resources and accelerate the drawing operation of the current frame.

[0137] Step 403: Determine the frequency boost duration based on the execution status of the subsequent frame drawing operation of the current frame after the frequency boost and the target frequency boost exit condition corresponding to the current application; different applications correspond to corresponding frequency boost exit conditions.

[0138] The frequency increase duration refers to the duration for maintaining the frequency of the target hardware resource at the second frequency after the frequency of the target hardware resource is increased to the second frequency.

[0139] The system resources required for different applications or different scenarios and different frame types within the same application often vary. After the frequency of the target hardware resources is increased, the required frequency increase duration often also varies. Based on this, after increasing the frequency of the target hardware resources from a first frequency to a second frequency, the embodiment of the present application continuously monitors the execution status of subsequent frame drawing operations of the current frame, and dynamically determines the frequency increase duration of this frequency increase based on the execution status of the subsequent frame drawing operations of the current frame and the target frequency increase exit condition corresponding to the current application, so as to meet the actual system resource requirements of the current application's frame drawing.

[0140] During implementation, a third thread may obtain the frame drawing configuration parameters of the current application from the policy service module, determine the target frequency boost exit condition corresponding to the current application based on the frame drawing configuration parameters, and further determine the frequency boost duration based on the execution status of subsequent frame drawing operations of the current frame and the target frequency boost exit condition corresponding to the current application. Different applications have different frame drawing configuration parameters, and the determined frequency boost durations are different.

[0141] Optionally, the frequency boost duration is determined according to the execution status of the subsequent frame drawing operation of the current frame after the frequency boost and the target frequency boost exit condition corresponding to the current application, which can be further implemented as follows:

[0142] a. If, in the multiple consecutive drawing operations following the current frame after the frequency increase, the duration of each drawing operation does not exceed the upper limit of the duration corresponding to a normal drawing event, and the number of frames corresponding to the multiple consecutive drawing operations reaches the upper limit of the frame number, and the total operation duration does not reach a third threshold, then the frequency increase duration is determined to be the total duration of the multiple drawing operations;

[0143] b. If, in the continuous multiple frame drawing operations subsequent to the current frame after the frequency increase, the duration of any frame drawing operation exceeds the upper limit of the duration corresponding to the normal drawing event, and the total operation duration reaches the upper limit of the frequency increase duration, then the frequency increase duration is determined to be the upper limit of the frequency increase duration;

[0144] c. If, in the continuous multi-frame drawing operations following the current frame after the frequency increase, the duration of each frame drawing operation does not exceed the upper limit of the duration corresponding to the normal drawing event, and the number of frames corresponding to the continuous multi-frame drawing operations does not reach the upper limit of the frame number, but the total operation duration reaches the upper limit of the frequency increase duration, then the frequency increase duration is determined to be the upper limit of the frequency increase duration.

[0145] The upper limit of the duration corresponding to the normal drawing event, the upper limit of the number of frames corresponding to the continuous multi-frame drawing operation, and the upper limit of the frequency boost duration are all included in the frequency boost exit condition parameter and can be specifically extracted from the frequency boost exit condition parameter.

[0146] Step 404: During the frequency-increasing duration, maintain the operating frequency of the target hardware at the second frequency after the frequency-increasing duration.

[0147] After determining the frequency boost duration corresponding to the current frequency boost, the operating frequency of the target hardware resource may be maintained at the second frequency after the frequency boost during the frequency boost duration. Specifically, the second frequency corresponding to the target hardware resource after the frequency boost may be maintained for the frequency boost duration. Subsequently, when the maintenance duration reaches the frequency boost duration, the operating frequency of the target hardware resource may be restored to the first frequency before the frequency boost.

[0148] This embodiment effectively meets the varying system resource demands and resource increases of different applications or different frame types within the same application by setting different frequency boost policies for different applications or different frame types within the same application. Furthermore, after a frequency boost, this embodiment continuously monitors the execution status of subsequent frame-drawing operations of the current frame and dynamically determines the frequency boost duration based on the execution status of subsequent frame-drawing operations and the target frequency boost exit conditions for the current application. This further meets the actual system resource demands of the current application's current frame-drawing operation, achieving the goal of both meeting the actual resource requirements of the application's frame-drawing operations and avoiding resource waste.

[0149] In an optional embodiment, the processing logic of the processing method provided in the embodiment of the present application is integrated into the operating system of the electronic device.

[0150] The first and second threads described above run on the first core of the corresponding target hardware resource respectively, and the third thread runs on the second core of the corresponding target hardware resource; the computing power and power consumption of the first core are respectively higher than the computing power and power consumption of the second core.

[0151] For example, the first thread such as the main thread of the current application runs on the large core of the CPU, the second thread such as the drawing thread of the current application runs on the large core of the GPU, and the third thread such as the monitoring thread runs on the small core of the CPU.

[0152] This embodiment integrates the processing logic of the method provided by this application into the operating system of an electronic device, so that hardware frequency boost processing under the premise of predicting jamming can be implemented based on the method of this application without modifying the application side, thereby reducing the difficulty of implementation and achieving the effect of easy implementation. In addition, this embodiment prevents the third thread from competing with the first thread and the drawing thread, etc., by running the first and second threads on the first core of the corresponding target hardware resource respectively, and running the third thread on the second core of the corresponding target hardware resource (corresponding to lower power consumption than the first core). This ensures the system resources required for the operation of the first and second threads and ensures the efficient operation of the first and second threads. At the same time, since the third thread is responsible for related monitoring tasks, its tasks are lighter than those of the first thread and the second vector, and the third thread is often in a low-power state such as dormancy / sleep, and has a low demand for system resources. Therefore, running the third thread on the second core of the corresponding target hardware resource, such as running on a small core of the CPU, can still ensure the normal operation of the third thread and will not affect the operation of the third thread.

[0153] Corresponding to the above-mentioned processing method, the embodiment of the present application further provides a processing device, the composition structure of which is as follows: Figure 5 As shown, including:

[0154] A monitoring module 501 is used to determine whether to execute a current frame drawing operation and monitor the execution status of the drawing operation;

[0155] The acceleration module 502 is configured to increase target hardware resource capabilities to accelerate the drawing operation of the current frame in response to the execution state indicating that a timeout event will occur in the drawing operation of the current frame.

[0156] The target hardware resources are used for frame drawing operations.

[0157] In an optional implementation, the monitoring module 501 is specifically configured to:

[0158] At a first moment of starting to execute a drawing operation of a current frame, monitoring the drawing operation is started simultaneously, and an execution state of the drawing operation is acquired at a second moment of the drawing operation, where the second moment is later than the first moment;

[0159] The second time is configured differently for current frames of different applications or different types of current frames of the same application.

[0160] In an optional embodiment, the acceleration module 502, when increasing the target hardware resource capability to accelerate the rendering operation of the current frame, is specifically configured to:

[0161] Determine a first hardware resource of a first thread and a second hardware resource corresponding to a second thread of an application corresponding to a current frame, and increase resource capabilities of the first hardware resource and the second hardware resource;

[0162] The first thread and the second thread are created when the application corresponding to the current frame is started, the first thread is used to initiate the drawing operation based on the first hardware resource, and the second thread is used to execute the drawing operation based on the second hardware resource.

[0163] In an optional implementation, the monitoring module 501 is specifically configured to:

[0164] obtaining a frame drawing configuration parameter of the current application through a third thread, where the third thread is created when the current application is started;

[0165] Feedback information of the second thread's drawing operation on the current frame is obtained through the third thread, and whether the timeout event will occur is determined based on the frame configuration parameters and the feedback information.

[0166] In an optional implementation manner, when the monitoring module 501 obtains the frame drawing configuration parameters of the current application through the third thread, it is specifically configured to:

[0167] obtaining, by a third thread based on intra-system communication, a frame drawing configuration parameter of the current application from a policy service module, and determining, by the third thread, the second moment according to the frame drawing configuration parameter;

[0168] Different applications have different frame drawing configuration parameters and different second moments.

[0169] In an optional embodiment, the above device further includes a control module, which is used to:

[0170] Controlling the third thread to be in a first state at the first moment;

[0171] At the second moment, the third thread is controlled to be in a second state, so as to determine an execution state of the drawing operation based on the third thread in the second state; wherein power consumption of the first state is less than power consumption of the second state.

[0172] In an optional embodiment, when increasing the target hardware resource capability, the acceleration module 502 is specifically configured to:

[0173] Determine the target frequency increase strategy for the application corresponding to the current frame;

[0174] increasing the operating frequency of the target hardware resource from a first frequency to a second frequency greater than the first frequency according to a target frequency increase strategy; different applications correspond to corresponding frequency increase strategies;

[0175] Determining a frequency boost duration based on an execution state of subsequent frame drawing operations of the current frame after the frequency boost and a target frequency boost exit condition corresponding to the current application; different applications correspond to corresponding frequency boost exit conditions;

[0176] During the frequency boosting duration, the operating frequency of the target hardware is maintained at the second frequency after the frequency boosting.

[0177] In an optional embodiment, when the acceleration module 502 determines the frequency boost duration based on the execution status information of the subsequent frame drawing operation of the current frame after the frequency boost and the target frequency boost exit condition corresponding to the current application, it is specifically configured to:

[0178] The frame drawing configuration parameters of the current application are obtained from the policy service module, and a target frequency boost exit condition is determined based on the frame drawing configuration parameters to determine the frequency boost duration. Different applications have different frame drawing configuration parameters, and thus different frequency boost durations are determined.

[0179] In an optional embodiment, the processing logic of each module in the processing device is integrated into the operating system of the electronic device;

[0180] The first thread and the second thread respectively run on the first core of the corresponding target hardware resource, and the third thread runs on the second core of the corresponding target hardware resource; the computing power and power consumption of the first core are respectively higher than the computing power and power consumption of the second core.

[0181] The present application also discloses an electronic device, the composition structure of the electronic device is as follows: Figure 6 As shown, including at least:

[0182] Memory 10, for storing a computer instruction set;

[0183] A set of computer instructions can be implemented in the form of a computer program.

[0184] The processor 20 is configured to implement the processing method provided in any of the above method embodiments by executing the computer instruction set in the memory.

[0185] The processor 20 may include but is not limited to a CPU, a GPU, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a neural network processor (NPU), a deep learning processor (DPU) or other programmable logic devices.

[0186] Optionally, the electronic device also includes storage resources such as memory and cache.

[0187] Optionally, the electronic device further includes a camera assembly, and / or is connected to an external camera assembly.

[0188] In addition, the electronic device may also include components such as a communication interface and a communication bus. The memory, processor, and communication interface communicate with each other via the communication bus.

[0189] Communication interfaces are used for communication between electronic devices and other devices. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, and a control bus.

[0190] An embodiment of the present application further discloses a storage medium, which carries one or more computer instruction sets. When the one or more computer instruction sets are executed by an electronic device, the electronic device can implement a processing method as provided in any of the above method embodiments.

[0191] In summary, the processing method and related apparatus provided in the embodiments of the present application have at least the following technical advantages:

[0192] 1) By predicting jamming and intervening in advance, jamming can be avoided or its impact can be mitigated.

[0193] Traditional solutions typically only increase the frequency after detecting lag, which is already happening, similar to trying to fix the problem after it has happened. The solution in this embodiment of the application starts the frequency increase in advance when the signs of lag are detected, that is, when lag is predicted to occur but has not yet occurred, thereby avoiding or mitigating lag.

[0194] 2) After the frequency is increased, the frequency increase duration is dynamically determined or adjusted by dynamically and continuously monitoring the drawing of subsequent frames in the application, which can meet the system resource requirements of different applications.

[0195] After detecting a freeze, the traditional solution performs a fixed-duration frequency boost operation, that is, the frequency after the boost is maintained for a fixed duration, which is obviously unable to adapt to different freeze scenarios. For example, in some large and complex game scenarios, if the frequency boost time is too short, it will not be able to meet its demand for system resources; for some applications with relatively small loads, if the frequency boost time is too long, it will cause a waste of power consumption. After starting the frequency boost, the embodiment of the present application will continuously monitor the drawing time of subsequent frames. When the drawing performance of the application returns to normal, the frequency boost will be stopped, thereby meeting the system resource requirements of different applications.

[0196] 3) Support parameter configuration for different applications to meet the individual needs of different applications and scenarios.

[0197] Different applications have different requirements for system resources. Even the same application, such as a game app, may have different requirements when running at different refresh rates. To address this need, the present embodiment defines a series of parameters for customizing conditions such as frequency boost activation and exit to meet the varying needs of different applications.

[0198] During implementation, this application can also support users to debug relevant parameters of different applications based on actual needs and perform personalized settings based on debugging to achieve a balance between performance and power consumption.

[0199] 4) The solution of the embodiment of the present application is integrated into the operating system, without the need to modify the application side, and is applicable to both system built-in applications and third-party applications.

[0200] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other.

[0201] For the convenience of description, the above systems or devices are described as being divided into various modules or units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0202] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform. Based on this understanding, the technical solution of the present application can be essentially or the part that makes a creative contribution in the form of a software product. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present application.

[0203] Finally, it should be noted that, in this document, relational terms such as first, second, third, and fourth are used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0204] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A processing method comprising: Determining to execute a current frame drawing operation, and monitoring an execution status of the drawing operation; In response to the execution state indicating that a timeout event will occur for the current frame drawing operation, increasing target hardware resource capabilities to accelerate the drawing operation for the current frame; The target hardware resources are used for frame drawing operations.

2. The processing method according to claim 1, wherein determining to execute the current frame drawing operation and monitoring the execution status of the drawing operation comprises: At a first moment of starting to execute a drawing operation of a current frame, monitoring the drawing operation is started simultaneously, and an execution state of the drawing operation is acquired at a second moment of the drawing operation, where the second moment is later than the first moment; The second time is configured differently for current frames of different applications or different types of current frames of the same application.

3. The processing method according to claim 2, wherein increasing the target hardware resource capability to accelerate the rendering operation of the current frame comprises: Determine a first hardware resource of a first thread and a second hardware resource corresponding to a second thread of an application corresponding to a current frame, and increase resource capabilities of the first hardware resource and the second hardware resource; The first thread and the second thread are created in response to the application corresponding to the current frame being started, the first thread is used to initiate the drawing operation based on the first hardware resource, and the second thread is used to execute the drawing operation based on the second hardware resource.

4. The processing method according to claim 3, wherein determining to execute the current frame drawing operation and monitoring the execution status of the drawing operation comprises: obtaining a frame drawing configuration parameter of the current application through a third thread, where the third thread is created when the current application is started; Feedback information of the second thread's drawing operation on the current frame is obtained through the third thread, and whether the timeout event will occur is determined based on the frame configuration parameters and the feedback information.

5. The processing method according to claim 4, wherein obtaining the frame drawing configuration parameters of the current application through the third thread comprises: The third thread obtains the frame drawing configuration parameters of the current application from the policy service module based on intra-system communication, and the third thread determines the second time according to the frame drawing configuration parameters; Different applications have different frame drawing configuration parameters and different second moments.

6. The processing method according to claim 4, further comprising: Controlling the third thread to be in a first state at the first moment; At the second moment, the third thread is controlled to be in a second state, so as to determine an execution state of the drawing operation based on the third thread in the second state; wherein power consumption of the first state is less than power consumption of the second state.

7. The processing method according to any one of claims 1 to 5, wherein increasing the target hardware resource capability comprises: Determine the target frequency increase strategy for the application corresponding to the current frame; increasing the operating frequency of the target hardware resource from a first frequency to a second frequency greater than the first frequency according to a target frequency increase strategy; different applications correspond to corresponding frequency increase strategies; Determining a frequency boost duration based on an execution state of subsequent frame drawing operations of the current frame after the frequency boost and a target frequency boost exit condition corresponding to the current application; different applications correspond to corresponding frequency boost exit conditions; During the frequency boosting duration, the operating frequency of the target hardware is maintained at the second frequency after the frequency boosting.

8. The processing method according to claim 7, wherein determining the frequency boost duration according to execution status information of subsequent frame drawing operations of the current frame after the frequency boost and a target frequency boost exit condition corresponding to the current application comprises: The frame drawing configuration parameters of the current application are obtained from the policy service module, and a target frequency boost exit condition is determined based on the frame drawing configuration parameters to determine the frequency boost duration. Different applications have different frame drawing configuration parameters, and thus different frequency boost durations are determined.

9. The processing method according to claim 5, wherein the processing logic of the processing method is integrated into an operating system of the electronic device; The first thread and the second thread respectively run on the first core of the corresponding target hardware resource, and the third thread runs on the second core of the corresponding target hardware resource; the computing power and power consumption of the first core are respectively higher than the computing power and power consumption of the second core.

10. A processing device comprising: A monitoring module, configured to determine whether to execute a current frame drawing operation and monitor the execution status of the drawing operation; an acceleration module, configured to increase target hardware resource capabilities in response to the execution state indicating that a timeout event will occur in the current frame drawing operation, so as to accelerate the drawing operation of the current frame; The target hardware resources are used for frame drawing operations.

Images