Task processing method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention discloses a task processing method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining user interaction data and system resource state data, determining equipment operation task state information based on the user interaction data and the system resource state data, and sending the equipment operation task state information to a server; and performing task scheduling processing based on the equipment operation task state information.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a task processing method, device, storage medium and electronic device. Background Art

[0002] With the rapid development of computer technology, terminal devices such as mobile phones and tablets have been rapidly popularized. In daily use, terminals will be involved in executing (foreground) application tasks and (background) service tasks. (Foreground) application tasks are related to foreground applications in the human-computer interaction process, and (background) service tasks are related to the kernel or system framework process that manages resources.

[0003] During the human-computer interaction process, there are often messages of resource competition between (background) service tasks and (foreground) application tasks. When resource competition occurs in the device running task status of the terminal device, it will cause phenomena such as a decrease in the foreground application frame rate and interface freeze. Summary of the invention

[0004] The present application provides a task processing method, device, storage medium and electronic device, and the technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a task processing method, the method comprising:

[0006] Obtain user interaction data and system resource status data;

[0007] Determining device operation task status information based on the user interaction data and the system resource status data;

[0008] Task scheduling is performed based on the device running task status information.

[0009] In a second aspect, an embodiment of the present application provides a task processing device, the device comprising:

[0010] A data acquisition module, used to acquire user interaction data and system resource status data;

[0011] An information determination module, used to determine device operation task status information based on the user interaction data and the system resource status data;

[0012] The scheduling processing module is used to perform task scheduling processing based on the device running task status information.

[0013] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0014] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0015] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0016] In one or more embodiments of the present application, the terminal device obtains user interaction data and system resource status data, predicts and determines the device running task status information of the current terminal device based on the user interaction data and the system resource status data, and performs task scheduling in a timely manner based on the device running task status information. The device running task status information is accurately predicted by the user interaction data and the system resource status data. The device running task status information can provide early feedback on the timing of resource competition between application tasks and service tasks. Based on this, task scheduling and service tasks are performed in a timely manner based on the device running task status information. Since resource competition can be predicted in advance from the user interaction dimension and the system resource task status dimension before it occurs, in this way, the resource consumption peak periods of service tasks and application tasks can be staggered through timely task scheduling, thereby achieving efficient task processing and greatly improving the utilization rate of system resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flowchart of a task processing method provided in an embodiment of the present application;

[0019] Figure 2 It is a flowchart of a task processing method provided in an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a task processing scenario provided in an embodiment of the present application;

[0021] Figure 4 It is a schematic diagram of a training scenario of a task state prediction model provided in an embodiment of the present application;

[0022] Figure 5 is a schematic diagram of a task scheduling scenario provided by an embodiment of the present application;

[0023] Figure 6This is a schematic diagram comparing the implementation effects of a solution provided in an embodiment of the present application;

[0024] Figure 7 This is a schematic diagram comparing the effects of another solution provided in the embodiment of the present application;

[0025] Figure 8 This is a schematic diagram of startup time comparison of a solution provided in an embodiment of the present application;

[0026] Fig. 9 This is a schematic diagram of startup time comparison of another solution provided in an embodiment of the present application;

[0027] Fig.10 is a comparative schematic diagram of a task processing device provided in an embodiment of the present application;

[0028] Fig.11 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0029] Fig.12 It is a schematic diagram of the structure of the operating system and user space provided in the embodiment of the present application;

[0030] Fig.13 yes Fig.12 The architecture diagram of the Android operating system;

[0031] Fig.14 yes Fig.12 Architecture diagram of the IOS operating system. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood in specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are an "or" relationship.

[0034] In the related art, most of the tasks running on the processor of the terminal device come from the service tasks of the kernel or system framework. After creative work, the inventors found that a large amount of resource competition occurs between service tasks (kernel or framework processes that manage resources) and application tasks (foreground application-related processes). Therefore, when the user is performing human-computer interaction, the user experience of the foreground application is negatively affected. In addition, service tasks and application tasks may often be busy at the same time. When the user stops the human-computer interaction process, such as turning off the terminal device screen, the activity of the service task will also be greatly reduced. Specifically, most users interact with terminal devices such as smartphones for more than four hours a day on average. In some cases, the system resources of the terminal device are wasted in more than 80% of the idle time every day, and these tasks are concentrated in a small range of time intervals used in daily life to compete fiercely for resources. The resource competition between service tasks and application tasks often leads to phenomena such as a decrease in the frame rate of the foreground application and a freeze in the interface. Based on this, an effective task processing method is needed.

[0035] However, the related technologies lack an effective way to improve or even completely avoid the frame rate drop and user interface freeze problems during the use of applications during human-computer interaction. Through creative work, the inventors found that the above problems are usually caused by competition between service tasks and application tasks for system resources. It often happens that busy application tasks will trigger busy service tasks at the same time, and the operation of service tasks will in turn block application tasks, which will in turn lead to the phenomenon of foreground application frame rate drop and interface freeze, and the terminal device system utilization rate is not high. For example, when the screen is turned off and the user behavior is idle, the resource utilization rate is greatly reduced, resulting in a waste of system resources.

[0036] In order to solve or even improve the above-mentioned phenomenon, one or more embodiments of the present specification propose a task processing method, which: obtains user interaction data and system resource status data, predicts and determines the device operation task status information of the current terminal device based on the user interaction data and the system resource status data, and performs task scheduling and processing in a timely manner based on the device operation task status information. The device operation task status information is accurately predicted through user interaction data and system resource status data. The device operation task status information can provide early feedback on the resource competition timing between application tasks and service tasks. Based on this, task scheduling and processing of service tasks are performed in a timely manner based on the device operation task status information. Since advance predictions can be made from the user interaction dimension and the system resource task status dimension before resource competition occurs, in this way, the resource consumption peak periods of service tasks and application tasks can be staggered through timely task scheduling, thereby achieving efficient task processing and greatly improving the utilization rate of system resources.

[0037] The present application is described in detail below with reference to specific embodiments.

[0038] In one embodiment, Figure 1 As shown, a task processing method is proposed, which can be implemented by a computer program and can be run on a task processing device based on the von Neumann system. The computer program can be integrated in an application or run as an independent tool application. The task processing device can be a terminal device, including but not limited to: a personal computer, a tablet computer, a handheld device, a vehicle-mounted device, a wearable device, a computing device or other processing device connected to a wireless modem. In different networks, terminal devices can be called different names, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user device, cellular phone, cordless phone, 5G network or terminal device in future evolution network, etc.

[0039] Specifically, the task processing method includes:

[0040] S102: Acquire user interaction data and system resource status data;

[0041] Schematically, the terminal device obtains user interaction data and system resource status data from the user experience perception dimension and the system resource perception dimension respectively;

[0042] Indicatively, the user (human-computer interaction behavior) busyness of the foreground user human-computer interaction process is predicted from the user experience perception dimension and / or the system resource perception dimension. Usually, when the user human-computer interaction behavior is idle, the current or subsequent device running task status information indicates that the foreground interaction is idle, and the foreground interaction idle corresponds to the (foreground) application task idle. Usually, when the user human-computer interaction behavior is busy, the current or subsequent device running task status information indicates that the foreground interaction is busy, and the foreground interaction busy corresponds to the (foreground) application task busy.

[0043] Indicatively, the busyness of the kernel service task is predicted from the user experience perception dimension and / or the system resource perception dimension. Usually, the device running task status information indicates that the kernel is idle at present or in the future, and the kernel idle corresponds to the idle service task (kernel or background). Usually, the device running task status information indicates that the kernel is busy at present or in the future, and the kernel busy corresponds to the busy service task (kernel or background);

[0044] For example, the terminal device can obtain information related to framework rendering and user interaction through the user experience perception part, which are collectively referred to as user interaction data. The system resource perception part can be responsible for obtaining information related to CPU, GPU, temperature, and memory, which are collectively referred to as system resource status data.

[0045] User interaction data includes but is not limited to the target frame rate T fps , the difference between the target frame rate and the system frame rate D fps , frame loss rate F lr , Human-computer interaction frequency U ui One or more types of fit.

[0046] Human-computer interaction frequency U ui For example, it can provide feedback on the number of interactions between the user and the terminal device within a unit time (such as 2s).

[0047] (Sample) System resource status data includes but is not limited to system free memory value M free , Application available memory value M val , average service task processor utilization C avg , processor core utilization, average processor utilization CU of application tasks usg , processor temperature, kswapd thread wake-up frequency, etc.

[0048] S104: Determine device operation task status information based on the user interaction data and the system resource status data;

[0049] The device running task status information may include foreground interactive task status information or kernel service task status information; the device running task status information may include foreground interactive task status information and kernel service task status information; the device running task status information may include foreground interactive task status information, kernel service task status information and resource bottleneck status information;

[0050] Optionally, the foreground interactive task status information may be divided into foreground application task busy and foreground application task idle;

[0051] Optionally, the kernel service task status information may be divided into kernel service task busy and kernel service task busy idle;

[0052] Optionally, the resource bottleneck status information may be divided into a processor resource bottleneck, a memory resource bottleneck, and an input / output resource bottleneck (I / O resource bottleneck);

[0053] In one or more embodiments of the present specification, a task status prediction model may be pre-trained based on a machine learning model, and the task status prediction model may be used to perform task status prediction.

[0054] The following is an explanation of the model training process of the task status prediction model:

[0055] Model creation: Create an initial task status prediction model for the terminal task status prediction scenario based on the machine learning model;

[0056] Sample data acquisition: Acquire a large amount of sample data, including sample user interaction data and sample system resource status data.

[0057] Sample data labeling: Based on the needs of terminal task status prediction scenarios, expert-side services are introduced to manually label the sample data with corresponding sample labels. The sample labels include foreground interactive task status labels and kernel service task status labels. In addition, resource bottleneck labels are also marked in some sample data, such as processor resource bottlenecks, memory resource bottlenecks, and input / output resource bottlenecks.

[0058] Model training process: input sample data into the initial task state prediction model for at least one round of model training to obtain the predicted device operation task state information. The predicted feature data includes one or more of the predicted foreground interaction task state, the predicted kernel service task state and the predicted resource bottleneck. The model loss function is used to determine the model loss value. The model loss value can be composed of the foreground interaction task prediction loss, the kernel service task prediction loss and the resource bottleneck task prediction loss. Based on the model loss value, the model parameters of the initial task state prediction model are adjusted until the model training end conditions are met to obtain the task state prediction model.

[0059] The foreground interaction task prediction loss can be obtained by using the first model loss calculation formula based on the predicted foreground interaction task state and the foreground interaction task state label;

[0060] The kernel service task prediction loss may be obtained by using a second model loss calculation formula based on the predicted kernel service task state and the kernel service task state label;

[0061] The resource bottleneck task prediction loss can be obtained based on the predicted resource bottleneck and resource bottleneck label using the third model loss calculation formula;

[0062] Among them, the first model loss calculation formula, the second model loss calculation formula, and the third model loss calculation formula can adopt the loss calculation function in the relevant technology, and can be the same or different;

[0063] Among them, the loss calculation function can be a hinge loss function, a cross entropy loss function, a feature distance loss function, etc.

[0064] Optionally, the model training end condition of the model may include, for example, the value of the loss function is less than or equal to a preset loss function threshold, the number of iterations reaches a preset number threshold, etc. The specific model training end condition can be determined based on actual conditions and is not specifically limited here.

[0065] It should be noted that the machine learning models involved in one or more embodiments of the present specification include but are not limited to the fitting of one or more machine learning models such as convolutional neural network (CNN) model, deep neural network (DNN) model, recurrent neural network (RNN) model, embedding model, gradient boosting decision tree (GBDT) model, logistic regression (LR) model, etc.

[0066] In a feasible implementation, the following method may be adopted:

[0067] A2: Determine foreground interaction task status information and kernel service task status information based on the user interaction data and the system resource status data;

[0068] A2-1: Determine device running task status information based on the foreground interaction task status information and the kernel service task status information;

[0069] For example, the terminal device can directly use the foreground interaction task status information and the kernel service task status information as the device running task status information;

[0070] For example, the terminal device may input the user interaction data and the system resource status data into the task status prediction model, and the task status prediction model may use the user interaction data and the system resource status data as references to perform status prediction processing to obtain the foreground interaction task status information and the kernel service task status information; the foreground interaction task status information may represent the status of the foreground application task at present or within a certain period of time to come, and the kernel service task status information may represent the status of the service task at present or within a certain period of time to come; and then the task status prediction model may output the device operation task status information including the foreground interaction task status information and the kernel service task status information;

[0071] In a feasible implementation, the following method may be adopted:

[0072] A2: Determine foreground interaction task status information and kernel service task status information based on the user interaction data and the system resource status data;

[0073] A2-2: When the kernel service task status information is of the kernel service task busy type, resource bottleneck prediction processing is performed based on the foreground interaction task status information and the kernel service task status information to obtain resource bottleneck status information, and the device operation task status information is determined based on the resource bottleneck status information, the foreground interaction task status information and the kernel service task status information.

[0074] For example, the terminal device may directly use the resource bottleneck status information, the foreground interaction task status information and the kernel service task status information as the device operation task status information;

[0075] For example, the terminal device may input the user interaction data and the system resource status data into the task status prediction model, and perform status prediction processing by the task status prediction model with reference to the user interaction data and the system resource status data to obtain the foreground interaction task status information and the kernel service task status information; and when the kernel service task status information is of the kernel service task busy type, the control task status prediction model performs resource bottleneck prediction processing based on the foreground interaction task status information and the kernel service task status information to obtain the resource bottleneck status information, and outputs the device operation task status information including the resource bottleneck status information, the foreground interaction task status information and the kernel service task status information;

[0076] S106: Perform task scheduling based on the device running task status information.

[0077] In a feasible implementation, based on the device running task status information, it is determined that the foreground interactive task state is the foreground application task idle, and the kernel service task can be actively triggered, that is, the service task execution is actively triggered, for example, the priority of the kernel service task can be increased;

[0078] In a feasible implementation, based on the device running task status information, it is determined that "the foreground interaction task status is the foreground application task busy, or the foreground interaction task status is the foreground application task busy and the kernel service task status is the kernel service task busy". The kernel service task can be delayed, that is, the kernel service task can be delayed;

[0079] Furthermore, when the terminal device determines that the kernel service task state is busy based on the device running task state information, the resource bottleneck state information can be obtained from the device running task state information, and the service task can be scheduled according to the resource bottleneck state information.

[0080] In one or more embodiments of the present specification, the terminal device obtains user interaction data and system resource status data, predicts and determines the device running task status information of the current terminal device based on the user interaction data and the system resource status data, and performs task scheduling in a timely manner based on the device running task status information. The device running task status information is accurately predicted by the user interaction data and the system resource status data. The device running task status information can provide early feedback on the timing of resource competition between application tasks and service tasks. Based on this, task scheduling is performed in a timely manner to process service tasks based on the device running task status information. Because resource competition can be predicted in advance from the user interaction dimension and the system resource task status dimension before it occurs, in this way, the resource consumption peak periods of service tasks and application tasks can be staggered through timely task scheduling, thereby achieving efficient task processing and greatly improving the utilization rate of system resources.

[0081] See also Figure 2 , Figure 2 This is a flowchart of another embodiment of a task processing method proposed in this application. Specifically:

[0082] S202: In response to the user's human-computer interaction operation, execute the step of acquiring user interaction data and system resource status data.

[0083] Optionally, the terminal device may obtain user interaction data and system resource status data based on a set monitoring period when each monitoring period arrives;

[0084] Optionally, the terminal device may monitor the user's human-computer interaction operation, and when the user's human-computer interaction operation is detected, obtain the user interaction data and system resource status data; the human-computer interaction operation may be a user touch screen operation, a user pressing a physical button operation, a user operating the terminal operation, etc.;

[0085] Through the above method, when the user is performing a human-computer interaction process, or when the user is performing a human-computer interaction process and the service task is busy, the step of obtaining user interaction data and system resource status data is executed. The task processing overhead can be saved. By identifying the behavior of human-computer interaction operations, the task processing method of one or more embodiments of this specification can be executed quickly to schedule tasks, and the time periods of frequent user interactions can be successfully staggered later. When executing the task processing method of one or more embodiments of this specification, the service task tends to run when the human-computer interaction is infrequent, such as in the device screen-off scenario and the user's low-frequency operation scenario, rather than occupying more device resources to execute the service task when the foreground user frequently interacts.

[0086] S204: inputting the user interaction data and the system resource status data into a task status prediction model;

[0087] Indicatively, Figure 3 As shown, Figure 3 is a schematic diagram of a task processing scenario. Figure 3 In the process, the terminal device detects the human-computer interaction operation between the user and the foreground application. The terminal device can obtain the framework rendering related information and the user interaction related information through the user experience perception part, and these information are collectively referred to as the user interaction data. The system resource perception part can be responsible for obtaining the CPU related information, GPU related information, temperature related information and memory related information, and these information are collectively referred to as the system resource status data. The user interaction data and the system resource status data are then input into the task status prediction model. The task status prediction model can perform user busy prediction (i.e., predict the foreground interaction task status), kernel busy prediction (i.e., predict the kernel service task status) and resource bottleneck prediction (i.e., predict the resource bottleneck status).

[0088] S206: Control the task state prediction model to predict the foreground interaction task state to obtain foreground interaction task state information, and predict the kernel service task state through the task state prediction model to obtain the kernel service task state information;

[0089] S208: Control the task status prediction model to output running task status information corresponding to the foreground interaction dimension and / or the kernel service task dimension;

[0090] For example, the terminal device can input user interaction data and system resource status data into a task status prediction model, and use the task status prediction model to perform status prediction processing to obtain foreground interaction task status information and kernel service task status information with reference to the user interaction data and system resource status data, and then control the task status prediction model to output the running task status information corresponding to the foreground interaction dimension and / or the kernel service task dimension.

[0091] S210: Control the task status prediction model to perform resource bottleneck prediction processing to obtain resource bottleneck status information when the kernel service task status information is of the kernel service task busy type, and output the device running task status information including the foreground interaction task status information, the kernel service task status information and the resource bottleneck status information.

[0092] For example, the terminal device may input the user interaction data and the system resource status data into the task status prediction model, and perform status prediction processing by the task status prediction model with reference to the user interaction data and the system resource status data to obtain the foreground interaction task status information and the kernel service task status information; and when the kernel service task status information is of the kernel service task busy type, the control task status prediction model performs resource bottleneck prediction processing based on the foreground interaction task status information and the kernel service task status information to obtain the resource bottleneck status information, and outputs the device operation task status information including the resource bottleneck status information, the foreground interaction task status information and the kernel service task status information;

[0093] Optionally, the resource bottleneck status information may be divided into a processor resource bottleneck, a memory resource bottleneck, and an input / output resource bottleneck (I / O resource bottleneck);

[0094] Furthermore, the following is a schematic diagram of a task state prediction model training method. Specifically:

[0095] S2002: Acquire sample data, and label the sample data with a device running task status information label, wherein the sample data includes sample user interaction data and sample system resource status data;

[0096] For example, Figure 3 As shown, the terminal device can maintain a pre-training part, an online prediction part and a task scheduling engine part. The pre-training part is responsible for training a task state prediction model. The online prediction part is responsible for using the task state prediction model to make online predictions of user busy states, kernel busy states and resource bottlenecks. The task scheduling engine part is responsible for dynamically scheduling tasks based on the output results of the task state prediction model, such as adjusting the running time of service tasks.

[0097] In a feasible implementation, the following method may be referred to:

[0098] B2: Obtain sample data including sample user interaction data and sample system resource status data;

[0099] First, in the task state prediction model training phase, sample data comes from the user experience perception (unit or module) and system resource perception (unit or module) in the pre-training part of one or more terminal devices, which respectively obtain the user interaction data and system resource status data in the sample phase. The user interaction data and system resource status data constitute a set of sample data, and the number of sample data is usually multiple;

[0100] B4: labeling the sample data with a foreground interactive task status label and a kernel service task status label using a scene label labeling condition;

[0101] The sample user interaction data at least includes the target frame rate, the difference between the target frame rate and the system frame rate, the frame loss rate, and the human-computer interaction frequency; the sample system resource status data at least includes the system free memory value, the application available memory value, and the average service task processor usage rate.

[0102] In some embodiments, the (sample) user interaction data includes but is not limited to the target frame rate T fps , the difference between the target frame rate and the system frame rate D fps , frame loss rate F lr , Human-computer interaction frequency U ui One or more types of fit.

[0103] In some embodiments, the (sample) system resource status data includes but is not limited to the system free memory value M. free , Application available memory value M val , average service task processor utilization C avg , processor core utilization, average processor utilization CU of application tasks usg , processor temperature, kswapd thread wake-up frequency, etc.

[0104] In some embodiments, an expert service can be introduced based on the needs of the terminal task status prediction scenario to manually annotate the sample data with corresponding sample labels. The sample labels include foreground interactive task status labels and kernel service task status labels. In addition, resource bottleneck labels are also annotated in some sample data, such as processor resource bottlenecks, memory resource bottlenecks, and input / output resource bottlenecks.

[0105] In the training phase, in order to collect data under various user and kernel states, for example, four data collection scenarios can be constructed: i) the scenario corresponding to the user idle state and the kernel idle state (corresponding to the label "foreground application task idle + kernel service task idle"); ii) the scenario corresponding to the user idle state and the kernel busy state (corresponding to the label "foreground application task idle + kernel service task busy"); iii) the scenario corresponding to the user busy state and the kernel idle state (corresponding to the label "foreground application task busy + kernel service task idle"); iv) the scenario corresponding to the user busy state and the kernel busy state (corresponding to the label "foreground application task busy + kernel service task busy"). The label annotated to the sample data is to annotate the corresponding data collection scenario. In order to construct a training data set for resource bottleneck prediction, the labels of the sample data records obtained under the CPU resource constraint, memory resource constraint, I / O resource constraint and other scenarios can be defined as the corresponding restricted system resources.

[0106] Furthermore, during the data collection process, a large amount of system status information and user behavior information can be read and parsed. Considering that it is usually a very time-consuming process. In order to simplify data collection and analysis and reduce IA overhead, the original sample data can be preprocessed, and only data closely related to user status, kernel status and resource bottlenecks can be extracted through data preprocessing. To this end, the terminal device can filter the original sample data in advance through the data preprocessing part (module or unit) that is not very relevant to the model status prediction, and finally output high-quality sample data;

[0107] Exemplarily, the terminal device executes the scenario labeling condition to label the sample data with a foreground interactive task status label and a kernel service task status label in the following manner:

[0108] Traverse each sample data and label the sample data in turn. Specifically, compare the target frame rate in the sample data with the target frame rate threshold, compare the difference rate with the difference rate threshold, compare the system free memory value with the system free memory threshold, compare the application available memory value with the application available memory threshold, and compare the average service task processor usage with the processor usage threshold, as follows:

[0109] If the first target frame rate in the first sample data is greater than or equal to the target frame rate threshold, the first difference rate is greater than or equal to the difference rate threshold, and the first human-computer interaction frequency is greater than the interaction frequency threshold, then marking the first sample data with a foreground application task busy label;

[0110] If the second target frame rate in the second sample data is less than the target frame rate threshold, and / or the second difference rate is less than the difference rate threshold, and / or the second human-computer interaction frequency is less than the interaction frequency threshold, then marking the second sample data with a foreground application task idle label;

[0111] If the third system free memory value in the third sample data is less than the system free memory threshold, the third application available memory value is less than the application available memory threshold, and the third average service task processor usage is greater than the processor usage threshold, then the third sample data is marked with a kernel service task busy label;

[0112] If the fourth system idle memory value in the fourth sample data is less than the system idle memory threshold, the fourth application available memory value is less than the application available memory threshold, and the fourth average service task processor usage is greater than the processor usage threshold, the fourth sample data is marked with a kernel idle label.

[0113] B6: Marking a resource bottleneck label on target sample data in the sample data, wherein the kernel service task status label of the target sample data is a kernel service task busy type.

[0114] Furthermore, in order to construct a training data set for resource bottleneck prediction based on the aforementioned sample data, the reference sample data labeled with the busy type of kernel service tasks can be further analyzed and processed, that is, to determine whether the reference sample data is sample data corresponding to the corresponding resource bottleneck type. The corresponding resource bottleneck type refers to CPU resource constraints, memory resource constraints, I / O resource constraints, etc. The obtained reference sample data are defined as corresponding restricted system resource labels, that is, labeled resource bottleneck labels. Optionally, resource bottleneck labels can be divided into processor resource bottlenecks, memory resource bottlenecks, and input / output resource bottlenecks (I / O resource bottlenecks);

[0115] For example, the relevant sample data x i The detailed characteristic data is shown in the following table:

[0116]

[0117] Exemplarily, the data preprocessing part adopts the above method to realize the construction of sample data in the model training phase, for example, constructing D fps and F lr To describe the degree of user experience, U ui To describe user interaction information, and N kw To describe the number of wake-up times of the kswapd process. According to the sample data x i Contains {{T fps , D fps , F lr , U ui , M free , M val , C avg , C4 usg , C5 usg , C6usg , C7 usg , C.U. avg , C temp , G temp , N kw} and label data y i , we can construct a sample data set (xi, yi), which can be expressed as D = {(x 1 ,y 1 ), (x 2 ,y 2 ),....,(x m ,y m )}.

[0118] S2004: Create an initial task status prediction model;

[0119] Create an initial task status prediction model for terminal task status prediction scenarios based on the machine learning model;

[0120] Exemplarily, the machine learning model may be a gradient boosting decision tree model, that is, the gradient boosting decision tree model may be used to create an initial task state prediction model;

[0121] S2006: Input the sample data into the initial task state prediction model for at least one round of model training, determine the predicted device operation task state information corresponding to the sample data through the initial task state prediction model in each round of model training, adjust the model parameters of the initial task state prediction model based on the predicted device operation task state information and the device operation task state information label, and obtain the task state prediction model after model training.

[0122] User busy state prediction, system busy state prediction, and current resource bottleneck prediction can all be considered classification problems. In order to accurately predict the user busy state, system busy state, and current resource bottleneck, a decision tree model can be used to create an initial task state prediction model for model training. The model contains multiple binary decision tree models corresponding to different prediction targets. Because the decision tree can fully explore the associations between data, it has the characteristics of strong explanatory power and simple implementation. In addition, the overhead of the decision tree is also very low. In order to further reduce the prediction overhead of the model, user busy prediction and resource bottleneck prediction are only performed when the user interacts with the screen and the service task is busy. The Touch-driven ML model takes user experience perception and system resource data after data preprocessing as input, and takes user busy state, kernel busy state, and current resource bottleneck (CPU, memory, or I / O) as output, as shown in 4, Figure 4 It is a schematic diagram of the training scenario of a task state prediction model.

[0123] The core of building a decision tree learning algorithm is how to select the optimal partitioning attribute. Information entropy can be used as a common indicator to measure the purity of a sample data set for a decision tree model. Assume that the proportion of the kth class of samples in the current sample data set D is p k (k=1,2,...n, n is the total number of categories), the information entropy of D is defined as:

[0124]

[0125] Among them, Ent(D) represents the entropy of the sample data set D, which is the expected value of the information randomly extracted from the data, P k It represents the probability of a certain category k appearing in the sample data set D. The smaller the entropy value, the higher the purity of D. On the contrary, the larger the entropy value, the higher the uncertainty of the data set.

[0126]

[0127] Where Cain(D,A) represents the information gain when using discrete feature a (which can be regarded as feature data a in a certain sample data) to segment the sample data set D; |D v | represents the size of the data subset under a certain value k of the discrete feature a, and |D| represents the total size of the sample data set D. Information gain is the difference between the entropy of the original data set and the weighted sum of the entropy of the segmented data subsets. The greater the information gain, the greater the "purity improvement" of using feature a to segment the sample data set D.

[0128] As can be understood, the discrete feature is defined as a, and ai represents the value taken. Assume that a has V values ​​{a 1 , a 2 , ..., a v}, if the sample data-feature a is used to divide the sample set D, V branch nodes will be generated, where the Vth branch node contains all samples in D whose feature a value is av, denoted as Dv. The information gain obtained by dividing the sample set D by the sample data-feature a can be calculated based on the number of sample data contained in the branch node and the information entropy. In general, the greater the information gain, the greater the purity gain obtained by using feature a, and the better the effect. As investigated, the information gain criterion has a preference for features with a larger number of values. In order to generate a task state prediction model corresponding to a classification decision tree with high generalization ability, you can choose to first find features with information gain higher than the average level from the candidate segmentation features corresponding to the sample data, and then select the highest gain rate from them. The gain rate is defined as:

[0129]

[0130]

[0131] Among them, IV(a) is called the intrinsic value of attribute a, and its value increases with the increase in the number of values ​​of attribute a. This effectively reduces the preference of information gain for attributes with a large number of values. In order to prevent the overfitting problem of the initial task state prediction model of the decision tree learning algorithm, its generalization performance can be estimated according to the accuracy of the validation set after node division to perform pre-pruning of the decision tree. During the training process, at least one round of model training is performed by inputting sample data into the initial task state prediction model. In each round of model training, the predicted device operation task state information corresponding to the sample data is determined by the initial task state prediction model. Based on the predicted device operation task state information and the device operation task state information label, the model parameters of the initial task state prediction model are adjusted to obtain the task state prediction model after model training.

[0132] Furthermore, in order to prevent the model from overfitting, the maximum depth of the decision tree can be set during the model parameter adjustment process. depth And the information gain threshold MIN during model training is , for example, MAX depth Set to 8, and the newly added gain threshold is set to 0.25; if the decision tree depth corresponding to the initial task state prediction model is higher than the maximum depth or the information gain is lower than the gain threshold, the split will not continue. These parameters can be adjusted according to the training environment.

[0133] S212: Determine a key scheduling task from at least one system running task;

[0134] The terminal device can maintain the task scheduling engine, which is used to determine the key scheduling tasks to be scheduled and the timing of the key scheduling tasks. Task Scheduling Engine (TSE): TSE is responsible for scheduling the target tasks by adjusting the task running timing according to the prediction results of the task status prediction model. Figure 5 As shown, Figure 5 It is a scenario diagram of task scheduling. TSE consists of two main parts: task extraction mechanism and task scheduler. The task extraction mechanism determines the key scheduling tasks that need to be scheduled based on the resource bottleneck prediction results. The task scheduler is responsible for specific task scheduling and adjusts the execution timing of key scheduling tasks to prevent them from competing with foreground application tasks for system resources. Among them, the execution timing of key scheduling tasks is predicted based on the busyness of foreground user interaction.

[0135] For terminal devices, their memory is limited. Since more applications can be opened at the same time, there will always be a time when the memory is exhausted. For this reason, the terminal device kernel maintains a memory recovery mechanism for this kind of insufficient memory, including kswapd memory compression, direct memory recovery, and GC. zRAM (a swappable partition architecture based on memory compression) is enabled to compress pages on DRAM and then transfer them to the RAM disk when the system is short of memory, which will bring more burden to the processor of the terminal device.

[0136] Optionally, when both the kernel service task and the foreground application task are very busy and the current resource bottleneck is memory, the task responsible for memory compression and recycling will be executed frequently, and its operation will inevitably occupy system resources, thereby competing with the application task for system resources.

[0137] In addition, when direct memory recycling is triggered due to insufficient memory during the execution of an application task, it will wait for the direct recycling task to complete before continuing to execute, which will undoubtedly damage the foreground user experience. Considering the characteristics of the user's foreground interaction behavior when using the terminal device, memory compression, memory recycling and GC tasks can be actively triggered when the user's foreground interaction is idle, and memory compression and memory recycling tasks can be actively suppressed or postponed when the user's foreground interaction is busy, so as to reduce competition with application tasks for system resources.

[0138] Furthermore, according to the characteristics of the task's demand for system computing and memory resources, it can be divided into computing-intensive tasks and I / O-intensive tasks. For computing-intensive tasks of kernel service tasks and foreground application tasks, the terminal device can execute them when the foreground interactive tasks are idle, such as executing memory compression tasks and video high-definition decoding tasks. For the user's I / O-intensive tasks, we can execute them in advance when the user is idle, such as loading data in advance according to the time and space positioning of the user's access to local storage or cloud storage to reduce the I / O task blocking problem. In order to locate the target task comprehensively and quickly, we designed static task extraction and dynamic task extraction. The static task extraction module directly obtains the target task by accessing the local file of the relationship between the storage resource bottleneck and the task to be scheduled. The dynamic task extraction module monitors the service tasks that occupy the most system resources in real time through system commands and uses them as key scheduling tasks.

[0139] In a feasible implementation, the following method may be performed:

[0140] C2: determining that the system task state is in a kernel service task busy state and a resource bottleneck task state based on the device running task state information, and the terminal device determines the target bottleneck resource from the device running task state information;

[0141] The device running task status information may include foreground interactive task status information, kernel service task status information and resource bottleneck status information;

[0142] Optionally, the foreground interactive task status information may be divided into foreground application task busy and foreground application task idle;

[0143] Optionally, the kernel service task status information may be divided into kernel service task busy and kernel service task busy idle;

[0144] Optionally, the resource bottleneck status information may be divided into a processor resource bottleneck, a memory resource bottleneck, and an input / output resource bottleneck (I / O resource bottleneck); it can be understood that the target bottleneck resource may be one or more of a processor resource bottleneck, a memory resource bottleneck, and an input / output resource bottleneck (I / O resource bottleneck);

[0145] Further, the terminal device parses the device running task status information. After parsing, one situation may be that the system task status is a kernel service task busy state and a resource bottleneck task state. The terminal device may determine the target bottleneck resource from the device running task status information.

[0146] C4: Determine a key static scheduling task and a key dynamic scheduling task from at least one system running task based on the target bottleneck resource.

[0147] 1. When the target bottleneck resource is a memory resource, the terminal device may obtain a preset resource bottleneck scheduling mapping relationship, and use the resource bottleneck scheduling mapping relationship to determine a static memory processing task from at least one system running task;

[0148] Among them, the resource bottleneck scheduling mapping relationship is the mapping relationship between the reference memory resource bottlenecks such as the processor resource bottleneck, the memory resource bottleneck, the input / output resource bottleneck (I / O resource bottleneck) and the corresponding static memory scheduling tasks (such as at least one of the memory compression task, the memory collection task and the memory garbage collection GC task); the mapping relationship between the reference memory resource bottleneck and the static memory scheduling task can be established in advance

[0149] The memory processing task includes at least one of a memory compression task, a memory collection task and a memory garbage collection GC task;

[0150] When it is predicted that the current resource bottleneck is memory resources, the key static scheduling tasks to be scheduled are responsible for memory compression tasks, memory collection tasks, and GC. For this type of task, its resource bottleneck scheduling mapping relationship can be stored in the local file system in advance. The corresponding tasks are directly read by the static task extraction part to save system overhead.

[0151] 2. When the target bottleneck resource is a processor resource, determining a dynamic processor occupancy task based on the processor resource occupancy from at least one system running task;

[0152] When it is predicted that the current resource bottleneck is the processor resource, a dynamic processor occupying task is determined from at least one system running task based on the processor resource occupancy. For example, the task to be scheduled is a service task that occupies a large amount of processor resource occupancy, that is, a dynamic processor occupying task. For such a dynamic processor occupying task, dynamic analysis and acquisition can be performed through system instructions, based on which dynamic analysis can be performed through the dynamic task extraction part to determine the dynamic processor occupying task from the system running tasks based on the processor resource occupancy.

[0153] 3. When the target bottleneck resource is an input / output resource, a dynamic input / output occupation task is determined from at least one system running task based on the input / output occupation amount.

[0154] When it is predicted that the current target bottleneck resource is I / O, a dynamic input / output occupying task is determined from at least one system running task based on the input / output occupying amount, for example, the task to be scheduled is a service task that occupies a large amount of I / O resources. For such a dynamic input / output occupying task, dynamic analysis and acquisition can be performed through system instructions, based on which dynamic analysis can be performed through the dynamic task extraction part through system instructions to determine the dynamic processor occupying task from the system running tasks based on the processor resource occupying amount.

[0155] S214: Perform task scheduling processing on the key scheduling task based on the device running task status information.

[0156] Finally, the task scheduler is responsible for scheduling static memory processing tasks, dynamic processor occupation tasks, and dynamic input / output occupation tasks through system calls and file configuration modification. For example, for the scheduling of memory recycling tasks, we can determine the triggering time of the memory recycling task by modifying the memory recycling value.

[0157] In a feasible implementation, the terminal device may determine that the foreground interactive task state is in the foreground interactive task idle state based on the device running task state information, and the terminal device performs task execution processing on the key scheduling task;

[0158] In a feasible implementation manner, the terminal device determines that the foreground interactive task state is in a foreground interactive task busy state based on the device running task state information, and then the terminal device performs task delay processing on the key scheduling task.

[0159] In one or more embodiments of the present specification, the terminal device obtains user interaction data and system resource status data, predicts and determines the device running task status information of the current terminal device based on the user interaction data and the system resource status data, and performs task scheduling in a timely manner based on the device running task status information. The device running task status information is accurately predicted by the user interaction data and the system resource status data. The device running task status information can provide early feedback on the timing of resource competition between application tasks and service tasks. Based on this, task scheduling is performed in a timely manner to process service tasks based on the device running task status information. Because resource competition can be predicted in advance from the user interaction dimension and the system resource task status dimension before it occurs, in this way, the resource consumption peak periods of service tasks and application tasks can be staggered through timely task scheduling, thereby achieving efficient task processing and greatly improving the utilization rate of system resources.

[0160] Furthermore, terminal devices dominate people's daily lives. Among them, a large number of applications and service tasks run on processors. The present invention has conducted relevant experiments on terminal devices and observed that service tasks always compete with applications for resources in a short time. On the other hand, surveys based on real mobile users show that applications and service tasks are idle for more than 80% of the time, which is a waste of resources. The task processing method involved in executing one or more embodiments is improved on the current mobile system task scheduling, and the task processing method, a user interaction-aware task scheduling optimization method, is proposed to reduce resource competition in the mobile system. The task processing method can utilize idle time that was previously neglected in daily use. By identifying the user's behavior on the touch screen, the task processing method can quickly schedule service tasks and successfully stagger the time periods of frequent user interactions. When executing the task processing method of one or more embodiments, service tasks tend to run when there is no interaction, such as when the terminal device turns off the screen, rather than when the user interacts frequently.

[0161] For example, in order to verify the task processing method involved in this specification, it is deployed on the terminal device in advance. The experimental results show that after using the task processing method, the user experience is significantly improved. Compared with the related art, the startup speed and frame rate of the application are increased by 38.89% and 7.97% respectively.

[0162] Deploy the experiment:

[0163] Implementation platform and workload: The present invention evaluates the task processing method IA on two terminal devices - mobile phones. Terminal device A is equipped with a Snapgragon 888 processor, 12GB DRAM and 256GB UFS flash memory. Terminal device B is equipped with a Snapgragon 865 processor, 8GB DRAM and 128GB UFS flash memory. Users often run more than ten applications, so 14 popular applications are pre-installed on terminal devices A and B to simulate daily use. These applications are selected from the most popular applications in various categories in the application market.

[0164] 14 popular applications are pre-installed on terminal device A and terminal device B, for example, they may be: PUBG MOBILE application, TikTok application, Facebook application, PayPal Chrome application, Twitter application, YouTube application, Weibo application, Skype Snapchat application, WeChat application, Instagram application, Uber application, Temu application;

[0165] Deployment comparison scheme and the task processing scheme involved in this specification:

[0166] Schedutil comparison scheme: In terminal device A and terminal device B, schedutil is deployed as the default scheduling scheme. Schedutil based on CPU frequency policy directly uses the load data of the scheduler. It allows the system to select the most appropriate frequency. In order to effectively utilize the ARM multi-core architecture, the Energy Aware Scheduler (EAS) was developed and applied to the Linux kernel of recent terminal devices. EAS organizes CPU hardware information into an energy model, which is used to improve CPU scheduling performance by allocating sufficient CPU cores.

[0167] SmartSwap Comparison Solution: SmartSwap is a high-performance and user-friendly intelligent memory swap solution in mobile systems. It swaps the corresponding pages in advance by predicting the applications with the lowest probability of use. Specifically, it uses the Naive Bayes algorithm to predict the applications that will not be used in the next user interaction period based on time, location, power, and the order of application usage. It adjusts the operation time of page swapping according to the prediction results. The data and programs of unused applications are swapped to the Flash swap area in advance. It will be used as one of our comparison experiments.

[0168] zTT Comparison Solution: zTT is a state-of-the-art resource management mechanism that learns thermal environment characteristics and jointly adjusts CPU and GPU frequencies to maximize application performance in an energy-efficient manner.

[0169] The task processing scheme involved in this specification reduces resource competition by isolating the resource demand peak of application tasks and service tasks during the busy period of the system. Service tasks are scheduled to be executed when the user is idle, based on the prediction of resource conflict time between application tasks and service tasks and the prediction of user idle state.

[0170] Beneficial effects on user experience: We evaluated four typical scenarios to understand the benefits of the described task handling scheme on user experience: video playback in the YouTube app, short video switching in the TikTok app, frequent scrolling of Moments in the Facebook app, and gaming in the PUBG app.

[0171] Frame rate: A potential user experience advantage of the task processing scheme is the improvement of frame rate. The present invention first evaluates FPS to understand the degree of improvement of smooth experience. We developed a script to simulate daily use, such as application startup, video playback, and scrolling. In addition, we developed a frame rate collection tool in C language. It collects frame rate information by using the dumpsurfaceflinger command. We first run the shell script to start an application every 5 seconds using the adb am command, in the order of the applications in Table 2, with a total of 14 applications. To simulate the user's idle state, we turn off the screen for 5 seconds after every 15 seconds. Secondly, we hot-start the application to be tested and continue to run the shell script to simulate the user's operations of scrolling pages, playing videos, pausing videos, and switching short videos. During this period, we run different strategies and execute the frame rate capture tool to obtain frame rate information. Finally, the frame rate information collected in different experimental environments is visualized. To reduce the impact of the environment, such as temperature, we restart the device before each experiment and wait for the temperature to drop below 25°C. In addition, each experiment is repeated ten times and the average is taken.

[0172] like Figure 6 As shown, Figure 6 This is a schematic diagram for comparing the schemes. Figure 6 The above-mentioned Schedutil comparison scheme, SmartSwap comparison scheme, zTT comparison scheme and the task processing scheme involved in this specification are respectively executed on the terminal device A ( Figure 6 iAware solution shown); Figure 7 This is another scheme comparison diagram. Figure 7 The above-mentioned Schedutil comparison scheme, SmartSwap comparison scheme, zTT comparison scheme and the task processing scheme involved in this specification are respectively executed on the terminal device A ( Figure 7 iAware solution shown);

[0173] See also Figure 6 and Figure 7 , evaluated the frame rendering performance of the iAware solution and compared it with the Schedutil comparison solution, SmartSwap comparison solution, and zTT comparison solution. The average frame rate of real-world mobile applications (YouTube, TikTok, Facebook, PUBG Mobile) when multiple applications are cached in the background can be directly derived from the task processing solution involved in this specification ( Figure 6 and Figure 7 Taking the TikTok application as an example, the frame rate in the original system of the terminal device is 45.88FPS, and after using the task processing scheme, this value has increased by an average of 13.14%. In addition, the FPS using the task processing scheme is 2.96% and 7.97% higher than SmartSwap and zTT, respectively. The benefits of the task processing scheme for gaming scenarios are more obvious than other scenarios because the resource requirements are more aggressive when users use the PUBG mobile game. Therefore, service tasks are woken up more frequently.

[0174] Further analysis was conducted in combination with the implementation effect of the scheme, and the CPU usage under different schemes was analyzed. The inventor found through experiments that when the Schedutil comparison scheme was used, the CPU usage reached 100% many times during the frequent use of the mobile phone. This will undoubtedly increase the number of resource competitions between application tasks and service tasks, resulting in a significant drop in frame rate. On the contrary, during the period when the terminal device screen is turned off, the CPU usage shows a significant decline. When using "the task processing scheme involved in this specification", the CPU usage during the screen off period is higher than in other cases. This is because "the task processing scheme involved in this specification" schedules and executes some service tasks (such as memory compression, memory recycling, GC, etc.) during this period in advance. Since "the task processing scheme involved in this specification" allocates service tasks during the period when the terminal device screen is turned off and the user is idle, the number of times the CPU usage exceeds 95% during the period when the user frequently interacts with the device is greatly reduced, thereby reducing the opportunity for resource competition between application tasks and service tasks. Therefore, "the task processing scheme involved in this specification" can effectively reduce the decline in frame rate.

[0175] When the screen of the terminal device is turned off, Smartswap can reclaim memory in advance. Therefore, the inventors found through experiments that when SmartSwap is used, the CPU usage during the screen off period is higher than the default solution. But it will fail while the user is using the mobile phone. Compared with "the task processing scheme involved in this specification", it has two shortcomings. On the one hand, its task scheduling granularity is large, while "the task processing scheme involved in this specification" can use the user's fine-grained idle time for task scheduling to reduce resource competition. On the other hand, the SmartSwap comparison scheme can only alleviate the frame rate drop caused by insufficient memory, but the frame rate drop may also be due to insufficient resources such as CPU and I / O. However, "the task processing scheme involved in this specification" can alleviate the frame rate drop by scheduling critical tasks under multiple resource bottlenecks.

[0176] The zTT comparison scheme establishes the relationship between CPU, GPU, power, temperature, frame rate information and CPU and GPU frequencies in a thermal environment based on reinforcement learning. And it achieves energy-saving application performance by jointly adjusting the CPU and GPU frequencies. However, the shortcomings of the zTT comparison scheme: sometimes the reason for the current application frame rate drop is not insufficient CPU and GPU resources, but memory pressure, I / O blocking, etc. Therefore, simply adjusting the CPU and GPU frequencies cannot fully guarantee the system resource availability of the foreground application.

[0177] Afterwards, we completed the same evaluation on terminal device B, which has a worse hardware configuration than terminal device A. Figure 7 The average frame rate of the four scenarios tested on terminal device B is shown. Compared with other scenarios, we can see that "the task processing scheme involved in this specification" attempts to meet the target frame rate as much as possible to reduce the drop in frame rate.

[0178] In one or more embodiments of the present specification, the effect of the "task processing scheme" involved in the present specification on the startup delay of the application is also evaluated: during the startup process, the application needs to build a new process, load data from the Flash storage, and render the user interface. Therefore, it always takes hundreds of milliseconds or even seconds to start an application. The startup delay of an application refers to the time from the first click of the user on the application icon in the Launcher to the display of all the contents of the homepage. In this evaluation, the device usage is divided into five stages: (a) select "14 popular applications pre-installed" and use the first five applications for three minutes; (b) sleep for one minute; (c) select "14 popular applications pre-installed" and use the last five applications for three minutes; (d) sleep for one minute, and (e) "14 popular applications pre-installed" and use the last four applications for two minutes. To ensure repeatability, the entire process is executed by a script. ADB (Android Debug Bridge) is then used to monitor the startup delay of each application.

[0179] like Figure 8 As shown, Figure 8 is the startup time of all test applications using the corresponding solution on the terminal device; specifically, it is the startup time of all test applications using the corresponding solution on the terminal device A;

[0180] like Fig. 9 As shown, Fig. 9 is the startup time of all test applications using the corresponding solution on the terminal device; specifically, it is the startup time of all test applications using the corresponding solution on the terminal device B;

[0181] When several applications are cached in the background, memory is often under pressure. In this case, the startup delay of the original system is extended compared to the ideal case where no applications are cached in the background. The delay is caused by the increase in service tasks, including kswapd, GC, and kworker. Using IA can improve the startup speed. Figure 8 and Fig. 9 As shown in the figure, after enabling IA on OPPO Find X3 and IQOO Neo3, the startup speed of applications increased by an average of 28.21% and 27.41%, respectively. Taking Skype as an example, compared with SmartSwap, IA increased the startup speed of Skype on OPPO Find X3 by 38.89%. Although SmartSwap can also recycle pages in advance, it only schedules kswapd tasks, and the scheduling timing is not as appropriate as the "task processing scheme involved in this specification" (iAware scheme), so its beneficial effects are limited.

[0182] Further, the inventors further evaluated the implementation analysis of the "task processing scheme involved in this specification" and the F1 score of the recall rate (Recall) to evaluate the overall performance of the "task processing scheme involved in this specification". Precision is the ratio of correct predictions to all positive predictions. Recall is the ratio of correct predictions to all actual positive predictions. Precision and recall affect each other, and the F1 score is used here, which is defined as:

[0183]

[0184] First, the accuracy of user busy prediction, kernel busy prediction and resource bottleneck prediction was verified with the aforementioned 14 real popular applications. A large amount of user experience perception and system resource perception data was collected, and a large number of applications were opened as background workloads. The accuracy was then measured by comparing the predicted values ​​with the actual labels. At the same time, the average accuracy of user busy prediction, kernel busy prediction and resource bottleneck prediction was calculated for all 14 applications. The user busy prediction accuracy reached 0.9184, the kernel busy prediction accuracy reached 0.9750, and the resource bottleneck prediction accuracy reached 0.8767; this shows that the "task processing solution involved in this specification" is of high quality.

[0185] Through the above analysis, in order to verify the task processing method involved in this specification, it was deployed on the terminal device in advance, and the effect was verified to be significant.

[0186] The following will be combined Fig.10 , the task processing device provided in the embodiment of the present application is introduced in detail. It should be noted that, Fig.10 The task processing device shown is used to execute the application Figures 1 to 10 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figures 1 to 9 The embodiment shown.

[0187] See also Fig.10 , which shows a schematic diagram of the structure of the task processing device of an embodiment of the present application. The task processing device 1 can be implemented as all or part of the terminal device through software, hardware or a combination of both. According to some embodiments, the task processing device 1 includes a data acquisition module 11, an information determination module 12 and a scheduling processing module 13, which are specifically used to:

[0188] Data acquisition module 11, used to acquire user interaction data and system resource status data;

[0189] An information determination module 12, configured to determine device operation task status information based on the user interaction data and the system resource status data;

[0190] The scheduling processing module 13 is used to perform task scheduling based on the device running task status information.

[0191] Optionally, the information determination module 12 is used to:

[0192] Determine foreground interaction task status information and kernel service task status information based on the user interaction data and the system resource status data;

[0193] The device running task status information is determined based on the foreground interaction task status information and the kernel service task status information; and / or, when the kernel service task status information is of the kernel service task busy type, resource bottleneck prediction processing is performed based on the foreground interaction task status information and the kernel service task status information to obtain resource bottleneck status information, and the device running task status information is determined based on the resource bottleneck status information, the foreground interaction task status information and the kernel service task status information.

[0194] Optionally, the information determination module 12 is used to:

[0195] Inputting the user interaction data and the system resource status data into a task status prediction model;

[0196] Controlling the task state prediction model to predict the foreground interaction task state to obtain foreground interaction task state information, and predicting the kernel service task state through the task state prediction model to obtain the kernel service task state information;

[0197] Control the task status prediction model to output running task status information corresponding to the foreground interaction dimension and / or the kernel service task dimension; and / or control the task status prediction model to perform resource bottleneck prediction processing to obtain resource bottleneck status information when the kernel service task status information is a kernel service task busy type, and output the device running task status information including the foreground interaction task status information, the kernel service task status information and the resource bottleneck status information.

[0198] Optionally, the device 1 is further used for:

[0199] Acquire sample data, and label the sample data with device running task status information tags, wherein the sample data includes sample user interaction data and sample system resource status data;

[0200] Create an initial task status prediction model;

[0201] The sample data is input into the initial task state prediction model for at least one round of model training. In each round of model training, the predicted device operation task state information corresponding to the sample data is determined through the initial task state prediction model. The model parameters of the initial task state prediction model are adjusted based on the predicted device operation task state information and the device operation task state information label to obtain the task state prediction model after model training.

[0202] Optionally, the device 1 is further used for:

[0203] Obtaining sample data including sample user interaction data and sample system resource status data;

[0204] Using the scene labeling condition to label the sample data with a foreground interactive task status label and a kernel service task status label;

[0205] And, a resource bottleneck label is marked on target sample data in the sample data, and the kernel service task status label of the target sample data is a kernel service task busy type.

[0206] Optionally, the sample user interaction data at least includes a target frame rate, a difference rate between the target frame rate and the system frame rate, a frame loss rate, and a human-computer interaction frequency; the sample system resource status data at least includes a system free memory value, an application available memory value, and an average service task processor usage rate; the device 1 is further used to:

[0207] If the first target frame rate in the first sample data is greater than or equal to the target frame rate threshold, the first difference rate is greater than or equal to the difference rate threshold, and the first human-computer interaction frequency is greater than the interaction frequency threshold, then marking the first sample data with a foreground application task busy label;

[0208] If the second target frame rate in the second sample data is less than the target frame rate threshold, and / or the second difference rate is less than the difference rate threshold, and / or the second human-computer interaction frequency is less than the interaction frequency threshold, then marking the second sample data with a foreground application task idle label;

[0209] If the third system free memory value in the third sample data is less than the system free memory threshold, the third application available memory value is less than the application available memory threshold, and the third average service task processor usage is greater than the processor usage threshold, then the third sample data is marked with a kernel service task busy label;

[0210] If the fourth system idle memory value in the fourth sample data is less than the system idle memory threshold, the fourth application available memory value is less than the application available memory threshold, and the fourth average service task processor usage is greater than the processor usage threshold, the fourth sample data is marked with a kernel idle label.

[0211] Optionally, the data acquisition module 11 is used to:

[0212] In response to the user's human-computer interaction operation, the step of acquiring user interaction data and system resource status data is executed.

[0213] Optionally, the scheduling processing module 13 is used to:

[0214] Determine a key scheduling task from at least one system operation task;

[0215] The key scheduling task is scheduled based on the device running task status information.

[0216] Optionally, the scheduling processing module 13 is used to:

[0217] Determine based on the device running task status information that the system task status is in a kernel service task busy state and a resource bottleneck task state, and determine the target bottleneck resource from the device running task status information;

[0218] A key static scheduling task and a key dynamic scheduling task are determined from at least one system operation task based on the target bottleneck resource.

[0219] Optionally, the scheduling processing module 13 is used to:

[0220] When the target bottleneck resource is a memory resource, a preset resource bottleneck scheduling mapping relationship is obtained, wherein the resource bottleneck scheduling mapping relationship is a mapping relationship between a reference memory resource bottleneck and a static memory scheduling task, and the resource bottleneck scheduling mapping relationship is used to determine a static memory processing task from at least one system running task, wherein the memory processing task includes at least one of a memory compression task, a memory collection task, and a memory garbage collection task;

[0221] When the target bottleneck resource is a processor resource, determining a dynamic processor occupation task based on the processor resource occupation from at least one system running task;

[0222] When the target bottleneck resource is an input / output resource, a dynamic input / output occupation task is determined from at least one system running task based on the input / output occupation amount.

[0223] Optionally, the scheduling processing module 13 is used to:

[0224] Determining based on the device running task status information that the foreground interactive task state is in the foreground interactive task idle state, then performing task execution processing on the key scheduling task;

[0225] If it is determined based on the device running task status information that the foreground interactive task status is in a foreground interactive task busy state, task delay processing is performed on the key scheduling task.

[0226] It should be noted that the task processing device provided in the above embodiment only uses the division of the above functional modules as an example when executing the task processing method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the task processing device provided in the above embodiment and the task processing method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0227] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0228] The present application also provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figures 1 to 9 The task processing method of the embodiment shown in the figure can be found in the specific execution process. Figures 1 to 9 The specific description of the illustrated embodiment will not be repeated here.

[0229] The present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor as described above. Figures 1 to 9 The task processing method of the embodiment shown in the figure can be found in the specific execution process. Figures 1 to 9 The specific description of the illustrated embodiment will not be repeated here.

[0230] Please refer to Fig.11 , which shows a block diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. The electronic device in the present application may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected via the bus 150.

[0231] The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 110 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0232] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The operating system may be an Android system, including a system deeply developed based on the Android system, an IOS system developed by Apple, including a system deeply developed based on the IOS system or other systems. The data storage area may also store data created by the electronic device during use, such as a phone book, audio and video data, chat record data, etc.

[0233] See also Fig.12As shown, the memory 120 can be divided into an operating system space and a user space. The operating system runs in the operating system space, and native and third-party applications run in the user space. In order to ensure that different third-party applications can achieve good operating results, the operating system allocates corresponding system resources to different third-party applications. However, different application scenarios in the same third-party application also have different requirements for system resources. For example, in the local resource loading scenario, the third-party application has higher requirements for disk reading speed; in the animation rendering scenario, the third-party application has higher requirements for GPU performance. The operating system and third-party applications are independent of each other, and the operating system often cannot perceive the current application scenario of the third-party application in a timely manner, resulting in the operating system being unable to perform targeted system resource adaptation according to the specific application scenario of the third-party application.

[0234] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0235] Taking the Android operating system as an example, the programs and data stored in the memory 120 are as follows: Fig.13As shown, the memory 120 may store a Linux kernel layer 320, a system runtime library layer 340, an application framework layer 360 and an application layer 380, wherein the Linux kernel layer 320, the system runtime library layer 340 and the application framework layer 360 belong to the operating system space, and the application layer 380 belongs to the user space. The Linux kernel layer 320 provides underlying drivers for various hardware of electronic devices, such as display drivers, audio drivers, camera drivers, Bluetooth drivers, Wi-Fi drivers, power management, etc. The system runtime library layer 340 provides the main feature support for the Android system through some C / C++ libraries. For example, the SQLite library provides database support, the OpenGL / ES library provides 3D drawing support, and the Webkit library provides browser kernel support, etc. The Android runtime library (Android runtime) is also provided in the system runtime library layer 340, which mainly provides some core libraries that allow developers to use the Java language to write Android applications. The application framework layer 360 provides various APIs that may be used when building applications. Developers can also use these APIs to build their own applications, such as activity management, window management, view management, notification management, content provider, package management, call management, resource management, and location management. At least one application runs in the application layer 380. These applications can be native applications that come with the operating system, such as contact applications, text messaging applications, clock applications, camera applications, etc.; they can also be third-party applications developed by third-party developers, such as game applications, instant messaging applications, photo beautification applications, etc.

[0236] Taking the operating system as an IOS system as an example, the programs and data stored in the memory 120 are as follows: Fig.14As shown, the IOS system includes: a core operating system layer 420 (Core OS layer), a core service layer 440 (Core Services layer), a media layer 460 (Media layer), and a touchable layer 480 (Cocoa Touch Layer). The core operating system layer 420 includes the operating system kernel, drivers, and underlying program frameworks, which provide functions closer to the hardware for use by the program framework located in the core service layer 440. The core service layer 440 provides system services and / or program frameworks required by the application, such as the foundation framework, account framework, advertising framework, data storage framework, network connection framework, geographic location framework, motion framework, etc. The media layer 460 provides audio-visual interfaces for the application, such as graphics and image related interfaces, audio technology related interfaces, video technology related interfaces, and wireless playback (AirPlay) interfaces for audio and video transmission technologies. The touchable layer 480 provides various commonly used interface-related frameworks for application development, and the touchable layer 480 is responsible for the user's touch interaction operations on the electronic device. For example, local notification service, remote push service, advertising framework, game tool framework, message user interface (UI) framework, user interface UIKit framework, map framework, etc.

[0237] exist Fig.14 Among the frameworks shown, the frameworks related to most applications include but are not limited to: the basic framework in the core service layer 440 and the UIKit framework in the touchable layer 480. The basic framework provides many basic object classes and data types, provides the most basic system services for all applications, and has nothing to do with UI. The classes provided by the UIKit framework are basic UI class libraries for creating touch-based user interfaces. iOS applications can provide UIs based on the UIKit framework, so it provides the basic architecture of applications for building user interfaces, drawing, processing and user interaction events, responding to gestures, etc.

[0238] Among them, the method and principle of implementing data communication between third-party applications and the operating system in the IOS system can be referred to the Android system, and this application will not go into details here.

[0239] Among them, the input device 130 is used to receive input instructions or data, and the input device 130 includes but is not limited to a keyboard, a mouse, a camera, a microphone or a touch device. The output device 140 is used to output instructions or data, and the output device 140 includes but is not limited to a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 are touch screen displays, which are used to receive touch operations on or near the user using any suitable object such as a finger or a touch pen, and to display the user interface of each application. The touch screen display is usually set on the front panel of the electronic device. The touch screen display can be designed as a full screen, a curved screen or a special-shaped screen. The touch screen display can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of the present application.

[0240] In addition, those skilled in the art will appreciate that the structure of the electronic device shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the electronic device also includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0241] In one or more embodiments of this specification, the execution subject of each step may be the electronic device described above. Optionally, the execution subject of each step is the operating system of the electronic device. The operating system may be an Android system, an IOS system, or other operating systems, which is not limited in the embodiments of this application.

[0242] The electronic device of the embodiment of the present application may also be equipped with a display device, which may be any device capable of realizing a display function, such as a cathode ray tube display (CR), a light-emitting diode display (LED), an electronic ink screen, a liquid crystal display (LCD), a plasma display panel (PDP), etc. The user may use the display device on the electronic device 101 to view displayed text, images, videos and other information. The electronic device may be a smart phone, a tablet computer, a gaming device, an AR (Augmented Reality) device, a car, a data storage device, an audio playback device, a video playback device, a notebook, a desktop computing device, a wearable device such as an electronic watch, an electronic glasses, an electronic helmet, an electronic bracelet, an electronic necklace, electronic clothing and other devices.

[0243] exist Fig.11 In the electronic device shown, the electronic device may be a terminal device, and the processor 110 may be used to call the application stored in the memory 120 and specifically execute the task processing method involved in one or more embodiments of this specification.

[0244] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0245] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A task processing method, It is characterized in that The method comprises: Obtain user interaction data and system resource status data; Determining device operation task status information based on the user interaction data and the system resource status data; Task scheduling is performed based on the device running task status information.

2. The method according to claim 1, It is characterized in that The determining, based on the user interaction data and the system resource status data, device operation task status information includes: Determine foreground interaction task status information and kernel service task status information based on the user interaction data and the system resource status data; The device running task status information is determined based on the foreground interaction task status information and the kernel service task status information; and / or, when the kernel service task status information is of the kernel service task busy type, resource bottleneck prediction processing is performed based on the foreground interaction task status information and the kernel service task status information to obtain resource bottleneck status information, and the device running task status information is determined based on the resource bottleneck status information, the foreground interaction task status information and the kernel service task status information.

3. The method according to claim 2, It is characterized in that Determining the foreground interaction task status information and the kernel service task status information based on the user interaction data and the system resource status data, and determining the device operation task status information based on the foreground interaction task status information and the kernel service task status information; And / or, when the kernel service task status information is of the kernel service task busy type, performing resource bottleneck prediction processing based on the foreground interaction task status information and the kernel service task status information to obtain resource bottleneck status information, and determining the device running task status information based on the resource bottleneck status information, the foreground interaction task status information and the kernel service task status information, including: Inputting the user interaction data and the system resource status data into a task status prediction model; Controlling the task state prediction model to predict the foreground interaction task state to obtain foreground interaction task state information, and predicting the kernel service task state through the task state prediction model to obtain the kernel service task state information; Control the task status prediction model to output running task status information corresponding to the foreground interaction dimension and / or the kernel service task dimension; and / or control the task status prediction model to perform resource bottleneck prediction processing to obtain resource bottleneck status information when the kernel service task status information is a kernel service task busy type, and output the device running task status information including the foreground interaction task status information, the kernel service task status information and the resource bottleneck status information.

4. The method according to claim 3, It is characterized in that The method further comprises: Acquire sample data, and label the sample data with device running task status information tags, wherein the sample data includes sample user interaction data and sample system resource status data; Create an initial task status prediction model; The sample data is input into the initial task state prediction model for at least one round of model training. In each round of model training, the predicted device operation task state information corresponding to the sample data is determined through the initial task state prediction model. The model parameters of the initial task state prediction model are adjusted based on the predicted device operation task state information and the device operation task state information label to obtain the task state prediction model after model training.

5. The method according to claim 4, It is characterized in that The acquiring of sample data and labeling of the sample data with a device running task status information label includes: Obtaining sample data including sample user interaction data and sample system resource status data; Using the scene labeling condition to label the sample data with a foreground interactive task status label and a kernel service task status label; And, a resource bottleneck label is marked on target sample data in the sample data, and the kernel service task status label of the target sample data is a kernel service task busy type.

6. The method according to claim 5, It is characterized in that The sample user interaction data at least includes the target frame rate, the difference between the target frame rate and the system frame rate, the frame loss rate, and the human-computer interaction frequency; the sample system resource status data at least includes the system free memory value, the application available memory value, and the average service task processor usage rate. The step of labeling the sample data with a foreground interactive task status label and a kernel service task status label using a scene label labeling condition includes: If the first target frame rate in the first sample data is greater than or equal to the target frame rate threshold, the first difference rate is greater than or equal to the difference rate threshold, and the first human-computer interaction frequency is greater than the interaction frequency threshold, then marking the first sample data with a foreground application task busy label; If the second target frame rate in the second sample data is less than the target frame rate threshold, and / or the second difference rate is less than the difference rate threshold, and / or the second human-computer interaction frequency is less than the interaction frequency threshold, then marking the second sample data with a foreground application task idle label; If the third system free memory value in the third sample data is less than the system free memory threshold, the third application available memory value is less than the application available memory threshold, and the third average service task processor usage is greater than the processor usage threshold, then the third sample data is marked with a kernel service task busy label; If the fourth system idle memory value in the fourth sample data is less than the system idle memory threshold, the fourth application available memory value is less than the application available memory threshold, and the fourth average service task processor usage is greater than the processor usage threshold, the fourth sample data is marked with a kernel idle label.

7. The method according to claim 1, It is characterized in that The obtaining of user interaction data and system resource status data includes: In response to the user's human-computer interaction operation, the step of acquiring user interaction data and system resource status data is executed.

8. The method according to claim 1, It is characterized in that The task scheduling process based on the device running task status information includes: Determine a key scheduling task from at least one system operation task; The key scheduling task is scheduled based on the device running task status information.

9. The method according to claim 8, It is characterized in that Determining a key scheduling task from at least one system running task includes: Determine based on the device running task status information that the system task status is in a kernel service task busy state and a resource bottleneck task state, and determine the target bottleneck resource from the device running task status information; A key static scheduling task and a key dynamic scheduling task are determined from at least one system operation task based on the target bottleneck resource.

10. The method according to claim 9, It is characterized in that The determining of a key static scheduling task and a key dynamic scheduling task from at least one system operation task based on the target bottleneck resource includes: When the target bottleneck resource is a memory resource, a preset resource bottleneck scheduling mapping relationship is obtained, wherein the resource bottleneck scheduling mapping relationship is a mapping relationship between a reference memory resource bottleneck and a static memory scheduling task, and the resource bottleneck scheduling mapping relationship is used to determine a static memory processing task from at least one system running task, wherein the memory processing task includes at least one of a memory compression task, a memory collection task, and a memory garbage collection task; When the target bottleneck resource is a processor resource, determining a dynamic processor occupancy task based on the processor resource occupancy from at least one system running task; When the target bottleneck resource is an input / output resource, a dynamic input / output occupation task is determined from at least one system running task based on the input / output occupation amount.

11. The method according to claim 8, It is characterized in that The performing task scheduling processing on the key scheduling task based on the device running task status information includes: Determining based on the device running task status information that the foreground interactive task state is in the foreground interactive task idle state, then performing task execution processing on the key scheduling task; If it is determined based on the device running task status information that the foreground interactive task status is in a foreground interactive task busy state, task delay processing is performed on the key scheduling task.

12. A task processing device, It is characterized in that The device comprises: A data acquisition module, used to acquire user interaction data and system resource status data; An information determination module, used to determine device operation task status information based on the user interaction data and the system resource status data; The scheduling processing module is used to perform task scheduling processing based on the device running task status information.

13. A computer storage medium, It is characterized in that The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 11.

14. An electronic device, It is characterized in that include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 11.

Images