Control method of electronic equipment and electronic equipment

Abstract

The invention provides a control method of electronic equipment and the electronic equipment. The method comprises the steps of monitoring system resource conditions of the electronic equipment; determining attribute information of the first target process under the condition of determining that a first regulation and control condition is met based on the system resource condition; controlling the running state of the first target process based on the attribute information to recycle system resources of the electronic equipment or provide corresponding target function service for a target application of the electronic equipment; wherein the first target process is at least one to-be-run process of an auxiliary application of the electronic equipment, and the auxiliary application is an application used for providing a corresponding function service for the target application.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of electronic technology, and in particular to a control method for an electronic device and an electronic device. Background Technology

[0002] In related technologies, electronic devices may experience problems such as process startup failure and lag when under high load, making it impossible to provide the functional services corresponding to the applications used by users in a timely and effective manner, thus degrading the user experience. Summary of the Invention

[0003] In view of this, embodiments of this application provide at least one control method for an electronic device and an electronic device.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a control method for an electronic device, comprising: monitoring the system resource status of the electronic device; determining attribute information of a first target process when a first control condition is met based on the system resource status; controlling the running state of the first target process based on the attribute information to reclaim system resources of the electronic device or provide corresponding target function services to a target application of the electronic device; wherein the first target process is at least one running process of an auxiliary application of the electronic device, and the auxiliary application is an application used to provide corresponding function services to the target application.

[0005] This application provides an electronic device, including at least one processor and an intelligent agent capable of running on the processor. The intelligent agent is capable of executing or invoking at least one processing model to perform the following operations: monitoring the system resource status of the electronic device; determining the attribute information of a first target process when a first control condition is met based on the system resource status; controlling the running state of the first target process based on the attribute information to reclaim the system resources of the electronic device or provide corresponding target function services to a target application of the electronic device; wherein, the first target process is at least one process to be run of an auxiliary application of the electronic device, and the auxiliary application is an application used to provide corresponding function services to the target application.

[0006] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this application. Attached Figure Description

[0007] Figure 1 This is a schematic diagram illustrating the implementation flow of a control method for an electronic device provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating the combination relationship between a game application and a first target process provided in an embodiment of this application; Figure 3This is a schematic diagram of the implementation process of a resource load-aware scheduling method provided in an embodiment of this application. Figure 1 ; Figure 4 This is a schematic diagram of a process priority configuration based on a state machine provided in an embodiment of this application; Figure 5 This is a schematic diagram of the implementation process of a resource load-aware scheduling method provided in an embodiment of this application. Figure 2 ; Figure 6 This is a schematic diagram illustrating the correspondence between time features and attenuation weights provided in an embodiment of this application; Figure 7 This is a schematic diagram of the implementation process of a resource load-aware scheduling method provided in an embodiment of this application. Figure 3 ; Figure 8 This is a schematic diagram illustrating the composition structure of the original feature library of a target game application provided in an embodiment of this application; Figure 9 This is a schematic diagram of the composition structure of a feature group of a target game application provided in an embodiment of this application; Figure 10 This is a schematic diagram of the composition structure of a feature pool provided in an embodiment of this application; Figure 11 This is a schematic diagram illustrating the switching of the usage state of a feature pool provided in an embodiment of this application; Figure 12 This is a schematic diagram of the implementation process of a resource load-aware scheduling method provided in an embodiment of this application. Figure 4 ; Figure 13 This is a schematic diagram of the implementation process of a resource load-aware scheduling method provided in an embodiment of this application. Figure 5 ; Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0008] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0010] In the following description, the terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first / second / third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application. It should also be noted that, for ease of description, only the parts relevant to the application are shown in the accompanying drawings.

[0011] This application provides a control method for an electronic device, which can be executed by the electronic device. The electronic device refers to a device with data processing capabilities, such as a server, laptop, tablet, desktop computer, smart TV, set-top box, or mobile device (e.g., mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device). Figure 1 As shown, the method includes the following steps S101 to S103: Step S101: Monitor the system resource status of electronic devices.

[0012] Here, system resource status represents the usage status of system resources in an electronic device. For example, system resource usage status may include, but is not limited to, at least one of the following: Central Processing Unit (CPU) utilization, memory utilization, Neural-network Processing Unit (NPU) utilization, Graphics Processing Unit (GPU) utilization, processor temperature, fan speed, etc.

[0013] System resource status can be used to assess whether an electronic device is currently under high load, thereby determining whether the running status of various processes within the device needs to be adjusted. For example, if the user's application requires running model-related processes, such as at least one of the following: Optical Character Recognition (OCR) model, Computer Vision (CV) model, or You Only Look Once (YOLO) model, system resources may be consumed rapidly, leading to slower function response or process startup failure.

[0014] During implementation, electronic devices can continuously collect system resource information, thereby providing a more accurate scheduling basis for subsequent dynamic process scheduling, improving the overall efficiency, stability and response speed of electronic devices, and enhancing the user experience.

[0015] Step S102: If the system resource status determines that the first control condition is met, determine the attribute information of the first target process.

[0016] Here, the control conditions are pre-set trigger conditions based on the system resource status. When the system resource status meets the control conditions, control operations on processes in electronic devices will be triggered to reclaim system resources and / or reallocate system resources to processes with higher priority.

[0017] The determination that the system resource status meets the first control condition indicates that the system resources have reached a saturation state.

[0018] In some implementations, the system resource status can be determined to meet the first control condition when the system resource usage parameter values ​​exceed preset thresholds. For example, the system resource status can be determined to meet the first control condition when the memory usage rate is higher than a preset memory usage rate upper limit. Similarly, the system resource status can be determined to meet the first control condition when the processor temperature is higher than a preset temperature threshold. It is understood that the system resource status can be determined to meet the first control condition when the usage parameters of one or more system resources exceed their corresponding preset thresholds.

[0019] The primary target process is a process or group of processes running on an electronic device. Attribute information may include, but is not limited to, at least one of the following: configuration priority, identification information, and description information of the process or group of processes, used to characterize the service type and / or priority of the process or group of processes. Configuration priority characterizes the priority of configuring system resources. For example, if the process's identification information indicates that its configuration priority is the highest level, the process will be prioritized for operation when system resources are scarce. Similarly, if the process's identification or description information indicates that its service type is a service unrelated to the necessary services for application operation, the resources occupied by this process may be released first when system resources are scarce.

[0020] In some implementations, the descriptive information may also be used to describe the size of system resources required by a process or combination of processes.

[0021] Step S103: Control the running state of the first target process based on attribute information to reclaim system resources of the electronic device or provide corresponding target function services to the target application of the electronic device; wherein, the first target process is at least one running process of an auxiliary application of the electronic device, and the auxiliary application is an application used to provide corresponding function services to the target application.

[0022] In some implementations, the control methods for the running state may include, but are not limited to, loading running, waiting to run (suspending), terminating or removing processes. For example, when system resources are scarce, low-priority processes can be terminated or removed from the waiting running processes to reclaim system resources. Conversely, when system resources are sufficient, highly active (frequently used) processes can be preloaded to reduce latency when users access corresponding functional services.

[0023] The target process is a process running by an auxiliary application in an electronic device, used to provide the corresponding functional services of that auxiliary application. Specifically, the execution of the first target process is used to provide the corresponding target functional services to the target application, and the running state of the first target process may affect the availability of the target functional services. The processes to be run are those that need to be loaded and executed.

[0024] Both target applications and auxiliary applications are applications running on electronic devices. It is understood that an auxiliary application can correspond to different target applications, or to different types of target applications. This application does not limit the type or number of target applications. For example, an auxiliary application can be an application corresponding to a specific function of a target application. As another example, an auxiliary application can be an application corresponding to at least one function of a target application of a certain application type.

[0025] For example, when the target application is a game application, the auxiliary application can be an artificial intelligence (AI) agent application that assists the game, such as a game assistant, and the first target process can be a process or a combination of processes that provides target functional services such as highlighting, strategy, and replay. When the target application is an audio-visual application, the auxiliary application can be a proxy application related to audio-visual entertainment, such as an audio-visual assistant, and the first target process can be a process or a combination of processes that provides target functional services such as editing exciting clips, generating highlight moments, and recording playback history. When the target application is a meeting application, the auxiliary application can be an agent application that supports meeting-related functional services, and the first target process can be a process or a combination of processes that provides target functional services such as summarizing meetings, speech recognition, counting attendees, and scheduling meeting times. When the target application is a learning application, the auxiliary application can be an agent application that supports learning functional services, and the first target process can be a process or a combination of processes that provides target functional services such as summarizing, generating training question banks, and conducting mock exams for the learning application.

[0026] In this embodiment, by monitoring the system resource status of the electronic device and, when the system resource status meets the first control condition, controlling the running state of the first target process based on its attribute information, system resources are reclaimed or target function services corresponding to the target application are provided on demand. In this way, the electronic device can, based on real-time resource load, control the running state of the process to achieve automatic system resource scheduling, reducing the problem of process startup failures or process lag under high load; furthermore, when system resources support it, the target function services corresponding to the target application can be pre-determined, and by controlling the operation of the first target process, the response latency of the target function services can be reduced, thereby improving the system stability and response efficiency of the electronic device.

[0027] In some embodiments, step S101 may include step S111 or step S112: Step S111: In response to the electronic device starting and running the first application, perform the step of monitoring the system resource status of the electronic device. The first target process includes at least one running process of a first auxiliary application for providing a first type of functional service to the first application.

[0028] Here, the first application is at least one application in the target application, the first type of functional service is at least one functional service in the target functional services, and the first auxiliary application is at least one auxiliary application corresponding to the first application.

[0029] For example, if the first application is a pre-defined game application, the first auxiliary application may include a game assistant, and the first target process may be a process related to the game assistant in providing game strategy services. The game strategy service-related processes may include, but are not limited to, frame capture processes, inference management processes, strategy business processes, and CV inference processes. It is understood that different auxiliary applications may run the same types of processes; for example, both the game strategy service and the highlighting service require the execution of frame capture and inference management processes. Therefore, when different functional services correspond to the same process, multiple corresponding functional services can be provided through a single process, without repeatedly calling the same process.

[0030] In practice, when a user activates the first application, the electronic device can determine the functional requirements of the first application and further identify the first auxiliary application corresponding to the first type of functional service.

[0031] Step S112: During the operation of the second application on the electronic device, the step of monitoring the system resource status of the electronic device is performed. The first target process includes at least one running process of the second auxiliary application used to provide the second type of functional services to the second application.

[0032] Here, the second application is at least one application in the target application, the second type of functional service is at least one functional service in the target functional services, and the second auxiliary application is at least one auxiliary application corresponding to the second application.

[0033] In some implementations, the system resource status of the electronic device can be monitored at a preset period during the operation of the second application, thereby gaining real-time insight into the system resource usage during the operation of the second application, and further intelligently adjusting the system resources based on the running status of the first target process.

[0034] In some implementations, during the operation of the first application, in response to the startup and operation of the second application, a step of monitoring the system resource status of the electronic device can also be performed. In this way, in response to the potential resource consumption of a newly added application in the system, by monitoring the usage of system resources, the system resources can be intelligently adjusted based on the running status of the first target process.

[0035] In some implementations, when the first application and the second application are of the same type, the first auxiliary application and the second auxiliary application are the same, and the first target process is the same or different. For example, if both the first application and the second application are game applications, then the corresponding auxiliary applications are both game assistants. Furthermore, the running state of the first target process can be controlled based on the process types required by the first application and the second application, respectively.

[0036] It is understood that the functional services required by the first application and the second application may be the same or different. Therefore, the first target process corresponding to the first application and the first target process corresponding to the second application may be the same or different. For example, such as Figure 2 As shown, game applications 1 and 2 both require highlight and replay services, game application 3 only requires highlight services, and game applications 4 and 5 both require highlight and strategy guide services. When the first and second applications are game application 1 and game application 2 respectively, or when the first and second applications are game application 4 and game application 5 respectively, the first target process of the first and second applications is the same. When the first and second applications are game application 1 and game application 3 respectively, the first target process of the first and second applications is partially the same.

[0037] In some implementations, when the first application and the second application belong to different application types, the first auxiliary application and the second auxiliary application are different, and their first target processes are different. It is understood that when the first application and the second application belong to different types, the first auxiliary application and the second auxiliary application they depend on will also be different, resulting in differences in their corresponding first target processes. For example, the first application is a video editing application that depends on an image processing auxiliary application, while the second application is a music creation application that depends on an audio analysis auxiliary application. Due to different functional requirements, the process combinations, number of processes, and / or process priorities triggered by the first application and the second application may differ. Therefore, resource management needs to handle them separately to ensure that each type of application can obtain the system resources it needs.

[0038] In this embodiment, system resource status is monitored when the electronic device starts the first application and runs the second application, and dynamic management is performed based on the pending processes of the required auxiliary applications. This allows for flexible selection of corresponding auxiliary applications and the first target process for different application scenarios. On the one hand, this improves the targeting and flexibility of system resource management; on the other hand, matching appropriate service modules (auxiliary applications) according to actual business needs further enhances the device's resource utilization efficiency and service response capabilities.

[0039] Understandably, when multiple applications are running concurrently, resource conflicts can be reduced and the response speed and stability of electronic devices can be improved by rationally arranging the running status of the primary target process of each application.

[0040] In some embodiments, step S102 may include steps S121 to S123: Step S121: If the usage parameter value of at least one target hardware of the electronic device representing the system resource status is greater than the corresponding preset parameter value, it is determined that the first control condition is met.

[0041] Here, the usage parameter values ​​of the target hardware are quantitative indicators that characterize the current load of the target hardware. For example, they may include, but are not limited to, the utilization rate, usage rate, and / or usage duration of any one of the CPU, memory, NPU, GPU, etc.

[0042] The preset parameter values ​​are upper limits set based on device performance evaluation and are used to determine whether to enter the resource control process. If at least one target hardware's usage parameter value is greater than the corresponding preset parameter value, it indicates that the system resources corresponding to the target hardware are under high load. For example, if the CPU utilization exceeds 80% for three consecutive minutes, it can be determined that the system resources are saturated, thus meeting the first control condition. Similarly, if the memory utilization reaches 60%, it can be determined that the first control condition is met, and memory needs to be released.

[0043] Step S122: If it is determined that there are processes waiting to be run, determine the priority information configured for the first target process. The priority information is used to configure the priority order in which the first target process is loaded or removed.

[0044] Here, there are processes in a waiting state, indicating that there are unexecuted process tasks waiting for resource release. For example, the existence of processes in a waiting state can include processes in a state machine that are in a waiting state, or processes suspended in the background (i.e., keeping the queue of suspended processes non-empty). Further, the running state of the first target process is controlled according to the priority information configured for each process.

[0045] Priority is used to define the order in which different processes are loaded or terminated when resources are limited.

[0046] In some implementations, priorities can be divided into multiple levels in sequence. For example, the first priority represents the highest priority, and processes with the first priority are typically those providing core services. The second priority represents a priority lower than the first priority; in the event of resource saturation, processes with the second priority can be terminated to release resources. The third priority represents the lowest priority, and processes with the third priority are typically those providing non-core services; in the event of resource saturation, processes with the third priority can be terminated first to release resources. It is understood that the number and type of priorities can be adjusted according to the total resources of the electronic device and the type of application, etc., and the embodiments of this application do not limit this.

[0047] In some implementations, a state machine can be used to manage process priorities. For example, based on the startup order, the first process started can be designated as the highest priority process (set to the first running state), and the priorities of subsequently started processes can be sequentially reduced according to their startup order (set to the second running state, third running state, etc.). Alternatively, based on user preferences, foreground processes can be designated as high-priority processes (set to the running state), and background processes as low-priority processes (set to the waiting state), allowing for further control.

[0048] Step S123: If it is determined that there are no processes waiting to run, return to the step of monitoring the system resources of the electronic device.

[0049] Here, if it's determined that there are no processes currently waiting to run, it indicates that all executable tasks have been completed, or that current system resources are insufficient to support the startup of any new processes. In this case, it's necessary to continue monitoring the electronic device's system resources to await the next wave of resource changes or task requests. This way, monitoring doesn't stop after a single resource usage assessment, allowing for rapid responses to changes in resource status, thereby improving device responsiveness and task execution efficiency.

[0050] In this embodiment, the system resource status is compared with preset parameters to determine whether control logic is triggered, and resource allocation is performed in conjunction with process priority information. This effectively reduces unnecessary resource consumption and achieves more refined process management. By monitoring system resources in real time and combining them with a process priority management mechanism, processes can be flexibly selected for loading or removal when resources are limited, reducing resource waste and process conflicts, further improving the stability and response speed of the device in a multi-process environment, thereby optimizing the user experience.

[0051] In some embodiments, step S103 may include at least one of steps S131 to S133: Step S131: When the attribute information characterizes the first target process as having the first priority, control the first target process to be in a loading and running state or a waiting loading state, so as to provide the corresponding target function service to the target application of the electronic device.

[0052] Here, when the first target process is configured as the first priority, it means that the functional services provided by this first target process are core services or important functions currently being used by the user, and should not be terminated or uninstalled. For example, in a game scenario, the frame capture process used to capture game frames is usually set to the first priority because this process provides basic support for services such as highlighting, strategy guides, and replay analysis. The first target process with the first priority can remain running and continue loading to respond to needs at any time, or enter a waiting loading state, that is, retain some resources in memory but do not immediately execute tasks, thereby reducing device latency and improving response speed. Through this control method, the problem of device performance degradation caused by frequent restarts of the first target process can be reduced.

[0053] Step S132: When the attribute information indicates that the first target process is configured as the second priority, control the first target process to enter the termination state in order to reclaim the system resources occupied by the first target process.

[0054] Step S133: When the attribute information indicates that the first target process is configured as the third priority, control the first target process to enter the termination state in order to reclaim the system resources occupied by the first target process.

[0055] Here, both second and third priorities indicate that the corresponding process is not essential for the current application's operation, or that the process's functional services can be reloaded later without affecting the user experience. The second priority is higher than the third priority. For example, the inference module of a smart assistant (such as an OCR model or CV model) may only be called in specific scenarios. When it is detected that such second-priority first-target processes are no longer active and the current resource load is high, these first-target processes are terminated, releasing the memory and computing power they occupy, thereby improving device performance. Similarly, processes handling non-real-time background tasks (such as data uploads and log recording) typically have a low trigger frequency. If the user has not used these functions for a long time, they can be terminated, freeing up resources for other higher-priority processes.

[0056] In this embodiment, by setting different priorities to control the running state of the first target process, it is possible to ensure that high-priority processes corresponding to critical functions run normally while releasing resources occupied by low-priority processes in a timely manner. Thus, by dynamically adjusting the running state of the first target process, resource allocation is optimized, reducing device lag or crashes caused by excessive resource consumption by the first target process, improving resource utilization and device response speed, and especially enhancing the user experience in multi-tasking environments.

[0057] In some embodiments, controlling the running state of the first target process based on attribute information in step S103 above may include the following steps S141 to S142: Step S141: Based on the historical behavior data of the target user and / or the demand information of the target application, predict the activity level of the first target process. The activity level represents the likelihood that the first target process will be used to provide corresponding functional services to the target application.

[0058] Here, activity level can represent the likelihood of a process being invoked within a certain time interval in the future. The higher the activity level, the more likely the process is to be invoked again, so the response latency of the corresponding service can be reduced by keeping it running or loading it in advance. The lower the activity level, the less likely the process is to be invoked in the near future, so it can be uninstalled or not started again in the short term, thereby reducing unnecessary waste of resources.

[0059] Historical behavioral data includes records of user behavior while using electronic devices or applications. For example, this may include, but is not limited to, records of opening, using, and closing electronic devices, applications, or processes. This behavioral data may include total usage time, usage frequency, duration of each usage session, and operation path information. By analyzing the historical behavioral data of target users, their usage habits and / or preferences can be inferred, thereby predicting the likelihood of using related processes and providing a basis for scheduling the primary target process.

[0060] The target application's requirement information characterizes its specific needs for resources and services, i.e., the types or content of functional services required, reflecting its dependence on the primary target process. For example, based on the frequency of user usage of the primary target process, if the frequency of use of the primary target process exceeds a preset limit (e.g., 10 times) within a single day, the activity level of the primary target process can be determined. Similarly, based on whether the target application relies on auxiliary applications for image recognition, speech processing, or data analysis, the activity level of the primary target process related to image recognition, speech processing, or data analysis can be determined.

[0061] Understandably, combining the target user's historical behavior data with the target application's demand information can comprehensively assess the activity level of the primary target process, thereby further improving the accuracy of activity prediction.

[0062] Step S142: Based on attribute information and activity level, control the first target process to enter any of the following states: terminated state, loading running state, or waiting to load state.

[0063] Here, when system resources are saturated, in addition to the service type and / or priority of the first target process determined based on attribute information, the activity level of the first target process, i.e., the likelihood of it being used, is further considered to control the running state of the first target process. This allows for a more accurate determination of the running state to be controlled for the first target process. In this way, when system resources are saturated and the activity level of the first target process is low, the first target process can be controlled to enter a terminated state; when system resources are saturated and the activity level of the first target process is low but there is still potential demand, the first target process can be controlled to enter a waiting-to-load state; and when system resources are saturated and the activity level of the first target process is high, the first target process can be controlled to enter a loading and running state, providing a rapid and timely response when the target application needs the relevant services of the first target process.

[0064] In this embodiment, the activity level of the first target process is predicted based on user behavior data and application demand information. Furthermore, the running state of the first target process is dynamically adjusted according to its attribute information and activity level. This allows for the early identification and handling of first target processes that may not require invocation, or the retention of first target processes with a high probability of invocation. This enables on-demand start / stop and resource allocation of the first target process, saving resources and time consumed by repeatedly loading processes.

[0065] In some embodiments, when the startup of a first application process is detected, the control method of the above-mentioned electronic device may further include at least one of the following steps S151 to S153: Step S151: If it is determined that there is a second application process that is currently being executed, add the first application process to the waiting execution state, and add the first application process to the waiting execution queue if the system resource status of the electronic device meets the second control condition.

[0066] Here, the system resource status meeting the second control condition indicates that the system resources have not reached a saturated state. In some implementations, the system resource status can be determined to meet the second control condition if the system resource usage does not exceed a preset threshold. It is understood that the system resource status can be determined to meet the second control condition if the usage of multiple system resources does not exceed their corresponding preset thresholds.

[0067] The "Running" state represents the current state where an application process is occupying system resources and running. In implementation, the "Running" state can be managed using a state machine. For example, if a state machine indicates that a second application process is currently in the "Running" state (i.e., in the "Running" state), then the second application process has higher priority and uses resources first. For instance, adding a "Pending" state to the state machine involves defining a specific state node to identify that the application process is currently in a waiting execution phase, and switching to a higher priority state (such as the "Running" state) when resources become available or the scheduling policy allows. For example, if a game application is already running on the device, in response to the launch of another game application, the running state of the launched game application will be added to the "Waiting to Execute" state, and when system resources become available, it will be added to the queue of applications waiting to be executed, running sequentially.

[0068] Understandably, when system resources are limited, multiple application processes may not be able to run simultaneously. In such cases, application processes can be added to a queue to wait. The execution queue is used to store application processes that cannot run immediately due to insufficient resources. During implementation, the application processes in the queue can be sorted according to their priority and waiting time, thereby improving the rationality and efficiency of resource allocation.

[0069] Step S152: If it is determined that there is no second application process in the running state, add the first application process as the running state, and load and run the first application process if the system resource status of the electronic device meets the second control condition.

[0070] The "Running" status indicates that no other application processes are currently consuming system resources. Newly launched application processes can be directly set to the running state, meaning they have the highest scheduling priority and can immediately acquire the necessary computing resources and begin executing their tasks. It's understandable that the resource status judgment threshold corresponding to the second control condition can be the same as or different from the resource status judgment threshold corresponding to the first control condition. When system resources meet the second control condition (i.e., when system resources are sufficient), newly launched application processes do not need to enter the waiting queue and can run directly, thereby improving device response speed and reducing user waiting time. This allows for rapid response to user operations when system resources are sufficient, shortening application process startup latency, improving user experience, user satisfaction, and device availability.

[0071] Step S153: After detecting that the first application process has entered the termination state, predict the activity level of the first target process based on the historical behavior data of the target user, and control the first target process to enter the waiting execution queue or enter the termination state based on the activity level.

[0072] Here, the first application process is related to the target user's historical behavior data. For example, if a user reopens the same or a different game application within one hour after closing it, and the first application process is the game process, it can be predicted that the user will use the frame capture process (the first target process) required by the game application again shortly after this closure. Therefore, the frame capture process is controlled to enter the waiting execution queue. Similarly, if a user does not reopen the same or a different game application within one day after repeatedly closing it, and the first application process is the game process, it can be predicted that the user will not use the non-essential strategy process of the game application shortly after this closure. Therefore, the strategy process is controlled to enter the terminated state.

[0073] The execution queue in this step is used to store the first target process that is not currently running but is predicted to be reused by the user. This reduces resource consumption by minimizing the repeated loading of the same first target process. The final state completely releases the first target process, freeing it from system resources for other processes. Thus, by optimizing the lifecycle management of the first target process through user behavior prediction, unnecessary resource waste and redundant process loading overhead can be reduced, improving device operating efficiency and responsiveness, and extending device battery life.

[0074] In this embodiment, by introducing an application process state and queue management mechanism, the application process and the first target process can be managed and scheduled in an orderly manner, which helps to balance resource allocation issues when multiple processes are executed concurrently. This reduces process conflicts, balances system load and functional service response speed, reduces device performance degradation under high load, and improves device stability.

[0075] In some embodiments, the control method for the above-mentioned electronic device may further include the following step S161: Step S161: Control the loading strategy of the feature pool called by the second target process based on the system resource status. The second target process is the feature recognition process of the auxiliary application of the electronic device. The feature pool includes reference features for identifying and comparing the original features of the target application captured by the auxiliary application. The loading strategy includes at least the switching of the usage state of the feature pool and / or the adjustment of the data volume of the reference features in the feature pool.

[0076] For example, the second target process may include a process capable of image recognition, behavior analysis, model inference, etc., used to identify and compare the original features captured by the target application, and further determine the feature loading progress of the target application.

[0077] A feature pool is a pre-built set of reference features used to match and identify the original features extracted from the target application. It can determine the current loading progress and pending data of the target application when the original features match the reference features. In implementation, the reference features in the feature pool can include various forms of data such as image templates, text patterns, and behavioral trajectories, depending on the application scenario. For example, in a game application, the feature pool could contain feature data such as player actions, weapon recognition, and scene elements.

[0078] Loading strategies are used to dynamically adjust the usage of the feature pool based on system resource conditions. This includes two aspects: first, usage state switching, i.e., pausing the loading or use of some feature pools when resources are scarce; and second, data volume adjustment, i.e., dynamically increasing or decreasing the number of reference features loaded in the feature pool based on available resources. For example, if CPU utilization exceeds a set threshold (e.g., 70%) or memory usage falls below a critical value (e.g., 500MB), the loading of new reference features can be reduced, or reference features that have already been compared can be removed, retaining only uncompared reference features to reduce resource overhead. By employing different loading strategies, feature comparison needs under different resource environments can be flexibly addressed, improving resource utilization while reducing process startup failures or lag caused by insufficient resources.

[0079] In this embodiment of the application, by dynamically adjusting the loading strategy of the feature pool called by the second target process, the usage of the feature pool can be dynamically adjusted according to the current resource usage of the electronic device, thereby optimizing resource allocation during the feature comparison process.

[0080] In some embodiments, step S161 may include at least one of steps S171 to S172: Step S171: Based on the system resource status, control the status of the feature pool that is in the waiting loading state and / or in the execution state, or the amount of data of the reference features in the feature pool.

[0081] Here, a feature pool in the "waiting to load" state indicates that the reference features in that feature pool have not been loaded. A feature pool in the "execution" state indicates that at least some of the reference features in that feature pool have been loaded.

[0082] In some implementations, step S171 may include at least one of steps S1711 to S1715: Step S1711: If the first control condition is met based on the system resource status, control the feature pool that is in the waiting loading state to enter the unloading state.

[0083] When the current resource load is high, performing an unloading operation on the feature pool that is waiting to be loaded can release resources based on the currently loaded feature pool without affecting feature comparison.

[0084] Step S1712: If the system resource status determines that the first control condition is met, reduce the amount of data of the reference features in the feature pool that is in the execution state.

[0085] During implementation, when the current resource load is high, resources can be released and reclaimed by removing reference features that are identical to the original features after comparison; or by reducing the amount of data of unloaded reference features during the subsequent loading of uncompared reference features, the use of resources can be reduced.

[0086] Step S1713: If the system resource status determines that the first control condition is met, reduce the amount of reference feature data in the feature pool that is in the waiting loading state.

[0087] Here, if all reference features in the feature pool in the execution state have been loaded, a feature pool in the waiting-to-load state can be added in response to the system resource status determining that the second control condition is met, and the reference features in the waiting-to-load feature pool can be loaded. During the loading process, if the system resource status determines that the first control condition is met, since the reference features in the waiting-to-load feature pool will not be used for comparison yet, the loading of reference features in the waiting-to-load feature pool can be reduced, thereby reducing the consumption of system resources.

[0088] Step S1714: If the second control condition is met based on the system resource status, increase the amount of reference feature data in the feature pool that is in the execution state.

[0089] Here, when system resources are sufficient, such as CPU space rate being higher than the preset lower limit and sufficient memory space, the amount of reference feature data in the feature pool that is in the execution state can be increased to increase the amount of reference feature data used for subsequent feature comparison, thereby improving the efficiency and stability of feature comparison and maximizing device performance.

[0090] Step S1715: If the second control condition is met based on the system resource status, increase the amount of reference feature data in the feature pool that is in the waiting loading state.

[0091] Here, assuming sufficient system resources and all reference features in the feature pool in the execution state have been loaded, the amount of pre-loaded reference features can be increased by increasing the amount of reference features in the feature pool in the waiting-to-load state. After the feature pool transitions from the waiting-to-load state to the execution state, identification and comparison can be performed directly based on the loaded reference features, thereby improving the efficiency and stability of feature comparison and maximizing device performance. During implementation, increasing or decreasing the amount of reference features in the feature pool in the waiting-to-load state can synchronously adjust the amount of reference features in the feature pool in the execution state.

[0092] Step S172: Dynamically adjust the amount of reference feature data loaded simultaneously in the feature pool that is in a waiting-to-load state based on the loading status of the feature pool.

[0093] Here, by monitoring the loading status of reference features in the feature pool that is in a waiting-to-load state, the amount of data of reference features loaded simultaneously in the feature pool can be dynamically adjusted, such as increasing or decreasing the number of reference features loaded simultaneously, thereby increasing the possibility of providing functional services based on the feature pool that is in an execution state. In implementation, the size of the data buffered simultaneously in the feature pool that is in an execution state and the feature pool that is in a waiting-to-load state can be increased or decreased.

[0094] In this embodiment, by dynamically adjusting the loading status and data volume of the feature pool, system resources can be balanced while ensuring the availability of the feature comparison function. Thus, compared to a static loading strategy, this solution can adapt to resource changes under different load scenarios, flexibly adjust the usage status of the feature pool, and / or increase or decrease the data volume of reference features, thereby achieving intelligent resource management during the feature comparison process.

[0095] In some embodiments, the control method for the above-described electronic device may further include at least one of the following steps S181 to S183: Step S181: If the target application is determined to be in an idle state by the original features captured by the auxiliary application, reduce the capture frequency of the original features and / or control the feature pool in the waiting loading state to enter the unloading state.

[0096] Here, the idle state indicates that the target application is not currently being actively used by the user, or is in a state where no critical task is in progress. When the target application is in the idle state, it indicates that the application has lower real-time requirements, which can reduce the application's resource consumption, thereby saving system resources and improving overall device performance. In some implementations, the idle state can be determined comprehensively based on user operation data (such as mouse movement trajectory, keyboard input, and changes in interface focus) and / or resource consumption.

[0097] The raw features may include feature data collected by the auxiliary application during the operation of the target application, such as user interface (UI) element information, image frame content, text content, window state, etc.

[0098] Fetch frequency represents the number of times raw features are collected per unit of time. When the target application is idle, reducing the fetch frequency can decrease the load on system resources such as CPU, memory, and GPU, thereby reducing stuttering or latency issues caused by resource contention. For example, the target application can be considered to be in an idle state when the user temporarily leaves the game or closes related functions (such as strategy services). At this time, the fetch frequency of raw features is automatically reduced, and unused feature pools that are waiting to be loaded are unloaded from the last to the first according to the order of addition, thereby freeing up resources.

[0099] Step S182: When the target application switches from an idle state to an active state, increase the capture frequency of the original features and / or load the feature pool that is in a waiting loading state.

[0100] Here, the active state represents the target application being used by the user again or in a state where a critical task is being initiated. In implementation, the switch from the idle state to the active state of the target application can be triggered by user interaction, such as clicking a button, opening a specific interface, or launching a specific function. Increasing the crawling frequency allows for timely acquisition of the latest target application state information, enabling a prompt and rapid response. Loading the feature pool that is in a pending loading state and preloading reference features eliminates the need for real-time feature data loading when the user uses the corresponding function, thereby shortening response time and improving user experience.

[0101] Step S183: Based on the feature recognition progress control of the second target process, switch the usage state between the feature pool in the waiting loading state and the feature pool in the execution state.

[0102] Here, the second target process can be a subprocess that works in collaboration with other processes. For example, in the application scenario of a game-based intelligent assistant, the second target process can be an independent process that works in conjunction with highlight detection or replay analysis. The feature recognition progress of the second target process will directly affect the efficiency and completion of highlight detection or replay analysis. Therefore, it is necessary to adjust the usage status of the feature pool in a timely manner according to the feature recognition progress of the second target process.

[0103] State switching refers to switching the state of a feature pool between a waiting-to-load state and an execution state. In implementation, when the reference features of a feature pool in the execution state have been identified, the state of that feature pool can be switched to a waiting-to-load state, and then switched back to the execution state for feature comparison of the new feature pool. In some implementations, when a feature pool is about to be used for feature comparison, its state can be set to a waiting-to-load state, thereby prioritizing resource allocation and completing the pre-loading of that feature pool.

[0104] In this embodiment, on the one hand, by dynamically adjusting the capture frequency of the original features and / or controlling the state of the feature pool waiting to be loaded, the resource investment for feature comparison can be flexibly adjusted according to the state and changes of the target application, thereby improving the accuracy and efficiency of resource use. On the other hand, by controlling the switching of the feature pool in the waiting-to-load state and the feature pool in the execution state according to the feature recognition progress of the second target process, the system resource utilization can be further optimized, and the response speed, smoothness and stability of the comparison reference features can be improved.

[0105] In AI-enabled gaming applications, AI models can assist game assistants. Leveraging the capabilities of models like OCR, YOLO, and CV, numerous game assistant services can be developed, such as highlight reels, strategy guides, and replay services. However, with the increasing number of services and models used, managing the lifecycle and resources of game assistant-related programs presents significant challenges for electronic devices. First, the lack of resource awareness during process startup and / or shutdown can lead to startup failures or stuttering (frame drops) under high load. Second, the absence of clear standards for process reuse can result in different games repeatedly creating frame capture processes, increasing memory usage. Third, the lack of activity-related predictions for process execution means that if a user re-activates the application shortly after it has been closed, the corresponding functional process must be reloaded, causing delays in functionality.

[0106] To address the above issues, based on the control methods of the aforementioned electronic devices, this application provides a resource load-aware scheduling method, applied to the usage scenario of game applications, which can be executed by the electronic device running the game application, such as... Figure 3 As shown, the method includes the following steps S301 to S320: Step S301: Start the AI ​​model.

[0107] Here, the AI ​​model can correspond to the auxiliary application in the control method of the aforementioned electronic device, namely a game assistant. In implementation, the AI ​​model can be activated after the electronic device is started; or it can be activated in response to the launch of the game application.

[0108] After executing step S301, execute steps S302 and S314.

[0109] Step S302: Monitor the game process startup.

[0110] Here, the game process can correspond to the application process in the control method of the aforementioned electronic device.

[0111] Step S303: Determine if the game hits.

[0112] Here, upon monitoring the launch of a game process, it is determined whether the game process is a game type suitable for the AI ​​model; if it is suitable, it's a match; otherwise, it's a miss. Understandably, for game processes that are not compatible, the AI ​​model cannot provide related functional services. In implementation, this can be done by determining whether the game application is compatible with the functional services in the AI ​​model upon its first installation or launch on an electronic device.

[0113] Step S304: Determine if there is a game process in a running state.

[0114] Here, if a game hit is confirmed, it is determined whether a higher-priority game process exists. The running state can correspond to the executing state in the control method of the aforementioned electronic device. If there is no game process in the running state, step S305 is executed; if there is a game process in the running state, step S310 is executed.

[0115] Step S305: Add the running state to the state machine.

[0116] In implementation, a state machine can be used to manage the state of the monitored game processes. For example, if the state machine is empty, the game process that starts at this time is automatically added to the state machine and set to the running state. When other game processes start subsequently, since there is already a running process in the state machine, the other game processes that start later will be added to the pending state. The pending state can correspond to the waiting execution state in the control method of the aforementioned electronic device.

[0117] For multiple processes in a pending state, the priority of the application processes can be determined based on their startup order when determining the monitoring order. For example, such as... Figure 4 As shown, according to the startup order, the game process 41, which starts first, is in the running state and has the highest priority; the game process 42, which starts after game process 41, is in the pending state and has a lower priority than game process 41; the game process 43, which starts after game process 42, is in the pending state and has a lower priority than game process 42.

[0118] Step S306: Real-time monitoring of resource load.

[0119] During implementation, the current system resource usage can be collected in real time.

[0120] Step S307: Determine whether the resource requirements are met.

[0121] Here, after adding a new game process to the state machine, it is necessary to determine whether the current resources are sufficient to support the operation of the business processes required by the game process. The business processes required by the game process correspond to the first target process in the control method of the aforementioned electronic device; meeting the resource requirements corresponds to the system resource status meeting the second control condition in the control method of the aforementioned electronic device.

[0122] For example, the business processes of a game application may include, but are not limited to, a game frame collection process (the frame capture process in the above embodiments), a highlight service process, a strategy guide service process, a replay service process, an inference management process, an OCR inference process, a CV inference process, and a YOLO inference process. The required process combinations differ for different game services: the highlight service requires a game frame collection process, a highlight service process, an inference management process, and an OCR inference process; the strategy guide service requires a game frame collection process, an inference management process, a strategy guide service process, and a CV inference process; and the replay service requires a game frame collection process, an inference management process, a replay service process, and a YOLO inference process. It is evident that the game frame collection process and the inference management process are common processes for all three game services; when providing multiple services, only one invocation of the game frame collection process and the inference management process is required.

[0123] The resource requirements for different processes can be the same or different. For example, the memory requirements for the game frame collection process, highlight service process, strategy service process, reasoning management process, OCR reasoning process and CV reasoning process can be 160 MB, 28 MB, 50 MB, 4 MB, 290 MB and 26 MB respectively.

[0124] In some implementations, resource requirements can be determined to be met when CPU utilization is less than a preset threshold (e.g., 70% or 80%), free memory is greater than a preset threshold (e.g., 500M), and NPU computing power idle rate is greater than a preset threshold (e.g., 50%).

[0125] Step S308: Start the business process and add it to the running state machine.

[0126] Here, if the resource requirements are met, the corresponding business process of the game application is started, and the corresponding business process is added to the running state machine of the game process. After executing step S308, step S309 is executed.

[0127] Step S309: Run the cycle.

[0128] Step S310: Add a pending state to the state machine.

[0129] Step S311: Real-time monitoring of resource load.

[0130] Step S312: Determine whether the resource requirements are met.

[0131] Step S313: Start the business process and add it to the pending state machine.

[0132] Here, if the resource requirements are met, the corresponding business process of the game application is started, and the corresponding business process is added to the pending state machine of the game process. After executing step S313, step S309 is executed.

[0133] Step S314: Control the operation of the resource load monitoring system.

[0134] During implementation, a resource load monitoring system can be used to monitor and collect the resource load information of the equipment.

[0135] Step S315: Real-time monitoring of resource load.

[0136] Step S316: Determine whether the system resources are saturated.

[0137] Here, system resource saturation can correspond to the system resource status satisfying the first control condition in the control method of the above-mentioned electronic device.

[0138] In some implementations, system resource saturation can be determined when CPU utilization exceeds a preset threshold (e.g., 90%), memory utilization exceeds a preset threshold (e.g., 90%), and NPU computing power utilization exceeds a preset threshold (e.g., 80%).

[0139] Step S317: Determine whether there is a pending process in the state machine.

[0140] Here, by determining whether there are any pending business processes in the state set, the corresponding processes can be terminated if they are, thereby releasing resources. In implementation, the order in which pending business processes are terminated can be determined based on their priority. For example, the game frame collection process and the inference management process can be assigned the highest priority (first priority); the OCR inference process, the CV inference process, and the YOLO inference process can be assigned the second priority (lower than the first priority but higher than the third priority); and the highlight process, the strategy process, and the replay process can be assigned the lowest priority (third priority).

[0141] Step S318: Determine whether a process with a third priority exists.

[0142] If it exists, proceed to step S320; if it does not exist, proceed to step S319.

[0143] Step S319: Determine whether a process with a second priority exists.

[0144] If it exists, proceed to step S320; if it does not exist, proceed to step S309.

[0145] Step S320: Terminate the corresponding process.

[0146] In this way, by reusing shared business processes, memory usage can be reduced (for example, by 200MB); by adjusting resource load, the failure rate of process startup under high load can be reduced; and by combining with a state machine, when users switch game applications or perform operations on business functions (such as opening or closing operations), rapid identification of business processes and lifecycle switching of business processes can be achieved, thereby improving the user experience of business functions.

[0147] In some implementations, such as Figure 5 As shown, the above resource load-aware scheduling method may further include the following steps S321 to S329: Step S321: Monitor whether the game has ended.

[0148] Here, we can refer to step S153 above, which involves monitoring whether the first application process has entered the termination state.

[0149] Step S322: Process activity prediction.

[0150] After the game process ends, the activity level of the business process (i.e., the activity level of the first target process) is predicted. In implementation, a sliding window can be used to store multiple consecutive user interactions with the game process, where user actions can correspond to starting and closing the game process. For example, as shown... Figure 6 As shown, a sliding window can record the interval between the last time a user closed the game application and the current time the game application was launched, which is used as the time feature corresponding to the launch time. Based on a preset correspondence between interval duration and feature value, a corresponding feature value is assigned to the event feature corresponding to each launch time. Specifically, the feature value is 1 for intervals within 1 minute; 2 for intervals within 5 minutes; 3 for intervals within 10 minutes; 4 for intervals within 30 minutes; and 5 for intervals exceeding 1 hour.

[0151] Furthermore, utilizing the time decay mechanism, the 20 time features are grouped into sets of 5. For example, time features t0 to t4 form the first group, t5 to t9 the second group, t10 to t14 the third group, and t15 to t19 the fourth group. The time features in the first group all correspond to a decay weight of 0.1, the time features in the second group all correspond to a decay weight of 0.2, the time features in the third group all correspond to a decay weight of 0.3, and the time features in the fourth group all correspond to a decay weight of 0.4. Using decay weights can improve the reference value of recently collected user behavior while considering user behavior. After determining the decay weight corresponding to each time feature, the weighted probability corresponding to each interval duration is calculated. For example, the process of calculating the weighted probability corresponding to each interval duration can be found in formula (1): (1); The denominator is the sum of the decay weights corresponding to the 20 time features, and the numerator is the weighted sum of each time feature and its corresponding decay weight. For the first The decay weights corresponding to each time feature exist It is 1 if it is true, otherwise it is 0.

[0152] Combination Figure 6According to formula (1), the correspondence between the preset interval duration, the feature value, and the weighted probability of the user restarting the game process under that interval duration can be obtained: when the interval duration is within 1 minute, the corresponding feature value is 1 and the weighted probability value is 1.9; when the interval duration is within 5 minutes, the corresponding feature value is 2 and the weighted probability value is 1.6; when the interval duration is within 10 minutes, the corresponding feature value is 3 and the weighted probability value is 0.2; when the interval duration is within 30 minutes, the corresponding feature value is 4 and the weighted probability value is 0.2; when the interval duration is more than 1 hour, the corresponding feature value is 5 and the weighted probability value is 1.1. Here, the maximum value among the weighted probability values ​​corresponding to each interval duration can correspond to the activity level of the first target process in the control method of the above electronic device.

[0153] Step S323: Determine the process duration.

[0154] Based on the maximum weighted probability value determined in step S322, i.e., the activity level corresponding to the business process, the retention duration of the business processes related to the game process is calculated based on the interval duration corresponding to this activity level. For example, the retention duration of the business processes is calculated... The process can be found in formula (2): (2); in, The feature value representing the interval duration corresponding to the maximum activity level. This is a preset base duration determined based on this interval. When the activity level is 1.9 for intervals within 1 minute, =1, If you set it to 1 minute, the resulting business process duration will be 4 minutes.

[0155] Step S324: Determine whether to terminate the process.

[0156] Here, if the duration of a business process exceeds a preset retention threshold (such as 10 minutes), it can be considered that the business process is no longer necessary to retain, and the business process can be terminated to release resources. If the game process is restarted subsequently, the decision on whether to restart the business process will be made based on the real-time system resource load.

[0157] Step S325: Enable load balancing scheduling.

[0158] Here, if the duration of a business process does not exceed a preset retention threshold, it can be considered that the business process can continue to be maintained, in order to reduce the resources and time of repeated startups and balance the load to schedule the allocation of resources among processes.

[0159] Step S326: Add the process to the holding queue.

[0160] Here, the business processes that need to be kept are added to the business process keep queue according to their process priority.

[0161] Step S327: Determine whether the queue is empty.

[0162] Here, if system resources are saturated, determine whether there are any business processes in the maintenance queue. If so, proceed to step S328.

[0163] Step S328: Remove processes from low to high priority.

[0164] Here, when system resources are saturated, business processes in the retention queue are removed in order of priority from low to high until system resources are no longer saturated.

[0165] Step S329, End.

[0166] In this way, by predicting activity levels, the switching latency of functional services can be optimized. Furthermore, by detecting the start and / or end status of game processes, combined with the support for game services, it is easier to maintain the list of subsequent business processes. Through mechanisms such as process lifecycle management, resource load awareness scheduling, and activity prediction based on user behavior, dynamic collaboration and resource optimization of multiple processes and multiple services are achieved.

[0167] Understandably, by sensing resource status (load), user behavior (activity), and task attributes (priority), this approach enables "on-demand resource allocation and on-demand process start / stop." This method can overcome the limitations of game scenarios and be reused in scenarios involving multi-process collaboration, limited or sensitive resources, and / or a need to balance efficiency and stability, providing a general resource governance framework. For example, in multimodal AI inference scenarios: enabling concurrent operation of multiple AI models on edge devices (such as smart cameras and drones) (e.g., face recognition, behavior analysis, anomaly detection); and mobile AI applications (e.g., scene recognition, filter processing, and real-time translation in photo apps). Edge devices and mobile devices have limited computing power. When multiple AI models (e.g., computer vision, natural language processing, and speech) are run concurrently, resource imbalances may lead to increased inference latency or model loading failures; repeatedly loading basic models (e.g., multiple services calling the feature extraction module) will consume additional memory. For example, in IoT and edge computing node scenarios: industrial IoT edge nodes (such as factory sensor data acquisition, equipment status monitoring, and AI fault diagnosis); smart home gateways (coordinating data analysis and linkage services from cameras, door locks, and temperature and humidity sensors). Edge nodes (such as industrial gateways) have limited computing power and storage resources, and need to handle multiple tasks simultaneously, including data acquisition, local analysis, and cloud synchronization. Improper resource allocation may lead to data loss or monitoring delays; when multiple devices are linked, repeatedly creating communication processes may also consume bandwidth and memory.

[0168] Furthermore, depending on the functional process, there are situations where feature matching using a CV feature library is required to provide a complete service. How to control the allocation of system resources to better serve users is also a concern. Taking a game strategy service as an example, providing a strategy service requires real-time feature matching between captured game frame data and the CV feature library. Since the image feature matching required for event trigger points (i.e., key points that trigger events in the game) in game applications is large (e.g., one event trigger point may correspond to 100 images), as the number of key points increases, the CV feature library also increases, leading to slower loading speeds, increased system memory consumption, and reduced CV matching speed.

[0169] Regarding this issue, such as Figure 7 As shown, the above resource load-aware scheduling method may further include the following steps S401 to S407: Step S401: Game frame capture.

[0170] During execution, game frames of the game process can be captured through the frame capture process in the above embodiments.

[0171] Step S402: Perform feature recognition.

[0172] Here, this corresponds to the identification of the original features of the target application and the reference features in the feature pool in the control method of the aforementioned electronic device. For example, taking a target game application as an example... Figure 8 As shown, the target game application includes 76 levels, each level corresponding to two key points (i.e., event trigger points), and each key point corresponds to 100 image frames. In related technologies, the original CV feature library is a continuous, unified feature library. The reference features in the feature pool are taken from the original feature library.

[0173] Step S403: Detect the hit progress of the current feature group.

[0174] Here, the hit progress refers to the feature matching progress. In this method, the levels in the original feature library are grouped to obtain multiple feature groups. For example, as shown... Figure 9 As shown, the feature library corresponding to the 76 levels of the target game application can be divided into 38 feature groups, with each feature group consisting of two image frames from each level. Each feature group corresponds to its own identifier.

[0175] During implementation, the feature comparison is based on a preset feature pool, and the image frames of each feature group are loaded into the feature pool in sequence.

[0176] In some implementations, when system resources are not saturated, a feature pool in the execution state and a feature pool in the waiting-to-load state can be set. The feature pool in the execution state includes image frames of the feature group currently undergoing feature comparison, while the feature pool in the waiting-to-load state can be empty or include feature groups in the original feature library that follow the feature group currently undergoing feature comparison. Figure 10 As shown, during the loading and comparison of reference features in the feature pool containing 5 feature groups and in the execution state, reference features in the feature pool containing 5 feature groups and in the waiting-to-load state are preloaded. The last buffered feature group in the feature pool in the execution state is the same as the first buffered feature group in the feature pool in the waiting-to-load state. It can be understood that by judging the feature recognition progress of the feature pool currently in the execution state, the system can switch to the feature pool in the waiting-to-load state with buffered reference features in a timely manner when the comparison of reference features in the feature pool is about to be completed, thereby improving the smoothness, response speed, and stability of feature comparison.

[0177] Step S404: Has the target progress of the current feature group been achieved?

[0178] Here, the target progress is a preset feature recognition progress threshold, indicating that the reference features of the feature group are about to be compared. For example, the target progress can be 75%, 80%, 85%, etc.

[0179] Step S405: Compare the identifiers of feature groups in the feature pool of the execution state with those in the feature pool of the waiting-to-load state.

[0180] Here, the system determines whether to switch the feature pool usage state by comparing the identifier of the last buffered feature group in the feature pool that is in the execution state with the identifier of the first buffered feature group in the feature pool that is in the waiting-to-load state.

[0181] Step S406: Switch the usage status of the feature pool.

[0182] Here, if it is determined that the current feature group has reached the target progress, and the next feature group is the last buffered feature group, the current feature pointer is switched to the feature pool in the waiting-to-load state; the original waiting-to-load feature pool is switched to the execution state; the original execution feature pool is switched to the waiting-to-load state, and feature group buffering begins from the last buffered feature group in the currently execution feature pool. For example, as... Figure 11 As shown, when the next feature group in the feature pool in the execution state is about to be compared with the feature group marked 9-10, the state of the feature pool in the execution state is switched to the waiting loading state, and buffering begins from the feature group marked 17-18; the state of the feature pool in the original waiting loading state is switched to the execution state, and comparison begins from the feature group marked 9-10.

[0183] Step S407: Determine whether the final feature group has been reached.

[0184] Here, after switching states, it is determined whether the feature groups in the feature pool include the last feature group in the original feature library. If they do, the comparison of the reference features corresponding to the feature groups in the feature pool currently in the execution state is completed and the process ends. If they do, proceed to step S408; if they do not, return to step S401.

[0185] Step S408, End.

[0186] like Figure 12 As shown, in some embodiments, the resource load-aware scheduling method described above may further include the following steps S501 to S515: Step S501: Game frame capture.

[0187] Step S502: Determine whether the information of the current frame is consistent with that of the historical frames.

[0188] Here, multi-dimensional and multi-modal perception methods such as CV comparison, camera information extraction, speech recognition, and OCR can be used to confirm whether the game frame captured at the current moment is consistent with the game frame captured at the previous moment (indicating whether the user had no further action at the previous moment). If they are consistent, proceed to step S503; if they are inconsistent, proceed to step S509. For example, it can be determined whether there is input for the current game application by recognizing voice, text, or system operations. Alternatively, it can be determined whether the user is focused on the game by capturing the user's current state through the camera.

[0189] Step S503: Accumulated management of identical frames.

[0190] In practice, identical frames can be accumulated, and the accumulated result can be compared with a preset threshold to improve the accuracy of determining whether the current frame is consistent with the information of historical frames.

[0191] Step S504: Confirm whether the number of identical frames is greater than the number threshold.

[0192] Here, the quantity threshold is a preset minimum number of identical frames that indicate the user is not currently focused on the game. For example, it can be 10, 20, 60, etc. If the number of identical frames is greater than the quantity threshold, step S505 is executed; if the number of identical frames is not greater than the quantity threshold, step S501 is executed. In some embodiments, it can be determined that the current user is not focused on the game if the number of consecutive identical frames reaches a first quantity threshold; it can also be determined that the current user is not focused on the game if not all consecutive identical frames reach a second quantity threshold. The second quantity threshold is not less than the first quantity threshold. It is understood that for a specific game scenario, there may be instances of identical game frames.

[0193] Step S505: Determine the execution of the idle state scheme.

[0194] Here, if the number of identical frames exceeds a threshold, it can be assumed that the user is not focused on the game application. Therefore, the game application is in an idle state, and system resources can be released by reducing the resource consumption of related processes within the game application. For example, reducing the screen refresh rate, entering power-saving mode, or lowering the volume. Steps S506 and S507 can also be executed.

[0195] Step S506: Reduce the frequency of game frame capture.

[0196] For example, the frame rate of the game can be reduced from 3 frames per second to 1 frame per second.

[0197] Step S507: Unload the feature pool that is in a waiting-to-load state.

[0198] Step S508: Enter the idle state.

[0199] Step S509: Determine whether the current game application is in an idle state.

[0200] Here, when the information in the current frame is inconsistent with that in the historical frames, but the current game application is in an idle state, it indicates that the user may refocus on the game or the game application may start performing tasks. Therefore, it is necessary to ensure that the current game application has available system resources. If it is in an idle state, proceed to step S510; if it is not in an idle state, proceed to step S515, and perform feature recognition and comparison based on the feature pool.

[0201] Step S510: Determine the execution activation scheme.

[0202] The active state scheme may include, but is not limited to, increasing the screen refresh rate, entering high-efficiency mode, and increasing the volume. Steps S511 and S513 may also be executed.

[0203] Step S511: Clear identical frames.

[0204] Here, the number of identical frames that have already been accumulated is set to 0, and identical frames are re-accumulated.

[0205] Step S512: Increase the frequency of game frame capture.

[0206] For example, the frame rate of the game can be increased from 1 frame per second to 3 frames per second.

[0207] Step S513: Load the feature pool that is in a waiting loading state.

[0208] Step S514: Enter the loop.

[0209] Here, the game application enters the loop in an active state.

[0210] Step S515: Workflow based on feature pool.

[0211] Here, the specific implementation of step S515 can be found in steps S401 to S407 above.

[0212] In this way, the activity level of a scene can be determined by the same frame, thereby dynamically adjusting device power consumption and device resources.

[0213] For resource scheduling during feature pool loading, such as Figure 13 As shown, the above resource load-aware scheduling method may further include the following steps S601 to S613: Step S601: Start the strategy service.

[0214] Here, the first target process corresponding to the strategy service includes the second target process in the control method of the aforementioned electronic device.

[0215] Step S602: The feature pool loading logic is started.

[0216] First, determine the amount of reference features to be loaded simultaneously into the feature pool (such as the number of feature groups in step S403 above). In implementation, taking the target game application in step S402 above as an example, the feature library corresponding to the 76 levels of the target game application is loaded into the feature pool in groups of two image frames from each level. Reference features for each feature group For example, Figure 10 middle The value is 5, which is the amount of reference feature data loaded simultaneously in the feature pool.

[0217] Step S603: Load the running feature pool.

[0218] Here, the running feature pool corresponds to the feature pool in the execution state in the control method of the aforementioned electronic device. The number of feature groups to be loaded is specified. Reference features in the original feature library are loaded in groups according to their order.

[0219] Step S604: Determine whether the running feature pool has been successfully loaded.

[0220] If unsuccessful, proceed to step S605; if successful, proceed to step S608.

[0221] Step S605: Adjust the number of feature groups loaded in the running feature pool.

[0222] Here, with For example, after a loading failure, the number of... The value is maintained until the running feature pool is successfully loaded.

[0223] Step S606: Determine whether the number of feature groups in the running feature pool is less than the preset number of groups.

[0224] Here, it is determined whether the number of feature groups loaded when the running feature pool is successfully loaded is less than a preset number of groups. The preset number of groups represents the minimum effective number of groups for feature loading based on the feature pool; for example, the preset number of groups can be 4. It is understood that in... If the value is less than 4 (e.g., 3), it indicates that the system resources cannot support the pending feature pool (corresponding to the feature pool in the control method of the aforementioned electronic device that is in a waiting-to-load state). Loading group feature group data makes it impossible to switch between the running feature pool and the pending feature pool. Therefore, it is possible to... If the value is less than 4, the running feature pool is considered to have failed to load, as the current system resources do not support the switching mechanism between the running and pending feature pools. If the running feature pool loads successfully... If the number of groups is less than the preset number, proceed to step S607; if the running feature pool is successfully loaded... If the number of groups is not less than the preset number, proceed to step S603.

[0225] Step S607: Loading failed.

[0226] Step S608: Load the pending feature pool.

[0227] Here, based on loading the running feature pool Determine the number of feature groups loaded in the pending feature pool.

[0228] Step S609: Determine whether the pending feature pool has been successfully loaded.

[0229] If unsuccessful, proceed to step S610; if successful, proceed to step S612.

[0230] Step S610: Adjust the number of feature groups loaded in the pending feature pool.

[0231] Here, with For example, after a loading failure, the number of... The value is maintained until the pending feature pool is successfully loaded.

[0232] Step S611: Determine whether the number of feature groups in the pending feature pool is less than the preset number of groups.

[0233] Here, it is determined whether the number of feature groups loaded when the pending feature pool is successfully loaded is less than a preset number. The preset number of groups used to determine whether the pending feature pool has been successfully loaded differs from the preset number used to determine the running feature pool. For example, the preset number of groups used to determine the pending feature pool can be 2. It is understood that in... If the value is less than 2 (e.g., 1), it means that the switching between the running and pending feature pools is actually the sequential loading of single feature groups, which is the same as the conventional approach. Therefore, it is possible to... If the value is less than 2, the pending feature pool loading is considered to have failed, as the current system resources do not support the switching mechanism between the running and pending feature pools. If the pending feature pool loading is successful... If the number of groups is less than the corresponding preset number, proceed to step S607; if the pending feature pool is successfully loaded... If the number of groups is not less than the corresponding preset number, proceed to step S608.

[0234] Step S612: Determine whether the number of feature groups loaded in the pending feature pool has changed.

[0235] Here, if the pending feature pool is successfully loaded, then determine... If a change has occurred, proceed to step S613; otherwise, both the running feature pool and the pending feature pool have been successfully loaded.

[0236] Step S613: Update the loading logic based on the number of feature groups loaded in the latest feature pool.

[0237] Here, we use the latest version. Update the size of the running feature pool so that the loading capacity of the running feature pool is the same as that of the pending feature pool.

[0238] In this way, on the one hand, it can improve feature loading efficiency, reduce memory consumption, and decrease inference latency while balancing system resources; on the other hand, it facilitates the modular updating logic of feature library files; furthermore, it can reduce the demand for electronic devices to run strategy services, thereby expanding the application scenarios and usage scope of the devices, such as enabling low-memory devices. It is understandable that, based on the above feature pool loading strategy, when system resources are sufficient, the running feature pool and the pending feature pool can be expanded accordingly. This improves the efficiency of feature matching and overall task execution.

[0239] This application provides an electronic device, including a memory and a processor. The electronic device can refer to a server, laptop computer, tablet computer, desktop computer, smart TV, set-top box, mobile device (such as a mobile phone, portable video player, personal digital assistant, dedicated messaging device, portable gaming device), or any other device with data processing capabilities.

[0240] like Figure 14As shown, the electronic device 1400 includes at least one processor 1410 and an intelligent agent 1411 capable of running on the processor. The intelligent agent 1411 is capable of executing or invoking at least one processing model to perform the following operations: monitoring the system resource status of the electronic device; determining the attribute information of a first target process when a first control condition is met based on the system resource status; controlling the running state of the first target process based on the attribute information to reclaim the system resources of the electronic device or provide corresponding target function services to the target application of the electronic device; wherein, the first target process is at least one running process of an auxiliary application of the electronic device, and the auxiliary application is an application used to provide corresponding function services to the target application.

[0241] In some embodiments, the aforementioned intelligent agent may also execute or invoke at least one processing model to execute any step in the control method of the aforementioned electronic device.

[0242] This application also proposes a computer program including computer-readable code. When the computer-readable code is run in an electronic device, the processor in the electronic device executes a control method for implementing the electronic device provided in this application.

[0243] This application provides a computer program product, including a computer program or computer-executable instructions. When the computer program or computer-executable instructions are executed by a processor, they implement the control method of the electronic device provided in this application.

[0244] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the control method of the electronic device provided in this application.

[0245] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A control method of an electronic device, comprising: monitoring a system resource status of the electronic device; in a case where it is determined that a first regulation condition is met based on the system resource status, determining attribute information of a first target process; controlling a running state of the first target process based on the attribute information, so as to recycle a system resource of the electronic device or provide a corresponding target function service to a target application of the electronic device; wherein the first target process is at least one to-be-run process of an auxiliary application of the electronic device, and the auxiliary application is an application for providing a corresponding function service to the target application.

2. The method of claim 1, wherein the monitoring of the system resource status of the electronic device comprises: in response to the electronic device starting to run a first application, performing the step of monitoring the system resource status of the electronic device, the first target process comprising at least one to-be-run process of a first auxiliary application for providing a first type of function service to the first application; or in a process in which the electronic device runs a second application, performing the step of monitoring the system resource status of the electronic device, the first target process comprising at least one to-be-run process of a second auxiliary application for providing a second type of function service to the second application; wherein in a case where the first application and the second application belong to the same type of application, the first auxiliary application and the second auxiliary application are the same, and the first target process is the same or different; in a case where the first application and the second application belong to different types of application, the first auxiliary application and the second auxiliary application are different, and the first target process is different.

3. The method of claim 1, wherein the determining of the attribute information of the target process in the case where it is determined that the first regulation condition is met based on the system resource status comprises: in a case where a usage parameter value of at least one target hardware of the electronic device represented by the system resource status is greater than a corresponding preset parameter value, it is determined that the first regulation condition is met; in a case where it is determined that there is a to-be-run process in a waiting state, priority information configured for the first target process is determined, the priority information being used to configure a priority order in which the first target process is loaded or removed; in a case where it is determined that there is no to-be-run process in a waiting state, the step of monitoring the system resource of the electronic device is returned.

4. The method of claim 1, wherein the controlling of the running state of the first target process based on the attribute information, so as to recycle the system resource of the electronic device or provide the corresponding target function service to the target application of the electronic device comprises at least one of the following: in a case where the attribute information represents that the first target process is configured as a first priority, the first target process is controlled to be in a loaded running state or a waiting loaded state, so as to provide the corresponding target function service to the target application of the electronic device; in a case where the attribute information represents that the first target process is configured as a second priority, the first target process is controlled to enter an end state, so as to recycle a system resource occupied by the first target process. In a case where the attribute information indicates that the first target process is configured as a third priority, the first target process is controlled to enter an end state to reclaim system resources occupied by the first target process.

5. The method of claim 1 or 4, wherein, Controlling a running state of the first target process based on the attribute information includes: predicting an activity level of the first target process based on historical behavior data of a target user and / or requirement information of a target application, the activity level indicating a possibility that the first target process is used to provide a corresponding functional service to the target application; controlling the first target process to enter any one of an end state, a loaded running state, or a waiting loaded state based on the attribute information and the activity level.

6. The method of claim 1, in a case where a first application process is monitored to be started, the method further includes at least one of: in a case where it is determined that there is a second application process in an executing state, adding the first application process as a waiting execution state, and in a case where a system resource condition of the electronic device meets a second regulation condition, adding the first application process to a waiting execution queue; in a case where it is determined that there is no second application process in an executing state, adding the first application process as an executing state, and in a case where a system resource condition of the electronic device meets a second regulation condition, loading and running the first application process; or, after monitoring that the first application process enters an end state, predicting an activity level of the first target process based on historical behavior data of a target user, and controlling the first target process to enter a waiting execution queue or an end state based on the activity level.

7. The method of claim 1, further including: controlling a loading strategy of a feature pool called by a second target process based on the system resource condition, wherein the second target process is a feature identification process of an auxiliary application of the electronic device, the feature pool includes reference features used to identify and compare original features of a target application grabbed by the auxiliary application, and the loading strategy includes at least a usage state switching of the feature pool and / or a data amount adjustment of the reference features in the feature pool.

8. The method of claim 7, wherein the controlling the loading strategy of the feature pool called by the second target process based on the system resource condition includes at least one of: controlling a state of the feature pool in a waiting loaded state and / or an executing state or a data amount of the reference features in the feature pool based on the system resource condition; dynamically adjusting the data amount of the reference features simultaneously loaded in the feature pool in the waiting loaded state based on a loading situation of the feature pool.

9. The method of claim 7, further including at least one of: in a case where the target application is determined to be in an idle state through the original features grabbed by the auxiliary application, reducing a grabbing frequency of the original features and / or controlling the feature pool in the waiting loaded state to enter an unloaded state; in a case where the target application switches from the idle state to an active state, increasing the grabbing frequency of the original features and / or loading the feature pool in the waiting loaded state; or, Based on the feature recognition progress of the second target process, the feature pool in the loading state and the feature pool in the execution state are switched in use state.

10. An electronic device comprising at least one processor and an agent capable of running on the processor, the agent being capable of executing or invoking at least one processing model to perform the following operations: monitoring a system resource condition of the electronic device; determining attribute information of a first target process if it is determined that a first regulation condition is met based on the system resource condition; controlling a running state of the first target process based on the attribute information to recycle system resources of the electronic device or provide a corresponding target function service to a target application of the electronic device; wherein the first target process is at least one to-be-run process of an auxiliary application of the electronic device, and the auxiliary application is an application for providing a corresponding function service to the target application.

Images