A resource reuse method, system and computer readable storage medium

Abstract

With the development of technologies such as artificial intelligence, edge computing, and the Internet of Things, modern computing systems often need to run multiple computationally intensive tasks simultaneously. These tasks have different requirements for computing resources. Allocating dedicated resources to each task independently would lead to a surge in hardware costs and low resource utilization. Furthermore, in real-time systems such as robotics, autonomous driving, and industrial control, competition for computing resources among different tasks (such as perception, decision-making, and control) is also prominent. Existing real-time operating systems mostly use fixed-priority scheduling, which cannot dynamically adjust the computing resource configuration of tasks, resulting in delays or resource waste in critical task responses. This application provides a resource reuse method, system, and computer-readable storage medium, aiming to achieve efficient multi-task concurrent processing by dynamically scheduling computing units and actively releasing idle resources.

Description

Technical Field

[0001] This application relates to the field of computer resource scheduling and multitasking technology, and more specifically, to a resource reuse method, system, and computer-readable storage medium. Background Technology

[0002] With the widespread application of artificial intelligence, edge computing, and the Internet of Things (IoT) technologies, modern computing systems face the challenge of multi-task concurrent processing. Various computationally intensive tasks, such as machine learning model inference, real-time data analysis, and control system operation, have significantly different requirements for computing resources. In traditional architectures, fixed resources are typically allocated to each task, such as reserving GPU memory or CPU computing power separately for models of different sizes. This leads to long-term idle hardware resources and low overall system utilization. Taking multi-model inference scenarios as an example, if large language models and small vision models are deployed as independent instances, GPU memory and computing power cannot be shared, resulting in significant waste. Although existing inference frameworks introduce model hibernation mechanisms to release idle GPU memory, they lack the ability to dynamically evaluate task priority and complexity, and cannot flexibly schedule resources according to request characteristics. In real-time systems such as robotics, autonomous driving, and industrial control, resource competition between tasks is particularly prominent. Existing scheduling strategies rely on fixed priorities or time-slice rotation, which is difficult to adapt to changes in task complexity, leading to delays in critical task response or long-term resource occupation by low-priority tasks. Furthermore, when system load fluctuates, traditional methods fail to promptly transition inactive computing units to a low-power state; for example, they fail to migrate state data from high-speed storage to low-speed storage, resulting in unnecessary energy consumption and increased hardware costs. These issues collectively constrain resource utilization efficiency and system real-time performance in multi-tasking environments, necessitating a universal solution that comprehensively considers request priority and task complexity, dynamically manages computing unit states, and optimizes resource allocation.

[0003] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0004] The purpose of this application is to provide a resource reuse method, system, and computer-readable storage medium, which has the advantages of improving computing resource utilization, reducing idle waste, and optimizing task scheduling efficiency.

[0005] This application provides a resource reuse method, the technical solution of which is as follows: Includes the following steps: Deploy multiple computing units with different resource requirements; Based on the priority and complexity of the request, dynamically select the target computing unit to process the request. Place inactive computing units in a low-resource-occupancy state; Requests are processed in order of priority.

[0006] Furthermore, this application also proposes that the low resource occupancy state includes offloading the state data of the computing unit from high-speed storage to low-speed storage and saving the state data for subsequent recovery.

[0007] Furthermore, this application also proposes that high-speed storage includes high-speed storage media for computing units, and low-speed storage includes storage media with larger capacity but relatively slower access speed.

[0008] Furthermore, this application proposes that the priority of a request be determined based on at least one of the following: request source type, user settings, or automatic system settings.

[0009] Furthermore, this application proposes that the task complexity be determined based on at least one of the following: input data volume, expected output data volume, required computational depth, or historical processing time.

[0010] Furthermore, this application also proposes processing requests in order of priority, including suspending or delaying the processing of low-priority requests when resources are scarce.

[0011] Furthermore, this application also proposes to include dynamically loading larger computing units to replace the current main computing unit based on user instructions or system requirements.

[0012] Furthermore, this application also proposes that the method be applied to intelligent agents, robots, industrial equipment, Internet of Things systems, smart home systems, in-vehicle systems, cloud systems, or any multitasking system.

[0013] Furthermore, this application also proposes a resource reuse system, comprising: A resource pool is used to store multiple computing units with different resource requirements; The routing module is used to select the target computing unit based on the priority of the request and the complexity of the task. The resource management module is used to place inactive computing units into a low-resource-occupancy state. The scheduling module is used to process requests in order of priority.

[0014] Furthermore, this application proposes that the resource management module be configured to offload the state data of the computing unit from high-speed storage to low-speed storage and save the context for rapid recovery.

[0015] Furthermore, this application also proposes that the scheduling module be configured to suspend or delay the processing of low-priority requests when resources are scarce.

[0016] Furthermore, this application also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0017] As can be seen from the above, the resource reuse method, system and computer-readable storage medium provided in this application effectively solve the problems of resource waste and low scheduling efficiency in a multi-tasking environment by deploying multiple computing units, dynamically selecting target units, managing the status of inactive units and processing requests according to priority. It has the advantages of improving computing resource utilization, reducing idle waste and optimizing task scheduling efficiency.

[0018] Comparative analysis with existing technologies: Regarding the granularity of resource management: existing technologies typically allocate fixed resources to a single task (such as one GPU per model); while the solution in this application can achieve on-demand allocation, dynamically selecting computing units based on task priority and complexity.

[0019] Regarding resource release strategies: Existing technologies typically release resources after the task is completed, or simply hibernate (such as vLLM); while the solution in this application can achieve proactive energy saving: placing inactive computing units in a low resource occupancy state and preserving the context for rapid recovery.

[0020] Regarding task scheduling criteria: existing technologies are based solely on time slices or fixed priorities; while the solution proposed in this application can achieve dual scheduling based on priority and complexity, comprehensively considering the urgency of the task and computational requirements.

[0021] In terms of resource utilization efficiency: existing technologies have low utilization rates and waste idle resources; while the solution proposed in this application has high utilization rates and idle resources are released for use by other tasks.

[0022] In terms of application scenarios: existing technologies are limited to specific fields; while the solution of this application can be applied to any multi-tasking system such as intelligent agents, robots, vehicles, industries, cloud computing, and the Internet of Things. As can be seen from the above comparison, this application extends resource reuse from a single model scenario to the domain of general computing tasks. Through dynamic scheduling based on "priority + complexity" and an "active energy-saving" mechanism, it significantly improves resource utilization and has remarkable creativity. Attached Figure Description

[0023] Several embodiments of this application are described below with reference to the accompanying drawings. It should be noted that the specific structures, modules, steps, parameters, and connections shown in the drawings are preferred embodiments of this application and not limitations on the scope of protection of this application. Those skilled in the art can make various modifications, substitutions, or combinations to the specific details shown in the drawings based on the teachings of this application, and these modified embodiments should still be considered to fall within the scope of protection of this application.

[0024] Figure 1This application provides an overall architecture diagram of a resource reuse system, which is an exemplary architecture of the resource reuse system. Each module can be adjusted according to actual applications and does not limit the scope of protection.

[0025] Figure 2 This application provides a resource scheduling flowchart, which illustrates the request processing flow and reflects the priority, complexity assessment, and dynamic loading mechanism. The diagram is only an example and does not constitute a limitation on the claims.

[0026] Figure 3 This diagram illustrates a resource-saving state switching process provided in this application. The diagram shows the switching process of a computing unit between an active and energy-saving state. This diagram is merely an example and does not constitute a limitation on the claims. Detailed Implementation

[0027] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application. Other technologies that may be mentioned in the embodiments can be implemented using existing technology or other patent applications filed by the applicant on the same day, and will not be repeated here. It should be particularly noted that the specific module divisions, process steps, data flow directions, status names, time values, etc., shown in the accompanying drawings are merely illustrative examples and should not constitute a limitation on the scope of protection of the claims of this application. The scope of protection of the claims is determined solely by their wording and should be interpreted in accordance with the overall content of the specification.

[0028] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] In traditional computing systems, when multiple computationally intensive tasks run simultaneously, the lack of dynamic resource allocation mechanisms leads to low resource utilization. Specifically, the system cannot dynamically adjust resource allocation based on the actual needs of the tasks, resulting in both idle and overloaded hardware resources. This rigid scheduling strategy causes high-priority tasks to be delayed due to resource contention, while resources occupied by low-priority tasks cannot be released in a timely manner for other tasks. Furthermore, the lack of a comprehensive assessment of task priority and complexity means that scheduling decisions rely on only a single factor, further exacerbating resource conflicts.

[0030] For example, when deploying a large language model, a small dialogue model, and an image recognition model in an intelligent agent system, after a user initiates a text dialogue request, the system needs to process subsequent image recognition requests. At this time, GPU memory is already occupied by the small model, while the large model is in a dormant state. Traditional solutions cannot quickly assess whether to activate the large model, resulting in either delays due to waiting for resource release or additional overhead caused by forcibly unloading the current model. Furthermore, under resource constraints, the system cannot dynamically select the most suitable computing unit based on the urgency of the request and computational needs, hindering the response of critical tasks while inactive models continuously occupy high-speed storage resources.

[0031] If the above problems are not addressed, the system will remain in a suboptimal operating state for an extended period. The coexistence of resource idleness and contention will intensify, increasing processing latency for critical tasks and decreasing overall system throughput. In scenarios with stringent real-time requirements, such as autonomous driving or industrial control, this latency can lead to safety risks. Furthermore, continuous resource waste will increase hardware deployment costs and reduce the system's economic viability. Therefore, deficiencies in the resource management mechanism will directly impact the reliability and scalability of multi-tasking systems.

[0032] In response, this application proposes a resource reuse method, comprising the following steps: Deploy multiple computing units with different resource requirements; Based on the priority and complexity of the request, dynamically select the target computing unit to process the request. Place inactive computing units in a low-resource-occupancy state; Requests are processed in order of priority.

[0033] For ease of understanding, the following explains some key terms in this embodiment: Computing unit: refers to a software or hardware entity capable of performing a specific computing task. These computing units can include, but are not limited to, machine learning model instances, data processing flows, control algorithm modules, or any reusable computing resources. Each computing unit typically has different resource requirements; for example, the requirements for a central processing unit (CPU), graphics processing unit (GPU), memory, or storage space may vary.

[0034] Resource requirements: refers to the amount of computing resources required for a computing unit to operate. For example, a large language model may require a large amount of GPU memory and computing power, while a small image processing model may only require less CPU resources.

[0035] Request priority: This refers to the importance or urgency of a computing task request within the system. High-priority requests typically need to be processed first to ensure system responsiveness and timely execution of critical functions. Priority can be determined based on various factors, such as the type of request origin (e.g., user interaction, internal system task, emergency event), user-preset levels, or the system's automatic assessment based on the current state.

[0036] Task complexity refers to the amount of computation or resources required to process a computational task. For example, tasks that process large amounts of input data are generally more complex than those that process small amounts of input data. Task complexity can be assessed based on factors such as the amount of input data, the expected amount of output data, the required computational depth, or historical processing time.

[0037] Low resource utilization state: This refers to an energy-saving or resource-releasing mode in which a computing unit operates during inactivity. In this state, some or all of the computing resources occupied by the computing unit are released to be used by other active tasks, thereby improving overall resource utilization. When the computing unit needs to become active again, it can recover from this state.

[0038] Traditional computing systems often face problems such as low resource utilization, rigid task scheduling, and untimely resource release when running multiple tasks simultaneously. To address this, this application proposes a resource reuse method that aims to achieve efficient multi-task concurrent processing by dynamically scheduling computing units and proactively releasing idle resources.

[0039] Specifically, this method includes the following steps: First, deploy multiple computing units with varying resource requirements. These units can be pre-configured and stored in a resource pool for system calls. For example, a computing unit for image recognition, a computing unit for speech processing, and a computing unit for data analysis can be deployed. Upon initial deployment, these units can be configured to utilize all their required resources or load only their core components. In one implementation, these computing units can be statically allocated across different physical or logical processors, such as deploying the image recognition unit on a GPU and the speech processing unit on a CPU. However, this static allocation method may result in some processors being idle for specific periods, while others are under resource strain.

[0040] Secondly, the target computing unit for processing the request is dynamically selected based on the request's priority and task complexity. When the system receives a new computing request, it needs to be evaluated. For example, its priority can be determined based on the request's source (such as a real-time interaction request from user A or a background data synchronization request from system B), and its task complexity can be evaluated based on the amount of data contained in the request or the expected computational steps. In one implementation, the system can maintain a simple mapping table to directly route specific types of requests to preset computing units. For example, all image processing requests are sent to the image recognition computing unit. However, this simple routing method may not fully utilize other computing units in the system that may be idle or more suitable for processing the current request.

[0041] Furthermore, inactive computing units are placed in a low-resource-consumption state. A computing unit is considered inactive when it has not received any requests for a period of time, or when its processed tasks have been completed. To conserve resources, the system can switch it to a low-resource-consumption state. For example, the computing unit can be switched from a running state to a paused state, so that it no longer consumes computing cycles. In one implementation, an inactive computing unit can simply be stopped, and all its occupied memory and computing resources are completely released. However, this complete stopping may cause the computing unit to need to be reloaded and initialized the next time it is activated, thus introducing additional latency.

[0042] Finally, requests are processed according to priority. The system maintains a queue of pending requests and sorts them according to their priority. High-priority requests are retrieved from the queue first and assigned to the corresponding computing units for processing. For example, when multiple requests arrive simultaneously, the system processes the highest-priority request first. In one implementation, the system can use a first-in, first-out (FIFO) strategy to process requests with the same priority. However, this approach may lead to low-priority requests remaining unprocessed for extended periods, or even being blocked by high-priority requests, when resources are scarce.

[0043] The following example will provide a more detailed explanation of the above technical solution: Suppose an edge computing device needs to simultaneously support three functions: smart home control, security monitoring, and voice assistant. To achieve this, three computing units with different resource requirements are deployed: a control unit for smart home control (low resource requirements), a video analytics unit for security monitoring (high resource requirements, requiring a GPU), and a voice recognition unit for the voice assistant (medium resource requirements). These computing units are deployed within the device's resource pool.

[0044] For example, user A requests "turn on the living room lights" via a voice assistant. This request is assessed as having "high" priority (real-time user interaction) and "low" task complexity (simple command). Based on the assessment results, the routing module dynamically selects a smart home control unit to handle the request. Because this control unit has low resource requirements and is likely to be active, the request is processed quickly.

[0045] Subsequently, the device receives a video stream from a security camera, requiring abnormal behavior detection. This request is assessed as having a "medium" priority (backend monitoring) and a "high" task complexity (continuous video analysis). The routing module selects the video analysis unit for processing. Meanwhile, if the voice recognition unit does not receive voice commands for a period of time, it will be placed in a low-resource-occupancy state by the resource management module; for example, only its core program will remain in memory, while releasing some of its computational cycles.

[0046] When the device simultaneously receives a voice command from user A (high priority) and a background data synchronization request (low priority), the scheduling module will process the requests in order of priority. User A's voice command will be processed first to ensure that the voice assistant can respond promptly. The background data synchronization request will be scheduled for execution only if resources allow, or after all high-priority requests have been processed.

[0047] As can be seen from the above examples, this method deploys diverse computing units and dynamically schedules target units based on request priority and task complexity. At the same time, it places inactive units in a low-resource state and prioritizes high-priority requests, thereby efficiently reusing resources, improving overall utilization, and ensuring timely response to critical tasks.

[0048] Based on the examples above, the technical concept of this method demonstrates significant technical contributions. Traditional solutions typically employ a "one model, one instance" deployment approach or schedule tasks based solely on fixed priorities, resulting in low resource utilization and slow response times in multi-task concurrent scenarios. For instance, in the smart home scenario described above, a traditional solution might require reserving fixed and independent computing resources for smart home control, security monitoring, and voice assistants. Even if a function is not used for an extended period, its resources cannot be reused by other functions, leading to substantial waste.

[0049] In contrast, this method achieves on-demand resource allocation and fine-grained scheduling by "deploying multiple computing units with different resource requirements" and "dynamically selecting the target computing unit based on the priority of the request and the complexity of the task." This allows the system to flexibly allocate computing resources to the most needed tasks according to the actual task load and importance, significantly improving resource utilization. For example, when the speech recognition unit is inactive, the resources it releases can be utilized by the video analysis unit, thereby supporting more functions on limited hardware resources.

[0050] Furthermore, this method achieves proactive resource release and energy saving by "placing inactive computing units in a low-resource-occupancy state." Traditional solutions may only release resources after a task is completed, or simply put the model into a dormant state without releasing core resources. This method, however, allows the system to proactively reduce the resource consumption of computing units when they are idle, such as releasing some memory or computing cycles, thereby reducing power consumption and operating costs. When reactivation is needed, it can be quickly restored, avoiding the time-consuming process of a complete reload.

[0051] Finally, this method ensures real-time response for critical tasks by processing requests in priority order. When multiple tasks compete for resources, this method prioritizes the execution of high-priority tasks, preventing low-priority tasks from blocking critical operations. This is particularly important in systems with high real-time requirements, such as smart homes, robots, or autonomous driving, effectively improving system stability and reliability. In summary, this method, by comprehensively utilizing dynamic scheduling, proactive energy saving, and priority processing mechanisms, effectively solves the problems of low resource utilization, rigid scheduling, and untimely resource release in existing technologies, demonstrating significant progress.

[0052] In some of the embodiments described above in this application, it is proposed to place inactive computing units in a low resource occupancy state to release resources. However, in the implementation process, if there is no specific mechanism to save the state data, the state data may be lost or the recovery process may be slow, thereby affecting the system response time and resource utilization efficiency.

[0053] In this regard, this application further proposes that the aforementioned low resource occupancy state includes offloading the state data of the computing unit from high-speed storage to low-speed storage and saving the state data for subsequent recovery.

[0054] Specifically, the low resource occupancy state refers to a mode in which the computing unit reduces its use of high-value computing resources when not performing tasks, thereby reducing power consumption and resource consumption. This state aims to maximize resource utilization while ensuring the computing unit can recover quickly. For example, in addition to the data offloading method described in this application, the low resource occupancy state can also be achieved by reducing the operating frequency of the computing unit, entering a sleep mode, or suspending some non-core functions.

[0055] The state data of the computing unit refers to information generated by the computing unit during task execution that is crucial to its current operating state. This data may include, but is not limited to, model weights, intermediate calculation results, context information, program counter, register contents, memory page tables, and any necessary configuration parameters for resuming its execution. This state data can be stored in the form of a binary stream, a structured data file, or a block of memory.

[0056] High-speed storage refers to storage media with extremely high data read and write speeds. It typically has a relatively small capacity but is more expensive, and is mainly used to store frequently accessed data to ensure the efficient operation of computing units. For example, high-speed storage can refer to the processor's internal cache (such as L1, L2, and L3 caches) or the video memory of a graphics processing unit (GPU). Alternatively, it can also be high-bandwidth memory (HBM) or non-volatile RAM (NVRAM) on the system motherboard.

[0057] Offloading state data from high-speed storage to low-speed storage refers to moving the state data of a computing unit currently residing in high-speed storage to a storage medium with relatively slower access speed but larger capacity through a data transfer mechanism. The purpose of this operation is to free up valuable high-speed storage resources for use by other active computing units or high-priority tasks, while reducing the power consumption of inactive computing units. This offloading process can be implemented using methods such as memory copying, file writing, or through efficient data transfer via a Direct Memory Access (DMA) controller.

[0058] Low-speed storage refers to storage media with relatively slow data read / write speeds but large capacity and low cost, typically used to store data that is not frequently accessed or needs to be preserved for a long time. For example, low-speed storage can refer to the system's main memory (RAM), or solid-state drives (SSDs) and hard disk drives (HDDs). Additionally, it can also be remote network storage (such as Network Attached Storage (NAS), Storage Area Network (SAN)) or cloud storage services.

[0059] Saving state data for subsequent recovery refers to ensuring that this data is properly stored and its storage location or identifier is recorded after it has been offloaded to low-speed storage. This allows for quick and accurate retrieval and loading of the data when the computing unit needs to be reactivated. This aims to avoid the need to initialize or recalculate the state from scratch when the computing unit is reactivated, thereby significantly shortening recovery time and achieving seamless switching of the computing unit.

[0060] This application's solution addresses the issue of critical state data loss or slow recovery during resource release by offloading the state data of inactive computing units from high-speed storage to low-speed storage and properly preserving this data for subsequent recovery. When a computing unit enters an inactive state, its critical state data is safely migrated to lower-cost, higher-capacity low-speed storage, freeing up valuable high-speed storage resources for other active tasks and optimizing overall resource allocation. Simultaneously, because the state data is fully preserved, when the computing unit needs to be reactivated, the system can directly load this data from low-speed storage, avoiding time-consuming re-initialization or data reconstruction processes and ensuring rapid recovery capabilities. This combined mechanism cleverly balances energy saving and system responsiveness under resource constraints, enabling the system to efficiently utilize resources while maintaining rapid adaptability to task changes, thereby improving overall system efficiency.

[0061] The following is a concrete example. In an intelligent agent system, multiple machine learning models are deployed as computing units, such as a large language model and a small dialogue model. When a user interacts with the small dialogue model, the large language model may be inactive. At this time, to optimize resource utilization, the system triggers a low-resource-consumption state switch. Specifically, the large language model's state data, such as model weights, intermediate activation values, and its runtime context, is unloaded from GPU memory (high-speed storage) and written to CPU memory (low-speed storage). The system generates a unique identifier for this unloaded data and records its storage location in a fast-access lookup table. When the large language model receives a request again and needs to be activated, the system quickly reads and loads its state data from CPU memory to GPU memory based on this identifier, allowing it to continue execution from the previously interrupted state without reloading the entire model or performing time-consuming initialization.

[0062] Through the above technical solution, this application effectively solves the problem of data loss or slow recovery caused by improper state data management when inactive computing units are placed in a low-resource-occupancy state. This solution ensures that while releasing high-speed storage resources, computing units can maintain their rapid recovery capabilities, thereby avoiding delays caused by re-initialization or data reconstruction. This enables the system to manage and schedule computing resources more flexibly and efficiently in a multi-tasking environment, significantly improving resource utilization and system response speed. Especially in scenarios requiring frequent switching of computing unit activity states, it enables seamless task switching and ensures timely response to high-priority tasks.

[0063] In some of the solutions described above in this application, the state data of inactive computing units is offloaded from high-speed storage to low-speed storage to reduce resource consumption. However, in this process, the specific types of high-speed storage and low-speed storage are not clearly defined, which may lead to inefficient offloading and recovery operations, affecting the compatibility of state migration and system response speed.

[0064] In this regard, this application further proposes a high-speed storage medium including a computing unit for high-speed storage, and a low-speed storage medium including a storage medium with a larger capacity but a relatively slower access speed.

[0065] High-speed storage refers to storage media with extremely high read and write speeds. It is mainly used to store real-time data and frequently accessed information from active computing units to ensure rapid response and efficient execution of computing tasks. For example, GPU memory is a high-speed memory dedicated to graphics processing units (GPUs). It is used to store large model weights, intermediate calculation results, and texture data. Its bandwidth far exceeds that of system memory, making it crucial for parallel computing.

[0066] Low-speed storage refers to storage media with slower read / write speeds compared to high-speed storage, but typically offering larger capacity and lower cost. It is primarily used to store state data of inactive computing units to facilitate resource release and subsequent rapid recovery. For example, CPU memory, used here as low-speed storage, is relative to GPU memory or CPU cache. It provides a relatively large and low-cost storage area to receive data unloaded from higher-speed storage (such as GPU memory).

[0067] This application optimizes the state data migration process in resource management by specifying the specific types of high-speed and low-speed storage. When a computing unit is active, its critical state data is stored in high-speed storage media to ensure extremely low access latency and efficient computing performance. Once the computing unit becomes inactive, its state data is offloaded from these high-speed storage media to low-speed storage media. This offloading operation frees up valuable high-speed resources for use by other active computing units, thereby improving overall resource utilization. Simultaneously, by specifying the types of storage media, the system can select the optimal data transfer path and management strategy based on the characteristics of different media. For example, it can utilize DMA (Direct Memory Access) technology to efficiently transfer data between GPU memory and CPU memory, or employ a specific file system to optimize hard disk read / write operations. This tiered storage strategy not only reduces the resource consumption of inactive computing units but also ensures that the state of the computing unit can be quickly restored from low-speed storage when needed by saving context data, avoiding the overhead of reloading from scratch. This saves resources while maintaining system responsiveness and compatibility.

[0068] The following is a concrete example. In an intelligent agent system, multiple computing units are deployed, including a large language model (LLM) and an image recognition model. When the large language model is active, its massive model weights, intermediate activation values, and key-value cache, among other critical state data, are loaded and reside in GPU memory to support rapid inference computation. In this case, GPU memory acts as high-speed storage, providing the extremely high bandwidth and low latency required for model operation. When the large language model does not receive requests for a period of time, or when a higher-priority task requires GPU resources, the resource management module determines it to be an inactive computing unit. At this point, the state data of the large language model (e.g., model weights and some contextual information) is offloaded from GPU memory to CPU memory. In this scenario, CPU memory acts as low-speed storage; although its access speed is slower than GPU memory, it has a larger capacity and lower cost, effectively preserving the state of the large model. If the system does not need the large language model for an extended period, or if CPU memory resources are also limited, its state data can even be further offloaded to a solid-state drive (a type of hard drive) for persistent storage. When new requests require processing large language models, their state data is reloaded from CPU memory or SSD back to GPU memory, quickly restoring their active state and allowing them to continue providing services. Conversely, for some CPU-intensive small control algorithms, their active state data may primarily reside in the CPU cache; when inactive, this data is flushed to CPU memory. In this way, the system can flexibly migrate state data between storage media of varying speeds and costs based on the activity level and resource requirements of the computing units.

[0069] Through the above technical solutions, this application clarifies the specific types of high-speed and low-speed storage, enabling the system to optimize the unloading and recovery process of state data based on the characteristics of different storage media. This significantly improves the efficiency and compatibility of data migration, ensuring that active computing units can fully utilize the performance advantages of high-speed storage, while the state data of inactive computing units can be economically and efficiently preserved. This effectively solves the problems of low efficiency in unloading and recovery operations, impacting state migration compatibility and system response speed caused by unclear storage types.

[0070] In some of the solutions mentioned above in this application, request priorities are proposed to dynamically select target computing units and the order of request processing. However, in this process, the specific method for determining the priority is not clear, which may lead to inflexible scheduling decisions, making it impossible to adapt to different request sources and user needs, thereby affecting the accuracy of resource allocation and system response efficiency.

[0071] In this regard, this application further proposes that the priority of a request be determined based on at least one of the following: request source type, user settings, or automatic system settings.

[0072] Specifically, request priority is an indicator that measures the importance or urgency of a request in resource contention and processing order. It determines the processing order and resource allocation priority of requests within the system. Request source type refers to the category of the entity or event initiating the request. For example, it could be a user interaction request, an internal system task, an emergency event trigger request, or a background data processing request. Different source types typically correspond to different business importance or real-time requirements. The system can pre-set a set of rules to mark requests from specific user interfaces as "user interaction" and requests from internal monitoring modules as "system tasks"; alternatively, the system can distinguish request types based on the API interface or message queue source identifier. For example, requests from real-time control APIs are identified as "real-time control type," while requests from data analysis APIs are identified as "data analysis type." User settings refer to the priorities manually configured by end users or system administrators for specific requests or request groups based on their business needs or preferences. This approach provides flexibility, allowing users to adjust the importance of tasks according to actual circumstances. When a user submits a request, a priority selection interface can be provided, allowing the user to choose "high," "medium," or "low" priority. Alternatively, system administrators can use a configuration interface to set default priorities for specific user groups or task types, allowing users to adjust them within a certain range. Automatic priority allocation refers to the system autonomously assigning priorities to requests based on preset strategies, algorithms, or real-time operational status. This approach enables dynamic and intelligent priority adjustment to adapt to constantly changing system load and task characteristics. The system can automatically calculate priorities based on factors such as expected processing time, deadline, and predicted resource consumption using scheduling algorithms; alternatively, the system can combine machine learning models to predict the urgency and importance of requests based on historical data and current system status, and dynamically adjust their priorities.

[0073] This application introduces a flexible priority determination mechanism, enabling resource reuse methods to respond more accurately to different request needs. When the system receives a new request, it no longer relies solely on a single or fixed priority setting, but comprehensively considers the request's source type, the user's specific needs, and the system's own operational status. For example, requests from real-time interactions (high-priority source type) or requests explicitly marked as urgent by the user (user-defined) will be assigned higher priority. Simultaneously, the system can also dynamically adjust priorities based on its internal load or task deadlines (automatically). This multi-dimensional, adaptive priority determination method provides a solid foundation for subsequent actions such as "dynamically selecting the target computing unit to process the request based on its priority and task complexity" and "processing requests in priority order." It ensures that, under resource constraints, critical tasks receive priority access to computing unit resources and are processed promptly, while secondary tasks can be queued or delayed reasonably according to their priority, thus avoiding unreasonable resource allocation or delayed response to important tasks due to rigid priority settings. In this way, the entire resource reuse system can perform task scheduling and resource management more intelligently and efficiently.

[0074] The following example illustrates this. Consider an intelligent agent system that needs to handle voice commands from users, background data analysis tasks, and internal health monitoring requests. When a user issues a command to "play music" via a voice assistant, because this is a user-initiated interaction, the request source type is identified as "user interaction," and the system automatically assigns it high priority. If a user submits a complex image processing task and manually selects the "urgent processing" option through the interface, even though the request source type is "background task," the system will still elevate its priority because the user has set it to high. Furthermore, the system periodically runs health monitoring tasks; these tasks have an "internal system" request source type and typically have lower priority. However, if the system detects an abnormally high temperature in a critical component, it can automatically elevate the priority of related health monitoring tasks (such as initiating diagnostic procedures) to the highest level to ensure timely response to potential faults. In this way, the system can flexibly and accurately determine the priority of requests based on their actual circumstances, thereby guiding the selection of subsequent computing units and request processing.

[0075] By introducing a method for determining request priority based on at least one of the following: request source type, user settings, or automatic system settings, this application significantly improves the flexibility and accuracy of priority setting. This multi-dimensional, adaptive priority determination mechanism enables the system to more precisely identify the true importance and urgency of different requests. For example, urgent user interaction requests or critical system tasks can be promptly identified and assigned high priority, ensuring they receive priority access to resources and processing, thereby guaranteeing the system's real-time responsiveness and user experience. Simultaneously, user settings allow the system to meet personalized business needs, while automatic system adjustments ensure continuous optimization of priorities in a dynamically changing system environment, avoiding problems such as unreasonable resource allocation or delays in important tasks due to rigid priorities. This ultimately optimizes the accuracy of resource allocation and improves the overall system response efficiency and resource utilization.

[0076] In some of the embodiments described above in this application, task complexity is proposed for dynamically selecting target computing units. However, in this process, the evaluation of task complexity may lack specific and objective standards, leading to inaccurate scheduling decisions and affecting the efficiency and real-time performance of resource allocation.

[0077] In this regard, this application further proposes that the task complexity is determined based on at least one of the following: input data volume, expected output data volume, required computational depth, or historical processing time.

[0078] The input data volume refers to the total amount of data that needs to be received and processed during the task processing. Its concept lies in quantifying the task's dependence on input data; a larger data volume usually means a higher computational load. For example, for image processing tasks, the input data volume can be represented by the number of pixels in the image or the file size; for text processing tasks, it can be represented by the number of characters, words, or bytes in the text. By accurately assessing the input data volume, the system can initially determine the scale of the task and its potential computational requirements. The expected output data volume refers to the total amount of data expected to be generated after the task is completed. Its concept lies in measuring the scale of the task's output; a larger output data volume may mean that the task requires more intermediate storage during processing or more resources in the result generation stage. For example, the expected output data volume of a data analysis task could be the number of lines in the generated report or the file size; the expected output data volume of a code generation task could be the number of lines of generated code or the number of characters. The required computational depth refers to the computational complexity or iterative level of the algorithm involved in the task execution. Its concept lies in reflecting the inherent computational intensity of the task; a deeper computational depth usually means more computation cycles and a longer processing time. For example, for deep learning model inference tasks, the required computational depth can refer to the number of layers in the model, the number of parameters, or the number of floating-point operations (FLOPs); for complex data processing workflows, it can refer to the number of steps in data transformation or aggregation operations. Historical processing time refers to the actual time the system has spent processing similar tasks in the past. The concept lies in using past experience to predict the execution time of future tasks; it is an evaluation method based on statistics and learning. For example, the system can record and analyze the average time, maximum time, or specific percentile time spent processing different types of requests; or build a model to predict the processing time based on certain characteristics of the task (such as the amount of input data and the type of request). This approach can effectively adapt to changes in system load and the dynamic evolution of task characteristics.

[0079] This solution addresses the lack of objectivity in task complexity assessment in traditional methods by providing specific and quantifiable evaluation criteria. In the resource reuse approach, when the system receives a request, it dynamically selects a target computing unit based on the request's priority and task complexity. Accurate assessment of task complexity is crucial in this process. By considering the amount of input data, the system can predict the initial processing load of the task; by considering the expected amount of output data, the system can estimate the final output scale and related resource requirements; by considering the required computational depth, the system can understand the computational intensity of the task and the inherent complexity of the algorithm; and by considering historical processing time, the system can use past actual running data to empirically predict the task's execution time. One of these metrics can be used in combination or individually to determine task complexity, making the assessment of task computational requirements more accurate and objective. This accurate complexity assessment allows the routing module to more accurately match task requirements with the capabilities of the computing units when dynamically selecting target computing units, avoiding improper resource allocation due to inaccurate complexity assessments, such as assigning complex tasks to undercapacitated computing units or simple tasks to overly powerful computing units. Therefore, this solution ensures the rationality and efficiency of resource allocation, thereby improving the overall resource utilization rate and guaranteeing the real-time response capability of high-priority tasks.

[0080] The following example illustrates this. In an intelligent agent system, when a user initiates a text generation request, the system needs to assess the task complexity of that request. As a specific implementation, the system can first assess the amount of input data, such as the number of tokens for the user's prompt. If the prompt is long, the input data volume is large. Simultaneously, the system can estimate the expected output data volume based on the type of user request (e.g., generating a short response or a long report), such as estimating the number of tokens for the generated text. Furthermore, the system can consider the required computational depth; for example, if the text generation task requires calling a large language model (LLM) for inference, the number of layers and parameters of the model can serve as indicators of computational depth. Finally, the system can query historical processing time data, for example, predicting the current task's time based on the average processing time for text generation tasks with similar input and output lengths in the past. By comprehensively considering at least one of these factors, such as using the number of input tokens and the expected number of output tokens as primary indicators, the system can objectively determine the task complexity of the text generation request. For example, a request with few input tokens and a small expected number of output tokens will be classified as a low-complexity task, while a request with many input tokens and a large expected number of output tokens will be classified as a high-complexity task. This determination method provides a solid foundation for the subsequent routing module to dynamically select appropriate computing units (e.g., selecting a small dialogue model to handle low-complexity tasks, or selecting a large language model to handle high-complexity tasks).

[0081] Through the above technical solution, this application provides a more accurate and objective task complexity assessment mechanism. Since task complexity is a key basis for dynamically selecting target computing units, by determining task complexity based on factors such as input data volume, expected output data volume, required computational depth, or historical processing time, the system can avoid inaccurate scheduling decisions caused by subjective or coarse assessments. This allows the routing module to more effectively match the actual computational needs of the task with the resource capabilities of the computing unit when selecting target computing units based on request priority and task complexity, thereby significantly improving the accuracy and efficiency of resource allocation. Ultimately, this not only optimizes the utilization of computing resources and reduces resource waste, but also ensures that high-priority tasks can obtain the necessary resources in a timely manner, effectively guaranteeing the system's real-time response capability and solving the problems of inaccurate task scheduling and low resource utilization.

[0082] In some of the solutions described above in this application, requests are processed in order of priority to ensure that high-priority tasks are processed first. However, in this process, when resources are scarce, low-priority requests may consume computing resources, causing high-priority requests to fail to respond in a timely manner.

[0083] In response, this application further proposes processing requests in order of priority, including pausing or delaying the processing of low-priority requests when resources are scarce.

[0084] "Resource strain" refers to a situation where the available computing resources of the system (such as CPU, GPU, memory, network bandwidth, etc.) are lower than the total resources required for all pending requests, or lower than a preset threshold. There are several ways to determine resource strain. For example, it can be determined by real-time monitoring of system resource utilization; when any critical resource (such as GPU memory usage or CPU utilization) exceeds a preset threshold (e.g., 80% or 90%), it is considered resource strain. Alternatively, a predictive model can be used to predict, based on the length of the current pending request queue, the total task complexity, and historical resource consumption data, that resources will be insufficient to meet all requests in the near future, thus determining resource strain.

[0085] The purpose of "pausing or delaying the processing of low-priority requests" is to ensure that high-priority requests can obtain the necessary resources in a timely manner, avoiding response delays or processing failures caused by low-priority requests consuming resources. Specifically, for low-priority requests that are currently executing, their current execution can be interrupted, their state data (e.g., intermediate calculation results, context information, etc.) can be saved to memory or disk, and the resources they occupy can be released. Execution can be resumed from the saved state once resource conditions improve. For low-priority requests that have not yet started execution, their execution time in the scheduling queue can be postponed until system resources are sufficient or high-priority requests are completed. Alternatively, a hybrid strategy can be adopted, that is, pausing low-priority requests that have already started execution and delaying low-priority requests that have not yet started execution.

[0086] This application's solution effectively addresses the problem of low-priority tasks hindering high-priority tasks when resources are insufficient in priority scheduling by proactively pausing or delaying the processing of low-priority requests during resource-scarce periods. In the aforementioned resource reuse method, the system first deploys multiple computing units with varying resource requirements and dynamically selects the target computing unit for processing based on the request's priority and task complexity. When the system detects resource scarcity, the scheduling module no longer simply queues requests according to priority but proactively intervenes, pausing or delaying low-priority requests. This mechanism ensures that resources that might otherwise be occupied by low-priority tasks are released or remain available, thus prioritizing allocation to high-priority requests. In this way, the solution enhances the efficiency of dynamic resource scheduling, improving the system's real-time performance and responsiveness by identifying resource scarcity and adjusting the processing order accordingly, especially in scenarios requiring urgent responses to critical tasks.

[0087] The following is a concrete example to illustrate this. In an intelligent agent system, multiple models run simultaneously, such as a large language model, a small dialogue model, and an image recognition model, each with different requirements for GPU memory and computing power. Assume the small dialogue model is currently running actively, while the large language model is in a low-resource-occupancy state (weights have been offloaded to CPU memory). At this time, the system receives a high-priority urgent user command request that requires activating the large language model or a computationally intensive processing unit. Simultaneously, a low-priority background data synchronization task is running or in a queue. When the system detects that GPU memory or CPU utilization has reached a preset stress threshold, the scheduling module immediately suspends the running background data synchronization task and saves its current state. If the background task has not yet started, its startup will be delayed. By releasing the resources occupied by the low-priority task, the system can quickly allocate the necessary computing resources to the high-priority urgent user command request, ensuring that the command is responded to and processed promptly. Once the high-priority request is processed and the resource situation eases, the suspended background data synchronization task can resume execution from its saved state.

[0088] The above technical solution ensures timely responses to high-priority tasks even under resource-constrained conditions, effectively preventing critical tasks from being blocked or delayed by low-priority tasks. This significantly improves the system's real-time performance and reliability, making it particularly suitable for applications with stringent requirements for response speed and task priority, such as emergency decision-making in autonomous driving, industrial control, or intelligent agent systems.

[0089] In some of the solutions mentioned above in this application, the target computing unit is dynamically selected based on request priority and task complexity to optimize resource allocation and task processing. However, in this process, when task requirements increase or system requirements change, the existing computing unit may be unable to meet the processing requirements due to resource limitations, resulting in processing delays, resource bottlenecks or untimely responses, which affect the overall system efficiency and real-time performance.

[0090] In response, this application further proposes a resource reuse method, which also includes dynamically loading a larger computing unit to replace the current main computing unit according to user instructions or system requirements.

[0091] "User instructions" refer to explicit commands issued by end users through human-computer interaction interfaces (such as voice commands, text input, and graphical interface operations) that request the system to adjust computing units. For example, a user might explicitly select "Enable advanced mode" or "Switch to high-precision model." "System requirements" refer to the system's automatic assessment of its operational status, environmental perception, task queue changes, or preset strategies, leading to demands for adjustments to computing resources. For instance, when the system detects that the amount of input data far exceeds expectations, task processing time increases significantly, or the external environment undergoes major changes (such as moving from the suburbs to the city in autonomous driving), it will trigger a demand for larger computing units. These two triggering mechanisms ensure that adjustments to computing units are based on actual needs rather than blindly implemented, thus avoiding unnecessary resource consumption.

[0092] "Dynamic loading" refers to loading computing units from storage media (such as hard drives and network storage) into active computing resources (such as GPU memory and CPU memory) in real time as needed during system runtime, making them executable. This loading process can be completed without interrupting the overall system operation. Dynamic loading methods can include: reading the binary file and weight data of the computing unit from local storage (such as solid-state drives) into memory or GPU memory; or downloading the computing unit from a remote server or cloud storage via a network. "Larger computing unit" can refer to another computing unit that is superior to the currently used computing unit in terms of processing power, resource consumption (such as GPU memory, number of CPU cores, and memory size), or model complexity (such as the number of parameters and layers). For example, it could be a machine learning model with a larger number of parameters, a more complex algorithm implementation, or a computing instance that requires more hardware resources.

[0093] "Replacement" refers to switching the currently processing or active computing unit to a newly loaded, larger computing unit, which then takes over subsequent task processing. This typically involves saving the state data of the original computing unit (e.g., offloading it to low-speed storage), releasing the high-speed resources it occupies, and then loading and activating the new computing unit. The "current primary computing unit" refers to the computing unit selected by the system at a given moment to handle a specific type of request or undertake the main computing task. It may be the initial computing unit dynamically selected based on request priority and task complexity, or it may be a computing unit that has been previously replaced and is currently running. The replacement process can employ various strategies, such as: smooth switching, where the new unit immediately takes over after the old unit completes its current processing; or switching at specific checkpoints to ensure data consistency.

[0094] This application's solution, by introducing a dynamic loading and replacement mechanism, combines it with the dynamic selection and low-resource-occupancy state management in the aforementioned resource reuse methods, forming a more flexible and powerful resource scheduling system. When the system receives a request, it first dynamically selects an initial target computing unit for processing based on the request's priority and task complexity. However, during processing, if the actual needs of the task change—for example, the amount of input data far exceeds expectations, or the user explicitly requests higher-precision processing—the system will trigger an evaluation process based on these "user instructions or system requirements." During this evaluation process, the system determines whether the currently running "main computing unit" can efficiently meet the new requirements. If the determination is that the current unit is insufficient, the system will "dynamically load a larger computing unit." This loading process is real-time and aims to bring more powerful computing resources into an active state. Once the larger computing unit is loaded, the system will perform the operation of "replacing the current main computing unit," placing the original, insufficiently capable computing unit in a low-resource-occupancy state (e.g., offloading it to low-speed storage), and the newly loaded, larger computing unit will take over subsequent task processing. This mechanism ensures that the system can not only optimize scheduling based on initial requests, but also seamlessly upgrade computing power during task execution according to dynamically changing needs, thereby avoiding performance bottlenecks and response delays caused by resource constraints. In this way, the system can always process tasks using the computing units best suited to the current task requirements, significantly improving resource utilization efficiency and the system's adaptability to dynamic environments.

[0095] The following is a concrete example. In an intelligent agent system, a small dialogue model might be deployed initially to handle short, routine interaction requests. When a user initiates a complex dialogue request requiring in-depth contextual understanding or multi-turn reasoning, the system recognizes this as a change in "system requirements," meaning the current small dialogue model may not be able to provide a high-quality response. At this point, the system dynamically loads a larger language model with more parameters and stronger reasoning capabilities into GPU memory. Once the large language model is loaded, it replaces the current main computing unit and takes over the user's complex dialogue request. The original small dialogue model can then be placed in a low-resource-consumption state, for example, by offloading its weights to CPU memory to free up GPU memory for the large model. When a short interaction request occurs again, the system can select the small dialogue model again based on priority and complexity, and quickly restore it from CPU memory to GPU memory. As another specific implementation, in an autonomous driving system, when the vehicle is traveling on a highway, the system might primarily use a relatively lightweight obstacle detection model, running at a lower frame rate to conserve computing resources. However, when a vehicle enters a complex urban road environment, the system triggers a dynamic adjustment process based on "system requirements" (i.e., a significant increase in environmental complexity). At this point, the system dynamically loads a more accurate, computationally intensive obstacle detection model and runs it at a higher frame rate. This high-precision model replaces the current main computing unit, taking over the obstacle detection task to ensure driving safety in complex traffic conditions. The original lightweight model can then be temporarily unloaded or placed in a low-power state.

[0096] Through the above technical solution, this application effectively addresses the problem that existing computing units may be unable to meet processing demands due to resource limitations when task requirements increase or system requirements change. The system can flexibly and dynamically load larger computing units and replace the current main computing unit based on user instructions or system requirements, thereby increasing processing capacity in real time without interrupting service and avoiding processing delays and resource bottlenecks. This allows the system to better adapt to dynamically changing workloads and environments, ensuring timely response and efficient processing of critical tasks, and significantly improving the system's robustness and overall performance.

[0097] In some of the solutions mentioned above in this application, a resource reuse method is proposed to achieve efficient multi-task processing through dynamic scheduling and resource release. However, in the implementation of this method, its application scenarios may be misunderstood as only applicable to a single or specific domain, and its generality cannot be clearly defined, resulting in uncertainty when implementing it in diverse multi-task systems, which limits the widespread promotion and actual deployment of the method.

[0098] In this regard, the present application further proposes that the application scenarios of the above method include, but are not limited to, any multi-task system such as an agent, a robot, a vehicle-mounted system, industrial equipment, the cloud, and an Internet of Things system.

[0099] Among them, the "application scenario of the method" specifies the specific environment and field to which the resource reuse method applies. This can be flexibly defined according to the system type, such as an embedded system or a distributed system, or according to business requirements, such as the requirements for real-time performance and data throughput, as well as the deployment environment, such as the edge side or the cloud. The expression "including, but not limited to" emphasizes that the listed scenarios are only examples and not exhaustive, thus highlighting the openness and scalability of the present method. This can be achieved by clearly stating its non-restrictiveness in the technical documentation or by designing a modular system architecture to support the access of future new scenarios. "Agents, robots, vehicle-mounted systems, industrial equipment, the cloud, and Internet of Things systems" are typical multi-task systems, which have relatively high requirements for resource management and scheduling. For example, an agent system may need to simultaneously process tasks such as speech recognition, natural language understanding, and decision-making generation; a robot system may need to simultaneously perform tasks such as perception, navigation, motion control, and human-computer interaction; a vehicle-mounted system, especially an autonomous driving system, needs to simultaneously process tasks such as environmental perception, path planning, vehicle control, and infotainment; industrial equipment, such as an intelligent manufacturing production line controller, needs to simultaneously perform tasks such as process monitoring, fault diagnosis, motion control, and data acquisition; a cloud system, such as a microservices architecture on a cloud computing platform, needs to simultaneously manage and schedule a large number of containerized applications, database services, and message queues; an Internet of Things system, such as a smart home gateway or a smart city platform, needs to simultaneously process data from a large number of sensors, execute linkage control, and perform data analysis and reporting. "Any multi-task system" further emphasizes the universality of the present method, indicating that it is applicable to all systems that need to simultaneously execute multiple tasks and perform resource scheduling. This can be achieved by providing a general application programming interface (API) and configurable scheduling policies, enabling the method to be integrated into various different multi-task operating systems or application frameworks.

[0100] This application's solution addresses the issue of insufficient versatility by illustrating application scenarios for the resource reuse method, ensuring its flexible adaptability to various multi-tasking systems. Specifically, by defining application scenarios as including but not limited to the listed system types, it emphasizes that the method is not limited to specific domains but covers diverse scenarios such as artificial intelligence, edge computing, and the Internet of Things, thereby eliminating the limitation of the application scope of existing technologies in the background. The characteristic of "application scenarios" indicates the applicable environment of the technical solution, providing a clear implementation background for the resource reuse method; "including but not limited to" indicates that the scenarios are extensible and open, avoiding the limitations of fixed listings and ensuring that the method can adapt to future emerging systems; "intelligent agents, robots, vehicle systems, industrial equipment, cloud, IoT systems, etc." specifically list typical multi-tasking system examples. These scenarios represent high-concurrency, resource-competitive domains, demonstrating the method's universality in solving priority scheduling and resource release problems; "any multi-tasking system" further strengthens the method's versatility, ensuring that the dynamic scheduling and energy-saving mechanisms of the resource reuse method can be seamlessly applied to all systems requiring multi-tasking, thereby improving resource utilization and real-time response capabilities. The resource reuse method deploys multiple computing units with varying resource requirements and dynamically selects the target computing unit based on request priority and task complexity. Simultaneously, inactive computing units are placed in a low-resource-occupancy state, and requests are processed according to priority. This dynamic and flexible resource management mechanism is highly compatible with the diverse needs of multi-task systems. For example, in intelligent agent systems, different tasks (such as speech recognition, natural language processing, and decision generation) have significantly different computational resource requirements, and their priorities may dynamically change with user interaction. This method can dynamically schedule computing units based on these changes, ensuring timely response to high-priority tasks. In vehicular systems, autonomous driving tasks have extremely high requirements for real-time performance and precise resource allocation. This method can flexibly adjust resource allocation according to the driving scenario and task urgency, ensuring the execution of critical tasks. Therefore, by clearly defining these application scenarios, this method not only solves the problems of low resource utilization and rigid scheduling in existing technologies but also greatly expands its application scope and practical value through its wide applicability.

[0101] The following is a concrete example. Taking a robot system as an example, robots typically need to run multiple computing units simultaneously, such as a visual perception model, a path planning algorithm, and a voice interaction model. When the robot is in standby mode, the resource management module can, according to this method, offload the state data of currently inactive computing units such as the voice interaction model from high-speed storage (e.g., GPU memory) to low-speed storage (e.g., CPU memory), putting them into a low-resource-occupancy state, while keeping only key units such as the visual perception model active, ensuring that the robot can quickly respond to environmental changes. When the user issues a voice command, the routing module dynamically selects and loads the voice interaction model based on the request's priority (user-initiated interactions usually have higher priority) and task complexity (speech recognition and understanding), restoring it from a low-resource-occupancy state to an active state and processing the request. The scheduling module ensures that voice command processing has higher priority than other background tasks, thereby achieving dynamic and efficient resource utilization.

[0102] Through the above technical solutions, this method can clearly demonstrate its versatility and effectiveness in various complex multi-tasking systems, eliminate the ambiguity of application scenarios, and thus promote the application of this method in a wide range of fields such as intelligent agents, robots, vehicle systems, industrial equipment, cloud computing, and Internet of Things systems, greatly enhancing its market potential and practical value.

[0103] Traditional computing systems often face problems such as low resource utilization, rigid task scheduling, and untimely resource release when running multiple computationally intensive tasks simultaneously. For example, in artificial intelligence, edge computing, and Internet of Things scenarios, allocating dedicated resources to each task independently will lead to a surge in hardware costs and idle resources; while existing scheduling mechanisms, such as fixed priority or simple sleep strategies, cannot dynamically adapt to changes in task requirements, resulting in delays in response to critical tasks or waste of resources.

[0104] To address this issue, this application proposes a resource reuse system, including a resource pool, a routing module, a resource management module, and a scheduling module. The system aims to achieve efficient multi-task concurrent processing by dynamically scheduling computing units and proactively releasing idle resources. The resource pool stores multiple computing units with varying resource requirements. These units can be machine learning models, data processing flows, or control algorithm instances, each with different resource consumption characteristics. The routing module is configured to dynamically select the most suitable computing unit to process a received request based on its priority and task complexity. Priority can be based on the request source type, user settings, or automatic system determination; task complexity can be determined based on the amount of input data, the expected amount of output data, or historical processing time. The resource management module is responsible for placing currently inactive computing units in a low-resource-occupancy state, for example, offloading the computing unit's state data from high-speed storage to low-speed storage and saving the context for quick recovery. The scheduling module processes requests according to their priority order, pausing or delaying low-priority requests when resources are scarce to ensure timely responses to high-priority tasks.

[0105] The core innovation of this embodiment lies in combining the routing module and the resource management module in a dynamic and collaborative manner. This allows for the dynamic selection of target computing units based on request priority and task complexity, while proactively releasing resources from inactive units. Compared to traditional solutions that rely solely on single-factor scheduling or simple sleep mode, this application achieves on-demand allocation and proactive energy saving, significantly improving resource utilization. Through this technical solution, the system can support more concurrent tasks on limited hardware resources while ensuring the real-time performance of critical operations, reducing overall power consumption and operating costs.

[0106] The following example illustrates this concept. In a smart home edge device, a resource pool deploys a smart home control unit, a video analytics unit, and a voice recognition unit, with their resource requirements increasing sequentially. When a user issues a voice request to "turn on the living room lights," this request is assessed as high priority and low task complexity, and the routing module selects the smart home control unit for processing. Subsequently, the video stream request from the security camera (medium priority, high complexity) is routed to the video analytics unit. If the voice recognition unit has no requests for a period of time, the resource management module unloads its state data from GPU memory to CPU memory, retaining only the recovery context, and the released resources are reused by the video analytics unit. When multiple requests arrive simultaneously, the scheduling module prioritizes high-priority user interaction requests, ensuring rapid system response. In this way, the device efficiently reuses computing resources without increasing hardware costs, avoiding the waste caused by reserving fixed resources for each function in traditional solutions.

[0107] In some of the solutions mentioned above in this application, a resource management module is proposed to place inactive computing units in a low resource consumption state. However, in this process, if there is no specific mechanism to save the state data and ensure efficient recovery, the recovery process may take too long, affecting the system response speed and overall efficiency, thus failing to meet the needs of real-time tasks.

[0108] In response, this application further proposes that the resource management module be configured to offload the state data of the computing unit from high-speed storage to low-speed storage and save the context for rapid recovery.

[0109] Specifically, the resource management module is responsible for monitoring the active status of computing units and optimizing the resources of inactive computing units according to preset strategies or system instructions. Its functions include, but are not limited to, state data migration, resource allocation and reclamation. The state data of a computing unit refers to all data generated during task execution that is necessary to maintain its current working state or for subsequent recovery. For example, for machine learning models, this might include model weights, intermediate activation values, optimizer status, cached data, etc.; for data processing flows, it might include current processing progress, temporary variables, buffer contents, etc. The integrity of this data is crucial for the seamless recovery of the computing unit. High-speed storage refers to storage media with extremely high read / write speeds and low access latency, typically used to store currently active or frequently accessed data to ensure the real-time performance and responsiveness of computing tasks. Examples include processor registers, L1 / L2 / L3 caches, or dedicated accelerator memory (such as GPU memory). Low-speed storage refers to storage media with slower read / write speeds but typically larger capacity and lower cost compared to high-speed storage, suitable for storing data that is not frequently accessed or needs to be stored long-term. For example, this could be main memory (RAM), solid-state drives (SSDs), hard disk drives (HDDs), or network-attached storage (NAS). Offloading to low-speed storage refers to migrating the state data of the computing unit currently residing in high-speed storage to low-speed storage. The purpose is to free up valuable high-speed storage resources for other active computing units, thereby improving overall resource utilization. This process typically involves data serialization and transmission. Saving the context means recording the computing unit's current operating environment, configuration parameters, and the location of the state data in low-speed storage while offloading the state data. This context information is crucial for quickly restoring the computing unit's operating state, ensuring that its previous operating environment can be accurately reconstructed upon reloading.

[0110] The solution in this application operates as follows: The resource management module, as part of the resource reuse system, continuously monitors the activity status of each computing unit in the resource pool. When a computing unit becomes inactive, for example, if it has not received any requests for a period of time or is idle, the resource management module initiates a process to convert it to a low-resource-occupancy state. This conversion process involves two key and coordinated actions: First, the resource management module identifies the critical state data currently residing in high-speed storage (e.g., GPU memory, CPU cache) of the inactive computing unit and systematically transfers this data to low-speed storage (e.g., CPU memory, hard disk). This directly frees up high-speed, high-value storage resources, making them available to other active or high-priority computing units. Second, while the data is unloaded, the resource management module captures and stores the necessary context information related to the computing unit. This context information includes the computing unit's metadata, its configuration, the exact location of the state data in the low-speed storage, and any other parameters used for seamless reinitialization. The combination of unloading state data and saving context ensures that the operational integrity of the computing unit is maintained when it is in a low-resource state. When a new request requires the computing unit, the resource management module can quickly retrieve the saved context, locate the state data in low-speed storage, and reload it back into high-speed storage, thus achieving rapid recovery. This mechanism enables the system to dynamically manage high-speed resources, avoiding idle occupation by inactive units and significantly improving the overall resource utilization efficiency of the resource reuse system.

[0111] As a specific implementation method, an intelligent agent system can be used as an example. In this system, multiple computing units are deployed, such as a large language model and a small dialogue model. When the large language model does not receive new requests for a period of time and is determined to be inactive by the resource management module, the module initiates a resource optimization process. Specifically, the resource management module unloads the model weights, intermediate activation values, and other state data currently occupied by the large language model from the GPU memory to the CPU memory via a data transfer channel. During the unloading process, the resource management module simultaneously records the large language model's storage address in the CPU memory, its current version information, and necessary configuration parameters, forming a complete context record. When a new high-priority request needs to be processed by the large language model, the resource management module quickly loads the model weights and other state data from the CPU memory back to the GPU memory based on the previously saved context information, enabling the large language model to quickly resume its working state and continue processing requests. In this way, GPU memory resources are released during the inactivity period of the large language model, making them available for other active computing units, while the large language model itself can respond quickly when needed, avoiding the delay caused by initialization from scratch.

[0112] Through the above technical solution, the resource management module can effectively offload the state data of inactive computing units from high-speed storage to low-speed storage and properly preserve their context information. This mechanism significantly solves the problem of excessively long recovery times for inactive units in traditional solutions. The timely release of high-speed storage resources allows the system to allocate these valuable resources to currently active or high-priority tasks, thereby greatly improving overall resource utilization. Simultaneously, by preserving the complete context, when computing units need to be reactivated, they do not need to be reinitialized or reloaded from scratch; instead, they can quickly restore to their previous running state based on the saved context information. This ensures that the system maintains a high response speed during dynamic scheduling and resource reuse, especially when handling tasks with high real-time requirements, guaranteeing timely response to critical tasks and avoiding impacts on system performance and user experience due to state recovery delays.

[0113] In some of the solutions described above in this application, a scheduling module is proposed to process requests in order of priority. However, in this process, when system resources are scarce, low-priority requests may continue to occupy computing resources, causing high-priority requests to fail to respond in a timely manner, which affects the system's real-time performance and resource utilization efficiency.

[0114] In response, this application further proposes configuring the scheduling module to pause or delay the processing of low-priority requests when resources are scarce.

[0115] The scheduling module is a core component of the resource reuse system, and its basic function is to process received requests according to a preset priority order. In this embodiment, the scheduling module further has the ability to intervene in request processing under specific conditions. "Resource stress" refers to the situation where the system's currently available computing resources (such as CPU, GPU, memory, network bandwidth, etc.) are insufficient to simultaneously meet the needs of all pending requests, or the resource utilization rate reaches a preset high threshold. This state can be determined in various ways, such as monitoring indicators such as the average load rate of the CPU or GPU, memory usage, task queue length, or waiting time. When these indicators exceed a predetermined safety threshold, the system can be determined to be in a resource stress state. Another way to determine this is when a high-priority request cannot start immediately due to insufficient resources or its response time exceeds expectations; in this case, the system can also be considered resource stressful. "Pausing" the processing of low-priority requests means interrupting the currently executing low-priority task, saving its execution state (such as intermediate calculation results, program counter, register state, etc.), and releasing some or all of the computing resources it occupies so that these resources can be used by high-priority requests. After the resource situation eases, the low-priority request can resume execution from the saved state point. The "delay" handling of low-priority requests typically refers to temporarily holding low-priority requests that have not yet started execution in a waiting queue, without immediately allocating resources to initiate them, or shifting their position in the queue until sufficient system resources are available or high-priority requests have been processed. Both of these approaches aim to free up necessary resources for high-priority requests, ensuring their timely response.

[0116] The resource reuse system of this application introduces a dynamic resource intervention mechanism in its scheduling module, building upon the basic priority-based request processing functionality. Specifically, when the system receives multiple requests, the routing module selects a suitable target computing unit based on the request's priority and task complexity. Subsequently, the scheduling module processes these requests according to their priority order. However, during system operation, the scheduling module continuously monitors the usage of various system resources. Once it detects a strain on critical resources such as CPU, GPU, and memory—for example, if resource utilization exceeds a preset threshold or the waiting time for high-priority requests is too long—the scheduling module immediately initiates an intervention strategy. At this point, the scheduling module no longer passively allocates resources according to priority but actively identifies and takes measures to "pause" or "delay" low-priority requests that are currently executing or waiting to be executed. By pausing, the scheduling module can save the current execution context of low-priority requests and release the computing resources they occupy, thereby freeing up necessary execution space for high-priority requests. By delaying, the scheduling module can prevent low-priority requests from starting when resources are scarce, avoiding further consumption of scarce resources. This dynamic adjustment mechanism ensures that even under extreme resource constraints, high-priority requests can be prioritized and effectively utilize system resources, thereby guaranteeing the real-time responsiveness of critical tasks and optimizing overall resource utilization efficiency. This complements the resource management module's strategy of placing inactive computing units in a low-resource-occupancy state, together forming the system's ability to perform refined resource management at different activity levels.

[0117] As a specific implementation method, an intelligent agent system can be used as an example. Suppose an intelligent agent system is simultaneously running a high-priority user interaction request (e.g., a user is engaged in a real-time voice conversation requiring a rapid response) and a low-priority background data analysis task. Under normal circumstances, the scheduling module processes these two requests in priority order. However, when the system detects a shortage of GPU memory or CPU computing resources—for example, due to the user interaction request needing to load a large language model, resulting in insufficient available GPU memory, or the CPU load being near saturation—the scheduling module immediately identifies the background data analysis task as a low-priority request. The scheduling module can adopt one of two strategies: If the background data analysis task is running, the scheduling module will "pause" it, that is, save the task's current execution context (such as computation progress, intermediate results, etc.) and release some or all of its occupied CPU and memory resources. These released resources are then allocated to the high-priority user interaction request, ensuring that it can obtain the necessary computing resources in a timely manner, thereby guaranteeing the smoothness of the conversation. If the background data analysis task has not yet started execution, the scheduling module will "delay" its start, keeping it in the waiting queue until the high-priority request is processed and system resources are restored to sufficient levels. In this way, even under critical conditions of resource scarcity, the intelligent agent system can prioritize the response speed and user experience of critical tasks such as user interaction.

[0118] Through the above technical solution, this application effectively solves the problem that low-priority requests may continuously occupy computing resources when resources are scarce, resulting in high-priority requests being unable to respond in a timely manner. The scheduling module actively pauses or delays the processing of low-priority requests when resources are scarce, ensuring that high-priority tasks can obtain and effectively utilize the required computing resources first, significantly improving the system's real-time response capability and the execution efficiency of critical tasks. This avoids the ineffective consumption of scarce resources by low-priority tasks under resource constraints, optimizes the resource allocation strategy, and enables the system to maintain a highly efficient and stable operating state even when facing high concurrency or resource bottlenecks. This dynamic resource intervention mechanism not only ensures the continuity of core business and user experience, but also, combined with the basic priority scheduling mechanism, jointly constructs a more robust and adaptive resource management framework, thereby enhancing the adaptability and overall performance of the entire system.

[0119] In some of the solutions described above in this application, a method for reusing resources using computer programs was proposed. However, when storing and executing the program, there are defects such as loading delay and inconvenient deployment, which leads to low efficiency in resource release and scheduling, affecting the overall system response speed and resource utilization.

[0120] In response, this application proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method.

[0121] The computer-readable storage medium refers to a physical medium capable of storing digital data and readable by a computer system. This medium can be a non-volatile storage medium, such as a hard disk drive (HDD), solid-state drive (SSD), flash memory (e.g., USB drive, SD card), or optical disc (e.g., CD-ROM, DVD, Blu-ray disc), used for persistent storage of programs. Alternatively, it can be a volatile storage medium, such as random access memory (RAM), but is typically used for data storage and fast access during program execution. As a carrier of the program, the computer-readable storage medium ensures that the program is preserved even after power is lost and can be loaded and executed by the processor.

[0122] The computer program refers to a set of instructions designed to guide a computer in performing specific tasks or operations. This program can be written in a compiled language (such as C / C++, Java bytecode), compiled to generate an executable file or bytecode, which is then executed directly by the processor or interpreted by a virtual machine. Alternatively, it can be written in an interpreted language (such as Python, JavaScript), which is interpreted and executed line by line by an interpreter at runtime. The computer program encodes the logic of resource reuse methods, enabling the computer to automatically execute these methods, achieving the deployment, dynamic selection, low-resource-occupancy state switching, and priority processing of requests.

[0123] The execution of the program by the processor refers to the process by which the processor (such as a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), etc.) reads and operates according to the instruction sequence in the computer program. This process can manifest as the CPU directly executing machine code instructions, completing instruction operations through stages such as instruction fetching, decoding, execution, memory access, and write-back. Alternatively, it can manifest as the GPU executing parallel computing instructions, which is particularly suitable for large-scale data processing and machine learning tasks. The execution of the program transforms the static program on the storage medium into a dynamically running instruction stream, driving the hardware to complete various functions of the resource reuse method and realizing effective management and scheduling of computing resources.

[0124] The solution presented in this application stores the computer program implementing the resource reuse method on a computer-readable storage medium and executes it via a processor, forming a complete automated execution system. Specifically, the computer-readable storage medium, as the physical carrier, ensures persistent storage and rapid loading of the computer program, thus solving the problem of inconvenient program deployment. The computer program stored on the medium encodes all the logic of the aforementioned resource reuse method, enabling the automated execution of a series of complex operations, such as deploying computing units, dynamically selecting target computing units based on request priority and task complexity, placing inactive computing units in a low-resource-occupancy state, and processing requests according to priority, without manual intervention or configuration. When the processor executes the program, it can dynamically achieve intelligent selection of computing units, timely release of resources, and priority processing of requests. Through standardized and automated program execution processes, loading latency is effectively reduced, and rapid recovery and efficient scheduling of computing units are achieved. This integrated implementation method enables the aforementioned resource reuse method to operate with extremely high efficiency and stability, thereby significantly improving the overall system's resource utilization and response speed.

[0125] The following is a concrete example. In an intelligent agent system, an embedded system flash memory chip (such as eMMC or NOR Flash) can be used as a computer-readable storage medium. This flash memory chip stores a firmware program written and compiled in C++ or Rust, which contains all the logic for the resource reuse method described above. When the intelligent agent device is powered on, its internal embedded microcontroller (such as an ARM Cortex-A series processor) loads the firmware program from the flash memory chip into its random access memory (RAM) and begins execution. After program initialization, multiple threads or processes are started to simulate different computing units. For example, one thread handles inference for a large language model, another handles image recognition processing, and yet another handles interaction for a small dialogue model. The program continuously monitors the various request queues received by the intelligent agent, dynamically activating or suspending the corresponding computing unit threads based on the request priority (e.g., high priority for user-initiated interaction requests, low priority for background data analysis requests) and task complexity (e.g., high complexity for long text generation, low complexity for short text question answering). For example, when a computing unit is inactive for an extended period, the program offloads its state data (such as model weights) from high-speed storage (such as GPU memory) to low-speed storage (such as CPU memory), placing it in a low-resource-occupancy state. When a new request requires that computing unit, the program evaluates its priority and complexity to decide whether to reload and activate it. In this way, the execution of the computer program on the processor ensures that the intelligent agent system can efficiently manage and schedule its computing resources, enabling concurrent multitasking.

[0126] Through the above technical solution, computer programs stored on computer-readable storage media can ensure stable and efficient automated execution of the resource reuse method under various complex scenarios during processor execution. This enables core functions such as dynamic resource scheduling, low-resource-occupancy state switching, and priority processing to operate with extremely low latency and high reliability, thereby significantly improving the overall performance and resource utilization efficiency of the multi-tasking system, ensuring the real-time response capability of critical tasks, effectively solving the defects of loading latency and deployment inconvenience in program storage and execution, and further improving resource release and scheduling efficiency, optimizing the overall system response speed and resource utilization.

[0127] Other application scenarios The following simplified embodiments illustrate the application of the present invention in other fields. Specific implementations of these embodiments can be found in the examples described in the detailed embodiments above, and will not be repeated here.

[0128] Simplified Example 1: In-vehicle System (Autonomous Driving) Autonomous vehicles simultaneously perform multiple computational tasks, including obstacle detection, lane keeping, path planning, and driver monitoring. The system dynamically adjusts resource allocation for each task based on the current driving scenario (e.g., highway vs. urban congestion). For example, during highway cruising, the frequency of obstacle detection is reduced to free up resources for long-distance path planning; during urban congestion, obstacle detection is prioritized to ensure safety.

[0129] Simplified Example 2: Industrial Control Equipment Industrial robots simultaneously perform tasks such as real-time control cycles, fault detection, and remote monitoring. When the equipment is running normally, the fault detection model operates at a low frequency (energy-saving state); when sensor data is abnormal, a high-precision fault diagnosis model is dynamically loaded to prioritize handling potential faults.

[0130] Simplified Example 3: Cloud Service The cloud-based inference platform provides model services of various specifications to different tenants. Model instances are dynamically scheduled based on the priority of tenant requests (paid users > free users) and task complexity (long text vs. short text). Low-priority requests are queued when resources are scarce, while high-priority requests receive resources first. Idle model instances automatically enter a power-saving state, releasing GPU memory.

[0131] Simplified Example 4: Internet of Things Gateway The IoT gateway simultaneously handles preprocessing, anomaly detection, and data reporting tasks from multiple sensors. It dynamically allocates computing resources based on the urgency of the sensor data (e.g., smoke alarm vs. temperature recording). When there are no urgent events, the anomaly detection model is placed in a low-power state, retaining only simple data acquisition functionality.

[0132] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0133] This solution is applicable not only to local devices but can also be deployed on cloud servers. Those skilled in the art should understand that the technical solution of this invention is not limited to a specific deployment environment, and any implementation based on the technical concept of this application should be considered to fall within the protection scope of this invention.

Claims

1. A resource reuse method, characterized in that, Includes the following steps: Deploy multiple computing units with different resource requirements; Based on the priority and complexity of the request, dynamically select the target computing unit to process the request. Place inactive computing units in a low-resource-occupancy state; Requests are processed in order of priority.

2. The method according to claim 1, characterized in that, The low resource usage state includes unloading the state data of the computing unit from high-speed storage to low-speed storage and saving the state data for subsequent recovery.

3. The method according to claim 2, characterized in that, The high-speed storage includes high-speed storage media for computing units, while the low-speed storage includes storage media with larger capacity but relatively slower access speed.

4. The method according to claim 1, characterized in that, The priority of the request is determined based on at least one of the following: request source type, user settings, or automatic system settings.

5. The method according to claim 1, characterized in that, The task complexity is determined based on at least one of the following: input data volume, expected output data volume, required computational depth, or historical processing time.

6. The method according to claim 1, characterized in that, The priority-based request processing includes pausing or delaying the processing of low-priority requests when resources are scarce.

7. The method according to claim 1, characterized in that, It also includes dynamically loading larger computing units to replace the current main computing unit based on user instructions or system requirements.

8. The method according to any one of claims 1 to 7, characterized in that, The method can be applied to, but is not limited to, intelligent agents, robots, industrial equipment, Internet of Things systems, smart home systems, in-vehicle systems, cloud systems, or any multitasking system.

9. A resource reuse system, characterized in that, include: A resource pool is used to store multiple computing units with different resource requirements; The routing module is used to select the target computing unit based on the priority of the request and the complexity of the task. The resource management module is used to place inactive computing units into a low-resource-occupancy state. The scheduling module is used to process requests in order of priority.

10. The system according to claim 9, characterized in that, The resource management module is configured to offload the computing unit's state data from high-speed storage to low-speed storage and save the context for quick recovery.

11. The system according to claim 9, characterized in that, The scheduling module is configured to pause or delay the processing of low-priority requests when resources are scarce.

12. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any one of claims 1 to 8.

Images