Model resource loading method and device, equipment, storage medium and program product

Abstract

Embodiments of the present application provide a model resource loading method, device, equipment, storage medium and program product. The method comprises: determining a loading strategy of the model according to a use condition of the model and a type of the model; the type of the model is related to a parameter quantity of the model; and performing a loading operation of the model according to the loading strategy of the model. The method is used to improve system resource utilization and model use efficiency.

Description

Technical Field

[0001] This application relates to the field of resource allocation technology, and in particular to a model resource loading method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the rapid development of big data and artificial intelligence technologies, models have been widely applied in numerous fields. Currently, in many complex multi-task scenarios, a single model often struggles to meet diverse task requirements. Therefore, multiple models are frequently used in combination to achieve efficient processing and accurate analysis of various complex tasks. However, because different models have varying requirements for computing and storage resources, and these resource requirements are even more complex and diverse in multi-task scenarios, existing resource allocation strategies struggle to achieve efficient and rational resource allocation, resulting in poor resource allocation performance. This, in turn, reduces the overall system's resource utilization and model efficiency.

[0003] Therefore, improving the resource utilization rate of model loading and the efficiency of model usage is an urgent problem to be solved. Summary of the Invention

[0004] The model resource loading method, apparatus, device, storage medium, and program product provided in this application are intended to improve the resource utilization rate and efficiency of model loading.

[0005] In a first aspect, embodiments of this application provide a model resource loading method, including:

[0006] The loading strategy for the model is determined based on its usage and type; the type of the model is related to the number of its parameters.

[0007] According to the loading strategy of the model, the loading operation of the model is executed.

[0008] Optionally, the usage scenarios include: usage frequency; determining the model loading strategy based on the model's usage scenarios and model type includes:

[0009] If the usage frequency of the model is greater than or equal to a first threshold and the number of parameters of the model is less than or equal to a first number of parameters, then the loading strategy of the model is determined to be the first strategy.

[0010] If the frequency of use of the model is less than the first threshold and the number of parameters of the model is greater than the number of parameters of the first threshold, then the loading strategy of the model is determined to be the second strategy.

[0011] Optionally, determining the model loading strategy based on the model's usage and type includes:

[0012] The loading strategy for the model is determined based on its usage, its type, and the resource usage.

[0013] Optionally, determining the model loading strategy based on the model's usage, the model's type, and resource usage includes:

[0014] If the model's usage frequency is less than a first threshold, the model's parameter count is less than or equal to a first parameter count, and the resource usage is less than or equal to a first resource occupancy rate, then the model's loading strategy is determined to be the first strategy.

[0015] If the model's usage frequency is less than the first threshold, the model's parameter count is less than or equal to the first parameter count, and the resource usage is greater than the first resource occupancy rate, then the model's loading strategy is determined to be the second strategy.

[0016] If the model's usage frequency is greater than or equal to the first threshold, the model's parameter count is greater than the first parameter count, and the resource usage is less than or equal to the first resource occupancy rate, then the model's loading strategy is determined to be the first strategy.

[0017] If the model's usage frequency is greater than or equal to the first threshold, the model's parameter count is greater than the first parameter count, and the resource usage is greater than the first resource occupancy rate, then the model's loading strategy is determined to be the second strategy.

[0018] Optionally, it also includes: monitoring the usage of the model.

[0019] Optionally, the usage scenarios include: usage frequency; the usage scenarios of the monitoring model include:

[0020] The popularity type of the model is determined based on the number of times the model is invoked in the first time window; the popularity type includes: cold model and hot model;

[0021] In the case that the model is a cold model, the usage frequency of the model is determined based on the number of times the model is called in the second time window; the second time window includes the first time window.

[0022] Optionally, the method further includes:

[0023] In the case that the model is a hot model, the loading strategy for the model is determined to be the first strategy.

[0024] Optionally, the resource usage includes at least one of CPU usage, memory usage, and GPU usage.

[0025] Secondly, embodiments of this application provide a model resource loading device, comprising:

[0026] The monitoring module is used to monitor the usage of the model;

[0027] The processing module is used to determine the loading strategy of the model based on the model's usage and type; the model type is related to the number of parameters in the model.

[0028] The control module is used to execute the loading operation of the model according to the loading strategy of the model.

[0029] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0030] The memory stores computer-executed instructions;

[0031] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0032] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0033] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0034] The model resource loading method, apparatus, device, storage medium, and program product provided in this application determine a model loading strategy based on the model's usage and the model type related to the number of model parameters. Then, the model loading operation is executed according to the loading strategy. The method of this application can achieve a balance between model resource loading's response speed and resource consumption through different loading strategies, thereby improving the rationality of model resource loading and ultimately enhancing system resource utilization and model usage efficiency. Attached Figure Description

[0035] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0036] Figure 1 A flowchart illustrating a model resource loading method provided in an embodiment of this application;

[0037] Figure 2 A flowchart illustrating another model resource loading method provided in an embodiment of this application;

[0038] Figure 3 A flowchart illustrating yet another model resource loading method provided in this application embodiment;

[0039] Figure 4 This is a schematic diagram of the structure of a model resource loading device provided in an embodiment of this application;

[0040] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0041] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0042] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0043] Currently, models can include mathematical statistics models, machine learning models, deep learning models, etc. Different types of models, based on their unique algorithmic principles and characteristics, play an indispensable role in application scenarios such as industrial production, healthcare, and security monitoring. In multi-task scenarios, multiple models are often needed to work together to meet diverse task requirements through a combination of models with different functions.

[0044] In the practical application of the model, the input data must first be extracted. Then, multiple models required to process the input data are loaded from storage media into the runtime environment. After the multiple models required to process the input data are loaded, the input data is input into the corresponding models for collaborative processing. Finally, the processing results are output through multiple models.

[0045] However, the current model loading process primarily employs a pre-set approach. This requires comprehensively considering factors such as the application scenario (e.g., model complexity, usage frequency) and hardware resources (e.g., processor type, memory size) to pre-debug the data processing program in order to achieve optimal detection efficiency under those conditions. Specifically, model loading is currently performed primarily through the following two methods:

[0046] Method 1: Loading Model One by One: Load the model each time it is detected that it is needed. Then, after the data processing is completed and the data processing results are output through the model, release the resources occupied by the model.

[0047] However, while this loading method avoids unnecessary resource consumption by loading models only when there is an actual need, frequent load and release operations place an additional burden on the system processor. Furthermore, each model loading requires a series of steps, including reading from storage and loading into memory, which reduces overall data processing efficiency to some extent. Especially in scenarios with high processing speed requirements, such frequent load and release operations may compromise the timeliness of data processing, failing to meet the needs of practical applications.

[0048] Method 2: Preload multiple models: This means preloading multiple models, for example, loading multiple models into memory in advance. The resources occupied by these models will not be released.

[0049] However, while this loading method can quickly respond to different data processing needs and avoid the latency issues of loading models one by one, it has certain advantages for scenarios that require rapid switching between models for processing. However, this method suffers from serious resource waste. Under this method, multiple models occupy system resources for extended periods. Even if some models are not used during certain periods, the system resources they occupy cannot be released. This not only limits the available resources for other tasks but also, with limited hardware resources, loading too many models may exceed the hardware's capacity, leading to system crashes and affecting the stability and reliability of the entire system.

[0050] In view of this, this application provides a model resource loading method. By monitoring the model usage in real time, and based on the monitored model usage and the model type determined by the model's parameters, different loading strategies are determined for the model. This allows for a balance between model resource loading and model response speed and resource consumption, thereby improving the rationality of model resource loading and ultimately enhancing system resource utilization and model usage efficiency.

[0051] The execution entity of this model resource loading method can be an electronic device with computing capabilities, such as a server, smartphone, computer, or tablet. This electronic device can be equipped with software or program code that runs the model resource loading method, thereby scheduling the loading of model resources. Alternatively, the execution entity of this model resource loading method can be a system, such as an industrial production system, medical testing system, or data processing system, which then schedules the loading of model resources.

[0052] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below through specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0053] Figure 1 This is a flowchart illustrating a model resource loading method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes:

[0054] S101. Determine the model loading strategy based on the model's usage and type.

[0055] The usage of the model can include, for example, the model's usage patterns, the duration of a single use, and the amount of data processed.

[0056] Invocation details can include invocation frequency, usage frequency, or whether the model is invoked within a specific time period. For example, invocation details can be determined by counting the number of times the model is invoked within a specific time period. A higher invocation frequency indicates that the model is used frequently in the current business scenario and may need to reside in memory for a longer period to ensure fast response.

[0057] The duration of a single use refers to the time spent by the model from the start of task processing to task completion. If the duration of a single use is long, it indicates that the model is processing a large amount of data or has high computational complexity. Frequent loading and unloading of such models will consume a lot of system resources, making it more suitable for them to reside in memory for a long time.

[0058] Data processing volume refers to the size of the data processed by the model. Models that process large amounts of data typically require more system resources and incur greater overhead during loading and unloading. For example, in a big data processing platform, the model responsible for full data cleaning and transformation processes a massive amount of data. By keeping the model loaded in memory for an extended period, the latency caused by frequent loading can be reduced.

[0059] The following embodiments of this application will be described using the model's usage as an example of model invocation. When the model's usage is other, the implementation method is similar.

[0060] The model type is related to the number of parameters in the model. The model type characterizes the model's size. For example, models can be categorized based on the number of parameters, such as large models and small models. Alternatively, the model type can be further subdivided based on the number of parameters; this application does not impose any limitations on this. Taking the classification of models into large and small models based on the number of parameters as an example, models with millions of parameters can be considered small models (e.g., MobileBERT, TinyBERT), while models with hundreds of millions of parameters can be considered large models (e.g., BERT, GPT).

[0061] Optionally, the model type can also be determined based on the type of neural network. For example, models with complex neural network structures (e.g., network layers greater than or equal to a preset layer threshold, number of neurons greater than or equal to a preset number threshold) can be classified as large models. Models with relatively simple neural network structures (e.g., network layers less than a preset layer threshold, number of neurons less than or equal to a preset number threshold) can be classified as small models, etc. The specific classification method can be set according to actual needs; this application is only an example and does not impose specific limitations.

[0062] The model loading strategy may include at least two loading strategies, such as pre-loading (e.g., pre-loading a portion of the model and keeping it in memory for a long time) as described in the previous embodiments, and real-time loading according to data processing needs (also known as on-demand loading).

[0063] In this step, we can determine whether the model needs to be called frequently based on its usage, and determine the size of the model based on its type, so as to predict the amount of system resources required for model loading.

[0064] Based on whether the model needs to be called frequently and the predicted amount of system resources required for model loading, a model loading strategy can be determined. For example, for models that need to be called frequently, a pre-loading strategy can be chosen to avoid loading delays during each call, ensuring that the model can quickly respond to business needs and improve the overall system efficiency. For models that require a large amount of system resources, they can be loaded in real time according to data processing needs, so as to ensure that data processing tasks can be carried out smoothly while minimizing the long-term occupation of system resources and improving the effective utilization of resources.

[0065] Alternatively, the two factors mentioned above can be combined to determine the corresponding loading strategy based on actual needs, in order to balance model response speed and resource consumption.

[0066] S102. Execute the model loading operation according to the model loading strategy.

[0067] The model resource loading method provided in this application determines a model loading strategy based on the model's usage and the model type related to its parameter count. Then, it executes the model loading operation according to the loading strategy. This method can achieve a balance between model response speed and resource consumption through different loading strategies, thereby improving the rationality of model resource loading and ultimately enhancing system resource utilization and model efficiency.

[0068] The following section will take usage scenarios, including usage frequency, as an example to provide a detailed explanation of how to determine the model loading strategy based on the model's usage scenarios and model type in the aforementioned step S102.

[0069] If the model's usage frequency is greater than or equal to the first threshold, and the number of model parameters is less than or equal to the first number of parameters, then the model's loading strategy is determined to be the first strategy.

[0070] If the model's usage frequency is less than the first threshold and the number of model parameters is greater than the first parameter number, then the model loading strategy is determined to be the second strategy.

[0071] The first strategy can be, for example, the pre-loading model strategy mentioned above, and the second strategy can be, for example, the on-demand loading model strategy mentioned above.

[0072] Model usage frequency can be determined by monitoring the number of times the model is called. For example, the number of times the model is called within a specific time window can be recorded. Then, the model usage frequency can be calculated based on the number of times the model is called.

[0073] For example, the frequency can be directly calculated using the number of times the model is called. Alternatively, the usage frequency can be calculated based on the number of calls and a reference parameter. When calculating the frequency based on the number of calls and the reference parameter, this reference parameter can be, for example, a time interval. For instance, a fixed time period can be set, and the usage frequency of the model can be obtained by dividing the number of times the model is called within that time window by the duration of that time period.

[0074] When the model's usage frequency is greater than or equal to the first threshold, it indicates that the model is a high-frequency model in the business process. High-frequency usage means that the model plays an important role in the real-time performance and continuity of the business, and frequent calls require the model to respond quickly. When the model's usage frequency is less than the first threshold, it indicates that the model is a low-frequency model in the business process. Low-frequency usage means that the model is not frequently necessary during business execution, has a smaller direct impact on the real-time performance and continuity of the business, and is called only under specific conditions or occasionally.

[0075] The number of model parameters can be determined by the model's structural information and parameter storage method. When the number of parameters is greater than the first parameter count, it indicates that the model is relatively large. Larger models usually have stronger expressive power, but require more system resources for loading and running. Conversely, when the number of parameters is less than or equal to the first parameter count, it indicates that the model is relatively small, requiring fewer system resources for loading and running.

[0076] When the model's usage frequency is greater than or equal to a first threshold, and the model's parameter count is less than or equal to a first parameter count, the high-frequency model needs to respond quickly to business requests, while the relatively small parameter count means the model consumes limited system resources. A preloading strategy can load the model into memory in advance. When a business request arrives, it can be used immediately without waiting for the loading process, significantly shortening response time and improving overall system efficiency. Since the model itself doesn't consume many resources, preloading won't place an excessive resource burden on the system. Therefore, in this case, the preloading strategy is a good choice to balance response speed and resource consumption.

[0077] Conversely, when the model's usage frequency is less than the first threshold, and the model's parameter count is greater than the first parameter count, low-frequency usage indicates that the model is not a core, high-frequency business requirement, while a large parameter count indicates that loading the model will consume a significant amount of system resources. In this case, using a pre-loading strategy would result in resources being occupied for extended periods despite infrequent actual usage, leading to resource waste. Therefore, a second strategy (such as an on-demand loading strategy) can be adopted, loading the model only when it is actually needed and releasing resources promptly after use. This effectively avoids unnecessary resource consumption and improves system resource utilization.

[0078] In one possible implementation, the model loading strategy can be further determined by considering resource usage. In this implementation, step S102, which determines the model loading strategy based on model usage and model type, may specifically include:

[0079] The model loading strategy is determined based on the model's usage, model type, and resource usage.

[0080] The resource usage information may include at least one of the following: CPU usage, memory usage, and GPU usage. Optionally, resource usage information may also include storage I / O information. This resource usage information may be obtained through monitoring by the aforementioned real-time system monitoring script, or it may be obtained based on data from the system's native resource manager, etc.

[0081] Regarding CPU usage:

[0082] When CPU utilization is high (or CPU usage is high), frequent loading of models with complex computational requirements may further increase the CPU load, leading to a significant degrade in system performance. Therefore, loading new models under high CPU load should be avoided as much as possible. When CPU utilization is low and there are plenty of idle resources, some models can be pre-loaded into memory to ensure a fast response and reduce waiting time when these models are needed.

[0083] Regarding GPU usage:

[0084] When GPU utilization is high, frequent loading of computationally intensive models that heavily rely on GPU resources can intensify GPU resource contention, further burdening the GPU and leading to a significant performance drop in tasks such as graphics processing and deep learning computations, even causing stuttering. In such cases, it's best to avoid loading new GPU-intensive models when the GPU is under high load; wait until the GPU load decreases before loading new models. When GPU utilization is low and there are many idle resources, computationally intensive models suitable for GPU acceleration can be pre-loaded into GPU memory. This allows for a faster response time when these models are needed, leveraging the GPU's powerful computing capabilities, significantly reducing processing time, improving model efficiency, and fully utilizing idle GPU resources for business applications.

[0085] Regarding memory usage:

[0086] When memory usage is high (i.e., high memory occupancy), system memory resources are strained. Loading models that require significant memory can lead to memory exhaustion, causing frequent memory swapping, severely impacting overall system performance, and potentially causing system crashes. Therefore, loading new models should be avoided under high memory load. When memory usage is low and there is ample free memory, frequently used models or models expected to be used soon can be pre-loaded into memory. This eliminates the need to wait for data to be read from storage into memory when these models are actually called, allowing for rapid model startup and processing, effectively shortening model response time.

[0087] Furthermore, in storage I / O scenarios, frequent read and write operations can create significant I / O pressure, and loading models may further exacerbate this pressure. In such cases, model loading operations that require substantial read and write access to storage devices should be minimized. When storage I / O load is low, it's advisable to preload some models from storage devices into memory to reduce loading latency during subsequent use.

[0088] For example, the following provides a specific model loading strategy, which includes the following four cases:

[0089] Case 1: If the model's usage frequency is less than the first threshold, the number of model parameters is less than or equal to the first number of parameters, and the resource usage is less than or equal to the first resource utilization rate, then the model's loading strategy is determined to be the first strategy.

[0090] Specifically, a model usage frequency below the first threshold indicates that the model is not frequently invoked in the business process, has a relatively small direct impact on business real-time performance, and is not part of the core high-frequency business requirements. A model parameter count less than or equal to the first parameter count indicates a relatively small model size, requiring relatively few system resources for loading and runtime, and will not place an excessive resource burden on the system. Resource utilization less than or equal to the first resource occupancy rate indicates that the current system resources are relatively abundant, in a low-load state, and have sufficient resources to handle various operations.

[0091] Therefore, although the model is used infrequently, given its small size and ample system resources, loading it into memory will not excessively consume system resources or cause resource contention issues. On the contrary, pre-loading the model allows it to be quickly retrieved and used directly from memory when needed by the business, avoiding the time overhead of temporary loading. This effectively improves the response speed of business processing, better meeting the occasional use needs of the model without affecting the efficient utilization of system resources, and ensuring the smooth operation of the entire business process.

[0092] Case 2: If the model's usage frequency is less than the first threshold, the number of model parameters is less than or equal to the first number of parameters, and the resource usage is greater than the first resource occupancy rate, then the model loading strategy is determined to be the second strategy.

[0093] Specifically, a model usage frequency below the first threshold indicates that the model is not frequently invoked in the business process and is not a critical model used frequently in core business processes. From the perspective of the overall business architecture and execution path, its impact on the immediacy and continuity of the business is relatively small. A model parameter count less than or equal to the first parameter count indicates that the model is relatively small in scale, with low complexity and parameter size, meaning that the model's demand on system resources during operation is relatively low. Resource usage exceeding the first resource occupancy rate indicates that the system is currently in a state of relatively tight resource utilization, with high loads on various resources (such as CPU utilization, GPU utilization, memory utilization, etc.).

[0094] Therefore, although the model is small in scale and not frequently used, due to the scarcity of system resources, in order to ensure that the core business and critical tasks running on the system can obtain sufficient resource support, maintain the stable operation of the system, and avoid system performance degradation or even failure due to excessive resource consumption, it is necessary to adopt an on-demand loading strategy. This will meet the occasional business needs for the model while minimizing the long-term occupation of system resources by the model, improving resource utilization, and avoiding system crashes or reduced model processing efficiency due to excessive resource load.

[0095] Case 3: If the model's usage frequency is greater than or equal to the first threshold, the number of model parameters is greater than the first number of parameters, and the resource usage is less than or equal to the first resource utilization rate, then the model's loading strategy is determined to be the first strategy.

[0096] Among these criteria, a model usage frequency greater than or equal to the first threshold indicates that the model is a core part of the business process and is frequently used, with frequent calls playing a decisive role in the real-time performance and continuity of the business. A model parameter count greater than the first parameter count indicates a large and complex model with numerous parameters and complex computational logic. This means the model consumes significant system resources during loading and execution, placing high demands on the system's computing power and memory capacity. Resource usage less than or equal to the first resource occupancy rate indicates that the overall system resources are relatively abundant, with sufficient spare capacity available for all types of resources (such as CPU, memory, GPU, etc.), and the system load is at a low level, providing ample resource guarantees for loading and running large and complex models.

[0097] Therefore, although the model is large in scale, the current resource usage indicates that resources are relatively abundant. Furthermore, the model is used frequently, and pre-loading it ensures that it can be quickly retrieved from memory and used directly when business requests are frequent, significantly shortening loading time and effectively improving the response speed and overall efficiency of business processing. At the same time, due to sufficient system resources, keeping this large model resident in memory will not cause resource contention or insufficient resources; instead, it can fully utilize idle system resources, creating a stable and efficient operating environment for the frequently used large model. This strongly guarantees the smooth progress of task processing and the optimization of system performance, ensuring the efficient and stable operation of the entire business process.

[0098] Case 4: If the model's usage frequency is greater than or equal to the first threshold, the number of model parameters is greater than the first number of parameters, and the resource usage is greater than the first resource occupancy rate, then the model loading strategy is determined to be the second strategy.

[0099] Among these criteria, a model usage frequency greater than or equal to the first threshold indicates that the model is a frequently used key component in the business process and is one of the core links ensuring the normal operation of the business. From the perspective of task processing continuity and real-time processing requirements, its frequent invocation demands place extremely high demands on system response speed; any loading delay may affect the smooth progress of task processing. A model parameter count greater than the first parameter count indicates that the model is large in scale and highly complex, requiring a large amount of system resources during operation, covering memory space, CPU computing resources, storage I / O, and other aspects. Resource usage exceeding the first resource utilization rate indicates that the system is currently operating under high load and resource constraints, with various resources (such as CPU utilization, GPU utilization, memory utilization, etc.) approaching or reaching saturation, making it difficult to load new models.

[0100] Therefore, although this model is used frequently, current resource usage indicates that resources are relatively scarce. To ensure that the core tasks and critical tasks currently in operation receive sufficient resource support, maintain stable system operation, and avoid resource exhaustion or system crashes due to overloading of large models, an on-demand loading strategy is required. This will meet the high-frequency business needs for the model while minimizing the model's long-term occupation of system resources, rationally allocating resources, and improving resource utilization efficiency.

[0101] In one possible implementation, the usage of the model can be monitored, and the method in the aforementioned method embodiment can be implemented based on the monitored model usage.

[0102] Optionally, model usage can be monitored based on log records. For example, logging functionality can be embedded in the model's code implementation, recording detailed timestamps, call parameters, task status, and other information each time the model is invoked, a task begins, and ends. Logs can be stored in the file system or a dedicated log management system. By periodically analyzing this log data, model usage can be monitored.

[0103] Optionally, system performance monitoring tools can be used to monitor model usage. For example, the performance monitoring tools built into the operating system or programming language can be used to monitor process resource usage in real time. Simultaneously, the event listening function provided by the monitoring tool can be used to capture model start and stop events, thereby statistically analyzing the number of model calls and usage duration.

[0104] Optionally, the model's usage can be monitored in real time via a pre-developed system monitoring script. This script can run in real time and interact with the detection program in the system, which can obtain information such as the current model usage, CPU / GPU utilization, and memory utilization in real time.

[0105] Figure 2 A flowchart illustrating another model resource loading method provided in this application embodiment is shown below. Figure 2 As shown, usage includes usage frequency. The usage of the above monitoring model can be specifically implemented through the following steps:

[0106] S201. Determine the model's popularity type based on the number of times the model is called in the first time window.

[0107] The popularity categories include: cold models and hot models. Cold models are those that were not called within the first time window, or those that were called infrequently; meaning there is no recent demand for these models, it can be predicted that they will not need to be called in the near future. Hot models are those that were called within the first time window, or those that were called frequently; meaning there is a recent demand for these models, it can be predicted that they will need to be called in the near future.

[0108] One possible implementation is to determine the model's popularity type based on whether the model has been called within the first time window. For example, if the model has been called within the first time window, its popularity type is a hot model; if the model has not been called within the first time window, its popularity type is a cold model.

[0109] Another possible implementation is to determine the model's popularity type based on whether the number of times the model is called within the first time window is greater than or equal to a preset threshold. For example, if the number of times the model is called within the first time window is greater than or equal to the preset threshold, its popularity type is a hot model; if the number of times the model is called within the first time window is less than the preset threshold, its popularity type is a cold model.

[0110] S202. When the model is a cold model, determine the model usage frequency based on the number of times the model is called in the second time window.

[0111] The second time window includes the first time window. That is, the length of the second time window is greater than that of the first time window.

[0112] Since the model is a cold model, it can be predicted that the model will not be needed for a period of time or in the future. It is necessary to further determine the usage frequency of the model, and then determine the loading strategy of the model based on the model type, resource usage, and usage frequency described in the foregoing embodiments.

[0113] The model's usage frequency can be calculated based on the number of times the model is invoked within the second time window. For example, the usage frequency can be calculated based on the number of times the model is invoked and reference parameters. For instance, a fixed time period can be set, and the number of times the model is invoked within the second time window can be divided by the duration of this time period to determine the model's usage frequency.

[0114] Alternatively, if the second time window corresponds to multiple models, these models can be sorted based on the number of times they are called. Then, the usage frequency of each model can be determined based on the sorting result. For example, if the second time window corresponds to three models A, B, and C, and model A is called twice, model B once, and model C three times, then the sorting result is C > A > B. Taking model B as an example, the usage frequency of model B can be determined to be medium frequency.

[0115] S203. When the model is a hot model, the loading strategy of the model is determined to be the first strategy.

[0116] If the model is a "hot" model, it indicates that there is a recent demand for the model, which can be used to predict whether the model will need to be called in the near future. Therefore, based on this prediction, the model can be loaded in advance so that when the model needs to be called in the near future, the model response can be accelerated. Compared with loading the model temporarily, this saves the model loading time during task processing and improves the efficiency of task processing.

[0117] The method provided in this application determines the model's popularity type based on the number of times the model is called in a first time window. If the model is a cold model, its usage frequency is determined based on the number of times it is called in a second time window. Then, based on this usage frequency, model type, and resource usage, a loading strategy for the model is further determined. If the model is a hot model, the loading strategy is determined to be the first strategy. This method improves the efficiency of hot model determination and reduces the time spent determining the loading strategy for hot models by first identifying hot models and executing the first strategy before determining the loading strategy based on the model's usage frequency, model type, and resource usage.

[0118] For ease of understanding, an exemplary complete method flow of this application is described below. Figure 3 This is a flowchart illustrating another model resource loading method provided in an embodiment of this application. Figure 3 As shown, the method may include:

[0119] First, based on the number of times the model is invoked within the first time window, the model's popularity type is determined (i.e., Figure 3 (The model is determined to be hot or cold). If the model is a hot model, the first loading strategy is determined; if the model is a cold model, the loading strategy is further determined based on the model's usage frequency, the number of model parameters, and resource usage.

[0120] If the model is a cold model, and the model's usage frequency is less than the first threshold (i.e., low frequency), then it is further determined whether the number of model parameters is less than or equal to the first number of parameters (i.e., size). If not (i.e., large size), then the model's loading strategy is determined to be the second strategy (i.e., no loading, meaning no preloading, and loading only when there is a corresponding demand for the model).

[0121] If so (i.e., small scale), then further determine whether the resource usage is greater than the first resource occupancy rate (i.e., determine whether resources are scarce or idle). If so (i.e., scarce resources), then determine the model loading strategy as the second strategy (i.e., no loading, meaning no preloading, loading only when the corresponding demand for the model exists); if not (i.e., idle resources), then determine the model loading strategy as the first strategy (i.e., loading, meaning preloading).

[0122] If the model is a cold model, and the model's usage frequency is greater than or equal to the first threshold (i.e., high frequency), then it is further determined whether the number of model parameters is less than or equal to the first number of parameters. If so (i.e., small scale), then the model's loading strategy is determined to be the first strategy (i.e., preloading).

[0123] If not (i.e., the scale is large), then further determine whether the resource usage exceeds the first resource occupancy rate (i.e., determine whether resources are scarce or idle). If yes (i.e., resources are scarce), then determine the model loading strategy as the second strategy; if no (i.e., resources are idle), then determine the model loading strategy as the first strategy.

[0124] Figure 4 This is a schematic diagram of a model resource loading device provided in an embodiment of this application. Figure 4 As shown, the model resource loading device may include: a processing module 11 and a control module 12. In one possible implementation, it may also include: a monitoring module 13.

[0125] Processing module 11 is used to determine the model loading strategy based on the model's usage and type. The model type is related to the number of model parameters.

[0126] The control module 12 is used to execute the model loading operation according to the model loading strategy. Optionally, the processing module 11 is specifically used to obtain the theoretical capacity change value based on the first SOC and the second SOC. Based on the theoretical capacity change value and the actual change value of the target battery capacity, the SOHC is obtained.

[0127] Optionally, when the usage scenario includes usage frequency, the processing module 11 is specifically configured to determine the model loading strategy as the first strategy if the model's usage frequency is greater than or equal to a first threshold and the model's parameter count is less than or equal to a first parameter count. If the model's usage frequency is less than the first threshold and the model's parameter count is greater than the first parameter count, then the model loading strategy is determined to be the second strategy.

[0128] Optionally, the processing module 11 is specifically used to determine the model loading strategy based on the model's usage, model type, and resource usage.

[0129] Optionally, the processing module 11 is specifically used to determine the model loading strategy as the first strategy when the model usage frequency is less than a first threshold, the number of model parameters is less than or equal to the first number of parameters, and the resource usage is less than or equal to the first resource occupancy rate.

[0130] If the model's usage frequency is less than the first threshold, the number of model parameters is less than or equal to the first number of parameters, and the resource usage is greater than the first resource occupancy rate, then the model loading strategy is determined to be the second strategy.

[0131] If the model's usage frequency is greater than or equal to the first threshold, the number of model parameters is greater than the first number of parameters, and the resource usage is less than or equal to the first resource occupancy rate, then the model's loading strategy is determined to be the first strategy.

[0132] If the model's usage frequency is greater than or equal to the first threshold, the number of model parameters is greater than the first number of parameters, and the resource usage is greater than the first resource occupancy rate, then the model loading strategy is determined to be the second strategy.

[0133] Optionally, monitoring module 13 is used to monitor the usage of the model.

[0134] Optionally, when usage includes usage frequency, the monitoring module 13 is specifically used to determine the model's popularity type based on the number of times the model is called in the first time window. Popularity types include: cold model and hot model. The processing module 11 is specifically used to determine the model's usage frequency based on the number of times the model is called in the second time window when the model is a cold model. The second time window includes the first time window.

[0135] Optionally, the processing module 11 is also used to determine the model loading strategy as the first strategy when the model is a hot model.

[0136] Optionally, resource usage includes at least one of CPU usage, memory usage, and GPU usage.

[0137] The model resource loading device provided in this application embodiment can execute the model resource loading method in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0138] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device is used to execute the aforementioned model resource loading method, and may be, for example, the aforementioned electronic device with data processing capabilities. Figure 5 As shown, the electronic device 500 may include at least one processor 501, a memory 502, and a communication interface 503.

[0139] The memory 502 is used to store programs. Specifically, the program may include program code, which includes computer operation instructions.

[0140] Memory 502 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0141] The processor 501 is used to execute computer execution instructions stored in the memory 502 to implement the method described in the foregoing method embodiments. The processor 501 may be a CPU, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0142] Processor 501 can communicate and interact with external devices through communication interface 503. In specific implementations, if communication interface 503, memory 502, and processor 501 are implemented independently, they can be interconnected via a bus to complete communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0143] Optionally, in a specific implementation, if the communication interface 503, memory 502, and processor 501 are integrated on a single chip, then the communication interface 503, memory 502, and processor 501 can communicate through an internal interface.

[0144] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0145] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0146] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0147] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0148] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0149] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0150] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0151] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0153] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for loading model resources, characterized in that, include: The loading strategy for the model is determined based on its usage and type. The type of the model is related to the number of parameters in the model; According to the loading strategy of the model, the loading operation of the model is executed.

2. The method according to claim 1, characterized in that, The usage scenario includes: usage frequency. The process of determining the model loading strategy based on the model's usage scenario and model type includes: If the usage frequency of the model is greater than or equal to a first threshold and the number of parameters of the model is less than or equal to a first number of parameters, then the loading strategy of the model is determined to be the first strategy. If the frequency of use of the model is less than the first threshold and the number of parameters of the model is greater than the number of parameters of the first threshold, then the loading strategy of the model is determined to be the second strategy.

3. The method according to claim 1, characterized in that, The step of determining the model loading strategy based on the model's usage and type includes: The loading strategy for the model is determined based on its usage, its type, and the resource usage.

4. The method according to claim 2, characterized in that, The step of determining the model loading strategy based on the model's usage, model type, and resource usage includes: If the model's usage frequency is less than a first threshold, the model's parameter count is less than or equal to a first parameter count, and the resource usage is less than or equal to a first resource occupancy rate, then the model's loading strategy is determined to be the first strategy. If the model's usage frequency is less than the first threshold, the model's parameter count is less than or equal to the first parameter count, and the resource usage is greater than the first resource occupancy rate, then the model's loading strategy is determined to be the second strategy. If the model's usage frequency is greater than or equal to the first threshold, the model's parameter count is greater than the first parameter count, and the resource usage is less than or equal to the first resource occupancy rate, then the model's loading strategy is determined to be the first strategy. If the model's usage frequency is greater than or equal to the first threshold, the model's parameter count is greater than the first parameter count, and the resource usage is greater than the first resource occupancy rate, then the model's loading strategy is determined to be the second strategy.

5. The method according to any one of claims 1-4, characterized in that, Also includes: Monitor the usage of the model.

6. The method according to claim 5, characterized in that, The usage details include: usage frequency; the usage details of the monitoring model include: The popularity type of the model is determined based on the number of times the model is invoked in the first time window; the popularity type includes: cold model and hot model; In the case that the model is a cold model, the usage frequency of the model is determined based on the number of times the model is called in the second time window; the second time window includes the first time window.

7. The method according to claim 6, characterized in that, The method further includes: In the case that the model is a hot model, the loading strategy for the model is determined to be the first strategy.

8. The method according to claim 3 or 4, characterized in that, The resource usage includes at least one of the following: CPU usage, memory usage, and GPU usage.

9. A model resource loading device, characterized in that, include: The monitoring module is used to monitor the usage of the model; The processing module is used to determine the loading strategy of the model based on the usage and type of the model. The type of the model is related to the number of parameters in the model; The control module is used to execute the loading operation of the model according to the loading strategy of the model.

10. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-8.

Images