Model file loading method and device, equipment, storage medium and product

Abstract

The embodiment of the invention relates to the technical field of computer application, and provides a model file loading method and device, equipment, a storage medium and a product. The method comprises the steps that a corresponding target node is determined according to a model file extension name; judging whether a model file object exists in a cache of the target node according to the model file path and a pre-constructed cache mapping table; if yes, returning the model file object to the calling party; if not, determining a corresponding loading strategy class from a pre-constructed strategy registry according to the model file extension name; and loading the model file object according to the model file path by using the loading strategy class, returning the model file object to the calling party, and storing the model file object in the cache of the target node. Through the embodiment of the invention, the loading efficiency and stability of different types of model files can be improved.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer application technology, and in particular to a model file loading method, device, equipment, storage medium and product. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, deep learning models are widely used across industries. However, these deep neural network models often have a large number of parameters and require extensive computation, making their deployment on resource-constrained embedded devices challenging. Model file loading involves reading a trained deep learning model (such as the neural network's structure and parameters) from a storage device (such as a hard drive or the cloud) into computer memory, preparing it for inference (prediction) or continued training. Model file loading bridges the gap between "trained models" and "real-world applications." Its efficiency and stability directly impact the model's implementation on embedded devices, servers, or mobile devices. Optimizing the loading process is key to improving model deployment performance.

[0003] However, the existing model file loading methods have the following main problems. On the one hand, most of the existing model engines only support loading a single model file type. If a new type of model file is to be loaded, the project source code needs to be customized and modified. Technical developers need to go through a series of processes such as scheduling, development, testing and regression, and deployment. This makes the change and iteration cycle of such model engines for new types of model files too long, and the model file loading efficiency is low. On the other hand, there is a problem of high coupling when loading different types of model files. Specifically, different types of model files may require different loading logic, such as parsing different file formats, calling different libraries or APIs, or processing different data structures. Existing loading methods usually write the loading logic of different types of model files into the main code, and then distinguish and process them through conditional judgment. In this loading method, the loading logic of different model files is not well isolated. When the loading logic of one model file needs to be modified, it may affect the loading process of other model files, thereby affecting the stability of the loading process. Therefore, there is an urgent need for a model file loading method to achieve efficient and stable loading of different types of model files. Summary of the Invention

[0004] In response to the above-mentioned problems in the prior art, the purpose of the embodiments of this specification is to provide a model file loading method, device, equipment, storage medium and product to solve the problems of low efficiency and poor stability of model file loading in the prior art.

[0005] In order to solve the above technical problems, the specific technical solutions of the embodiments of this specification are as follows:

[0006] On the one hand, an embodiment of this specification provides a model file loading method, the method comprising:

[0007] Receive the model file loading request sent by the caller;

[0008] Parsing the loading request to obtain the model file extension and model file path;

[0009] Determine the corresponding target node according to the model file extension;

[0010] Determining whether a model file object exists in the cache of the target node according to the model file path and a pre-built cache mapping table, wherein the cache mapping table includes a mapping relationship between a node cache address and a model file path;

[0011] If so, the model file object is returned to the caller;

[0012] If not, determining the corresponding loading policy class from a pre-built policy registry according to the model file extension, wherein the policy registry contains a mapping relationship between the model file extension and the loading policy class;

[0013] The model file object is loaded according to the model file path using the loading strategy class, the model file object is returned to the caller, and the model file object is stored in the cache of the target node.

[0014] Furthermore, determining the corresponding target node according to the model file extension includes:

[0015] Determining a target node group according to the model file extension and a pre-built node mapping table, wherein the node mapping table includes a mapping relationship between the model file extension and the node group;

[0016] A polling strategy is adopted to select a node from the target node group as the target node of the model file.

[0017] Furthermore, before storing the model file object in the cache of the target node, the method further includes:

[0018] Calculate the model file object size;

[0019] Determining whether the storage space of the target node is larger than the size of the model file object according to the size of the model file object;

[0020] If yes, storing the model file object in the cache of the target node;

[0021] If not, the model file object in the cache of the target node is cleaned up according to the preset cache elimination strategy until the storage space in the cache is larger than the size of the model file object, and the model file object is stored in the cache of the target node.

[0022] Furthermore, the method further comprises:

[0023] Determining a target storage block from the cache space of the target node according to the size of the model file object;

[0024] Generate a memory address of the target storage block according to the model file path;

[0025] Generate a node cache address of the model file object according to the target node identifier and the memory address;

[0026] A mapping relationship is formed between the model file path and the node cache address, and the mapping relationship is written into the cache mapping table.

[0027] Furthermore, the cleaning of the model file objects in the cache of the target node according to the preset cache elimination strategy includes:

[0028] Obtain cache information of each model file object in the cache of the target node, wherein the cache information includes the latest call time, call frequency, loading time and memory usage;

[0029] The idle time of each model file object is calculated based on the difference between the latest call time of each model file object and the current time;

[0030] Calculate the call activity of each model file object according to the idle time and call frequency of each model file object;

[0031] Calculate the loading cost of each model file object based on the loading time and memory usage of each model file object;

[0032] Calculate the cleanup priority of each model file object according to the loading cost and the call activity;

[0033] The model file objects whose cleaning priority is greater than a preset threshold are removed from the cache of the target node.

[0034] Furthermore, the method further comprises:

[0035] If the loading policy class corresponding to the model file extension does not exist in the policy registry, creating a corresponding loading policy class for the model file extension, the loading policy class including the loading logic of the model file extension;

[0036] Add the mapping relationship between the loading strategy class and the model file extension to the strategy registry.

[0037] On the other hand, an embodiment of this specification provides a model file loading device, the device comprising:

[0038] The receiving module is used to receive the model file loading request sent by the caller;

[0039] A parsing module, configured to parse the loading request to obtain a model file extension and a model file path;

[0040] A first determining module, configured to determine a corresponding target node according to the model file extension;

[0041] a determination module, configured to determine whether a model file object exists in the cache of the target node according to the model file path and a pre-built cache mapping table, wherein the cache mapping table includes a mapping relationship between a node cache address and a model file path;

[0042] A return module, configured to return the model file object to the caller when the model file object exists in the cache of the target node;

[0043] a second determining module configured to determine, when the model file object does not exist in the cache of the target node, a corresponding loading policy class from a pre-built policy registry according to the model file extension, wherein the policy registry contains a mapping relationship between the model file extension and the loading policy class;

[0044] A loading module is used to load the model file object according to the model file path using the loading strategy class, return the model file object to the caller, and store the model file object in the cache of the target node.

[0045] On the other hand, an embodiment of this specification further provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein when the computer program is run by the processor, the computer program executes instructions of any one of the above methods.

[0046] On the other hand, an embodiment of the present specification further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor of a computer device, executes instructions of any one of the above methods.

[0047] On the other hand, the embodiments of this specification further provide a computer program product, which, when executed by a processor of a computer device, executes instructions of any one of the above methods.

[0048] By adopting the above technical solution, the model file loading method provided in the embodiment of this specification establishes a mapping relationship between the model file extension and the loading policy class through a policy registry. Different types of model files have their own corresponding loading policy classes, thereby realizing the decoupling of the loading logic of different types of model files. When it is necessary to modify the loading logic of one of the model files, only the corresponding loading policy class needs to be modified without affecting the loading logic of other types of models, thus avoiding the mixing of code logic. And when it is necessary to load a new model file type, only a new mapping relationship and a corresponding loading policy class need to be added without modifying the existing main code, thereby improving the scalability and maintainability of the system. By introducing a node cache and a cache mapping table, different types of model files can be stored in different target node caches, thus realizing resource isolation and reducing the mutual influence between the loading of different model files, thereby improving the stability of model file loading. At the same time, the node cache is checked before loading the model file. If the model file object exists in the node cache, it is directly returned, thus avoiding repeated loading and improving loading efficiency.

[0049] The above description is only an overview of the technical solutions of some embodiments of this specification. In order to more clearly understand the technical means of some embodiments of this specification, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiments of this specification more obvious and easy to understand, the following specifically cites preferred embodiments and provides detailed descriptions in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A schematic diagram showing the steps of a model file loading method in some embodiments of this specification is shown;

[0052] Figure 2 A schematic diagram of a process of executing an online model by an existing model engine in some embodiments of this specification is shown;

[0053] Figure 3 A flow chart showing model file loading using a strategy pattern molecule class in some embodiments of this specification is shown;

[0054] Figure 4 A schematic diagram of determining a target node in some embodiments of this specification is shown;

[0055] Figure 5A flowchart of determining how to store a model file object in a target node in some embodiments of this specification is shown;

[0056] Figure 6 A schematic diagram showing the steps of clearing model file objects in the cache of a target node according to a preset cache elimination strategy in some embodiments of this specification is shown;

[0057] Figure 7 A schematic diagram showing the steps of determining a node cache address of a model file object in some embodiments of this specification is shown;

[0058] Figure 8 A schematic structural diagram of a model file loading device in some embodiments of this specification is shown;

[0059] Figure 9 A schematic structural diagram of a computer device in this specification is shown.

[0060] Description of the accompanying symbols:

[0061] 801, receiving module;

[0062] 802, parsing module;

[0063] 803. First determination module;

[0064] 804, judgment module;

[0065] 805, return module;

[0066] 806. Second determination module;

[0067] 807, load module;

[0068] 902. Computer equipment;

[0069] 904, processor;

[0070] 906. Memory;

[0071] 908, driving mechanism;

[0072] 910, input / output module;

[0073] 912. Input devices;

[0074] 914. Output device;

[0075] 916. Presentation equipment;

[0076] 918. Graphical User Interface;

[0077] 920, network interface;

[0078] 922, communication link;

[0079] 924. Communication bus. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this specification.

[0081] It should be noted that the terms "first," "second," and the like in this specification, the claims, and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this specification described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products, or devices.

[0082] Existing model file loading methods have the following major problems. First, most existing model engines only support loading a single model file type. Loading a new model file type requires customization and modification of the project source code. Technical developers need to go through a series of processes such as scheduling, development, testing, regression, and deployment. This makes the modification and iteration cycle of such model engines for new model files too long, and the model file loading efficiency is low. Second, the loading of different types of model files is highly coupled. Specifically, different types of model files may require different loading logic, such as parsing different file formats, calling different libraries or APIs, or processing different data structures. Existing loading methods often write the loading logic of different model file types into the main code and then distinguish them through conditional judgments, such as directly coupling the loading logic of different model file types (such as .pkl and .py) through conditional statements. In this loading method, the loading logic of different model files is not well isolated. When the loading logic of one model file needs to be modified, it may affect the loading process of other model files, thereby affecting the stability of the loading process.

[0083] In order to solve the above problems, this specification provides a model file loading method. Figure 1This is a step diagram of a model file loading method provided in the embodiment of this specification. This specification provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many steps, and does not represent the only execution order. When the actual system or device product is executed, it can be executed in the order or in parallel according to the method shown in the embodiment or the accompanying drawings. Specifically, Figure 1 As shown, the method may include:

[0084] S101: receiving a model file loading request sent by a caller;

[0085] S102: Parse the loading request to obtain the model file extension and model file path;

[0086] S103: Determine the corresponding target node according to the model file extension;

[0087] S104: Determine whether a model file object exists in the cache of the target node according to the model file path and a pre-built cache mapping table;

[0088] S105: If yes, return the model file object to the caller;

[0089] S106: If not, determining a corresponding loading policy class from a pre-built policy registry according to the model file extension, wherein the policy registry contains a mapping relationship between the model file extension and the loading policy class;

[0090] S107: Using the loading strategy class to load the model file object according to the model file path, returning the model file object to the caller, and storing the model file object in the cache of the target node.

[0091] By adopting the above technical solution, the model file loading method provided in the embodiment of this specification establishes a mapping relationship between the model file extension and the loading policy class through a policy registry. Different types of model files have their own corresponding loading policy classes, thereby realizing the decoupling of the loading logic of different types of model files. When it is necessary to modify the loading logic of one of the model files, only the corresponding loading policy class needs to be modified without affecting the loading logic of other types of models, thus avoiding the mixing of code logic. And when it is necessary to load a new model file type, only a new mapping relationship and a corresponding loading policy class need to be added without modifying the existing main code, thereby improving the scalability and maintainability of the system. By introducing a node cache and a cache mapping table, different types of model files can be stored in different target node caches, thus realizing resource isolation and reducing the mutual influence between the loading of different model files, thereby improving the stability of model file loading. At the same time, the node cache is checked before loading the model file. If the model file object exists in the node cache, it is directly returned, thus avoiding repeated loading and improving loading efficiency.

[0092] A model engine is a system used to execute algorithm models. It is usually used to generate related model files based on different algorithms, and dynamically load model files for service-based call execution. Model files are the product of deep learning model training, and are generated based on but not limited to machine learning and deep learning algorithms such as xgboost, lightgbm, transformer, lr, torch, etc. Common model file formats include .pkl, .pmml, .so, etc. In some high-demand scenarios, model files also include files for large models. Model file loading refers to the process of reading a trained deep learning model from local storage or remote storage (such as file system, cloud, database, etc.) into computer memory and preparing it for inference or continued training. The process of executing online models by existing model engines is as follows: Figure 2 As shown, the model is first trained using offline data. After training, it is packaged into a model file. The model engine then loads the model file to generate an online real-time model. Finally, the server calls the input model variables to perform model calculations. The model file loading process is a high I / O read and write scenario, while model execution is a high CPU computing scenario. To improve the efficiency of model execution, it is necessary to isolate the model file loading and calculation processes. The key to isolating the loading and calculation processes is to ensure that the loading of different types of model files does not affect each other. In other words, the loading of different types of models needs to be decoupled. Therefore, different types of model files can be stored in different node caches to achieve resource isolation, reduce the mutual impact between the loading of different model files, and thus improve the stability of model file loading.

[0093] In some embodiments of this specification, the model file is loaded using the strategy mode molecule class, and the loading process is as follows: Figure 3 As shown, first, a FileStrategy interface is defined. This interface can be thought of as a contract, specifying the methods that all model file loading strategy classes must implement. This allows the FileContext class to call different loading strategy classes in a unified manner. For example, the FileStrategy interface defines a load method for loading model files. Next, a corresponding loading strategy class is created for each model file type. Each loading strategy class encapsulates the loading logic for that specific model file type. For example, for .py model files, a PyFileStrategy class can be created, and for .pkl model files, a PklFileStrategy class can be created. Each class implements the load method of the FileStrategy interface and contains the specific loading logic. Then, categorized loading is performed through the FileContext class. The FileContext class maintains a strategy registry that maps different model file extensions to loading strategy classes. When receiving a model file loading request, the FileContext class parses the model file extension in the request, searches the strategy registry for the corresponding loading strategy class based on the model file extension, and finally calls the load method of that class to load the model file. When the loading strategy class corresponding to the model file extension does not exist in the strategy registry, a corresponding loading strategy class is created for the model file extension and the FileStrategy interface is implemented. At the same time, the mapping relationship between the loading strategy class and the model file extension is added to the strategy registry. There is no need to modify the existing loading strategy class, thereby achieving isolation and decoupling of the loading logic and improving the scalability and maintainability of the model engine.

[0094] In some embodiments of this specification, the loading strategy class encapsulates the loading logic of each model file. For example, for .pkl files, this file is a binary file format used in Python to serialize and deserialize objects. Simply put, it can convert objects in Python (such as lists, dictionaries, custom classes, etc.) into a format that can be stored or transmitted. When needed, these serialized objects can be deserialized back to their original form for continued use by the program. In some embodiments of this specification, the file is loaded using: model = pickle.load (file_path, encoding = "utf-8"), and it is loaded into the node cache using: cacheManger.put (model_name, model). For .so files, this file is a shared object file, commonly found in Unix and Linux systems. It is similar to the ".dll" file in the Windows system, which is a dynamic link library file when the program is running. .so files are widely used in various software development and system management. They allow developers to write efficient and maintainable code while reducing the use of storage space and improving the scalability of the system. In some embodiments of this specification, the following statement is used to load .so files:

[0095] module_spec=importlib.util.spec_from_file_location(file_name,local_model_path);

[0096] module=importlib.util.module_from_spec(module_spec);

[0097] module_spec.loader.exec_module(module).

[0098] In some embodiments of this specification, node memory is used as a cache during the loading process, leveraging its high read and write performance to accelerate model loading and avoid resource competition through physical isolation, thereby improving model file loading efficiency. The target node can be an independent computing resource in a distributed system, such as a server, a virtual machine, or a container. Each node has independent hardware resources, including CPU, memory, disk, etc. Each node has an independent data storage area for storing loaded model files. The role of the node cache is to utilize the high read and write performance of the node memory to avoid repeatedly loading the same model file, thereby improving the efficiency of model file usage. When a model file needs to be loaded, it first checks whether the model file already exists in the node's cache. If it does, it is directly retrieved from the cache without having to reload it from disk or other storage devices, which can significantly reduce loading time. Because the memory of each node is physically isolated, the cache of each node is also independent. Models stored in the cache of one node do not affect the caches of other nodes. This can better utilize the resources of each node and avoid data interference between different nodes.

[0099] In some embodiments of this specification, reference is made to Figure 4 , the determining the corresponding target node according to the model file extension includes:

[0100] Determining a target node group according to the model file extension and a pre-built node mapping table, wherein the node mapping table includes a mapping relationship between the model file extension and the node group;

[0101] A polling strategy is adopted to select a node from the target node group as the target node of the model file.

[0102] It is understandable that, considering that different model file types have different demands for hardware resources, the model file can be routed to the most appropriate node group by a pre-configured node mapping table, thereby avoiding waste of resources. In this specification embodiment, the node mapping table allocates appropriate node groups according to the model file type. For example, .py files have high memory requirements, so it is possible to define that .py files correspond to gpu_nodes groups (containing 3 GPU nodes), and .pkl files correspond to cpu_nodes groups (containing 5 CPU nodes). Then, when loading .py files, .py files will be assigned to gpu_nodes groups. After determining the target node group, it is necessary to select a node from the target node group as its target node. In some embodiments of this specification, a polling strategy is used to determine the target node of the model file. Specifically, each node group maintains a pointer, which initially points to the first node. Each time a request is made, the pointer is incremented and circulated to ensure that all nodes are fairly accessed. For example, the gpu_nodes group includes three GPU nodes. When a node allocation request is received for the first time, the first one is selected, the second one for the second time, and so on, and the cycle repeats. In other embodiments, factors such as the busyness, load, storage capacity and performance indicators of the node can also be comprehensively considered to select a suitable node for the model file to avoid the loading of the model file being affected by excessive load or insufficient performance of the node.

[0103] In some embodiments of this specification, in order to avoid the problem of model file caching failure due to insufficient node storage space, such as Figure 5 As shown, before storing the model file object in the cache of the target node, the method further includes:

[0104] S501: Calculate the size of the model file object;

[0105] S502: Determine whether the storage space of the target node is larger than the size of the model file object according to the size of the model file object;

[0106] S503: If yes, store the model file object in the cache of the target node;

[0107] S504: If not, the model file object in the cache of the target node is cleaned up according to a preset cache elimination strategy until the storage space in the cache is larger than the size of the model file object, and the model file object is stored in the cache of the target node.

[0108] It's understood that the model file object size refers to the amount of storage space occupied by the object in memory. Calculating this size can help determine whether the target node's cache has sufficient space to accommodate the model file object. The size calculation method may vary for different types of model file objects, but generally, the number of bytes occupied is determined based on the object's data structure and stored content. For example, for a neural network model file object, the model file object size is calculated by counting the total number of bytes occupied by its weight parameters, structural information, and other information. After determining the model file object size, it needs to be compared with the remaining storage space in the target node's cache. The target node's cache typically has a fixed total capacity and records the currently used and remaining free space in real time. By comparing the model file object size with the remaining storage space, it can be determined whether there is sufficient space to store the object. If the target node's remaining storage space is greater than the model file object size, it indicates that there is sufficient space to store the object. The model file object is then stored in the target node's cache, and the cache usage status is updated, increasing the used space and decreasing the remaining free space. Simultaneously, the cache mapping table is updated to associate the model file object's path with the newly allocated node cache address. If the remaining storage space of the target node is smaller than the size of the model file object, it means that the current cache space is insufficient and the object cannot be directly stored. At this time, the model file object in the cache of the target node needs to be cleaned up according to the preset cache elimination strategy.

[0109] In some embodiments of this specification, reference is made to Figure 6 , the cleaning of the model file objects in the cache of the target node according to the preset cache elimination strategy includes:

[0110] S601: Obtain cache information of each model file object in the cache of the target node, wherein the cache information includes the latest call time, call frequency, loading duration, and memory usage.

[0111] It is understandable that in order to reasonably decide which model file objects should be removed from the cache, it is first necessary to collect the relevant cache information of each model file object, where the latest call time is the time point when the model file object was last used, which can be used to calculate how long the model object has been idle. The call frequency refers to the number of times the model file object is called within a certain period of time, reflecting how frequently it is used. The loading time is the time it takes to load the model file object, which is usually related to factors such as the complexity of the model and the file size. The memory usage size indicates the amount of memory space occupied by the model file object in the cache.

[0112] S602: Calculate the idle time of each model file object according to the difference between the latest call time of each model file object and the current time.

[0113] By subtracting the latest call time of each model file object from the current time, we can get the idle time of the object. The idle time is an important indicator to measure whether the model file object has been frequently used recently.

[0114] S603: Calculate the calling activity of each model file object according to the idle time and calling frequency of each model file object.

[0115] The call activity comprehensively considers the idle time and call frequency of the model file object. Generally speaking, the shorter the idle time and the higher the call frequency, the higher the call activity. In the embodiment of this specification, the call activity can be calculated by a weighted summation method. For example, the idle level is first determined based on the calculated idle time and the preset level threshold. For example, if the idle time is 3 days, the idle level can be 3. The smaller the idle level, the higher the call activity. The idle level is then normalized, and then a weight is set for the idle level and call frequency respectively. The inverse of the idle level and the call frequency are added according to the set weight to obtain the call activity value. In other embodiments, the call activity can be calculated according to specific business needs and scenarios, and this specification does not make specific restrictions on this.

[0116] S604: Calculate the loading cost of each model file object according to the loading time and memory usage of each model file object.

[0117] The loading cost combines the loading time and memory usage of the model file object. The longer the loading time and the larger the memory usage, the higher the loading cost. In some embodiments of this specification, taking into account the problem that the loading time and the memory usage are different in dimension, in order to eliminate the influence of dimension, the loading time and the memory usage are first normalized, and then the loading cost is calculated by weighted summation. The normalized loading time can be calculated by (loading time-minimum loading time) / (maximum loading time-minimum loading time), and the normalized memory usage can be calculated by (memory usage-minimum memory usage) / (maximum memory usage-minimum memory usage). In other embodiments, the loading cost can also be calculated based on the node load and node performance.

[0118] S605: Calculate the cleanup priority of each model file object according to the loading cost and the calling activity.

[0119] S606: Remove the model file objects whose cleaning priority is greater than a preset threshold from the cache of the target node.

[0120] The cleaning priority is a value obtained by comprehensively considering the loading cost and the calling activity. Model file objects with high loading cost and low calling activity have higher cleaning priority. In some embodiments of this specification, the cleaning priority can be calculated by dividing the loading cost by the calling activity. The higher the cleaning priority value, the more priority the model file object should be removed from the cache. In some embodiments of this specification, when the cleaning priority of a model file object is greater than a preset threshold, it is removed from the cache of the target node. In other embodiments, if the target node cache storage space is still smaller than the model file object size after the model file object with a cleaning priority greater than the preset threshold is removed, the model file objects in the node can be removed in descending order of the cleaning priority until the target node cache storage space is larger than the model file object size. This ensures that when the cache space is insufficient, those model file objects that have less impact on system performance and have less recent usage frequency are cleared first, thereby releasing sufficient cache space to store new model file objects, thereby maximizing cache utilization and avoiding unnecessary waste of resources.

[0121] In some embodiments of this specification, after the model file object is stored in the target node cache, in order to facilitate the subsequent re-loading of the file, the model file can be quickly found from the cache, it is necessary to generate a corresponding node cache address for the newly cached model file object, and write the mapping relationship between the path of the newly cached model file object and the node cache address into the cache mapping table, refer to Figure 7 As shown, the method further includes:

[0122] S701: Determine a target storage block from the cache space of the target node according to the size of the model file object.

[0123] It is understandable that the cache space of the target node is usually divided into multiple storage blocks, each of which has a fixed or variable size. When a model file object needs to be stored, a storage block that is large enough to store the model file object is found in the cache space of the target node according to the size of the model file object. In some embodiments of this specification, when determining the target storage block, the search can be started from the starting position of the cache space, and when the first storage block that is large enough to accommodate the model file object is found, it can be determined as the target storage block. In other embodiments, the entire cache space can be traversed, and the storage block with a size closest to (but not less than) the size of the model file object can be used as the target storage block, which can minimize memory fragmentation and improve the utilization of the node storage space.

[0124] S702: Generate the memory address of the target storage block according to the model file path.

[0125] The model file path is a string that describes the logical or physical location of the file in the storage system. It is a unique identifier of the file in the storage system and is used to accurately locate and access the model file. It usually includes the following parts: storage protocol: such as file: / / (local file system), s3: / / (AWS S3 storage bucket), hdfs: / / (Hadoop distributed file system); directory hierarchy: the folder structure where the file is located, such as / models / classification / ; and file name: the complete file name including the extension, such as resnet50.pth. A complete model file path is: file: / / mnt / models / resnet50.pth, indicating that it is the resnet50.pth file stored in the local / mnt / models / directory. Since the model file path contains the key information of the model file and uniquely identifies the model file object, the memory address of the target storage block can be generated based on the model file path. In some embodiments of this specification, the model file path is first hashed, and then the calculated hash value is mapped to the physical address range of the target storage block by modulo operation. For example, if the memory space of the target node is 1GB and its physical address range is 0 to 0x3FFFFFFF, and the hash value calculated according to the model file path is 0x89ABCDEF, then the memory address of the target storage block is 0x89ABCDEF%0x40000000=0x49ABCDEF.

[0126] S703: Generate a node cache address of the model file object according to the target node identifier and the memory address.

[0127] The target node identifier uniquely identifies a target node, while the memory address determines the specific location of the model file object within the node cache space. Combining the two can generate the node cache address of the model file object. In some embodiments of this specification, the target node identifier and the memory address can be concatenated to obtain the node cache address of the model file object. For example, if the node identifier is Node1 and the memory address is 0x49ABCDEF, the node cache address can be represented as Node1:0x49ABCDEF.

[0128] S704: Form a mapping relationship between the model file path and the node cache address, and write the mapping relationship into the cache mapping table.

[0129] The cache mapping table contains a mapping between the node cache address and the model file path. This allows the model file path to be used to quickly locate the model file object in the cache when it is subsequently accessed. In some embodiments of this specification, the cache mapping table may be implemented using a data structure such as a hash table or dictionary, with the model file path as the key and the node cache address as the value, storing the mapping in the cache mapping table.

[0130] In some embodiments of the present specification, when using the above method to load model files in batches, it is necessary to check the available space of the node memory in advance to ensure that it can accommodate all the model file objects to be loaded. If the total size of all the model files to be loaded is less than the available memory, they are loaded directly, otherwise, batch loading or memory cleaning is triggered. In other embodiments, when the number of loaded files is greater than the set maximum number of files in a single batch, batch loading can be used to reduce memory pressure. During the model file loading process, model files that failed to load need to be retried multiple times to improve reliability. In some embodiments of the present specification, model files that failed to load can be retried a set number of times according to a preset interval duration, where the interval duration can increase exponentially to avoid the problem of unstable loading caused by repeated requests in a short period of time.

[0131] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, storage, and display, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. The acquisition, storage, use, and processing of data in the technical solutions described in the embodiments of this application comply with relevant regulations.

[0132] Based on the above-mentioned model file loading method, the embodiment of this specification also provides a model file loading device. The device may include a system (including a distributed system), software (application), module, component, server, client, etc. using the method described in the embodiment of this specification and combined with the necessary implementation hardware. Based on the same innovative concept, the device in one or more embodiments provided in the embodiment of this specification is as described in the following embodiments. Since the implementation scheme and method for solving the problem of the device are similar, the implementation of the specific device in the embodiment of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0133] Specifically, Figure 8 This is a schematic diagram of the module structure of an embodiment of a model file loading device provided in the embodiment of this specification, referring to Figure 8 As shown, a model file loading device provided in an embodiment of this specification includes:

[0134] Receiving module 801, used to receive a model file loading request sent by a caller;

[0135] The parsing module 802 is used to parse the loading request to obtain the model file extension and the model file path;

[0136] A first determining module 803 is configured to determine a corresponding target node according to the model file extension;

[0137] A determination module 804 is configured to determine whether a model file object exists in the cache of the target node according to the model file path and a pre-built cache mapping table, wherein the cache mapping table includes a mapping relationship between a node cache address and a model file path;

[0138] The return module 805 is configured to return the model file object to the caller when the model file object exists in the cache of the target node;

[0139] A second determining module 806 is configured to determine a corresponding loading policy class from a pre-built policy registry according to the model file extension when the model file object does not exist in the cache of the target node, wherein the policy registry contains a mapping relationship between the model file extension and the loading policy class;

[0140] The loading module 807 is used to load the model file object according to the model file path using the loading strategy class, return the model file object to the caller, and store the model file object in the cache of the target node.

[0141] The beneficial effects achieved by the device provided in the embodiments of this specification are consistent with the beneficial effects achieved by the above-mentioned method and will not be repeated here.

[0142] Reference Figure 9As shown, based on the above-described model file loading method, an embodiment of this specification further provides a computer device 902, wherein the above-described method is executed on the computer device 902. The computer device 902 may include one or more processors 904, such as one or more central processing units (CPUs), each of which may implement one or more hardware threads. The computer device 902 may also include any memory 906 for storing any type of information such as code, settings, data, etc. For example, without limitation, the memory 906 may include any one or more combinations of the following: any type of RAM, any type of ROM, a flash memory device, a hard disk, an optical disk, etc. More generally, any memory may use any technology to store information. Furthermore, any memory may provide volatile or non-volatile retention of information. Furthermore, any memory may represent a fixed or removable component of the computer device 902. In one embodiment, when the processor 904 executes associated instructions stored in any memory or combination of memories, the computer device 902 may perform any operation of the associated instructions. The computer device 902 also includes one or more drive mechanisms 908 for interacting with any storage, such as a hard disk drive mechanism, an optical disk drive mechanism, and the like.

[0143] The computer device 902 may also include an input / output module 910 (I / O) for receiving various inputs (via input devices 912) and for providing various outputs (via output devices 914). A specific output mechanism may include a presentation device 916 and an associated graphical user interface (GUI) 918. In other embodiments, the input / output module 910 (I / O), input devices 912, and output devices 914 may not be included, and the computer device 902 may simply be a computer device in a network. The computer device 902 may also include one or more network interfaces 920 for exchanging data with other devices via one or more communication links 922. One or more communication buses 924 couple the components described above together.

[0144] The communication link 922 may be implemented in any manner, for example, via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 922 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0145] Corresponding to Figures 1 to 7 In addition to the method shown, an embodiment of this specification also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are executed.

[0146] The embodiment of this specification also provides a computer-readable instruction, wherein when the processor executes the instruction, the program therein causes the processor to execute the following Figures 1 to 7 The method shown.

[0147] The embodiment of this specification also provides a computer program product, including at least one instruction or at least one program, which is loaded and executed by a processor to implement the following Figures 1 to 7 The method shown.

[0148] It should be understood that in the various embodiments of this specification, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this specification.

[0149] It should also be understood that in the embodiments of this specification, the term "and / or" is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. Furthermore, the character " / " in this specification generally indicates that the associated objects are in an "or" relationship.

[0150] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this specification can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this specification.

[0151] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0152] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be an electrical, mechanical or other form of connection.

[0153] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this specification.

[0154] In addition, the functional units in the various embodiments of this specification may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0155] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0156] Specific embodiments are used in this specification to illustrate the principles and implementation methods of this specification. The description of the above embodiments is only used to help understand the methods and core ideas of this specification. At the same time, for those skilled in the art, based on the ideas of this specification, there will be changes in the specific implementation methods and application scope. In summary, the contents of this specification should not be understood as limiting this specification.

Claims

1. A model file loading method, characterized in that: The method comprises: Receive the model file loading request sent by the caller; Parsing the loading request to obtain the model file extension and model file path; Determine the corresponding target node according to the model file extension; Determining whether a model file object exists in the cache of the target node according to the model file path and a pre-built cache mapping table, wherein the cache mapping table includes a mapping relationship between a node cache address and a model file path; If so, the model file object is returned to the caller; If not, determining the corresponding loading policy class from a pre-built policy registry according to the model file extension, wherein the policy registry contains a mapping relationship between the model file extension and the loading policy class; The model file object is loaded according to the model file path using the loading strategy class, the model file object is returned to the caller, and the model file object is stored in the cache of the target node.

2. The method according to claim 1, characterized in that The determining the corresponding target node according to the model file extension includes: Determining a target node group according to the model file extension and a pre-built node mapping table, wherein the node mapping table includes a mapping relationship between the model file extension and the node group; A polling strategy is adopted to select a node from the target node group as the target node of the model file.

3. The method according to claim 1, characterized in that Before storing the model file object in the cache of the target node, the method further includes: Calculate the model file object size; Determining whether the storage space of the target node is larger than the size of the model file object according to the size of the model file object; If yes, storing the model file object in the cache of the target node; If not, the model file object in the cache of the target node is cleaned up according to the preset cache elimination strategy until the storage space in the cache is larger than the size of the model file object, and the model file object is stored in the cache of the target node.

4. The method according to claim 3, characterized in that The method further comprises: Determining a target storage block from the cache space of the target node according to the size of the model file object; Generate a memory address of the target storage block according to the model file path; Generate a node cache address of the model file object according to the target node identifier and the memory address; A mapping relationship is formed between the model file path and the node cache address, and the mapping relationship is written into the cache mapping table.

5. The method according to claim 3, characterized in that The step of clearing the model file objects in the cache of the target node according to the preset cache elimination strategy includes: Obtain cache information of each model file object in the cache of the target node, wherein the cache information includes the latest call time, call frequency, loading time and memory usage; The idle time of each model file object is calculated based on the difference between the latest call time of each model file object and the current time; Calculate the call activity of each model file object according to the idle time and call frequency of each model file object; Calculate the loading cost of each model file object based on the loading time and memory usage of each model file object; Calculate the cleanup priority of each model file object according to the loading cost and the call activity; The model file objects whose cleaning priority is greater than a preset threshold are removed from the cache of the target node.

6. The method according to claim 1, characterized in that The method further comprises: If the loading policy class corresponding to the model file extension does not exist in the policy registry, creating a corresponding loading policy class for the model file extension, the loading policy class including the loading logic of the model file extension; Add the mapping relationship between the loading strategy class and the model file extension to the strategy registry.

7. A model file loading device, characterized in that: The device comprises: The receiving module is used to receive the model file loading request sent by the caller; A parsing module, configured to parse the loading request to obtain a model file extension and a model file path; A first determining module, configured to determine a corresponding target node according to the model file extension; a determination module, configured to determine whether a model file object exists in the cache of the target node according to the model file path and a pre-built cache mapping table, wherein the cache mapping table includes a mapping relationship between a node cache address and a model file path; A return module, configured to return the model file object to the caller when the model file object exists in the cache of the target node; a second determining module configured to determine, when the model file object does not exist in the cache of the target node, a corresponding loading policy class from a pre-built policy registry according to the model file extension, wherein the policy registry contains a mapping relationship between the model file extension and the loading policy class; A loading module is used to load the model file object according to the model file path using the loading strategy class, return the model file object to the caller, and store the model file object in the cache of the target node.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The method comprises at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded and executed by a processor to implement the method according to any one of claims 1 to 6.

Images