Dependence-aware offline model migration method and apparatus

Abstract

The present application relates to the technical field of artificial intelligence, and provides a large model offline migration method and device based on dependency perception. The method comprises: in the process of loading a model file, capturing runtime dependency behaviors in real time to generate runtime dependency behavior logs; performing dependency component pruning based on the runtime dependency behavior logs to generate a minimum necessary dependency set; packaging the model file and the minimum necessary dependency set to obtain an offline installation package, and performing offline migration based on the offline installation package. The method provided by the present application generates logs by capturing runtime dependency behaviors when loading a model, prunes a minimum necessary dependency set according to the logs, and finally packages the model file and the minimum necessary dependency set together into an offline installation package for migration, thereby overcoming the defect that traditional manual migration is prone to missing components, achieving accurate dependency extraction and one-time packaging for deployment on any offline device, and significantly improving the success rate of model offline migration and the deployment implementation efficiency.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for offline migration of large models based on dependency awareness. Background Technology

[0002] Ollama is a widely used open-source local inference framework for Large Language Models (LLMs). It allows users to quickly pull and run various mainstream large models, such as Llama, Mistral, and Gemma, via command line. Its typical workflow relies on the internet to download the model manifest and model data blocks from the official repository and build the runtime environment locally. It's important to note that in confidential or offline environments, Ollama requires offline model loading.

[0003] However, since Ollam uses a runtime written in Go, its dependencies include operating system libraries, Compute Unified Device Architecture (CUDA) drivers, quantization backends, Python bindings, etc., but it lacks a unified dependency description file, which makes it easy to miss key components when manually packaging, thus making it impossible for the existing offline deployment method to load the target model. Summary of the Invention

[0004] This invention provides a dependency-aware offline migration method and apparatus for large models, which solves the problem that in the prior art, when running a large language model framework in an offline environment, key components are easily missed due to manual packaging, resulting in a high failure rate for offline deployment.

[0005] This invention provides a dependency-aware offline migration method for large models, comprising: Based on the summary identifier of the target model, extract the model file corresponding to the target model; During the loading of the model file, runtime dependency behavior is captured in real time to generate runtime dependency behavior logs; Based on the runtime dependency behavior log, dependency components are trimmed to generate a minimum necessary dependency set; The model file and the minimum necessary dependency set are packaged to obtain an offline installation package, and offline migration is performed based on the offline installation package.

[0006] According to the present invention, a method for offline migration of large models based on dependency awareness, wherein the dependency component pruning based on the runtime dependency behavior log to generate a minimum necessary dependency set includes: Extract the call frequency and execution path coverage of each dependent component from the runtime dependency behavior log; Based on the call frequency of each dependent component and the execution path coverage, the necessity score of each dependent component is calculated. Compare the necessity scores of each dependent component with a preset threshold, remove dependent components whose necessity scores are lower than the preset threshold, and use the remaining dependent components as the minimum necessary dependency set.

[0007] According to a dependency-aware offline migration method for large models provided by the present invention, the method involves capturing runtime dependency behavior in real time during the loading of the model file to generate a runtime dependency behavior log, including: Inject a probe program to intercept and record system call behavior during the loading process of the model file; Configure a dynamic link library preloading mechanism to intercept and record the dynamic link library loading behavior during the loading process of the model file; Record the access paths to environment variables during the loading process of the model file; The system call behavior, the dynamic link library loading behavior, and the access path are combined to generate the runtime dependency behavior log.

[0008] According to the dependency-aware offline migration method for large models provided by the present invention, before extracting the model file corresponding to the target model based on the target model's summary identifier, the method further includes: Parse the model manifest file corresponding to the target model in the model directory of the large model execution framework; Extract the hash value of the corresponding data block from the model manifest file, and use the hash value as the digest identifier.

[0009] According to the present invention, a method for offline migration of large models based on dependency awareness is provided, wherein packaging the model file and the minimum necessary dependency set to obtain an offline installation package includes: Obtain multiple pre-compiled inference backends adapted to different hardware environments, as well as hardware adaptive scripts; The model file, the minimum necessary dependency set, the multiple pre-compiled inference backends, and the hardware adaptive script are used as data to be packaged. The data to be packaged is compressed to generate the offline installation package.

[0010] According to the present invention, a dependency-aware offline migration method for large models is provided, wherein performing offline migration based on the offline installation package includes: Unzip the offline installation package on the offline device; The hardware environment information of the offline device is detected; the hardware environment information includes at least one of the following: operating system type, processor architecture, and accelerator card type. By executing the hardware adaptive script, a target inference backend that matches the offline device is selected from the plurality of pre-compiled inference backends based on the hardware environment information; Based on the target inference backend and the minimum necessary dependency set, the model file is loaded on the offline device to start the model service.

[0011] According to the dependency-aware offline migration method for large models provided by the present invention, the step of decompressing the offline installation package on the offline device includes: In response to a target model loading command, a subprocess independent of the current model service is created on the offline device; The offline installation package is asynchronously decompressed in the subprocess.

[0012] According to a dependency-aware offline migration method for large models provided by the present invention, the data to be packaged further includes an integrity check code; the integrity check code is calculated based on the model file, the minimum necessary dependency set, the multiple pre-compiled inference backends, and the hardware adaptive script. The process of unzipping the offline installation package on the offline device further includes: Extract the integrity check code from the data to be packaged; Based on the integrity check code, a consistency check is performed on the restored data obtained by decompressing the offline installation package; If the consistency check passes, the hardware environment information of the offline device is detected.

[0013] According to a dependency-aware offline migration method for large models provided by the present invention, the step of compressing the data to be packaged to generate the offline installation package includes: The data to be packaged is divided into multiple data blocks according to a preset size, and compression is performed on each data block to obtain each compressed data block. Extract the position offset information and logical mapping relationship of each compressed data block, and construct a block index table; The block index table is combined with each compressed data block to generate the offline installation package.

[0014] The present invention also provides a dependency-aware large model offline migration device, comprising: The model file extraction unit extracts the model file corresponding to the target model based on the summary identifier of the target model; The dependency capture unit captures runtime dependency behavior in real time during the loading of the model file to generate a runtime dependency behavior log. The trimming unit trims dependent components based on the runtime dependency behavior log to generate a minimum necessary dependency set. The migration unit packages the model file and the minimum necessary dependency set to obtain an offline installation package, and performs offline migration based on the offline installation package.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the dependency-aware large model offline migration method as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dependency-aware large model offline migration method as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the dependency-aware large model offline migration method as described above.

[0018] The dependency-aware large model offline migration method and apparatus provided by this invention extracts model files based on the target model's summary identifier, dynamically captures runtime dependency behavior to generate logs during model loading, and then generates a minimum necessary dependency set based on the logs. Finally, the model files and the minimum necessary dependency set are packaged together into an offline installation package for migration. This overcomes the shortcomings of traditional manual migration, which is prone to missing components, and achieves accurate dependency extraction and deployment on any offline device with a single package, significantly improving the success rate and deployment efficiency of offline model migration. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the offline migration method for large models based on dependency awareness provided by this invention. Figure 2 This is a flowchart illustrating the dependency awareness and pruning method provided by the present invention; Figure 3 This is a schematic diagram of the offline migration process of the target model provided by the present invention; Figure 4 This is a schematic diagram of the internal structure of the offline installation package provided by the present invention; Figure 5 This is a schematic diagram of the architecture of the dependency-aware large model offline migration system provided by the present invention; Figure 6 This is a schematic diagram of the structure of the dependency-aware large model offline migration device provided by the present invention; Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] It should be understood that, in the description of the embodiments of the present invention, "multiple" means two or more, unless otherwise explicitly specified. Furthermore, all actions involving the acquisition of signals, information, or data in this application are performed in accordance with the applicable data protection laws and regulations of the locality and with permission granted by the owner of the respective device.

[0023] To address the aforementioned issues, this invention provides a dependency-aware offline migration method for large models, enabling automated model packaging and efficient offline deployment. Figure 1 This is a flowchart illustrating the dependency-aware offline migration method for large models provided by this invention, as shown below. Figure 1 As shown, the method includes: Step 110: Extract the model file corresponding to the target model based on the summary identifier of the target model.

[0024] The target model here refers to the open-source large language model specified by the user that needs to be migrated offline. In a practical scenario, the digest identifier here refers to the digital fingerprint or hash feature code used to uniquely identify a specific data block of the large model.

[0025] Specifically, the summary identifier of the target model can be obtained first, and then the model file corresponding to the target model can be accurately located in the local storage path or file system based on the summary identifier. This is the physical file that actually carries the model weights, network structure and pre-trained parameters, such as the model file encapsulated in a general weight format.

[0026] It should be understood that extracting model files through summary tags can ensure the accuracy of the extraction process and avoid data extraction errors caused by models with the same name or version conflicts.

[0027] Step 120: During the loading of the model file, capture runtime dependency behavior in real time to generate a runtime dependency behavior log.

[0028] Here, runtime dependency behavior refers to the low-level resource requests that the program makes to the operating system during model startup and loading and inference service initialization, which broadly covers operations such as system calls, dynamic link library loading, and access to environment variables.

[0029] Specifically, the process of loading model files can be triggered on the source network device. During this loading process, all runtime dependency behaviors that occur can be captured in real time by attaching underlying monitoring probes or interception mechanisms. Then, these captured underlying actions and resource requests are recorded, summarized, and formatted to generate a runtime dependency behavior log.

[0030] It should be noted that by dynamically capturing dependencies during the actual loading process, implicit dependencies that are difficult to identify by static code scanning can be accurately discovered, effectively solving the problem that the lack of transparency in framework component dependencies in existing technologies makes it easy to miss them during manual packaging.

[0031] Step 130: Based on the runtime dependency behavior log, perform dependency component pruning to generate a minimum necessary dependency set.

[0032] Here, the minimum necessary dependency set refers to the minimum combination of components necessary to maintain the target model's normal operation on the target hardware architecture, excluding redundant library files or driver components that are not actually called due to differences in system environment.

[0033] Specifically, the runtime dependency behavior logs generated earlier can be analyzed to identify which dependent components actually participated in the model loading and inference process, and redundant optional components that were not actually called can be removed or filtered out. Thus, the core component set retained after intelligent pruning is the minimum necessary dependency set. For example, if the logs show that the model only used CPU-related libraries for a specific architecture during loading, redundant dependencies related to the graphics processor can be pruned. It's important to understand that intelligent pruning based on real runtime logs can significantly reduce the size of the subsequently packaged data packets, greatly reducing storage costs and network distribution overhead during offline transmission.

[0034] Step 140: Package the model file and the minimum necessary dependency set to obtain an offline installation package, and perform offline migration based on the offline installation package.

[0035] Here, the line installation package refers to an independent distribution file that encapsulates the model data and its supporting underlying runtime environment into one unit, which can complete the environment restoration locally without relying on the external Internet.

[0036] Specifically, the extracted model files and the minimal set of necessary dependencies generated through trimming can be physically merged, and then compressed together using a compression algorithm to package them into a single offline installation package. Subsequently, the offline installation package is copied, transferred, or deployed to the target offline device, where it is decompressed and the environment is built, thus executing and completing the offline migration. Here, packaging the model files and their corresponding necessary runtime dependencies into independent units for unified migration achieves a strong binding delivery of the model and the execution environment, ensuring a very high success rate and cross-platform compatibility in offline environments lacking network connectivity.

[0037] The method provided in this invention extracts model files based on the summary identifier of the target model, dynamically captures runtime dependency behavior to generate logs when the model is loaded, generates a minimum necessary dependency set based on the logs, and finally packages the model files and the minimum necessary dependency set together into an offline installation package for migration. This overcomes the shortcomings of traditional manual migration, which is prone to missing components, and achieves accurate dependency extraction and deployment on any offline device with a single package, significantly improving the success rate and deployment efficiency of offline model migration.

[0038] Based on any of the above embodiments, step 130 includes: Extract the call frequency and execution path coverage of each dependent component from the runtime dependency behavior log; Based on the call frequency of each dependent component and the execution path coverage, the necessity score of each dependent component is calculated. Compare the necessity scores of each dependent component with a preset threshold, remove dependent components whose necessity scores are lower than the preset threshold, and use the remaining dependent components as the minimum necessary dependency set.

[0039] In practice, the first step is to extract the call frequency and execution path coverage of each dependent component from the runtime dependency behavior log. The call frequency here refers to the total number of times a specific underlying component or library file is requested and triggered by the system during model loading and initialization. The execution path coverage refers to the proportion of the code logic branch actually executed within a specific component relative to its overall code architecture.

[0040] Specifically, the collected log files can be deeply analyzed, and statistical analysis algorithms can be used to extract the specific number of times each dependent component was called, while simultaneously analyzing tracking data to obtain the coverage ratio of the corresponding component's execution path. For example, the actual utilization depth of a specific graphics processor dynamic library on the current hardware node can be analyzed. It is important to understand that extracting these quantitative indicators can provide multi-dimensional data support for the subsequent objective evaluation of the true operational value of underlying components, avoiding biases caused by human experience-based judgments.

[0041] Then, based on the call frequency and execution path coverage of each dependent component, a necessity score is calculated for each dependent component. Here, the necessity score refers to a quantitative score that comprehensively evaluates the indispensability of a dependent component in maintaining the normal operation of the model.

[0042] Specifically, the obtained call frequency and execution path coverage can be used as input variables, and a pre-set weighted calculation model or multi-dimensional scoring algorithm can be used for comprehensive calculation to obtain a single numerical score for each underlying component.

[0043] Furthermore, the necessity scores of each dependent component are compared with a preset threshold. Dependent components with necessity scores below the preset threshold are eliminated, and the remaining dependent components are used as the minimum necessary dependency set. Here, the preset threshold refers to a baseline score line set in advance by the system to determine whether a component is a core dependency that must be retained or a redundant item that can be discarded.

[0044] Specifically, the calculated score of each component can be compared one by one with a set benchmark score. If some components are found to be below the benchmark score, such as redundant computing libraries that are not called at all in non-graphics processor paths, they are directly filtered out or deleted from the package list; while components with scores equal to or higher than the benchmark score are retained, and all retained components are merged to form the final streamlined component combination, so as to eliminate optional dependencies that are not actually used, and significantly reduce the space occupied by the subsequent installation package without affecting the normal operation of the program.

[0045] The method provided in this invention extracts the call frequency and execution path coverage of each dependent component and calculates the necessity score. Based on a preset threshold, it accurately eliminates dependent components, realizing quantitative evaluation and intelligent trimming. This eliminates the problem of redundant component accumulation caused by traditional static packaging, maximizes the reduction of offline package size, and greatly improves the efficiency of data distribution and deployment in offline environments.

[0046] Based on any of the above embodiments, step 120 includes: Inject a probe program to intercept and record system call behavior during the loading process of the model file; Configure a dynamic link library preloading mechanism to intercept and record the dynamic link library loading behavior during the loading process of the model file; Record the access paths to environment variables during the loading process of the model file; The system call behavior, the dynamic link library loading behavior, and the access path are combined to generate the runtime dependency behavior log.

[0047] In practical implementation, runtime dependency behaviors can be captured in real time through a hybrid monitoring mechanism. This includes injecting probes to intercept and record system call behaviors during model file loading. Here, a probe refers to a miniature monitoring code that can be non-intrusively mounted in the operating system kernel mode to listen for and capture specific program instructions in real time, such as an extended Berkeley packet filter, i.e., an eBPF kernel probe. System call behaviors refer to the low-level interactions between the user-mode runtime framework and the operating system kernel requesting higher-privilege services, such as file reading and memory allocation.

[0048] Specifically, probe code can be dynamically deployed to the operating system kernel of the current host machine the instant the target model is triggered to load. The probe code constantly monitors the kernel layer, and once it detects a system call request initiated by the target model's loading process, it immediately intercepts and records the specific system call type, timestamp, and return result. It's important to understand that monitoring and recording at the kernel level can penetrate high-level application layer encapsulation to obtain the most original, lowest-level operating system dependencies, ensuring that no hidden system-level component interactions are missed.

[0049] Additionally, a dynamic link library preloading mechanism can be configured to intercept and record dynamic link library loading behavior during the model file loading process. Here, the dynamic link library preloading mechanism refers to a user-space technique, such as the LD_PRELOAD mechanism, that forces the operating system to prioritize loading user-specified interceptor libraries before program startup, thereby proxying and monitoring all subsequent dynamic library loading requests. Dynamic link library loading behavior refers to the actions of the runtime framework mapping and calling external shared library files in memory, such as calling inference backends or driver libraries like libcuda.so and libggml.so.

[0050] Specifically, a custom interception and monitoring library can be pre-injected into the environment configuration for starting the model service process. When the model process attempts to load external shared dependencies, the interception library will take over the request, extract the specific name, physical path, and version number of the library to be loaded, and record these details. It is important to understand that by using pre-loading interception technology, critical dynamic library dependencies located in user space can be accurately captured, effectively overcoming the problem of poor cross-platform compatibility caused by the lack of supporting drivers or inference backends in existing offline migrations.

[0051] Furthermore, it can also record the access paths to environment variables during the model file loading process. Here, environment variables refer to global or local dynamic values ​​in the operating system used to specify program runtime environment configuration parameters, such as CUDA_VISIBLE_DEVICES for the computing graphics card.

[0052] Specifically, the aforementioned monitoring system can be used to specifically track actions related to the model process reading system environment variables, fully recording all environment variable keys queried by the model during initialization of the runtime environment, along with their corresponding access paths or return values. It's important to understand that tracking environment configuration parameters preserves the model's implicit configuration requirements on the source device, preventing startup crashes or initialization failures caused by missing configurations when deployed on the target offline device.

[0053] Finally, system call behavior, dynamic link library loading behavior, and access paths are merged to generate a runtime dependency behavior log. Specifically, system calls captured in kernel mode, shared library loading information intercepted in user mode, and environment configuration read records can be uniformly summarized. Through data cleaning and formatting transformation, this multi-dimensional fragmented information is integrated into a structured document, such as JSON format, thus forming a complete log file that presents the overall runtime situation of the program.

[0054] The method provided in this invention records system calls by injecting a probe program, configures a dynamic link library preloading mechanism to record library loading actions, and combines the recording of environment variable access paths. Finally, the three are merged to generate a unified log, constructing a hybrid monitoring mechanism covering kernel mode and user mode. This solves the black-box problem of opaque dependencies during the runtime of open-source large language models, and ensures extremely high accuracy and comprehensive coverage of dependency awareness.

[0055] In one embodiment, Figure 2 This is a flowchart illustrating the dependency-aware and pruning method provided by the present invention, as follows: Figure 2 As shown, the system first requires the user to input the target model name, such as llama3:8b. Then, it parses the manifest file in the model directory to obtain the model's summary identifier, i.e., it automatically parses manifests / library / <model>The directory is used to obtain the digest and summary identifier of the model. Next, the exact path is located to blobs / sha256- <digest>The path is determined, and the corresponding GGUF file, i.e., the model file, is extracted. Next, Ollam is launched and the model is loaded, with eBPF and LD_PRELOAD enabled simultaneously during loading to monitor runtime behavior. That is, the framework is started and the model is loaded in a sandbox, with extended Berkeley packet filters and dynamic link library preloading mechanisms enabled in real time to monitor all underlying runtime behaviors. Subsequently, a dependency call graph is constructed by fully recording system calls, library loading, and environment variable accesses. Then, intelligent pruning is performed: low-scoring redundant items are removed based on call frequency and path coverage, ultimately outputting the minimum necessary dependency set, successfully concluding the entire process.

[0056] Based on any of the above embodiments, the method further includes the following steps before step 110: Parse the model manifest file corresponding to the target model in the model directory of the large model execution framework; Extract the hash value of the corresponding data block from the model manifest file, and use the hash value as the digest identifier.

[0057] In practice, the first step is to parse the model manifest file corresponding to the target model within the model directory of the large model runtime framework. Here, the large model runtime framework refers to the local software system used to coordinate the download, construction, and inference of large language models, such as the Ollam framework. The model directory refers to the framework's dedicated folder in the local file system used to store various model metadata and configuration information, such as the manifests directory. The model manifest file refers to a metadata description file using a structured language to describe the model version, configuration structure, and components.

[0058] Specifically, when a user enters the name of the model to be migrated, such as "llama3:8b", the system automatically navigates to the directory path where the local framework stores metadata, finds the description file that exactly matches the name, and uses the parsing engine to read and parse its internal structure. It should be noted that this automatic parsing of the framework manifest's metadata is the first time that the user-friendly model identifier has been integrated with the underlying complex file system, achieving complete automation of the location process and avoiding path matching errors that are common in manual searches.

[0059] Then, the hash value of the corresponding data block is extracted from the model manifest file, and the hash value is used as a digest identifier. Here, the data block refers to the underlying storage unit, such as blobs, where the unstructured model weight data is actually stored.

[0060] Specifically, within the parsed list structure, key fields pointing to the core weights of the model, such as the `digest` field, can be located. From this field, the hash value of the unique digital fingerprint of the record data block can be completely extracted. This hash feature is then used directly as the digest identifier for retrieving physical files in subsequent steps. It's important to understand that using the encrypted hash value as a retrieval credential establishes an absolutely precise binding relationship between the logical model declaration and the underlying physical storage unit, ensuring uniqueness and tamper resistance when extracting model files from massive local data blocks.

[0061] The method provided in this invention parses the model manifest file in the model directory of the large model running framework and accurately extracts the hash value of the corresponding data block as a digest identifier. Combined with the framework's static model description mechanism, it achieves efficient and accurate mapping from model metadata to physical storage data, which greatly improves the accuracy of subsequent model file extraction.

[0062] Based on any of the above embodiments, in order to achieve efficient cross-platform compatibility and containerized encapsulation of the model, step 140 packages the model file and the minimum necessary dependency set to obtain an offline installation package, including: Obtain multiple pre-compiled inference backends adapted to different hardware environments, as well as hardware adaptive scripts; The model file, the minimum necessary dependency set, the multiple pre-compiled inference backends, and the hardware adaptive script are used as data to be packaged. The data to be packaged is compressed to generate the offline installation package.

[0063] In practice, the first step is to acquire multiple pre-compiled inference backends adapted to different hardware environments, as well as hardware adaptive scripts. Here, the pre-compiled inference backends refer to low-level inference execution binaries pre-compiled from source code for different CPU architectures or GPU ecosystems, such as low-level library packages suitable for different computing chips. The hardware adaptive scripts refer to automated program code that can automatically detect physical hardware parameters on the target deployment machine and dynamically schedule the corresponding execution engine.

[0064] Specifically, during the packaging process, not only are the dependencies on the current source device collected, but the system also proactively pulls various inference backend files adapted to pure CPUs, mainstream GPUs, and various domestic neural network accelerators from the resource repository, along with automatic adaptation scripts. It should be noted that pre-preparing multiple architecture backend components provides a resource pool guarantee for the execution engine to run models across heterogeneous hardware platforms, breaking down barriers caused by differences in underlying architecture.

[0065] Next, the model files, the minimum necessary dependency set, multiple pre-compiled inference backends, and the hardware adaptive scripts are used as the data to be packaged. The data to be packaged is compressed to generate an offline installation package.

[0066] Specifically, the formatted model weights extracted earlier, the trimmed and optimized basic dependency libraries, various pre-compiled backend components, and the adaptation scripts used for environment sniffing are combined together as a unified processing object. Then, a high-performance compression algorithm is called to compress this collection of hashed files into a single, high-density offline deployment file.

[0067] It should be noted that by binding and encapsulating multiple hardware engines with the model itself, the offline delivery of the model as a service concept is realized. This ensures that the generated package file can be successfully unpacked and run on any heterogeneous offline node, solving the pain point of strong coupling between the model and a single hardware.

[0068] The method provided in this invention obtains multiple pre-compiled inference backends and hardware adaptive scripts adapted to different hardware environments, and compresses them together with model files and necessary dependencies as packaged data. It realizes cross-platform containerization of models and their runtime environments by using self-contained binaries and adapter programs, which solves the problem of offline migration failure caused by differences in application binary interfaces or underlying computing interfaces, and greatly improves the general distribution capability of model files.

[0069] Based on any of the foregoing embodiments, in order to achieve highly automated environmental awareness and model mounting at the target end, for offline migration based on offline installation packages, the specific steps include: Unzip the offline installation package on the offline device; The hardware environment information of the offline device is detected; the hardware environment information includes at least one of the following: operating system type, processor architecture, and accelerator card type. By executing the hardware adaptive script, a target inference backend that matches the offline device is selected from the plurality of pre-compiled inference backends based on the hardware environment information; Based on the target inference backend and the minimum necessary dependency set, the model file is loaded on the offline device to start the model service.

[0070] In practice, the first step is to unzip the offline installation package on the offline device. Specifically, once the packaged single file is securely transferred to an offline device without a network connection, the deployment agent will trigger the unzipping process, restoring the internal weight data, component libraries, and automation scripts to a specified directory on the local operating system.

[0071] Next, the hardware environment information of the offline device is detected. This hardware environment information includes at least one of the following: operating system type, processor architecture, and accelerator card type. The hardware environment information here refers to the characteristic parameters of the underlying operating environment of the target physical machine.

[0072] Specifically, by deploying an agent program to read the system registry or kernel device tree of the current machine, the type of operating system, the central processing unit instruction set architecture, and whether it is equipped with a dedicated computing accelerator card from a specific manufacturer and its specific model can be accurately detected. This allows for a clear understanding of the underlying computing power attributes of the current computing node, thereby enabling precise matching and execution of the backend.

[0073] Then, by executing a hardware adaptive script, a target inference backend that matches the offline device is selected from multiple pre-compiled inference backends based on the hardware environment information. Here, the target inference backend refers to the program version that best fits the current physical machine's computing power ecosystem and can achieve the greatest acceleration performance among many alternative execution engines.

[0074] Specifically, the automated script code obtained from the decompression process can be launched, passing the recently detected machine environment parameters as input parameters. The script will use preset routing matching logic to select the uniquely matching underlying binary file from the decompressed candidate execution engine libraries. For example, when a specific acceleration chip is detected, the corresponding plugin version will be precisely mounted. It should be explained that by dynamically selecting the optimal execution engine, the multi-branch encapsulation of the source end and the single-path execution of the target end are cleverly combined and converged, greatly releasing the extreme inference performance under heterogeneous hardware.

[0075] Finally, based on the target inference backend and the minimum necessary dependency set, the model file is loaded on the offline device to start the model service. Specifically, after determining the core computing engine, the minimum dependency environment variables required by the engine are configured, and then the sandboxed service subprocess is officially started, successfully loading the hundreds of gigabytes of model weight files into memory, thereby exposing usable generative application programming interfaces. Understandably, using the matched native components to start inference avoids crashes caused by runtime environment mismatches, ensuring stable operation at the business level.

[0076] The method provided in this invention decompresses the installation package on an offline device and detects hardware environment information, including the operating system and accelerator card type. Then, it dynamically selects the matching target inference backend and loads the model file using a hardware adaptive script. This eliminates the tedious manual environment configuration and troubleshooting process, thereby reducing the deployment time of the migrated model service by orders of magnitude.

[0077] Based on any of the foregoing embodiments, in order to seamlessly switch or update the target model without affecting the production environment, the following measures are taken for unpacking the offline installation package on an offline device: In response to a target model loading command, a subprocess independent of the current model service is created on the offline device; The offline installation package is asynchronously decompressed in the subprocess.

[0078] First, in response to the target model loading command, a child process independent of the current model service is created on the offline device. The target model loading command here refers to the control signal sent by upper-layer business applications or operations personnel via inter-process communication mechanisms to trigger the loading of a new version or entirely new model entity. The child process refers to a newly allocated runtime entity in the operating system, containing its own independent memory space and file descriptors, and isolated from the existing main inference service. Specifically, when the offline device is providing existing model generation services, the deployment agent receives a signal to load a new model data package. At this time, the agent does not suspend or interrupt the running main service, but instead requests the operating system to create a completely new process space strictly isolated by a sandbox mechanism. Creating an isolated runtime space ensures that the operations of the new package during the preparation phase will not cause negative interference such as memory leaks or resource contention to the current high-concurrency inference service.

[0079] Finally, the offline installation package is asynchronously decompressed in a child process. Asynchronous decompression here refers to a data processing method that silently performs file release operations in the system background without causing the main control communication thread to be in a long-term blocked waiting state. Specifically, within the newly created isolated process, input / output bandwidth and CPU time slices are independently allocated to perform streaming decompression of the offline package. Before decompression and initialization are complete, the front-end application programming interface (API) routes still point to the old model. An atomic-level routing table switch is only performed when the new environment is available, allowing the target model to be called. It's important to understand that this background independent decompression strategy overcomes the limitation of traditional frameworks requiring a complete restart to load new weights, effectively eliminating the switching vacuum period.

[0080] The method provided in this invention creates a subprocess independent of the current service by responding to the target model loading command, and processes the new offline installation package in the subprocess using an asynchronous decompression method. This constructs a model hot loading architecture in an offline environment, realizing the smooth replacement and seamless switching of old and new model resources, and ensuring the high availability and business continuity of government and enterprise core businesses during model iteration.

[0081] In one embodiment, Figure 3 This is a schematic diagram of the offline migration process of the target model provided by the present invention, such as... Figure 3 As shown, when an administrator needs to update the underlying weights or switch business models, they first send a loading command containing the new model path to the lightweight offline deployment agent via a unified inter-process communication mechanism. Upon receiving the command, the lightweight offline deployment agent (Ollama-OfflineAgent) immediately sends a control signal to the system's internal hot-load manager to initiate an asynchronous loading task. Then, the hot-load manager requests the operating system to create a strictly isolated child process, namely the Ollam service process. This child process independently executes the background mounting and initialization tasks of the new model, ensuring that the normal response of the current main inference service is never blocked. Once the new model is fully loaded and ready, the child process returns a ready signal to the hot-load manager. Subsequently, the hot-load manager atomically registers the newly launched model service to the system's global routing table, completing a smooth traffic transition. Finally, the lightweight offline deployment agent returns a successful loading response to the administrator, achieving a seamless and smooth update with zero business interruption throughout the entire process.

[0082] Based on any of the foregoing embodiments, in order to ensure the reliability of gigabit or even 10-gigabit volume data during the transportation process, the data to be packaged also includes an integrity check code; the integrity check code is calculated based on the model file, the minimum necessary dependency set, the multiple pre-compiled inference backends, and the hardware adaptive script.

[0083] The process of unzipping the offline installation package on the offline device further includes: Extract the integrity check code from the data to be packaged; Based on the integrity check code, a consistency check is performed on the restored data obtained by decompressing the offline installation package; If the consistency check passes, the hardware environment information of the offline device is detected.

[0084] In practice, the first step is to extract the integrity check code from the data to be packaged. This integrity check code refers to a tamper-proof digital signature generated during the packaging phase using a highly secure hash algorithm, such as the third-generation Secure Hash Algorithm standard, SHA3-256.

[0085] Specifically, before an offline device receives a packet file but has not yet performed any in-depth operations, it can first extract the characteristic digital string attached to the packet header or footer, i.e., the integrity check code, by deploying an agent program.

[0086] Next, based on the integrity checksum, a consistency check is performed on the restored data obtained from the decompressed offline installation package. Here, restored data refers to the file entities released from compressed form back to the local disk directory. Specifically, after the file is stream-decompressed and stored on disk, the underlying computation module can be started to recalculate the massive model weight file and various script dependencies using the same hash algorithm, generating a completely new on-site hash value. Then, the on-site hash value is rigorously compared with the previously extracted original signature feature to determine whether the file content has undergone subtle changes due to network fluctuations, bad sector copying, or malicious tampering. It should be explained that implementing strict consistency comparison can accurately detect underlying weight corruption that is imperceptible to the naked eye, preventing problematic data from being loaded into video memory and causing logical inconsistencies.

[0087] Finally, if the consistency check passes, the hardware environment information of the offline device is checked. Specifically, only when the on-site comparison result declares complete consistency, proving that the transmitted data packets are 100% pure and lossless, will the next step of physical underlying environment parameter detection and backend configuration be allowed. If the comparison fails, the deployment is terminated directly and a security alert is issued.

[0088] In one embodiment, Figure 4 This is a schematic diagram of the internal structure of the offline installation package provided by the present invention, as shown below. Figure 4 As shown, this offline installer, acting as a single distribution entity, highly encapsulates all the core digital assets required to maintain the offline operation of the model. Specifically, the most fundamental components of the package are the extracted general weight format model ontology file, namely the model.geff file, and the dependency manifest file, namely dependency.json, recorded in a structured language. The package also integrates a dedicated backend folder, backends / , which categorizes and stores pre-compiled inference backends adapted to different hardware architectures, such as versions adapted to general-purpose CPUs, versions adapted to mainstream GPUs, and versions adapted to specific domestic neural network accelerators. Furthermore, the installer also includes a hardware adaptation script (autio_adaptt.py) for dynamically probing the target hardware environment and a sandbox configuration template (sangbox.conf) for isolating the running process. Within the package's independent data segment, an integrity checksum (checksum.sha3) generated based on a secure hash algorithm is also included to comprehensively ensure the security, tamper-proof nature, and strong consistency of all data within the package during offline transfer and streaming decompression.

[0089] The method provided in this invention extracts the integrity check code from the packaged file, performs a consistency check on the restored data after decompression, and then performs hardware environment information detection after the check passes. This establishes a mandatory security verification barrier based on hashing algorithms, which makes up for the lack of automated troubleshooting methods in traditional solutions and ensures the consistency and high security and reliability of inference results after transmission through a poorly configured network outage medium.

[0090] Based on any of the foregoing embodiments, in order to improve the compression ratio of the extremely large volume model and the decompression efficiency at the target end, the data to be packaged is compressed to generate the offline installation package, including: The data to be packaged is divided into multiple data blocks according to a preset size, and compression is performed on each data block to obtain each compressed data block. Extract the position offset information and logical mapping relationship of each compressed data block, and construct a block index table; The block index table is combined with each compressed data block to generate the offline installation package.

[0091] In practice, the data to be packaged is first divided into multiple data blocks according to a preset size. Compression is then performed on each data block to obtain compressed data blocks. Here, data block segmentation refers to the forced division of a massive aggregated data set into smaller data units based on a fixed byte capacity threshold. Specifically, when the packaging pipeline starts, the model file and the data awaiting packaging from the underlying execution components are used as input streams. Then, the data is truncated according to a predefined data length boundary, breaking down the giant data pipeline into hundreds or thousands of independent micro-data blocks. Next, a compression engine supporting high-speed processing algorithms, such as Zstandard, is invoked. The machine's multi-core computing power is mobilized to perform high-intensity data dictionary compression operations on each of these segmented data units in parallel, thereby generating independently shrunk units. It should be noted that introducing this block-based independent operation mechanism greatly unleashes the machine's concurrent processing potential and avoids the memory overflow risk that easily occurs when processing extremely large files with a single thread.

[0092] Next, the position offset information and logical mapping relationship of each compressed data block are extracted to construct a block index table. Here, the position offset information refers to the absolute physical start and end addresses of each compressed unit in the final merged file sequence. The block index table is a structured navigation directory used to record and maintain the correct combination order of all fragments and their physical and logical correspondences.

[0093] Specifically, while generating independent compressed blocks, the system meticulously records the starting pointer and data length of each block relative to the file header, and extracts the original filename and associated path of that block. These addressing parameters and associated logic are all written into a unified structured dictionary list. A global navigation directory is constructed, providing precise jump-style addressing coordinates for subsequent reading of large encapsulated files by the target device, making the individual extraction of local files possible.

[0094] Finally, the block index table is combined with each compressed data block to generate an offline installation package. Specifically, this meticulously constructed directory file is placed in a designated area, such as the core header of the package file. Then, it is physically stitched together with the neatly arranged, highly compressed data fragments, ultimately delivering a well-structured, independent package containing a high-performance access channel. It's important to understand that this combination of index and content structure gives the offline deployment package an inherent advantage in terms of streaming parallel decompression.

[0095] The method provided in this invention divides the data to be packaged into multiple data blocks for parallel compression, extracts information such as position offset to construct a block index table, and finally combines the index table with each block to generate an offline installation package. It adopts a block index-based high-performance compression architecture, which enables the parallel streaming processing capability of ultra-large language model data, greatly reducing the generation and restoration time of gigabyte-level files during offline migration.

[0096] Based on any of the above embodiments Figure 5 This is a schematic diagram of the architecture of the dependency-aware large model offline migration system provided by the present invention, such as... Figure 5 As shown, the system architecture mainly consists of two physical endpoints: a network-connected source device and a target offline device. On the network-connected source device, a series of modules are connected, including a manifest file parsing module, a data block extraction module, a hybrid dependency monitoring module, an intelligent dependency pruning module, and a binding and encapsulation module. The dependency monitoring module combines an Extended Berkeley Packet Filter (eBPF) and a dynamic link library preloading mechanism. After the extracted model and streamlined dependencies are packaged into an offline installation package by these modules, it is transmitted across network isolation to the target offline device. On the target offline device, a lightweight offline deployment agent (Ollama-Offline Agent, OOA) is deployed. Upon receiving the offline installation package, it uses a hardware adaptive module to perform low-level environment detection and dynamic matching of the computation backend, ultimately starting and running the framework model service in a sandboxed isolated environment. Simultaneously, the target system is also configured with a hot-loading management module to receive hot-loading commands to achieve asynchronous background loading and seamless switching of the model service.

[0097] For example, migrating Mistral-7B from Ubuntu 22.04 (NVIDIA A10) to Kylin V10 (Ascend 910B). First, on the source machine, execute: `ollm-pack --model mistral:7b --output mistral7b.ollm`. Then, the system automatically parses the manifest, extracts the blob, and monitors calls to libcuda.so.1 and libggml-cuda.so. Next, after intelligent trimming, it packages llama.cpp, including both CPU and NPU plugin versions. Then, copy the .ollm file to the Kylin device. Next, run `ollm-agent --install mistral7b.ollm`, which automatically detects the Ascend NPU and loads the CANN backend. Finally, you can verify that the target model inference is normal.

[0098] In addition, in specific applications, the system is running Llama3, i.e., the old model. After the administrator uploads phi3.ollm, the OOA receives the instruction and loads the phi3 model in the background; 5 seconds later, the API can simultaneously respond with / api / generate?model=llama3 and / api / generate?model=phi3, without service interruption.

[0099] It should be noted that the migration system provided in this embodiment of the invention increases the migration success rate from approximately 60% to over 99%, reduces the size of the offline installation package by about 30%, enables cross-platform migration, reduces deployment time to only about 1 / 10 of the original, and supports hot reloading and integrity assurance based on SHA3-256 combined with sandbox isolation.

[0100] Based on any of the above embodiments Figure 6 This is a schematic diagram of the offline migration device for large models based on dependency awareness provided by the present invention, as shown below. Figure 6 As shown, the device includes: The model file extraction unit 610 extracts the model file corresponding to the target model based on the summary identifier of the target model; The dependency capture unit 620 captures runtime dependency behavior in real time during the loading of the model file to generate a runtime dependency behavior log. The trimming unit 630 trims dependent components based on the runtime dependency behavior log to generate a minimum necessary dependency set. Migration unit 640 packages the model file and the minimum necessary dependency set to obtain an offline installation package, and performs offline migration based on the offline installation package.

[0101] The apparatus provided in this invention extracts model files based on the summary identifier of the target model, dynamically captures runtime dependency behavior to generate logs when the model is loaded, then generates a minimum necessary dependency set based on the logs, and finally packages the model files and the minimum necessary dependency set together into an offline installation package for migration. This overcomes the shortcomings of traditional manual migration, which is prone to missing components, and achieves accurate dependency extraction and deployment on any offline device with a single package. It also significantly improves the success rate of offline model migration and the efficiency of deployment.

[0102] Based on any of the above embodiments, the trimming unit is specifically used for: Extract the call frequency and execution path coverage of each dependent component from the runtime dependency behavior log; Based on the call frequency of each dependent component and the execution path coverage, the necessity score of each dependent component is calculated. Compare the necessity scores of each dependent component with a preset threshold, remove dependent components whose necessity scores are lower than the preset threshold, and use the remaining dependent components as the minimum necessary dependency set.

[0103] Based on any of the above embodiments, the dependency capture unit is specifically used for: Inject a probe program to intercept and record system call behavior during the loading process of the model file; Configure a dynamic link library preloading mechanism to intercept and record the dynamic link library loading behavior during the loading process of the model file; Record the access paths to environment variables during the loading process of the model file; The system call behavior, the dynamic link library loading behavior, and the access path are combined to generate the runtime dependency behavior log.

[0104] Based on any of the above embodiments, an identifier extraction unit is further included before the model file extraction unit, and the identifier extraction unit is specifically used for: Parse the model manifest file corresponding to the target model in the model directory of the large model execution framework; Extract the hash value of the corresponding data block from the model manifest file, and use the hash value as the digest identifier.

[0105] Based on any of the above embodiments, the migration unit is specifically used for: Obtain multiple pre-compiled inference backends adapted to different hardware environments, as well as hardware adaptive scripts; The model file, the minimum necessary dependency set, the multiple pre-compiled inference backends, and the hardware adaptive script are used as data to be packaged. The data to be packaged is compressed to generate the offline installation package.

[0106] Based on any of the above embodiments, the migration unit is further specifically used for: Unzip the offline installation package on the offline device; The hardware environment information of the offline device is detected; the hardware environment information includes at least one of the following: operating system type, processor architecture, and accelerator card type. By executing the hardware adaptive script, a target inference backend that matches the offline device is selected from the plurality of pre-compiled inference backends based on the hardware environment information; Based on the target inference backend and the minimum necessary dependency set, the model file is loaded on the offline device to start the model service.

[0107] Based on any of the above embodiments, the migration unit is further specifically used for: In response to a target model loading command, a subprocess independent of the current model service is created on the offline device; The offline installation package is asynchronously decompressed in the subprocess.

[0108] Based on any of the above embodiments, the data to be packaged further includes an integrity check code; the integrity check code is calculated based on the model file, the minimum necessary dependency set, the multiple pre-compiled inference backends, and the hardware adaptive script; The migration unit is also specifically used for: Extract the integrity check code from the data to be packaged; Based on the integrity check code, a consistency check is performed on the restored data obtained by decompressing the offline installation package; If the consistency check passes, the hardware environment information of the offline device is detected.

[0109] Based on any of the above embodiments, the migration unit is further specifically used for: The data to be packaged is divided into multiple data blocks according to a preset size, and compression is performed on each data block to obtain each compressed data block. Extract the position offset information and logical mapping relationship of each compressed data block, and construct a block index table; The block index table is combined with each compressed data block to generate the offline installation package.

[0110] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a dependency-aware large model offline migration method. This method includes: extracting the model file corresponding to the target model based on the target model's summary identifier; capturing runtime dependency behavior in real time during the loading of the model file to generate a runtime dependency behavior log; pruning dependency components based on the runtime dependency behavior log to generate a minimum necessary dependency set; packaging the model file and the minimum necessary dependency set to obtain an offline installation package; and performing offline migration based on the offline installation package.

[0111] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0112] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the dependency-aware large model offline migration method provided by the above methods. The method includes: extracting the model file corresponding to the target model based on the summary identifier of the target model; capturing runtime dependency behavior in real time during the loading of the model file to generate a runtime dependency behavior log; pruning dependency components based on the runtime dependency behavior log to generate a minimum necessary dependency set; packaging the model file and the minimum necessary dependency set to obtain an offline installation package; and performing offline migration based on the offline installation package.

[0113] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the dependency-aware large model offline migration method provided by the above methods. The method includes: extracting the model file corresponding to the target model based on the summary identifier of the target model; capturing runtime dependency behavior in real time during the loading of the model file to generate a runtime dependency behavior log; pruning dependency components based on the runtime dependency behavior log to generate a minimum necessary dependency set; packaging the model file and the minimum necessary dependency set to obtain an offline installation package; and performing offline migration based on the offline installation package.

[0114] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0115] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.< / digest> < / model>

Claims

1. A dependency-aware offline migration method for large models, characterized in that, include: Based on the summary identifier of the target model, extract the model file corresponding to the target model; During the loading of the model file, runtime dependency behavior is captured in real time to generate runtime dependency behavior logs; Based on the runtime dependency behavior log, dependency components are trimmed to generate a minimum necessary dependency set; The model file and the minimum necessary dependency set are packaged to obtain an offline installation package, and offline migration is performed based on the offline installation package.

2. The offline migration method for large models based on dependency awareness according to claim 1, characterized in that, The process of trimming dependency components based on the runtime dependency behavior log to generate a minimum necessary dependency set includes: Extract the call frequency and execution path coverage of each dependent component from the runtime dependency behavior log; Based on the call frequency of each dependent component and the execution path coverage, the necessity score of each dependent component is calculated. Compare the necessity scores of each dependent component with a preset threshold, remove dependent components whose necessity scores are lower than the preset threshold, and use the remaining dependent components as the minimum necessary dependency set.

3. The offline migration method for large models based on dependency awareness according to claim 1, characterized in that, During the loading of the model file, runtime dependency behavior is captured in real time to generate a runtime dependency behavior log, including: Inject a probe program to intercept and record system call behavior during the loading process of the model file; Configure a dynamic link library preloading mechanism to intercept and record the dynamic link library loading behavior during the loading process of the model file; Record the access paths to environment variables during the loading process of the model file; The system call behavior, the dynamic link library loading behavior, and the access path are combined to generate the runtime dependency behavior log.

4. The offline migration method for large models based on dependency awareness according to any one of claims 1 to 3, characterized in that, Before extracting the model file corresponding to the target model based on the summary identifier of the target model, the method further includes: Parse the model manifest file corresponding to the target model in the model directory of the large model execution framework; Extract the hash value of the corresponding data block from the model manifest file, and use the hash value as the digest identifier.

5. The offline migration method for large models based on dependency awareness according to any one of claims 1 to 3, characterized in that, The step of packaging the model file and the minimum necessary dependency set to obtain an offline installation package includes: Obtain multiple pre-compiled inference backends adapted to different hardware environments, as well as hardware adaptive scripts; The model file, the minimum necessary dependency set, the multiple pre-compiled inference backends, and the hardware adaptive script are used as data to be packaged. The data to be packaged is compressed to generate the offline installation package.

6. The offline migration method for large models based on dependency awareness according to claim 5, characterized in that, The offline migration based on the offline installation package includes: Unzip the offline installation package on the offline device; The hardware environment information of the offline device is detected; the hardware environment information includes at least one of the following: operating system type, processor architecture, and accelerator card type. By executing the hardware adaptive script, a target inference backend that matches the offline device is selected from the plurality of pre-compiled inference backends based on the hardware environment information; Based on the target inference backend and the minimum necessary dependency set, the model file is loaded on the offline device to start the model service.

7. The offline migration method for large models based on dependency awareness according to claim 6, characterized in that, The step of decompressing the offline installation package on the offline device includes: In response to a target model loading command, a subprocess independent of the current model service is created on the offline device; The offline installation package is asynchronously decompressed in the subprocess.

8. The offline migration method for large models based on dependency awareness according to claim 6, characterized in that, The data to be packaged also includes an integrity check code; the integrity check code is calculated based on the model file, the minimum necessary dependency set, the multiple pre-compiled inference backends, and the hardware adaptive script. The process of unzipping the offline installation package on the offline device further includes: Extract the integrity check code from the data to be packaged; Based on the integrity check code, a consistency check is performed on the restored data obtained by decompressing the offline installation package; If the consistency check passes, the hardware environment information of the offline device is detected.

9. The offline migration method for large models based on dependency awareness according to claim 5, characterized in that, The step of compressing the data to be packaged to generate the offline installation package includes: The data to be packaged is divided into multiple data blocks according to a preset size, and compression is performed on each data block to obtain each compressed data block. Extract the position offset information and logical mapping relationship of each compressed data block, and construct a block index table; The block index table is combined with each compressed data block to generate the offline installation package.

10. A dependency-aware offline migration device for large models, characterized in that, include: The model file extraction unit extracts the model file corresponding to the target model based on the summary identifier of the target model; The dependency capture unit captures runtime dependency behavior in real time during the loading of the model file to generate a runtime dependency behavior log. The trimming unit trims dependent components based on the runtime dependency behavior log to generate a minimum necessary dependency set. The migration unit packages the model file and the minimum necessary dependency set to obtain an offline installation package, and performs offline migration based on the offline installation package.

Images