Model encapsulation method, model decapsulation method, apparatus, device and storage medium

Abstract

This application provides a model encapsulation method, a model decapsulation method, an apparatus, a device, and a storage medium. In one example of this application, the model encapsulation method includes: obtaining a file to be encapsulated, the file to be encapsulated including a model file and a model description file; encapsulating the model file to obtain an encapsulated model file; and generating a model encapsulation representation file based on the encapsulated model file and the model description file.

Description

[Technical Field]

[0001] This application relates to the field of deep learning neural networks, and in particular to a model encapsulation method, a model decapsulation method, an apparatus, a device, and a storage medium. [Previous Technology]

[0002] Training deep learning neural networks requires a lot of computing resources and time. Using massive amounts of training data, a model whose performance meets the task requirements is obtained. This model can be packaged for transmission.

[0003] However, with the continuous development of deep learning, on the one hand, the number of models and the types of tasks are increasing, and on the other hand, the specifications of the platforms that can be deployed and run are different. Currently, there is a lack of an efficient, secure and widely applicable model encapsulation and transmission method. [Summary of the Invention]

[0004] In view of the above, this application provides a model encapsulation method, a model decapsulation method, an apparatus, a device, and a storage medium.

[0005] Specifically, this application is implemented through the following technical solution:

[0006] According to a first aspect of the embodiments of this application, a model encapsulation method is provided, comprising:

[0007] Obtain the file to be packaged; wherein the file to be packaged includes a model file and a model description file; wherein a model file contains information about a complete model, or information about a portion of the model's structure;

[0008] The model file is encapsulated to obtain an encapsulated model file; wherein, an encapsulated model file contains one or more model data segments, and a model data segment is used to store the complete data of a model or part of the data of the model; multiple model data segments used to store the data of the same model are set with the same identifier.

[0009] Based on the encapsulation model file and the model description file, a model encapsulation representation file is generated; wherein, the model encapsulation representation file is used to transmit to the receiving device.

[0010] According to a second aspect of the embodiments of this application, a model decapsulation method is provided, comprising:

[0011] Obtain the model encapsulation representation file; wherein the model encapsulation representation file is obtained by encapsulating using the method provided in the first aspect;

[0012] The model encapsulation representation file is decapsulated to obtain a decapsulated file; wherein, the decapsulated file includes a model file and a model description file;

[0013] The model used for task processing is determined based on the decapsulated file; wherein the model is used to obtain inference output based on the model input.

[0014] According to a third aspect of the embodiments of this application, a model packaging apparatus is provided, comprising:

[0015] Acquisition unit, used to acquire the file to be packaged; wherein, the file to be packaged includes a model file and a model description file; wherein, a model file contains information about a complete model, or information about a portion of the structure of the model;

[0016] Encapsulation unit, used to encapsulate the model file to obtain an encapsulated model file; wherein, an encapsulated model file contains one or more model data segments, a model data segment is used to store the complete data of a model or part of the data of the model; multiple model data segments used to store the data of the same model are set with the same identifier.

[0017] The generation unit is used to generate a model encapsulation representation file based on the encapsulation model file and the model description file; wherein the model encapsulation representation file is used to transmit to the receiving device.

[0018] According to a fourth aspect of the embodiments of this application, a model decapsulation apparatus is provided, comprising:

[0019] An acquisition unit is used to acquire a model encapsulation representation file; wherein the model encapsulation representation file is obtained by encapsulating the model using the method provided in the first aspect;

[0020] Decapsulation unit, used to decapsulate the model encapsulation representation file to obtain a decapsulated file; wherein, the decapsulated file includes a model file and a model description file;

[0021] The determining unit is used to determine the model for task processing based on the depackaged file; wherein the model is used to obtain inference output based on the model input.

[0022] According to a fifth aspect of the present application, an electronic device is provided, including a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor being configured to execute the machine-executable instructions to implement the method provided in the first or second aspect.

[0023] According to a sixth aspect of the embodiments of this application, a machine-readable storage medium is provided, wherein machine-executable instructions are stored therein, and when the machine-executable instructions are executed by a processor, the method provided in the first aspect or the second aspect is implemented.

[0024] According to a seventh aspect of the embodiments of this application, a model repository sending device is provided, comprising: a model pool, a device library, and a communication and control module; wherein:

[0025] The model pool is used to store standard model files;

[0026] The device library is used to store model converters corresponding to different model deployment platforms. The model converters are used to convert standard models into models that can be executed by the model deployment platform.

[0027] The communication and control module is used to encapsulate the standard model to be transmitted using the method provided in the first aspect, and transmit the obtained model encapsulation representation file to the receiving device.

[0028] According to an eighth aspect of the embodiments of this application, a model repository receiving device is provided, comprising: a model pool, a device library, and a communication and control module; wherein:

[0029] The communication and control module is used to receive the model encapsulation representation file sent by the transmitting device, and to decapsulate the received model encapsulation representation file using the method provided in the second aspect to obtain a standard model;

[0030] The device library is used to store model converters for corresponding model warehouse receiving devices. The model converters are used to convert the standard model into a model that can be executed by the model warehouse receiving device.

[0031] The model pool is used to store executable models of the model warehouse receiving device.

[0032] The model encapsulation method provided in this application embodiment obtains a file to be encapsulated, which includes a model file and a model description file. For the model file, data encapsulation can be performed to obtain an encapsulated model file. Based on the encapsulated model file and the model description file, a model encapsulation representation file for transmission to the receiving device is generated. A model file can contain data of a complete model or data of a portion of the model's structure. An encapsulated model file can contain one or more model data segments, where each segment stores complete data or partial data of a model. Therefore, the data of a complete model can be encapsulated into an encapsulated model file, or the data of a portion of the model's structure can be encapsulated into an encapsulated model file, improving the flexibility of model data encapsulation. Consequently, when transmitting model data, the data of a complete model or the data of a portion of the model's structure can be transmitted, further improving the flexibility of model data transmission.

Implementation Method

[0034] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0035] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms "a," "the," and "the" as used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0036] In order to enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, some terms involved in the embodiments of this application will be explained below.

[0037] 1. Deep learning model: refers to the model parameters and structure obtained by using deep neural networks and training with data. Generally speaking, they are fixed during inference.

[0038] 2. Model encapsulation: refers to the process of arranging data and supplementing information in a more reasonable manner on the original model used for inference, so as to facilitate the transmission of the model.

[0039] In some examples, through model encapsulation, the parameters, structure, meta-information, and additional files required for running a deep learning model can be encapsulated into an independent unit to facilitate the storage, transmission, and subsequent deployment of the model.

[0040] In order to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions of the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0041] It should be noted that the sequence number of each step in the embodiments of this application does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0042] Please refer to Figure 1, which is a flowchart illustrating a model encapsulation method provided in an embodiment of this application. As shown in Figure 1, the model encapsulation method may include the following steps:

[0043] Step S100: Obtain the file to be packaged; wherein, the file to be packaged includes a model file and a model description file; wherein, a model file contains information about a complete model, or information about a part of the structure of the model.

[0044] For example, the model file may include model data such as model weight information and structure information.

[0045] For example, a model file may contain weight information and structure information of a complete model, or a model file may contain weight information and structure information of a partial structure of a model.

[0046] For example, the model description file is used to describe the model's attribute information and technical information.

[0047] For example, the attribute information of the model refers to the basic entity information related to the model, which may include, but is not limited to, information such as the name and size of the model. The technical information of the model refers to the algorithm-level information related to the model, which may include, but is not limited to, information such as the model version, model data type, running requirements, and input / output.

[0048] Step S110: Encapsulate the model file to obtain an encapsulated model file; wherein, an encapsulated model file contains one or more model data segments, and a model data segment is used to store the complete data of a model or part of the data of a model; multiple model data segments used to store the data of the same model are set with the same identifier.

[0049] In this embodiment of the application, in order to improve the flexibility of model data transmission, when encapsulating model files, model data in the same model file can be stored in one or more model data segments of the encapsulated model file.

[0050] For example, model data in the model file can be stored in the same model data segment, or, for the data of a complete model, it can be segmented and each segment of model data can be stored in a different model data segment.

[0051] For example, in order for the receiving end to accurately identify the model data of the same model, the same identifier (also called model identifier) ​​can be set for multiple model data segments used to store the data of the same model.

[0052] Step S120: Generate a model encapsulation representation file based on the encapsulation model file and the model description file; wherein, the model encapsulation representation file is used to transmit to the receiving device.

[0053] In this embodiment of the application, when the encapsulated model file is obtained in the manner described above, the encapsulated model file and the model description file can be organized into a single file (which can be called a model encapsulation representation file), which can be sent to the receiving device.

[0054] For example, the model description file can be stored in the form of binary data at the end of the encapsulated model file to obtain the model encapsulated representation file.

[0055] For example, when the receiving device receives the model encapsulation representation file, it can decapsulate the model encapsulation representation file to obtain the model file and the model description file, and perform relevant processing based on the obtained files. The specific implementation can be found in the relevant description in the method flow shown in Figure 2. The embodiments of this application will not be described in detail here.

[0056] As can be seen, in the method flow shown in Figure 1, the file to be packaged is obtained, which includes a model file and a model description file. For the model file, data can be packaged to obtain a packaged model file, and based on the packaged model file and the model description file, a model packaged representation file for transmission to the receiving device is generated. A model file can contain the data of a complete model or the data of a part of the model's structure, and a packaged model file can contain one or more model data segments. A model data segment is used to store the complete data of a model or the data of a part of the model. Thus, the data of a complete model can be packaged into a packaged model file, or the data of a part of the model's structure can be packaged into a packaged model file, improving the flexibility of model data packaging. Therefore, when transmitting model data, the data of a complete model or the data of a part of the model's structure can be transmitted, improving the flexibility of model data transmission.

[0057] In some embodiments, a packaged model file includes a file header, and one or more model data segments as described above;

[0058] The file header includes the file header start code, the file header marker value, the file version, and the number of model data segments in the encapsulated model file.

[0059] For example, the file header start code can be used to identify the file header, making it easier for personnel to locate the file header, and its value is usually a preset value.

[0060] The file header mark value can be used to verify the integrity of the file header. Its value is usually a preset value. If the file header mark value is not the preset value during the process of reading the file header, it indicates that the integrity of the file header has been damaged, indicating that the file may be damaged. In this case, subsequent data reading can be skipped.

[0061] The file version is used to identify the version information of the file, and its value can be customized.

[0062] The number of model data segments in the encapsulated model file is used to identify the number of model data segments included in an encapsulated model file, and its value can be set according to requirements.

[0063] In some embodiments, a model data segment includes a model header and model data;

[0064] The model header includes the model header start code, the model header identifier, the verification message, and the model data size; wherein, the model header identifier is the same in multiple model data segments used to store data of the same model.

[0065] For example, the model data segment may include a model header and model data.

[0066] For example, the model header may include a model header start code to identify the model header and facilitate the location of the model header.

[0067] The model header may also include a model header identifier, verification messages, and model data size.

[0068] The model header identifier can be used to identify the correspondence between the model data segment and the model. When the model is too large or there is a need for segmented transmission, the model data of a model can be stored in multiple model data segments with the same model header identifier. The value of the model header identifier can be customized.

[0069] The verification message can be used to verify the model data in the model data segment, and its value can be determined based on the specific data calculation results.

[0070] The model data size can be used to identify the size of the model data stored in the model data segment.

[0071] In one example, the data of a complete model is stored in the same model data segment; or, the data of a complete model is divided into at least two parts, and the model data of the at least two parts are stored in different model data segments of the same packaged model file.

[0072] For example, in order to improve the flexibility of model data encapsulation, the data of a complete model can be encapsulated in the same model data segment, or the data of the complete model can be divided into at least two parts as required.

[0073] For example, assuming the length of the data of a complete model is 100, it can be divided into two parts with lengths of 60 and 40, or 70 and 30, or 50 and 50, etc.; or it can be divided into three parts with lengths of 50, 30 and 20, or 40, 10 and 50, or 30, 30 and 40, etc.

[0074] For example, when the data of a complete model is divided into at least two parts, the model data of the at least two parts can be stored in different model data segments of the same packaged model file.

[0075] For example, the model header identifiers in the model data segments used to store the model data of the at least two parts are the same, which is used to identify that the model data stored in the multiple model data segments belong to the same model.

[0076] In some embodiments, the above-described data encapsulation of the model file to obtain the encapsulated model file may include:

[0077] When it is determined that the model needs to be processed in a specific way, the data of the model corresponding to the model file is processed in a specific way; wherein, the specific processing includes model compression processing and / or model encryption processing;

[0078] The data of the specified processed model is encapsulated to obtain the encapsulated model file.

[0079] For example, during the process of encapsulating data in the model file, the data of the model can also be specified for processing according to actual needs.

[0080] For example, the specified processing may include, but is not limited to, model compression processing and / or model encryption processing.

[0081] For example, model encryption processing may include encrypting model data in the model data segment.

[0082] In one example, the processing methods for specifying the processing of the same model's data stored in different model data segments are allowed to be different. During the encapsulation of model data, it is possible to determine which part of the model data will be stored in which model data segment. Therefore, before the final encapsulation processing, the model data can be specified according to the model data segments according to actual needs. The specified processing methods for each model data segment used to store the data of the same model may be the same or different.

[0083] For example, when the data of the same model is to be stored in multiple different model data segments, different processing methods can be used to specify the processing for the data of the same model to be stored in different model data segments.

[0084] For example, the processing method for model compression may include, but is not limited to, model compression methods such as pruning, quantization and / or low-rank decomposition.

[0085] For example, the processing method of model encryption processing may include, but is not limited to, symmetric encryption methods (such as AES (Advanced Encryption Standard), SM4 (a block cipher standard)), asymmetric encryption methods (such as RSA (an asymmetric encryption algorithm), SM2 (an elliptic curve public key cryptography algorithm standard)) or multi-level encryption methods.

[0086] For example, the processing method for the specified processing of the same model data stored in different model data segments is the same; or, the processing method for the specified processing of the same model data stored in different model data segments is not completely the same. For example, assuming that the same model data is stored in model data segments 1 to 3, the processing method for the specified processing of the model data in model data segment 1 can be the same as the processing method for the specified processing of the model data in model data segment 2, and different from the processing method for the specified processing of the model data in model data segment 3; or, the processing method for the specified processing of the same model data stored in different model data segments is completely different.

[0087] In some embodiments, the model description file may include a model attribute message file (or a model attribute message sub-file) and a model technology message file (or a model technology message sub-file).

[0088] The model attribute information file is used to store model attribute information, which includes the model name and model size information;

[0089] The model technical information file is used to store model technical information, which includes model version information, model data type information, hardware support information required by the model, basic environment information for running the model, model input information, and model output information.

[0090] For example, model size information may include, but is not limited to, the number of model parameters and / or the amount of model computation.

[0091] The unit of model parameter quantity may include, but is not limited to, MB or GB; the unit of model computation quantity may include, but is not limited to, MFLOP (million floating-point operations per second) or GFLOPS (billion floating-point operations per second).

[0092] For example, the model data type may include, but is not limited to, FP32 (single-precision floating-point number), FP16 (half-precision floating-point number), BF16 (BFloat16), etc.

[0093] For example, the hardware support information required by the model may include the requirements of one or more of the hardware such as the Central Processing Unit (CPU), the Graphics Processing Unit (GPU), and memory.

[0094] For example, the basic environment information of the running model may include, but is not limited to, basic environments such as systems and programs.

[0095] For example, the model input information may include the type of data input by the model, such as image, text, video, or speech.

[0096] The model output information may include the type of data output by the model, such as image, text, video, speech, coordinates, probability, etc.

[0097] In one example, the model attribute message may also include one or more of the following messages:

[0098] Model task information, model evaluation metric information, model open source license, model production date, algorithm description file, and script description file.

[0099] For example, the model task message is used to describe the type of model task, such as image classification task, image segmentation task, image style transfer task, OCR character recognition task, object detection task, translation task, or image generation task.

[0100] For example, the model evaluation metric message is used to describe the evaluation metrics of the model, such as precision, recall, F1 score, or accuracy.

[0101] In one example, the model technology message may also include one or more of the following messages:

[0102] Model runtime configuration file, model runtime framework, model input resolution type, model converter input configuration file, pre-trained large model structure information, and model file information.

[0103] For example, the configuration file for model runtime can be used to modify settings such as input and output path, threshold, and output format setting.

[0104] For example, the model input resolution type may include fixed resolution or dynamic resolution, etc.

[0105] In some embodiments, the file to be packaged also includes a model program file; the model program file includes an execution library and / or a runtime script;

[0106] The above-mentioned generation of a model encapsulation representation file based on the encapsulated model file and the model description file may include:

[0107] Generate a model encapsulation representation file based on the encapsulated model file, model description file, and model program file.

[0108] For example, the file to be packaged may also include a model program file. The model program file may include an execution library and / or a runtime script.

[0109] For example, the execution library may include program files required by the runtime framework and runtime environment.

[0110] For example, the running script may include model-related program scripts that obtain running results through input data, such as interface calls, model inference, etc.

[0111] For example, when the encapsulated model file is obtained in accordance with the manner described in the above embodiments, the encapsulated model file, the model description file and the model program file can be organized into a model encapsulated representation file.

[0112] In one example, the model program file may include an execution library, which includes a model converter for converting a standard model into a model for a specified platform.

[0113] For example, in order to reduce the number of models that the model sender needs to store, the model converter can be sent to the model receiver along with the standard model. The model receiver can then use the model converter to convert the standard model to obtain the model that can be deployed on the specified platform.

[0114] Please refer to Figure 2, which is a flowchart illustrating a model decapsulation method provided in an embodiment of this application. As shown in Figure 2, the model decapsulation method may include the following steps:

[0115] Step S200: Obtain the model encapsulation representation file.

[0116] For example, the model encapsulation representation file can be encapsulated in the manner described in the method flow shown in Figure 1.

[0117] Step S210: Decapsulate the model encapsulation representation file to obtain the decapsulated file; wherein, the decapsulated file includes the model file and the model description file.

[0118] In this embodiment of the application, when a model encapsulation representation file is obtained, for example, when the receiving device receives a model encapsulation representation file sent by the sending device, the model encapsulation representation file can be decapsulated to obtain a decapsulated file.

[0119] For example, the decapsulated file may include a model file and a model description file.

[0120] For example, in the process shown in FIG1, if the file to be packaged also includes a model program file, the depackaged file may also include a model program file.

[0121] For example, the method of decomposing the model representation file can be determined based on the encapsulation method when generating the model representation file, and the model representation file can be re-splittered into a model file and a model description file, or a model file, a model description file and a model program file, based on the decapsulation method.

[0122] Step S220: Determine the model for task processing based on the depackaged file; wherein, the model is used to obtain inference output based on model input.

[0123] In this embodiment of the application, when the depackaged file is obtained, the model used for task processing can be determined based on the depackaged file.

[0124] For example, the task may include, but is not limited to, image processing tasks, video processing tasks, text processing tasks, or voice processing tasks.

[0125] Taking image processing tasks as an example, image processing tasks may include, but are not limited to, image classification tasks, image segmentation tasks, image style transfer tasks, optical character recognition (OCR), character recognition tasks, object detection tasks, translation tasks, or image generation tasks.

[0126] For example, a model for task processing can be constructed based on the weight and structure information in the model file.

[0127] For example, when the data of a complete model is contained in multiple model files, that is, a model file contains data of a partial structure of the model, multiple model files of the same model can be determined based on the model description file. For example, multiple model files of the same model can be determined based on the model_file_info (information used to associate the model and the model file) in the model description file. Then, a model for task processing can be constructed based on the multiple model files.

[0128] In one example, if the model encapsulation representation file includes partial data of the model, and the model encapsulation representation file is decapsulated in the manner described above, and the corresponding model structure (partial structure of the model) is obtained based on the decapsulated file, the final model for task processing can be obtained based on the model structure and the local model data.

[0129] For example, this model structure can be used to replace the relevant parts in the local corresponding model to obtain the final model for task processing.

[0130] For example, the model corresponding to the model structure can be determined based on the identifier (model header identifier) ​​in the model data segment, and the relevant part in the corresponding model can be replaced by the model structure to obtain the final model for task processing.

[0131] As can be seen, through the above solution, when the model is partially updated, the updated model data can be encapsulated and sent to the receiving end. The receiving end can then update the local model based on the updated model data. Thus, model updates can be achieved without transmitting the complete model data, reducing the amount of data transmitted during model updates.

[0132] For example, the specific implementation of model decapsulation can refer to the implementation process of model encapsulation.

[0133] For example, if the data of the model is processed in the model encapsulation process, the reverse processing of the specified processing can also be performed in the model decapsulation process.

[0134] For example, when the specified process includes model compression, the inverse process of the specified process includes model decompression.

[0135] When the specified processing includes model encryption, the inverse processing of the specified processing includes model decryption.

[0136] For example, if it is determined that the model needs to be converted, the converter included in the model program file or a local tool (if the model needs to be converted and the model program file does not include a converter) can also be used to convert the standard model file into a structure and form that can be used for local inference, that is, to convert the standard model into a model that can be executed on the local platform.

[0137] In this embodiment of the application, when a model for task processing is obtained, the model data can be loaded using the execution library, and given relevant inputs, the inference output can be obtained.

[0138] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of this application, the technical solutions provided by the embodiments of this application will be described below with reference to specific examples.

[0139] In this embodiment, a standard model encapsulation and transmission process is defined, and the algorithm repository pattern is used to improve the efficiency of model storage, management and distribution.

[0140] For example, the model file, model description file and model program file related to model deployment correspond to three modules respectively, as shown in Figure 3. Different encapsulation processes are adopted for the three modules respectively.

[0141] For example, the model file contains the model's weight information and structure information, and uses the subsequent model encapsulation representation definition method to convert the raw data into encapsulated data.

[0142] The model description file mainly contains model attribute information and model technical information.

[0143] The model program file mainly includes two parts: the first part is the execution library, which includes the running framework and the program files required for the running environment; the second part is the running script, which includes program scripts related to the model that obtain running results through input data, such as interface calls, model inference, etc.

[0144] For example, to increase transmission efficiency and improve security, model compression and data encryption processes can be optionally performed.

[0145] The defined encapsulation method can be used to organize files from different sources together as required to form an encapsulated representation file (i.e., the above model encapsulated representation file) for subsequent transmission.

[0146] For example, the flowchart of the overall model encapsulation and distribution is shown in Figure 4.

[0147] For example, in contrast to the model encapsulation process, when the receiving end obtains the model encapsulation representation file, it divides the model encapsulation representation file into a model file, a model description file, and a model program file, and performs decryption and decompression operations on the model that needs to be decrypted and decompressed. The overall process can be shown in Figure 5.

[0148] In addition, in order to meet the deployment requirements of different platforms, a model converter can be used to convert the original unified model representation into a terminal model representation specifically for the target platform. That is, the standard model is converted into a model that can be executed on the target platform, thereby increasing the use cases of the same model and reducing the storage requirements of the server.

[0149] The following describes some implementation details of model encapsulation / decapsulation in the embodiments of this application.

[0150] I. Model encapsulation represents the file directory structure:

[0151] The model encapsulation means that the file needs to follow the following file directory structure, as shown in Table 1:

[0152] 1.1 Model (Model Directory): Stores encapsulated model data.

[0153] 1.2, Program (program directory): stores execution libraries, script files, etc.

[0154] 1.3, Meta-info (Description Message Directory): Stores the model's attribute information, technical information, etc.

[0155] Table 1, Model and Archive Directory Structure Directory Name necessity Archives / Subdirectories describe Model necessary Model files AI Model Archives Program Optional Library (archives) The library files required to execute the model Script (script file) Execute the script file to control startup, shutdown, and status checks. Meta-info necessary Add subdirectories based on model identifiers Management info Model Attributes and Management Information Archives Technical info Model technology information description archive

[0156] The model identifier is a unique identifier for the model. Different models can correspond to different model description files and different description information (including attribute information and technical information).

[0157] II. Definition of Model Data Structure:

[0158] The encapsulated model data structure includes a file header, a model header, and model data. Each encapsulated model file has a separate file header and may contain model data segments from multiple model files. The model headers and corresponding data segments of multiple model data segments are arranged sequentially after the file header, as shown in Table 2.

[0159] Table 2. Arrangement of file header information and data information File Header Model Header Model Data Model Header Model Data

[0160] For example, the data structure definition of the file header can be as shown in Table 3, and the data structure definition of the model header can be as shown in Table 4:

[0161] Table 3, Structure of Archive Header Data column Data type value definition Start_code unsigned int 0x5352434D Start of header code (SRCM) Magic_number unsigned int 0x47D02F93 File header mark value Version unsigned int Custom Archive version Model_number unsigned int Custom The archive contains the following number of model blocks (i.e., model data segments):

[0162] Where, unsigned int is an unsigned integer, and its range is from zero to the maximum positive value.

[0163] Table 4, Model Header Data Structure column Data type value definition Start_code unsigned int 0x486F4D52 Model Header Start Code (HoMR) Identifier unsigned int Custom Model header identifier. When the model is too large or there is a need for segmented transmission, the model data of a single model can be divided into multiple model data segments with the same model header identifier. Check_sum unsigned int Custom 32-bit MD5 hash checksum Compression method unsigned int Custom The compression method used in the model can be specifically defined through the model description file. Encryption unsigned int Custom The encryption method used by the model can be specifically defined through the model description file. Data size unsigned int Custom Model size

[0164] III. Definition of Model Program File:

[0165] The model program file mainly consists of two parts: the execution library and the runtime scripts. The execution library mainly includes the runtime environment on which the algorithm depends, the algorithm library file (core) that is self-compiled according to the scene and model, and the converter used to convert the unified model representation into a model representation that can be used on different platforms. The runtime scripts include the script files related to model operation and control.

[0166] For example, the file structure of the model program file can be as shown in Table 5.

[0167] Table 5, Model Program File Structure Program Directory Name necessity Archives / Subdirectories describe Library Optional environment The environment dependencies required to run the model, such as PyTorch, MXNet, numpy, pandas, and Scikit-learn. core Some models require pre-compiled libraries, such as data preprocessing and visualization libraries. converter Tools for converting unified model representations into model representations that run on different device platforms. Scripts Optional Run.py Execute model script Stop.py Stop model script Status.py Check Model Status Script Convert.py Transformation Model Expression Script [Else] Other script files

[0168] IV. Model Description File Definition:

[0169] The model description file is used to describe the model's attribute information and technical information. It mainly includes two sub-files: managementinfo.json (i.e., the model attribute file mentioned above) and technicalinfo.json (i.e., the model technical file mentioned above). managementinfo.json mainly stores attribute information, and its file structure can be shown in Table 6. technicalinfo.json mainly stores technical information, and its file structure can be shown in Table 7.

[0170] Table 6, Model Attribute File Field Name necessity type describe model_name necessary String The name of the model, such as ResNet101 model_size necessary Dict Describe the model's size, including the number of model parameters (units can be MB / GB) and the computational cost in FLOPs (units can be MFLOP / GFLOPS). For example, {"params": "10MB", "FLOPSs": "1GFLOPs"} model_task Optional String The modeling task is described in Table 8. model_metrics Optional Dict Model evaluation metrics, such as precision, recall, F1-score, and accuracy. model_license Optional String The model is open-source licensed, such as GPL-2.0, MIT, etc. model_date Optional String Model production date, such as 2024 / 3 / 20 algorithm_doc Optional String Evolution Algorithm Documentation scripts_description Optional String Script description file

[0171] Where String is a string data type and Dict is a dictionary data type.

[0172] Table 7, Technical Information Archive Field Name necessity type describe model_version necessary unsigned int Model version data_type necessary String Model data types, such as FP32, FP16, BF16, etc. model_requirement necessary String The hardware support required by the model, such as CPU / GPU and memory requirements. For example: "GPU: type A, type B, GPUmem: 8G, CPU: type C, Mem: 8G". model_env necessary String The basic environment for running the model, including the system and programs, such as "Ubuntu 18.04-PYTHON 3.7-PyTorch 1.7-Cuda 9.0". model_inputs necessary List Model inputs are detailed in Table 9. model_outputs necessary List Model output model_config Optional Dict The configuration file for model runtime, used to modify settings such as input and output paths, thresholds, and output format settings. model_framework Optional String Model execution framework, such as PyTorch mode_resolution_type Optional unsigned int Model input resolution type, for example, 0: fixed resolution, 1: dynamic resolution, etc. converter_confg Optional Dict The model converter input configuration file is detailed in Table 10. PTM_info Optional Dict For details on the pre-trained large model structure, please refer to Table 11. model_file_info Optional Dict Model file information, linking files and model information, used for transferring multiple files at once.

[0173] Where, List is a list data structure.

[0174] Table 8, Model Tasks Field Name necessity type describe image_classification Optional String Image classification task image_segmentation Image segmentation task image_style_transfer Image style transfer task ocr_recognition OCR character recognition task object_detection Object detection task translation Translation task text_to_image Image generation task other Other tasks

[0175] Table 9, Model Input Description Field Name necessity type describe input_type necessary String Input data types (modalities) include images, text, video, and audio. input_name Optional String Enter the variable name input_requirement Optional String Input data format requirements, such as PNG, CSV and PNG type, input size limits, and input dimension limits. model_mean Optional List Mean of model input data model_scale Optional List Variance of model input data

[0176] Among them, the input size limit is used to limit the size of the input file data; the input size limit is used to limit the resolution of the input file.

[0177] The mean and variance of the model input data can be used to normalize the model input data.

[0178] Table 10, Converter Configuration Description Column Name necessity type describe multi-batch_type Optional String Device-side inference supports multiple batch types, such as static-batch and dynamic-batch. input_shape Optional List Input a list of resolutions, for example: [[1, 1, 112, 112]] resize_parameters Optional Dict Input data preprocessing parameters, such as resize_height, resize_wide, etc. output_node Optional Dict Specify the name of the network output node model-optimization-setting Optional Dict Optimization parameter configuration for the model transformation process, such as BN fusion, YOLO split, etc.

[0179] Where resize_height is the target height of the image after adjustment, and resize_wide is the target width of the image after adjustment.

[0180] bn fusion refers to the process of merging multiple Batch Normalization layers into a larger Batch Normalization layer.

[0181] YOLO split refers to splitting or adjusting the original YOLO (a real-time object detection algorithm) model or its network structure in order to better adapt to specific hardware platforms (such as edge devices or low-power devices).

[0182] Table 11, Information Table of Pre-trained Large Model Column Name necessity type describe architecture Optional String Model infrastructure, such as "llama", "Qwen", "mpt", "falcon" attention Optional String The descriptions of the attention modules used, such as "FlashAttention," "SparseAttention," and "MQA," can be written together for multiple technologies. PE Optional String Position encoding methods, such as Relative, Learned, RoPE, etc. max_input_length Optional Unsigned int Longest Input Length max_output_length Optional Unsigned int Maximum output length blocks Optional Unsigned int Model block layers embedding_length Optional Unsigned int Hidden state number expert_count Optional Unsigned int MoE Model Expert Module Count expert_used_count Optional Unsigned int Number of expert modules used by the MoE model during inference

[0183] For example, when the model encapsulated and transmitted in the above process is a pre-trained large model, the model description file may also include the pre-trained large model information shown in Table 11. This part of the information belongs to the model technical information file content, which facilitates quick access to technical details through text.

[0184] V. Model Packaging Process:

[0185] In this embodiment, the inputs to the packaging process include:

[0186] 5.1 Model File (Files containing model weights, structure, etc., used for model representation and inference);

[0187] 5.2 Model program files (libraries required for operation, converters required for deployment, and related scripts, etc.);

[0188] 5.3 Model Description File (a description file that stores model attribute information and model technical information).

[0189] The specific process for model encapsulation is as follows:

[0190] S1. Obtain the file to be packaged;

[0191] S2. When the model needs to be compressed, perform the model compression process;

[0192] For example, model compression methods may include general methods such as pruning, quantization, and low-rank decomposition.

[0193] For example, the agreed model compression method can be specified in the Compression method field of the data structure in the model header.

[0194] S3. When the model needs to be encrypted, perform the model encryption process;

[0195] For example, the model encryption method may include symmetric encryption methods (such as AES, SM4), asymmetric encryption methods (such as RSA, SM2), multi-level encryption methods, etc.

[0196] For example, the agreed model encryption method can be specified in the Encryption field of the model header data structure.

[0197] S4. Encapsulate the model file header and model header information, and arrange the data as required.

[0198] For example, for the complete model file model_origin output by the original training framework, it can be split into multiple model segments model_0, model_1, ..., model_n as needed. The division of model segments can be based on different modules, different structures, or other situations that require splitting.

[0199] Each model segment can be stored in the same or different files, which is suitable for various types of models.

[0200] The following will be explained with examples.

[0201] Example 1

[0202] The original complete model is segmented and stored in different model headers and model data. For example, an original model with a data length of 100 can, according to the specification, use only one model header message and model data to record its complete data content. Alternatively, the original data content can be split into multiple model segments according to different needs.

[0203] For example, as shown in Table 12, the complete data of 100 can be divided into two parts of model data with lengths of 40 and 60, or model data with lengths of 40, 10 and 50. The model data should be preceded by the corresponding model header.

[0204] For example, different compression and encryption methods can be used for different model data segments as needed, making model storage more flexible.

[0205] In addition, the header of the segmented model contains the verification information of each model data. When a problem occurs due to data transmission error, it can be detected in the current data segment in a timely manner, reducing the cost of error correction.

[0206] Table 12. Classification of data for different models File header Model Head 0 Model data 0 (length 100) File header Model Head 0 Model data 0 (length 40) Model Head 1 Model Data 1 (Length 60) File header Model Head 0 Model data 0 (length 40) Model Head 1 Model Data 1 (Length 10) Model Head 2 Model Data 2 (Length 50)

[0207] Example 2

[0208] Assume that only some parameters in the model are updated, mainly for model upgrades. After the first transmission, both the sender and receiver have a complete model, model_A. However, in subsequent training iterations, only some structures or weights in model_A change. This is common in visual tasks where the backbone is frozen and only the classification head is trained, or in language models where the Adapter learns downstream knowledge. After some modules, such as the Adapter, are updated, the second transmission only needs to transmit the corresponding Adapter structure parameters, which are then replaced and merged by the receiver. The implementation flowchart is shown in Figure 6.

[0209] As shown in Figure 6, the implementation process may include:

[0210] S1. Organize the data as required and generate a complete initial model model_A.

[0211] For example, for parameter parts that need to be transmitted separately (such as Adapter), they can be included as part of the model header and model data, and the Identifier information in the model header at this time can be recorded.

[0212] S2, Transmit model_A to the receiving end.

[0213] S3. After the model training is updated, the Adapter parameters change. The second transmission will only use the updated Adapter as the model data in the model file to keep the Identifier information consistent.

[0214] For example, compression and encryption steps can be added as needed.

[0215] S4. Transmit the model parameters (Adapter) that need to be updated to the receiving end.

[0216] S5. After the receiving end obtains the updated Adapter, it decompresses and decrypts it as needed, replaces the model data of the Identifier corresponding to the original model file model_A, and obtains model_B consistent with the sending end (i.e. model_A with the updated Adapter).

[0217] For example, the complete model includes a fixed part (Freeze) and a learned variable part (Adapter), i.e., model_A includes the fixed part (Freeze) and the learned variable part (Adapter_A). During the first transmission, the sending end transmits the complete model_A to the receiving end. The receiving end receives the complete model_A. Subsequently, the sending end's model_A is trained and updated to model_B (including the fixed part (Freeze) and the learned variable part (Adapter_B)), i.e., the learned variable part (Adapter_A) of model_A is updated to Adapter_B. During the second transmission, the sending end only needs to transmit Adapter_B to the receiving end, and the receiving end replaces Adapter_A with the received Adapter_B to obtain model_B.

[0218] VI. Model De-encapsulation Process:

[0219] In this embodiment, the inputs to the decapsulation process include:

[0220] Model encapsulation representation file obtained from the sending end.

[0221] For example, the decapsulation process is as follows:

[0222] S1. Obtain the file to be depackaged (model encapsulation representation file), and re-divide (depackage) the model encapsulation representation file into model file, model description file and model program file;

[0223] S2. When the model needs to be decrypted, the model is decrypted using a pre-agreed model decryption method.

[0224] For example, the sending device and the receiving device can pre-agree on optional model encryption and decryption methods. The sending device can send the identification information (such as a serial number) of the relevant model encryption method in the encapsulation file to the receiving device according to the actual model encryption method used, so that the receiving device can determine the model decryption method according to the identification information.

[0225] The processing of model compression and model decompression can be obtained in the same way.

[0226] S3. When the model needs to be decompressed, use the pre-agreed model decompression method to decompress the model.

[0227] S4. If the model needs to be converted, use the converter included in the model program file, or a local tool (if the model needs to be converted and the model program file does not include a converter) to convert the standard model file into a structure and form that can be used for local reasoning.

[0228] For example, if the model file received this time contains part of the model structure, the model can be updated or replaced by combining the existing local parameters.

[0229] S5. Load model data using the execution library, give relevant inputs, and obtain inference output.

[0230] The method provided by this application has been described above. The apparatus provided by this application is described below:

[0231] Please refer to Figure 7, which is a schematic diagram of the structure of a model packaging device provided in an embodiment of this application. As shown in Figure 7, the model packaging device 700 may include:

[0232] The acquisition unit 710 is used to acquire the file to be packaged; wherein the file to be packaged includes a model file and a model description file; wherein a model file contains information about a complete model, or information about a part of the structure of the model;

[0233] The encapsulation unit 720 is used to encapsulate the model file to obtain an encapsulated model file; wherein, an encapsulated model file contains one or more model data segments, and a model data segment is used to store the complete data of a model or part of the data of the model; multiple model data segments used to store the data of the same model are set with the same identifier.

[0234] The generation unit 730 is used to generate a model encapsulation representation file based on the encapsulation model file and the model description file; wherein the model encapsulation representation file is used to transmit to the receiving device.

[0235] In some embodiments, a packaged model file includes a file header, and the one or more model data segments;

[0236] The file header includes a file header start code, a file header marker value, a file version, and the number of model data segments in the encapsulated model file; and / or,

[0237] A model data segment contains a model header and model data;

[0238] The model header includes a model header start code, a model header identifier, a verification message, and a model data size; wherein, the model header identifier is the same in multiple model data segments used to store data of the same model.

[0239] In some embodiments, the data of a complete model is stored in the same model data segment; or,

[0240] The data of a complete model is divided into at least two parts, and the data of the at least two parts are stored in different model data segments of the same packaged model file.

[0241] In some embodiments, the encapsulation unit 720 encapsulates the model file to obtain an encapsulated model file, including:

[0242] Perform the specified processing on the data of the model corresponding to the model file; wherein, the specified processing includes model compression processing and / or model encryption processing;

[0243] The data of the specified processed model is encapsulated to obtain the encapsulated model file.

[0244] In some embodiments, for different model data segments used to store data of the same model, the processing method of the specified processing of the data stored in each model data segment is the same or different.

[0245] In some embodiments, the model description file includes a model attribute information file and a model technical information file;

[0246] The model attribute information file is used to store model attribute information, which includes: model name and model size information;

[0247] The model technical information file is used to store model technical information, which includes: model version information, model data type information, hardware support information required by the model, basic environment information for running the model, model input information, and model output information.

[0248] Among them, the model input information includes the type of data input to the model.

[0249] In some embodiments, the model attribute information further includes one or more of the following information:

[0250] Model task information, model evaluation metric information, model open-source license, model production date, algorithm description file, and script description file; and / or,

[0251] The model technology information also includes one or more of the following information:

[0252] Model runtime configuration file, model runtime framework, model input resolution type, model converter input configuration file, pre-trained large model structure information, and model file information.

[0253] In some embodiments, the file to be packaged further includes a model program file; the model program file includes an execution library and / or a runtime script;

[0254] The step of generating a model encapsulation representation file based on the encapsulation model file and the model description file includes:

[0255] Generate a model encapsulation representation file based on the encapsulated model file, the model description file, and the model program file.

[0256] In some embodiments, the model program file includes an execution library, the execution library including a model converter for converting a standard model into a model used by a specified platform.

[0257] Please refer to Figure 8, which is a schematic diagram of a model unpacking and depackaging device provided in an embodiment of this application. As shown in Figure 8, the model unpacking and depackaging device 800 may include:

[0258] Acquisition unit 810, used to acquire model encapsulation representation file;

[0259] For example, the model encapsulation represents that the file is encapsulated using the manner described in the above method embodiments;

[0260] Decapsulation unit 820 is used to decapsulate the model encapsulation representation file to obtain a decapsulated file; wherein, the decapsulated file includes a model file and a model description file;

[0261] The determining unit 830 is used to determine the model for task processing based on the depackaged file; wherein the model is used to obtain inference output based on the model input.

[0262] This application provides an electronic device, including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor is used to execute the machine-executable instructions to implement the model encapsulation method or model decapsulation method described above.

[0263] Please refer to Figure 9, which is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device 900 may include a processor 901 and a memory 902 storing machine-executable instructions. The processor 901 and the memory 902 can communicate via a system bus 903. Furthermore, by reading and executing the machine-executable instructions in the memory 902 corresponding to the model encapsulation or model decapsulation logic, the processor 901 can execute the model encapsulation method or model decapsulation method described above.

[0264] The memory 902 mentioned herein can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage device (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0265] In some embodiments, a machine-readable storage medium is also provided, such as memory 902 in FIG9. The machine-readable storage medium stores machine-executable instructions, which, when executed by a processor, implement the model encapsulation method or model decapsulation method described above. For example, the storage medium may be ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0266] This application embodiment also provides a model repository sending device, including: a model pool, a device library, and a communication and control module; wherein:

[0267] The model pool is used to store standard model files;

[0268] The device library is used to store model converters corresponding to different model deployment platforms. The model converters are used to convert standard models into models that can be executed by the model deployment platform.

[0269] The communication and control module is used to encapsulate the standard model to be transmitted using the model encapsulation method described in the above embodiments, and transmit the obtained model encapsulation representation file to the receiving device.

[0270] In this embodiment of the application, in view of the problem of difficult model management under multiple models and multiple inference platforms, this embodiment of the application proposes a model warehouse management method, which, combined with the above-mentioned encapsulation and transmission process, further improves management efficiency and reduces storage requirements.

[0271] For example, the model pool of the model repository sending device stores standard models instead of storing platform-specific models.

[0272] In addition, in order for standard models to be deployed and run on various platforms, the model repository may also include a device library for storing model converters for different model deployment platforms.

[0273] For example, a model converter is used to convert a standard model into a model that can be executed by a model deployment platform.

[0274] Compared with the traditional storage method, assuming there are m models and n target devices, since the optimization and operators of each device platform are different, a total of m*n different model files need to be stored for a sender that supports all models and platforms.

[0275] However, when using the unified encapsulation standard and model repository management method provided in the embodiments of this application, the sending end only needs to store m standard model files. These standard model files are not designed for any specific device; instead, they are transmitted to the target device and then converted into device model files that can be run on that device. This significantly reduces the storage requirements of the sending end.

[0276] For example, the model repository sending device may also include a communication and control module, which can use the model encapsulation method described in the above embodiments to encapsulate the standard model to be transmitted and transmit the obtained model encapsulation representation file to the receiving device.

[0277] For example, the communication and control module of the model warehouse sending device can communicate with the receiving device to determine the model acquisition requirements of the receiving end.

[0278] For example, when a model repository sending device receives a model acquisition request sent by a receiving device, it can determine the model that the receiving device needs to acquire based on the model acquisition request, and send the standard model file corresponding to the determined model that the receiving device needs to acquire to the receiving device.

[0279] For example, the model repository sending device can also communicate with the receiving device through the communication and control module to determine whether the receiving device has a corresponding model converter. If it is determined that the receiving device does not have a corresponding model converter, the model converter and the standard model file are transmitted to the receiving device.

[0280] For example, the standard model files stored in the model pool of the model repository sending device include model files corresponding to complete models, and may also include model files corresponding to partial structures of models.

[0281] For example, if a partial structure update of a model occurs on the model repository sending device side, the model repository sending device can save the standard model file corresponding to the complete model before the update, as well as the standard model file corresponding to the partial structure of the updated model.

[0282] Among them, the model repository sending device can record the model version information corresponding to each standard model file.

[0283] For example, the model version information of the standard model file (assuming it corresponds to model A) corresponding to the complete model before the update can be v1.0, and the model version information of the standard model file corresponding to a part of the structure of the updated model can be v1.1.

[0284] When the model repository sending device receives a model update request sent by the receiving device, it can determine the standard model file to be transmitted based on the model version information carried in the model update request.

[0285] For example, suppose the receiving device sends a model update request to update model A, and the model version information carried in the model update request is v1.0, that is, the receiving device already has a complete model of version v1.0 locally. In this case, the model repository sending device can transmit the standard model file corresponding to the partial structure of the updated model of version v1.1 (whose model version information is v1.1) to the receiving device. The receiving device then updates its local model according to the standard model file corresponding to the partial structure of the updated model to obtain the updated complete model.

[0286] It should be noted that, in this embodiment of the application, when the model repository sending device receives a model download request from the receiving device, the model repository sending device can query and determine the standard model files of different versions of the model stored in the model pool. If the updated standard model file is a standard model file corresponding to a part of the model's structure, the model repository sending device can send both the standard model file corresponding to the complete model of the old version and the standard model file corresponding to the part of the structure of the new version to the receiving device. The receiving device can then obtain the latest version of the complete model based on the standard model file corresponding to the complete model of the old version and the standard model file corresponding to the part of the structure of the new version. Alternatively, the model repository sending device can obtain the standard model file corresponding to the latest version of the complete model based on the standard model file corresponding to the complete model of the old version and the standard model file corresponding to the part of the structure of the new version, and send it to the receiving device.

[0287] In some embodiments, the model repository sending device may further include a designated processing module for performing designated processing on the model data.

[0288] For example, the specified processing may include model compression and / or model encryption.

[0289] For example, the specific implementation method of the model warehouse sending end device in the model encapsulation process for specifying the processing of model data can be found in the relevant description in the above-mentioned model encapsulation method embodiment, and will not be repeated here in the embodiments of this application.

[0290] For example, the structure of the model repository sender 1000 can be as shown in Figure 10, mainly including (taking model encryption as an example of specified processing):

[0291] Model Pool 1001: Stores files of different standard models (standard models A to C in Figure 10). Standard models A to C are unified representation models and do not depend on any specific device for execution.

[0292] Equipment Library 1002: Stores model converters for the required deployment devices, converting the unified representation model (i.e., the standard model) into an executable model on the target device, which can be transmitted along with the package;

[0293] For example, as shown in FIG10, the device X converter can be used to convert the unified representation model into a special model of device X; the device Y converter can be used to convert the unified representation model into a special model of device Y; and the device Z converter can be used to convert the unified representation model into a special model of device Z.

[0294] Encryption / decryption module 1003: Primarily used at the sending end to generate keys and encrypt data;

[0295] Communication and Control Module 1004: Used to implement the model encapsulation process, establish communication with other sending and receiving ends, and transmit the model.

[0296] As can be seen, by separating the model and the converter, the model repository sender does not need to store the dedicated model files for the corresponding deployment platform, but can store standard model files, which effectively reduces the number of model files stored on the model repository sender, saves storage space, and reduces management difficulty.

[0297] For example, the communication and control module of the above-mentioned model repository sending device may include: a software module, a hardware module, or a combination of both. In the embodiments, the implementation form of the communication and control module is not specifically limited, as long as it can achieve the functions described in the embodiments herein. For example, the communication and control module of the above-mentioned model repository sending device may be program code, a processor executing the program code, a chip integrating communication and software control functions, a computer, an electronic device, etc.

[0298] This application embodiment also provides a model repository receiving device, including: a model pool, a device library, and a communication and control module; wherein:

[0299] The communication and control module is used to receive the model encapsulation representation file sent by the transmitting device, and to decapsulate the received model encapsulation representation file using the model decapsulation method described in the above embodiments to obtain a standard model;

[0300] The device library is used to store model converters corresponding to this device, and the model converters are used to convert the standard model into a model executable by this device;

[0301] The model pool is used to store the executable models of the device.

[0302] For example, the specific implementation of the model repository receiving device obtaining the model encapsulation representation file and performing model decapsulation can be found in the relevant descriptions in the above method embodiments and the relevant descriptions in the implementation of the model repository sending device. This application embodiment will not repeat the details here.

[0303] In some embodiments, the model repository receiving device may further include a designated processing module for inverse processing of the model to perform designated processing.

[0304] For example, when the specified process includes model compression, the inverse process of the specified process includes model decompression; when the specified process includes model encryption, the inverse process of the specified process includes model decryption.

[0305] For example, the structure of the model repository receiver 1100 can be as shown in Figure 11, mainly including (taking model encryption as an example of specified processing):

[0306] Model Pool 1101: Includes various model files that can be run on the receiving end, which are converted from a unified representation model;

[0307] For example, as shown in Figure 11, model A_X can be obtained by converting standard model A; model B_X can be obtained by converting standard model B; and model C_X can be obtained by converting standard model C.

[0308] Equipment Library 1102: Stores the model converter corresponding to this device, used to convert the unified representation model into a model that can be executed by this device;

[0309] Encryption / decryption module 1103: mainly used at the receiving end to decrypt encrypted model data;

[0310] Communication and Control Module 1104: Used to implement the model decapsulation process, establish communication with other sending and receiving ends, and receive the model.

[0311] For example, the communication and control module of the above-mentioned model repository receiving device may include: a software module, a hardware module, or a combination of both. In the embodiments, the implementation form of the communication and control module is not specifically limited, as long as it can achieve the functions described in the embodiments herein. For example, the communication and control module of the above-mentioned model repository receiving device may be program code, a processor executing the program code, a chip integrating communication and software control functions, a computer, an electronic device, etc.

[0312] For example, a schematic diagram of a model sending end (hereinafter referred to as the sender) sending a model to a model receiving end (hereinafter referred to as the receiver) can be shown in Figure 12.

[0313] As shown in Figure 12, assuming that the receiving end X requests model A_X from the sending end 1, the sending end 1 can encapsulate the standard model file corresponding to the standard model A corresponding to model A_X in the manner described in the above embodiment, and send it to the receiving end X together with the model converter (such as converter X) corresponding to the receiving end X.

[0314] The receiver X can use the converter X to convert the standard model A into a locally executable model A_X.

[0315] Assuming that the receiving end Z has a local model converter (used to convert the unified representation model into a locally executable model), when the receiving end Z requests model C_Z from the sending end 2, the sending end 2 can encapsulate the standard model file corresponding to the standard model C corresponding to model C_Z in the manner described in the above embodiment and send it to the receiving end Z.

[0316] The receiver Z can use a local converter to convert the standard model C into a locally executable model C_Z. [Simplified Explanation of the Diagram]

[0033] Figure 1 is a flowchart illustrating a model encapsulation method according to an exemplary embodiment of this application. Figure 2 is a flowchart illustrating a model decapsulation method according to an exemplary embodiment of this application. Figure 3 is a structural diagram illustrating a model encapsulation representation file according to an exemplary embodiment of this application. Figure 4 is a flowchart illustrating an overall model encapsulation distribution according to an exemplary embodiment of this application. Figure 5 is a flowchart illustrating a model decapsulation according to an exemplary embodiment of this application. Figure 6 is a schematic diagram illustrating a model partial update transmission process according to an exemplary embodiment of this application. Figure 7 is a structural diagram illustrating a model encapsulation device according to an exemplary embodiment of this application. Figure 8 is a structural diagram illustrating a model decapsulation device according to an exemplary embodiment of this application. Figure 9 is a hardware structure diagram illustrating an electronic device according to an exemplary embodiment of this application. Figure 10 is a structural diagram illustrating a model repository sending end according to an exemplary embodiment of this application. Figure 11 is a structural diagram illustrating a model repository receiving end according to an exemplary embodiment of this application. Figure 12 is a schematic diagram illustrating model sending and receiving according to an exemplary embodiment of this application.

Claims

1. A model encapsulation method, comprising: Obtain a file to be packaged, wherein the file to be packaged includes a model file and a model description file, wherein one of the model files contains data of a complete model, or data of a partial structure of the model; Package the model file to obtain a packaged model file, wherein one of the packaged model files contains one or more model data segments, one of the model data segments is used to store the complete data of the model or the partial data of the model; the multiple model data segments used to store the data of the same model are set with the same identifier, wherein one of the model data segments contains a model header and model data, wherein the model header includes a model header start code, a model header identifier, a verification message and the model data size, wherein the model header identifier is the same among the multiple model data segments used to store the data of the same model; Generate a model packaged representation file based on the packaged model file and the model description file, wherein the model packaged representation file is used to transmit to the receiving device.

2. As described in claim 1, a packaged model file includes a file header and one or more model data segments; the file header includes a file header start code, a file header flag value, a file version, and the number of model data segments in the packaged model file.

3. As described in claim 2, the data of a complete model is stored in the same model data segment; or, the data of a complete model is divided into at least two parts, and the model data of the at least two parts are stored in different model data segments of the same packaged model file.

4. The method as described in claim 1, wherein encapsulating the model file to obtain an encapsulated model file includes: The data of the model corresponding to the model file is subjected to specified processing, wherein the specified processing includes model compression processing and / or model encryption processing; And encapsulate the data of the specified processed model to obtain the encapsulated model file.

5. As described in claim 4, for different model data segments used to store data of the same model, the processing method of the specified processing of the data stored in each of the model data segments is the same or different.

6. The method as described in claim 1, wherein the model description file includes a model attribute information file and a model technical information file; the model attribute information file is used to store model attribute information, the model attribute information including: Model name and model size information; The model technical information file is used to store model technical information, which includes: model version information, model data type information, hardware support required by the model, basic environment information for running the model, model input information, and model output information; wherein, the model input information includes the type of data input to the model.

7. As described in Request 6, the model attribute information further includes one or more of the following: model task information, model evaluation metric information, model open-source license, model production date, algorithm description file, and script description file; and / or, the model technical information further includes one or more of the following: model runtime configuration file, model runtime framework, model input resolution type, model converter input configuration file, pre-trained large model structure information, and model file information.

8. The method as described in claim 1, wherein the file to be packaged further includes a model program file; the model program file includes an execution library and / or a runtime script; the step of generating a model package representation file based on the packaged model file and the model description file includes: Based on the encapsulated model file, the model description file, and the model program file, a model encapsulated representation file is generated.

9. The method as described in claim 8, wherein the model file includes an execution library, the execution library includes a model converter for converting a standard model into a model for use on a specified platform.

10. A method for decapsulating a model, comprising: Obtain the model encapsulation representation file, wherein the model encapsulation representation file is encapsulated using the method described in any one of requests 1-9; The model encapsulation representation file is decapsulated to obtain a decapsulated file, wherein the decapsulated file includes a model file and a model description file; a model for task processing is determined based on the decapsulated file, wherein the model is used to obtain inference output based on model input.

11. A model packaging device, comprising: An acquisition unit is used to acquire a file to be packaged, wherein the file to be packaged includes a model file and a model description file, wherein one model file contains data of a complete model or data of a partial structure of the model; an packaging unit is used to package the model file to obtain a packaged model file, wherein one packaged model file contains one or more model data segments, one model data segment is used to store the complete data of a model or the partial data of the model; multiple model data segments used to store data of the same model are set with the same identifier, one model data segment includes a model header and model data, wherein the model header includes a model header start code, a model header identifier, a verification message and a model data size, wherein the model header identifier is the same among the multiple model data segments used to store data of the same model; a generation unit is used to generate a model packaged representation file based on the packaged model file and the model description file, wherein the model packaged representation file is used to transmit to the receiving device.

12. A model de-encapsulation device, comprising: The acquisition unit is used to acquire a model encapsulation representation file, wherein the model encapsulation representation file is encapsulated using the method described in any one of requests 1-9; the decapsulation unit is used to decapsulate the model encapsulation representation file to obtain a decapsulated file, wherein the decapsulated file includes a model file and a model description file; the determination unit is used to determine a model for task processing based on the decapsulated file, wherein the model is used to obtain inference output based on model input.

13. An electronic device comprising a processor and memory, the memory storing machine-executable instructions executable by the processor, the processor being configured to execute the machine-executable instructions to implement the method of any one of claims 1-9 or the method of claim 10.

14. A machine-readable storage medium storing machine-executable instructions that, when executed by a processor, implement the method of any one of claims 1-9 or the method of claim 10.

15. A model repository sending device, comprising: The system includes a model pool, a device library, and a communication and control module; wherein: the model pool is used to store standard model files; the device library is used to store model converters corresponding to different model deployment platforms, the model converters being used to convert standard models into models executable by the model deployment platform; and the communication and control module is used to encapsulate the standard model to be transmitted using the method described in any one of requests 1-9, and transmit the resulting encapsulated model representation file to the receiving device.

16. A model repository receiving device, comprising: The system comprises a model pool, a device library, and a communication and control module; wherein: the communication and control module is used to receive a model encapsulation representation file sent by a sending device, and to decapsulate the received model encapsulation representation file using the method described in request item 10 to obtain a standard model; the device library is used to store model converters corresponding to the receiving devices of the model library, and the model converters are used to convert the standard model into a model executable by the receiving devices of the model library; the model pool is used to store the models executable by the receiving devices of the model library.

Images