Model serialization and deserialization methods and devices, electronic equipment and storage medium

By segmenting and serializing the object files of the large language model and encrypting and decrypting them in combination with hardware features, the problem of low storage and deployment efficiency caused by excessive serialization of the large language model is solved, and the file size reduction and deployment are achieved Speed ​​improvement.

CN119003454APending Publication Date: 2024-11-22SHANGHAI BIREN TECH CO LTD
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411216852.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The large language model generates too large files during the serialization and deserialization process, resulting in increased storage costs and reduced deployment efficiency, and the encryption and decryption process takes a long time, affecting the deployment and inference performance of the model.

Method used

By segmenting the target files of the model, serializing the split files based on the preset serialization format and hardware characteristics, and generating unique identifiers and digital encodings for each file, realizing file encryption and decryption, and optimizing data layout To reduce file size and improve encryption and decryption efficiency.

Benefits of technology

It effectively reduces the size of the file after model serialization, avoids storage and deployment problems caused by excessive files, solves the confusion problem of multiple models and multiple files, and improves the deployment and inference speed of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003454A_ABST
    Figure CN119003454A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of artificial intelligence, and provides a model serialization and deserialization method and device, electronic equipment and a storage medium, and the method comprises the steps: segmenting a target file of a model based on the size of the model to obtain a plurality of segmented files, the target file being a file with the file size exceeding a preset threshold value in the model; on the basis of a preset serialization format, serializing the plurality of segmented files and non-target files of the model to obtain a plurality of initial serialized files; generating a model identifier, and writing the model identifier as a first byte into each initial serialized file; and performing digital coding on each initial serialization file, and writing a digital code as a second byte into each initial serialization file to obtain a serialization file of the model. According to the method, the size of the file after model serialization can be reduced, the problem that the model cannot be stored and deployed due to the fact that a single file is too large is avoided, and meanwhile the problem of file confusion during multi-model management can be solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a model serialization and deserialization method, device, electronic device and storage medium. Background Art

[0002] In the field of artificial intelligence and machine learning, with the rise of large language models, these models have attracted much attention for their excellent text generation and understanding capabilities. However, these models are backed by huge computational workloads and complex model parameters, which directly leads to many challenges in the process of model deployment, storage, and transmission. Serialization and deserialization, as key technologies for model deployment, storage, exchange, and persistence in different systems or environments, are crucial for the widespread application of large language models.

[0003] Although serialization and deserialization are crucial for the application of large language models, there are still some defects in practical applications. Due to the large number of parameters, the serialized files generated by large language models are often very large, which not only takes up a lot of disk space but also increases storage costs. At the same time, the reading and writing operations of files also become more time-consuming due to the increase in file size, affecting the deployment efficiency of the model. Summary of the invention

[0004] The present invention provides a model serialization and deserialization method, device, electronic device and storage medium, which are used to solve the defects in the related art that the file after model serialization is too large, the storage cost is increased, and the deployment efficiency is affected.

[0005] The present invention provides a model serialization method, comprising: Based on the model size, the target file of the model is segmented to obtain a plurality of segmented files, wherein the target file is a file in the model whose file size exceeds a preset threshold; Based on a preset serialization format, serialize the multiple segmented files and the non-target files of the model to obtain multiple initial serialized files; Generate a model identifier, and write the model identifier as the first byte into each initial serialization file; Each initial serialization file is digitally encoded, and the digital encoding is written into each initial serialization file as the second byte to obtain a serialization file of the model.

[0006] A model serialization method provided by the present invention further includes: Based on the hardware characteristics, any serialized file is encrypted to obtain an encrypted serialized file corresponding to the any serialized file.

[0007] According to a model serialization method provided by the present invention, the hardware characteristics include the number of hardware computing cores, and based on the hardware characteristics, any serialization file is encrypted to obtain an encrypted serialization file corresponding to the any serialization file, including: Based on the number of hardware computing cores and the storage type of model data, the data in any serialized file is divided into a plurality of data blocks; Based on the data arrangement of the model layer corresponding to any serialized file, the data in each data block is rearranged to obtain an encrypted serialized file corresponding to any serialized file.

[0008] According to a model serialization method provided by the present invention, the data in any serialization file is divided into multiple data blocks based on the number of hardware computing cores and the storage type of model data, including: Determining a data alignment target based on a storage type of the model data; If any of the serialized files does not meet the data alignment target, performing data padding on any of the serialized files; Based on the number of hardware computing cores, any serialized file filled with data is divided into multiple data blocks.

[0009] According to a model serialization method provided by the present invention, the data arrangement of the model layer corresponding to any serialization file is based on which the data in each data block is rearranged to obtain an encrypted serialization file corresponding to any serialization file, including: Along the specified dimension direction of the data arrangement, data in any data block is divided based on a preset block size to obtain a plurality of blocks; When the size of the last block in the blocks does not reach the preset block size, filling the last block with data; The multiple cut blocks are arranged based on the cut-up order of the multiple cut blocks to obtain the encrypted serialized file.

[0010] According to a model serialization method provided by the present invention, the target file of the model is segmented based on the model size to obtain multiple segmented files, including: Based on the model size, the preset file size and the model layer, the target file of the model is segmented to obtain the multiple segmented files.

[0011] The present invention also provides a model deserialization method, comprising: Reading the first byte and the second byte of each serialized file, wherein the serialized file is obtained based on any of the above-described model serialization methods; Based on the first byte of each serialized file, determine the model identifier corresponding to each serialized file, and apply the model identifier to determine all serialized files to be parsed; Based on the second byte of each serialized file to be parsed, the digital code of each serialized file to be parsed is determined, and each serialized file to be parsed is deserialized according to the order of the digital codes to obtain a deserialized file of the model.

[0012] A model deserialization method provided by the present invention also includes: Based on hardware characteristics, the deserialized file is decrypted to obtain model data, and the model data is applied to perform reasoning.

[0013] The present invention also provides a model serialization device, comprising: A file segmentation unit, configured to segment a target file of the model based on the model size to obtain a plurality of segmented files, wherein the target file is a file in the model whose file size exceeds a preset threshold; A serialization unit, configured to serialize the plurality of segmented files and the non-target files of the model based on a preset serialization format to obtain a plurality of initial serialization files; An identifier writing unit, used for generating a model identifier and writing the model identifier as the first byte into each initial serialization file; The encoding writing unit is used to digitally encode each initial serialization file and write the digital code as the second byte into each initial serialization file to obtain the serialization file of the model.

[0014] The present invention also provides a model deserialization device, comprising: A reading unit, used for reading the first byte and the second byte of each serialized file, wherein the serialized file is obtained based on any one of the above-mentioned model serialization methods; A determination unit, configured to determine a model identifier corresponding to each serialized file based on the first byte of each serialized file, and determine all serialized files to be parsed by applying the model identifier; The deserialization unit is used to determine the digital code of each serialized file to be parsed based on the second byte of each serialized file to be parsed, and deserialize each serialized file to be parsed in the order of the digital code to obtain a deserialized file of the model.

[0015] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements any one of the above-mentioned model serialization methods or any one of the above-mentioned model deserialization methods.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned model serialization methods or any of the above-mentioned model deserialization methods.

[0017] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned model serialization methods or any of the above-mentioned model deserialization methods.

[0018] The model serialization and deserialization method, device, electronic device and storage medium provided by the present invention can reduce the file size by segmenting the target file of the model based on the model size, so that after the segmented file and the non-target file of the model are serialized, the obtained serialized file will not be too large, thereby reducing the size of the file after the model is serialized, and avoiding the problem that a single file is too large to be stored and deployed. In addition, by generating a unique model identifier for each model and writing it as the first byte into each serialized file, the confusion problem of multiple models and multiple files can be effectively solved, so that when managing and maintaining multiple models, each serialized file can be quickly and clearly identified to which model it belongs, thereby facilitating the deployment and reasoning application of subsequent models. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of a model serialization method provided by the present invention; Figure 2 It is a schematic diagram of slicing based on data arrangement provided by the present invention; Figure 3 It is a flow chart of the model deserialization method provided by the present invention; Figure 4 It is a schematic diagram of the process of serializing a large language model provided by the present invention; Figure 5 It is a schematic diagram of the process of deserialization of a large language model provided by the present invention; Figure 6 It is a structural schematic diagram of the model serialization device provided by the present invention; Figure 7 It is a structural schematic diagram of the model deserialization device provided by the present invention; Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Model serialization refers to converting the model's weights, parameters and other data structures into a format that can be stored for a long time (such as binary format) so that it can be stored or transmitted to another environment. Here, another environment refers to a computing environment or physical environment that is different from the current model training or running environment. For example, after the model is trained on a high-performance computing server, it may need to be deployed to another server for production or testing. At this time, the serialized model can be transferred to the target server and reloaded and used there. By serializing the model, the trained model can be saved to a hard disk or other persistent storage device so that it can be reloaded and used when needed without having to re-do the time-consuming training process. Model deserialization is the inverse operation of model serialization, that is, restoring the storage or transmission format to the original model structure and data. For large language models, effective serialization and deserialization mechanisms can significantly improve the flexibility and scalability of the model.

[0023] However, due to the large amount of computation and model parameters of large language models, the use of serialization methods in related technologies (such as JSON and XML) will cause the generated serialized files to be too large, resulting in a waste of disk space and increased storage costs. At the same time, file read and write operations also become more time-consuming due to the increase in file size, affecting the deployment efficiency of the model. In addition, in order to improve data security, data is usually encrypted during the model serialization process, but complex encryption algorithms are often accompanied by high computational costs, resulting in a long encryption and decryption process, which affects the deployment and use of the model.

[0024] In this regard, the present invention provides a model serialization method, which avoids the problem of a single file being too large by splitting the file, and generates a unique model identifier for each model, which can solve the confusion problem when there are multiple models and multiple files. At the same time, by integrating hardware characteristics and model encryption and decryption technology, not only can the model be encrypted, but the speed of model reasoning can also be greatly improved, thereby overcoming the above defects.

[0025] Figure 1It is a flow chart of the model serialization method provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, based on the model size, segmenting the target file of the model to obtain a plurality of segmented files, wherein the target file is a file in the model whose file size exceeds a preset threshold; Specifically, before serializing the model, the target file of the model can be split according to the model size to reduce the file size and avoid a single file being too large. Here, in the field of machine learning and artificial intelligence, model size refers to the size of the storage space required to save the neural network model, which can be expressed in units such as MB and GB. The model size is directly related to the number of parameters in the model, including weights and biases, which are stored in the model in numerical form. The larger the model, the more parameters it contains and the more complex data and tasks it can handle.

[0026] The target file of the model refers to the file whose file size exceeds the preset threshold in the model. These files can be model parameter files, weight files, or other key data files related to the operation of the model. Since large language models usually contain a large number of parameters and data, the corresponding files may be very large. In order to prevent the size of the file after the model is serialized from exceeding the system limit, the embodiment of the present invention proposes a strategy for segmenting the target file.

[0027] Specifically, the target file can be segmented by the following steps: First, a preset threshold of the file size can be set, which can be determined according to the actual application scenario, storage system limitations, and performance requirements. Then, all files in the model are traversed to find those files whose file size exceeds the preset threshold as target files. For each target file, multiple strategies can be used for segmentation. One strategy is to segment according to a fixed size (such as 2G, 4G, etc.), dividing a large target file into multiple small files. Another strategy is to segment according to the logical structure of the file content. For example, the model parameters can be divided according to the model layer. It should be understood that in the large language model, the model layer refers to the internal components of the model, which are responsible for processing input data and generating output. For example, the model layer of the large language model may include an embedding layer, an encoding layer, a decoding layer, an attention layer, and the like.

[0028] It is understandable that for large language models, weight files often occupy the vast majority of the model files, while other types of files (such as model configuration files) are relatively much smaller. Therefore, when splitting files, usually only the model weight files need to be split.

[0029] Step 120, serializing the plurality of segmented files and the non-target files of the model based on a preset serialization format to obtain a plurality of initial serialized files; Specifically, after the file segmentation is completed, the segmented files and non-target files of the model can be serialized according to the preset serialization format. Here, non-target files refer to files in the model whose file size does not exceed the preset threshold. These files do not need to be segmented, but they also need to be serialized for storage or transmission.

[0030] The preset serialization format refers to the convention or specification for converting the data structure or object state of the model into a series of bytes during the serialization process. In an embodiment of the present invention, the preset serialization format is the format of the serialization file predefined by Protobuf (Protocol Buffers, a cross-platform, cross-language serialization tool) according to the needs of the large language model inference engine, which includes the model data to be serialized and the representation of these model data in the serialization file. It should be understood that when the preset serialization format is defined using Protobuf, the Protobuf syntax is first used to define the data structure to be serialized (such as model weights, parameters, etc.), and then the Protobuf compiler is used to compile these data structures into source code of the required programming language. These source codes are the preset serialization formats, which include functions and methods for serializing and deserializing data. Subsequently, according to the preset serialization format, the file can be serialized.

[0031] Specifically, the data to be serialized is first loaded, including multiple segmentation files and non-target files of the model; then, according to the preset serialization format, the corresponding serialization object is created or the corresponding serialization function is called, and the loaded data is written to the serialization object, or the data is processed using the serialization function to convert the data into a byte stream; finally, the serialized byte stream is written to the file to obtain the initial serialization file. Here, the initial serialization file refers to the file obtained after serialization processing, which contains the serialized data (ie, byte stream). In subsequent steps, these initial serialization files will be further processed (such as adding model identifiers and digital codes) to obtain the final serialization files.

[0032] It can be understood that the serialization of target files and non-target files can be performed independently. After identifying the target files and non-target files of the model, the non-target files can be serialized directly to generate the corresponding initial serialization files. For the target files, they can be split first, and then the split files can be serialized, and the save path of the generated initial serialization files can be specified to save the initial serialization files corresponding to the target files and non-target files together.

[0033] Step 130, generating a model identifier, and writing the model identifier as the first byte into each initial serialization file; It should be noted that in order to facilitate the subsequent rapid finding of the serialized file of the required model and deserialization in the correct order and manner, some metadata, such as model identification, file number, etc., can be added to the specified position (such as the beginning or end) of each initial serialized file during the process of generating the initial serialized file or after the initial serialized file is obtained.

[0034] Specifically, the model identifier is an identifier used to uniquely identify different models. During the serialization process, the model identifier can help distinguish and manage the data files of multiple models, ensuring that the corresponding data can be correctly identified and used when deserializing or loading the model. When generating a model identifier, an ID (Identity) generator can be used to implement it. Here, the ID generator refers to a tool or algorithm for generating a unique identifier. It should be understood that when using the ID generator to generate a unique identifier for a model, it can be generated based on some parameters of the model, such as model architecture parameters (such as the number of model layers, the type of each layer, etc.), model training parameters (such as learning rate, batch size, training rounds, etc.), model feature parameters (such as the number of attention heads), other auxiliary parameters (such as timestamps), etc. Not all of the above parameters need to be used to generate a unique identifier for the model. In actual operation, some of the parameters can be selected to generate a unique identifier of a fixed length.

[0035] After the model identifier is generated, the identifier can be written as the first byte to each initial serialized file. Here, in order to quickly read the model identifier of each serialized file when the model is deserialized, the model identifier can be written as the first byte to a specified position of the initial serialized file, such as the header of the file. In this case, the first byte is the header byte. The header byte refers to a portion of bytes at the beginning of the file, which is usually used to store metadata. The header of the initial serialized file refers to a portion of the area at the beginning of the file, which can be used to store the written model identifier. When reading or processing a serialized file, the header byte is first read to obtain the metadata information of the file, such as the model identifier, and then subsequent processing or verification is performed based on this.

[0036] Specifically, after the model identifier is generated, the model identifier can be converted into a byte sequence so that it can be written into the initial serialization file. When creating or opening each initial serialization file, the byte sequence of the model identifier is written into the beginning of the file (ie, the header bytes).

[0037] Step 140, digitally encode each initial serialization file, and write the digital code as the second byte into each initial serialization file to obtain the serialization file of the model.

[0038] Specifically, digital encoding refers to assigning a unique digital number (or digital identifier) ​​to each initial serialized file. The digital number can be an integer and is used to provide additional differentiation and sorting capabilities among multiple serialized files. By digitally encoding each initial serialized file, it can be ensured that for a single large language model, each serialized file has a unique identifier, which is easy to manage and track, and the files can be sorted and processed in digital order when necessary. It should be understood that the generation of digital encoding can be implemented using a simple counter. For example, an incremental counter can be used to assign a unique digital number to each file.

[0039] When a unique digital code is assigned to each initial serialized file, the digital code can be written as the second byte into the initial serialized file. Here, in order to facilitate the rapid reading of the digital code of each serialized file when the model is deserialized, the digital code can also be written as the second byte into the specified position of the initial serialized file, such as the tail of the file, in which case the second byte is the tail byte. The tail byte refers to a portion of bytes at the end of the file, corresponding to the head byte. The tail of the initial serialized file refers to a portion of the area at the end of the file, which can be used to store the written digital code. Specifically, the digital code can be converted into a byte sequence, and then the byte sequence of the digital code is written as the tail byte at the end of the serialized file. It should be understood that the model identifier can also be written to the tail of the file, and the digital code can be written to the head of the file. Therefore, the above-mentioned first byte can also be a tail byte, and the second byte can be a head byte. The embodiment of the present invention does not specifically limit this.

[0040] In the above steps, the final model serialization file can be obtained through the steps of segmentation, serialization, adding model identification and digital encoding. These files can be stored on disk, transmitted over the network, or used in other scenarios where model data is required.

[0041] The method provided by the embodiment of the present invention can reduce the file size by segmenting the target file of the model based on the model size, so that after the segmented file and the non-target file of the model are serialized, the obtained serialized file will not be too large, thereby reducing the size of the file after the model is serialized, and avoiding the problem that a single file is too large to be stored and deployed. In addition, by generating a unique model identifier for each model and writing it as the first byte into each serialized file, the confusion problem of multiple models and multiple files can be effectively solved, so that when managing and maintaining multiple models, each serialized file can be quickly and clearly identified to which model it belongs, thereby facilitating the deployment and reasoning application of subsequent models.

[0042] Based on any of the above embodiments, step 110 specifically includes: Based on the model size, the preset file size and the model layer, the target file of the model is segmented to obtain the multiple segmented files.

[0043] Specifically, when the target file of the model (such as the weight file) is segmented, it can be implemented according to the model size, preset file size and model layer. Here, the preset file size refers to the maximum size that each segmented file should reach in advance during the model file segmentation process. The size can be a specific value based on units such as bytes, MB, GB, etc., which is used to control the size of the segmented file for storage, transmission or processing. The selection of the preset file size depends on many factors, including the total size of the model, the limitations of the target platform, processing performance, etc. For example, the preset file size can be set to 2G, 4G, etc. according to actual needs.

[0044] It is understandable that before performing file segmentation, the model size can be determined and a reasonable preset file size can be set, wherein the model size can be obtained by viewing the model file properties or using a special tool. The preset file size can be set according to the storage, transmission or processing requirements, and it should not be set too large (to avoid exceeding the limitations of the target platform) nor too small (to avoid generating too many segmented files and increasing the complexity of management).

[0045] Subsequently, the model can be segmented according to the determined model size, preset file size, model layer, etc. For example, the model can be segmented directly according to the preset file size, that is, the target file of the model is divided into N parts, and the number N is determined by dividing the model size by the preset file size. This segmentation method is simple and direct. For another example, the model can be segmented according to the structure of the layer, that is, each segmented file contains a complete layer or part of a layer to ensure the integrity and independence of the data in the layer. For another example, the preset file size and model layer can be combined for segmentation, that is, the model layer is first preliminarily segmented, and then the oversized layer is segmented according to the preset file size.

[0046] Based on any of the above embodiments, after step 140, the method further includes: Step 150: encrypt any serialized file based on hardware characteristics to obtain an encrypted serialized file corresponding to the any serialized file.

[0047] It should be noted that in the process of model storage, transmission and deployment, in order to improve the security of model data, serialized files can be encrypted according to hardware characteristics, which is an effective way to improve data security. By utilizing the encryption capabilities and uniqueness of hardware, the security of the encryption process can be increased, thereby protecting model data from being illegally obtained or tampered with.

[0048] Specifically, hardware characteristics refer to the unique properties or capabilities of a computer hardware system. For example, hardware characteristics can be the unique arrangement characteristics of a GPU (Graphics Processing Unit) or an ASIC (Application Specific Integrated Circuit). Here, hardware characteristics refer to the hardware characteristics of the target hardware on which the model will be deployed. This is because the encryption process optimizes data arrangement and segmentation based on the target hardware's parallel processing capabilities, memory access architecture, SPC (Streaming Processing Core) number and other characteristics, thereby ensuring that the encrypted data can be efficiently processed and calculated on the target hardware.

[0049] It is understandable that, considering that in a large language model, the weight file usually occupies the vast majority of the model file, and the weight file is also the main training file during model training, from a value perspective, the weight is the core value of a large language model. Therefore, when encrypting the serialized file of the model, only the serialized file of the weight can be encrypted. Other types of serialized files may be encrypted or not, and the embodiments of the present invention do not specifically limit this.

[0050] Exemplarily, when the serialized file of weights is encrypted according to the hardware characteristics, the weight data in the serialized file can be rearranged according to the memory access mode, parallel processing characteristics, and data arrangement characteristics of the GPU or ASIC, thereby obtaining an encrypted serialized file. Here, the above-mentioned encrypted serialized file refers to an encrypted model serialized file, and the content of these files is encrypted and protected during storage or transmission to prevent unauthorized access or tampering. It should be understood that by rearranging the weight data using hardware characteristics, the original arrangement of the data in the serialized file is changed, thereby playing an encryption role. At the same time, the weight data is rearranged according to the hardware characteristics of the target hardware, so that after decryption, it can be ensured that the data can be arranged in a manner that best suits the target hardware, thereby maximizing the utilization efficiency and computing performance of the hardware.

[0051] Based on any of the above embodiments, the hardware characteristics include the number of hardware computing cores, and step 150 specifically includes: Step 151, based on the number of hardware computing cores and the storage type of model data, dividing the data in any serialized file into a plurality of data blocks; Specifically, hardware characteristics may include the number of hardware computing cores. Here, the number of hardware computing cores refers to the number of cores that independently execute instructions or process data in a processor or similar computing hardware, such as a GPU, GPGPU (General-Purpose Graphics Processing Unit), TPU (Tensor Processing Unit), etc., such as the number of SPCs in a GPU.

[0052] The storage type of model data refers to the mode of managing and storing model data (such as weights) on a multi-processor or multi-core system to optimize performance and access speed. The model data here mainly refers to the model weight. For example, depending on the model, the weights of different models can be set to different storage types, such as NUMA (Non-Uniform Memory Access), UMA (Uniform Memory Access), UMA4, UMA8, etc. In the NUMA architecture, the memory is distributed among multiple processors (or nodes), and each processor can access its own local memory and access the memory of other processors through the interconnection network. The speed of accessing local memory is usually faster than accessing remote memory, so it is called "non-uniform". In the UMA architecture, all processors have the same speed to access any memory location because the memory is globally shared and there is no distinction between local and remote memory. UMA4 refers to a UMA architecture with four processors (or cores), and UMA8 refers to a UMA architecture with eight processors (or cores).

[0053] Specifically, when encrypting the weight serialization file, you first need to understand the number and performance of the hardware computing cores to determine the size of the data blocks that can be processed in parallel, and understand the storage type and structure of the model data to determine how to effectively split the data into multiple parts. Subsequently, based on the number of hardware computing cores and the storage type of the model data, the data in the weight serialization file can be split into multiple data blocks of appropriate sizes.

[0054] Step 152: based on the data arrangement of the model layer corresponding to any serialized file, rearrange the data in each data block to obtain an encrypted serialized file corresponding to any serialized file.

[0055] Specifically, after the data in the serialized file of the model weights is segmented, the data in each data block can be further rearranged according to the data arrangement characteristics of the model layer to change the original arrangement of the data in the serialized file, thereby achieving the encryption effect. Here, the data arrangement of the model layer refers to the organization and actual arrangement order of the data of each layer of the large language model in the hardware physical memory, including the dimension, shape, order, etc. of the data.

[0056] For example, the data arrangement of the weights of different layers of the model can be NCHW (batch size N, number of channels C, height H, width W), NHW, HW, N, etc. Different data arrangements will result in different order of data in memory, which has a significant impact on data access speed, cache utilization, and parallel computing performance. It should be understood that for the weights of different layers of the model, the corresponding data arrangement may be different. Therefore, when the serialized file of the weights is split, it can be split according to the model layer, so that when the data in the serialized file of the weights is rearranged, it can be implemented according to the data arrangement of the model layer corresponding to the serialized file.

[0057] In an embodiment of the present invention, by rearranging the data in each data block according to the data arrangement of the weights of different layers of the model, the layout of the data in the memory can be made more consistent with the access mode of the hardware. While achieving encryption, the large language model can have a faster advantage in the decryption and reasoning process.

[0058] Based on any of the above embodiments, step 151 specifically includes: Step 1511, determining a data alignment target based on the storage type of the model data; Specifically, the data alignment target refers to a normalized or standardized setting of the data storage method in the memory in order to optimize memory access efficiency and reduce cache misses during data processing or calculation. For example, according to different storage types, data needs to be aligned, such as 4096-byte alignment or 1024-byte alignment. Here, 4096 bytes or 1024 bytes are the goals to be achieved when aligning data.

[0059] Step 1512, when any of the serialized files does not meet the data alignment target, perform data padding on any of the serialized files; Specifically, in order to ensure that the data alignment goal is achieved, the serialized file can be judged before the weighted serialized file is split into different data blocks. For example, assuming that the data alignment is performed according to 4096 bytes, if the size of the serialized file cannot be divided by 4096, it means that the serialized file does not meet the data alignment goal. If it is split directly, the last data block will not be able to be aligned. In this case, the serialized file can be padded with data to ensure that all data blocks can be aligned after the split.

[0060] It is understandable that when padding the serialized file with data, the padding data can be arbitrary, such as 0 or other data. Random numbers can also be used for padding to further play the role of encryption obfuscation. By padding the data, it can be ensured that the data blocks in the serialized file meet the data alignment target, thereby improving data access efficiency and reducing cache misses.

[0061] Step 1513, based on the number of hardware computing cores, divide any serialized file after data filling into multiple data blocks.

[0062] Specifically, after the data is filled, the data in the weight serialization file can be divided into multiple data blocks according to the number of hardware SPCs (computing cores). For example, the data in the serialization file can be divided into N equal parts according to the number of hardware SOCs, where N can be set according to the number of hardware SPCs.

[0063] In the embodiment of the present invention, by performing data alignment, padding and segmentation operations on the serialized file, the storage and access efficiency of the data on the hardware can be optimized. At the same time, random numbers are used to fill the excess space as encryption obfuscation, thereby increasing the security of the data.

[0064] Based on any of the above embodiments, step 152 specifically includes: Step 1521, dividing the data in any data block based on a preset block size along the specified dimension direction of the data arrangement to obtain a plurality of blocks; Specifically, when rearranging the data in each data block after segmentation according to the data arrangement of weights of different model layers, the data in the data block can be segmented first along the specified dimension direction of the data arrangement, so that multiple segments can be arranged later, thereby achieving the rearrangement of the serialized data into different blocks. Here, the specified dimension direction of the data arrangement refers to the dimension or dimensions along which the data is segmented in the model data. For example, assuming that the data arrangement of the weights of a certain layer of the model is NHW, it can be segmented in the W dimension and H dimension directions, where the W dimension and H dimension are the specified dimensions.

[0065] The preset chunk size is a fixed value set before data segmentation, which specifies the amount of data (such as the number of bytes, the number of elements, etc.) that each chunk should contain. This value can be determined based on factors such as the capabilities of the hardware computing core, memory limitations, and data transmission bandwidth to optimize computing efficiency and memory usage.

[0066] Specifically, for any data block, when data is segmented, the dimension along which the data is segmented can be determined according to the specified dimension direction of the data arrangement, and the number of segments that can be segmented can be calculated according to the preset segment size and the total size of the specified dimension. Then, the data can be segmented along the specified dimension according to the preset segment size to obtain multiple segments.

[0067] Step 1522, when the size of the last block in the blocks does not reach the preset block size, fill the last block with data; Specifically, if the size of the last block does not reach the preset block size, it indicates that when the block is split along the specified dimension, the remaining amount of data is insufficient to form a complete block of the preset size. In this case, the last block can be padded with data to ensure that each block is aligned and further achieve the encryption effect. Specifically, when filling the last block with data, the difference between the last block and the preset block size can be determined first, that is, the amount of data to be filled; then, the filling data is generated according to the needs. The filling data can be arbitrary, such as 0 or other values, or it can be filled with random numbers for confusion; finally, the filling data is added to the end of the last block until its size reaches the preset block size.

[0068] Step 1523: Arrange the multiple blocks based on the segmentation order of the multiple blocks to obtain the encrypted serialized file.

[0069] Specifically, the order of slicing multiple blocks refers to the order in which the blocks are created and processed when the data is slicing. This order is related to the traversal order of the specified dimension. For example, if the slicing is performed along the height dimension, the blocks will be created in the order from top to bottom or from bottom to top. By rearranging and saving multiple blocks in the slicing order, an encrypted serialized file can be formed.

[0070] Figure 2 is a schematic diagram of slicing based on data arrangement provided by the present invention, such as Figure 2 As shown, the data arrangement of the model layer weights is NHW. When rearranging the data in the data block, n blocks can be cut along the W dimension first, and then n blocks can be cut along the H dimension. The size of the block can be two-dimensional 4×4, or other shapes. The size of n can be determined according to the size of the data block in the W and H dimensions and the size of the block. When the nth block is cut, due to insufficient data volume, data needs to be filled. At this time, random data can be filled for encryption obfuscation. Similarly, data needs to be filled for obfuscation when the H dimension is cut. It should be understood that when arranging the data for other model layer weights, it will also be cut according to the cut dimension and direction and the size of the block, and the different data will be aligned and filled.

[0071] In an embodiment of the present invention, the data in each data block is rearranged and aligned to a certain number according to the data arrangement of different layers of the model. The entire rearrangement process changes the original arrangement of the data in the serialized file, and at the same time adds random data to participate in the obfuscation, which plays a role of encryption.

[0072] Based on any of the above embodiments, Figure 3 It is a flow chart of the model deserialization method provided by the present invention, such as Figure 3 As shown, the method includes: Step 310, reading the first byte and the second byte of each serialized file, wherein the serialized file is obtained based on any of the above-mentioned model serialization methods; Step 320: determining a model identifier corresponding to each serialized file based on the first byte of each serialized file, and applying the model identifier to determine all serialized files to be parsed; It should be noted that after serializing the file of the large language model, when the model needs to be loaded from the storage medium (such as hard disk, cloud storage, etc.) into the memory for reasoning or further training, the serialized model file can be deserialized. Through deserialization, the model data is converted from the storage format (such as binary, etc.) back to the data structure in the memory, so that the model can be recognized and used by the programming language or framework.

[0073] Specifically, when deserializing a model, since there may be multiple serialized files of the model, all serialized files in the folder may be traversed first, and the first byte and the second byte of each serialized file may be read to obtain the model identifier and digital code corresponding to each serialized file. Here, the first byte may be the header byte and the second byte may be the tail byte, or the first byte may be the tail byte and the second byte may be the header byte, so that the first byte and the second byte may be quickly read from the header and the tail of each serialized file, thereby improving efficiency.

[0074] According to the model identifier corresponding to each serialized file, it can be determined whether the serialized file is the serialized file of the large language model to be loaded, thereby obtaining all the serialized files to be parsed. Here, the serialized files to be parsed refer to the files determined to be required to be deserialized.

[0075] For example, suppose that model A generates 5 serialized files, and the corresponding digital codes are 1, 2, 3, 4, and 5 respectively. Model B also generates 5 serialized files, and the corresponding digital codes are also 1, 2, 3, 4, and 5 respectively. The serialized files of these two models are saved in the same folder. When deserializing, you can first traverse and obtain the header bytes and tail bytes of each serialized file, and determine which serialized files are the files corresponding to the model to be loaded according to the header bytes, and then read the serialized files with corresponding digital codes from the folder in turn according to the tail bytes of these serialized files for deserialization.

[0076] Step 330, based on the second byte of each serialized file to be parsed, determine the digital code of each serialized file to be parsed, and deserialize each serialized file to be parsed according to the order of the digital code to obtain a deserialized file of the model.

[0077] Specifically, after all the serialized files to be parsed are determined, their digital codes can be determined according to the second bytes of these serialized files, and the corresponding serialized files can be deserialized in sequence according to the order of the codes. Here, when deserializing, the main content of the file can be read and converted back to the original data structure or model instance using a predefined deserialization algorithm.

[0078] The method provided by the embodiment of the present invention identifies and processes serialized files through the first byte and the second byte, so that the method can flexibly process serialized files of different models, effectively solve the confusion problem when there are multiple models and multiple files, and can quickly and clearly identify which model each serialized file belongs to when loading and deploying models, avoiding errors. By deserializing in the order of digital coding, the processing flow can be optimized and performance problems caused by incorrect file order can be reduced.

[0079] Based on any of the above embodiments, after step 330, the method further includes: Based on hardware characteristics, the deserialized file is decrypted to obtain model data, and the model data is applied to perform reasoning.

[0080] Specifically, during the model serialization process, in order to improve data security, the serialized file will be encrypted according to the hardware characteristics. Therefore, when the model is deserialized, the deserialized file can also be decrypted according to the hardware characteristics to extract the model data. Here, model data refers to the data structure used to represent the machine learning model or algorithm that is decrypted and restored from the deserialized file. This data usually includes model parameters (such as weights and biases). Model data is necessary in the machine learning reasoning process and is used to generate prediction results based on input data.

[0081] Specifically, the decryption process is the reverse process of the above encryption process. When decrypting according to hardware characteristics, according to the hardware calculation and the number of cores and the storage type of the model weights, it is possible to calculate how many data blocks the data in the serialized file is divided into, and determine the data alignment target at the same time. According to the number of data blocks and the data alignment target, it is possible to calculate which data is randomly encrypted, so that some of the randomly encrypted data can be discarded and only a specific part of the data can be taken out. Similarly, according to the data arrangement of the model layer weights corresponding to the serialized file, as well as the shape and block size of different layers of the model, it is possible to calculate how many blocks each data block is divided into and which block is filled with random data. When taking data, the random data in the block can be discarded according to the above calculation, so as to take out a specific part of the data. Since the arrangement of the retrieved data already meets the arrangement required by the hardware calculation, the retrieved data can be directly given to the hardware SPC for use, and there is no need to sort the weights according to the data arrangement required by the hardware. Therefore, these model data can be directly applied for reasoning, thereby improving the overall reasoning speed of the model.

[0082] Based on any of the above embodiments, the present invention provides a method for serializing and deserializing a large language model. Figure 4 is a schematic diagram of the process of serializing a large language model provided by the present invention, Figure 5 It is a schematic diagram of the process of deserializing a large language model provided by the present invention. Figure 4 and Figure 5 As shown, the method includes: Step S1, according to the inference engine of the large language model, using Protobuf to define a method for generating a serialized file; Step S2, split the weight file of the large language model into N parts, where the value of N can be determined by dividing the size of the model by the preset file size. The preset file size here can be 4G to avoid a single file being too large, which makes the model unable to be transplanted across platforms; Step S3, according to the method for generating serialized files defined in step S1, serialize the segmented weight files and other types of files of the model to obtain a serialized file of the model; Step S4, using an ID generator to uniquely encode the model to obtain a model identifier, and write the model identifier into the header of all serialized files, and digitally encode all serialized files at the same time, and write the digital code into the tail of all serialized files; Step S5, according to the hardware characteristics, such as the unique arrangement characteristics of GPU or ASIC, the data in the serialized file of the model weight is rearranged, and the redundant space is filled with random numbers as encryption obfuscation to achieve the purpose of hardware encryption; Step S6, when deserializing the model, judging whether it is the same or the required large language model according to the header bytes of the serialized file, and then deserializing these serialized files according to the tail bytes of the serialized files of the determined large language model using a parser; Step S7, according to the hardware characteristics, decrypt the deserialized model file and take out the model data in the file. Since the arrangement of the model data already conforms to the hardware characteristics, the decryption process will be particularly fast and the weight sorting step is omitted. Therefore, these model data can be directly applied for reasoning, thereby improving the reasoning speed.

[0083] The method provided by the embodiment of the present invention can not only reduce the size of the file after serialization of the large language model, and solve the problem that the file is too large to be stored, but also because it is lighter and the encryption process integrates the characteristics of hardware data arrangement, the large language model has the advantage of faster decryption and reasoning.

[0084] Based on any of the above embodiments, Figure 6 is a schematic diagram of the structure of the model serialization device provided by the present invention, such as Figure 6 As shown, the device comprises: A file segmentation unit 610 is used to segment a target file of the model based on the model size to obtain a plurality of segmented files, wherein the target file is a file in the model whose file size exceeds a preset threshold; A serialization unit 620, configured to serialize the plurality of segmented files and the non-target files of the model based on a preset serialization format to obtain a plurality of initial serialization files; An identifier writing unit 630 is used to generate a model identifier and write the model identifier as the first byte into each initial serialization file; The encoding writing unit 640 is used to digitally encode each initial serialization file, and write the digital code as the second byte into each initial serialization file to obtain the serialization file of the model.

[0085] The device provided by the embodiment of the present invention can reduce the file size by segmenting the target file of the model based on the model size, so that after the segmented file and the non-target file of the model are serialized, the serialized file obtained will not be too large, thereby reducing the size of the file after the model is serialized, and avoiding the problem that a single file is too large to be stored and deployed. In addition, by generating a unique model identifier for each model and writing it as the first byte into each serialized file, the confusion problem of multiple models and multiple files can be effectively solved, so that when managing and maintaining multiple models, each serialized file can be quickly and clearly identified to which model it belongs, thereby facilitating the deployment and reasoning application of subsequent models.

[0086] Based on any of the above embodiments, the device further includes an encryption unit, and the encryption unit is used to: Based on the hardware characteristics, any serialized file is encrypted to obtain an encrypted serialized file corresponding to the any serialized file.

[0087] Based on any of the above embodiments, the hardware characteristics include the number of hardware computing cores, and the encryption unit specifically includes: A segmentation subunit, used for segmenting the data in any serialized file into a plurality of data blocks based on the number of hardware computing cores and the storage type of model data; The rearrangement subunit is used to rearrange the data in each data block based on the data arrangement of the model layer corresponding to any serialization file to obtain an encrypted serialization file corresponding to any serialization file.

[0088] Based on any of the above embodiments, the cutting subunit is specifically used for: Determining a data alignment target based on a storage type of the model data; If any of the serialized files does not meet the data alignment target, performing data padding on any of the serialized files; Based on the number of hardware computing cores, any serialized file filled with data is divided into multiple data blocks.

[0089] Based on any of the above embodiments, the rearrangement subunit is specifically used for: Along the specified dimension direction of the data arrangement, data in any data block is divided based on a preset block size to obtain a plurality of blocks; When the size of the last block in the blocks does not reach the preset block size, filling the last block with data; The multiple cut blocks are arranged based on the cut-up order of the multiple cut blocks to obtain the encrypted serialized file.

[0090] Based on any of the above embodiments, the file segmentation unit 610 is specifically used for: Based on the model size, the preset file size and the model layer, the target file of the model is segmented to obtain the multiple segmented files.

[0091] Based on any of the above embodiments, Figure 7 is a schematic diagram of the structure of the model deserialization device provided by the present invention, such as Figure 7 As shown, the device comprises: A reading unit 710 is used to read the first byte and the second byte of each serialized file, where the serialized file is obtained based on any of the above-mentioned model serialization methods; A determination unit 720 is used to determine a model identifier corresponding to each serialized file based on the first byte of each serialized file, and to determine all serialized files to be parsed by applying the model identifier; The deserialization unit 730 is used to determine the digital code of each serialized file to be parsed based on the second byte of each serialized file to be parsed, and deserialize each serialized file to be parsed in the order of the digital code to obtain a deserialized file of the model.

[0092] The device provided by the embodiment of the present invention identifies and processes serialized files through the first byte and the second byte, so that the method can flexibly process serialized files of different models, effectively solve the confusion problem when there are multiple models and multiple files, and can quickly and clearly identify which model each serialized file belongs to when loading and deploying models, avoiding errors. By deserializing in the order of digital coding, the processing flow can be optimized and performance problems caused by incorrect file order can be reduced.

[0093] Based on any of the above embodiments, the device further includes a decryption unit, and the decryption unit is used to: Based on hardware characteristics, the deserialized file is decrypted to obtain model data, and the model data is applied to perform reasoning.

[0094] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the model serialization method, which includes: based on the model size, segmenting the target file of the model to obtain a plurality of segmented files, wherein the target file is a file in the model whose file size exceeds a preset threshold; based on a preset serialization format, serializing the plurality of segmented files and the non-target files of the model to obtain a plurality of initial serialized files; generating a model identifier, and writing the model identifier as the first byte into each initial serialized file; digitally encoding each initial serialized file, and writing the digital code as the second byte into each initial serialized file to obtain a serialized file of the model.

[0095] In addition, the processor 810 can call the logic instructions in the memory 830 to execute the model deserialization method, which includes: reading the first byte and the second byte of each serialized file, and the serialized file is obtained based on any one of the model serialization methods described above; based on the first byte of each serialized file, determining the model identifier corresponding to each serialized file, and applying the model identifier to determine all serialized files to be parsed; based on the second byte of each serialized file to be parsed, determining the digital code of each serialized file to be parsed, and deserializing each serialized file to be parsed in the order of the digital code to obtain a deserialized file of the model.

[0096] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0097] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the model serialization method provided by the above methods, which method includes: based on the model size, splitting the target file of the model to obtain multiple split files, and the target file is a file in the model whose file size exceeds a preset threshold; based on a preset serialization format, serializing the multiple split files and the non-target files of the model to obtain multiple initial serialized files; generating a model identifier, and writing the model identifier as the first byte into each initial serialized file; digitally encoding each initial serialized file, and writing the digital code as the second byte into each initial serialized file to obtain a serialized file of the model.

[0098] In addition, when the computer program is executed by the processor, the computer can also execute the model deserialization method provided by the above methods, the method including: reading the first byte and the second byte of each serialized file, the serialized file being obtained based on any one of the model serialization methods described above; based on the first byte of each serialized file, determining the model identifier corresponding to each serialized file, and applying the model identifier to determine all serialized files to be parsed; based on the second byte of each serialized file to be parsed, determining the digital code of each serialized file to be parsed, and deserializing each serialized file to be parsed in the order of the digital code to obtain a deserialized file of the model.

[0099] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the model serialization method provided by the above-mentioned methods, the method comprising: based on the model size, segmenting the target file of the model to obtain multiple segmented files, the target file being a file in the model whose file size exceeds a preset threshold; based on a preset serialization format, serializing the multiple segmented files and the non-target files of the model to obtain multiple initial serialized files; generating a model identifier, and writing the model identifier as the first byte into each initial serialized file; digitally encoding each initial serialized file, and writing the digital code as the second byte into each initial serialized file to obtain a serialized file of the model.

[0100] In addition, when the computer program is executed by the processor, it is implemented to execute the model deserialization method provided by the above-mentioned methods, and the method includes: reading the first byte and the second byte of each serialized file, and the serialized file is obtained based on the model serialization method described in any one of the above-mentioned methods; based on the first byte of each serialized file, determining the model identifier corresponding to each serialized file, and applying the model identifier to determine all serialized files to be parsed; based on the second byte of each serialized file to be parsed, determining the digital code of each serialized file to be parsed, and deserializing each serialized file to be parsed in the order of the digital code to obtain a deserialized file of the model.

[0101] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A model serialization method, characterized in that: include: Based on the model size, the target file of the model is segmented to obtain a plurality of segmented files, wherein the target file is a file in the model whose file size exceeds a preset threshold; Based on a preset serialization format, serialize the multiple segmented files and the non-target files of the model to obtain multiple initial serialized files; Generate a model identifier, and write the model identifier as the first byte into each initial serialization file; Each initial serialization file is digitally encoded, and the digital encoding is written into each initial serialization file as the second byte to obtain a serialization file of the model.

2. The model serialization method according to claim 1, characterized in that: Also includes: Based on the hardware characteristics, any serialized file is encrypted to obtain an encrypted serialized file corresponding to the any serialized file.

3. The model serialization method according to claim 2, characterized in that: The hardware characteristics include the number of hardware computing cores, and the step of encrypting any serialized file based on the hardware characteristics to obtain an encrypted serialized file corresponding to any serialized file includes: Based on the number of hardware computing cores and the storage type of model data, the data in any serialized file is divided into a plurality of data blocks; Based on the data arrangement of the model layer corresponding to any serialized file, the data in each data block is rearranged to obtain an encrypted serialized file corresponding to any serialized file.

4. The model serialization method according to claim 3, characterized in that: The dividing the data in any serialized file into a plurality of data blocks based on the number of hardware computing cores and the storage type of the model data includes: Determining a data alignment target based on a storage type of the model data; If any of the serialized files does not meet the data alignment target, performing data padding on any of the serialized files; Based on the number of hardware computing cores, any serialized file filled with data is divided into multiple data blocks.

5. The model serialization method according to claim 3, characterized in that: The data arrangement of the model layer corresponding to any serialized file is based on the data arrangement of the model layer corresponding to any serialized file, and the data in each data block is rearranged to obtain the encrypted serialized file corresponding to any serialized file, including: Along the specified dimension direction of the data arrangement, data in any data block is divided based on a preset block size to obtain a plurality of blocks; When the size of the last block in the blocks does not reach the preset block size, filling the last block with data; The multiple cut blocks are arranged based on the cutting order of the multiple cut blocks to obtain the encrypted serialized file.

6. The model serialization method according to any one of claims 1 to 5, characterized in that: Based on the model size, the target file of the model is segmented to obtain multiple segmented files, including: Based on the model size, the preset file size and the model layer, the target file of the model is segmented to obtain the multiple segmented files.

7. A model deserialization method, characterized in that: include: Reading the first byte and the second byte of each serialized file, wherein the serialized file is obtained based on the model serialization method according to any one of claims 1 to 6; Based on the first byte of each serialized file, determine the model identifier corresponding to each serialized file, and apply the model identifier to determine all serialized files to be parsed; Based on the second byte of each serialized file to be parsed, the digital code of each serialized file to be parsed is determined, and each serialized file to be parsed is deserialized according to the order of the digital codes to obtain a deserialized file of the model.

8. The model deserialization method according to claim 7, characterized in that: Also includes: Based on hardware characteristics, the deserialized file is decrypted to obtain model data, and the model data is applied to perform reasoning.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the model serialization method according to any one of claims 1 to 6 or the model deserialization method according to any one of claims 7 to 8 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the model serialization method according to any one of claims 1 to 6 or the model deserialization method according to any one of claims 7 to 8 is implemented.

Citation Information

Cited By

  • Model file loading method and device and related equipment

    CN119537029A

  • Model encryption method and device and model decryption method and device

    CN120602154A

  • Model construction method, deployment method and electronic equipment

    CN121168512A