ONNX Runtime-based model protection method and apparatus, and related medium

Abstract

The invention discloses a model protection method and device based on ONNX Runtime and a related medium, and the method comprises the steps: obtaining a core description file, and carrying out the recombination optimization of a calculation chart in the core description file; modifying the character string type of the identifier in the calculation chart into an integer to obtain a modified core description file; obtaining a standard core description model, analyzing the standard core description model, and exporting the standard core description model as a model intermediate file; recompiling the model intermediate file into an ORT model file by utilizing the modified core description file; and constructing a running environment based on the modified core description file so as to load the ORT model file. According to the method, the core description file continues to be modified, and the operation environment is constructed based on the modified core description file, so that the model can be effectively prevented from being illegally extracted, copied or reversely analyzed, and the safety of model protection is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a model protection method, device and related media based on ONNX Runtime. Background Art

[0002] With the rapid development of artificial intelligence technology, the deployment of deep learning models in user-end devices has gradually become a mainstream application method. In the context of the widespread use of inference acceleration frameworks, model files are easily directly extracted or copied, resulting in the risk of leakage of the company's core technology assets. At present, the following technical solutions are mainly used to protect model security: (1) Model encryption and decryption mechanism, which encrypts and stores model files and dynamically decrypts them at runtime to prevent copying and reverse analysis, while combining hardware feature code binding to ensure that the model can only run on authorized devices; (2) Model segmentation and remote inference, which divides the key parts of the model to run in the cloud to reduce the risk of the complete model being extracted; (3) Dynamic key management, which regularly updates the decryption key through an authorization mechanism based on time or number of uses to prevent the model from being illegally used for a long time.

[0003] However, the above schemes still have defects: (1) The model encryption and decryption mechanism requires a complex key management and distribution system, which increases the system maintenance cost. The decrypted plaintext model is easily obtained by memory extraction tools. (2) The model segmentation and remote reasoning schemes are highly dependent on the stability of the network connection and cannot be used in offline scenarios. In addition, the communication delay and server operation and maintenance costs are high, and there is a security risk of data transmission being intercepted. (3) The dynamic key management scheme relies on a complex key update and synchronization mechanism. The system reliability is difficult to guarantee, and there is a risk of key update interruption or cracking. Therefore, the above defects lead to low security of model protection. Summary of the invention

[0004] The embodiments of the present invention provide a model protection method, device and related media based on ONNX Runtime, aiming to solve the problem of low security of model protection in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides a model protection method based on ONNX Runtime, including:

[0006] Obtaining a core description file, and reorganizing and optimizing a computational graph in the core description file;

[0007] Modify the string type of the identifier in the calculation graph to an integer type to obtain a modified core description file;

[0008] Obtaining a standard core description model, parsing the standard core description model and exporting it as a model intermediate file;

[0009] Recompiling the model intermediate file into an ORT model file using the modified core description file;

[0010] An operating environment is constructed based on the modified core description file to load the ORT model file.

[0011] In a second aspect, an embodiment of the present invention provides a model protection device based on ONNX Runtime, including:

[0012] A data reorganization unit, used to obtain a core description file and reorganize and optimize the calculation graph in the core description file;

[0013] A data modification unit, used for modifying the string type of the identifier in the calculation graph to an integer type, so as to obtain a modified core description file;

[0014] A data parsing unit, used for obtaining a standard core description model, parsing the standard core description model and exporting it as a model intermediate file;

[0015] A data compiling unit, used to recompile the model intermediate file into an ORT model file using the modified core description file;

[0016] A data loading unit is used to build an operating environment based on the modified core description file to load the ORT model file.

[0017] In a third aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the model protection method based on ONNX Runtime of the first aspect is implemented.

[0018] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the model protection method based on ONNX Runtime of the first aspect is implemented.

[0019] The embodiment of the present invention provides a model protection method, device and related media based on ONNX Runtime, the method comprising obtaining a core description file, and reorganizing and optimizing the calculation graph in the core description file; modifying the string type of the identifier in the calculation graph to an integer to obtain a modified core description file; obtaining a standard core description model, parsing the standard core description model and exporting it as a model intermediate file; using the modified core description file to recompile the model intermediate file into an ORT model file; and constructing an operating environment based on the modified core description file to load the ORT model file. The present invention continues to modify the core description file, and constructs an operating environment based on the modified core description file, so that the model can be effectively prevented from being illegally extracted, copied or reverse analyzed, greatly improving the security of model protection.

[0020] The embodiment of the present invention also provides a model protection device, a computer device and a storage medium based on ONNX Runtime, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0022] Figure 1 A flowchart of a model protection method based on ONNX Runtime provided in an embodiment of the present invention;

[0023] Figure 2 Another flowchart of a model protection method based on ONNX Runtime provided by an embodiment of the present invention;

[0024] Figure 3 A schematic block diagram of a model protection device based on ONNX Runtime provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0027] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0028] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0029] See below Figure 1 , Figure 1 A flow chart of a model protection method based on ONNX Runtime provided in an embodiment of the present invention specifically includes steps S101 to S105.

[0030] S101, obtaining a core description file, and reorganizing and optimizing a calculation graph in the core description file;

[0031] S102, modifying the string type of the identifier in the calculation graph to an integer type, and obtaining a modified core description file;

[0032] S103, obtaining a standard core description model, parsing the standard core description model and exporting it as a model intermediate file;

[0033] S104, recompiling the model intermediate file into an ORT model file using the modified core description file;

[0034] S105. Building an operating environment based on the modified core description file to load the ORT model file.

[0035] In the present invention, ONNX Runtime is taken as an example. ONNX Runtime is a high-performance machine learning inference engine developed and open sourced by Microsoft. The main goal is to optimize the deployment and execution efficiency of machine learning models. ONNX Runtime supports the ONNX (Open Neural Network Exchange) format, which is an open neural network exchange format that enables models trained by different frameworks to interoperate. ONNX Runtime provides a variety of optimization techniques to improve model performance, including operator fusion, memory planning optimization, parallel computing, etc. It also supports a variety of hardware backends, including CPU, GPU, DirectML, etc., and can automatically select the optimal execution scheme according to the actual operating environment. In practical applications, ONNX Runtime can often significantly improve the reasoning speed of the model while reducing resource consumption. ONNX Runtime also supports cross-platform, such as supporting multiple operating systems such as Windows, Linux, macOS, Android, IOS, and provides API interfaces for multiple programming languages, including Python, C++, C#, etc. A suitable deployment scheme can be selected according to specific needs. In a production environment, ONNX Runtime is widely used in a variety of scenarios such as edge devices, mobile terminals, and cloud servers. The ORT model is adopted in the present invention, and the ORT model is a proprietary model of ONNX Runtime. The ORT format is a format supported by a reduced-size version of the ONNX Runtime. Reduced-size builds are more suitable for use in size-constrained environments such as mobile and web applications.

[0036] Combination Figure 2 As shown, in step S101, the core description file (ort.fbs, the same below) is extracted from the ONNX Runtime source code. The core description file defines the underlying data structure of the ORT model, including the element layout and related fields of the calculation graph (GraphTable). When the calculation graph is reorganized and optimized, a specific reorganization algorithm is used to adjust the data structure order of the core description file to ensure that while maintaining the integrity of the data structure, it is differentiated from the standard parser, laying the foundation for the security protection of the model.

[0037] In one embodiment, the step S101 includes:

[0038] Adjusting the output node of the computation graph to the first position of the data structure;

[0039] Adjusting the computation node of the computation graph to the second position of the data structure;

[0040] Adjust the initializer of the computation graph to the third last position of the data structure;

[0041] The adjusted data structures of the output nodes, computing nodes and initializers are verified.

[0042] In this embodiment, first, the output node (Outputs) field in the calculation graph is extracted and its position is adjusted to the first position of the data structure. The output node is the final output of the model reasoning result, which is used to represent the calculation result of the model. By placing the output node in the first position, the data parsing process starts from the most critical result node. This structural adjustment not only changes the original parsing order, but also provides a protection mechanism for the concealment of subsequent fields. Next, the calculation node (Nodes) field in the calculation graph is adjusted to the second position of the data structure. The calculation node represents the core operation unit in the model, including operators and their connection relationships. Adjusting the position of the calculation node further breaks the parsing order of the standard structure, so that the parser based on the default order cannot correctly extract the node information. At the same time, this adjustment ensures that the important structural information in the calculation graph can be arranged immediately after the output node, which helps to optimize the logical organization of the data structure. Then, the initializer (Initializers) field in the calculation graph is adjusted to the third from the bottom of the data structure. Initializers are usually used to store the weights or other initial parameters of the model. The adjustment of its position can increase the difficulty of parsing and prevent attackers from reversing the model by reading the weight information. In addition, this adjustment does not affect the relationship between the initializer and other fields, ensuring the structural integrity of the model. Finally, the data structure of the adjusted output nodes, calculation nodes, and initializers is verified to ensure that the data structure after the position adjustment is logically complete and correct. The verification process includes the following steps:

[0043] Verify that output nodes, compute nodes, and initializer fields are rearranged in the expected order;

[0044] Check the relevance of each field data to ensure that the logical relationship of the model calculation chart will not be destroyed after adjusting the position;

[0045] Verify that the data type complies with the original definition of the core description file to ensure that the parser can read and process the data correctly.

[0046] In step S102, in order to enhance the security of the model file, this step performs type conversion on all identifiers in the computational graph (such as the names of input nodes, output nodes, and computational nodes), mapping them from string types to integer values.

[0047] In one embodiment, the step S102 includes:

[0048] Respectively extracting data identifiers corresponding to output nodes, computation nodes, and initializers in the computation graph;

[0049] Defining an integer mapping rule for the data identifier according to a predetermined hash algorithm;

[0050] Convert the string type of the data identifier into an integer value using the integer mapping rule to replace the original string type;

[0051] The converted integer value is rewritten into the calculation graph to obtain a modified core description file.

[0052] In this embodiment, all data identifiers corresponding to output nodes, computation nodes, and initializers are extracted from the computation graph (Graph Table) in the core description file. These identifiers usually exist in the form of strings and are used to uniquely identify important structures such as the input and output, computation nodes, and parameter initializers of the model. The extraction process needs to traverse the relevant fields of the computation graph one by one, and store the identifier information in temporary variables for subsequent processing. The integer mapping rules of the data identifier are defined according to a predetermined hash algorithm. The hash algorithm must be deterministic, that is, the same string input always generates the same integer value, and should have a low conflict rate to ensure the uniqueness of the identifier. Commonly used algorithms include hash algorithms (such as SHA256, MD5, etc.) or other mapping functions based on specific rules. On this basis, the string identifier can be converted to an integer value of 64 bits or other suitable length. Using the integer mapping rules defined above, the extracted identifier strings are converted one by one to corresponding integer values. In this process, for each identifier:

[0053] Input the identifier string to the hash algorithm;

[0054] The generated hash value is mapped to an integer value according to the rules;

[0055] Replace the original string identifier with an integer value.

[0056] This conversion operation can hide the original meaning of the identifier, increase the difficulty of parsing and reverse engineering, and thus improve the security of the model file.

[0057] Furthermore, the converted integer value is rewritten back to the calculation graph field in the core description file, replacing the original string identifier. This ensures that all identifiers in the calculation graph have completed type conversion and the data structure conforms to the modified core description file definition. During the rewriting process, data integrity must be strictly verified to ensure that each integer value is correctly associated with the corresponding field, while keeping other fields of the calculation graph unaffected.

[0058] In one embodiment, rewriting the converted integer value into the computation graph to obtain a modified core description file includes:

[0059] Maintaining the order of the original fields in the computation graph to obtain an updated computation graph;

[0060] The updated computation graph is rewritten into the core description file, and the rewritten core description file is recompiled using a compilation mode script to generate the modified core description file.

[0061] In this embodiment, after replacing the original string identifier in the calculation graph with an integer value, the data structure of the entire calculation graph needs to be updated while strictly maintaining the order of the original fields. The specific process is as follows:

[0062] Check the fields in the computational graph (such as output nodes, computation nodes, initializers, etc.) one by one to ensure that their logical order is consistent with that in the original core description file. Any change in order may affect the parseability of the model;

[0063] Insert the fields after integer value replacement into the new computational graph one by one, ensuring that all identifiers have been successfully converted while retaining the original data of other field contents (such as weights, tensor shapes, etc.);

[0064] Based on the field order, all updated field information is integrated to form an updated calculation graph that is complete in structure and meets the standards.

[0065] After obtaining the updated calculation graph, it needs to be rewritten into the core description file. The specific steps are as follows:

[0066] Extract the storage location and structure definition of the computational graph from the core description file (which can be described by the FlatBuffers file) to ensure that other core description fields are not destroyed during the rewriting process;

[0067] Rewrite the updated calculation graph field by field to the corresponding position of the core description file, replacing the content of the original calculation graph. During this process, ensure that the data type, field format, and memory alignment are consistent with the original definition of the core description file;

[0068] Verify the calculation graph fields in the core description file to ensure that all updated field contents are written correctly without any missing or abnormal data.

[0069] Furthermore, to ensure that the rewritten core description file complies with the standard definition of FlatBuffers files and that the changes take effect, it needs to be recompiled, including:

[0070] Load the source code compilation toolchain of ONNX Runtime, especially the compilation script of the core description file (such as compile_schema.py);

[0071] Call the compilation script to recompile the modified core description file to generate a new description file header file (such as .h file) and parsing code. The specific instructions are as follows:

[0072] python compile_schema.py --input ort.fbs --output ort.fbs.h

[0073] This command will regenerate the parsing code of the core description file based on the modified ort.fbs file to ensure consistency with the updated calculation graph.

[0074] Furthermore, after recompilation, the generated core description file is the modified core description file, in which the identifiers of the calculation graph have all been converted to integer values, and the overall data structure has been verified and compiled, with good consistency and integrity. This file provides a reliable foundation for subsequent model protection processes (such as model parsing and construction of the operating environment), while significantly improving the security of the model.

[0075] In step S103, the standard core description model (i.e., standard .ort file) can be parsed by the tool chain provided by FlatBuffers (such as the flatc tool) and exported as a model intermediate file. During the parsing process, all the structure and attribute information of the model are retained, and the identifier is converted according to the integer mapping rule of the aforementioned step S102 to ensure that the generated model intermediate file matches the modified core description file format.

[0076] In one embodiment, the step S103 includes:

[0077] Parsing the standard core description model according to a predetermined format using a compilation tool to obtain a parsed core description model;

[0078] The compiled tool is used to combine the parsed core description model with the input model file and export the model into the model intermediate file.

[0079] In this embodiment, the compilation tool (such as flatc tool) provided by FlatBuffers is used to parse the standard core description model according to a predetermined format. The core description model is usually stored in the form of a .fbs file, which defines the underlying data structure of the model. The specific parsing process is as follows:

[0080] Load the standard core description file ort.fbs, which contains the structure definition of all key fields in the model, including calculation graphs, node information, initializers, etc.

[0081] Use the compilation tool to parse the core description file into a readable JSON format to extract detailed information and data structure of all fields. The command example is as follows:

[0082] flatc--raw-binary-t schema / ort.fbs--input / model.ort--strict-json

[0083] --raw-binary: Instructs the tool to read the input .ort file in binary format;

[0084] -t: Generate parsing results in JSON format;

[0085] schema / ort.fbs: specifies the core description file path;

[0086] input / model.ort: input model file path;

[0087] --strict-json: Generate parsing results that strictly conform to the JSON format;

[0088] Furthermore, based on parsing the standard core description model, the input model file content is further combined. The specific steps are as follows:

[0089] Load the input standard ORT model file and read its calculation graph, node information, weight data and other structural contents;

[0090] Match and merge the data content in the input model file with the parsed core description model structure.

[0091] Furthermore, the combined core description model and input model file data are exported as a model intermediate file. The export process needs to regenerate a .json file that conforms to the core description file definition to ensure the integrity and readability of the model structure. Use the compilation tool to rewrite the combined data structure into the intermediate file. This file retains all field definitions of the parsing model and contains the complete content of the input model file;

[0092] The following are examples of export commands:

[0093] flatc--raw-binary-o output / schema / ort.fbs model.json

[0094] --raw-binary: save the output file in binary format;

[0095] -o output / : specifies the output directory;

[0096] schema / ort.fbs: core description file path;

[0097] model.json: intermediate file of the combined model.

[0098] In step S104, the model intermediate file is recompiled by the flatc tool using the modified core description file (after reorganization optimization and identifier type conversion) to generate a new ORT model file. During recompilation, it is ensured that all structure definitions and identifier mapping rules conform to the modified core description file standard so that the generated ORT model file is unique.

[0099] In one embodiment, the step S104 includes:

[0100] Loading the modified core description file and model intermediate file respectively;

[0101] Comparing the data structures of the modified core description file and the model intermediate file, and adjusting the compatibility of the modified core description file and the model intermediate file in terms of node order, data format and string type;

[0102] Calling a compilation tool to recompile the loaded modified core description file and model intermediate file to generate a compiled model file;

[0103] The compiled model file is verified, and the ORT model file is generated according to the result of the verification.

[0104] In this embodiment, the modified core description file (ort.fbs) is read from the file system. Using the file loading tool provided by FlatBuffers, the file content is parsed into a data structure in memory to ensure that the field order and type comply with the modification rules. The model intermediate file (JSON format) is read from the file system. Using standard JSON parsing tools (such as Python's json module), the file content is parsed into a data structure object in memory to extract the model's calculation chart, nodes, weights and other information. After loading is completed, the data structures of the modified core description file and the model intermediate file are compared one by one to ensure the compatibility of the two in terms of node order, data format and string type, specifically including the following steps:

[0105] Extract the node list of the computational graph in the core description file and the model intermediate file;

[0106] Rearrange the node information in the model intermediate file according to the order specified in the core description file to ensure that the execution order of the nodes is consistent with the core description file;

[0107] Compare the data types of the two fields (such as integer, floating point, string, etc.); for compatible data types, adjust the relevant field formats of the model intermediate file to match the core description file. For example, convert the identifier field from string type to integer, or adjust the precision of the floating point number to the precision defined in the core description file;

[0108] For the identifier field in the model intermediate file, ensure that it conforms to the integer identifier format defined in the core description file. If the identifier is still a string type, call the integer mapping rule defined in step S102 to perform mapping conversion on it and update the model intermediate file.

[0109] Furthermore, after the compatibility adjustment is completed, the FlatBuffers compilation tool (flatc) is called to recompile the modified core description file and model intermediate file to generate the target compiled model file. The specific operation is as follows: Call the following command to complete the recompilation operation:

[0110] flatc--binary--strict-json-o output / modified_ort.fbs model.json

[0111] --binary: specifies the output to be in binary format;

[0112] --strict-json: Ensure that the input JSON format strictly complies with the specification;

[0113] -o output / : specifies the storage directory of the output file;

[0114] modified ort.fbs: modified core description file;

[0115] model.json: model intermediate file;

[0116] The compilation tool reparses the model intermediate file according to the definition of the core description file and generates a compiled model file in binary format. The output file is in .ort format and contains complete model structure and parameter information. After the compilation is completed, the generated compiled model file is verified to ensure the structural integrity and compatibility of the file. For example, verify whether the compiled model file contains all the fields defined in the core description file, check whether the field order meets the requirements of the core description file, and ensure that there are no omissions or errors. You can also compare the data values ​​(such as node parameters, initializer weights, etc.) in the model intermediate file and the compiled model file to ensure that the data has not been changed or lost during the compilation process. On the premise that the verification is correct, the compiled model file is converted into the final ORT model file. The output file is named protected_model.onnx and stored in the specified directory for subsequent use.

[0117] In step S105, a dedicated operating environment is constructed based on the modified core description file so that it can correctly load and run the generated ORT model file. During the construction of the operating environment, a reverse mapping strategy is used for all integer identifiers in the model to restore the integer values ​​to the original string identifiers and correctly load the corresponding model parameters and node attributes.

[0118] In one embodiment, the step S105 includes:

[0119] Parsing the modified core description file using a parsing tool to generate an operating environment suitable for model execution;

[0120] Defining integer restoration mapping rules according to the modified core description file;

[0121] Each node parameter and its corresponding tensor information in the ORT model file are loaded based on the integer restoration mapping rule to load the ORT model file.

[0122] In one embodiment, the modified core description file (modified_ort.fbs) is parsed using a FlatBuffers tool (such as flatc). The computational graph, node structure, tensor data structure, and initializer field information defined in the core description file are extracted. The data structure of the model runtime environment is constructed in memory through the core description file definition obtained by parsing. The runtime environment includes information such as the parsing order of the computational graph, the execution order of the nodes, and the allocation method of the tensors to ensure that the model can run correctly according to the predetermined logic during actual execution. The consistency of the data structure defined in the core description file and the actual parsing result is verified to ensure that the generated runtime environment conforms to the file definition.

[0123] Furthermore, since the model identifier has been converted from a string type to an integer value in the previous step, it is necessary to define an integer restoration mapping rule in this step to correctly map the integer identifier back to the original string identifier. The specific steps are as follows:

[0124] According to the integer mapping rule (such as hash algorithm) used when modifying the core description file, the corresponding reverse mapping rule is constructed;

[0125] The reverse mapping rule must be able to parse the integer value and restore it to a unique string identifier;

[0126] For each integer value, call the restore function (such as hash_reverse(int)) to convert it into the corresponding string identifier, ensuring that the restored string identifier is consistent with the original identifier defined in the core description file to avoid parsing errors;

[0127] Load the integer restoration mapping rules into the running environment for use in the subsequent loading process of node parameters and tensor information.

[0128] Furthermore, after the running environment is built and the restoration mapping rules are loaded, the parameters and tensor information of each node in the ORT model file are loaded based on the environment. The specific steps are as follows:

[0129] Load the protected ORT model file (such as protected_model.onnx);

[0130] Parse the node information, parameter fields, and tensor definitions in the model file;

[0131] Traverse all node information in the model file and restore the string identifier of the node according to the integer restoration mapping rule;

[0132] For each node, extract its operation type (such as operator type), input-output relationship, and required parameters, and load them into the running environment;

[0133] Parse all tensor data in the model file, including the dimensions, data types, and storage locations of the tensors;

[0134] Using the integer restoration mapping rule, the tensor identifier is restored to string form and associated with the node information in the running environment;

[0135] The loaded tensor information is stored in the running environment for the actual execution of model inference.

[0136] Verify that the parameters and tensors of each node are loaded correctly and ensure that they are consistent with the core description file definition, and verify that the loaded operating environment can fully cover all computing requirements of the model file.

[0137] The present invention solves the problem of traditional model protection solutions' dependence on encryption and decryption technology and key management mechanisms through a model protection method based on ONNX Runtime technology, and realizes localized model protection without network connection or remote server support. The protection method of the present invention can effectively prevent the model from being illegally extracted, copied or reverse analyzed, and provides an efficient protection solution designed for end-side deployment.

[0138] Compared with the prior art, the present invention has the following significant advantages:

[0139] No need for key management and network dependence: The present invention abandons traditional encryption and decryption technology and key management mechanism, and there is no need to consider the risk of key leakage or the increased system complexity due to key synchronization.

[0140] Lossless performance and security: By modifying the underlying framework of ONNX Runtime, the present invention does not generate additional performance overhead during model protection and avoids the risk of memory plaintext data leakage.

[0141] Simple and reliable implementation method: The present invention adopts technical means such as field adjustment and identifier mapping of the core description file to achieve structural protection of the model, which has both offensive and defensive capabilities, and the technical implementation process is simple and efficient.

[0142] Combination Figure 3 As shown, Figure 3 A schematic block diagram of a model protection device based on ONNX Runtime provided in an embodiment of the present invention, the model protection device 300 based on ONNX Runtime includes:

[0143] A data reorganization unit 301 is used to obtain a core description file and reorganize and optimize the calculation graph in the core description file;

[0144] A data modification unit 302, used to modify the string type of the identifier in the calculation graph to an integer type, so as to obtain a modified core description file;

[0145] The data parsing unit 303 is used to obtain a standard core description model, parse the standard core description model and export it as a model intermediate file;

[0146] A data compiling unit 304 is used to recompile the model intermediate file into an ORT model file using the modified core description file;

[0147] The data loading unit 305 is used to build an operating environment based on the modified core description file to load the ORT model file.

[0148] In this embodiment, the data reorganization unit 301 obtains the core description file, and reorganizes and optimizes the calculation chart in the core description file; the data modification unit 302 modifies the string type of the identifier in the calculation chart to an integer, and obtains a modified core description file; the data parsing unit 303 obtains the standard core description model, parses the standard core description model and exports it as a model intermediate file; the data compilation unit 304 uses the modified core description file to recompile the model intermediate file into an ORT model file; the data loading unit 305 builds an operating environment based on the modified core description file to load the ORT model file.

[0149] In one embodiment, the data reassembly unit 301 includes:

[0150] A first sorting unit, configured to adjust the output node of the computational graph to the first position of the data structure;

[0151] A second sorting unit, used to adjust the computing nodes of the computing graph to the second position of the data structure;

[0152] A third sorting unit, configured to adjust the initializer of the computation graph to the third to last position of the data structure;

[0153] The sorting verification unit is used to verify the adjusted data structures of the output nodes, computing nodes and initializers.

[0154] In one embodiment, the data modification unit 302 includes:

[0155] A data extraction unit, used to extract data identifiers corresponding to output nodes, computation nodes and initializers in the computation graph respectively;

[0156] A rule definition unit, used to define an integer mapping rule of the data identifier according to a predetermined hash algorithm;

[0157] A data replacement unit, used to convert the string type of the data identifier into an integer value by using the integer mapping rule to replace the original string type;

[0158] A data rewriting unit is used to rewrite the converted integer value into the calculation graph to obtain a modified core description file.

[0159] In one embodiment, the data rewriting unit includes:

[0160] A data updating unit, used to maintain the order of the original fields in the calculation graph to obtain an updated calculation graph;

[0161] The file compilation unit is used to rewrite the updated calculation graph into the core description file, and recompile the rewritten core description file using a compilation mode script to generate the modified core description file.

[0162] In one embodiment, the data parsing unit 303 includes:

[0163] A model parsing unit, used to parse the standard core description model according to a predetermined format using a compilation tool to obtain a parsed core description model;

[0164] The model combining unit is used to combine the parsed core description model with the input model file by using the compilation tool, and export it as the model intermediate file.

[0165] In one embodiment, the data compiling unit 304 includes:

[0166] A file loading unit, used to load the modified core description file and model intermediate file respectively;

[0167] A file adjustment unit, used for comparing the data structure of the modified core description file with the model intermediate file, and adjusting the compatibility of the modified core description file with the model intermediate file in terms of node sequence, data format and string type;

[0168] A tool compilation unit, used for calling a compilation tool to recompile the loaded modified core description file and model intermediate file to generate a compiled model file;

[0169] The file verification unit is used to verify the compiled model file and generate the ORT model file according to the verification result.

[0170] In one embodiment, the data loading unit 305 includes:

[0171] An environment building unit, used to parse the modified core description file using a parsing tool to generate an operating environment suitable for model execution;

[0172] A mapping definition unit, used to define integer restoration mapping rules according to the modified core description file;

[0173] A model loading unit is used to load each node parameter and its corresponding tensor information in the ORT model file based on the integer restoration mapping rule to load the ORT model file.

[0174] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.

[0175] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0176] The embodiment of the present invention also provides a computer device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the computer device may also include various network interfaces, power supplies and other components.

[0177] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0178] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A model protection method based on ONNX Runtime, characterized in that: include: Obtaining a core description file, and reorganizing and optimizing a computational graph in the core description file; Modify the string type of the identifier in the calculation graph to an integer type to obtain a modified core description file; Obtaining a standard core description model, parsing the standard core description model and exporting it as a model intermediate file; Recompiling the model intermediate file into an ORT model file using the modified core description file; An operating environment is constructed based on the modified core description file to load the ORT model file.

2. The model protection method based on ONNX Runtime according to claim 1, characterized in that: The reorganizing and optimizing the computational graph in the core description file includes: Adjusting the output node of the computation graph to the first position of the data structure; Adjusting the computation node of the computation graph to the second position of the data structure; Adjust the initializer of the computation graph to the third last position of the data structure; The adjusted data structures of the output nodes, computing nodes and initializers are verified.

3. The model protection method based on ONNX Runtime according to claim 1, characterized in that: The step of modifying the string type of the identifier in the computation graph to an integer type, and obtaining a modified core description file, includes: Respectively extracting data identifiers corresponding to output nodes, computation nodes, and initializers in the computation graph; Defining an integer mapping rule for the data identifier according to a predetermined hash algorithm; Convert the string type of the data identifier into an integer value using the integer mapping rule to replace the original string type; The converted integer value is rewritten into the calculation graph to obtain a modified core description file.

4. The model protection method based on ONNX Runtime according to claim 3, characterized in that: The converted integer value is rewritten into the calculation graph to obtain a modified core description file, including: Maintaining the order of the original fields in the computation graph to obtain an updated computation graph; The updated computation graph is rewritten into the core description file, and the rewritten core description file is recompiled using a compilation mode script to generate the modified core description file.

5. The model protection method based on ONNX Runtime according to claim 1, characterized in that: The core description model of the standard is parsed and exported as a model intermediate file, including: Parsing the standard core description model according to a predetermined format using a compilation tool to obtain a parsed core description model; The compiled tool is used to combine the parsed core description model with the input model file and export the model into the model intermediate file.

6. The model protection method based on ONNX Runtime according to claim 1, characterized in that: The method of recompiling the model intermediate file into an ORT model file by using the modified core description file includes: Loading the modified core description file and model intermediate file respectively; Comparing the data structures of the modified core description file and the model intermediate file, and adjusting the compatibility of the modified core description file and the model intermediate file in terms of node order, data format and string type; Calling a compilation tool to recompile the loaded modified core description file and model intermediate file to generate a compiled model file; The compiled model file is verified, and the ORT model file is generated according to the result of the verification.

7. The model protection method based on ONNX Runtime according to claim 1, characterized in that: The step of constructing an operating environment based on the modified core description file to load the ORT model file comprises: Parsing the modified core description file using a parsing tool to generate an operating environment suitable for model execution; Defining integer restoration mapping rules according to the modified core description file; Each node parameter and its corresponding tensor information in the ORT model file are loaded based on the integer restoration mapping rule to load the ORT model file.

8. A model protection device based on ONNX Runtime, characterized in that: include: A data reorganization unit, used to obtain a core description file and reorganize and optimize the calculation graph in the core description file; A data modification unit, used for modifying the string type of the identifier in the calculation graph to an integer type, so as to obtain a modified core description file; A data parsing unit, used for obtaining a standard core description model, parsing the standard core description model and exporting it as a model intermediate file; A data compiling unit, used to recompile the model intermediate file into an ORT model file using the modified core description file; A data loading unit is used to build an operating environment based on the modified core description file to load the ORT model file.

9. A computer device, characterized in that: The invention comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the model protection method based on ONNX Runtime as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the model protection method based on ONNXRuntime is implemented as described in any one of claims 1 to 7.

Images