Electronic equipment, model file generation method, medium and system

Abstract

The embodiment of the invention discloses an electronic device, a model file generation method, a medium and a system, a first intermediate representation corresponding to a neural network model can be determined based on the neural network model to be compiled, and according to various types of calculation units of the neural network model to be executed and each operator in the first intermediate representation, the model file can be generated. Splitting the first intermediate representation to generate a plurality of second intermediate representations, and based on the plurality of second intermediate representations, generating a model file corresponding to the neural network model, since each second intermediate representation comprises an operator operated by the same type of calculation unit, decoupling operators supported by different types of calculation units, and generating a model file corresponding to the neural network model. Therefore, the generated model file can comprise instructions of multiple types of calculation units, the instructions of different types of calculation units are decoupled, debugging is convenient, a new calculation unit is convenient to extend, and the flexibility and effectiveness of the model file are improved.

Description

Technical Field

[0001] This disclosure relates to neural network model compilation techniques, and in particular to an electronic device, a method, medium, and system for generating model files. Background Technology

[0002] In fields such as autonomous driving, smart cockpits, and security monitoring, neural network models play a crucial role. To deploy neural network models to terminal devices, the neural network models need to be compiled into model files, which are then deployed to the terminal devices. The terminal devices parse the model files using a runtime library, generate computation tasks, and distribute these tasks to various heterogeneous computing units. These heterogeneous computing units then execute the computation tasks, thereby enabling inference of the neural network model on the terminal devices.

[0003] In related technologies, for a neural network model to be compiled, the compiler generates an intermediate representation of the neural network model during the code generation stage, and then generates the target model file corresponding to the neural network model to be compiled based on the intermediate representation. The target model file can only store model information executed by specific types of heterogeneous computing units, and the model information of each heterogeneous computing unit is strongly coupled, making debugging difficult and making it difficult to expand to new computing units. Summary of the Invention

[0004] Embodiments of this disclosure provide an electronic device, a method for generating model files, a medium, and a system to decouple instructions of different types of computing units in the model file, facilitating debugging and expansion of new computing units.

[0005] A first aspect of this disclosure provides an electronic device, including: one or more first processors; wherein the one or more first processors are configured to:

[0006] Based on the neural network model to be compiled, a first intermediate representation corresponding to the neural network model is determined; the first intermediate representation includes multiple operators; according to the various types of computing units of the neural network model to be executed and each operator in the first intermediate representation, the first intermediate representation is split to generate multiple second intermediate representations, each of the second intermediate representations including at least one operator operated by computing units of the same type; based on the multiple second intermediate representations, a model file corresponding to the neural network model is generated.

[0007] A second aspect of this disclosure provides a method for generating a model file, comprising: determining a first intermediate representation corresponding to a neural network model to be compiled; the first intermediate representation including a plurality of operators; splitting the first intermediate representation according to various types of computing units of the neural network model to be executed and each operator in the first intermediate representation to generate a plurality of second intermediate representations, each second intermediate representation including at least one operator operated by computing units of the same type; and generating a model file corresponding to the neural network model based on the plurality of second intermediate representations.

[0008] A third aspect of this disclosure is to provide a computer-readable storage medium storing a computer program that is executed by a processor to perform the method for generating model files as described in any of the above embodiments of this disclosure.

[0009] A fourth aspect of this disclosure provides an electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the method for generating model files as described in any of the above embodiments of this disclosure.

[0010] A fifth aspect of this disclosure provides a computer program product that, when instructions in the computer program product are executed by a processor, performs a method for generating a model file provided in any of the above embodiments of this disclosure.

[0011] A sixth aspect of this disclosure provides a processing system, including a memory and an electronic device provided in any of the above embodiments of this disclosure. The processor is used to store model files.

[0012] Based on the electronic device, model file generation method, medium, and system provided in the above embodiments of this disclosure, a first intermediate representation corresponding to the neural network model is determined based on the neural network model to be compiled. According to the various types of computational units of the neural network model to be executed and the operators in the first intermediate representation, the first intermediate representation is split to generate multiple second intermediate representations. Each second intermediate representation includes at least one operator operated by the same type of computational unit. Based on the multiple second intermediate representations, a model file corresponding to the neural network model is generated. It can be seen that, in this disclosure, because the first intermediate representation is split based on the type of computational unit to obtain the second intermediate representation, and each second intermediate representation includes operators operated by the same type of computational unit, the operator coupling of different types of computational units in heterogeneous computing platforms is reduced. This further adapts to different types of heterogeneous computing platforms, improves the flexibility of the model file, and makes debugging based on the model file more convenient. Attached Figure Description

[0013] Figure 1 This is an exemplary application scenario of the electronic device provided in this disclosure;

[0014] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this disclosure;

[0015] Figure 3 This is a schematic diagram of the structure of an electronic device provided in another exemplary embodiment of this disclosure;

[0016] Figure 4 This is a flowchart of the process for generating model files in related technologies;

[0017] Figure 5 This is a flowchart illustrating the generation of model files via a processor, provided in an exemplary embodiment of this disclosure.

[0018] Figure 6 This is a schematic diagram of the data structure of the target model file provided in an exemplary embodiment of this disclosure;

[0019] Figure 7 This is a structural diagram of an electronic device provided in yet another exemplary embodiment of this disclosure;

[0020] Figure 8 This is a flowchart illustrating a method for generating model files provided in an exemplary embodiment of this disclosure;

[0021] Figure 9 This is a flowchart illustrating a method for generating model files provided in another exemplary embodiment of this disclosure;

[0022] Figure 10 This is a flowchart illustrating a method for generating model files provided in yet another exemplary embodiment of this disclosure;

[0023] Figure 11 This is a flowchart illustrating a method for generating model files provided in yet another exemplary embodiment of this disclosure;

[0024] Figure 12 This is a flowchart illustrating a method for generating model files provided in yet another exemplary embodiment of this disclosure;

[0025] Figure 13 This is a schematic diagram of the structure of an apparatus for generating model files provided in an exemplary embodiment of the present disclosure;

[0026] Figure 14 This is a schematic diagram of the structure of an apparatus for generating model files provided in another exemplary embodiment of this disclosure;

[0027] Figure 15This is a schematic diagram of the structure of an apparatus for generating model files provided in another exemplary embodiment of the present disclosure. Detailed Implementation

[0028] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0029] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0030] This disclosure outlines

[0031] In developing this disclosure, the inventors discovered that neural network models play a crucial role in fields such as intelligent driving, intelligent cockpits, and security monitoring. To deploy neural network models to terminal devices and enable inference on these devices, the neural network model needs to be compiled into a model file. This model file is then deployed to the terminal device. The terminal device parses the model file using a runtime library, generates computational tasks, and distributes these tasks to various heterogeneous computing units. These heterogeneous computing units then execute the computational tasks, thereby enabling inference on the neural network model on the terminal device. Terminal devices typically include various heterogeneous computing units, such as ACU (Accelerator Control Unit), ARM (Advanced RISC Machines), VPU (Vector Processing Unit), SPU (Scalar Processing Unit), DSP (Digital Signal Processor), NPU (Neural Network Processing Unit), and GPU (Graphics Processing Unit).

[0032] In related technologies, for a neural network model to be compiled, the compiler generates an intermediate representation of the neural network model during the code generation stage, and then generates the target model file corresponding to the neural network model to be compiled based on the intermediate representation. The target model file can only store model information executed by specific types of heterogeneous computing units, such as model information executed by ACU and ARM. Moreover, the model information of different types of heterogeneous computing units has strong coupling, making debugging difficult and making it difficult to expand to new execution units.

[0033] Exemplary Overview

[0034] Figure 1 This is an exemplary application scenario of the electronic device provided in this disclosure. For example... Figure 1 As shown, when a neural network model to be compiled needs to be compiled, the electronic device 11 of this disclosure compiles the neural network model to generate a model file corresponding to the neural network model. The model file is deployed to a terminal device 12, which can then perform computational tasks based on the model file. The electronic device 11 is, for example, but not limited to, a server or PC. The terminal device 12 is, for example, but not limited to, an in-vehicle terminal, a mobile phone, or a tablet. Optionally, the model file can be deployed to a system-on-a-chip (SoC) in the terminal device 12. The SoC includes, for example, but not limited to, an intelligent driving SoC, an intelligent cockpit SoC, and a cockpit-driver integrated SoC. The terminal device 12 or the SoC can include various types of computing units. Various types of computing units include, but are not limited to, ACU (Accelerator Control Unit), ARM (Advanced RISC Machines), VPU (Vector Processing Unit), SPU (Scalar Processing Unit), DSP (Digital Signal Processor), GPU (Graphics Processing Unit), and NPU (Neural Network Processing Unit). Optionally, a CPU can also be used as a computing unit.

[0035] The electronic device 11 may include one or more first processors; wherein the one or more first processors are configured to: determine a first intermediate representation corresponding to the neural network model based on the neural network model to be compiled; the first intermediate representation includes multiple operators; split the first intermediate representation according to the various types of computing units of the neural network model to be executed and each operator in the first intermediate representation to generate multiple second intermediate representations, each second intermediate representation including at least one operator operated by the same type of computing unit; and generate a model file corresponding to the neural network model based on the multiple second intermediate representations. It can be seen that, in this disclosure, because the first intermediate representation is split based on the type of computing unit to obtain the second intermediate representation, and each second intermediate representation includes operators operated by the same type of computing unit, the operator coupling of different types of computing units in heterogeneous computing platforms is reduced, further adapting to different types of heterogeneous computing platforms, improving the flexibility of the model file, and making debugging based on the model file more convenient.

[0036] Exemplary electronic devices

[0037] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of this disclosure. The electronic device 20 provided in this embodiment includes, but is not limited to, one or more of a server, a PC, and other electronic devices with corresponding functions. Figure 2 As shown, the electronic device 20 of this embodiment includes one or more first processors 21. Optionally, the electronic device 20 may further include a memory for storing a neural network model to be compiled and / or a model file generated by the first processor 21. The one or more first processors 21 are configured to:

[0038] Based on the neural network model to be compiled, a first intermediate representation corresponding to the neural network model is determined; the first intermediate representation includes multiple operators; according to the various types of computing units of the neural network model to be executed and each operator in the first intermediate representation, the first intermediate representation is split to generate multiple second intermediate representations, each of which includes at least one operator operated by computing units of the same type; based on the multiple second intermediate representations, a model file corresponding to the neural network model is generated.

[0039] Wherein, any of the first processors 21 is, for example, but not limited to, a Central Processing Unit (CPU) or other types of processors. The neural network model can be a neural network model of any task type and any network structure, which can be set according to actual needs. The task type includes, for example, but not limited to, object detection, semantic segmentation, image classification, speech recognition, text recognition, image perception, etc. The network structure includes, for example, but not limited to, network structures based on convolutional neural networks and their series, network structures based on Transformers and their series, network structures based on RNNs (Recurrent Neural Networks) and their series, network structures based on Multilayer Perceptrons (MLPs) and their series, etc.

[0040] The first intermediate representation of a neural network model is a multi-level intermediate representation (MLIR) optimized and transformed during the code generation stage. The processing operations during code generation include, but are not limited to, operator fusion, memory folding, and instruction scheduling. The first intermediate representation comprises multiple operators, each used to perform a specific computation or operation.

[0041] The various types of computing units that will execute the neural network model include, but are not limited to, CPUs, ACUs, ARM processors, VPUs, SPUs, DSPs, and GPUs. Each type of computing unit can support one or more operators. The specific operators supported by a computing unit can be determined based on its functional characteristics.

[0042] Based on the operators supported by various types of computing units, the operators in the first intermediate representation are split according to the type of computing unit to generate multiple second intermediate representations. Each second intermediate representation includes one or more operators operated on by the same type of computing unit. In other words, operators in the first intermediate representation that are operated on by the same type of computing unit are grouped together to obtain second intermediate representations corresponding to different types of computing units. For example, the first intermediate representation can be split into second intermediate representations corresponding to SPUs, VPUs, and ARMs.

[0043] After generating multiple second intermediate representations, a model file corresponding to the neural network model is generated based on these representations. For example, the multiple second intermediate representations are converted into a model file according to the format conditions of the model file. The model file is a standardized format file that can be deployed on the terminal device where the neural network model is to be executed. The model file may include the executable file of the computational unit corresponding to the neural network model, model parameters, and logical relationships such as the layer structure and connection methods of the neural network model.

[0044] In some optional embodiments, the number of neural network models can be one or more. For example, in intelligent driving scenarios, multiple neural network models are typically required to work together to achieve functions such as perception, localization, planning, and decision-making. Each of the multiple neural network models can generate a corresponding model file according to the above process. A complete target model file can then be generated through linking. Optionally, if there is only one neural network model, after generating the model file corresponding to that model, the model file can be linked using a linker to obtain the target model file.

[0045] The electronic device provided in this disclosure determines a first intermediate representation corresponding to a neural network model based on the neural network model to be compiled. According to the various types of computing units in the neural network model to be executed and the operators in the first intermediate representation, the first intermediate representation is split to generate multiple second intermediate representations. Each second intermediate representation includes at least one operator operated by the same type of computing unit. Based on the multiple second intermediate representations, a model file corresponding to the neural network model is generated. It can be seen that, in this disclosure, because the first intermediate representation is split based on the type of computing unit to obtain the second intermediate representation, and each second intermediate representation includes operators operated by the same type of computing unit, the operator coupling of different types of computing units in heterogeneous computing platforms is reduced. This further adapts to different types of heterogeneous computing platforms, improves the flexibility of the model file, and makes debugging based on the model file more convenient.

[0046] In some alternative embodiments, in the above... Figure 2 Based on the illustrated embodiment, one or more first processors 21, according to the various types of computational units of the neural network model to be executed and the operators in the first intermediate representation, split the first intermediate representation to generate multiple second intermediate representations, which may include:

[0047] Based on the various types of computational units in the neural network model to be executed and the operators supported by each type of computational unit, the computational units corresponding to each operator in the first intermediate representation are determined respectively; based on the computational units corresponding to each operator in the first intermediate representation, the first intermediate representation is split into multiple second intermediate representations, and the types of computational units corresponding to any two second intermediate representations are different.

[0048] Different types of computational units may support different operators. Each type of computational unit and its supported operator types can be pre-stored in the storage space. During model compilation, the computational units corresponding to each operator in the first intermediate representation can be determined based on the type of computational units in the neural network model to be executed and the types of operators supported by different types of computational units pre-stored in the storage space.

[0049] In some optional embodiments, determining the computational unit corresponding to each operator in the first intermediate representation may include: allocating a corresponding computational unit to each operator according to a preset rule based on the dependencies between operators in the first intermediate representation and the operators supported by each type of computational unit. For example, if an operator A can be operated on by multiple types of computational units, the computational unit corresponding to operator A can be determined from the idle computational units that support operator A, based on the occupied computational units such as the preceding operator B that operator A depends on and the operator C that the preceding operator B depends on. The specific preset rule for determining the computational unit corresponding to an operator can be set according to actual needs, and this embodiment of the disclosure does not limit it.

[0050] Specifically, based on the computational units corresponding to each operator in the first intermediate representation, the first intermediate representation is split into second intermediate representations corresponding to each type of computational unit. Any two second intermediate representations correspond to different types of computational units. For example, if the neural network model to be executed corresponds to three types of computational units—SPU, VPU, and CPU—then the first intermediate representation can be split into three second intermediate representations. Each second intermediate representation is used for execution in a specific type of computational unit. The first second intermediate representation is used for computation via the SPU, the second second intermediate representation is used for computation via the VPU, and the third second intermediate representation is used for computation via the CPU.

[0051] In the embodiments of this disclosure, each operator in the first intermediate representation is split according to the type of computing unit based on the operators supported by each type of computing unit to obtain multiple second intermediate representations, so that each second intermediate representation corresponds to a different type of computing unit. This enables each operator in the neural network model to be computed through different types of computing units, thereby reducing the coupling of operators in different types of computing units, making it easier to debug based on the type of computing unit, and facilitating the expansion of new computing units.

[0052] In some optional embodiments, based on any of the above embodiments, one or more first processors 21 generate a model file corresponding to the neural network model based on multiple second intermediate representations, including:

[0053] Based on multiple second intermediate representations and the computational units corresponding to each second intermediate representation, file information and model description information in a preset format are generated for each computational unit. The file information in the preset format includes an executable file in a preset format corresponding to the second intermediate representation. The model description information includes the structural parameters of the neural network model. Based on the file information in the preset format and the model description information corresponding to each computational unit, a model file corresponding to the neural network model is generated.

[0054] The file information in this disclosure is in a predefined format, including but not limited to Executable and Linkable Format (ELF) file information, to facilitate the reuse of various tools of LLVM (Low Level Virtual Machine). LLVM is a framework system for architecture compilers. The predefined file information includes a second intermediate representation of the corresponding predefined format executable file. The executable file is, for example, but not limited to, an ELF file. ELF is a standard file format used to store executable programs, shared libraries, etc. For example, an ELF file can store compiled model code, model parameters, and linking information, enabling the model file to be loaded and executed.

[0055] In this disclosure, the model description information describes the structural parameters of the model according to a predefined data structure. Structural parameters include, but are not limited to, the model name, the number of computation graphs included in the model, the name of each computation graph, the task nodes included in each computation graph, input information, weight information, output information, compilation options used during compilation, and environment variables. Input information includes, but is not limited to, the data type of the input tensor (e.g., INT8, FP32), dimension information (i.e., shape), and stride information. Weight information includes, but is not limited to, the data type of the weight tensor, dimension information, and stride information. Output information includes, but is not limited to, the data type of the output tensor, dimension information, and stride information. The data types include, but are not limited to, INT4, INT8, INT16, FP16, and FP32. Dimension information indicates the number of dimensions of the tensor and the size of each dimension. For example, a tensor with dimensions H×W indicates that it is a two-dimensional tensor with a height (or number of rows) of H and a width (or number of columns) of W, where H and W are both positive integers. Stride information indicates the number of memory bytes that need to be skipped when moving from one element to the next along a certain dimension. For example, a tensor with dimensions H×W and a stride of (4,1) indicates a stride of 4 bytes in the H direction and a stride of 1 byte in the W direction. This means the tensor is row-oriented. When reading data from memory, if data needs to be read along the H direction, after reading the first data in the first column, 4 bytes need to be skipped to read the second data in the first column. If data needs to be read along the W direction, after reading the first data in the first row, only 1 byte needs to be skipped to read the second data in the first row.

[0056] In this disclosure, the pre-formatted file information corresponding to each computing unit includes the executable code of each computing unit, and the model description information includes the structural parameters of the overall computing logic of the model, which can be used for parameter configuration of the computing unit. Based on the pre-formatted file information and model description information corresponding to each computing unit, a deployable model file corresponding to the neural network model can be generated, enabling the model to be loaded and executed.

[0057] For example, the model description information may include multiple levels of information. The first level of model description information may include a computation graph array, computation graph information, storage space information, node information, quantization information, etc. The various information at the first level may further include second-level information. For example, the computation graph information may further include name, tool component version, node information, input information, output information, storage space information, variables, compilation options, environment variables, etc. The various information at the second level may further include third-level information, and so on. The specific number of levels and the information content included in each level can be set according to actual needs, and this embodiment of the disclosure does not limit this.

[0058] In the embodiments of this disclosure, for the second intermediate representation corresponding to various types of computing units, file information and model description information in preset formats corresponding to each type of computing unit are generated. Then, based on the file information and model description information in preset formats corresponding to each type of computing unit, a model file corresponding to the neural network model that can be deployed to the computing unit is generated. The file information in preset formats represents the executable file information of the computing unit, so that the model file can be loaded and executed. Moreover, different types of computing units in the model file correspond to their respective preset format file information, so that the executable files of different types of computing units are decoupled, which is convenient for debugging and easy to expand new computing units.

[0059] In some optional embodiments, one or more first processors 21 generate a model file corresponding to the neural network model based on file information and model description information in a preset format corresponding to each computing unit, including:

[0060] The file information in the preset format corresponding to each computing unit is determined as the file information segment in the model file; the model description information is determined as the description information segment in the model file; an initial model file is generated based on the file information segment and the description information segment; a first hash code is generated based on the initial model file; the first hash code is written into the hash code segment in the initial model file to obtain the model file corresponding to the neural network model.

[0061] The model file records the file information and model description information of each computational unit in a preset format through multiple fields (sections). That is, the file information of each computational unit in a preset format is determined as one field in the model file (called the file information section), and the model description information is determined as another field in the model file (called the description information section). The initial model file is generated based on each file information section and the description information section.

[0062] The first hash code can be used to verify the model file, ensuring its integrity and uniqueness. Optionally, the first hash code of the initial model file can be calculated based on a preset hash algorithm. Hash algorithms include, but are not limited to, MD5 (Message Digest Algorithm 5) and SHA-256 (Secure Hash Algorithm 256-bit). MD5 maps data of any length to a 128-bit hash code (or hash value). SHA-256 maps data of any length to a 256-bit hash code. When generating the initial model file, a preset hash code of a certain length can be used as the hash code segment in the initial file. The initial hash code is, for example, but not limited to, an initial value of all zeros. The number of bits in the hash code is determined based on the hash algorithm. For example, if the preset hash algorithm is MD5, then the hash code is 128 bits, the hash code segment is a 128-bit field, and the initial hash code is 128 zeros. Based on the initial model file, a hash function is used to generate a 128-bit first hash code, and the first hash code is written into the hash code segment to obtain the model file corresponding to the neural network model.

[0063] In some optional embodiments, the model description information can be defined as a description segment in the model file using a cross-platform serialization library (flatbuffer). Flatbuffer is a zero-copy serialization scheme whose core is to directly store and access model parameters through a predefined binary data model, enabling efficient execution on hardware without parsing. In other words, model weights and computation graphs are stored in a contiguous binary format and accessed directly through memory pointers, avoiding the parsing overhead of traditional serialization. When performing model inference on the terminal device, model data can be directly mapped from the model file to memory for the computation unit to access.

[0064] In the embodiments of this disclosure, by determining the file information in a preset format corresponding to each computing unit as the file information segment in the model file, and determining the model description information as the description information segment in the model file, an initial model file is generated based on the description information segment and each file information segment. This allows the initial model file to store executable files of various types of computing units, without being limited by the type of computing unit. Furthermore, the executable files of different types of computing units are decoupled, which facilitates debugging and makes it easy to expand new computing units.

[0065] Figure 3 This is a schematic diagram of the structure of an electronic device provided in another exemplary embodiment of this disclosure.

[0066] In some optional embodiments, based on any of the above embodiments, the number of neural network models to be compiled is multiple; such as Figure 3As shown, the electronic device 20 also includes a second processor 22, which is configured to:

[0067] For each neural network model, after generating the corresponding model file based on multiple second intermediate representations, the model files corresponding to the multiple neural network models are linked to obtain the target model file.

[0068] The second processor 22 may be, but is not limited to, a central processing unit (CPU). The second processor 22 may be coupled to the first processor 21. The second processor 22 can obtain model files corresponding to multiple neural network models from the first processor 21. Alternatively, after the first processor 21 generates model files corresponding to multiple neural network models, it stores the model files in a preset storage space, and the second processor 22 can obtain the model files corresponding to each neural network model from the preset storage space. Each neural network model's model file is a single model file (or a single-model relocatable file). Since multiple neural network models need to collaborate to complete corresponding functions, during model inference, multiple neural network models can share the computing resources of the computing unit in a time-sharing manner. Therefore, the model files of multiple neural network models need to be linked to generate a target model file to ensure that multiple neural network models can be loaded and executed normally.

[0069] In some optional embodiments, the process of linking the model files corresponding to multiple neural network models may include, but is not limited to, operations such as merging the information segments of multiple model files and static relocation in the file information segments and description information segments corresponding to the executable files of heterogeneous computing units, so that multiple neural network models can be executed in an orderly manner on the terminal device.

[0070] In some optional embodiments, a linker can be used to link the model files corresponding to multiple neural network models to obtain the target model file.

[0071] In some alternative embodiments, the first processor 21 may be a processor running a compiler, and the second processor 22 may be a processor running a linker.

[0072] In the embodiments of this disclosure, a target model file is obtained by linking multiple single model files, so that the model file can store the executable files of multiple models, ensuring that multiple neural network models can be executed collaboratively and orderly on the terminal device, improving model execution efficiency, and avoiding conflicts in computational tasks between models.

[0073] In some optional embodiments, the second processor 22 links the model files corresponding to multiple neural network models to obtain a target model file, including:

[0074] Multiple model files are parsed to obtain information segments from the multiple model files; the information segments are merged to obtain a merged result; in response to the presence of multiple sets of identical model parameters in the merged result, the multiple sets of identical model parameters are compressed to obtain a compressed result; the compressed result is statically relocated to obtain the target model file.

[0075] The parsing of multiple model files involves extracting various information segments from each model file according to its data structure. These information segments include, for example, file information segments, description information segments, and hash code segments corresponding to different types of computational units. After obtaining these information segments, they can be merged according to pre-configured merging rules. The merging process includes merging information segments of the same type from multiple model files; for example, merging the first SPU file information segment of model file A with the second SPU file information segment of model file B into a third SPU file information segment, merging the first description information segment of model file A with the second description information segment of model file B into a third description information segment, and so on. Model parameters are the network parameters of a neural network model, including, but not limited to, weight parameters or other related parameters. Multiple sets of identical model parameters refer to multiple sets of parameters with the same values. Identifiable model parameters can be determined by recognizing duplicate parameters among the model parameters. Compression of multiple sets of identical model parameters refers to compressing two or more sets of model parameters with identical values ​​into a single set of model parameters to reduce the number of model parameters in the model file. Optionally, a corresponding storage location can be allocated for the compressed set of model parameters. The parts of each model file that use these model parameters can be replaced with references to their storage location. This allows for storing one set of model parameters and using them in multiple places within each information segment of the merged result. By merging the information segments, a merged result is obtained. Finally, by compressing the model parameters within the merged result, a compressed result is obtained.

[0076] Static relocation is a technique that integrates scattered code and data segments from a compressed file into a single, executable block that can be stored at a specific memory address. Specifically, static relocation is an address mapping technique used during the program linking or loading phase. Its core is to integrate the scattered code and data segments from the executable file (i.e., the compressed file) into a continuous module that can run in the target logical address space, and to correct all unbound symbolic addresses (or relative addresses) within the executable file to fixed absolute logical addresses after program loading. The specific operation of static relocation includes two core steps: Step 1, allocating the actual starting address and storage range within the target logical address space for the code and data segments in the executable file, clarifying the address boundaries of each module; Step 2, based on the allocated absolute logical addresses, correcting the address references to code or data in the instructions line by line, replacing the original relative offsets or symbolic identifiers. Through this process, after loading, the program can directly execute instructions based on the fixed logical addresses without additional address translation, ensuring that instructions accurately point to the corresponding code or data storage location, enabling independent program execution.

[0077] In the embodiments of this disclosure, for each neural network model among multiple neural network models, after generating the single model file corresponding to each neural network model, operations such as parsing, merging information segments, compressing model parameters, and static relocation can be performed on the multiple single model files to generate target model files corresponding to multiple neural network models. This ensures that multiple models can be executed normally and in an orderly manner, improves model execution efficiency, and avoids task conflicts between multiple models. Furthermore, by compressing model parameters, the number of model parameters that need to be stored in the model file can be reduced, thereby reducing the storage space occupied by the model file.

[0078] In some optional embodiments, the compression result includes a file information segment and a description information segment after merging and compressing the model parameters; the file information segment includes a sub-information segment storing static relocation items. The second processor 22 performs static relocation on the compression result to obtain the target model file, including:

[0079] Based on the sub-information segment containing the static relocation item in the file information segment of the compression result, the content of the storage address pointed to by the static relocation item in the compression result is corrected to obtain the correction result; based on the correction result, the second hash code segment is generated; based on the correction result and the second hash code segment, the target model file is generated.

[0080] The storage address information in the compressed result is the symbolic address (or relative address) in a single model file. If the terminal device where the computing unit resides only needs to run a single model, each computing unit only needs to execute the computing task of that single model, and storage address conflicts will not occur. However, if the terminal device needs to run multiple models simultaneously, each computing unit needs to collaboratively execute the computing tasks of multiple models, and storage address conflicts may occur. Therefore, the storage address information in the compressed result needs to be corrected to ensure that multiple models can run normally and in an orderly manner. The file information segment in the compressed result includes a sub-information segment storing static relocation items. Static relocation items are information marking items that need to be corrected. Therefore, based on the sub-information segment storing static relocation items in the file information segment of the compressed result, the storage addresses that need to be corrected in the compressed result can be determined. Based on this, the content of the storage addresses pointed to by the static relocation items in the compressed result can be corrected to obtain the corrected result.

[0081] Storage addresses may include, but are not limited to, register numbers and offsets, or they may also include memory addresses. For example, register R stores the starting address of model parameters. If multiple models do not need to run simultaneously, each model's starting address is obtained from register R during runtime. Combined with the relative addresses in a single model file, the absolute logical address can be accurately obtained, thus avoiding storage address conflicts. If multiple models need to run simultaneously, and model file A is linked with model file B, and model file A precedes model file B, then during actual runtime, the starting addresses of model parameters for both model file A and model file B will be obtained from register R, leading to conflicts. The register number for model file B needs to be corrected; for example, the register corresponding to the starting address of model parameters in model file B could be changed to register R+1. Furthermore, due to model parameter compression, the model parameter addresses will also shift, so the storage addresses of model parameters in model files A and B need to be corrected to ensure accuracy. By correcting the storage address information in the compression result, a corrected result is obtained. Based on this corrected result, a second hash code is generated, and the second hash code segment is determined. The principle behind generating the second hash code is similar to that of generating the first hash code, and will not be repeated here. Based on the correction result and the generated second hash code segment, the target model file is generated. For example, the information segments from the correction result and the second hash code segment are combined according to a preset data format to generate the target model file. The second hash code can be used to verify the integrity and uniqueness of the target model file.

[0082] In some optional embodiments, after modifying the content of the storage address pointed to by the static relocation item in the file information segment based on the compression result, the method further includes deleting the static relocation item to obtain the modified result. Since the static relocation item is no longer useful after static relocation is completed, it can be deleted so that the modified result no longer includes the static relocation item, further reducing storage space usage.

[0083] In some optional embodiments, the target model file may include dynamic relocation items. Dynamic relocation items are those whose storage addresses can only be determined at runtime. For example, relocation involving cross-computation units requires runtime determination of the storage address across computing units. For instance, if the ACU needs to start the VPU, the address of the VPU file can only be determined after the VPU is loaded into memory. In this case, the VPU's storage address needs to be relocated at runtime. Specific relocation items are set according to actual application requirements, and will not be described in detail in this disclosure.

[0084] In some optional embodiments, the functions of the second processor 22 described above can also be implemented by the first processor 21. That is, the first processor 21 is further configured to, for each neural network model, after generating a model file corresponding to the neural network model based on multiple second intermediate representations, link the model files corresponding to the multiple neural network models respectively to obtain a target model file. Specific linking operations can be found in the linking operations of the second processor 22 in the foregoing embodiments.

[0085] In some alternative embodiments, Figure 4 This is a flowchart illustrating the process of generating model files in related technologies. For example... Figure 4 As shown, the process for generating model files in related technologies includes:

[0086] S31, based on a neural network model, generates intermediate representations (MLIRs) through a code generation stage.

[0087] S32, based on the intermediate representation, generates model files through the model file generation (which can be represented as gen-.m) stage.

[0088] The model file generation schemes in related technologies produce a large structure that couples ACU instructions and CPU-called operators into custom description information. There are no intermediate products, no distinction between computing units, making debugging extremely difficult, and it is also difficult to add other heterogeneous computing units, resulting in poor scalability.

[0089] In response to the aforementioned problems with related technologies, Figure 5This is a flowchart illustrating the process of generating a model file using a processor, provided in an exemplary embodiment of this disclosure. For example... Figure 5 As shown, the process of generating model files through a processor in this embodiment of the disclosure includes the following steps:

[0090] S41, based on multiple neural network models, generates first intermediate representations (MLIRs) corresponding to each of the multiple neural network models through the code generation stage.

[0091] S42, based on the first intermediate representation corresponding to each neural network model, generates multiple second intermediate representations (dialect) corresponding to each first intermediate representation through a splitting operation. The splitting operation includes splitting the first intermediate representation according to the various types of computational units of the neural network model to be executed and the operators in the first intermediate representation to generate multiple second intermediate representations.

[0092] S43, for each neural network model corresponding to multiple second intermediate representations, generates the model file corresponding to that neural network model through a single model file generation operation. The single model file is the object file of the unrelocated single model generated by the compiler. The single model file includes ELF files corresponding to various types of computational units, model description information, and hash codes (i.e., first hash codes), etc. Figure 5 The various types of computing units include ACU, VPU, and host. The host is the execution unit in the terminal device that loads and parses model files and generates computing tasks. The host can be the CPU or main controller in the terminal device. In other words, the CPU or main controller in the terminal device can also be used as a computing unit. For computing tasks that cannot be executed by computing units such as ACU and VPU, they can be executed by the CPU or main controller.

[0093] S44 generates a target model file by linking (lld) the individual model files corresponding to multiple neural network models. The target model file can be debugged using various LLVM tools.

[0094] contrast Figure 4 and Figure 5As shown, compared to the process of generating life model files in related technologies, the process of generating model files in this embodiment of the present disclosure, after generating the first intermediate representation MLIR, can split the first intermediate representation into multiple second intermediate representations according to the computational units corresponding to each operator in the first intermediate representation. Then, based on the multiple second intermediate representations, ELF files corresponding to different types of computational units, model description information, and hash codes for model file verification can be generated respectively. This decouples the ELF files of different types of computational units and decouples the ELF files from the model description information, improving the flexibility of the model file. Furthermore, linking multiple single model files into a target model file enables multiple models to be correctly loaded and executed in an orderly manner on the terminal device, helping to improve execution efficiency. In addition, since this embodiment of the present disclosure adds multiple stages of intermediate products during the model file generation process, it facilitates debugging and allows for the expansion of new computational units.

[0095] In some alternative embodiments, Figure 6 This is a schematic diagram of the data structure of a target model file provided in an exemplary embodiment of this disclosure. For example... Figure 6As shown, the target model file includes ELF files from multiple neural network models, processed through parsing, information segment merging, model parameter compression, and static relocation. These ELF files contain information segments, description segments, and hash code segments. Specifically, `_info` represents the description segment, with the `.m_info` field storing model description information in flatbuffer format. The `.string_info` field stores string data used in the flatbuffer. The `.binary_info` field stores binary data used in the flatbuffer. `.acu_elf`, `.vpu_elf`, and `.spu_elf` represent the file information segments for the ACU, VPU, and SPU respectively, resulting from the merging of information segments from multiple single model files. These segments store the dynamic libraries for each type of computational unit. `.m_checksum` stores the unique hash code of the target model file. Taking `.acu_elf` as an example, `.acu_elf` includes `.text`, `.rodata`, and `.rel.dyn` fields after static relocation by the linker (LLD). The `.text` field stores the ACU code, `.rodata` stores CPU and ACU constant data, and `.rel.dyn` stores dynamic relocation entries. The information segments of other types of computational units' ELF files are similar to `.acu_elf` and will not be elaborated further. CPU and ACU constant data includes parameters and other necessary constants. `_info` is not an ELF file. For example, `.string_info` records all strings as a long array in a section. When a string is needed in `.m_info`, its position information in `.string_info`, including the start and end positions, is used. Similarly, all binary data is stored in `.binary_info`, and the start and end positions of the binary data are referenced in `.m_info`. At runtime, the corresponding binary data can be obtained from `.binary_info` based on the start and end positions. In the target model file, the corresponding ELF files for each computational unit are essentially recorded in binary format and can be parsed into ELF files. According to the protocol, model-related information for the computational unit can be obtained from the _elf file. Therefore, in this embodiment of the disclosure, the file for each heterogeneous computational unit is embedded as an information segment in the target model file. Each file information segment is an ELF file, which stores the code executed on that computational unit and the necessary data. It should be noted that... Figure 6 The representation methods for various information segments or sub-information segments are merely examples, and are not limited to those described above in practical applications. For instance, .m_info and .binary_info can be represented using any other method capable of representing the corresponding information segments.

[0096] The embodiments described above can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit them.

[0097] Figure 7 This is a structural diagram of an electronic device provided in another exemplary embodiment of the present disclosure, including at least one processor 91 and a memory 92. The at least one processor 91 includes, for example, the first processor 21 and the second processor 22 described above.

[0098] The processor 91 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions.

[0099] The memory 92 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 91 may execute one or more computer program instructions to implement the methods and / or other desired functions of the various embodiments of this disclosure described above.

[0100] In one example, the electronic device 90 may also include an input device 93 and an output device 94, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0101] The input device 93 may also include, for example, a touchscreen, a microphone, various sensors, etc. Sensors may include, for example, image sensors (e.g., cameras, webcams), LiDAR, millimeter-wave radar, ultrasonic radar, positioning sensors, pressure sensors, air quality sensors, temperature sensors, etc. Image sensors, LiDAR, millimeter-wave radar, ultrasonic radar, etc., can be used for environmental perception, i.e., detecting moving and static objects in the surrounding environment. Moving and static objects may include, for example, static objects such as lane lines, curbs, arrows, signs, trees, and buildings, as well as dynamic objects such as surrounding vehicles, pedestrians, and cyclists. Positioning sensors are used to locate the mobile device (e.g., a bicycle, a robot, etc.) where the electronic device is located. Positioning sensors may include, for example, an Inertial Measurement Unit (IMU), a Global Positioning System (GPS), etc. Pressure sensors can be used to detect seat pressure. Temperature sensors can be used to detect the temperature inside the vehicle cabin. Air quality sensors can be used to detect the air quality inside the vehicle cabin.

[0102] The output device 94 can output various information to the outside, including, for example, a display, a speaker, a communication network and its connected remote output devices, etc.

[0103] Of course, for the sake of simplicity, Figure 7 Only some of the components of the electronic device 90 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 90 may include any other suitable components depending on the specific application.

[0104] Exemplary methods

[0105] Figure 8 This is a schematic flowchart illustrating a method for generating a model file according to an exemplary embodiment of this disclosure. The method for generating a model file provided in this embodiment can be implemented using any of the electronic devices provided in the above embodiments of this disclosure, such as... Figure 8 The method shown may include the following steps:

[0106] Step 510: Based on the neural network model to be compiled, determine the first intermediate representation corresponding to the neural network model.

[0107] The first intermediate means that it includes multiple operators.

[0108] Step 520: Based on the various types of computational units of the neural network model to be executed and the operators in the first intermediate representation, the first intermediate representation is split to generate multiple second intermediate representations.

[0109] Each second intermediate representation includes at least one operator operated by the same type of computational unit.

[0110] Step 530: Generate the model file corresponding to the neural network model based on multiple second intermediate representations.

[0111] Figure 9 This is a flowchart illustrating a method for generating model files provided in another exemplary embodiment of this disclosure.

[0112] In some alternative embodiments, in the above... Figure 8 Based on the illustrated embodiments, as Figure 9 As shown, in step 520, the first intermediate representation is split according to the various types of computational units of the neural network model to be executed and the operators in the first intermediate representation to generate multiple second intermediate representations, which may include:

[0113] Step 5210: Based on the various types of computational units of the neural network model to be executed and the operators supported by each type of computational unit, determine the computational units corresponding to each operator in the first intermediate representation.

[0114] Step 5220: Based on the computational units corresponding to each operator in the first intermediate representation, the first intermediate representation is split into multiple second intermediate representations.

[0115] In this context, any two second intermediate values ​​indicate that the corresponding computational units are of different types.

[0116] Figure 10 This is a flowchart illustrating a method for generating model files provided in yet another exemplary embodiment of this disclosure.

[0117] In some alternative embodiments, based on any of the above embodiments, such as Figure 10 As shown, step 530, which generates the model file corresponding to the neural network model based on multiple second intermediate representations, may include:

[0118] Step 5310: Based on multiple second intermediate representations and the computing units corresponding to each second intermediate representation, generate file information and model description information in preset formats corresponding to each computing unit.

[0119] The preset format file information includes the second intermediate representation of the corresponding preset format executable file. The model description information includes the structural parameters of the neural network model.

[0120] Step 5320: Generate the model file corresponding to the neural network model based on the file information and model description information in the preset format corresponding to each computing unit.

[0121] In some optional embodiments, step 5320, which generates a model file corresponding to the neural network model based on the file information and model description information in a preset format corresponding to each computing unit, may include:

[0122] The file information in the preset format corresponding to each computing unit is determined as the file information segment in the model file; the model description information is determined as the description information segment in the model file; an initial model file is generated based on the file information segment and the description information segment; a first hash code is generated based on the initial model file; the first hash code is written into the hash code segment in the initial model file to obtain the model file corresponding to the neural network model.

[0123] In some optional embodiments, the model description information is determined as a description information segment in the model file, including:

[0124] A cross-platform serialization library is used to define the model description information as the description information segment in the model file.

[0125] Figure 11 This is a flowchart illustrating a method for generating model files provided in yet another exemplary embodiment of this disclosure.

[0126] In some optional embodiments, based on any of the above embodiments, the number of neural network models to be compiled is multiple; such as Figure 11 As shown, for each neural network model, after generating the model file corresponding to the neural network model based on steps 510 to 530, the method of this embodiment may further include:

[0127] Step 610: Link the model files corresponding to the multiple neural network models to obtain the target model file.

[0128] Figure 12 This is a flowchart illustrating a method for generating model files provided in yet another exemplary embodiment of this disclosure.

[0129] In some alternative embodiments, such as Figure 12 As shown, step 610 involves linking the model files corresponding to multiple neural network models to obtain the target model file, which may include:

[0130] Step 6110: Parse multiple model files to obtain information segments from the multiple model files.

[0131] Step 6120: Merge the information segments to obtain the merged result.

[0132] Step 6130: In response to the presence of multiple sets of identical model parameters in the merged result, the multiple sets of identical model parameters are compressed to obtain the compressed result.

[0133] Step 6140: Perform static relocation on the compression result to obtain the target model file.

[0134] In some optional embodiments, the compression result includes a file information segment and a description information segment after merging and compressing the model parameters; the file information segment includes a sub-information segment storing static relocation items. Step 6140, which performs static relocation on the compression result to obtain the target model file, may include:

[0135] Based on the sub-information segment containing the static relocation item in the file information segment of the compression result, the content of the storage address pointed to by the static relocation item in the compression result is corrected to obtain the correction result; based on the correction result, the second hash code segment is generated; based on the correction result and the second hash code segment, the target model file is generated.

[0136] The beneficial technical effects corresponding to the exemplary embodiments of this method can be found in the corresponding beneficial technical effects of the exemplary electronic device section described above, and will not be repeated here.

[0137] Any method for generating a model file provided in this disclosure can be executed by any suitable electronic device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any method for generating a model file provided in this disclosure can be executed by a processor, such as by a processor executing any method for generating a model file mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0138] Exemplary device

[0139] Figure 13 This is a schematic diagram of the structure of an apparatus for generating model files provided in an exemplary embodiment of this disclosure. For example... Figure 13 As shown, the apparatus 70 for generating model files includes a memory 71 and an electronic device 20 provided in any of the above embodiments of this disclosure. The electronic device 20 is used to generate model files, and the memory 71 is used to store the model files generated by the electronic device 20.

[0140] Optionally, when there are multiple neural network models to be compiled, the memory 71 is also used to store the target model file obtained by linking the model files of multiple neural network models.

[0141] Figure 14 This is a schematic diagram of an apparatus for generating model files provided in another exemplary embodiment of this disclosure. The apparatus for generating model files provided in this embodiment can be used to execute the method for generating model files provided in any of the above embodiments of this disclosure. Figure 14As shown, the apparatus for generating model files in this embodiment of the present disclosure may include: a determining unit 81, a splitting unit 82, and a model file generating unit 83.

[0142] The determining unit 81 is used to determine the first intermediate representation corresponding to the neural network model based on the neural network model to be compiled; the first intermediate representation includes multiple operators.

[0143] The splitting unit 82 is used to split the first intermediate representation according to the various types of computing units of the neural network model to be executed and the operators in the first intermediate representation to generate multiple second intermediate representations, each of which includes at least one operator operated by computing units of the same type.

[0144] The model file generation unit 83 is used to generate the model file corresponding to the neural network model based on multiple second intermediate representations.

[0145] In some optional embodiments, the splitting unit 82 is specifically used for:

[0146] Based on the various types of computational units in the neural network model to be executed and the operators supported by each type of computational unit, the computational units corresponding to each operator in the first intermediate representation are determined. Based on the computational units corresponding to each operator in the first intermediate representation, the first intermediate representation is split into multiple second intermediate representations. The types of computational units corresponding to any two second intermediate representations are different.

[0147] In some optional embodiments, based on any of the above embodiments, the model file generation unit 83 is specifically used for:

[0148] Based on multiple second intermediate representations and the corresponding computational units for each second intermediate representation, pre-formatted file information and model description information are generated for each computational unit. The pre-formatted file information includes executable files in a pre-formatted format corresponding to the second intermediate representations. The model description information includes the structural parameters of the neural network model. Based on the pre-formatted file information and model description information for each computational unit, a model file corresponding to the neural network model is generated.

[0149] In some optional embodiments, the model file generation unit 83 is specifically used for:

[0150] The file information in the preset format corresponding to each computing unit is determined as the file information segment in the model file; the model description information is determined as the description information segment in the model file; an initial model file is generated based on the file information segment and the description information segment; a first hash code is generated based on the initial model file; the first hash code is written into the hash code segment in the initial model file to obtain the model file corresponding to the neural network model.

[0151] In some optional embodiments, the model file generation unit 83 is specifically used for:

[0152] A cross-platform serialization library is used to define the model description information as the description information segment in the model file.

[0153] Figure 15 This is a schematic diagram of the structure of an apparatus for generating model files provided in another exemplary embodiment of the present disclosure.

[0154] In some optional embodiments, based on any of the above embodiments, the number of neural network models to be compiled is multiple; such as Figure 15 As shown, the apparatus in this embodiment further includes a linking unit 84. After generating a model file corresponding to each neural network model, the linking unit 84 is further configured to link the model files corresponding to multiple neural network models respectively to obtain a target model file.

[0155] In some alternative embodiments, the linking unit 84 is specifically used for:

[0156] Multiple model files are parsed to obtain information segments. These segments are then merged to obtain a merged result. If multiple sets of identical model parameters are found in the merged result, these parameters are compressed to obtain a compressed result. Finally, the compressed result is statically relocated to obtain the target model file.

[0157] In some optional embodiments, the compression result includes a file information segment and a description information segment after merging and compressing the model parameters; the file information segment includes a sub-information segment storing static relocation items. Link unit 84 is specifically used for:

[0158] Based on the sub-information segment containing the static relocation item in the file information segment of the compression result, the content of the storage address pointed to by the static relocation item in the compression result is corrected to obtain the correction result; based on the correction result, the second hash code segment is generated; based on the correction result and the second hash code segment, the target model file is generated.

[0159] The beneficial technical effects corresponding to the exemplary embodiments of this device can be found in the corresponding beneficial technical effects in the above-described exemplary electronic device or exemplary method sections, and will not be repeated here.

[0160] Exemplary computer program products and computer-readable storage media

[0161] In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods in the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0162] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0163] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the methods in the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0164] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0165] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0166] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. An electronic device, comprising: One or more first processors; wherein the one or more first processors are configured to: Based on the neural network model to be compiled, a first intermediate representation corresponding to the neural network model is determined; the first intermediate representation includes multiple operators. Based on the various types of computing units of the neural network model to be executed and the operators in the first intermediate representation, the first intermediate representation is split to generate multiple second intermediate representations, each of which includes at least one operator operated by computing units of the same type. Based on multiple second intermediate representations, a model file corresponding to the neural network model is generated.

2. The electronic device according to claim 1, wherein, The one or more first processors, based on the various types of computational units of the neural network model to be executed and the operators in the first intermediate representation, split the first intermediate representation to generate multiple second intermediate representations, including: Based on the various types of computational units of the neural network model to be executed and the operators supported by each type of computational unit, determine the computational units corresponding to each operator in the first intermediate representation; Based on the computational units corresponding to each operator in the first intermediate representation, the first intermediate representation is split into multiple second intermediate representations, and the computational units corresponding to any two second intermediate representations are of different types.

3. The electronic device according to claim 1, wherein, The one or more first processors generate a model file corresponding to the neural network model based on multiple second intermediate representations, including: Based on multiple second intermediate representations and the computing unit corresponding to each second intermediate representation, file information and model description information in a preset format are generated for each computing unit; the file information in the preset format includes an executable file in a preset format corresponding to the second intermediate representation; the model description information includes the structural parameters of the neural network model. Based on the file information in the preset format and the model description information corresponding to each of the computing units, the model file corresponding to the neural network model is generated.

4. The electronic device according to claim 3, wherein, The one or more first processors generate the model file corresponding to the neural network model based on the file information in the preset format corresponding to each of the computing units and the model description information, including: The file information in the preset format corresponding to each computing unit is determined as the file information segment in the model file; The model description information is determined as the description information segment in the model file; Based on the file information segment and the description information segment, an initial model file is generated; Based on the initial model file, generate the first hash code; The first hash code is written into the hash code segment of the initial model file to obtain the model file corresponding to the neural network model.

5. The electronic device according to any one of claims 1-4, wherein, The number of neural network models to be compiled is multiple; the electronic device further includes a second processor, which is configured to: For each neural network model, after generating the model file corresponding to the neural network model based on multiple second intermediate representations, the model files corresponding to the multiple neural network models are linked to obtain the target model file.

6. The electronic device according to claim 5, wherein, The second processor links the model files corresponding to the multiple neural network models to obtain a target model file, including: The multiple model files are parsed to obtain information segments from the multiple model files; The information segments are merged to obtain the merged result; In response to the presence of multiple sets of identical model parameters in the merged result, the multiple sets of identical model parameters are compressed to obtain a compressed result; The compression result is statically relocated to obtain the target model file.

7. The electronic device according to claim 6, wherein, The compression result includes a file information segment and a description information segment after merging and compressing model parameters; the file information segment includes a sub-information segment storing static relocation items; The second processor performs static relocation on the compression result to obtain the target model file, including: Based on the sub-information segment containing the static relocation item in the file information segment of the compression result, the content of the storage address pointed to by the static relocation item in the compression result is modified to obtain the correction result; Based on the correction results, a second hash code segment is generated; Based on the correction result and the second hash code segment, the target model file is generated.

8. A method for generating model files, comprising: Based on the neural network model to be compiled, a first intermediate representation corresponding to the neural network model is determined; the first intermediate representation includes multiple operators. Based on the various types of computing units of the neural network model to be executed and the operators in the first intermediate representation, the first intermediate representation is split to generate multiple second intermediate representations, each of which includes at least one operator operated by computing units of the same type. Based on multiple second intermediate representations, a model file corresponding to the neural network model is generated.

9. The method according to claim 8, wherein, The first intermediate representation is split according to the various types of computational units of the neural network model to be executed and the operators in the first intermediate representation to generate multiple second intermediate representations, including: Based on the various types of computational units of the neural network model to be executed and the operators supported by each type of computational unit, determine the computational units corresponding to each operator in the first intermediate representation; Based on the computational units corresponding to each operator in the first intermediate representation, the first intermediate representation is split into multiple second intermediate representations, and the computational units corresponding to any two second intermediate representations are of different types.

10. The method according to claim 8, wherein, The step of generating the model file corresponding to the neural network model based on multiple second intermediate representations includes: Based on multiple second intermediate representations and the computing unit corresponding to each second intermediate representation, file information and model description information in a preset format are generated for each computing unit; the file information in the preset format includes an executable file in a preset format corresponding to the second intermediate representation; the model description information includes the structural parameters of the neural network model. Based on the file information in the preset format and the model description information corresponding to each of the computing units, the model file corresponding to the neural network model is generated.

11. The method according to claim 10, wherein, The step of generating the model file corresponding to the neural network model based on the file information in the preset format corresponding to each of the computing units and the model description information includes: The file information in the preset format corresponding to each computing unit is determined as the file information segment in the model file; The model description information is determined as the description information segment in the model file; Based on the file information segment and the description information segment, an initial model file is generated; Based on the initial model file, generate the first hash code; The first hash code is written into the hash code segment of the initial model file to obtain the model file corresponding to the neural network model.

12. The method according to any one of claims 8-11, wherein, The number of neural network models to be compiled is multiple; the method further includes: For each neural network model, after generating the model file corresponding to the neural network model based on multiple second intermediate representations, the model files corresponding to the multiple neural network models are linked to obtain the target model file.

13. The method according to claim 12, wherein, The step of linking the model files corresponding to the multiple neural network models to obtain the target model file includes: The multiple model files are parsed to obtain information segments from the multiple model files; The information segments are merged to obtain the merged result; In response to the presence of multiple sets of identical model parameters in the merged result, the multiple sets of identical model parameters are compressed to obtain a compressed result; The compression result is statically relocated to obtain the target model file.

14. The method according to claim 13, wherein, The compression result includes a file information segment and a description information segment after merging and compressing model parameters; the file information segment includes a sub-information segment storing static relocation items; The step of performing static relocation on the compression result to obtain the target model file includes: Based on the sub-information segment containing the static relocation item in the file information segment of the compression result, the content of the storage address pointed to by the static relocation item in the compression result is modified to obtain the correction result; Based on the correction results, a second hash code segment is generated; Based on the correction result and the second hash code segment, the target model file is generated.

15. A computer-readable storage medium storing a computer program that is executed by a processor to perform the method for generating a model file according to any one of claims 8-14.

16. An apparatus for generating model files, comprising: A determining unit is used to determine a first intermediate representation corresponding to the neural network model based on the neural network model to be compiled; The first intermediate representation includes multiple operators; The splitting unit is used to split the first intermediate representation according to the various types of computing units of the neural network model to be executed and the operators in the first intermediate representation to generate a plurality of second intermediate representations, each of the second intermediate representations including at least one operator operated by computing units of the same type; The model file generation unit is used to generate a model file corresponding to the neural network model based on multiple second intermediate representations.

Images