Model processing method and apparatus, and terminal device

Abstract

The present application relates to the technical field of artificial intelligence. Provided are a model processing method and apparatus, and a terminal device, which are used for reducing the size of a neural network model. The method comprises: acquiring first model information of a neural network model, wherein the first model information comprises a plurality of operator nodes; determining at least one executable file corresponding to each of the plurality of operator nodes, wherein each of the at least one executable file comprises a kernel function corresponding to the operator node; and sending second model information of the neural network model, wherein the second model information comprises a plurality of different executable files among the executable files corresponding to the plurality of operator nodes.

Description

Model processing method, device and terminal equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on August 23, 2023, with application number 202311072000.8 and application name “A model processing method, device and terminal device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence technology, and in particular to a model processing method, apparatus, and terminal device. Background Art

[0003] Artificial intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. AI is a branch of computer science that seeks to understand the essence of intelligence and enable machines to possess the capabilities of perception, reasoning, and decision-making.

[0004] With the development of artificial intelligence (AI) technology, neural network models, which serve as AI models, have become increasingly complex, and as a result, their sizes have also grown. Some neural network models are even gigabytes (GB) or larger. Consequently, when a terminal device compiles and executes a neural network model deployed on that terminal device, problems such as high power consumption and high memory usage arise.

[0005] Summary of the Invention

[0006] The present application provides a model processing method, apparatus, and terminal device for reducing the size of a neural network model, thereby reducing the power consumption and memory occupied by the neural network model during compilation and execution.

[0007] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0008] In a first aspect, a model processing method is provided, the method comprising: obtaining first model information of a neural network model, the first model information including multiple operator nodes, for example, the neural network model may be a tf model, a pytorch model, a caffe model, or a davinci model, etc.; determining at least one executable file corresponding to each of the multiple operator nodes, wherein at least one executable file corresponding to each operator node may mean that the operator node may obtain one or more executable files after compilation, and each executable file in the at least one executable file includes a kernel function corresponding to the operator node. Exemplarily, the extension of the executable file may be ".o"; and sending second model information of the neural network model, wherein the second model information includes multiple different executable files among the executable files corresponding to the multiple operator nodes.

[0009] In the above technical solution, the first model information of the neural network model obtained includes multiple operator nodes. When determining the executable files corresponding to the multiple operator nodes, the executable files are obtained based on the kernel function level, that is, each executable file includes a kernel function corresponding to the operator node, thereby realizing the reuse of executable files at the kernel function level, thereby reducing the repeated compilation of kernel functions and reducing compilation time; at this time, the second model information sent includes multiple different executable files among the executable files corresponding to the multiple operator nodes, thereby avoiding the repeated sending of the same executable file, and thereby reducing the memory occupied during the execution of the neural network model.

[0010] In one possible implementation of the first aspect, determining at least one executable file corresponding to each of the multiple operator nodes includes: for each of the multiple operator nodes, compiling at least one kernel function corresponding to the operator node to obtain at least one executable file corresponding to the operator node. In the above possible implementation, when determining the executable files corresponding to the multiple operator nodes, the operator nodes are compiled based on the kernel function level, and each compiled executable file includes a kernel function corresponding to the operator node, thereby achieving reuse of executable files at the kernel function level, thereby reducing repeated compilation of kernel functions and reducing compilation time.

[0011] In a possible implementation of the first aspect, determining at least one executable file corresponding to each of the multiple operator nodes includes: for each of the multiple operator nodes, obtaining at least one executable file corresponding to each of the multiple operator nodes from an executable file library, wherein the executable file library includes an executable file corresponding to each of a plurality of preset kernel functions. In the above possible implementation, when determining the executable files corresponding to the multiple operator nodes, at least one executable file corresponding to each of the multiple operator nodes can be obtained from the executable file library, and the obtained executable file can realize the reuse of executable files at the kernel function level, thereby reducing the compilation time of the operator nodes.

[0012] In a possible implementation of the first aspect, the method further includes determining mapping relationship information indicating a correspondence between the multiple operator nodes, the different executable files, and at least one kernel function corresponding to each operator node; wherein the second model information also includes the mapping relationship information. In this possible implementation, determining the mapping relationship information enables different operator nodes to reuse the same executable file, thereby reducing the size of the compiled second model information.

[0013] In one possible implementation of the first aspect, the method further includes packaging multiple different executable files among the executable files corresponding to the multiple operator nodes to obtain second model information of the neural network model. In this possible implementation, packaging multiple different executable files among the executable files corresponding to the multiple operator nodes avoids repeated packaging and reduces the size of the packaged model.

[0014] In one possible implementation of the first aspect, the multiple computing nodes include a pending computing node, the pending computing node corresponds to a first executable file, and sending the second model information of the neural network model includes: sending the first executable file when the first executable file has not been sent. In this possible implementation, when sending the executable files corresponding to the multiple operator nodes, repeated sending can be avoided, reducing sending time and memory usage.

[0015] In one possible implementation of the first aspect, when the second executable file corresponding to the neural network model is the same as the second executable file corresponding to another neural network model, the neural network model and the other neural network model share the same second executable file. In the above possible implementation, different neural network models can also reuse the executable file of the same kernel function, thereby further reducing the size of the neural network model and lowering memory usage.

[0016] In a second aspect, a model processing method is provided, which includes: obtaining second model information of a neural network model, the neural network model corresponding to multiple computing nodes, the second model information including multiple different executable files in the executable files corresponding to the multiple operator nodes, each executable file including a kernel function; executing the multiple different executable files in the second model information.

[0017] In the above technical solution, the second model information obtained includes multiple different executable files among the executable files corresponding to the multiple operator nodes, and each executable file includes a kernel function, thereby avoiding repeated sending of the same executable file, and when executing the second model information, the executable file at the kernel function level can be reused, thereby reducing the size of the network model and reducing memory usage.

[0018] In a possible implementation of the second aspect, the second model information further includes mapping relationship information, which is used to indicate the correspondence between the multiple operator nodes, the different executable files, and at least one kernel function corresponding to each operator node. The method further includes: determining a first kernel function corresponding to a pending computing node among the multiple computing nodes based on parameter information of the pending computing node; and obtaining a first executable file from the multiple different executable files based on the first kernel function and the mapping relationship information, wherein the first executable file includes the first kernel function. In the above possible implementation, by determining the mapping relationship information, different operator nodes can reuse the same executable file, thereby reducing the memory occupied during model execution.

[0019] In one possible implementation of the second aspect, the multiple computing nodes include a pending computing node, the pending computing node corresponds to a first kernel function, and obtaining second model information of the neural network model includes receiving a first executable file, the first executable file including the first kernel function. In this possible implementation, repeated distribution of the same executable file can be avoided, thereby reducing distribution time and memory usage.

[0020] In one possible implementation of the second aspect, when the second executable file corresponding to the neural network model is the same as the second executable file corresponding to another neural network model, the neural network model and the other neural network model share the same second executable file. In the above possible implementation, different neural network models can also reuse the executable file of the same kernel function, thereby further reducing the size of the neural network model and lowering memory usage.

[0021] In a third aspect, a model processing device is provided, which includes: an acquisition unit for acquiring first model information of a neural network model, the first model information including multiple operator nodes; a processing unit for determining at least one executable file corresponding to each of the multiple operator nodes, each executable file in the at least one executable file including a kernel function corresponding to the operator node; and a sending unit for sending second model information of the neural network model, the second model information including multiple different executable files in the executable files corresponding to the multiple operator nodes.

[0022] In a possible implementation of the third aspect, the processing unit is further used to: for each operator node among the multiple operator nodes, compile at least one kernel function corresponding to the operator node to obtain at least one executable file corresponding to the operator node.

[0023] In a possible implementation of the third aspect, the processing unit is also used to: for each operator node among the multiple operator nodes, obtain at least one executable file corresponding to each operator node in the multiple operator nodes from an executable file library, and the executable file library includes an executable file corresponding to each kernel function in a preset multiple kernel functions.

[0024] In a possible implementation of the third aspect, the processing unit is also used to: determine mapping relationship information, which is used to indicate the correspondence between the multiple operator nodes, the different executable files and at least one kernel function corresponding to each operator node; wherein the second model information also includes the mapping relationship information.

[0025] In a possible implementation of the third aspect, the processing unit is further used to: package multiple different executable files among the executable files corresponding to the multiple operator nodes to obtain second model information of the neural network model.

[0026] In a possible implementation of the third aspect, the multiple computing nodes include a to-be-processed computing node, which corresponds to a first executable file; and the sending unit is further configured to send the first executable file when the first executable file has not been sent.

[0027] In a possible implementation of the third aspect, when the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models share the same second executable file.

[0028] In a fourth aspect, a model processing device is provided, which includes: a receiving unit for obtaining second model information of a neural network model, where the neural network model corresponds to multiple computing nodes, and the second model information includes multiple different executable files among the executable files corresponding to the multiple operator nodes, and each executable file includes a kernel function; and a processing unit for executing the multiple different executable files in the second model information.

[0029] In a possible implementation of the fourth aspect, the second model information also includes mapping relationship information, which is used to indicate the correspondence between the multiple operator nodes, the different executable files and at least one kernel function corresponding to each operator node; the processing unit is also used to determine the first kernel function corresponding to the node to be processed based on the parameter information of the node to be processed among the multiple computing nodes; the processing unit is also used to obtain a first executable file from the multiple different executable files based on the first kernel function and the mapping relationship information, and the first executable file includes a first kernel function.

[0030] In a possible implementation of the fourth aspect, the multiple computing nodes include a computing node to be processed, and the computing node to be processed corresponds to a first kernel function; the receiving unit is further used to receive a first executable file, and the first executable file includes the first kernel function.

[0031] In a possible implementation of the fourth aspect, when the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models share the same second executable file.

[0032] In a fifth aspect, a model processing device is provided, which includes a processor and a memory, wherein instructions are stored in the memory. When the processor executes the instructions, the model processing device executes the model processing method provided in the first aspect or any possible implementation of the first aspect.

[0033] In a sixth aspect, a model processing device is provided, which includes a processor and a memory, wherein instructions are stored in the memory. When the processor executes the instructions, the model processing device executes the model processing method provided in the second aspect or any possible implementation of the second aspect.

[0034] In another aspect of the present application, a terminal device is provided, which includes: the third aspect or any possible implementation of the third aspect, or the model processing device provided by the fifth aspect, and the fourth aspect or any possible implementation of the fourth aspect, or the model processing device provided by the sixth aspect.

[0035] In another aspect of the present application, a computer-readable storage medium is provided, in which a computer program or instruction is stored. When the computer program or instruction is executed, the model processing method provided by the first aspect or any possible implementation of the first aspect is implemented.

[0036] In another aspect of the present application, a computer-readable storage medium is provided, in which a computer program or instruction is stored. When the computer program or instruction is executed, the model processing method provided in the second aspect or any possible implementation of the second aspect is implemented.

[0037] In another aspect of the present application, a computer program product is provided, which includes: a computer program (also referred to as code, or instructions), which, when executed, enables a computer to execute the model processing method provided in the first aspect or any possible implementation of the first aspect.

[0038] In another aspect of the present application, a computer program product is provided, which includes: a computer program (also referred to as code, or instructions), which, when executed, enables a computer to execute the model processing method provided in the second aspect or any possible implementation of the second aspect.

[0039] It can be understood that the beneficial effects that can be achieved by any of the model processing devices, terminal devices, computer-readable storage media and computer program products provided above can correspond to the beneficial effects of the model processing method provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] FIG1 is a schematic structural diagram of a model processing device provided in an embodiment of the present application;

[0041] FIG2 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0042] FIG3 is a flow chart of a model processing method provided in an embodiment of the present application;

[0043] FIG4 is a schematic diagram of a model compilation provided in an embodiment of the present application;

[0044] FIG5 is a flow chart of another model processing method provided in an embodiment of the present application;

[0045] FIG6 is a schematic diagram of a model packaging provided in an embodiment of the present application;

[0046] FIG7 is a flow chart of another model processing method provided in an embodiment of the present application;

[0047] FIG8 is a schematic diagram of a model execution provided in an embodiment of the present application;

[0048] FIG9 is a schematic structural diagram of a model processing device provided in an embodiment of the present application;

[0049] FIG10 is a schematic structural diagram of another model processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The following sections discuss the making and use of various embodiments in detail. However, it should be understood that many applicable inventive concepts provided herein can be implemented in a variety of specific contexts. The specific embodiments discussed are merely illustrative of specific ways to implement and use this description and technology and do not limit the scope of this application.

[0051] Unless defined otherwise, all technical and scientific terms used herein have the same meanings as commonly understood by one of ordinary skill in the art.

[0052] Various circuits or other components may be described or referred to as being "configured to" perform one or more tasks. In this case, "configured to" is used to imply structure by indicating that the circuit / component includes structure (e.g., circuitry) that performs the one or more tasks during operation. Thus, even when a specified circuit / component is not currently operational (e.g., not turned on), the circuit / component may be referred to as being configured to perform the task. Circuits / components used with the phrase "configured to" include hardware, such as circuitry that performs an operation, etc.

[0053] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, a, b and c; where a, b and c can be single or multiple.

[0054] The embodiments of this application use terms such as "first" and "second" to distinguish objects with similar names, functions, or effects. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or order of execution. The term "coupled" is used to indicate an electrical connection, including direct connection via wires or connectors or indirect connection via other devices. Therefore, "coupling" should be considered a broadly defined electronic communication connection.

[0055] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0056] Before introducing the embodiments of the present application, the relevant background technology involved in the present application is first introduced and explained.

[0057] A neural network (NN) is a network structure that processes information by mimicking the behavioral characteristics of animal neural networks. A NN is composed of a large number of interconnected nodes (also called neurons). It processes information by learning and training input information based on a specific computational model. A NN consists of an input layer, a hidden layer, and an output layer. The input layer receives input signals, while the output layer outputs the neural network's computational results. The hidden layer handles learning, training, and other computational processes and serves as the network's memory unit. The memory function of the hidden layer is represented by a weight matrix, with each neuron typically assigned a weight (also called a weight coefficient).

[0058] A neural network model may refer to a network model obtained by model training based on a neural network. A neural network model generally includes the following information: network structure description information, weight information, and multiple operators (also referred to as operator nodes). The network structure description information can be used to describe the network structure of the neural network model, the weight information can include the weights in the neural network model, and the multiple operators can perform calculations and processing according to the network structure based on the weight information.

[0059] An operator is a node with data transmission and processing capabilities. Each operator can have the following parameter information: operator type, data type, input data size (shape), data layout, etc. The input data size type can include static and dynamic. When an operator does not involve multiple branches (for example, the input data size type is static), the operator can correspond to a kernel function; when an operator has multiple branches (for example, the input data size type is dynamic), the operator can correspond to multiple kernel functions.

[0060] A kernel function may refer to the compiled code of an operator function that can be executed directly on the hardware. A function may correspond to one or a class of execution strategies.

[0061] An executable file may refer to a compiled binary file of an operator that can be executed directly on hardware. An executable file may include one or more kernel functions. In an embodiment of the present application, an executable file includes one kernel function. In one example, the extension of the executable file may be ".o".

[0062] The kernel function number may be a number defined for quickly finding a kernel function, and the number may be used to index the kernel function. Kernel function numbers in the same executable file are unique.

[0063] In recent years, artificial intelligence (AI) technology has developed rapidly and has been widely applied in various fields. AI is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use this knowledge to achieve optimal results. AI is a branch of computer science that seeks to understand the essence of intelligence and enable machines to possess the capabilities of perception, reasoning, and decision-making.

[0064] With the development of artificial intelligence (AI) technology, neural network models, which serve as AI models, have become increasingly complex, and as a result, their sizes have also grown. Some neural network models are even gigabytes (GB) or larger. Consequently, when a terminal device compiles and executes a neural network model deployed on that terminal device, problems such as high power consumption and high memory usage arise.

[0065] In the related art, when compiling a neural network model (such as a tf model, a pytorch model, a caffe model, and a davinci model, etc.), the kernel functions corresponding to the neural network model are compiled together and packaged into the model or application, that is, each operator in the neural network model corresponds to an executable file after compilation, and the packaged model includes the executable file compiled after all operators of the neural network model, and the executable file usually includes multiple kernel functions. When the model is sent for execution, the entire binary file (such as a .o file) compiled by the neural network model is sent. In addition, for the use of the acceleration library of the provided neural network model (such as cuDNN provided by NV), the acceleration library is also packaged into the model when the model is compiled. The above solution still has problems such as high power consumption and high memory usage when compiling and executing the neural network model.

[0066] Based on this, the embodiments of the present application provide a model processing method that can be used to: implement kernel function-level reuse during model compilation, thereby reducing repeated compilation of kernel functions and shortening compilation time; avoid repeated packaging during model packaging, reducing the size of the packaged model; and implement kernel function-level dispatching when the model is dispatched for execution, avoiding repeated dispatching, reducing dispatching time and memory usage. Therefore, the embodiments of the present application can solve the aforementioned problems of high power consumption and high memory usage during the compilation and execution of neural network models.

[0067] The technical solution provided in the embodiment of the present application can be applied to a model processing device. As shown in Figure 1, the model processing device may include a compiler, a packager, and an executor. Among them, the compiler can be used to compile multiple operator nodes in the neural network model to obtain an executable file corresponding to each operator node. Each executable file may include a kernel function of the operator node. The packager can be used to package the executable files corresponding to the multiple operator nodes, for example, to perform deduplication and packaging on the executable files corresponding to the multiple operator nodes. The executor can be used to receive and execute the executable file issued by the packager, and can also be used to receive network structure description information and weight information of the neural network model, etc. The network structure description information and the weight information can also be packaged and issued together with the above-mentioned executable file.

[0068] Optionally, the model processing device may not include the packager. After compiling the executable file corresponding to each operator node, the compiler may also send the executable file to the executor by deduplicating it, and the executor receives and executes the executable file sent by the compiler.

[0069] In a possible embodiment, the model processing device may be a terminal device, which may include but is not limited to: a mobile phone, a tablet computer, a laptop computer, a PDA, an ultra-mobile personal computer (umPC), a mobile internet device (MID), a netbook, a camcorder, a camera, a wearable device (such as a smart watch and a smart bracelet), an in-vehicle device (such as a car, a bicycle, an electric car, an airplane, a ship, a train, a high-speed train, etc.), a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a smart home device (such as a refrigerator, a television, an air conditioner, an electric meter, etc.), an intelligent robot, a workshop device, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, or a wireless terminal in a smart home, a flying device (such as an intelligent robot, a hot air balloon, a drone, an airplane), etc.

[0070] The following takes a mobile phone as an example to introduce the specific structure of the terminal device.

[0071] FIG2 is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. The terminal device may include components such as a radio frequency (RF) circuit 110, a memory 120, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a processor 170, and a power supply 180. The following is a detailed introduction to the various components of the terminal device:

[0072] RF circuitry 110 can be used to send and receive information, or receive or send signals during a call. Specifically, it receives downlink information from the base station and passes it to processor 170 for processing; it also sends uplink data to the base station. Typically, RF circuitry 110 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, and more. RF circuitry 110 can also communicate with the network and other devices via wireless communication.

[0073] The memory 120 can be used to store data, software programs, and modules. It primarily includes a program storage area and a data storage area. The program storage area can store an operating system and at least one application required for a function, such as sound playback or image playback. The data storage area can store data generated based on the use of the terminal device, such as audio data, image data, and a phone book. Furthermore, the terminal device may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state memory device.

[0074] The input unit 130 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the terminal device. Specifically, the input unit 130 may include a touch screen 131 and other input devices 132. The touch screen 131 can collect user touch operations on or near it (such as operations performed by the user using any suitable object or accessory such as a finger, stylus, etc. on or near the touch screen) and drive the corresponding connection device according to a pre-set program. Optionally, the other input devices 132 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, power switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0075] The display unit 140 can be used to display information input by the user or information provided to the user, as well as various menus of the terminal device. In one example, the display unit 140 may include a display screen 141, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. Furthermore, the touch screen 131 may cover the display screen 141. When the touch screen 131 detects a touch operation on or near it, it transmits the information to the processor 170 to determine the type of touch event. The processor 170 then provides a corresponding visual output on the display screen 141 based on the type of touch event. Although the touch screen 131 and the display screen 141 are shown as two independent components to implement the input and output functions of the terminal device in the figure, in some embodiments, the touch screen 131 and the display screen 141 may be integrated to implement the input and output functions of the terminal device.

[0076] Sensor 150 may include one or more sensors for providing various status assessments for the terminal device. Specifically, sensor 150 may include a light sensor, which can be used in imaging applications, i.e., as a component of a camera or webcam. Sensor 150 may also include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor. Sensor 150 can detect the acceleration / deceleration, orientation, open / closed state of the terminal device, relative positioning of components, or temperature changes of the terminal device.

[0077] The audio circuit 160, speaker, and microphone provide an audio interface between the user and the terminal device. The audio circuit 160 converts received audio data into electrical signals and transmits them to the speaker, which then converts them into sound signals for output. The microphone, on the other hand, converts collected sound signals into electrical signals, which are then received by the audio circuit 160 and converted into audio data. The audio data is then output to the RF circuit 110 for transmission to, for example, another mobile phone, or to the memory 120 for further processing.

[0078] The processor 170 is the control center of the terminal device. It uses various interfaces and lines to connect the various parts of the terminal device. By running or executing software programs and / or modules stored in the memory 120 and calling data stored in the memory 120, it performs various functions of the terminal device and processes data, thereby monitoring the terminal device as a whole. Optionally, the processor 170 may include one or more processing units, which may include but are not limited to: a central processing unit (CPU), an application processor, a digital signal processor (DSP), a network processor (NPU), a graphics processing unit (GPU), an image signal processor (ISP), a microcontroller or a microprocessor. Furthermore, the processor 170 may also include other hardware circuits or accelerators, such as an application specific integrated circuit (ASIC), a complex programmable logic device (CPLD) or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. Optionally, the processor 170 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, etc.

[0079] In an embodiment of the present application, the processor 170 may include a general-purpose processor and an acceleration processor. The general-purpose processor may have general data processing, instruction or command execution, and task delivery capabilities, and the acceleration processor may have data acceleration processing capabilities. In one possible example, the general-purpose processor may include a CPU, a DSP, an application processor, a microcontroller, or a microprocessor, and the general-purpose processors listed above may include a compiler and a packager; in another possible example, the acceleration processor may include an NPU, a GPU, a CPU, an ASIC, or a CPLD, and the acceleration processors listed above may include an executor.

[0080] The terminal device may also include a power supply 180 (e.g., a battery) for powering each component. The power supply 180 may be logically connected to the processor 170 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. In an embodiment of the present application, the power supply 180 may include multiple batteries, and the power management system may support both fast charging technology and non-fast charging technology, so that the power management system may charge the multiple batteries through fast charging technology, or may charge the multiple batteries through non-fast charging technology.

[0081] Although not shown, the terminal device may also include a wireless fidelity (WiFi) module, a Bluetooth module, etc., which will not be described in detail in the embodiments of the present application. Those skilled in the art will understand that the terminal device structure shown in the figure does not constitute a limitation on the terminal device, and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0082] Figure 3 is a flow chart of a model processing method provided in an embodiment of the present application. The method can be applied to the model processing device provided above. The model processing device is used as an example for explanation below. The method includes the following steps.

[0083] S201: Obtain first model information of a neural network model, where the first model information includes multiple operator nodes.

[0084] The neural network model may be any network model obtained based on neural network training. For example, the neural network model may be a TF model, a PyTorch model, a Caffe model, or a DaVinci model. The neural network model may be stored in the terminal device. For example, after the neural network model is trained, the neural network model may be stored in the memory of the terminal device to deploy the neural network model in the terminal device.

[0085] In addition, the multiple operator nodes can be connected according to a certain network structure. Each of the multiple operator nodes can have the following parameter information: operator type, data type, input data size (shape), etc. Among them, the type of the input data size can include static and dynamic. When an operator does not involve multiple branches (for example, the type of the input data size is static), the operator can correspond to a kernel function; when an operator has multiple branches (for example, the type of the input data size is dynamic), the operator can correspond to multiple kernel functions.

[0086] Optionally, the first model information may also include network structure description information and / or weight information, where the network structure description information may be used to indicate the network structure of the neural network model, and the weight information may include the weight of the neural network model.

[0087] In a possible embodiment, the terminal device may include a general-purpose processor and a memory, in which first model information of the neural network model is stored. When the general-purpose processor needs to use the neural network model, the general-purpose processor can obtain the first model information from the memory.

[0088] S202: Determine at least one executable file corresponding to each operator node among the multiple operator nodes, each executable file in the at least one executable file includes a kernel function corresponding to the operator node.

[0089] Each operator node corresponds to at least one executable file, which may mean that after compiling the operator node, one or more executable files can be generated. The executable file may be a directly executable binary file generated after compilation, and thus the executable file may also be referred to as a binary file. Exemplarily, the extension of the executable file may be ".o".

[0090] In the embodiments of this application, an executable file includes only one kernel function, that is, one executable file corresponds to one kernel function. When an operator node corresponds to one kernel function, the operator node generates one executable file after compilation. When an operator node corresponds to multiple kernel functions, the operator node generates multiple executable files after compilation, and the multiple kernel functions correspond one-to-one to the multiple executable files.

[0091] In one possible embodiment, determining the at least one executable file corresponding to each of the multiple operator nodes may include: for each of the multiple operator nodes, compiling the at least one kernel function corresponding to the operator node to obtain the at least one executable file corresponding to the operator node. Specifically, when compiling each operator node, the compilation is performed based on the principle that one executable file includes only one kernel function, to obtain the at least one executable file corresponding to each operator node.

[0092] The operation of compiling an operator node can be specifically performed by a compiler in the general-purpose processor of the terminal device. The compiler can compile different executable files based on different operators and parameter information. If an operator node has multiple branches, multiple executable files can be generated through compilation, each executable file containing a kernel function. If an operator node does not involve multiple branches, a single executable file can be generated through compilation.

[0093] For example, as shown in FIG4 , if the multiple computing nodes include a first computing node op N1 and a second computing node op N2, and the first computing node op N1 and the second computing node op N2 both correspond to multiple kernel functions, then the multiple executable files obtained after compilation of the first computing node op N1 may include kern1 (TK1_1), kern2 (TK1_2), kern3 (TK1_3), …, and the multiple executable files obtained after compilation of the second computing node op N2 may include kern2 (TK2_1), kern3 (TK2_2), kern4 (TK2_3), ….

[0094] Figure 4 illustrates an example where both the first compute node op N1 and the second compute node op N2 correspond to multiple executable files. TK1_1 through TK1_3, and TK2_1 through TK2_3, represent kernel function numbers within the executable files. Kern1 through Kern4 can be used to represent different executable files, specifically hash values. The hash value of an executable file can include information such as function parameters and function content in addition to the function name. This hash value can be used for duplication checking.

[0095] It can be understood that the embodiments of the present application use the hash value of the executable file to perform a duplication check as an example. In actual applications, when an executable file is functionally or logically similar to other executable files, they can also be reused, and the keywords used for duplication checking can also be customized. The embodiments of the present application do not impose specific restrictions on this.

[0096] In another possible embodiment, determining the at least one executable file corresponding to each of the multiple operator nodes may include: for each of the multiple operator nodes, obtaining the at least one executable file corresponding to each of the multiple operator nodes from an executable file library, where the executable file library includes an executable file corresponding to each of the multiple kernel functions. In other words, when compiling each operator node, the corresponding executable file can be directly searched from the executable file library without performing a corresponding compilation operation.

[0097] It is understandable that the executable files in the executable file library may be pre-compiled and pre-configured in the terminal device. The executable file library may include a large number of executable files, and each executable file includes only one kernel function.

[0098] S203: Send second model information of the neural network model, where the second model information includes multiple different executable files among the executable files corresponding to the multiple operator nodes.

[0099] Among them, sending the second model information of the neural network model can also be called sending the second model information of the neural network model. In one possible example, the terminal device may include a general-purpose processor and an acceleration processor, and the general-purpose processor may send the second model information of the neural network model to the acceleration processor to send the second model information to the acceleration processor.

[0100] Optionally, the second model information may also include network structure description information and / or weight information of the neural network model. In one possible example, the general processor of the terminal device may obtain the network structure description information and the weight information from the first model information, and send the network structure description information, the weight information, and the above-mentioned multiple different executable files together in the second model information to the acceleration processor.

[0101] In one possible embodiment, as shown in FIG5 , before the general-purpose processor of the terminal device sends the second model information of the neural network model, the method may further include: S204. Packaging multiple different executable files among the executable files corresponding to the multiple operator nodes to obtain the second model information of the neural network model. In other words, the general-purpose processor of the terminal device may package the multiple different executable files together by de-packaging, so that the second model information can be sent to the acceleration processor at one time.

[0102] The operation of deduplicating and packaging the executable files corresponding to the multiple operator nodes can be specifically performed by a packager in the general processor of the terminal device. Exemplarily, in combination with the compilation result shown in FIG4 , as shown in FIG6 , the packager deduplicates and packages the executable files corresponding to the first computing node op N1 and the second computing node op N2, and the multiple different executable files in the obtained second model information can include kern1, kern2, kern3, kern4, .... Among them, kern2 (TK1_2) and kern3 (TK1_3) in the executable file corresponding to the first computing node op N1 are respectively the same as kern2 (TK2_1) and kern3 (TK2_2) in the executable file corresponding to the second computing node op N2, so that the second model information after deduplication and packaging only includes one kern2 and one kern3.

[0103] In another possible embodiment, the general-purpose processor of the terminal device may send the executable file corresponding to each of the multiple computing nodes to the acceleration processor in sequence according to the execution order of the multiple computing nodes in the neural network model. Taking a certain computing node to be processed among the multiple computing nodes as an example, if the computing node to be processed corresponds to the first executable file, sending the second model information of the neural network model may include: sending the first executable file when the first executable file has not been sent. If the first executable file has been sent before, then not sending the first executable file again.

[0104] Exemplarily, for any node to be processed, determining the executable file corresponding to the node to be processed and sending the executable file may include: determining the kernel function corresponding to the computing node to be processed based on parameter information of the computing node to be processed (for example, the type of input data size); if the node to be processed corresponds to the first kernel function, determining the first executable file based on the first kernel function, the first executable file includes the first kernel function; when the first executable file has not been sent, sending the first executable file.

[0105] Furthermore, the method may further include: determining mapping relationship information, the mapping relationship information being used to indicate a correspondence between the plurality of operator nodes, the plurality of different executable files, and at least one kernel function corresponding to each operator node, wherein the second model information also includes the mapping relationship information.

[0106] In a possible embodiment, after determining at least one executable file corresponding to each of the multiple nodes, the mapping relationship information can also be determined, that is, a correspondence between the multiple computing nodes, the multiple different executable files, and the at least one kernel function corresponding to each operator node is established. Exemplarily, taking the compilation result shown in FIG4 as an example, the mapping relationship information can be [op N1 (TK1_1, TK1_2, TK1_3, ...), op N2 (TK2_1, TK2_2, TK2_3, ...)] --> [kern1 (TK1_1), kern2 (TK1_2, TK2_1), kern3 (TK1_3, TK2_2), kern4 (TK2_3), ...].

[0107] Optionally, when the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network model can share the same second executable file. That is, different neural network models can also reuse the executable file of the same kernel function. Exemplarily, when the terminal device is compiling, packaging and issuing model information of other neural network models, if the executable file corresponding to the other neural network model has been compiled or issued before, the terminal device may no longer perform the compilation, packaging and issuing operations corresponding to the executable file.

[0108] Furthermore, in actual applications, the terminal device can also directly obtain the second model information in the above S201, that is, the compiled model information is configured for the terminal device at the beginning. Specifically, after the training of the neural network is completed, the first model information of the neural network can be compiled, and the executable files of the multiple computing nodes obtained after compilation can be deduplicated and packaged, so that the packaged second model information can be configured in the terminal device. Exemplarily, the binary file package of the neural network model is published offline, and when the binary file package is published, the operator compiler is called by the publishing script to compile in a similar manner as described above, an executable file is formed based on the kernel function level, and the above mapping relationship information is recorded.

[0109] Furthermore, as shown in Figure 7 , the method may further include the following steps, which may be executed by an accelerated processor (eg, NPU or GPU) in the terminal device. S201 to S204 are not shown in Figure 7 .

[0110] S301: Obtain second model information of a neural network model, where the second model information includes multiple different executable files among the executable files corresponding to multiple operator nodes, and each executable file includes a kernel function.

[0111] Among them, the neural network model can be a network model obtained based on neural network training, and the neural network model can include multiple operator nodes, and the multiple operator nodes can be connected according to a certain network structure. The second model information can be model information obtained by compiling the first model information of the neural network model. The relevant description of the second model information is consistent with the description of the second model information in S203 above. For details, please refer to the description in S203 above, and the embodiment of the present application will not be repeated here.

[0112] In addition, each of the multiple operator nodes may correspond to at least one executable file. If an operator node has multiple branches, the operator node may correspond to multiple executable files, each of which includes a kernel function. If an operator node does not involve multiple branches, the operator node may correspond to a single executable file. The multiple different executable files may be obtained by deduplicating the executable files corresponding to the multiple operator nodes.

[0113] In a possible embodiment, when the above-mentioned second model information is sent at one time by the general processor of the terminal device according to the execution order of the multiple computing nodes in the neural network model, the acceleration processor of the terminal device can receive the executable file corresponding to one computing node each time. Exemplarily, taking a certain computing node to be processed among the multiple computing nodes as an example, if the computing node to be processed corresponds to the first executable file, the first executable file can include the first kernel function corresponding to the computing node to be processed, then receiving the second model information of the neural network model can include: receiving the first executable file; wherein the first executable file is sent by the general processor when the first executable file has not been sent. If the first executable file has been sent before, the first executable file will no longer be sent.

[0114] In another possible embodiment, when the second model information is a general processor of the terminal device, and the multiple different executable files are packaged and sent, the acceleration processor of the terminal device can receive the second model information at one time, that is, obtain the multiple different executable files through one reception.

[0115] Optionally, if the second model information further includes mapping relationship information, the mapping relationship information is used to indicate the correspondence between the multiple operator nodes, the multiple different executable files, and the at least one kernel function corresponding to each operator node.

[0116] S302: Execute multiple different executable files in the second model information.

[0117] In a possible embodiment, for the multiple computing nodes, the acceleration processor of the terminal device can execute the executable files corresponding to the multiple computing nodes in sequence according to the execution order of the multiple computing nodes in the neural network model and according to the second model information.

[0118] Furthermore, before executing the executable file corresponding to each computing node in the multiple computing nodes, the method may further include: S303. For each computing node in the multiple computing nodes, obtain the executable file corresponding to the computing node from the second model information.

[0119] Specifically, based on the parameter information of the computing node to be processed among the multiple computing nodes, at least one kernel function corresponding to the node to be processed is determined; based on the at least one kernel function and the mapping relationship information, at least one corresponding executable file is obtained from the multiple different executable files, and the at least one executable file corresponds one-to-one to the at least one kernel function. For example, each executable file includes a kernel function number among the at least one kernel function.

[0120] In one possible example, if the computing node to be processed has multiple branches (for example, the size of the input data is dynamic), the acceleration processor of the terminal device can obtain multiple kernel function numbers corresponding to the computing node to be processed, and then obtain multiple executable files corresponding to the multiple kernel function numbers from the multiple different executable files based on the multiple kernel function numbers and the above-mentioned mapping relationship information. If the computing node to be processed does not involve multiple branches (for example, the size of the input data is static), the acceleration processor of the terminal device can obtain a kernel function number corresponding to the computing node to be processed, and then obtain an executable file corresponding to the kernel function number from the multiple different executable files based on the kernel function number and the above-mentioned mapping relationship information.

[0121] Among them, the operation of running the executable file corresponding to the operator node can be specifically executed by the executor in the acceleration processor of the terminal device. Exemplarily, in combination with Figure 4, as shown in Figure 8, if the computing node to be processed is the first computing node op N1, the executor can determine that the kernel function number corresponding to the first computing node op N1 includes TK1_1, for example, obtained by calculating according to the actual shape through a preset operator function (op tiling); then, according to the kernel function number TK1_1 and the mapping relationship information, the executable file kern1 corresponding to TK1_1 is obtained and executed from multiple different executable files (i.e., kern1, kern2, kern3, kern4, ...), and the corresponding execution result can be output; similarly, the executor can also obtain and execute the executable file kern2 corresponding to TK1_2, the executable file kern3 corresponding to TK1_3, ... in the above manner. The figure takes the example of obtaining and executing the executable file kern1 corresponding to TK1_1 for explanation.

[0122] Optionally, when the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network model can share the same second executable file. That is, different neural network models can also reuse the executable file of the same kernel function. Exemplarily, when the terminal device downloads and executes the model information of other neural network models, if the executable file corresponding to the other neural network model has been issued before, the terminal device may no longer execute the issuing operation corresponding to the executable file, but instead obtain the executable file that has been issued before and execute it.

[0123] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of model processing. It is understandable that the model processing device for performing the above processing includes a hardware structure and / or software module corresponding to the execution of each function in order to realize the above functions. Those skilled in the art should easily realize that, in combination with the equipment and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0124] The embodiment of the present application can divide the functional modules of the model processing device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. In actual implementation, there may be other division methods.

[0125] FIG9 shows a possible structural diagram of the model processing apparatus involved in the above embodiments, where the functional modules are divided according to their respective functions. The model processing apparatus can be a model processing device or a chip used in a model processing device. The model processing apparatus includes an acquisition unit 401 and a processing unit 402. Furthermore, the model processing apparatus may also include a sending unit 403.

[0126] In one possible embodiment, acquisition unit 401 is configured to support the model processing apparatus in executing S201 of the aforementioned method embodiment; processing unit 402 is configured to support the model processing apparatus in executing S202 of the aforementioned method embodiment; and sending unit 403 is configured to enable the model processing apparatus to execute S203 of the aforementioned method embodiment. Furthermore, processing unit 402 may also be configured to support the model processing apparatus in executing S204 of the aforementioned method embodiment and / or other technical processes described herein.

[0127] In another possible embodiment, the acquisition unit 401 is used to support the model processing device to execute S301 in the above method embodiment; the processing unit 402 is used to support the model processing device to execute S302 and S303 in the above method embodiment, and / or other technical processes described herein; the sending unit 403 is used to execute the step of returning the execution result of the neural network model in the above method embodiment by the model processing device.

[0128] It can be understood that all the contents in the above method embodiment can be correspondingly referred to in the embodiment of the model processing device, and the embodiment of the present application will not be repeated here.

[0129] Based on the hardware implementation, the processing unit 402 in this application can be a processor of the model processing device, the acquisition unit 401 can be a receiver of the model processing device, and the sending unit 403 can be a transmitter of the model processing device. The transmitter can usually be integrated with the receiver to be used as a transceiver. The specific transceiver can also be called a communication interface.

[0130] As shown in Figure 10, it is a structural diagram of another model processing device provided in an embodiment of the present application. The model processing device can be a model processing device, or a chip applied to a model processing device. The model processing device includes: a communication interface 413 and a processor 412. Among them, the processor 412 is used to control and manage the actions of the model processing device. For example, the processor 412 can be used to support the model processing device to execute at least one of steps S202, S204, S302 and S303 in the above-mentioned method embodiment, and / or other processes for the technology described herein; the communication interface 413 can be used to support the model processing device to communicate. Optionally, the model processing device may also include a memory 411, which is used to store program code and data of the model processing device.

[0131] The processor 412 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a processing chip, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. It may implement or execute various logic blocks, modules, and circuits described in conjunction with the disclosure of the embodiments of this application. The processor 412 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like. The communication interface 413 may be a transceiver, a transceiver circuit, or a transceiver interface. The memory 411 may be a volatile memory or a non-volatile memory.

[0132] For example, communication interface 413, processor 412, and memory 411 are interconnected via bus 414, which may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. Bus 414 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the figure shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0133] It can be understood that all the contents in the above method embodiment can be correspondingly referred to in the embodiment of the model processing device, and the embodiment of the present application will not be repeated here.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical functional division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another device, or ignoring or not implementing certain features.

[0135] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0137] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. The readable storage medium may include: a USB flash drive, a mobile hard drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc., which can store program code. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product.

[0138] In another embodiment of the present application, a readable storage medium is also provided, which stores a computer program or instructions. When the computer program or instructions are executed by a processor in a device, the device executes the steps of the compiler in the above method embodiment, or executes the steps of the compiler and packager in the above method embodiment.

[0139] In another embodiment of the present application, a readable storage medium is provided, in which a computer program or instruction is stored. When the computer program or instruction is executed by a processor in a device, the device executes the steps of the executor in the above method embodiment.

[0140] In another embodiment of the present application, a computer program product is further provided, which includes a computer program (also referred to as code, or instructions), which is stored in a computer-readable storage medium; at least one processor of the device can read the computer program from the computer-readable storage medium, and at least one processor executes the computer program so that the device performs the steps of the compiler in the above method embodiment, or performs the steps of the compiler and packager in the above method embodiment.

[0141] In another embodiment of the present application, a computer program product is also provided, which includes a computer program (also referred to as code, or instructions), which is stored in a computer-readable storage medium; at least one processor of the device can read the computer program from the computer-readable storage medium, and at least one processor executes the computer program so that the device performs the steps of the executor in the above method embodiment.

[0142] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A model processing method, characterized in that: The method comprises: Acquire first model information of a neural network model, wherein the first model information includes a plurality of operator nodes; Determine at least one executable file corresponding to each of the multiple operator nodes, each executable file in the at least one executable file includes a kernel function corresponding to the operator node; Send second model information of the neural network model, wherein the second model information includes multiple different executable files among the executable files corresponding to the multiple operator nodes.

2. The method according to claim 1, characterized in that The determining at least one executable file corresponding to each of the multiple operator nodes includes: For each operator node among the multiple operator nodes, at least one kernel function corresponding to the operator node is compiled respectively to obtain at least one executable file corresponding to the operator node.

3. The method according to claim 1, characterized in that The determining at least one executable file corresponding to each of the multiple operator nodes includes: For each of the multiple operator nodes, at least one executable file corresponding to each of the multiple operator nodes is obtained from an executable file library, and the executable file library includes an executable file corresponding to each kernel function of a preset multiple kernel functions.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Determine mapping relationship information, where the mapping relationship information is used to indicate a correspondence between the multiple operator nodes, the different executable files, and at least one kernel function corresponding to each operator node; Wherein, the second model information also includes the mapping relationship information.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: A plurality of different executable files among the executable files corresponding to the plurality of operator nodes are packaged to obtain second model information of the neural network model.

6. The method according to any one of claims 1 to 4, characterized in that: The plurality of computing nodes include a to-be-processed computing node, the to-be-processed computing node corresponds to a first executable file, and the sending of the second model information of the neural network model includes: When the first executable file has not been sent, the first executable file is sent.

7. The method according to any one of claims 1 to 6, characterized in that: When the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models share the same second executable file.

8. A model processing method, characterized in that: The method comprises: Acquire second model information of a neural network model, the neural network model corresponds to a plurality of computing nodes, the second model information includes a plurality of different executable files among the executable files corresponding to the plurality of operator nodes, each executable file includes a kernel function; Execute a plurality of different executable files in the second model information.

9. The method according to claim 8, characterized in that The second model information further includes mapping relationship information, where the mapping relationship information is used to indicate a corresponding relationship between the multiple operator nodes, the different executable files, and at least one kernel function corresponding to each operator node. The method further includes: Determining, according to parameter information of a to-be-processed computing node among the plurality of computing nodes, a first kernel function corresponding to the to-be-processed node; According to the first kernel function and the mapping relationship information, a first executable file is obtained from the multiple different executable files, where the first executable file includes a first kernel function.

10. The method according to claim 8, characterized in that The multiple computing nodes include a to-be-processed computing node, the to-be-processed computing node corresponds to a first kernel function, and the obtaining of second model information of the neural network model includes: A first executable file is received, wherein the first executable file includes the first kernel function.

11. The method according to any one of claims 8 to 10, characterized in that: When the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models share the same second executable file.

12. A model processing device, characterized in that: The device comprises: An acquisition unit, used to acquire first model information of a neural network model, wherein the first model information includes a plurality of operator nodes; A processing unit, configured to determine at least one executable file corresponding to each of the plurality of operator nodes, each executable file of the at least one executable file including a kernel function corresponding to the operator node; A sending unit is used to send second model information of the neural network model, wherein the second model information includes multiple different executable files among the executable files corresponding to the multiple operator nodes.

13. The device according to claim 12, characterized in that The processing unit is also used for: For each operator node among the multiple operator nodes, at least one kernel function corresponding to the operator node is compiled respectively to obtain at least one executable file corresponding to the operator node.

14. The device according to claim 12, characterized in that The processing unit is also used for: For each of the multiple operator nodes, at least one executable file corresponding to each of the multiple operator nodes is obtained from an executable file library, and the executable file library includes an executable file corresponding to each kernel function of a preset multiple kernel functions.

15. The device according to any one of claims 12 to 14, characterized in that: The processing unit is also used for: Determine mapping relationship information, where the mapping relationship information is used to indicate a correspondence between the multiple operator nodes, the different executable files, and at least one kernel function corresponding to each operator node; Wherein, the second model information also includes the mapping relationship information.

16. The device according to any one of claims 12 to 15, characterized in that: The processing unit is also used for: A plurality of different executable files among the executable files corresponding to the plurality of operator nodes are packaged to obtain second model information of the neural network model.

17. The device according to any one of claims 12 to 15, characterized in that: The plurality of computing nodes include a to-be-processed computing node, and the to-be-processed computing node corresponds to a first executable file; The sending unit is further configured to send the first executable file when the first executable file has not been sent.

18. The device according to any one of claims 12 to 17, characterized in that: When the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models share the same second executable file.

19. A model processing device, characterized in that: The device comprises: A receiving unit, configured to obtain second model information of a neural network model, wherein the neural network model corresponds to a plurality of computing nodes, and the second model information includes a plurality of different executable files among the executable files corresponding to the plurality of operator nodes, and each executable file includes a kernel function; A processing unit is used to execute multiple different executable files in the second model information.

20. The device according to claim 19, characterized in that The second model information further includes mapping relationship information, where the mapping relationship information is used to indicate a corresponding relationship between the multiple operator nodes, the different executable files, and at least one kernel function corresponding to each operator node; The processing unit is further configured to determine a first kernel function corresponding to a to-be-processed computing node among the plurality of computing nodes according to parameter information of the to-be-processed computing node; The processing unit is further configured to obtain a first executable file from the multiple different executable files according to the first kernel function and the mapping relationship information, where the first executable file includes a first kernel function.

21. The device according to claim 19, characterized in that The multiple computing nodes include a to-be-processed computing node, and the to-be-processed computing node corresponds to a first kernel function; The receiving unit is further configured to receive a first executable file, where the first executable file includes the first kernel function.

22. The device according to any one of claims 19 to 21, characterized in that When the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models share the same second executable file.

23. A model processing device, characterized in that: The model processing device includes a processor and a memory, wherein instructions are stored in the memory. When the processor executes the instructions, the model processing device executes the model processing method according to any one of claims 1 to 7.

24. A model processing device, characterized in that: The model processing device includes a processor and a memory, wherein instructions are stored in the memory. When the processor executes the instructions, the model processing device executes the model processing method according to any one of claims 8 to 11.

25. A terminal device, characterized in that: The terminal device comprises: a model processing device as described in any one of claims 12-18 or claim 23, and a model processing device as described in any one of claims 19-22 or claim 24.

26. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed by a device, the device executes the model processing method according to any one of claims 1 to 11.

Images