Model processing method and device and terminal equipment

Abstract

The invention provides a model processing method and device and terminal equipment, relates to the technical field of artificial intelligence, and is used for reducing the size of a neural network model. The method comprises the following steps: acquiring first model information of a neural network model, wherein the first model information comprises a plurality of operator nodes; determining at least one executable file corresponding to each operator node in the plurality of operator nodes, wherein each executable file in the at least one executable file comprises a kernel function corresponding to the operator node; and sending second model information of the neural network model, wherein the second model information comprises a plurality of different executable files in the executable files corresponding to the plurality of operator nodes.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model processing method, apparatus and terminal device. Background Art

[0002] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and enable machines to have the functions of perception, reasoning and decision-making.

[0003] With the development of artificial intelligence technology, neural network models as AI models have become more and more complex, and the size of neural network models has also become larger and larger. The size of some neural network models is even as large as gigabytes (GB) or more. Therefore, when a terminal device compiles and executes a neural network model deployed on the terminal device, there will be problems such as high power consumption and high memory usage. Summary of the invention

[0004] The present application provides a model processing method, apparatus and terminal device for reducing the size of a neural network model, thereby reducing the power consumption and memory occupied by the neural network model during compilation and execution.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, a model processing method is provided, the method comprising: obtaining first model information of a neural network model, the first model information comprising multiple operator nodes, for example, the neural network model may be a tf model, a pytorch model, a caffe model or a davinci model, etc.; determining at least one executable file corresponding to each of the multiple operator nodes, the at least one executable file corresponding to each operator node may mean that the operator node may obtain one or more executable files after compilation, each executable file in the at least one executable file comprises a kernel function corresponding to the operator node, exemplarily, the extension of the executable file may be ".o"; sending second model information of the neural network model, the second model information comprising multiple different executable files in the executable files corresponding to the multiple operator nodes.

[0007] In the above technical scheme, the first model information of the neural network model obtained includes multiple operator nodes. When determining the executable files corresponding to the multiple operator nodes, the executable files are obtained based on the kernel function level, that is, each executable file includes a kernel function corresponding to the operator node, thereby realizing the reuse of executable files at the kernel function level, thereby reducing the repeated compilation of kernel functions and reducing the compilation time; at this time, the second model information sent includes multiple different executable files among the executable files corresponding to the multiple operator nodes, thereby avoiding the repeated sending of the same executable file, and further reducing the memory occupied when the neural network model is executed.

[0008] In a possible implementation of the first aspect, determining at least one executable file corresponding to each of the multiple operator nodes includes: for each of the multiple operator nodes, compiling at least one kernel function corresponding to the operator node to obtain at least one executable file corresponding to the operator node. In the above possible implementation, when determining the executable files corresponding to the multiple operator nodes, the operator nodes are compiled based on the kernel function level, and each executable file obtained by compilation includes a kernel function corresponding to the operator node, thereby realizing the reuse of executable files at the kernel function level, thereby reducing repeated compilation of kernel functions and reducing compilation time.

[0009] In a possible implementation of the first aspect, determining at least one executable file corresponding to each of the multiple operator nodes includes: for each of the multiple operator nodes, obtaining at least one executable file corresponding to each of the multiple operator nodes from an executable file library, wherein the executable file library includes executable files corresponding to each of the preset multiple kernel functions. In the above possible implementation, when determining the executable files corresponding to the multiple operator nodes, at least one executable file corresponding to each of the multiple operator nodes can be obtained from the executable file library, and the obtained executable file can realize the reuse of executable files at the kernel function level, thereby reducing the compilation time of the operator nodes.

[0010] In a possible implementation of the first aspect, the method further includes: determining mapping relationship information, the mapping relationship information being used to indicate a correspondence between the multiple operator nodes, the different executable files, and at least one kernel function corresponding to each operator node; wherein the second model information also includes the mapping relationship information. In the above possible implementation, by determining the mapping relationship information, different operator nodes can reuse the same executable file, thereby reducing the size of the compiled second model information.

[0011] In a possible implementation of the first aspect, the method further includes: packaging multiple different executable files in the executable files corresponding to the multiple operator nodes to obtain second model information of the neural network model. In the above possible implementation, by packaging multiple different executable files in the executable files corresponding to the multiple operator nodes, repeated packaging can be avoided and the size of the packaged model can be reduced.

[0012] In a possible implementation of the first aspect, the multiple computing nodes include a computing node to be processed, the computing node to be processed corresponds to a first executable file, and the sending of the second model information of the neural network model includes: when the first executable file has not been sent, sending the first executable file. In the above possible implementation, when sending the executable files corresponding to the multiple operator nodes, repeated sending can be avoided, the time consumption of sending can be reduced, and the memory usage can be reduced.

[0013] In a possible implementation of the first aspect, when the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network model share the same second executable file. In the above possible implementations, different neural network models can also reuse the executable file of the same kernel function, thereby further reducing the size of the neural network model and reducing memory usage.

[0014] In a second aspect, a model processing method is provided, the method comprising: obtaining second model information of a neural network model, the neural network model corresponding to multiple computing nodes, the second model information comprising multiple different executable files in the executable files corresponding to the multiple operator nodes, each executable file comprising a kernel function; executing the multiple different executable files in the second model information.

[0015] In the above technical solution, the second model information obtained includes multiple different executable files among the executable files corresponding to the multiple operator nodes, and each executable file includes a kernel function, thereby avoiding repeated sending of the same executable file, and when executing the second model information, the executable file at the kernel function level can be reused, thereby reducing the size of the network model and reducing memory usage.

[0016] In a possible implementation of the second aspect, the second model information also includes mapping relationship information, which is used to indicate the correspondence between the multiple operator nodes, the different executable files, and at least one kernel function corresponding to each operator node, and the method also includes: determining the first kernel function corresponding to the to-be-processed computing node according to the parameter information of the to-be-processed computing node among the multiple computing nodes; obtaining the first executable file from the multiple different executable files according to the first kernel function and the mapping relationship information, the first executable file including the first kernel function. In the above possible implementation, by determining the mapping relationship information, different operator nodes can reuse the same executable file, thereby reducing the memory occupied when the model is executed.

[0017] In a possible implementation of the second aspect, the multiple computing nodes include a computing node to be processed, the computing node to be processed corresponds to a first kernel function, and the second model information of the neural network model is obtained, including: receiving a first executable file, the first executable file including the first kernel function. In the above possible implementation, repeated distribution of the same executable file can be avoided, thereby reducing the time consumption of distribution and reducing memory usage.

[0018] In a possible implementation of the second aspect, when the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network model share the same second executable file. In the above possible implementations, different neural network models can also reuse the executable file of the same kernel function, thereby further reducing the size of the neural network model and reducing memory usage.

[0019] In a third aspect, a model processing device is provided, which includes: an acquisition unit, used to acquire first model information of a neural network model, the first model information including multiple operator nodes; a processing unit, used to determine at least one executable file corresponding to each of the multiple operator nodes, each executable file in the at least one executable file including a kernel function corresponding to the operator node; and a sending unit, used to send second model information of the neural network model, the second model information including multiple different executable files in the executable files corresponding to the multiple operator nodes.

[0020] In a possible implementation manner of the third aspect, the processing unit is further used to: for each operator node among the multiple operator nodes, compile at least one kernel function corresponding to the operator node to obtain at least one executable file corresponding to the operator node.

[0021] In a possible implementation of the third aspect, the processing unit is also used to: for each operator node among the multiple operator nodes, obtain at least one executable file corresponding to each operator node among the multiple operator nodes from an executable file library, and the executable file library includes an executable file corresponding to each kernel function among a preset multiple kernel functions.

[0022] In a possible implementation of the third aspect, the processing unit is also used to: determine mapping relationship information, which is used to indicate the correspondence between the multiple operator nodes, the different executable files and at least one kernel function corresponding to each operator node; wherein the second model information also includes the mapping relationship information.

[0023] In a possible implementation of the third aspect, the processing unit is further used to: package multiple different executable files among the executable files corresponding to the multiple operator nodes to obtain second model information of the neural network model.

[0024] In a possible implementation of the third aspect, the multiple computing nodes include a to-be-processed computing node, which corresponds to a first executable file; and the sending unit is further used to send the first executable file when the first executable file has not been sent.

[0025] In a possible implementation of the third aspect, when the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models share the same second executable file.

[0026] In a fourth aspect, a model processing device is provided, which includes: a receiving unit, used to obtain second model information of a neural network model, the neural network model corresponds to multiple computing nodes, the second model information includes multiple different executable files in the executable files corresponding to the multiple operator nodes, and each executable file includes a kernel function; a processing unit, used to execute the multiple different executable files in the second model information.

[0027] In a possible implementation of the fourth aspect, the second model information also includes mapping relationship information, which is used to indicate the correspondence between the multiple operator nodes, the different executable files and at least one kernel function corresponding to each operator node; the processing unit is also used to determine the first kernel function corresponding to the node to be processed according to the parameter information of the node to be processed among the multiple computing nodes; the processing unit is also used to obtain a first executable file from the multiple different executable files according to the first kernel function and the mapping relationship information, and the first executable file includes the first kernel function.

[0028] In a possible implementation of the fourth aspect, the multiple computing nodes include a computing node to be processed, and the computing node to be processed corresponds to a first kernel function; the receiving unit is also used to receive a first executable file, and the first executable file includes the first kernel function.

[0029] In a possible implementation of the fourth aspect, when the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models share the same second executable file.

[0030] In a fifth aspect, a model processing device is provided, which includes a processor and a memory, wherein instructions are stored in the memory. When the processor executes the instructions, the model processing device executes the model processing method provided in the first aspect or any possible implementation of the first aspect.

[0031] In a sixth aspect, a model processing device is provided, which includes a processor and a memory, wherein instructions are stored in the memory. When the processor executes the instructions, the model processing device executes the model processing method provided in the second aspect or any possible implementation of the second aspect.

[0032] In another aspect of the present application, a terminal device is provided, which includes: the third aspect or any possible implementation of the third aspect, or a model processing device as provided in the fifth aspect, and the fourth aspect or any possible implementation of the fourth aspect, or a model processing device as provided in the sixth aspect.

[0033] In another aspect of the present application, a computer-readable storage medium is provided, in which a computer program or instruction is stored. When the computer program or instruction is executed, the model processing method provided by the first aspect or any possible implementation of the first aspect is implemented.

[0034] In another aspect of the present application, a computer-readable storage medium is provided, in which a computer program or instruction is stored. When the computer program or instruction is executed, the model processing method provided by the second aspect or any possible implementation of the second aspect is implemented.

[0035] In another aspect of the present application, a computer program product is provided, which includes: a computer program (also referred to as code, or instructions), which, when executed, enables a computer to execute a model processing method as provided in the first aspect or any possible implementation of the first aspect.

[0036] In another aspect of the present application, a computer program product is provided, which includes: a computer program (also referred to as code, or instructions), which, when executed, enables a computer to execute a model processing method provided in the second aspect or any possible implementation of the second aspect.

[0037] It can be understood that the beneficial effects that can be achieved by any of the model processing devices, terminal devices, computer-readable storage media and computer program products provided above can correspond to the beneficial effects of the model processing methods provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the structure of a model processing device provided in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0040] Figure 3 A flow chart of a model processing method provided in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of a model compilation provided in an embodiment of the present application;

[0042] Figure 5 A flowchart of another model processing method provided in an embodiment of the present application;

[0043] Figure 6 A schematic diagram of a model packaging provided in an embodiment of the present application;

[0044] Figure 7 A flowchart of another model processing method provided in an embodiment of the present application;

[0045] Figure 8 A schematic diagram of a model execution provided in an embodiment of the present application;

[0046] Fig. 9 A schematic diagram of the structure of a model processing device provided in an embodiment of the present application;

[0047] Fig.10 A schematic diagram of the structure of another model processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will discuss the making and use of various embodiments in detail. However, it should be understood that many applicable inventive concepts provided by this application can be implemented in a variety of specific environments. The specific embodiments discussed are only illustrative of specific ways to implement and use this description and this technology, and do not limit the scope of this application.

[0049] Unless defined otherwise, all technical and scientific terms used herein have the same meanings as commonly understood by one of ordinary skill in the art.

[0050] Various circuits or other components may be described or referred to as being "configured to" perform one or more tasks. In this case, "configured to" is used to imply structure by indicating that the circuit / component includes structure (e.g., circuitry) that performs the one or more tasks during operation. Thus, even when the specified circuit / component is not currently operational (e.g., not turned on), the circuit / component may be referred to as being configured to perform the task. Circuits / components used with the phrase "configured to" include hardware, such as circuits that perform an operation, etc.

[0051] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the present application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, a, b and c; where a, b and c can be single or multiple.

[0052] The embodiments of the present application use words such as "first" and "second" to distinguish objects with similar names, functions or effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order. The term "coupled" is used to indicate electrical connection, including direct connection through wires or connection terminals or indirect connection through other devices. Therefore, "coupled" should be regarded as a broad electronic communication connection.

[0053] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0054] Before introducing the embodiments of the present application, the relevant background technology involved in the present application is first introduced and explained.

[0055] A neural network (NN) is a network structure that processes information by imitating the behavioral characteristics of animal neural networks. The structure of a neural network is composed of a large number of nodes (also called neurons) connected to each other. It processes information by learning and training the input information based on a specific operation model. A neural network includes an input layer, a hidden layer, and an output layer. The input layer is responsible for receiving input signals, the output layer is responsible for outputting the calculation results of the neural network, and the hidden layer is responsible for the calculation process such as learning and training. It is the memory unit of the network. The memory function of the hidden layer is represented by a weight matrix. Usually, each neuron corresponds to a weight (also called a weight coefficient).

[0056] A neural network model may refer to a network model obtained by model training based on a neural network. A neural network model may generally include the following information: network structure description information, weight information, and multiple operators (also referred to as operator nodes). The network structure description information may be used to describe the network structure of the neural network model, the weight information may include the weights in the neural network model, and the multiple operators may perform calculations and processing according to the network structure based on the weight information.

[0057] An operator can be a node with data transmission and processing capabilities. Each operator can have the following parameter information: operator type, data type, input data size (shape), data arrangement, etc. The type of the input data size can include static and dynamic. Among them, when an operator does not involve multiple branches (for example, the type of the input data size is static), the operator can correspond to a kernel function; when an operator has multiple branches (for example, the type of the input data size is dynamic), the operator can correspond to multiple kernel functions.

[0058] A kernel function may refer to the compiled code of an operator function that can be directly executed on hardware. A function may correspond to one or a class of execution strategies.

[0059] An executable file may refer to a compiled binary file of an operator that can be directly executed on hardware. An executable file may include one or more kernel functions, and an executable file in an embodiment of the present application includes a kernel function. In an example, the extension of the executable file may be ".o".

[0060] The kernel function number may refer to a number defined for quickly finding a kernel function, and the number may be used to index the kernel function. Kernel function numbers in the same executable file are not repeated.

[0061] In recent years, artificial intelligence (AI) technology has developed rapidly and has been widely used in various fields. AI is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and enable machines to have the functions of perception, reasoning and decision-making.

[0062] With the development of artificial intelligence technology, neural network models as AI models have become more and more complex, and the size of neural network models has also become larger and larger. The size of some neural network models is even as large as gigabytes (GB) or more. Therefore, when a terminal device compiles and executes a neural network model deployed on the terminal device, there will be problems such as high power consumption and high memory usage.

[0063] In the related art, when compiling a neural network model (for example, a tf model, a pytorch model, a caffe model, and a davinci model, etc.), the kernel functions corresponding to the neural network model are compiled together and packaged into a model or application, that is, each operator in the neural network model corresponds to an executable file after compilation, and the packaged model includes the executable files compiled after all operators of the neural network model, and the executable file usually includes multiple kernel functions. When the model is sent down for execution, the entire binary file (for example, a .o file) compiled by the neural network model is sent down. In addition, for the use of the acceleration library of the provided neural network model (for example, cuDNN provided by NV), the acceleration library is also packaged into the model when the model is compiled. The above solution still has problems such as high power consumption and high memory usage when the neural network model is compiled and executed.

[0064] Based on this, the embodiment of the present application provides a model processing method, which can be used to: realize the reuse of kernel function level when compiling the model, thereby reducing the repeated compilation of kernel functions and reducing the compilation time; avoid repeated packaging when packaging the model, and reduce the size of the packaged model; realize the distribution of kernel function level when the model is distributed for execution, avoid repeated distribution, reduce the distribution time, and reduce memory usage. Therefore, the embodiment of the present application can solve the above-mentioned problems of high power consumption and high memory usage when compiling and executing the neural network model.

[0065] The technical solution provided in the embodiment of the present application can be applied to model processing equipment, such as Figure 1As shown, the model processing device may include a compiler, a packager and an executor. Among them, the compiler can be used to compile multiple operator nodes in the neural network model to obtain an executable file corresponding to each operator node. Each executable file may include a kernel function of the operator node. The packager can be used to package the executable files corresponding to the multiple operator nodes, for example, to perform repetitive packaging on the executable files corresponding to the multiple operator nodes. The executor can be used to receive and execute the executable file issued by the packager, and can also be used to receive the network structure description information and weight information of the neural network model, etc. The network structure description information and the weight information can also be packaged and issued together with the above-mentioned executable file.

[0066] Optionally, the model processing device may not include the packager. After compiling the executable file corresponding to each operator node, the compiler may also send the executable file to the executor by deduplicating the executable file, and the executor may receive and execute the executable file sent by the compiler.

[0067] In a possible embodiment, the model processing device may be a terminal device, which may include but is not limited to: a mobile phone, a tablet computer, a laptop computer, a PDA, an ultra-mobile personal computer (umPC), a mobile internet device (MID), a netbook, a camera, a camera, a wearable device (such as a smart watch and a smart bracelet, etc.), an in-vehicle device (such as a car, a bicycle, an electric car, an airplane, a ship, a train, a high-speed rail, etc.), a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a smart home device (such as a refrigerator, a television, an air conditioner, an electric meter, etc.), an intelligent robot, a workshop device, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, or a smart home. wireless terminals in the home), flying equipment (e.g., smart robots, hot air balloons, drones, airplanes), etc.

[0068] The following takes a mobile phone as an example to introduce the specific structure of the terminal device.

[0069] Figure 2A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. The terminal device may include: a radio frequency (RF) circuit 110, a memory 120, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a processor 170, and a power supply 180. The following is a detailed introduction to the various components of the terminal device:

[0070] The RF circuit 110 can be used to send and receive information, or receive or send signals during a call. In particular, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the uplink data is sent to the base station. Generally, the RF circuit 110 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 110 can also communicate with the network and other devices through wireless communication.

[0071] The memory 120 can be used to store data, software programs and modules; it mainly includes a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function, such as a sound playback function, an image playback function, etc.; the data storage area can store data created according to the use of the terminal device, such as audio data, image data, phone book, etc. In addition, the terminal device may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0072] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the terminal device. Specifically, the input unit 130 may include a touch screen 131 and other input devices 132. The touch screen 131 can collect user touch operations on or near it (such as operations performed by the user using any suitable object or accessory such as a finger, stylus, etc. on or near the touch screen), and drive the corresponding connection device according to a pre-set program. Optionally, other input devices 132 may include but are not limited to one or more of a physical keyboard, function keys (such as a volume control key, a power switch key, etc.), a trackball, a mouse, a joystick, etc.

[0073] The display unit 140 can be used to display information input by the user or information provided to the user and various menus of the terminal device. In an example, the display unit 140 may include a display screen 141, and the display screen 141 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch screen 131 may cover the display screen 141, and when the touch screen 131 detects a touch operation on or near it, it is transmitted to the processor 170 to determine the type of the touch event, and then the processor 170 provides a corresponding visual output on the display screen 141 according to the type of the touch event. Although in the figure, the touch screen 131 and the display screen 141 are used as two independent components to implement the input and output functions of the terminal device, in some embodiments, the touch screen 131 and the display screen 141 can be integrated to implement the input and output functions of the terminal device.

[0074] The sensor 150 may include one or more sensors for providing status assessment of various aspects for the terminal device. The sensor 150 may include a light sensor, which may be used in imaging applications, i.e., become a component of a camera or a camera head. In addition, the sensor 150 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor, through which the acceleration / deceleration, orientation, open / closed state, relative positioning of components, or temperature changes of the terminal device, etc., may be detected.

[0075] The audio circuit 160, the speaker and the microphone can provide an audio interface between the user and the terminal device. The audio circuit 160 can transmit the received audio data to the speaker after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 160 and converted into audio data, and then the audio data is output to the RF circuit 110 to be sent to, for example, another mobile phone, or the audio data is output to the memory 120 for further processing.

[0076] The processor 170 is the control center of the terminal device, which uses various interfaces and lines to connect the various parts of the entire terminal device, and executes various functions and processes data of the terminal device by running or executing the software programs and / or modules stored in the memory 120, and calling the data stored in the memory 120, so as to monitor the terminal device as a whole. Optionally, the processor 170 may include one or more processing units, which may include but are not limited to: a central processing unit (CPU), an application processor, a digital signal processor (DSP), a network processor (NPU), a graphics processing unit (GPU), an image signal processor (ISP), a microcontroller or a microprocessor, etc. Further, the processor 170 may also include other hardware circuits or accelerators, such as an application specific integrated circuit (ASIC), a complex programmable logic device (CPLD) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. Optionally, the processor 170 may also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, etc.

[0077] In an embodiment of the present application, the processor 170 may include a general-purpose processor and an accelerated processor. The general-purpose processor may have general data processing, execution of instructions or commands, and task delivery capabilities, and the accelerated processor may have data acceleration processing capabilities. In a possible example, the general-purpose processor may include a CPU, a DSP, an application processor, a microcontroller or a microprocessor, etc., and the general-purpose processors listed above may include a compiler and a packager; in a possible example, the accelerated processor may include an NPU, a GPU, a CPU, an ASIC or a CPLD, etc., and the accelerated processors listed above may include an executor.

[0078] The terminal device may also include a power supply 180 (e.g., a battery) for supplying power to each component. The power supply 180 may be logically connected to the processor 170 through a power management system, so that functions such as charging, discharging, and power consumption management can be implemented through the power management system. In an embodiment of the present application, the power supply 180 may include multiple batteries, and the power management system may support both fast charging technology and non-fast charging technology, so that the power management system may charge the multiple batteries through fast charging technology, or may charge the multiple batteries through non-fast charging technology.

[0079] Although not shown, the terminal device may also include a wireless fidelity (WiFi) module, a Bluetooth module, etc., which will not be described in detail in the embodiments of the present application. It can be understood by those skilled in the art that the terminal device structure shown in the figure does not constitute a limitation on the terminal device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0080] Figure 3 A flow chart of a model processing method provided in an embodiment of the present application is provided. The method can be applied to the model processing device provided above. The model processing device is taken as an example of a terminal device for explanation below. The method includes the following steps.

[0081] S201: Obtain first model information of a neural network model, where the first model information includes multiple operator nodes.

[0082] The neural network model may be any network model obtained based on neural network training. For example, the neural network model may be a TF model, a PyTorch model, a Caffe model, or a DaVinci model. The neural network model may be stored in the terminal device. For example, after the neural network model is obtained through training, the neural network model may be stored in the memory of the terminal device to deploy the neural network model in the terminal device.

[0083] In addition, the multiple operator nodes can be connected according to a certain network structure. Each of the multiple operator nodes can have the following parameter information: operator type, data type, input data size (shape), etc. Among them, the type of the input data size can include static and dynamic. When an operator does not involve multiple branches (for example, the type of the input data size is static), the operator can correspond to a kernel function; when an operator has multiple branches (for example, the type of the input data size is dynamic), the operator can correspond to multiple kernel functions.

[0084] Optionally, the first model information may also include network structure description information and / or weight information, where the network structure description information may be used to indicate the network structure of the neural network model, and the weight information may include the weight of the neural network model.

[0085] In a possible embodiment, the terminal device may include a general-purpose processor and a memory, in which first model information of the neural network model is stored. When the general-purpose processor needs to use the neural network model, the general-purpose processor can obtain the first model information from the memory.

[0086] S202: Determine at least one executable file corresponding to each operator node among the multiple operator nodes, each executable file among the at least one executable file includes a kernel function corresponding to the operator node.

[0087] Each operator node corresponds to at least one executable file, which may mean that the operator node can obtain one or more executable files after compilation. The executable file may refer to a binary file that can be directly executed after compilation, so the executable file may also be called a binary file. Exemplarily, the extension of the executable file may be ".o".

[0088] In the embodiment of the present application, an executable file includes only one kernel function, that is, one executable file corresponds to one kernel function. When an operator node corresponds to one kernel function, the operator node can obtain one executable file after compilation; when an operator node corresponds to multiple kernel functions, the operator node can obtain multiple executable files after compilation, and the multiple kernel functions correspond to the multiple executable files one by one.

[0089] In a possible embodiment, determining at least one executable file corresponding to each of the multiple operator nodes may include: for each of the multiple operator nodes, compiling at least one kernel function corresponding to the operator node to obtain at least one executable file corresponding to the operator node. That is, when compiling each operator node, compiling is performed according to the principle that one executable file only includes one kernel function to obtain at least one executable file corresponding to each operator node.

[0090] The operation of compiling the operator node can be specifically performed by a compiler in the general processor of the terminal device. The compiler can compile different executable files according to different operators and parameter information. If an operator node has multiple branches, multiple executable files can be obtained by compiling, and each executable file includes a kernel function; if an operator node does not involve multiple branches, one executable file can be obtained by compiling.

[0091] For example, Figure 4 As shown, if the multiple computing nodes include a first computing node op N1 and a second computing node op N2, and the first computing node op N1 and the second computing node op N2 both correspond to multiple kernel functions, then the multiple executable files obtained after compilation of the first computing node op N1 may include kern1 (TK1_1), kern2 (TK1_2), kern3 (TK1_3), ..., and the multiple executable files obtained after compilation of the second computing node op N2 may include kern2 (TK2_1), kern3 (TK2_2), kern4 (TK2_3), ...

[0092] Above Figure 4 In the example, the first computing node op N1 and the second computing node op N2 both correspond to multiple executable files. The above TK1_1 to TK1_3, and TK2_1 to TK2_3 represent the kernel function numbers included in the executable files. The above kern1 to kern4 can be used to represent different executable files, specifically hash values. The hash value of a certain executable file can be a hash value of information such as function parameters and function content in addition to the function name, and the hash value can be used for repeatability checking.

[0093] It can be understood that the embodiments of the present application use the hash value of the executable file to perform a duplication check for illustration. In actual applications, when an executable file is functionally or logically the same as other executable files, they can also be reused, and the keywords used for the duplication check can also be customized. The embodiments of the present application do not impose any specific restrictions on this.

[0094] In another possible embodiment, determining at least one executable file corresponding to each of the multiple operator nodes may include: for each of the multiple operator nodes, obtaining at least one executable file corresponding to each of the multiple operator nodes from an executable file library, wherein the executable file library includes an executable file corresponding to each of the preset multiple kernel functions. That is, when compiling each operator node, the corresponding executable file can be directly searched from the executable file library without performing a corresponding compilation operation.

[0095] It is understandable that the executable files in the executable file library may be pre-compiled and pre-configured in the terminal device. The executable file library may include a large number of executable files, and each executable file includes only one kernel function.

[0096] S203: Send second model information of the neural network model, where the second model information includes multiple different executable files among the executable files corresponding to the multiple operator nodes.

[0097] Among them, sending the second model information of the neural network model can also be called sending the second model information of the neural network model. In a possible example, the terminal device may include a general processor and an acceleration processor, and the general processor may send the second model information of the neural network model to the acceleration processor to send the second model information to the acceleration processor.

[0098] Optionally, the second model information may also include network structure description information and / or weight information of the neural network model. In a possible example, the general processor of the terminal device may obtain the network structure description information and the weight information from the first model information, and carry the network structure description information, the weight information, and the above-mentioned multiple different executable files in the second model information to the acceleration processor.

[0099] In a possible embodiment, Figure 5 As shown, before the general processor of the terminal device sends the second model information of the neural network model, the method may also include: S204. Packing multiple different executable files in the executable files corresponding to the multiple operator nodes to obtain the second model information of the neural network model. That is, the general processor of the terminal device can pack multiple different executable files together by de-packaging, so that they can be sent to the acceleration processor at one time through the second model information.

[0100] The operation of de-duplicating and packaging the executable files corresponding to the multiple operator nodes may be specifically performed by a packager in a general processor of the terminal device. Figure 4 The compilation results shown are as follows Figure 6 As shown, the packager performs de-duplication and packaging on the executable files corresponding to the first computing node op N1 and the second computing node op N2, and the obtained multiple different executable files in the second model information may include kern1, kern2, kern3, kern4, ... Among them, kern2 (TK1_2) and kern3 (TK1_3) in the executable file corresponding to the first computing node op N1 are respectively the same as kern2 (TK2_1) and kern3 (TK2_2) in the executable file corresponding to the second computing node op N2, so that the second model information after de-duplication and packaging includes only one kern2 and one kern3.

[0101] In another possible embodiment, the general processor of the terminal device can send the executable file corresponding to each of the multiple computing nodes to the acceleration processor in turn according to the execution order of the multiple computing nodes in the neural network model. Taking a certain computing node to be processed among the multiple computing nodes as an example, if the computing node to be processed corresponds to the first executable file, sending the second model information of the neural network model may include: sending the first executable file when the first executable file has not been sent. If the first executable file has been sent before, the first executable file is no longer sent.

[0102] Exemplarily, for any node to be processed, determining the executable file corresponding to the node to be processed and sending the executable file may include: determining the kernel function corresponding to the computing node to be processed based on parameter information of the computing node to be processed (for example, the type of input data size); if the node to be processed corresponds to a first kernel function, determining a first executable file based on the first kernel function, the first executable file including the first kernel function; when the first executable file has not been sent, sending the first executable file.

[0103] Furthermore, the method may also include: determining mapping relationship information, the mapping relationship information being used to indicate the corresponding relationship between the multiple operator nodes, the multiple different executable files, and at least one kernel function corresponding to each operator node. The second model information also includes the mapping relationship information.

[0104] In a possible embodiment, after determining at least one executable file corresponding to each of the multiple nodes, the mapping relationship information may also be determined, that is, a corresponding relationship between the multiple computing nodes, the multiple different executable files, and the at least one kernel function corresponding to each operator node is established. Figure 4 Taking the compilation result shown as an example, the mapping relationship information may be [op N1 (TK1_1, TK1_2, TK1_3, ...), op N2 (TK2_1, TK2_2, TK2_3, ...)] --> [kern1 (TK1_1), kern2 (TK1_2, TK2_1), kern3 (TK1_3, TK2_2), kern4 (TK2_3), ...].

[0105] Optionally, when the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models can share the same second executable file. That is, different neural network models can also reuse the executable file of the same kernel function. Exemplarily, when the terminal device is compiling, packaging and issuing model information of other neural network models, if the executable files corresponding to other neural network models have been compiled or issued before, the terminal device may no longer perform the compilation, packaging and issuing operations corresponding to the executable file.

[0106] Furthermore, in actual applications, the terminal device can also directly obtain the second model information in the above S201, that is, the compiled model information is configured for the terminal device at the beginning. Specifically, after the training of the neural network is completed, the first model information of the neural network can be compiled, and the executable files of the multiple computing nodes obtained after compilation can be de-duplicated and packaged, so as to configure the packaged second model information in the terminal device. Exemplarily, the binary file package of the neural network model is published offline, and when the binary file package is published, the operator compiler is called by the publishing script to compile in a similar manner as described above, an executable file is formed based on the kernel function level, and the above mapping relationship information is recorded.

[0107] Further, such as Figure 7 As shown, the method may further include the following steps, which may be executed by an accelerated processor (eg, NPU or GPU) in the terminal device. Figure 7 The above-mentioned S201 to S204 are not shown in FIG.

[0108] S301: Obtain second model information of the neural network model, where the second model information includes multiple different executable files among executable files corresponding to multiple operator nodes, and each executable file includes a kernel function.

[0109] Among them, the neural network model can be a network model obtained based on neural network training, and the neural network model can include multiple operator nodes, and the multiple operator nodes can be connected according to a certain network structure. The second model information can be model information obtained by compiling the first model information of the neural network model. The relevant description of the second model information is consistent with the description of the second model information in S203 above, and specifically refer to the description in S203 above, and the embodiment of the present application will not be repeated here.

[0110] In addition, each of the multiple operator nodes may correspond to at least one executable file. If an operator node has multiple branches, the operator node may correspond to multiple executable files, each executable file including a kernel function; if an operator node does not involve multiple branches, the operator node may correspond to one executable file. The above multiple different executable files may be obtained by deduplicating the executable files corresponding to the multiple operator nodes.

[0111] In a possible embodiment, when the second model information is sent by the general processor of the terminal device at one time according to the execution order of the multiple computing nodes in the neural network model, the acceleration processor of the terminal device can receive an executable file corresponding to one computing node each time. Exemplarily, taking a certain computing node to be processed among the multiple computing nodes as an example, if the computing node to be processed corresponds to the first executable file, the first executable file can include the first kernel function corresponding to the computing node to be processed, then receiving the second model information of the neural network model can include: receiving the first executable file; wherein the first executable file is sent by the general processor when the first executable file has not been sent. If the first executable file has been sent before, the first executable file is no longer sent.

[0112] In another possible embodiment, when the second model information is a general processor of the terminal device, and the multiple different executable files are packaged and sent, the acceleration processor of the terminal device can receive the second model information at one time, that is, obtain the multiple different executable files through one reception.

[0113] Optionally, if the second model information also includes mapping relationship information, the mapping relationship information is used to indicate the corresponding relationship between the multiple operator nodes, the multiple different executable files, and at least one kernel function corresponding to each operator node.

[0114] S302: Execute multiple different executable files in the second model information.

[0115] In a possible embodiment, for the multiple computing nodes, the acceleration processor of the terminal device can execute the executable files corresponding to the multiple computing nodes in sequence according to the execution order of the multiple computing nodes in the neural network model and according to the second model information.

[0116] Furthermore, before executing the executable file corresponding to each computing node among the multiple computing nodes, the method may also include: S303. For each computing node among the multiple computing nodes, obtain the executable file corresponding to the computing node from the second model information.

[0117] Specifically, based on the parameter information of the computing node to be processed among the multiple computing nodes, at least one kernel function corresponding to the node to be processed is determined; based on the at least one kernel function and the mapping relationship information, at least one corresponding executable file is obtained from the multiple different executable files, and the at least one executable file corresponds one-to-one to the at least one kernel function. For example, each executable file includes a kernel function number among the at least one kernel function.

[0118] In a possible example, if the computing node to be processed has multiple branches (for example, the type of the size of the input data is dynamic), the acceleration processor of the terminal device can obtain multiple kernel function numbers corresponding to the computing node to be processed, and then obtain multiple executable files corresponding to the multiple kernel function numbers from the multiple different executable files based on the multiple kernel function numbers and the above-mentioned mapping relationship information. If the computing node to be processed does not involve multiple branches (for example, the type of the size of the input data is static), the acceleration processor of the terminal device can obtain a kernel function number corresponding to the computing node to be processed, and then obtain an executable file corresponding to the kernel function number from the multiple different executable files based on the kernel function number and the above-mentioned mapping relationship information.

[0119] The operation of running the executable file corresponding to the operator node may be specifically performed by an executor in the acceleration processor of the terminal device. Figure 4 ,like Figure 8 As shown, if the computing node to be processed is the first computing node op N1, the executor can determine that the kernel function number corresponding to the first computing node op N1 includes TK1_1, for example, it is obtained by calculating according to the actual shape through a preset operator function (op tiling); then, according to the kernel function number TK1_1 and the mapping relationship information, the executable file kern1 corresponding to TK1_1 is obtained and executed from multiple different executable files (i.e., kern1, kern2, kern3, kern4, ...), and the corresponding execution result can be output; similarly, the executor can also obtain and execute the executable file kern2 corresponding to TK1_2, the executable file kern3 corresponding to TK1_3, ... in the above manner. The figure takes the example of obtaining and executing the executable file kern1 corresponding to TK1_1 as an example for explanation.

[0120] Optionally, when the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models can share the same second executable file. That is, different neural network models can also reuse the executable file of the same kernel function. Exemplarily, when the terminal device downloads and executes the model information of other neural network models, if the executable file corresponding to the other neural network model has been issued before, the terminal device may no longer execute the issuing operation corresponding to the executable file, but obtain the executable file that has been issued before and execute it.

[0121] The above mainly introduces the scheme provided by the embodiment of the present application from the perspective of model processing. It is understandable that the model processing device for performing the above processing includes a hardware structure and / or software module corresponding to each function in order to realize the above functions. Those skilled in the art should easily realize that, in combination with the equipment and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0122] The embodiment of the present application can divide the functional modules of the model processing device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0123] In the case of dividing each functional module into corresponding functional modules, Fig. 9 A possible structural diagram of the model processing device involved in the above embodiment is shown, and the model processing device can be a model processing device, or a chip applied to the model processing device. The model processing device includes: an acquisition unit 401 and a processing unit 402. Further, the model processing device can also include a sending unit 403.

[0124] In a possible embodiment, the acquisition unit 401 is used to support the model processing device to perform S201 in the above method embodiment; the processing unit 402 is used to support the model processing device to perform S202 in the above method embodiment; the sending unit 403 is used to execute the model processing device to perform S203 in the above method embodiment. Further, the processing unit 402 can also be used to support the model processing device to perform S204 in the above method embodiment, and / or other technical processes described herein.

[0125] In another possible embodiment, the acquisition unit 401 is used to support the model processing device to execute S301 in the above method embodiment; the processing unit 402 is used to support the model processing device to execute S302 and S303 in the above method embodiment, and / or other technical processes described herein; the sending unit 403 is used to execute the step of returning the execution result of the neural network model in the above method embodiment by the model processing device.

[0126] It can be understood that all the contents in the above method embodiment can be correspondingly referred to in the model processing device embodiment, and the embodiment of the present application will not be repeated here.

[0127] Based on the hardware implementation, the processing unit 402 in the present application can be a processor of the model processing device, the acquisition unit 401 can be a receiver of the model processing device, and the sending unit 403 can be a transmitter of the model processing device. The transmitter can usually be integrated with the receiver to be used as a transceiver, and the specific transceiver can also be called a communication interface.

[0128] like Fig.10 As shown, it is a structural schematic diagram of another model processing device provided in an embodiment of the present application. The model processing device can be a model processing device, or a chip applied to a model processing device. The model processing device includes: a communication interface 413 and a processor 412. Among them, the processor 412 is used to control and manage the actions of the model processing device. For example, the processor 412 can be used to support the model processing device to execute at least one of steps S202, S204, S302 and S303 in the above-mentioned method embodiment, and / or other processes for the technology described herein; the communication interface 413 can be used to support the model processing device to communicate. Optionally, the model processing device may also include a memory 411, and the memory 411 is used to store the program code and data of the model processing device.

[0129] Among them, the processor 412 can be a central processing unit, a general processor, a digital signal processor, an application-specific integrated circuit, a processing chip, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute various logic boxes, modules and circuits described in conjunction with the disclosure of the embodiments of the present application. The processor 412 can also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like. The communication interface 413 can be a transceiver, a transceiver circuit or a transceiver interface, and the like. The memory 411 can be a volatile memory or a non-volatile memory, and the like.

[0130] For example, the communication interface 413, the processor 412, and the memory 411 are interconnected via a bus 414, and the bus 414 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 414 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0131] It can be understood that all the contents in the above method embodiment can be correspondingly referred to in the model processing device embodiment, and the embodiment of the present application will not be repeated here.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0134] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0135] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium, which can include: a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., which can store program codes. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product.

[0136] In another embodiment of the present application, a readable storage medium is also provided, in which a computer program or instruction is stored. When the computer program or instruction is executed by a processor in a device, the device executes the steps of the compiler in the above method embodiment, or executes the steps of the compiler and packager in the above method embodiment.

[0137] In another embodiment of the present application, a readable storage medium is provided, in which a computer program or instruction is stored. When the computer program or instruction is executed by a processor in a device, the device executes the steps of the executor in the above method embodiment.

[0138] In another embodiment of the present application, a computer program product is also provided, which includes a computer program (also referred to as code, or instructions), which is stored in a computer-readable storage medium; at least one processor of the device can read the computer program from the computer-readable storage medium, and at least one processor executes the computer program so that the device performs the steps of the compiler in the above method embodiment, or performs the steps of the compiler and packager in the above method embodiment.

[0139] In another embodiment of the present application, a computer program product is also provided, which includes a computer program (also referred to as code, or instructions), which is stored in a computer-readable storage medium; at least one processor of the device can read the computer program from the computer-readable storage medium, and at least one processor executes the computer program so that the device performs the steps of the executor in the above method embodiment.

[0140] Finally, it should be noted that the above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model processing method, characterized in that: The method comprises: Acquire first model information of a neural network model, wherein the first model information includes a plurality of operator nodes; Determine at least one executable file corresponding to each of the multiple operator nodes, each executable file in the at least one executable file includes a kernel function corresponding to the operator node; Send second model information of the neural network model, wherein the second model information includes multiple different executable files among the executable files corresponding to the multiple operator nodes.

2. The method according to claim 1, characterized in that The determining at least one executable file corresponding to each of the multiple operator nodes includes: For each operator node among the multiple operator nodes, at least one kernel function corresponding to the operator node is compiled respectively to obtain at least one executable file corresponding to the operator node.

3. The method according to claim 1, characterized in that: The determining at least one executable file corresponding to each of the multiple operator nodes includes: For each of the multiple operator nodes, at least one executable file corresponding to each of the multiple operator nodes is obtained from an executable file library, and the executable file library includes an executable file corresponding to each kernel function of a preset multiple kernel functions.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Determine mapping relationship information, where the mapping relationship information is used to indicate a correspondence between the multiple operator nodes, the different executable files, and at least one kernel function corresponding to each operator node; Wherein, the second model information also includes the mapping relationship information.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: A plurality of different executable files among the executable files corresponding to the plurality of operator nodes are packaged to obtain second model information of the neural network model.

6. The method according to any one of claims 1 to 4, characterized in that: The plurality of computing nodes include a to-be-processed computing node, the to-be-processed computing node corresponds to a first executable file, and the sending of the second model information of the neural network model includes: When the first executable file has not been sent, the first executable file is sent.

7. The method according to any one of claims 1 to 6, characterized in that: When the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models share the same second executable file.

8. A model processing method, characterized in that: The method comprises: Acquire second model information of a neural network model, the neural network model corresponds to a plurality of computing nodes, the second model information includes a plurality of different executable files among the executable files corresponding to the plurality of operator nodes, each executable file includes a kernel function; Execute a plurality of different executable files in the second model information.

9. The method according to claim 8, characterized in that The second model information further includes mapping relationship information, where the mapping relationship information is used to indicate a corresponding relationship between the multiple operator nodes, the different executable files, and at least one kernel function corresponding to each operator node. The method further includes: Determining, according to parameter information of a to-be-processed computing node among the plurality of computing nodes, a first kernel function corresponding to the to-be-processed node; According to the first kernel function and the mapping relationship information, a first executable file is obtained from the multiple different executable files, where the first executable file includes a first kernel function.

10. The method according to claim 8, characterized in that The multiple computing nodes include a to-be-processed computing node, the to-be-processed computing node corresponds to a first kernel function, and the obtaining of second model information of the neural network model includes: A first executable file is received, wherein the first executable file includes the first kernel function.

11. The method according to any one of claims 8 to 10, characterized in that: When the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models share the same second executable file.

12. A model processing device, characterized in that: The device comprises: An acquisition unit, configured to acquire first model information of a neural network model, wherein the first model information includes a plurality of operator nodes; A processing unit, configured to determine at least one executable file corresponding to each of the plurality of operator nodes, each executable file of the at least one executable file including a kernel function corresponding to the operator node; A sending unit is used to send second model information of the neural network model, wherein the second model information includes multiple different executable files among the executable files corresponding to the multiple operator nodes.

13. The device according to claim 12, characterized in that The processing unit is also used for: For each operator node among the multiple operator nodes, at least one kernel function corresponding to the operator node is compiled respectively to obtain at least one executable file corresponding to the operator node.

14. The device according to claim 12, characterized in that The processing unit is also used for: For each of the multiple operator nodes, at least one executable file corresponding to each of the multiple operator nodes is obtained from an executable file library, and the executable file library includes an executable file corresponding to each kernel function of a preset multiple kernel functions.

15. The device according to any one of claims 12 to 14, characterized in that: The processing unit is also used for: Determine mapping relationship information, where the mapping relationship information is used to indicate a correspondence between the multiple operator nodes, the different executable files, and at least one kernel function corresponding to each operator node; Wherein, the second model information also includes the mapping relationship information.

16. The device according to any one of claims 12 to 15, characterized in that: The processing unit is also used for: A plurality of different executable files among the executable files corresponding to the plurality of operator nodes are packaged to obtain second model information of the neural network model.

17. The device according to any one of claims 12 to 15, characterized in that: The plurality of computing nodes include a to-be-processed computing node, and the to-be-processed computing node corresponds to a first executable file; The sending unit is further configured to send the first executable file when the first executable file has not been sent.

18. The device according to any one of claims 12 to 17, characterized in that: When the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models share the same second executable file.

19. A model processing device, characterized in that: The device comprises: A receiving unit, configured to obtain second model information of a neural network model, wherein the neural network model corresponds to a plurality of computing nodes, and the second model information includes a plurality of different executable files among the executable files corresponding to the plurality of operator nodes, and each executable file includes a kernel function; A processing unit is used to execute multiple different executable files in the second model information.

20. The device according to claim 19, characterized in that The second model information further includes mapping relationship information, where the mapping relationship information is used to indicate a corresponding relationship between the multiple operator nodes, the different executable files, and at least one kernel function corresponding to each operator node; The processing unit is further configured to determine a first kernel function corresponding to a to-be-processed computing node among the plurality of computing nodes according to parameter information of the to-be-processed computing node; The processing unit is further configured to obtain a first executable file from the multiple different executable files according to the first kernel function and the mapping relationship information, where the first executable file includes a first kernel function.

21. The device according to claim 19, characterized in that The plurality of computing nodes include a to-be-processed computing node, and the to-be-processed computing node corresponds to a first kernel function; The receiving unit is further configured to receive a first executable file, where the first executable file includes the first kernel function.

22. The device according to any one of claims 19 to 21, characterized in that When the second executable file corresponding to the neural network model is the same as the second executable file corresponding to other neural network models, the neural network model and the other neural network models share the same second executable file.

23. A model processing device, characterized in that: The model processing device includes a processor and a memory, wherein instructions are stored in the memory. When the processor executes the instructions, the model processing device executes the model processing method according to any one of claims 1 to 7.

24. A model processing device, characterized in that: The model processing device includes a processor and a memory, wherein instructions are stored in the memory. When the processor executes the instructions, the model processing device executes the model processing method according to any one of claims 8 to 11.

25. A terminal device, characterized in that: The terminal device comprises: a model processing device as described in any one of claims 12-18 or claim 23, and a model processing device as described in any one of claims 19-22 or claim 24.

26. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed by a device, the device executes the model processing method according to any one of claims 1 to 11.

Images