Neural network model conversion acceleration method, device, computing device and storage medium

Abstract

The present application provides a neural network model conversion acceleration method, apparatus, computing device, and storage medium. The neural network model conversion acceleration method includes: receiving at least one neural network model to be processed; grouping the at least one neural network model to be processed, and sending the at least one neural network model group obtained by grouping to at least one preset parsing device; loading a layer information cache file; and parsing the corresponding neural network model group to be processed through the at least one parsing device based on the layer information cache file, and storing the parsing results in the layer information cache file. This method fully utilizes hardware performance, greatly improves model conversion efficiency, and reduces user waiting time.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and more particularly to a method for accelerating neural network model conversion. The present application also relates to a neural network model conversion acceleration device, a computing device, and a computer-readable storage medium. Background Art

[0002] In the field of deep learning, there are many neural network frameworks, such as TensorFlow, PyTorch, and PaddlePaddle. After training the model using these frameworks, the model will be converted to a unified, framework-independent ONNX format.

[0003] NVIDIA TensorRT is a framework for high-performance deep learning reasoning on NVIDIA GPU devices. It mainly consists of two parts: a model parser and an inference engine. To use TensorRT to reason about an ONNX model, you must first use the TensorRT ONNX parser to convert the ONNX model into a TensorRT Engine. The TensorRT Engine is a model that is specifically optimized for reasoning on a specified CUDA device. Then, use TensorRT TEngine to perform model reasoning.

[0004] Converting an ONNX model to a TensorRT Engine is very time-consuming. In actual use cases, the time required varies depending on the graphics card, and can sometimes take up to 10 minutes for a single model. To avoid this time consuming process each time the program is launched, a common practice is to serialize the TensorRT Engine to obtain a TensorRT Plan upon first obtaining it, and then save the TensorRT Plan to the local disk. In subsequent use, the TensorRT Plan can be directly deserialized to obtain the TensorRT Engine, and then loaded onto the GPU for model inference.

[0005] However, some scenarios require more than ten different models, and it takes hundreds of minutes for TensorRT to convert the ONNX model into the TensorRT Engine, which seriously affects usage efficiency and user experience. Summary of the Invention

[0006] In view of this, the embodiments of the present application provide a neural network model conversion acceleration method to address the technical defects existing in the prior art. The embodiments of the present application also provide a neural network model conversion acceleration device, a computing device, and a computer-readable storage medium.

[0007] According to a first aspect of an embodiment of the present application, a method for accelerating neural network model conversion is provided, comprising:

[0008] S10: Receive at least one neural network model to be processed;

[0009] S20: Grouping the at least one neural network model to be processed, and sending the at least one neural network model group to be processed obtained by grouping to at least one preset parsing device;

[0010] S30: Loading layer information cache file;

[0011] S40: Based on the layer information cache file, the corresponding neural network model group to be processed is parsed through the at least one parsing device, and the parsing result is stored in the layer information cache file.

[0012] Optionally, in step S20, grouping the at least one neural network model to be processed includes:

[0013] S21: Determine the number of groups and the number of group devices according to the number of devices of the at least one parsing device and the number of models of the at least one neural network model to be processed;

[0014] S22: Sort the at least one neural network model to be processed according to model size to obtain a sequence of neural network models to be processed;

[0015] S23: Grouping the at least one neural network model to be processed according to the sequence of neural network models to be processed, the number of groups and the number of group devices to obtain the at least one neural network model group to be processed.

[0016] Optionally, step S23 includes:

[0017] S231: Alternately selecting the neural network model to be processed from the sequence of neural network models to be processed;

[0018] S232: Add the selected neural network model to be processed to the current group of neural network models to be processed until the number of models in the current group of neural network models to be processed is equal to the number of devices in the group, and select the next group of neural network models to be processed until all the neural network models to be processed are selected, thereby obtaining all the groups of neural network models to be processed.

[0019] Optionally, the step S30 includes:

[0020] Determine whether the layer information cache file exists on the local device;

[0021] If so, load the layer information cache file;

[0022] If not, create and load an empty cache file of the layer information.

[0023] Optionally, the step S40 includes:

[0024] During the process of parsing the corresponding neural network model group to be processed, by the at least one parsing device, querying the layer information cache file to determine whether the layer information cache file contains the layer file of the neural network model to be processed currently being parsed;

[0025] If so, the corresponding layer file is used as the layer parsing result of the neural network model to be processed;

[0026] If not, parse the neural network model currently being parsed and store the parsing results in the layer information cache file.

[0027] According to a second aspect of an embodiment of the present application, a neural network model conversion acceleration device is provided, comprising:

[0028] A receiving module configured to receive at least one neural network model to be processed;

[0029] a grouping module configured to group the at least one neural network model to be processed, and send the at least one neural network model group to be processed obtained by grouping to at least one preset parsing device;

[0030] A loading module is configured to load a layer information cache file;

[0031] The parsing module is configured to parse the corresponding neural network model group to be processed based on the layer information cache file through the at least one parsing device, and store the parsing results in the layer information cache file.

[0032] According to a third aspect of an embodiment of the present application, a computing device is provided, including:

[0033] memory and processor;

[0034] The memory is used to store computer-executable instructions, and the processor implements the steps of the neural network model conversion acceleration method when executing the computer-executable instructions.

[0035] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions, which, when executed by a processor, implement the steps of the neural network model conversion acceleration method.

[0036] According to a fifth aspect of an embodiment of the present application, a chip is provided, which stores a computer program, and when the computer program is executed by the chip, the steps of the neural network model conversion acceleration method are implemented.

[0037] The neural network model conversion acceleration method provided in this application comprises the following steps: S10: receiving at least one neural network model to be processed; S20: grouping the at least one neural network model to be processed and sending the grouped at least one neural network model group to at least one preset parsing device; S30: loading a layer information cache file; and S40: parsing the corresponding neural network model group to be processed by the at least one parsing device based on the layer information cache file, and storing the parsing results in the layer information cache file. This method fully utilizes hardware performance, greatly improves model conversion efficiency, and reduces user waiting time. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 This is a flowchart of a neural network model conversion acceleration method provided by one embodiment of the present application;

[0040] Figure 2 This is a model grouping diagram of a neural network model conversion acceleration method provided by an embodiment of the present application;

[0041] Figure 3 This is a single model conversion flow chart of a neural network model conversion acceleration method provided by one embodiment of the present application;

[0042] Figure 4 This is a schematic diagram of the structure of a neural network model conversion acceleration device provided by one embodiment of the present application;

[0043] Figure 5 This is a structural block diagram of a computing device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0044] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0045] The terms used in one or more embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present application. The singular forms "a", "the" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0046] It should be understood that although the terms "first," "second," and the like may be used to describe various information in one or more embodiments of the present application, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, "first" may also be referred to as "second," and similarly, "second" may also be referred to as "first," without departing from the scope of one or more embodiments of the present application.

[0047] First, the terms involved in one or more embodiments of the present invention are explained.

[0048] ONNX: Open Neural Network Exchange, is an open neural network exchange format designed to facilitate model conversion and communication between different deep learning frameworks.

[0049] TensorRT: A high-performance deep learning inference engine provided by NVIDIA. It is designed for deployment in production environments to improve the inference speed and efficiency of deep learning models running on NVIDIA GPUs.

[0050] CUDA: Compute Unified Device Architecture, is a computing platform launched by graphics card manufacturer NVIDIA.

[0051] TensorRT Engine: A model that has been optimized specifically for TensorRT and is used to perform inference.

[0052] This application provides a neural network model conversion acceleration method. This application also relates to a neural network model conversion acceleration device, a computing device, and a computer-readable storage medium, which are described in detail in the following embodiments.

[0053] Figure 1 A flowchart of a neural network model conversion acceleration method provided according to an embodiment of the present application is shown, which specifically includes the following steps:

[0054] Step S10: receiving at least one neural network model to be processed;

[0055] Step S20: grouping the at least one neural network model to be processed, and sending the at least one neural network model group to be processed obtained by grouping to at least one preset parsing device;

[0056] Step S30: Loading layer information cache file;

[0057] Step S40: Based on the layer information cache file, the corresponding neural network model group to be processed is parsed through the at least one parsing device, and the parsing result is stored in the layer information cache file.

[0058] Among them, the neural network model to be processed is a model in ONNX format; the parsing device is various types of GPUs, etc. It should be noted that the type of parsing device is not fixed, and the specific type of parsing device used is determined by the actual usage scenario, which is not limited in this embodiment. In addition, in actual usage scenarios, the parsing device is more often chosen as a GPU.

[0059] Based on this, in actual usage scenarios, for multi-graphics card devices, multi-threading technology is used to designate different graphics cards to be responsible for processing multiple neural network models to be processed, thereby improving hardware utilization. In addition, multiple neural network models to be processed are placed in multi-threaded tasks for simultaneous processing, greatly reducing the total time required for model conversion. If the number of parsing devices is dev_num, the time required is reduced to 1 / dev_num of the original time required.

[0060] In addition, in actual usage scenarios, when converting the pending neural network model in ONNX format to TensorRTEngine, TensorRT will calculate an optimal strategy for each layer of the ONNX model to achieve the fastest inference, which is the most time-consuming step. The conversion information of each layer of the pending neural network model is saved by layer information caching. If there are other layers with the same IO tensor configuration and layer parameters as the layer information cache in the layer information cache file, the layer information cache can be directly referenced. This avoids repeated conversion of repeated layers for the conversion process of the pending neural network model, reduces repeated and redundant calculation processes, and improves conversion efficiency.

[0061] Furthermore, in step S20, the process of grouping at least one neural network model to be processed is specifically implemented as follows in this embodiment:

[0062] S21: Determine the number of groups and the number of group devices based on the number of devices of the at least one parsing device and the number of models of the at least one neural network model to be processed; S22: Sort the at least one neural network model to be processed according to the model size to obtain a sequence of neural network models to be processed; S23: Group the at least one neural network model to be processed based on the sequence of neural network models to be processed, the number of groups and the number of group devices to obtain the at least one neural network model group to be processed.

[0063] If the number of devices is dev_num and the number of models is model_num, then we can determine that the number of groups is equal to the number of devices, and then divide the number of models by the number of devices, that is, , obtaining the quotient and remainder. This determines that each thread is responsible for converting at least quotient models, and the first remainder threads process one additional model. This means that the first remainder groups have quotient + 1 group devices, while the remaining groups have quotient group devices. It should be noted that one parsing device corresponds to one group, and therefore one thread.

[0064] Based on this, all the neural network models to be processed are sorted according to the model size, and the neural network models to be processed in each group are selected from the sorted sequence of neural network models to be processed, that is, the neural network models to be processed that need to be converted and analyzed by each analysis device are determined.

[0065] Furthermore, in step S23, at least one neural network model to be processed is grouped according to the sequence of neural network models to be processed, the number of groups, and the number of group devices to obtain at least one neural network model group to be processed. In this embodiment, the specific implementation process is as follows:

[0066] S231: Alternately select the neural network models to be processed from the sequence of neural network models to be processed; S232: Add the selected neural network models to be processed to the current group of neural network models to be processed, until the number of models in the current group of neural network models to be processed is equal to the number of devices in the group, and select the next group of neural network models to be processed, until all the neural network models to be processed are selected, and obtain all the groups of neural network models to be processed.

[0067] Among them, the number of neural network model groups to be processed is equal to the number of groups; after obtaining the sequence of neural network models to be processed, each thread will select the model it is responsible for processing from the sorted model array in turn. Specifically, the first group corresponding to the first parsing device will first select the model from the starting position of the sequence, and then select from the end of the sequence, and so on alternately until the first group meets the number of group devices corresponding to itself. After that, the second group will operate from the remaining models in the sequence according to the same alternating selection principle until all groups are successfully assigned to the models they need to process.

[0068] Based on this, because model size and conversion time are positively correlated, the total model size of each group of models divided in the above way should be as close as possible to ensure that the time consumed by each thread is close, ultimately ensuring that the total time consumed is minimized.

[0069] Specifically, such as Figure 2 The model grouping diagram of a neural network model conversion acceleration method provided is shown as follows. If there are 5 neural network models to be processed, the sequence of neural network models to be processed obtained after sorting according to model size is shown in the model list. The 5 neural network models to be processed are models 1-5. If there are 3 parsing devices, there will be 3 groups, that is, there will be 3 working threads, namely working threads 1-3. The grouping results are as follows: Figure 2 shown.

[0070] Furthermore, in step S30, the process of loading the layer information cache file is specifically implemented as follows in this embodiment:

[0071] Determine whether the layer information cache file exists on the local device; if so, load the layer information cache file; if not, create and load an empty layer information cache file.

[0072] Specifically, such as Figure 3 As shown in the single model conversion flowchart of the provided neural network model conversion acceleration method, the ONNX model is input, that is, the neural network model to be processed is sent to the parsing device, and then it is determined whether there is a cache file locally, that is, whether there is a layer information cache file. If so, the cache file is loaded, and if not, an empty cache is created.

[0073] Furthermore, in step S40, based on the layer information cache file, at least one parsing device parses the corresponding neural network model group to be processed, and stores the parsing results in the layer information cache file. In this embodiment, the specific implementation method is as follows:

[0074] During the process of parsing the corresponding group of neural network models to be processed through the at least one parsing device, the layer information cache file is queried to determine whether the layer information cache file contains the layer file of the neural network model to be processed currently being parsed; if so, the corresponding layer file is used as the layer parsing result of the neural network model to be processed currently being parsed; if not, the neural network model to be processed currently being parsed is parsed, and the parsing result is stored in the layer information cache file.

[0075] Among them, for the neural network models to be processed, many model structures may be similar. For example, the license plate detection model and the vehicle detection model are both target detection models. The structures of these two models are the same, and the only difference is the data. After TensorRT conversion, the optimal strategies calculated for the same layers in the two models are consistent. Repeated parsing and conversion will result in repeated redundant calculation processes.

[0076] Therefore, by caching the layer information during the conversion of the ONNX model to the TensorRT Engine, for the same ONNX layer, the optimal strategy can be obtained directly by reading the cache, thereby accelerating the model conversion process. In other words, for the same model layer, TensorRT does not need to search for the optimal operator, but directly selects the operator based on the information in the cache file. The model compilation process is greatly accelerated, and the layer information cache file will be updated after any model compilation is completed, so that the cache information of the model layer will increase, and the subsequent model compilation time will be further shortened. The more models there are, the more obvious the acceleration effect is. According to actual measured data, the subsequent single model time will be reduced by 5-10 times.

[0077] Specifically, such as Figure 3 As shown in the single model conversion flowchart of the provided neural network model conversion acceleration method, the cache is added to the TensorRT configuration so that the parsing device can call the data in the layer information cache file, and then perform the model conversion and update the layer information cache. That is, after completing the current pending neural network model conversion, the parsing result is stored in the layer information cache file, and finally the layer information cache is saved locally to facilitate the direct call of the layer information cache file during the next execution of the pending neural network model conversion.

[0078] Corresponding to the above method embodiment, the present application also provides an embodiment of a neural network model conversion acceleration device, Figure 4 FIG. 1 shows a schematic diagram of a neural network model conversion acceleration device provided by an embodiment of the present application. Figure 4 As shown, the device includes:

[0079] A receiving module 402 is configured to receive at least one neural network model to be processed;

[0080] A grouping module 404 is configured to group the at least one neural network model to be processed, and send the at least one neural network model group to be processed obtained by grouping to at least one preset parsing device;

[0081] A loading module 406 is configured to load a layer information cache file;

[0082] The parsing module 408 is configured to parse the corresponding neural network model group to be processed based on the layer information cache file through the at least one parsing device, and store the parsing results in the layer information cache file.

[0083] In an optional embodiment, the grouping module 404 is further configured to:

[0084] Determine the number of groups and the number of group devices based on the number of devices of the at least one parsing device and the number of models of the at least one neural network model to be processed; sort the at least one neural network model to be processed according to the model size to obtain a sequence of neural network models to be processed; group the at least one neural network model to be processed based on the sequence of neural network models to be processed, the number of groups and the number of group devices to obtain the at least one neural network model group to be processed.

[0085] In an optional embodiment, the grouping module 404 is further configured to:

[0086] The neural network models to be processed in the sequence of neural network models to be processed are selected alternately from the beginning to the end; the selected neural network models to be processed are added to the current group of neural network models to be processed until the number of models in the current group of neural network models to be processed is equal to the number of devices in the group, and the next group of neural network models to be processed is selected until all the neural network models to be processed are selected, thereby obtaining all the groups of neural network models to be processed.

[0087] In an optional embodiment, the loading module 406 is further configured to:

[0088] Determine whether the layer information cache file exists on the local device; if so, load the layer information cache file; if not, create and load an empty layer information cache file.

[0089] In an optional embodiment, the parsing module 408 is further configured to:

[0090] During the process of parsing the corresponding group of neural network models to be processed through the at least one parsing device, the layer information cache file is queried to determine whether the layer information cache file contains the layer file of the neural network model to be processed currently being parsed; if so, the corresponding layer file is used as the layer parsing result of the neural network model to be processed currently being parsed; if not, the neural network model to be processed currently being parsed is parsed, and the parsing result is stored in the layer information cache file.

[0091] The neural network model conversion acceleration device provided in this application receives at least one neural network model to be processed; groups the at least one neural network model to be processed, and sends the at least one grouped neural network model to be processed to at least one preset parsing device; loads a layer information cache file; and, based on the layer information cache file, parses the corresponding neural network model group to be processed via the at least one parsing device, and stores the parsing results in the layer information cache file. This fully utilizes hardware performance, greatly improves model conversion efficiency, and reduces user waiting time.

[0092] The above is a schematic scheme of a neural network model conversion acceleration device of this embodiment. It should be noted that the technical solution of the neural network model conversion acceleration device and the technical solution of the above-mentioned neural network model conversion acceleration method belong to the same concept. For details not described in detail in the technical solution of the neural network model conversion acceleration device, please refer to the description of the technical solution of the above-mentioned neural network model conversion acceleration method. In addition, the various components in the device embodiment should be understood as functional modules that must be established to implement each step of the program flow or each step of the method, and each functional module is not an actual functional division or separation definition. The device claim defined by such a group of functional modules should be understood as a functional module architecture that mainly implements the solution through the computer program recorded in the specification, and should not be understood as a physical device that mainly implements the solution through hardware.

[0093] Figure 5 The block diagram shows a structure of a computing device 500 according to an embodiment of the present application. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0094] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)), whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0095] In one embodiment of the present application, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 5 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0096] Computing device 500 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 500 can also be a mobile or stationary server.

[0097] Among them, the processor 520 is used to execute computer executable instructions of each step of the neural network model conversion acceleration method.

[0098] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the aforementioned neural network model conversion acceleration method are based on the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the aforementioned neural network model conversion acceleration method.

[0099] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to execute the steps of the neural network model conversion acceleration method.

[0100] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of this storage medium and the technical scheme of the aforementioned neural network model conversion acceleration method are based on the same concept. For details not described in detail in the technical scheme of the storage medium, please refer to the description of the technical scheme of the aforementioned neural network model conversion acceleration method.

[0101] An embodiment of the present application also provides a chip, which stores a computer program, and when the computer program is executed by the chip, implements the steps of the neural network model conversion acceleration method.

[0102] The foregoing description describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0103] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content of the computer-readable medium may be appropriately increased or decreased based on the requirements of legislation and patent practice within a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0104] It should be noted that for the aforementioned method embodiments, for ease of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0105] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0106] The preferred embodiments of the present application disclosed above are intended only to help illustrate the present application. The optional embodiments do not describe all details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of this application. This application selects and describes these embodiments in detail in order to better explain the principles and practical applications of this application, so that those skilled in the art can better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A neural network model conversion acceleration method, characterized in that: include: S10: Receive at least one neural network model to be processed, wherein the neural network model to be processed is a model in ONNX format; S20: Grouping the at least one neural network model to be processed, and sending the at least one neural network model group to be processed obtained by grouping to at least one preset parsing device. Specifically, the process of grouping the at least one neural network model to be processed includes: S21: Determining the number of groups and the number of group devices based on the number of devices of the at least one parsing device and the number of models of the at least one neural network model to be processed; S22: Sorting the at least one neural network model to be processed according to the model size to obtain a sequence of neural network models to be processed; S23: Alternating between the beginning and the end of the neural network models to be processed in the sequence of neural network models to be processed, adding the selected neural network models to be processed to the current group of neural network models to be processed, until the number of models in the current group of neural network models to be processed is equal to the number of group devices, and then selecting the next group of neural network models to be processed, until all the neural network models to be processed are selected, and all the groups of neural network models to be processed are obtained; S30: Loading layer information cache file; S40: Based on the layer information cache file, the corresponding neural network model group to be processed is parsed through the at least one parsing device, and the parsing result is stored in the layer information cache file. Specifically, the conversion information of each layer of the neural network model to be processed is saved. If there are other layers with the same IO tensor configuration and layer parameters as the layer information cache in the layer information cache file, the layer information cache can be directly referenced.

2. The method according to claim 1, characterized in that The step S30 includes: Determine whether the layer information cache file exists on the local device; If so, load the layer information cache file; If not, create and load an empty cache file of the layer information.

3. The method according to claim 1, characterized in that The step S40 includes: During the process of parsing the corresponding neural network model group to be processed, by the at least one parsing device, querying the layer information cache file to determine whether the layer information cache file contains the layer file of the neural network model to be processed currently being parsed; If so, the corresponding layer file is used as the layer parsing result of the neural network model to be processed; If not, the neural network model to be processed that is currently being parsed is parsed, and the parsing result is stored in the layer information cache file.

4. A neural network model conversion acceleration device, characterized in that: include: A receiving module is configured to receive at least one neural network model to be processed, wherein the neural network model to be processed is a model in ONNX format; The grouping module is configured to group the at least one neural network model to be processed, and send the at least one neural network model group to be processed obtained by grouping to at least one preset parsing device. Specifically, the process of grouping the at least one neural network model to be processed includes: S21: determining the number of groups and the number of group devices according to the number of devices of the at least one parsing device and the number of models of the at least one neural network model to be processed; S22: sorting the at least one neural network model to be processed according to the model size to obtain a sequence of neural network models to be processed; S23: alternately selecting the neural network models to be processed in the sequence of neural network models to be processed, adding the selected neural network models to be processed to the current group of neural network models to be processed, until the number of models in the current group of neural network models to be processed is equal to the number of group devices, and then selecting the next group of neural network models to be processed, until all the neural network models to be processed are selected, and all the groups of neural network models to be processed are obtained; A loading module is configured to load a layer information cache file; The parsing module is configured to parse the corresponding neural network model group to be processed based on the layer information cache file through the at least one parsing device, and store the parsing results in the layer information cache file. Specifically, the conversion information of each layer of the neural network model to be processed is saved. If there are other layers with the same IO tensor configuration and layer parameters as the layer information cache in the layer information cache file, the layer information cache can be directly referenced.

5. A computing device, characterized in that include: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the steps of the neural network model conversion acceleration method according to any one of claims 1 to 3.

6. A computer-readable storage medium storing computer instructions, characterized in that: When the instruction is executed by the processor, the steps of the neural network model conversion acceleration method described in any one of claims 1 to 3 are implemented.

Images