Neural network model test management system and model test method

Abstract

The invention relates to the technical field of neural network management and test, in particular to a neural network model test management system and a model test method. The system comprises a digital signal processor and a neural network processing unit, a storage module, a query interface, an uploading interface, a reasoning interface, a deletion interface and a configuration interface which are electrically connected with one another are integrated in the digital signal processor, and a neural network model list is stored in the storage module; the neural network model list comprises a plurality of neural network models and configuration information and parameters of the plurality of neural network models; and the neural network processing unit is electrically connected with the digital signal processor and is used for testing the neural network model transmitted by the digital signal processor and returning a test result. According to the neural network model test management system provided by the invention, the management of the neural network model on the DSP and the operation test of the neural network model on the NPU are simplified, and the system interaction efficiency and the user development experience are improved.

Description

Technical Field

[0001] The present invention relates to the field of neural network management and testing, and in particular to a neural network model testing management system and a model testing method. Background Art

[0002] In the field of modern artificial intelligence and embedded systems, neural network processing units (NPUs) are dedicated hardware accelerators that can significantly improve the efficiency and performance of neural network reasoning. NPUs usually work in conjunction with other processors (such as digital signal processors, DSPs) to achieve complex computing tasks through high-speed data transmission and processing.

[0003] A common existing technical solution is to transfer the neural network model to the DSP, set the neural network operating parameters, initialize the neural network model, and return the result after inference in the NPU. Test another neural network model, reset the neural network parameters, reinitialize and return the result after inference. In the entire test process, only the final inference result is retained; however, the existing technology still has many problems in practical applications, as follows:

[0004] a. Because there is no management system designed for multiple neural network models and the situation of multiple neural network models is not considered, the traditional testing method needs to be re-initialized before testing the neural network model, which is time-consuming and inefficient;

[0005] b. Since the neural network model is run in a linear and fixed process without sorting out and separating the process, it is impossible to manage multiple neural network models and lacks model management functions;

[0006] c. In the process of testing the neural network model, the traditional test has a single process control, ignores the importance of the intermediate results, and does not plan the input and output formats. As a result, only the final test results are retained, the intermediate results are ignored, and the input and output formats are not unified.

[0007] d. Since multiple external interfaces are not reasonably arranged on the digital signal processor, the existing digital signal processor cannot be adapted to various external requirements. Summary of the invention

[0008] The present invention provides a neural network model test management system and a model test method to solve the technical problems mentioned in the background technology.

[0009] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0010] The present invention provides a neural network model test management system, comprising:

[0011] A digital signal processor DSP internally integrates a storage module, a query interface, an upload interface, an inference interface, a deletion interface, and a configuration interface that are electrically connected to each other, wherein the storage module internally stores a neural network model list, and the neural network model list includes multiple neural network models, and configuration information and parameters of the multiple neural network models;

[0012] The neural network processing unit NPU is electrically connected to the digital signal processor DSP, and is used to test the neural network model transmitted from the digital signal processor DSP and return the test results.

[0013] Further, the configuration information and parameters respectively include all configuration information and all parameters required for each neural network model to call the neural network processing unit NPU;

[0014] The configuration information at least includes whether to perform pre-processing, whether to perform post-processing, whether to save normalized data, whether to save normalized quantized data, whether to save tensor data, and whether to save tensor dequantized data;

[0015] The parameters include at least NPU configuration parameters and model parameters, wherein the NPU configuration parameters include memory allocation information and input and output data buffer addresses; the model parameters include at least the number of channels, mean, and scaling ratio of the neural network model.

[0016] Furthermore, the query interface is used to query relevant information of the neural network model in the neural network model list, and the relevant information of the neural network model includes at least one or more of the name, type, and input and output information of the neural network model.

[0017] Further, the configuration interface is used to configure the neural network model in the neural network model list;

[0018] The configuration interface provides at least one or more of the following configuration information:

[0019] Whether to perform pre-processing, whether to perform post-processing, whether to save normalized data, whether to save normalized quantized data, whether to save tensor data, whether to save tensor dequantized data, whether the configuration information is saved on the digital signal processor DSP, and whether to return information on whether the save is successful.

[0020] Furthermore, the inference interface is mainly used to pre-process the input data to be inferred and / or post-process the test data obtained by the neural network processing unit NPU according to the configuration information of the neural network model being tested;

[0021] The information carried by the inference interface includes at least the input data to be inferred and the name of the neural network model.

[0022] Another aspect of the present invention further provides a model testing method for a neural network model testing management system, comprising the following steps:

[0023] S2. Find the neural network model to be tested in the neural network model list through the query interface;

[0024] S3. Call the configuration interface to configure the neural network model to be tested, and obtain the configuration parameters of the neural network model to be tested;

[0025] S4. The inference interface pre-processes the input data to be inferred according to the configuration parameters, and then uses the input data before or after pre-processing to test the neural network model to be tested in the neural network processing unit NPU to obtain the test results, and then post-processes the test results according to the configuration parameters to obtain the final test results.

[0026] Furthermore, before S2, the following steps are also included:

[0027] S1. By calling the upload interface, the neural network model file, parameter file, model name and model type to be tested are saved to the storage module of the digital signal processor DSP, the neural network model to be tested is initialized on the storage module, and then the initialized neural network model to be tested is written into the neural network model list.

[0028] Furthermore, the S1 specifically includes the following steps:

[0029] S11, uploading a file including a neural network model file, model parameters, model name, and model type to a digital signal processor DSP through an upload interface; wherein the model parameters are the number of channels, mean value, and scaling ratio of the uploaded neural network model;

[0030] S12. Allocate storage space for storing parameters, storing input data, and storing output data for the neural network model to be tested on the storage module, so as to obtain NPU configuration parameters, and then complete the initialization of the neural network model to be tested. If the initialization is successful, the configuration information and parameters of the uploaded neural network model are saved to the neural network model list. If the initialization is unsuccessful, the uploaded neural network model will be deleted in the digital signal processor DSP.

[0031] Furthermore, after S4, the following steps are also included:

[0032] S5. Call the deletion interface to destroy the tested neural network model file and the parameter file of the model in the digital signal processor DSP, thereby deleting the tested neural network model from the neural network model list.

[0033] Furthermore, the S4 specifically includes the following steps:

[0034] S41, passing the input data to be inferred and the name of the neural network model to be tested to the inference interface, and then searching for the corresponding neural network model in the neural network model list. If not found, return the information of inference failure and enter S48; if found, enter S42;

[0035] S42, judging whether pre-processing is required according to the configuration information of the neural network model to be tested, and if pre-processing is not required, entering S44; if pre-processing is required, normalizing the input data to be inferred, and then saving the normalized data according to the configuration information of the neural network model to be tested;

[0036] S43, quantizing the normalized data, and then saving the quantized data according to the configuration information of the neural network model to be tested;

[0037] S44, sending the currently obtained data to the neural network processing unit NPU, and testing the neural network model to be tested in the neural network processing unit NPU using the currently obtained data, so as to obtain a test result, wherein the currently obtained data is the input data to be inferred in S41 or the data obtained by the quantization processing in S43;

[0038] S45, judging whether post-processing is required based on the configuration information of the neural network model to be tested, if post-processing is required, if not, entering S48; saving the tensor data inferred by the inference interface based on the configuration information of the neural network model to be tested;

[0039] S46, performing inverse quantization processing on the test results, and saving the data obtained by the inverse quantization processing according to the configuration information of the neural network model to be tested;

[0040] S47. Since different neural network models have different post-processing, they need to be distinguished according to their types, and then the data obtained by the dequantization processing needs to be analyzed according to the types;

[0041] S48, packaging the latest data and sending it to an external device connected to the digital signal processor DSP, the latest data being the final test result.

[0042] Beneficial effects of the present invention:

[0043] 1. The present invention provides a neural network model test management system, which includes multiple standardized interfaces, including a query interface, an upload interface, an inference interface, a deletion interface, and a configuration interface. It simplifies the management and testing operations of the neural network model, improves the usability and reusability of the neural network model after deployment, thereby improving the user development experience and reducing the complexity of software development. In addition, it is also convenient for users to connect through standardized interfaces.

[0044] 2. The neural network model test management system of the present invention discloses a neural network model list including multiple neural network models, which simplifies the model parameter management of the neural network model, so that the user only needs to process one neural network model list when calling various neural network models, without having to provide the model parameters of each neural network model one by one. The flexibility of model use is improved. Since the neural network model list encapsulates the necessary configuration information and parameters of each model, fast switching between different models can be achieved through this neural network model list without initialization every time.

[0045] In addition, the neural network model list also enhances the consistency and reliability of the neural network model test management system. By defining and using a unified neural network model list, the consistency of different models calling the neural network processing unit NPU is ensured. All operations on the neural network model obtain parameters from the neural network model list, reducing system crashes or errors caused by parameter passing errors. This consistency design improves the reliability and stability of the system. It facilitates resource management and release. The neural network model list not only contains the parameters required for initialization and runtime, but also includes information on resource allocation and release. Users can easily initialize and destroy each neural network model to ensure that multiple neural network models can be managed and utilized.

[0046] 3. On the other hand, the present invention also provides a model testing method based on a neural network model testing management system. During the testing process, the configuration interface can be used to configure the neural network model to be tested according to user needs. Through configuration, the user can decide whether to pre-process and post-process the data that needs to be inferred, and realize customized storage of intermediate data.

[0047] In addition, the model testing method in the present invention also provides five standardized interface processing flows, namely, query interface, upload interface, inference interface, deletion interface and configuration interface, providing users with a standardized process for model testing. At the same time, the processing flow and return results of each interface are simple and clear, making it easier for users to get started. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a structural block diagram of the neural network model test management system in the present invention;

[0049] Figure 2 is a flow chart of the model testing method in the present invention;

[0050] Figure 3 This is a workflow diagram of the upload interface in the present invention;

[0051] Figure 4 It is a workflow diagram of the query interface in the present invention;

[0052] Figure 5 It is a workflow diagram for configuring the interface in the present invention;

[0053] Figure 6 A workflow diagram for deleting an interface in the present invention;

[0054] Figure 7 This is a workflow diagram of the reasoning interface in the present invention. DETAILED DESCRIPTION

[0055] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Preferred embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many other different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0056] It should also be noted that, in the embodiments of the present application, the same figure mark is used to represent the same component or the same part. For the same parts in the embodiments of the present application, the figure may only mark one of the parts or components as an example. It should be understood that the figure mark also applies to other identical parts or components.

[0057] Reference Figure 1 , the embodiment of the present application provides a neural network model test management system, including:

[0058] A digital signal processor DSP internally integrates a storage module, a query interface, an upload interface, an inference interface, a deletion interface, and a configuration interface that are electrically connected to each other, wherein the storage module internally stores a neural network model list, and the neural network model list includes multiple neural network models, and configuration information and parameters of the multiple neural network models;

[0059] The neural network processing unit NPU is electrically connected to the digital signal processor DSP, and is used to test the neural network model transmitted from the digital signal processor DSP and return the test results.

[0060] The neural network model test management system of the present invention discloses a neural network model list including multiple neural network models, which simplifies the model parameter management of the neural network model, so that the user only needs to process one neural network model list when calling various neural network models, without having to provide the model parameters of each neural network model one by one. The flexibility of model use is improved. Since the neural network model list encapsulates the necessary configuration information and parameters of each model, switching between different models can be achieved through this neural network model list without initialization every time.

[0061] In addition, the neural network model list also enhances the consistency and reliability of the neural network model test management system. By defining and using a unified neural network model list, the consistency of different models calling the neural network processing unit NPU is ensured. All operations on the neural network model obtain parameters from the neural network model list, reducing system crashes or errors caused by parameter passing errors. This consistency design improves the reliability and stability of the system. It facilitates resource management and release. The neural network model list not only contains the parameters required for initialization and runtime, but also includes information on resource allocation and release. Users can easily initialize and destroy each neural network model to ensure that multiple neural network models can be managed and utilized.

[0062] In some embodiments, the configuration information and parameters respectively include all configuration information and all parameters required for each neural network model to call the neural network processing unit NPU;

[0063] The configuration information at least includes whether to perform pre-processing, whether to perform post-processing, whether to save normalized data, whether to save normalized quantized data, whether to save tensor data, and whether to save tensor dequantized data;

[0064] The parameters include at least NPU configuration parameters and model parameters, wherein the NPU configuration parameters include memory allocation information and input and output data buffer addresses; the model parameters include at least the number of channels, mean, and scaling ratio of the neural network model.

[0065] In some embodiments, the query interface is used to query relevant information of a neural network model in a neural network model list, and the relevant information of the neural network model includes at least one or more of the name, type, and input and output information of the neural network model.

[0066] In some embodiments, the configuration interface is used to configure a neural network model in the neural network model list;

[0067] The configuration interface provides at least one or more of the following configuration information:

[0068] Whether to perform pre-processing, whether to perform post-processing, whether to save normalized data, whether to save normalized quantized data, whether to save tensor data, whether to save tensor dequantized data, whether the configuration information is saved on the digital signal processor DSP, and whether to return information on whether the save is successful.

[0069] In some embodiments, the inference interface is mainly used to pre-process the input data to be inferred and / or post-process the test data obtained by the neural network processing unit NPU according to the configuration information of the neural network model being tested;

[0070] The information carried by the inference interface includes at least the input data to be inferred and the name of the neural network model.

[0071] In some embodiments, the information carried by the upload interface includes: neural network model file, parameter file, model name, model type. The above information is received and parsed, and the neural network model file, parameter file, type are saved to the digital signal processor DSP and named with the above name.

[0072] In some embodiments, the information carried by the deletion interface includes: the neural network model name, and the corresponding model file and parameter file on the digital signal processor DSP can be deleted through the neural network model name.

[0073] Reference Figure 2 On the other hand, the present invention also provides a model testing method of a neural network model testing management system, and the method of testing using the above neural network model testing management system includes the following steps:

[0074] S2. Use the query interface to search for the neural network model to be tested in the neural network model list; the workflow of the query interface is shown in Figure 4 As shown;

[0075] S3. Call the configuration interface to configure the neural network model to be tested and obtain the configuration parameters of the neural network model to be tested. The workflow of the configuration interface is shown in Figure 5 As shown;

[0076] S4. The inference interface pre-processes the input data to be inferred according to the configuration parameters, and then uses the input data before or after pre-processing to test the neural network model to be tested in the neural network processing unit NPU to obtain the test results, and then post-processes the test results according to the configuration parameters to obtain the final test results.

[0077] The model testing method based on the neural network model testing management system provided by the present invention can utilize the configuration interface to configure the neural network model to be tested according to user needs during the testing process. Through the configuration, the user can decide whether to pre-process and post-process the data to be inferred, and realize the customized storage of intermediate data.

[0078] In addition, the model testing method in the present invention also provides five standardized interface processing flows, namely, query interface, upload interface, inference interface, deletion interface and configuration interface, providing users with a standardized process for model testing. At the same time, the processing flow and return results of each interface are simple and clear, making it easier for users to get started.

[0079] Reference Figure 2 and Figure 3 In some embodiments, the step S2 further includes the following steps:

[0080] S1. By calling the upload interface, the neural network model file, parameter file, model name and model type to be tested are saved to the storage module of the digital signal processor DSP, the neural network model to be tested is initialized on the storage module, and then the initialized neural network model to be tested is written into the neural network model list.

[0081] Reference Figure 2 and Figure 3 In some embodiments, the S1 specifically includes the following steps:

[0082] S11, uploading a file including a neural network model file, model parameters, model name, and model type to a digital signal processor DSP through an upload interface; wherein the model parameters are the number of channels, mean value, and scaling ratio of the uploaded neural network model;

[0083] S12. Allocate storage space for storing parameters, storing input data, and storing output data for the neural network model to be tested on the storage module, so as to obtain NPU configuration parameters, and then complete the initialization of the neural network model to be tested. If the initialization is successful, the configuration information and parameters of the uploaded neural network model are saved to the neural network model list. If the initialization is unsuccessful, the uploaded neural network model will be deleted in the digital signal processor DSP.

[0084] In some embodiments, the step S4 further includes the following steps:

[0085] S5. Call the deletion interface to destroy the tested neural network model file and the parameter file of the model in the digital signal processor DSP, thereby deleting the tested neural network model from the neural network model list. See the workflow of the deletion interface for details. Figure 6 shown.

[0086] Reference Figure 2 and Figure 7 In some embodiments, S4 specifically includes the following steps:

[0087] S41, passing the input data to be inferred and the name of the neural network model to be tested to the inference interface, and then searching for the corresponding neural network model in the neural network model list. If not found, return the information of inference failure and enter S48; if found, enter S42;

[0088] S42, judging whether pre-processing is required according to the configuration information of the neural network model to be tested, and if pre-processing is not required, entering S44; if pre-processing is required, normalizing the input data to be inferred, and then saving the normalized data according to the configuration information of the neural network model to be tested;

[0089] S43, quantizing the normalized data, and then saving the quantized data according to the configuration information of the neural network model to be tested;

[0090] S44, sending the currently obtained data to the neural network processing unit NPU, and testing the neural network model to be tested in the neural network processing unit NPU using the currently obtained data, so as to obtain a test result, wherein the currently obtained data is the input data to be inferred in S41 or the data obtained by the quantization processing in S43;

[0091] S45, judging whether post-processing is required according to the configuration information of the neural network model to be tested, if post-processing is required, if not, entering S48; saving the tensor data obtained by the inference interface according to the configuration information of the neural network model to be tested;

[0092] S46, performing inverse quantization processing on the test results, and saving the data obtained by the inverse quantization processing according to the configuration information of the neural network model to be tested;

[0093] S47. Since different neural network models have different post-processing, they need to be distinguished according to their types, and then the data obtained by the dequantization processing needs to be analyzed according to the types;

[0094] S48, packaging the latest data and sending it to an external device connected to the digital signal processor DSP, the latest data being the final test result.

[0095] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered within the protection scope of the present invention. In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A neural network model test management system, characterized in that: include: A digital signal processor DSP internally integrates a storage module, a query interface, an upload interface, an inference interface, a deletion interface, and a configuration interface that are electrically connected to each other, wherein the storage module internally stores a neural network model list, and the neural network model list includes multiple neural network models, and configuration information and parameters of the multiple neural network models; The neural network processing unit NPU is electrically connected to the digital signal processor DSP, and is used to test the neural network model transmitted from the digital signal processor DSP and return the test results.

2. The neural network model test management system according to claim 1, characterized in that: The configuration information and parameters respectively include all configuration information and all parameters required for each neural network model to call the neural network processing unit NPU; The configuration information at least includes whether to perform pre-processing, whether to perform post-processing, whether to save normalized data, whether to save normalized quantized data, whether to save tensor data, and whether to save tensor dequantized data; The parameters include at least NPU configuration parameters and model parameters, wherein the NPU configuration parameters include memory allocation information and input and output data buffer addresses; the model parameters include at least the number of channels, mean, and scaling ratio of the neural network model.

3. The neural network model test management system according to claim 1, characterized in that: The query interface is used to query relevant information of a neural network model in a neural network model list, and the relevant information of the neural network model includes at least one or more of the name, type, and input and output information of the neural network model.

4. The neural network model test management system according to claim 1, characterized in that: The configuration interface is used to configure the neural network model in the neural network model list; The configuration interface provides at least one or more of the following configuration information: Whether to perform pre-processing, whether to perform post-processing, whether to save normalized data, whether to save normalized quantized data, whether to save tensor data, whether to save tensor dequantized data, whether the configuration information is saved on the digital signal processor DSP, and whether to return information on whether the save is successful.

5. The neural network model test management system according to claim 1, characterized in that: The inference interface is mainly used to pre-process the input data to be inferred and / or post-process the test data obtained by the neural network processing unit NPU according to the configuration information of the tested neural network model; The information carried by the inference interface includes at least the input data to be inferred and the name of the neural network model.

6. A model testing method for a neural network model testing management system, characterized in that: The neural network model test management system according to any one of claims 1 to 5 is used for testing, comprising the following steps: S2. Find the neural network model to be tested in the neural network model list through the query interface; S3. Call the configuration interface to configure the neural network model to be tested, and obtain the configuration parameters of the neural network model to be tested; S4. The inference interface pre-processes the input data to be inferred according to the configuration parameters, and then uses the input data before or after pre-processing to test the neural network model to be tested in the neural network processing unit NPU to obtain the test results, and then post-processes the test results according to the configuration parameters to obtain the final test results.

7. The model testing method according to claim 6, characterized in that: The step S2 also includes the following steps: S1. By calling the upload interface, the neural network model file, parameter file, model name and model type to be tested are saved to the storage module of the digital signal processor DSP, the neural network model to be tested is initialized on the storage module, and then the initialized neural network model to be tested is written into the neural network model list.

8. The model testing method according to claim 7, characterized in that: The S1 specifically includes the following steps: S11, uploading a file including a neural network model file, model parameters, model name, and model type to a digital signal processor DSP through an upload interface; wherein the model parameters are the number of channels, mean value, and scaling ratio of the uploaded neural network model; S12. Allocate storage space for storing parameters, storing input data, and storing output data for the neural network model to be tested on the storage module, so as to obtain NPU configuration parameters, and then complete the initialization of the neural network model to be tested. If the initialization is successful, the configuration information and parameters of the uploaded neural network model are saved to the neural network model list. If the initialization is unsuccessful, the uploaded neural network model will be deleted in the digital signal processor DSP.

9. The model testing method according to claim 6, characterized in that: The step S4 also includes the following steps: S5. Call the deletion interface to destroy the tested neural network model file and the parameter file of the model in the digital signal processor DSP, thereby deleting the tested neural network model from the neural network model list.

10. The model testing method according to any one of claims 6 to 9, characterized in that: The S4 specifically includes the following steps: S41, passing the input data to be inferred and the name of the neural network model to be tested to the inference interface, and then searching for the corresponding neural network model in the neural network model list. If not found, return the information of inference failure and enter S48; if found, enter S42; S42, judging whether pre-processing is required according to the configuration information of the neural network model to be tested, and if pre-processing is not required, entering S44; if pre-processing is required, normalizing the input data to be inferred, and then saving the normalized data according to the configuration information of the neural network model to be tested; S43, quantizing the normalized data, and then saving the quantized data according to the configuration information of the neural network model to be tested; S44, sending the currently obtained data to the neural network processing unit NPU, and testing the neural network model to be tested in the neural network processing unit NPU using the currently obtained data, so as to obtain a test result, wherein the currently obtained data is the input data to be inferred in S41 or the data obtained by the quantization processing in S43; S45, judging whether post-processing is required based on the configuration information of the neural network model to be tested, if post-processing is required, if not, entering S48; saving the tensor data inferred by the inference interface based on the configuration information of the neural network model to be tested; S46, performing inverse quantization processing on the test results, and saving the data obtained by the inverse quantization processing according to the configuration information of the neural network model to be tested; S47, analyzing the data obtained by the dequantization process according to the type; S48, packaging the latest data and sending it to an external device connected to the digital signal processor DSP, the latest data being the final test result.

Images