A neural network model test management system and model testing method

Abstract

This invention relates to the field of neural network management and testing technology, specifically a neural network model test management system and model testing method. The system includes a digital signal processor (DSP) and a neural network processing unit (NN unit). The DSP integrates electrically connected storage modules, query interfaces, upload interfaces, inference interfaces, deletion interfaces, and configuration interfaces. The storage module stores a list of neural network models, containing multiple neural network models and their configuration information and parameters. The NN unit is electrically connected to the DSP and is used to test the neural network models transmitted from the DSP and return the test results. This invention proposes a neural network model test management system that simplifies the management of neural network models on a DSP and their operation and testing on an NPU, improving system interaction efficiency and user development experience.

Description

Technical Field

[0001] This invention relates to the field of neural network management and testing, and in particular to a neural network model testing management system and a model testing method. Background Technology

[0002] In the fields of modern artificial intelligence and embedded systems, neural network processing units (NPUs) serve as dedicated hardware accelerators, significantly improving the efficiency and performance of neural network inference. NPUs typically work in conjunction with other processors (such as digital signal processors and DSPs) to perform complex computational tasks through high-speed data transmission and processing.

[0003] A common existing technical solution is to transmit the neural network model to a DSP, set the neural network's operating parameters, initialize the neural network model, and then perform inference in the NPU and return the result. To test another neural network model, the parameters are reset, the model is re-initialized, and inference is performed again to return the result. Throughout the entire testing process, only the final inference result is retained. However, this existing technology still has several problems in practical applications, specifically as follows:

[0004] a. Because there is no management system for multiple neural network models and the situation of multiple neural network models is not considered, traditional testing methods require re-initialization before testing neural network models, which is time-consuming and inefficient.

[0005] b. Because the neural network model is run in a linear and fixed process without streamlining and separating the process, it is impossible to manage multiple neural network models and lacks model management functions.

[0006] c. Traditional testing methods for neural network models rely on simplistic process control, neglecting the importance of intermediate results and failing to plan input / output formats. This results in only the final test results being retained, while intermediate results are ignored, leading to inconsistent input / output formats.

[0007] d. Due to the lack of a reasonable layout of multiple external interfaces on the digital signal processor, the existing digital signal processor cannot be adapted to various external needs. Summary of the Invention

[0008] This invention provides a neural network model test management system and a model testing method to solve the technical problems mentioned in the background.

[0009] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0010] This invention provides a neural network model testing and management system, comprising:

[0011] The digital signal processor (DSP) integrates interconnected storage modules, query interfaces, upload interfaces, inference interfaces, delete interfaces, and configuration interfaces. The storage module stores a list of neural network models, which contains multiple neural network models and their configuration information and parameters.

[0012] The Neural Processing Unit (NPU) is electrically connected to the Digital Signal Processor (DSP) and is used to test the neural network model transmitted from the DSP and return the test results.

[0013] Furthermore, the configuration information and parameters include all the configuration information and all the parameters required for each neural network model to call the neural network processing unit (NPU);

[0014] The configuration information includes at least whether to perform preprocessing, whether to perform postprocessing, whether to save normalized data, whether to save normalized quantized data, whether to save tensor data, and whether to save tensor inverse quantization data.

[0015] The parameters include at least NPU configuration parameters and model parameters. The NPU configuration parameters include memory allocation information and input / output data buffer addresses; the model parameters include at least the number of channels, mean, and scaling ratio of the neural network model.

[0016] Furthermore, the query interface is used to query relevant information about neural network models in the list of neural network models. The relevant information about neural network models includes at least one or more of the following: the name, type, and input / output information of the neural network model.

[0017] Furthermore, the configuration interface is used to configure the neural network models in the list of neural network models;

[0018] The configuration interface provides at least one or more of the following configuration information:

[0019] Whether to perform preprocessing, whether to perform postprocessing, whether to save normalized data, whether to save normalized quantized data, whether to save tensor data, whether to save tensor dequantized data, whether to save configuration information to the digital signal processor (DSP), and whether to return information on whether the saving was successful.

[0020] Furthermore, the inference interface is mainly used to preprocess the input data to be inferred and / or postprocess the test data obtained by the neural network processing unit (NPU) based on the configuration information of the neural network model being tested.

[0021] The inference interface carries at least the input data to be inferred and the name of the neural network model.

[0022] In another aspect, the present invention provides a model testing method for a neural network model testing management system, comprising the following steps:

[0023] S2. Find the neural network model to be tested by querying the list of neural network models through the query interface;

[0024] S3. Call the configuration interface to configure the neural network model to be tested and obtain the configuration parameters of the neural network model to be tested;

[0025] S4. The inference interface preprocesses the input data to be inferred according to the configuration parameters. Then, it uses the input data before or after preprocessing to test the neural network model to be tested in the neural network processing unit (NPU) to obtain the test results. Finally, it performs postprocessing on the test results according to the configuration parameters to obtain the final test results.

[0026] Furthermore, the following steps are included before step S2:

[0027] S1. By calling the upload interface, the neural network model file, parameter file, model name and model type to be tested are saved to the storage module of the digital signal processor (DSP). The neural network model to be tested is initialized in the storage module, and then the initialized neural network model to be tested is written to the neural network model list.

[0028] Furthermore, step S1 specifically includes the following steps:

[0029] S11. Upload the file including the neural network model file, model parameters, model name, and model type to the digital signal processor (DSP) via the upload interface; where the model parameters are the number of channels, mean, and scaling ratio of the uploaded neural network model.

[0030] S12. Allocate storage space for storing parameters, input data, and output data for the neural network model to be tested on the storage module to obtain NPU configuration parameters, thereby completing the initialization of the neural network model to be tested. If the initialization is successful, the configuration information and parameters of the uploaded neural network model are saved to the neural network model list. If the initialization fails, the uploaded neural network model is deleted in the digital signal processor DSP.

[0031] Furthermore, the following steps are included after S4:

[0032] S5. Call the deletion interface to destroy the tested neural network model file and its parameter file in the digital signal processor (DSP), thereby deleting the tested neural network model from the neural network model list.

[0033] Furthermore, step S4 specifically includes the following steps:

[0034] S41. Pass the input data to be inferred and the name of the neural network model to be tested to the inference interface, and then search for the corresponding neural network model in the list of neural network models. If it is not found, return the inference failure information and proceed to S48; if it is found, proceed to S42.

[0035] S42. Determine whether preprocessing is required based on the configuration information of the neural network model to be tested. If no preprocessing is required, proceed to S44. If preprocessing is required, normalize the input data to be inferred, and then save the normalized data based on the configuration information of the neural network model to be tested.

[0036] S43. Quantize the normalized data, and then save the quantized data according to the configuration information of the neural network model to be tested.

[0037] S44. Send the currently obtained data to the neural network processing unit (NPU), and use the currently obtained data to test the neural network model to be tested in the neural network processing unit (NPU) to obtain the test result. The currently obtained data is either the input data to be inferred in S41 or the data obtained by quantization in S43.

[0038] S45. Determine whether post-processing is required based on the configuration information of the neural network model to be tested. If post-processing is required, proceed to S48. Save the tensor data obtained by inference through the inference interface based on the configuration information of the neural network model to be tested.

[0039] S46. Perform dequantization on the test results and save the data obtained from the dequantization process according to the configuration information of the neural network model to be tested.

[0040] S47. Since different neural network models have different post-processing, it is necessary to distinguish them according to their types, and then it is necessary to parse the data obtained by inverse quantization according to their types.

[0041] S48. Pack the latest data and send it to the external device connected to the digital signal processor (DSP). The latest data is the final test result.

[0042] The beneficial effects of this invention are:

[0043] 1. This invention provides a neural network model test management system, which includes multiple standardized interfaces, namely query interface, upload interface, inference interface, deletion interface and configuration interface. It simplifies the management and testing of neural network models, improves the usability and reusability of neural network models after deployment, thereby enhancing the user development experience and reducing the complexity of software development. In addition, it also facilitates user integration through standardized interfaces.

[0044] 2. This invention discloses a neural network model list, which includes multiple neural network models, within the neural network model testing and management system. This simplifies the management of neural network model parameters, allowing users to handle only one neural network model list when calling various neural network models, without needing to provide the model parameters for each individual model. This improves the flexibility of model usage. Because the neural network model list encapsulates the necessary configuration information and parameters for each model, it enables quick switching between different models without requiring initialization each time.

[0045] Furthermore, the neural network model list enhances the consistency and reliability of the neural network model test management system. By defining and using a unified neural network model list, consistency in calling the Neural Processing Unit (NPU) is ensured across different models. All operations on neural network models obtain parameters from the neural network model list, reducing system crashes or errors caused by incorrect parameter passing. This consistent design improves system reliability and stability. It also facilitates resource management and release. The neural network model list not only contains parameters required for initialization and runtime but also includes information on resource allocation and release. Users can easily initialize and destroy individual neural network models, ensuring the management and utilization of multiple neural network models.

[0046] 3. In another aspect, the present invention also provides a model testing method based on a neural network model test management system. During the testing process, the neural network model to be tested can be configured using a configuration interface according to user requirements. Through configuration, the user can decide whether to perform pre-processing and post-processing on the data to be inferred, and realize the custom saving of intermediate data.

[0047] Furthermore, the model testing method in this invention provides a standardized processing flow for five interfaces: query interface, upload interface, inference interface, deletion interface, and configuration interface. This provides users with a standardized process for model testing, and the processing flow and return results of each interface are simple and clear, making it easier for users to get started. Attached Figure Description

[0048] Figure 1 This is a structural block diagram of the neural network model test management system in this invention;

[0049] Figure 2 This is a flowchart of the model testing method in this invention;

[0050] Figure 3 This is a flowchart of the upload interface in this invention;

[0051] Figure 4 This is a flowchart of the query interface in this invention;

[0052] Figure 5 This is a flowchart illustrating the process of configuring the interface in this invention;

[0053] Figure 6 This is a flowchart illustrating the process of deleting an interface in this invention.

[0054] Figure 7 This is a flowchart of the inference interface in this invention. Detailed Implementation

[0055] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many other different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0056] It should also be noted that in the embodiments of this application, the same reference numerals are used to represent the same component or part. For the same part in the embodiments of this application, the reference numerals may only be used to mark one part or component as an example in the figure. It should be understood that the reference numerals are also applicable to other identical parts or components.

[0057] Reference Figure 1 This application provides a neural network model testing and management system, including:

[0058] The digital signal processor (DSP) integrates interconnected storage modules, query interfaces, upload interfaces, inference interfaces, delete interfaces, and configuration interfaces. The storage module stores a list of neural network models, which contains multiple neural network models and their configuration information and parameters.

[0059] The Neural Processing Unit (NPU) is electrically connected to the Digital Signal Processor (DSP) and is used to test the neural network model transmitted from the DSP and return the test results.

[0060] This invention discloses a neural network model list, comprising multiple neural network models, within a neural network model testing and management system. This simplifies the management of neural network model parameters, allowing users to access various neural network models by simply processing a single neural network model list, eliminating the need to provide the parameters for each individual model. This enhances the flexibility of model usage. Since the neural network model list encapsulates the necessary configuration information and parameters for each model, switching between different models can be achieved through this list without requiring initialization each time.

[0061] Furthermore, the neural network model list enhances the consistency and reliability of the neural network model test management system. By defining and using a unified neural network model list, consistency in calling the Neural Processing Unit (NPU) is ensured across different models. All operations on neural network models obtain parameters from the neural network model list, reducing system crashes or errors caused by incorrect parameter passing. This consistent design improves system reliability and stability. It also facilitates resource management and release. The neural network model list not only contains parameters required for initialization and runtime but also includes information on resource allocation and release. Users can easily initialize and destroy individual neural network models, ensuring the management and utilization of multiple neural network models.

[0062] In some embodiments, the configuration information and parameters include all configuration information and all parameters required for each neural network model to call the neural network processing unit (NPU);

[0063] The configuration information includes at least whether to perform preprocessing, whether to perform postprocessing, whether to save normalized data, whether to save normalized quantized data, whether to save tensor data, and whether to save tensor inverse quantization data.

[0064] The parameters include at least NPU configuration parameters and model parameters. The NPU configuration parameters include memory allocation information and input / output data buffer addresses; the model parameters include at least the number of channels, mean, and scaling ratio of the neural network model.

[0065] In some embodiments, the query interface is used to query relevant information about neural network models in a list of neural network models. The relevant information about neural network models includes at least one or more of the following: the name, type, and input / output information of the neural network model.

[0066] In some embodiments, the configuration interface is used to configure neural network models in the list of neural network models;

[0067] The configuration interface provides at least one or more of the following configuration information:

[0068] Whether to perform preprocessing, whether to perform postprocessing, whether to save normalized data, whether to save normalized quantized data, whether to save tensor data, whether to save tensor dequantized data, whether to save configuration information to the digital signal processor (DSP), and whether to return information on whether the saving was successful.

[0069] In some embodiments, the inference interface is mainly used to preprocess the input data to be inferred and / or postprocess the test data obtained by the neural network processing unit (NPU) based on the configuration information of the neural network model being tested.

[0070] The information carried by the inference interface includes at least the input data to be inferred and the name of the neural network model.

[0071] In some embodiments, the upload interface carries the following information: neural network model file, parameter file, model name, and model type. The system receives and parses this information, saves the neural network model file, parameter file, and model type to the digital signal processor (DSP), and names it using the aforementioned names.

[0072] In some embodiments, the deletion interface carries the following information: neural network model name, through which the corresponding model file and parameter file on the digital signal processor (DSP) can be deleted.

[0073] Reference Figure 2 In another aspect, the present invention provides a model testing method for a neural network model testing management system, which includes the following steps for testing using the aforementioned neural network model testing management system:

[0074] S2. Locate the neural network model to be tested by searching the list of neural network models through the query interface; see the workflow of the query interface. Figure 4 As shown;

[0075] S3. Call the configuration interface to configure the neural network model under test and obtain the configuration parameters of the neural network model under test; see the workflow of the configuration interface. Figure 5 As shown;

[0076] S4. The inference interface preprocesses the input data to be inferred according to the configuration parameters. Then, it uses the input data before or after preprocessing to test the neural network model to be tested in the neural network processing unit (NPU) to obtain the test results. Finally, it performs postprocessing on the test results according to the configuration parameters to obtain the final test results.

[0077] The model testing method based on the neural network model test management system provided by this invention allows users to configure the neural network model to be tested using a configuration interface according to their needs during the testing process. Through configuration, users can decide whether to perform pre-processing and post-processing on the data to be inferred, and to customize the saving of intermediate data.

[0078] Furthermore, the model testing method in this invention provides a standardized processing flow for five interfaces: query interface, upload interface, inference interface, deletion interface, and configuration interface. This provides users with a standardized process for model testing, and the processing flow and return results of each interface are simple and clear, making it easier for users to get started.

[0079] Reference Figure 2 and Figure 3 In some embodiments, the following steps are included before step S2:

[0080] S1. By calling the upload interface, the neural network model file, parameter file, model name and model type to be tested are saved to the storage module of the digital signal processor (DSP). The neural network model to be tested is initialized in the storage module, and then the initialized neural network model to be tested is written to the neural network model list.

[0081] Reference Figure 2 and Figure 3 In some embodiments, step S1 specifically includes the following steps:

[0082] S11. Upload the file including the neural network model file, model parameters, model name, and model type to the digital signal processor (DSP) via the upload interface; where the model parameters are the number of channels, mean, and scaling ratio of the uploaded neural network model.

[0083] S12. Allocate storage space for storing parameters, input data, and output data for the neural network model to be tested on the storage module to obtain NPU configuration parameters, thereby completing the initialization of the neural network model to be tested. If the initialization is successful, the configuration information and parameters of the uploaded neural network model are saved to the neural network model list. If the initialization fails, the uploaded neural network model is deleted in the digital signal processor DSP.

[0084] In some embodiments, the following steps are further included after step S4:

[0085] S5. Call the deletion interface to destroy the tested neural network model file and its parameter file in the digital signal processor (DSP), thereby deleting the tested neural network model from the neural network model list. See [link to deletion interface workflow] for details. Figure 6 As shown.

[0086] Reference Figure 2 and Figure 7 In some embodiments, step S4 specifically includes the following steps:

[0087] S41. Pass the input data to be inferred and the name of the neural network model to be tested to the inference interface, and then search for the corresponding neural network model in the list of neural network models. If it is not found, return the inference failure information and proceed to S48; if it is found, proceed to S42.

[0088] S42. Determine whether preprocessing is required based on the configuration information of the neural network model to be tested. If no preprocessing is required, proceed to S44. If preprocessing is required, normalize the input data to be inferred, and then save the normalized data based on the configuration information of the neural network model to be tested.

[0089] S43. Quantize the normalized data, and then save the quantized data according to the configuration information of the neural network model to be tested.

[0090] S44. Send the currently obtained data to the neural network processing unit (NPU), and use the currently obtained data to test the neural network model to be tested in the neural network processing unit (NPU) to obtain the test result. The currently obtained data is either the input data to be inferred in S41 or the data obtained by quantization in S43.

[0091] S45. Determine whether post-processing is required based on the configuration information of the neural network model to be tested. If post-processing is required, proceed to S48. Save the tensor data obtained by inference through the inference interface based on the configuration information of the neural network model to be tested.

[0092] S46. Perform dequantization on the test results and save the data obtained from the dequantization process according to the configuration information of the neural network model to be tested.

[0093] S47. Since different neural network models have different post-processing, it is necessary to distinguish them according to their types, and then it is necessary to parse the data obtained by inverse quantization according to their types.

[0094] S48. Pack the latest data and send it to the external device connected to the digital signal processor (DSP). The latest data is the final test result.

[0095] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A model testing method for a neural network model test management system, characterized in that, Includes the following steps: S2. Find the neural network model to be tested by querying the list of neural network models through the query interface; S3. Call the configuration interface to configure the neural network model to be tested and obtain the configuration parameters of the neural network model to be tested; S4. The inference interface preprocesses the input data to be inferred according to the configuration parameters. Then, it uses the input data before or after preprocessing to test the neural network model to be tested in the neural network processing unit (NPU) to obtain the test results. Finally, it performs postprocessing on the test results according to the configuration parameters to obtain the final test results. S4 specifically includes the following steps: S41. Pass the input data to be inferred and the name of the neural network model to be tested to the inference interface, and then search for the corresponding neural network model in the list of neural network models. If it is not found, return the inference failure information and proceed to S48; if it is found, proceed to S42. S42. Determine whether preprocessing is required based on the configuration information of the neural network model to be tested. If no preprocessing is required, proceed to S44. If preprocessing is required, normalize the input data to be inferred, and then save the normalized data based on the configuration information of the neural network model to be tested. S43. Quantize the normalized data, and then save the quantized data according to the configuration information of the neural network model to be tested. S44. Send the currently obtained data to the neural network processing unit (NPU), and use the currently obtained data to test the neural network model to be tested in the neural network processing unit (NPU) to obtain the test result. The currently obtained data is either the input data to be inferred in S41 or the data obtained by quantization in S43. S45. Determine whether post-processing is required based on the configuration information of the neural network model to be tested. If post-processing is not required, proceed to S48. If post-processing is required, save the tensor data obtained by inference through the inference interface based on the configuration information of the neural network model to be tested. S46. Perform dequantization on the test results and save the data obtained from the dequantization process according to the configuration information of the neural network model to be tested. S47. Data obtained by inverse quantization based on type parsing; S48. Pack the latest data and send it to the external device connected to the digital signal processor (DSP). The latest data is the final test result.

2. The model testing method according to claim 1, characterized in that, The following steps are included before step S2: S1. By calling the upload interface, the neural network model file, parameter file, model name and model type to be tested are saved to the storage module of the digital signal processor (DSP). The neural network model to be tested is initialized in the storage module, and then the initialized neural network model to be tested is written to the neural network model list.

3. The model testing method according to claim 2, characterized in that, S1 specifically includes the following steps: S11. Upload the file including the neural network model file, model parameters, model name, and model type to the digital signal processor (DSP) via the upload interface; where the model parameters are the number of channels, mean, and scaling ratio of the uploaded neural network model. S12. Allocate storage space for storing parameters, input data, and output data for the neural network model to be tested on the storage module to obtain NPU configuration parameters, thereby completing the initialization of the neural network model to be tested. If the initialization is successful, the configuration information and parameters of the uploaded neural network model are saved to the neural network model list. If the initialization fails, the uploaded neural network model is deleted in the digital signal processor DSP.

4. The model testing method according to claim 1, characterized in that, The following steps are included after step S4: S5. Call the deletion interface to destroy the tested neural network model file and its parameter file in the digital signal processor (DSP), thereby deleting the tested neural network model from the neural network model list.

5. A neural network model test management system, characterized in that, Model testing is performed using the model testing method according to any one of claims 1 to 4, including: The digital signal processor (DSP) integrates interconnected storage modules, query interfaces, upload interfaces, inference interfaces, delete interfaces, and configuration interfaces. The storage module stores a list of neural network models, which contains multiple neural network models and their configuration information and parameters. The Neural Processing Unit (NPU) is electrically connected to the Digital Signal Processor (DSP) and is used to test the neural network model transmitted from the DSP and return the test results.

6. The neural network model test management system according to claim 5, characterized in that, The configuration information and parameters include all the configuration information and all the parameters required for each neural network model to call the neural network processing unit (NPU); The configuration information includes at least whether to perform preprocessing, whether to perform postprocessing, whether to save normalized data, whether to save normalized quantized data, whether to save tensor data, and whether to save tensor inverse quantization data. The parameters include at least NPU configuration parameters and model parameters, wherein the NPU configuration parameters include memory allocation information and input / output data buffer addresses; the model parameters include at least the number of channels, mean, and scaling ratio of the neural network model.

7. The neural network model test management system according to claim 5, characterized in that, The query interface is used to query relevant information about neural network models in the list of neural network models. The relevant information about neural network models includes at least one or more of the following: the name, type, and input / output information of the neural network model.

8. The neural network model test management system according to claim 5, characterized in that, The configuration interface is used to configure the neural network models in the list of neural network models; The configuration interface provides at least one or more of the following configuration information: Whether to perform preprocessing, whether to perform postprocessing, whether to save normalized data, whether to save normalized quantized data, whether to save tensor data, whether to save tensor dequantized data, whether to save configuration information to the digital signal processor (DSP), and whether to return information on whether the saving was successful.

9. The neural network model test management system according to claim 5, characterized in that, The inference interface is used to preprocess the input data to be inferred and / or postprocess the test data obtained by the neural network processing unit (NPU) based on the configuration information of the neural network model being tested. The information carried by the inference interface includes at least the input data to be inferred and the name of the neural network model.

Images