Artificial intelligence model test method, mobile terminal, readable storage medium and program product

Abstract

The invention relates to an artificial intelligence model test method, a mobile terminal, a computer readable storage medium and a computer program product, relates to the technical field of artificial intelligence, and can improve the test efficiency of an artificial intelligence model on the mobile terminal. The method comprises the following steps: in response to an artificial intelligence model test operation triggered by a model test page provided by a mobile terminal, determining a to-be-tested artificial intelligence model and test configuration parameters; based on the test configuration parameters, the artificial intelligence model is initialized at the mobile terminal, and model reasoning is carried out by using the initialized artificial intelligence model; monitoring the operation process of the artificial intelligence model to obtain operation condition data of the artificial intelligence model in a model initialization stage and a model reasoning stage; and outputting a performance test result of the artificial intelligence model on the mobile terminal according to the operation condition data.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an artificial intelligence model testing method, a mobile terminal, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of computer technology, artificial intelligence (AI) models have been widely used. In order to improve the performance of AI models, performance testing of AI models can be performed. In related technologies, the performance of AI models is mainly tested on the PC (Personal Computer). Summary of the invention

[0003] Based on this, it is necessary to provide an artificial intelligence model testing method, a mobile terminal, a computer-readable storage medium and a computer program product applied to a mobile terminal to address the above technical problems.

[0004] In a first aspect, the present application provides an artificial intelligence model testing method, comprising:

[0005] In response to an artificial intelligence model test operation triggered by a model test page provided on the mobile terminal, determining an artificial intelligence model to be tested and test configuration parameters;

[0006] Initializing the artificial intelligence model on the mobile terminal based on the test configuration parameters and performing model reasoning using the initialized artificial intelligence model;

[0007] Monitoring the operation process of the artificial intelligence model to obtain operation status data of the artificial intelligence model in the model initialization stage and the model inference stage;

[0008] Based on the operating status data, the performance test results of the artificial intelligence model on the mobile terminal are output.

[0009] In one embodiment, in response to an artificial intelligence model test operation triggered by a model test page provided on a mobile terminal, determining test configuration parameters includes:

[0010] In response to an artificial intelligence model test operation triggered by a model test page provided on a mobile terminal, determining a first configuration parameter for setting an initialization method of the artificial intelligence model, and / or a second configuration parameter for setting a model reasoning method of the artificial intelligence model;

[0011] Based on the first configuration parameter and / or the second configuration parameter, a test configuration parameter is acquired.

[0012] In one of the embodiments, in response to the artificial intelligence model test operation triggered by the model test page provided on the mobile terminal, determining a first configuration parameter for setting an initialization method of the artificial intelligence model and / or a second configuration parameter for setting a model reasoning method of the artificial intelligence model includes:

[0013] In response to an artificial intelligence model test operation triggered by a model test page provided on the mobile terminal, determining a target inference engine selected by a user for running the artificial intelligence model;

[0014] Obtain a model operation strategy configured for the target inference engine, and determine, based on the model operation strategy, a first configuration parameter for setting an initialization method for the artificial intelligence model, and / or a second configuration parameter for setting a model reasoning method for the artificial intelligence model.

[0015] In one embodiment, based on the test configuration parameters, using the initialized artificial intelligence model to perform model reasoning includes:

[0016] Based on the number of inference cycles in the test configuration parameters, controlling the initialized artificial intelligence model to cyclically execute the model inference task corresponding to the number of inference cycles;

[0017] Obtaining the operation data of the artificial intelligence model in the model reasoning stage, including:

[0018] Acquire the memory state information of the mobile terminal and the operation operator running information corresponding to the artificial intelligence model each time the artificial intelligence model executes the model reasoning task;

[0019] According to the operation operator running information and the memory status information, the running status data when the model reasoning task is executed each time is obtained.

[0020] In one embodiment, obtaining the memory status information of the mobile terminal each time the artificial intelligence model performs the model reasoning task includes:

[0021] Each time the artificial intelligence model executes the model reasoning task, monitoring the reasoning interpreter corresponding to the artificial intelligence model on the mobile terminal;

[0022] When an inference start signal is heard from the inference interpreter, the current first memory state information of the inference interpreter is obtained; when an inference end signal is heard from the inference interpreter, the current second memory state information of the inference interpreter is obtained;

[0023] The memory state information of the artificial intelligence model each time executing the model reasoning task is obtained according to the first memory state information and the second memory state information.

[0024] In one embodiment, obtaining the operation operator running information corresponding to the artificial intelligence model each time the artificial intelligence model performs the model reasoning task includes:

[0025] Determine an operation operator that is called each time the artificial intelligence model executes the model reasoning task, and obtain at least one of an operator type, an operator calling order, and an operator calling number corresponding to the operation operator;

[0026] The operation operator running information is determined according to at least one of the operator type, the operator calling order and the operator calling number.

[0027] In one embodiment, outputting the performance test result of the artificial intelligence model on the mobile terminal according to the operation status data includes:

[0028] Performing statistical analysis on the operation status data according to one or more preset model performance evaluation dimensions to determine indicator information of at least one model performance indicator under each of the model performance evaluation dimensions;

[0029] generating a test report file from the indicator information of the at least one model performance indicator, and / or selecting key indicator information from the indicator information of the at least one model performance indicator;

[0030] Output the performance test result of the artificial intelligence model on the mobile terminal, and the performance test result includes the test report file and / or the key indicator information.

[0031] In a second aspect, the present application further provides a mobile terminal, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0032] In response to an artificial intelligence model test operation triggered by a model test page provided on the mobile terminal, determining an artificial intelligence model to be tested and test configuration parameters;

[0033] Initializing the artificial intelligence model on the mobile terminal based on the test configuration parameters and performing model reasoning using the initialized artificial intelligence model;

[0034] Monitoring the operation process of the artificial intelligence model to obtain operation status data of the artificial intelligence model in the model initialization stage and the model inference stage;

[0035] Based on the operating status data, the performance test results of the artificial intelligence model on the mobile terminal are output.

[0036] In a third aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0037] In response to an artificial intelligence model test operation triggered by a model test page provided on the mobile terminal, determining an artificial intelligence model to be tested and test configuration parameters;

[0038] Initializing the artificial intelligence model on the mobile terminal based on the test configuration parameters and performing model reasoning using the initialized artificial intelligence model;

[0039] Monitoring the operation process of the artificial intelligence model to obtain operation status data of the artificial intelligence model in the model initialization stage and the model inference stage;

[0040] Based on the operating status data, the performance test results of the artificial intelligence model on the mobile terminal are output.

[0041] In a third aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0042] In response to an artificial intelligence model test operation triggered by a model test page provided on the mobile terminal, determining an artificial intelligence model to be tested and test configuration parameters;

[0043] Initializing the artificial intelligence model on the mobile terminal based on the test configuration parameters and performing model reasoning using the initialized artificial intelligence model;

[0044] Monitoring the operation process of the artificial intelligence model to obtain operation status data of the artificial intelligence model in the model initialization stage and the model inference stage;

[0045] Based on the operating status data, the performance test results of the artificial intelligence model on the mobile terminal are output.

[0046] The above-mentioned artificial intelligence model testing method, mobile terminal, computer-readable storage medium and computer program product, in response to the artificial intelligence model testing operation triggered by the model testing page provided on the mobile terminal, can determine the artificial intelligence model to be tested and the test configuration parameters, and then, based on the test configuration parameters, initialize the artificial intelligence model on the mobile terminal and use the initialized artificial intelligence model to perform model reasoning, and monitor the operation process of the artificial intelligence model to obtain the operation status data of the artificial intelligence model in the model initialization stage and the model reasoning stage, and then output the performance test results of the artificial intelligence model on the mobile terminal according to the operation status data. In this embodiment, on the one hand, the test configuration parameters can be quickly set and the model test can be started through the model testing page provided by the mobile terminal itself, which effectively reduces the threshold of artificial intelligence model testing and improves the popularity of artificial intelligence model testing methods in production environments; on the other hand, by directly running the artificial intelligence model on the mobile terminal and outputting the performance test results according to the operation status data obtained by monitoring, the performance of the artificial intelligence model on the mobile terminal can be quickly and effectively determined, meeting the performance evaluation requirements of multiple types of artificial intelligence models in the mobile terminal fragmentation differentiated operation environment; thus, the test of the artificial intelligence model on the mobile terminal can be realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic diagram of a flow chart of an artificial intelligence model testing method in one embodiment;

[0049] Figure 2 A schematic diagram of a model test page in one embodiment;

[0050] Figure 3 A schematic diagram of a framework structure of an artificial intelligence model testing application in one embodiment;

[0051] Figure 4 A flowchart of the steps of running an artificial intelligence model during a test in one embodiment;

[0052] Figure 5 A flowchart of another artificial intelligence model testing method in one embodiment;

[0053] Figure 6It is a structural schematic diagram of an operation operator calculation information monitoring module in one embodiment;

[0054] Figure 7 A schematic diagram of the structure of a memory status information monitoring module in one embodiment;

[0055] Figure 8 It is a structural schematic diagram of a performance test result output module in one embodiment;

[0056] Fig. 9 The figure is a diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] In one embodiment, Figure 1 As shown, an artificial intelligence model testing method is provided, and this embodiment is described by applying the method to a mobile terminal. The mobile terminal may be, but is not limited to, various smart phones, tablet computers, handheld terminals, and portable wearable devices. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc.

[0059] In this embodiment, the method includes the following steps:

[0060] S101, in response to an artificial intelligence model test operation triggered by a model test page provided on a mobile terminal, determining an artificial intelligence model to be tested and test configuration parameters.

[0061] In the related technology, the performance of the artificial intelligence model is mainly tested on the PC side. Taking the mobile terminal of the Android system as an example, for the Android mobile terminal deployed with the artificial intelligence model, the mobile terminal will be connected to the PC side when testing the artificial intelligence model, and the debugging tool will be installed on the PC side at the same time, such as installing the debugging tool Android DebugBridge provided by the Android platform, and then professionals will use the debugging tool to run professional program files or enter debugging instructions to test the artificial intelligence model deployed on the mobile terminal. It can be understood that the above-mentioned testing method actually relies on the PC side and has a high testing threshold, and cannot be applied to ordinary daily production environments. When it is necessary to test the performance of the artificial intelligence model on different mobile terminals, it is necessary to repeatedly connect different mobile terminals to the PC side and have professionals enter test instructions, and it is impossible to quickly obtain the performance of the artificial intelligence model on various mobile terminals.

[0062] In this regard, in some embodiments, an artificial intelligence model test application can be provided and deployed on a mobile terminal. Among them, the artificial intelligence model test application can collect the evaluation information inside the artificial intelligence model and output key information after normalization and standardization. Due to the variety of mobile terminal models used by users, the running performance of artificial intelligence models on mobile terminals of different models may vary greatly. Through the artificial intelligence model test application of this embodiment, it can be easily installed on different mobile terminals to evaluate the running performance of various artificial intelligence models on the user's mobile terminal.

[0063] In a specific implementation, the mobile terminal may provide a visual model test page, and detect whether the user triggers an artificial intelligence model operation on the model test page, wherein the artificial intelligence model test operation may include an operation indicating the content of the artificial intelligence model test (for example, including indicating which artificial intelligence model needs to be tested, and may further include test configuration parameters), and may also include an operation indicating the start of the model test. After detecting the artificial intelligence model test operation, the mobile terminal may determine the artificial intelligence model to be tested and obtain the corresponding test configuration parameters. The model test page may be provided with one or more controls for setting test configuration parameters, wherein the test configuration parameters are parameters for controlling the artificial intelligence model test method. By executing relevant triggers in the model test page, the user can flexibly and dynamically configure the test configuration parameters in each artificial intelligence model test process.

[0064] S102: Initialize the artificial intelligence model in the mobile terminal based on the test configuration parameters and perform model reasoning using the initialized artificial intelligence model.

[0065] After obtaining the test configuration parameters, the artificial intelligence model can be run on the mobile terminal according to the test process and the test configuration parameters. Specifically, the test process may include multiple links involved in running the artificial intelligence model. In this embodiment, the multiple links involved in the test process may include key links related to the effect of model use: model initialization and model reasoning. By processing the artificial intelligence model according to the test process, the artificial intelligence model can enter an executable state and smoothly perform subsequent model reasoning. The test configuration parameters can adjust the parameters involved in one or more links in the artificial intelligence model test process.

[0066] In this step, the artificial intelligence model can be initialized on the mobile terminal based on the test configuration parameters, and the initialized artificial intelligence model can be used to perform model reasoning based on the test configuration parameters. By initializing the artificial intelligence model, the artificial intelligence model can be made executable, and then the model that has reached the executable state can be used to perform data reasoning in the model reasoning stage.

[0067] S103, monitor the operation process of the artificial intelligence model to obtain the operation status data of the artificial intelligence model in the model initialization stage and the model reasoning stage.

[0068] Among them, the operating status data can represent the operating status of the artificial intelligence model on the current mobile terminal. For the same artificial intelligence model, the operating status data of the artificial intelligence model on different mobile terminals may be different.

[0069] In practical applications, during the process of initializing the artificial intelligence model according to the test process and using the artificial intelligence model for reasoning, the operation process can be monitored, and the operation data of the artificial intelligence model in the initialization stage and the model reasoning stage can be obtained according to the monitoring results. Specifically, on the one hand, model initialization can make the artificial intelligence model executable. Wrong or abnormal initialization may cause the model performance to decline, while reasonable or correct initialization can provide a basis for the subsequent normal operation of the model. On the other hand, the artificial intelligence model obtains the output results mainly through the processing of the input data in the model reasoning stage. The situation in the model reasoning stage may be directly related to the evaluation dimensions such as model performance and reasoning speed. In this regard, in this embodiment, by obtaining the operation data of the model in the initialization stage and the model reasoning stage, the model performance can be comprehensively detected in combination with multiple key links of the model operation to improve the test accuracy of the artificial intelligence model on the mobile terminal.

[0070] S104: Output the performance test results of the artificial intelligence model on the mobile terminal based on the operating status data.

[0071] Since the operation status data can characterize the operation status of the artificial intelligence model in the initialization stage and the model reasoning stage on the mobile terminal, after obtaining the operation status data, the performance test result of the artificial intelligence model on the current mobile terminal is determined and output according to the operation status data. For example, the various operation status data obtained can be summarized as the performance test result, or the operation status data can be further processed, such as data screening and data processing, to obtain the performance test result according to the data processing result.

[0072] In the above-mentioned artificial intelligence model testing method, in response to the artificial intelligence model testing operation triggered by the model testing page provided on the mobile terminal, the artificial intelligence model to be tested and the test configuration parameters can be determined. Then, based on the test configuration parameters, the artificial intelligence model is initialized on the mobile terminal and the initialized artificial intelligence model is used for model reasoning, and the operation process of the artificial intelligence model is monitored to obtain the operation data of the artificial intelligence model in the model initialization stage and the model reasoning stage, and then the performance test results of the artificial intelligence model on the mobile terminal are output according to the operation data. In this embodiment, on the one hand, the test configuration parameters can be quickly set and the model test can be started through the model testing page provided by the mobile terminal itself, which effectively reduces the threshold of artificial intelligence model testing and improves the popularity of artificial intelligence model testing methods in production environments; on the other hand, by directly running the artificial intelligence model on the mobile terminal and outputting the performance test results according to the operation data obtained by monitoring, the performance of the artificial intelligence model on the mobile terminal can be quickly and effectively determined, meeting the performance evaluation requirements of multiple types of artificial intelligence models in the fragmented and differentiated operation environment of the mobile terminal; thus, the test of the artificial intelligence model on the mobile terminal can be realized.

[0073] In an exemplary embodiment, step S101 may include the following steps:

[0074] In response to an artificial intelligence model testing operation triggered by a model testing page provided on a mobile terminal, determine a first configuration parameter for setting an initialization method of the artificial intelligence model, and / or a second configuration parameter for setting a model reasoning method of the artificial intelligence model; based on the first configuration parameter and / or the second configuration parameter, obtain a test configuration parameter.

[0075] In a specific implementation, the user can perform an artificial intelligence model test operation on the model test page provided by the mobile terminal, and set configuration parameters related to the initialization method of the artificial intelligence model. For ease of distinction, the configuration parameters used to set the initialization method of the artificial intelligence model are referred to as the first configuration parameters. Specifically, the initialization method of the artificial intelligence model may be affected by the test configuration parameters. For example, if the user sets an acceleration agent such as a graphics processing unit (GPU), related functions such as the ModifyGraphWithDelegate function will be called during the initialization process to replace the proxy operator of the model calculation graph corresponding to the artificial intelligence model. In this regard, the corresponding first configuration parameters can be set on the model test page. In some exemplary embodiments, the first configuration parameters related to the initialization of the artificial intelligence model may include one or more of the following: the inference engine running the artificial intelligence model, whether to enable the neural network inference library acceleration agent, whether to enable the GPU acceleration agent, the amount of memory, and the input data format. Among them, the inference engine parameters can be used to initialize the inference engine used when running the artificial intelligence model and the computing resources related to the inference engine; whether to enable the neural network inference library acceleration agent and the GPU acceleration agent can be used to initialize the operation operators used when the artificial intelligence model is running; the amount of memory can be used to initialize the memory size allocated to each tensor of the model; the input data format can be used to perform initialization processing such as data format conversion on the input data to be processed by the artificial intelligence model.

[0076] In addition, the user can also set configuration parameters related to the model reasoning mode of the artificial intelligence model by performing an artificial intelligence model test operation. For the sake of distinction, this configuration parameter is called a second configuration parameter. In some exemplary embodiments, the second configuration parameter related to the initialization of the artificial intelligence model may include one or more of the following: the number of reasoning cycles of the model, whether to enable or disable parallel computing, the number of threads for performing reasoning, whether to enable or disable mobile terminal cache, and the cache size that can be used during model operation.

[0077] In addition, in some embodiments, the user can also set configuration parameters related to the content of the performance test results through the model test page. For the sake of distinction, the configuration parameters can be referred to as third configuration parameters. Exemplarily, the third configuration parameter can include one or more of the following: model performance evaluation dimension, one or more model performance evaluation indicators under the model performance evaluation dimension, whether to generate a test report file, output key indicator information, and performance test result output path. For example, by setting the third configuration parameter, the user can decide whether to print the internal structure of the model's reasoning interpreter, and can also decide whether to output the test report file or the selected key indicator information under a certain path.

[0078] It is understandable that the user can set the first configuration parameters, the second configuration parameters and the third configuration parameters at the same time, or can only set some of the first configuration parameters, the second configuration parameters and the third configuration parameters. When the user only sets some of the configuration parameters, the mobile terminal can automatically supplement the remaining configuration parameters according to the default configuration policy, for example, obtain the test configuration parameters based on the user-defined first configuration parameters and the second configuration parameters and the third configuration parameters automatically supplemented by the mobile terminal.

[0079] Figure 2 An example of a model test page is shown. Figure 2 As shown, through the model test page, the user can set the test configuration parameters by entering custom information in the input box. For example, for the number of inference cycles in the second configuration parameter, the user can enter a specified value in the input box, or enter the name of the artificial intelligence model to be tested "XXX model" in the input box.

[0080] Users can also set test configuration parameters by turning on or off the evaluation sub-item switch input, for example, by switching Figure 2 In the "Use GPU proxy" or "Use Xnnpack (a neural network inference library) proxy" evaluation sub-item switch state, set the first configuration parameter, and then switch Figure 2 In "Open OpProfiling analysis", "Output pre-inference status" and "Output post-inference status", you can set the third configuration parameters to control the output content of the performance test results.

[0081] Users can also set test configuration parameters by selecting or entering file path information on the model test page. For example, they can provide the local path of the model file of the artificial intelligence model on the mobile terminal (for example Figure 2 The XXXX directory in the model file), the artificial intelligence model to be tested can be indicated by the local path of the model file, and the evaluation result output file path can be provided to configure the performance test result output path in the third configuration parameter. After completing the artificial intelligence model test, the performance test results can be displayed on the model test page. For example, after the user selects the relevant configuration options and clicks the start button, the baseline performance evaluation of the artificial intelligence model can be triggered, and the evaluation results can be displayed on the model test page in levels.

[0082] In some embodiments, when an artificial intelligence model testing application is deployed on a mobile terminal, the framework structure of the application can be as follows: Figure 3As shown, for the client benchmark evaluation API layer, the JNI (Java NativeInterface) bridge code is encapsulated, and a Java interface is provided for App to quickly access and use. The ModelBenchmarkConfigParam evaluation configuration parameters can be configured before starting the evaluation, and the results after the evaluation are returned as BenchmarkResultSummaryInfo. In this framework structure, after encapsulating the model benchmark evaluation encapsulation layer, it can be installed and compiled through a cross-platform installation and compilation tool. The installation and compilation tool can be configured to support dynamic link libraries (such as SO libraries) and Git Submodule source code dependencies through specific macros. The upper-level access party only needs to connect to the Benchmarker of the interface layer.

[0083] In addition, the application framework structure also includes the TensorFlow underlying evaluation capability layer, which mainly consists of the following modules:

[0084] (1) FlatBufferModel module, Interpreter module and TfLiteDelegatePtr module: They are the basic modules for model reasoning and operation.

[0085] (2) InputLayerInfo module and BenchmarkParams module: used to obtain model input data and model evaluation configuration parameters.

[0086] (3) BufferedProfiler module: used to analyze the running time of operators in the model initialization and inference phases.

[0087] (4) InterpreterStatePrinter model: It is the model reasoning interpreter state analysis module, which analyzes the memory allocation details before and after reasoning, and supports analysis of dimensions such as Subgraph, operator, Node, and Tensor.

[0088] (5) BenchmarkListeners module: Test process callback monitoring module, including monitoring processing at different stages of the evaluation and support for external business monitoring.

[0089] (6) BenchmarkKernelErrorReporter module: Customize the core error reporting module for TFLiteContext to collect information thrown by the underlying inference engine.

[0090] (7) BenchmarkInfoLogCollector module: Integrates all information of the evaluation process and returns the final result through a unified interface. The file name of the final result can be named BenchmarkSummaryInfo.

[0091] In this embodiment, by providing a model testing page and responding to the triggered artificial intelligence model testing operation, one or more of the first configuration parameters and the second configuration parameters are determined, so that the user can freely control the artificial intelligence model initialization process or the model reasoning process when testing the artificial intelligence model on the mobile terminal, which helps to improve the flexibility of the artificial intelligence model when testing the mobile terminal.

[0092] In one embodiment, Figure 4 As shown, in response to an artificial intelligence model test operation triggered by a model test page provided on a mobile terminal, determining a first configuration parameter for setting an artificial intelligence model initialization method, and / or a second configuration parameter for setting a model reasoning method of the artificial intelligence model may include the following steps:

[0093] S401, in response to an artificial intelligence model testing operation triggered by a model testing page provided on a mobile terminal, determining a target inference engine selected by a user for running the artificial intelligence model.

[0094] In actual applications, the inference engines used to run artificial intelligence models may be different. For example, some artificial intelligence models can be run through the CPU, and some artificial intelligence models can be run through the GPU. In this regard, the user can select the target inference engine to run the artificial intelligence model to be tested by performing the artificial intelligence model test operation on the model test page provided by the mobile terminal.

[0095] S402, obtaining a model operation strategy configured for a target inference engine, and determining a first configuration parameter for setting an initialization method of an artificial intelligence model and / or a second configuration parameter for setting a model reasoning method of the artificial intelligence model according to the model operation strategy.

[0096] Different inference engines may have different supporting capabilities or computing effects on different systems. In this regard, the test application can provide corresponding model operation strategies for different inference engines in advance. After determining the target inference engine, the model operation strategy configured for the target inference engine can be obtained, and one or more of the first configuration parameter and the second configuration parameter can be obtained based on the model operation strategy.

[0097] In some embodiments, for the same inference engine, one or more model operation strategies under the inference framework can also be provided. The model operation strategy under each inference framework can be provided by the implementation class corresponding to the inference framework. For example, for the TFL inference framework, the Mnn inference framework and the Onnx inference framework, the following can be set respectively: Figure 3 The TFL_Benchmarker implementation class, Mnn_Benchmarker implementation class, Onnx_Benchmarker implementation class, etc. shown in the figure can provide a model operation strategy that matches the corresponding reasoning framework according to the reasoning engine selected by the user. When testing the artificial intelligence model, the mobile terminal can call the corresponding implementation class interface according to the target reasoning engine, objectify the implementation class through the interface, and obtain a specific implementation subclass. For example, a BenchmarkTfLiteModel subclass can be created, and one or more of the first configuration parameter and the second configuration parameter can be parsed through the subclass to execute the subsequent specific test process.

[0098] In this embodiment, by determining the target inference engine selected by the user and determining one or more of the first configuration parameters and the second configuration parameters according to the model operation strategy configured for the target inference engine, on the one hand, the user can select the inference engine of the artificial intelligence model, which is helpful to test the performance of the artificial intelligence model under different inference engines on the same mobile terminal. On the other hand, by determining one or more of the first configuration parameters and the second configuration parameters according to the model operation strategy, it is possible to reduce the time for users to manually adjust the configuration parameters while taking into account the test flexibility, thereby effectively taking into account the test flexibility and test efficiency of the artificial intelligence model.

[0099] In one embodiment, step S102 may include the following steps: based on the number of inference cycles in the test configuration parameters, controlling the initialized artificial intelligence model to cyclically execute the model inference task corresponding to the number of inference cycles.

[0100] In actual applications, users can set the artificial intelligence model to be tested and the test configuration parameters on the model test page, and then trigger the model test process. In response to the trigger operation, the mobile terminal can parse the test configuration parameters and initialize the artificial intelligence model based on the first configuration parameters obtained by the parsing.

[0101] For example, Figure 5As shown, after the user triggers the start of testing the artificial intelligence model, the mobile terminal can start the Benchmark evaluation, load (LoadMode) the artificial intelligence model to be tested into the mobile terminal memory, and then build the inference interpreter corresponding to the artificial intelligence model through the InitInterpreter function. The inference interpreter is used to perform various data processing tasks in the artificial intelligence model reasoning process. If it is determined according to the first configuration parameter that an acceleration agent is configured, such as turning on the GPU acceleration agent, the model calculation graph can be further replaced with a proxy operator (ModifyGraphWithDelegate).

[0102] Subsequently, the data format corresponding to the model input data can be determined and whether the data format is consistent with the data format Tensor Shape defined by the model. The model input data supports custom input. If the user does not provide it, random data can be generated as the model input data. If the data format is inconsistent, the format of the model input data can also be adjusted during the initialization phase, such as calling the ResizeInputTensor function to adjust the model input data. After that, after successfully allocating tensor memory (AllocateTensors), the executable reasoning state is reached. Finally, a listener for the current model test can be added, such as through the function addOwnerBenchmarkListeners to implement the addition of a listener, so that the subsequent operation process of the artificial intelligence model can be monitored. At this point, the initialization processing of the artificial intelligence model can be completed.

[0103] In addition, the second configuration parameter can be obtained based on the parsed test configuration parameter, and the second configuration parameter can include the number of reasoning cycles. After completing the initialization, the mobile terminal controls the initialized artificial intelligence model to perform model reasoning and executes at least one model reasoning task corresponding to the number of reasoning cycles.

[0104] For example, Figure 5As shown, after initialization is completed, the infrastructure information of the artificial intelligence model can be collected (CollectInterpreterBasicInfo), and then the prepared model input data (PrepareInputData) can be obtained, such as the model input data provided by the user or the randomly generated model input data. After that, in the model inference phase, the artificial intelligence model is preheated and the model inference task is formally executed. In the inference phase, the mobile terminal loops the model inference task according to the number of inference cycles set in the test configuration parameters. Before each model inference task starts, you can first call the ResetInputsAndOutputs function to reset the current model input data and model output results, and then call back the OnSingleRunStart function to notify the start of a single model inference task, and start the inference interpreter Interpreter to complete the model inference task.

[0105] Accordingly, in step S103, obtaining the operating status data of the artificial intelligence model in the model reasoning stage may include the following steps: obtaining the memory status information of the mobile terminal and the operation operator operation information corresponding to the artificial intelligence model each time the artificial intelligence model executes a model reasoning task; and obtaining the operating status data each time the model reasoning task is executed based on the operation operator operation information and the memory status information.

[0106] In the specific implementation, the model initialization stage and the model inference stage can be monitored. For the model initialization stage, one or more of the following information can be monitored: initialization time, memory consumption occupied by initialization, and model infrastructure information after the artificial intelligence model is initialized. For the model inference stage, since the model inference link will process a large number of data processing tasks by multiple operation operators in multiple artificial intelligence models, it will have a significant impact on the overall operation of the artificial intelligence model. Therefore, when monitoring the model inference stage, the memory status information of the model inference link and the operation information of the operation operator corresponding to the artificial intelligence model can be obtained.

[0107] Among them, the memory status information can be information that characterizes the memory usage of the artificial intelligence model. For example, the memory status information can include at least one of the following: the type of memory occupied (AllocationType), the occupied time, the memory usage of various tensors in each Subgraph, the operation operator nodes that occupy more memory (such as the K operation operators that occupy the most memory), and the overall memory usage of the Subgraph. The operation operator operation information can be information that characterizes the operation status of the operation operator inside the artificial intelligence model in the reasoning link.

[0108] When executing model reasoning tasks according to the number of reasoning cycles, this embodiment can monitor each model reasoning task, determine the memory status information and operation operator running information of the artificial intelligence model when executing the current model reasoning task, and obtain the running status data corresponding to each model reasoning task based on the memory status information and the operation operator running information.

[0109] For example, Figure 5 As shown in the figure, after the OnSingleRunStart function is called back to notify the start of a single inference, the inference interpreter is started and the inference is completed, the mobile terminal can collect the time-consuming information and memory occupied by the single inference, and then call back OnSingleRunEnd to notify the end of the single inference, and determine whether the number of inference cycles has been reached. If so, the corresponding running data of all model inference tasks are integrated, if not, the next model inference task is continued.

[0110] In some embodiments, Figure 5 As shown in the figure, the model running process can be monitored through the BenchmarkListener interface. BenchmarkListener is a monitoring callback API (Application ProgrammingInterface) that can be used to notify the start and end of the overall evaluation and the start and end of each model reasoning task execution. The monitoring modules contained in it can have one or more of the following: LoggingListener log output module; ProfilingListener operator analysis module; InterpreterStatePrinter memory state analysis module; OutputSaverListener model running result output. Of course, it can also support external registration of its own monitoring and customize the processing of various evaluation events. Among them, the ProfilingListener module is used to monitor the operation operator running information, the InterpreterStatePrinter module can monitor the memory state information of the model initialization phase and the model reasoning phase, the LoggingListener log output module can be used to generate and output the monitoring log, and the OutputSaverListener module can monitor and output the model running results obtained by the model reasoning task execution.

[0111] In this embodiment, on the one hand, the number of task executions of the artificial intelligence model can be conveniently controlled by setting the number of inference cycles, which helps to determine the performance of the artificial intelligence model during multiple task executions, reduce the impact of occasional errors, and provide more powerful data support for the performance of the artificial intelligence model. On the other hand, by using the operation operator running information and memory status information as operating status data, the key factors closely related to the reasoning performance of the artificial intelligence model can be accurately identified, which helps to accurately identify the performance of the artificial intelligence model on the mobile terminal.

[0112] In one embodiment, obtaining the memory status information of the mobile terminal each time the artificial intelligence model performs a model reasoning task may include the following steps:

[0113] Each time the artificial intelligence model executes a model reasoning task, the inference interpreter corresponding to the artificial intelligence model on the mobile terminal is monitored; when the inference start signal is monitored from the inference interpreter, the current first memory state information of the inference interpreter is obtained; when the inference end signal is monitored from the inference interpreter, the current second memory state information of the inference interpreter is obtained; based on the first memory state information and the second memory state information, the memory state information of the artificial intelligence model each time it executes a model reasoning task is obtained.

[0114] In a specific implementation, since the artificial intelligence model can be provided with a corresponding reasoning interpreter to perform multiple data processing tasks in the model reasoning process during operation, the performance of the artificial intelligence model in the reasoning process can be obtained in a timely manner by monitoring the reasoning interpreter. In this embodiment, each time the artificial intelligence model performs a model reasoning task, the reasoning interpreter corresponding to the artificial intelligence model on the mobile terminal can be monitored.

[0115] Each time a model reasoning task is executed, when an inference start signal is heard from the inference interpreter, it can be determined that the inference interpreter is about to perform a data processing task related to model reasoning. At this time, the current memory state information of the inference interpreter can be obtained. For the sake of distinction, the memory state information recorded when the inference start signal is received is referred to as the first memory state information. On the other hand, when an inference end signal is heard from the inference interpreter, it can be determined that the inference interpreter is about to end the data processing task related to model reasoning. At this time, the current memory state information of the inference interpreter can be obtained. For the sake of distinction, the memory state information collected when the inference end signal is received is referred to as the second memory state information. Then, by comparing the first memory state information and the second memory state information, the memory state information corresponding to the inference link in the model operation process can be determined.

[0116] In some exemplary embodiments, Figure 5 and Figure 6As shown, the mobile terminal can obtain memory status information through the InterpreterStatePrinter model interpreter state analysis module. After completing the preparations before inference, you can call back the OnBenchmarkStart function to notify the start of the monitoring evaluation, and after the inference is completed, call back OnBenchmarkEnd to notify the end of the overall evaluation. When the InterpreterStatePrinter module listens to OnBenchmarkStart and OnBenchmarkEnd, it can determine the start and end of inference respectively, and then obtain the first memory state information before preinvoke_state starts the inference and the second memory state information when postinvoke_state ends the inference execution. Among them, the memory state information in the inference interpreter can be obtained through the ModelTensorMemoryInfo function and the MemoryArenaInfo function.

[0117] In this embodiment, by obtaining the memory status information of the inference interpreter before the start and end of inference, the memory occupancy of the artificial intelligence model each time it processes a model inference task can be quickly and accurately identified.

[0118] In one embodiment, obtaining the operation operator running information corresponding to the artificial intelligence model each time the artificial intelligence model performs a model reasoning task may include the following steps:

[0119] Determine the operation operator that is called each time the artificial intelligence model executes a model reasoning task, and obtain at least one of the operator type, operator calling order, and operator calling times corresponding to the operation operator; determine the operation operator running information based on at least one of the operator type, operator calling order, and operator calling times.

[0120] In the specific implementation, the artificial intelligence model includes multiple operators, which are the basic units for realizing model functions, performing data processing and computing tasks. Operators are used to perform specific operations or calculations on input data and generate corresponding outputs. In some exemplary embodiments, operators such as convolution kernels can be used to extract image features; Sobel operators can be used for edge detection, and operators such as activation functions are used to introduce nonlinear characteristics, enabling the network to learn and approximate complex functions. Operators have a direct and significant impact on model performance. For example, efficient matrix multiplication can improve model calculation efficiency, while inefficient or unnecessary operators will increase computational overhead.

[0121] In this regard, in this embodiment, the operation operator called each time the artificial intelligence model executes a model reasoning task can be determined, and then at least one of the information including the operator type, operator calling order and operator calling number can be obtained to obtain the operation operator running information.

[0122] In some exemplary embodiments, Figure 7 As shown, the ProfilingListener operator operation analysis can be used to obtain the operation operator operation information. Among them, the ProfilingListener module inherits from BenchmarkListener, and is internally associated with the Interpret inference interpreter. The operation operator operation information can be obtained by monitoring the inference interpreter. The ProfilingListener module can include the BufferedProfiler module, as well as the two ProfileSummarizer modules init_summarizer_&run_summarizer_. The two ProfileSummarizer modules are used to collect the operation information of the initialization link and the inference link respectively. The BufferedProfiler module can collect internal information from the inference interpreter through the Interpreter::SetProfiler interface. In the event callbacks received from different stages of BenchmarkListener, the ProfilingListener module calls the Profiler capability to collect the operation information of the current stage. The operation operator operation information finally collected can be sorted and integrated according to any of the following methods: Run Order execution order; Computation Time; Times Called operator call times; Node Type operator type.

[0123] In this embodiment, by obtaining the operation operator running information based on at least one of the operator type, operator calling order and operator calling number, the operation status of the artificial intelligence model in the mobile terminal can be accurately determined based on various operation information related to the operation operator, which helps to quickly identify performance bottlenecks in the model, and provides an important basis for model debugging and fault diagnosis, so as to obtain reliable and useful performance test results.

[0124] In one embodiment, step S104 may include the following steps:

[0125] Perform statistical analysis on the operation data according to one or more preset model performance evaluation dimensions to determine the indicator information of at least one model performance indicator under each model performance evaluation dimension; generate a test report file with the indicator information of at least one model performance indicator, and / or select key indicator information from the indicator information of at least one model performance indicator; output the performance test results of the artificial intelligence model on the mobile terminal, the performance test results including the test report file and / or the key indicator information.

[0126] Among them, the model performance index can be an index that can reflect the performance of the artificial intelligence model on the mobile terminal. In some examples, the model performance index can be multiple indicators related to time consumption, number of calls, and memory usage; the model performance index can be an indicator set for one or more evaluation dimensions, for example, it can be any one or more of the following dimensions: artificial intelligence model, subgraph, node, tensor.

[0127] In practical applications, after obtaining the operation data, the obtained operation data can be statistically analyzed from one or more model performance evaluation dimensions to determine the performance of the artificial intelligence model under specific dimensions. Specifically, one or more model performance evaluation dimensions can be determined, and one or more model performance indicators can be determined under each model performance evaluation dimension, and then the operation data can be statistically analyzed to obtain the indicator information of each model performance indicator. Exemplarily, the model performance indicators may include one or more of the following: model file size, model initialization interpretation time, model infrastructure information, reasoning execution before start-up state, reasoning execution after start-up state, model contains subgraphs, specified subgraphs contain Tensor tensors and Node nodes, tensor information of various types of memory, the tensor with the highest memory usage under the specified type of memory, the total memory consumption of the specified Subgraph subgraph, the overall average reasoning time of the model, and the Node nodes contained in the specified Subgraph subgraph operation.

[0128] Then, a test report file can be generated, which can include indicator information of one or more model performance indicators, for example, all determined indicator information. In addition, key indicator information to be directly displayed can also be selected from the indicator information of multiple model performance indicators. Afterwards, the performance test results of the artificial intelligence model on the mobile terminal can be obtained and output based on one or more of the test report file and the key indicator information.

[0129] In one embodiment, Figure 8As shown in the figure, the mobile terminal can process the obtained operation status data through the BenchmarkInfoLogCollector information collection module. Specifically, after the BenchmarkTfLiteModel is created, it can collect and integrate all key information in the evaluation process from each evaluation sub-module (including all BenchmarkListener listening objects) in the artificial intelligence model test application, obtain the indicator information under multiple model performance indicators, and then obtain the kernel_stream and whole_stream two output streams. When the test report file only records part of the indicator information, the two output streams can be Figure 8 The three objects pre-encapsulated in the get the specified indicator information. Among them, the BenchmarkInfoKernelStream object can be used to filter out the indicator information that is only the key indicator information from multiple indicator information, BenchmarkinfoKernelWithWholeStream can be used to determine the indicator information that is also the key indicator information and is recorded in the test report file from multiple indicator information; BenchmarkinfoWholeStream can be used to determine the indicator information that is only recorded in the test report file from multiple indicator information. The three objects have corresponding C++ macros, which facilitates each evaluation submodule to collect data through BenchmarkInfoLogCollector. After the evaluation is completed, all the information collected by the internal output stream will be output to the externally configured output_kernel_filepath (core information) and output_whole_filepath (all detailed information) local files.

[0130] After obtaining the test report file and key indicator information, on the one hand, the test report file can be stored locally on the mobile terminal for reference and subsequent upload, and on the other hand, the key indicator information can be displayed on the model test page, so that users can quickly and timely understand the performance of the artificial intelligence model on the current mobile terminal.

[0131] In addition, in some other embodiments, the test report file and key indicator information can both be stored in BenchmarkResultSummaryInfo, mainly including core output information (i.e., key indicator information) and complete output information (i.e., test report file), and their corresponding local information storage file paths can be displayed on the test page.

[0132] In order to enable those skilled in the art to better understand the above steps, Figure 3 The embodiments of the present application are described by way of example only, but it should be understood that the embodiments of the present application are not limited thereto.

[0133] In a specific implementation, in response to an artificial intelligence model testing operation triggered by a model testing page provided on a mobile terminal, the artificial intelligence model to be tested can be determined, the corresponding model file can be obtained, and the test configuration parameters set by the user can be obtained.

[0134] Then you can start the Benchmark evaluation. After creating the BenchmarkTfLiteModel, first parse and verify the test configuration parameters. After the verification is completed, initialize the artificial intelligence model according to the model running process, and obtain the time and memory consumption of the initialization phase.

[0135] After initialization, we first collect the basic structure information of the model, and then prepare the model input data. At this point, the preparations before the benchmark inference run are complete, and then we can call back OnBenchmarkStart to notify the monitoring evaluation to start.

[0136] The model inference phase includes warmup operation and formal inference. It executes multiple model inference tasks according to the number of thrust cycles set by the user, and obtains the operation data corresponding to each model inference task. After the inference is completed, OnBenchmarkEnd is called back to notify the end of the overall evaluation. The final evaluation results are packaged in BenchmarkSummaryInfo and returned.

[0137] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0138] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig. 9As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, an artificial intelligence model testing method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0139] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0140] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0141] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0142] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0144] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0146] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An artificial intelligence model testing method, characterized in that: Applied to a mobile terminal, the method comprises: In response to an artificial intelligence model test operation triggered by a model test page provided on the mobile terminal, determining an artificial intelligence model to be tested and test configuration parameters; Initializing the artificial intelligence model on the mobile terminal based on the test configuration parameters and performing model reasoning using the initialized artificial intelligence model; Monitoring the operation process of the artificial intelligence model to obtain operation status data of the artificial intelligence model in the model initialization stage and the model inference stage; Based on the operating status data, the performance test results of the artificial intelligence model on the mobile terminal are output.

2. The method according to claim 1, characterized in that In response to an artificial intelligence model test operation triggered by a model test page provided on a mobile terminal, test configuration parameters are determined, including: In response to an artificial intelligence model test operation triggered by a model test page provided on a mobile terminal, determining a first configuration parameter for setting an initialization method of the artificial intelligence model, and / or a second configuration parameter for setting a model reasoning method of the artificial intelligence model; Based on the first configuration parameter and / or the second configuration parameter, a test configuration parameter is acquired.

3. The method according to claim 2, characterized in that The step of determining, in response to an artificial intelligence model test operation triggered by a model test page provided on a mobile terminal, a first configuration parameter for setting an initialization method of the artificial intelligence model, and / or a second configuration parameter for setting a model reasoning method of the artificial intelligence model, includes: In response to an artificial intelligence model test operation triggered by a model test page provided on the mobile terminal, determining a target inference engine selected by a user for running the artificial intelligence model; Obtain a model operation strategy configured for the target inference engine, and determine, based on the model operation strategy, a first configuration parameter for setting an initialization method for the artificial intelligence model, and / or a second configuration parameter for setting a model reasoning method for the artificial intelligence model.

4. The method according to claim 1, characterized in that Based on the test configuration parameters, the initialized artificial intelligence model is used to perform model reasoning, including: Based on the number of inference cycles in the test configuration parameters, controlling the initialized artificial intelligence model to cyclically execute the model inference task corresponding to the number of inference cycles; Obtaining the operation data of the artificial intelligence model in the model reasoning stage, including: Acquire the memory state information of the mobile terminal and the operation operator running information corresponding to the artificial intelligence model each time the artificial intelligence model executes the model reasoning task; According to the operation operator running information and the memory status information, the running status data when the model reasoning task is executed each time is obtained.

5. The method according to claim 4, characterized in that Acquiring the memory status information of the mobile terminal each time the artificial intelligence model performs the model reasoning task, including: Each time the artificial intelligence model executes the model reasoning task, monitoring the reasoning interpreter corresponding to the artificial intelligence model on the mobile terminal; When an inference start signal is heard from the inference interpreter, the current first memory state information of the inference interpreter is obtained; when an inference end signal is heard from the inference interpreter, the current second memory state information of the inference interpreter is obtained; The memory state information of the artificial intelligence model each time executing the model reasoning task is obtained according to the first memory state information and the second memory state information.

6. The method according to claim 4, characterized in that Obtaining the operation operator running information corresponding to the artificial intelligence model each time the artificial intelligence model executes the model reasoning task, including: Determine an operation operator that is called each time the artificial intelligence model executes the model reasoning task, and obtain at least one of an operator type, an operator calling order, and an operator calling number corresponding to the operation operator; The operation operator running information is determined according to at least one of the operator type, the operator calling order and the operator calling number.

7. The method according to any one of claims 1 to 6, characterized in that Outputting the performance test result of the artificial intelligence model on the mobile terminal according to the operation status data includes: Performing statistical analysis on the operation status data according to one or more preset model performance evaluation dimensions to determine indicator information of at least one model performance indicator under each of the model performance evaluation dimensions; generating a test report file from the indicator information of the at least one model performance indicator, and / or selecting key indicator information from the indicator information of the at least one model performance indicator; Output the performance test result of the artificial intelligence model on the mobile terminal, and the performance test result includes the test report file and / or the key indicator information.

8. A mobile terminal, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images