Backend testing method, apparatus, storage medium, and computer device

Abstract

Embodiments of the present application provide a backend test method and device, a storage medium and a computer device. The method comprises: obtaining a parameter data packet, a network model file and test data; generating an inference result according to the parameter data packet, the network model file and the test data; generating baseline data according to the network model file and the test data; generating a backend test result according to the baseline data and the inference result; and verifying the numerical stability of the trained model on different backends in real time according to the backend accessed in the process of model training. Thus, the numerical stability of the backend inference can be quickly tested; different backends can be accessed to achieve the purpose of testing software simulation hardware, principle verification hardware or real backend hardware in real time; the performance of an existing backend on different network models can also be quickly verified; and through the backend test result, a technical personnel can easily find logical design defects of the backend, providing a guidance direction for later modification and reducing the research and development cycle.

Description

[Technical Field]

[0001] The embodiments of the present invention relate to the field of testing technology, specifically to a backend testing method, apparatus, storage medium, and computer device. [Background Technology]

[0002] In the design of backend systems such as Neural-network Processing Units (NPUs), Tensor Processing Units (TPUs), and Deep Learning Processing Units (DPUs), the hardware design cycle is relatively long. During user experience testing after the backend logic design is completed, design flaws in the backend can lead to significant accuracy errors in the network model's inference results, further extending the development cycle. [Summary of the Invention]

[0003] In view of this, embodiments of the present invention provide a backend testing method, apparatus, storage medium, and computer device to solve the problem that design flaws in the backend of the prior art can lead to large accuracy errors in the inference results of network models, thus increasing the development cycle.

[0004] In a first aspect, embodiments of the present invention provide a backend testing method, including:

[0005] Obtain parameter data packets, network model files, and test data;

[0006] Inference results are generated based on the parameter data packet, the network model file, and the test data.

[0007] Baseline data is generated based on the network model file and the test data;

[0008] Backend test results are generated based on the baseline data and the inference results.

[0009] In one possible implementation, the parameter data packet includes first test environment parameters, and the step of generating inference results based on the parameter data packet, the network model file, and the test data includes:

[0010] In the first backend environment corresponding to the first test environment parameters, a first inference graph is generated based on the network model file;

[0011] Determine whether to update the first inference graph based on the parameter data packet;

[0012] If it is determined that the first inference graph needs to be updated, then a second inference graph is generated based on the parameter data packet and the first inference graph;

[0013] A first execution queue is generated based on the second inference graph;

[0014] The inference result is generated based on the test data and the first execution queue.

[0015] In one possible implementation, determining whether to update the first inference graph based on the parameter data packet includes:

[0016] Determine whether the parameter data packet includes a matching backend replacement identifier;

[0017] If it is determined that the parameter data packet includes the matching backend replacement identifier, then the first inference graph is updated.

[0018] In one possible implementation, the parameter data packet includes second test environment parameters, and the step of generating inference results based on the parameter data packet, the network model file, and the test data includes:

[0019] In the second backend environment corresponding to the second test environment parameters, a third inference graph is generated based on the network model file;

[0020] Determine whether to update the third inference graph based on the parameter data packet;

[0021] If it is determined that the third inference graph needs to be updated, then a fourth inference graph is generated based on the parameter data packet and the third inference graph.

[0022] Generate a first sequence and a first execution master file based on the fourth reasoning graph;

[0023] The inference result is generated based on the first sequence, the first execution master file, and the test data.

[0024] In one possible implementation, the fourth inference graph includes at least one subgraph, and the first total execution file includes at least one first execution file; the step of generating the first sequence and the first total execution file based on the fourth inference graph includes:

[0025] Compile at least one of the subgraphs to generate a first executable file corresponding to each subgraph;

[0026] The first order is generated according to the order in which the subgraphs are compiled.

[0027] In one possible implementation, the parameter data packet includes third test environment parameters, and the step of generating inference results based on the parameter data packet, the network model file, and the test data includes:

[0028] In the third backend environment corresponding to the third test environment parameters, inference results are generated based on the network model file and the test data.

[0029] In one possible implementation, the parameter data packet includes a fourth test environment parameter, and the step of generating inference results based on the parameter data packet, the network model file, and the test data includes:

[0030] In the fourth backend environment corresponding to the fourth test environment parameter, obtain the backend configuration parameter file;

[0031] A second execution master file is generated based on the backend configuration parameter file and the network model file;

[0032] The inference result is generated based on the second execution master file and the test data.

[0033] Secondly, embodiments of the present invention provide a backend testing device, comprising:

[0034] The acquisition module is used to acquire parameter data packets, network model files, and test data.

[0035] The first generation module is used to generate inference results based on the parameter data packet, the network model file, and the test data;

[0036] The second generation module is used to generate baseline data based on the network model file and the test data;

[0037] The third generation module is used to generate backend test results based on the baseline data and the inference results.

[0038] Thirdly, embodiments of the present invention provide a storage medium including a stored program, wherein, when the program is running, it controls the device where the storage medium is located to execute the backend testing method in the first aspect or any possible implementation of the first aspect.

[0039] Fourthly, embodiments of the present invention provide a computer device, including a memory and a processor. The memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions. When the program instructions are loaded and executed by the processor, they implement the steps of the back-end testing method in the first aspect or any possible implementation of the first aspect.

[0040] This invention provides a backend testing method, apparatus, storage medium, and computer device. The method involves acquiring parameter data packets, network model files, and test data; generating inference results based on the parameter data packets, network model files, and test data; generating baseline data based on the network model files and test data; generating backend test results based on the baseline data and inference results; and using the computer device to connect to the backend during model training to verify the numerical stability of the trained model on different backends in real time. This allows for rapid testing of the stability of backend inference values; decoupling of backend devices; access to backends at different stages to achieve real-time testing of software simulation hardware, principle verification hardware, or real backend hardware; rapid verification of the performance of existing backends on different network models; and backend test results that allow engineers to easily identify logical design flaws in the backend, providing guidance for later modifications and reducing the development cycle. [Attached Image Description]

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart of a backend testing method provided in an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of a testing framework provided in an embodiment of the present invention;

[0044] Figure 3 A flowchart for generating reasoning results is provided as an embodiment of the present invention;

[0045] Figure 4 A schematic diagram of a test framework for a software-simulated backend online processing pipeline provided in an embodiment of the present invention;

[0046] Figure 5 Another flowchart for generating reasoning results provided in this embodiment of the invention;

[0047] Figure 6 A schematic diagram of a test framework for a software-simulated offline processing pipeline provided in an embodiment of the present invention;

[0048] Figure 7 This is a schematic diagram of a test framework for a real backend online processing pipeline provided in an embodiment of the present invention;

[0049] Figure 8 Another flowchart for generating reasoning results provided in this embodiment of the invention;

[0050] Figure 9 A schematic diagram of a test framework for a real backend offline processing pipeline provided in an embodiment of the present invention;

[0051] Figure 10 This is a schematic diagram of the structure of a backend testing device provided in an embodiment of the present invention;

[0052] Figure 11 This is a schematic diagram of a computer device provided in an embodiment of the present invention.

Detailed Implementation Methods

[0053] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0054] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0055] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0056] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0057] It should be understood that although terms such as first, second, third, etc., may be used to describe numbers in embodiments of the present invention, these numbers should not be limited to these terms. These terms are only used to distinguish numbers from each other. For example, without departing from the scope of embodiments of the present invention, a first number may also be referred to as a second number, and similarly, a second number may also be referred to as a first number.

[0058] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0059] Figure 1 A flowchart of a backend testing method provided in an embodiment of the present invention is shown below. Figure 1 As shown, the method includes:

[0060] Step 101: The computer device acquires the parameter data packet, network model file, and test data.

[0061] In this embodiment of the invention, the computer device includes a server, or a combination of a server and electronic devices. The server includes a prototype of a back-end chip, a Field-Programmable Gate Array (FPGA) verification board for the back-end chip, a prototype development board for the back-end chip, etc. The server can also be a device with simulation software for the back-end chip. The back-end chip includes, but is not limited to, any combination of one or more chips such as NPU chips, TPU chips, and DPU chips. The electronic devices include, but are not limited to, mobile phones, tablets, portable PCs, desktop computers, wearable devices, etc.

[0062] The process includes, prior to step 101: the user equipment acquiring a network model file and test data; generating a parameter data packet in response to at least one setting parameter input by the user; and sending the parameter data packet, network model file, and test data to the computer equipment. Step 101 includes: the computer equipment receiving the parameter data packet, network model file, and test data sent by the user equipment.

[0063] The user equipment in this embodiment of the invention includes, but is not limited to, mobile phones, tablets, portable PCs, desktop computers, wearable devices, etc. The user equipment displays at least one setting parameter, including test environment parameters. The test environment parameters include backend setting parameters and compilation environment parameters. The backend setting parameters include software simulation backend parameters or real backend parameters; the compilation environment parameters include online compilation parameters or offline compilation parameters. Users can select at least one setting parameter on the user equipment through drag-and-drop, input, or other methods. The user equipment receives network model files and test data uploaded by the user.

[0064] Figure 2 This is a schematic diagram of a testing framework provided in an embodiment of the present invention. This testing framework is a general framework that allows for rapid verification, summarization, and analysis of the numerical stability and inference performance data of different backends when performing inference accuracy and performance analysis tests on existing backend hardware. For example... Figure 2As shown, this testing framework can be divided into six modules. Each module can add different test process components based on different backends and test modes. This allows for testing inference accuracy for backends at different design stages, obtaining the numerical stability of the entire model under multi-backend inference. The UX module represents the user device; the computer device includes a Performance Analyzer, a Prepare NN Model module, an Execution Scheduler module, a Backends module, and an Invoke Application Programming Interface (API) module. The Performance Analyzer collects and analyzes system operation information; the Prepare NN Model module selects different processing components for the model based on different backends and test modes; the Execution Scheduler module performs inference to obtain model output and baseline output, with different built-in processing flows under different backends and test modes; the Backends module contains different backend components, which need to be specified by the user when selecting settings parameters in the user device's interface; the API module provides API interfaces, supporting the use of APIs during model training to view the model's training progress. Figure 2 As shown, the user equipment is used to interact with the user and send the parameter data packets, network model files and test data input by the user to the computer equipment.

[0065] Step 102: The computer device generates inference results based on the parameter data packet, network model file, and test data.

[0066] In this embodiment of the invention, the computer device encapsulates processing pipelines for different backends, enabling the decoupling of the technical solution framework from the backend based on the user's selection. This allows for the rapid acquisition of the inference results of the entire model across different backends. For example, when the parameter data package includes software simulation backend parameters and online compilation parameters, the computer device uses the online processing pipeline of the software simulation backend to obtain the inference results of the network model on the software simulation backend. When the parameter data package includes software simulation backend parameters and offline compilation parameters, the computer device uses the offline processing pipeline of the software simulation backend to obtain the inference results of the network model on the software simulation backend. When the parameter data package includes real backend parameters and online compilation parameters, the computer device uses the online processing pipeline of the real backend to obtain the inference results of the network model on the real backend. When the parameter data package includes real backend parameters and offline compilation parameters, the computer device uses the offline processing pipeline of the real backend to obtain the inference results of the network model on the real backend.

[0067] Step 103: The computer equipment generates baseline data based on the network model file and test data.

[0068] In this embodiment of the invention, step 103 can be executed simultaneously with step 102, or it can be executed before step 102, thereby achieving the simultaneous development of backend hardware and algorithm design, algorithm model development, and hardware logic development. The computer device processes the pipeline using the determined scheme, and infers on the baseline backend based on the network model file and test data uploaded by the user, thus quickly obtaining baseline data.

[0069] Step 104: The computer equipment generates backend test results based on the baseline data and inference results.

[0070] In this embodiment of the invention, after step 104, the method further includes: the computer device sending the backend test results to the user device. The user device displays the backend test results. The user device may also provide a download function, allowing the user to download and save the backend test results.

[0071] This invention provides a backend testing method that acquires parameter data packets, network model files, and test data; generates inference results based on the parameter data packets, network model files, and test data; generates baseline data based on the network model files and test data; and generates backend test results based on the baseline data and inference results. This allows computer devices to access the hardware backend during model training based on the backend settings parameters included in the parameter data packets, enabling real-time verification of the numerical stability of the trained model on the backend. This allows for rapid testing of the stability of the backend's inference values. It can access backends at different stages, achieving real-time testing of software simulation hardware, principle verification hardware, or real backend hardware, thus decoupling the backend devices. It also allows for rapid verification of the performance of existing backends on different network models. The backend test results enable technicians to easily identify logical design flaws in the backend, providing guidance for later modifications and reducing the development cycle.

[0072] In one possible implementation, the parameter data package includes first test environment parameters. Figure 3 A flowchart for generating reasoning results is provided as an embodiment of the present invention, such as... Figure 3 As shown, step 102 may specifically include:

[0073] Step 1021: The computer device generates a first inference graph based on the network model file in the first backend environment corresponding to the first test environment parameters.

[0074] In this embodiment of the invention, the first test environment parameters include software simulation backend parameters and online compilation parameters. The software simulation backend parameters indicate that the user-specified backend is a virtual backend generated by a computer device using simulation software, and the online compilation parameters indicate that the user-specified compilation environment is an online compilation environment. The computer device configures the first backend environment according to the first test environment parameters. In the first backend environment, the computer device parses the network model file into an inference graph represented by IR. The network model includes a deep learning network model. The computer device includes a server.

[0075] Figure 4 This is a schematic diagram of a test framework for a software-simulated backend online processing pipeline provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the user equipment includes a Web Front End module and a Web Backend module. The Web Front End module includes a select test mode component, a set parameters component, a select operator component, an upload data component, and a display module. The Web Backend module includes a parameter analysis component, an effectiveness check component, and a parameter subcontract component. The computer equipment includes a performance analyzer, a neural network model (NN Model) module, a neural network model preparation module, an execution scheduler module, a simulation backends module, an API module, a CPU backend, and a CPU executor. The performance analyzer includes a user log component and a debug log component. The neural network model module includes TFLite model components, TF PB model components, and ONNX model components. The neural network model preparation module includes a TFLite parser, a TF PB parser, an ONNX parser, a subgraph component, and an OP replacement component. The execution scheduler module includes an assignment backend component, a computer optimizer component, a subgraph queue component, and a result collector. Figure 4The data in the text refers to test data; the simulation backend module includes multiple backend components, such as backend 0, backend 1, backend 2, and backend 3; the API module includes a training model component, a test data component, a set params component, an inference result component, and a simulation converter.

[0076] like Figure 4 As shown, the computer device selects the model component corresponding to the network model file from the TFLite model component, TF PB model component, and ONNX model component based on the network model file. Then, it parses the network model file according to the parser corresponding to the model component and generates a first inference graph in the subgraph component. The first inference graph includes at least one subgraph.

[0077] Step 1022: The computer device determines whether to update the first inference graph based on the parameter data packet. If yes, proceed to step 1023; otherwise, proceed to step 1026.

[0078] In this embodiment of the invention, the computer device determines whether the parameter data packet includes a matching backend replacement identifier; if it determines that the parameter data packet includes a matching backend replacement identifier, then it determines to update the first inference graph.

[0079] Computer devices detect whether users have set backend operator nodes that need to be replaced during user interaction. Backend operators are the basic units of neural network computation, including convolution, pooling, activation, and normalization. A network model can correspond to at least one backend operator. For example, ... Figure 4 As shown, the user sets an equivalent substitution for conv to maxpool by selecting the operator component, and the computer device processes the data according to the equivalent substitution set by the user. If the user sets a matching backend replacement operator by selecting the operator component, the parameter data packet will include the matching backend replacement identifier, thereby enabling the user to specify the execution backend for any backend operator in real time.

[0080] Step 1023: The computer device generates a second inference graph based on the parameter data packet and the first inference graph.

[0081] In this embodiment of the invention, the computer device performs statistical analysis on the backend operators supported by the user-specified backend, and, based on the user-set matching backend replacement identifier, can replace backend operators not supported by the backend with those supported. The first inference graph includes at least one subgraph, and at least one backend operator can correspond to one subgraph; for example, each backend operator can correspond to one subgraph, or multiple backend operators can correspond to one subgraph. Figure 4 As shown, after the computer device replaces the backend operator that is not supported by the backend with the backend operator that is supported by the backend in the OP replacement component, it also needs to update the first inference graph according to the replaced backend operator; and use the updated first inference graph as the second inference graph.

[0082] Step 1024: The computer device generates a first execution queue based on the second inference graph.

[0083] In this embodiment of the invention, the first execution queue has an execution order. Since the second inference graph includes at least one subgraph, and there is a correspondence between the back-end operators and the subgraphs, and a correspondence between the back-end operators and the back-ends, such as... Figure 4 As shown, the computer device maps at least one subgraph to the backend within the computer optimization program component, and then performs the segmentation of the at least one subgraph in the backend; in the backend, it calls the corresponding compiler to compile the subgraph; and then saves the compiled subgraph to the subgraph queue component and saves the execution order of the subgraph.

[0084] Step 1025: The computer device generates inference results based on the test data and the first execution queue.

[0085] In this embodiment of the invention, the computer device reads the test data and, according to the execution order of the first execution queue, sequentially calls the backend corresponding to the subgraph to complete the subgraph inference, and finally obtains the inference result of the network model, thereby obtaining the stability result of the model value when running on a specific backend after replacing the backend operator node.

[0086] Step 1026: The computer device generates a second execution queue based on the first inference graph.

[0087] In this embodiment of the invention, no backend operator node needs to be replaced when the user interacts with the user device.

[0088] Step 1027: The computer device generates inference results based on the test data and the second execution queue.

[0089] In this embodiment of the invention, steps 1026 to 1027 can be referred to the above. Figure 3 Steps 1024 to 1025 are shown.

[0090] In one possible implementation, the parameter data package includes second test environment parameters. Figure 5 Another flowchart for generating reasoning results provided in an embodiment of the present invention, such as Figure 5 As shown, step 102 may specifically include:

[0091] Step 102A: The computer device generates a third inference graph based on the network model file in the second backend environment corresponding to the second test environment parameters.

[0092] In this embodiment of the invention, the second test environment parameters include software simulation backend parameters and offline compilation parameters. The software simulation backend parameters indicate that the user-specified backend is a virtual backend generated by a computer device using simulation software, and the offline compilation parameters indicate that the user-specified compilation environment is an offline compilation environment. In the second backend environment, the computer device includes a server. Figure 6 This is a schematic diagram of a test framework for a software-simulated offline processing pipeline provided in an embodiment of the present invention, as shown below. Figure 6 As shown, Figure 6 The user equipment shown is Figure 4 The user equipment shown is the same. Figure 6 The execution scheduler module of the computer device shown is Figure 5 The execution scheduler module of the computer device shown differs from the one described above. The execution scheduler module includes an allocation backend component, a computer optimization component, an offline scheduler component, and a result collector. The network model includes a deep learning network model, and the network model file includes a deep learning network model file. The computer device parses the deep learning network model file into a third inference graph represented by IR in the subgraph component.

[0093] Step 102B: The computer device determines whether to update the third inference graph based on the parameter data packet. If yes, proceed to step 102C; otherwise, proceed to step 102F.

[0094] Step 102C: The computer device generates a fourth inference graph based on the parameter data packet and the third inference graph.

[0095] In this embodiment of the invention, Figure 5 Steps 102B to 102C shown can be referenced. Figure 3 Steps 1022 to 1023 are shown.

[0096] Step 102D: The computer device generates the first sequence and the first execution master file according to the fourth reasoning diagram.

[0097] In this embodiment of the invention, the fourth inference graph includes at least one subgraph, and the first execution file includes at least one first execution file; the computer device compiles at least one subgraph to generate a first execution file corresponding to each subgraph; and a first sequence is generated according to the order in which the subgraphs are compiled.

[0098] like Figure 6 As shown, the computer device, within its computer optimization program component, performs a backend segmentation of at least one subgraph, mapping each subgraph to a backend. In the backend, the corresponding compiler is invoked to compile the subgraphs, generating a first executable file for each subgraph. The computer device maintains a compilation order during subgraph compilation; it records this order and, after the last subgraph is compiled, uses the compilation order of at least one subgraph as the first order. The computer device then stores the first executable file and the first order on its hard drive.

[0099] Step 102E: The computer device generates inference results based on the first sequence, the first executable file, and the test data.

[0100] In this embodiment of the invention, the first executable file includes a binary file. The computer device enters the offline scheduling execution inference stage. The computer device first configures the execution order of the offline backend executable binary files according to the first order; then it reads in the test data and calls the execution backends corresponding to the subgraphs in the first order to complete the subgraph inference; finally, it obtains the inference result of the network model.

[0101] Step 102F: The computer device generates the second sequence and the third execution master file based on the third inference graph.

[0102] Step 102G: The computer device generates inference results based on the second sequence and the third execution of the master file and test data.

[0103] In this embodiment of the invention, steps 102F to 102G can be referred to Figure 5 Steps 102D to 102E are shown.

[0104] In one possible implementation, the parameter data package includes third test environment parameters, and step 102 may specifically include: the computer device generating inference results based on the network model file and test data in the third backend environment corresponding to the third test environment parameters.

[0105] In this embodiment of the invention, the third test environment parameters include real backend parameters and online compilation parameters. The real backend parameters indicate that the user-specified backend is the actual backend hardware of the computer device, and the online compilation parameters indicate that the user-specified compilation environment is an online compilation environment. The computer device includes servers and electronic devices. Figure 7This is a schematic diagram of a test framework for a real backend online processing pipeline provided in an embodiment of the present invention, as shown below. Figure 7 As shown, Figure 7 The user equipment shown is Figure 4 or Figure 6 Compared to the user device shown, the web front-end module of the user device does not include a selection operator component. For example... Figure 7 As shown, the computer device includes a performance analyzer, a neural network model module, an allocation backend component, an ADB tunnel, a result collector, a hardware backend module, and an API module. Figure 7 The performance analyzer and neural network model module shown are respectively with Figure 4 or Figure 6 The performance analyzer shown is the same as the neural network model module. The hardware backend module includes multiple backend components, such as backend 0, backend 1, backend 2, and backend 3; the API module includes a training model component, a test data component, a parameter setting component, an inference result component, and a hardware converter.

[0106] The server uses ADB tools to push network model files and test data uploaded by user devices to electronic devices. The electronic devices, including real backend hardware, perform online compilation and inference on the network model files and test data, generating inference results. The electronic devices then transmit the inference results to the server. The server outputs the inference results to the result collector and executes them. Figure 1 Steps 103 and 104 are shown.

[0107] In one possible implementation, the parameter data package includes fourth test environment parameters. Figure 8 Another flowchart for generating reasoning results provided in an embodiment of the present invention, such as Figure 8 As shown, step 102 may specifically include:

[0108] Step 102a: The computer device obtains the backend configuration parameter file in the fourth backend environment corresponding to the fourth test environment parameter.

[0109] In this embodiment of the invention, the backend configuration parameter file includes at least one preset configuration parameter for the backend. The fourth test environment parameters include real backend parameters and offline compilation parameters, wherein the real backend parameters indicate that the user-specified backend is the actual backend hardware of the computer device, and the offline compilation parameters indicate that the user-specified compilation environment is an offline compilation environment. In the fourth backend environment, the computer device includes servers and electronic devices.

[0110] Figure 9 This is a schematic diagram of a test framework for a real backend offline processing pipeline provided in an embodiment of the present invention, as shown below. Figure 9As shown, Figure 9 The user equipment shown is Figure 7 The user equipment shown is the same, and the computer equipment includes a performance analyzer, a neural network model module, a neural network model preparation module, a distribution backend component, an ADB tunnel, a result collector, a hardware backend module, and an API module. Figure 9 The performance analyzer, neural network model module, hardware backend module, and API module shown are respectively related to... Figure 7 The performance analyzer, neural network model module, hardware backend module, and API module shown are the same. Preparing the neural network model module includes an ADB tunnel, a Native Development Kit (NDK) component, and configuration files.

[0111] like Figure 9 As shown, after receiving the network model file, the neural network model preparation module calls the corresponding NDK toolkit on the backend to read the preset configuration parameters of the specified backend.

[0112] Step 102b: The computer device generates a second executable file based on the backend configuration parameter file and the network model file.

[0113] In this embodiment of the invention, the second executable file includes at least one second executable file, and the second executable file includes a binary file.

[0114] like Figure 9 As shown, the neural network model module performs offline compilation of the network model file, compiling the network model file into at least one executable binary file.

[0115] Step 102c: The computer device generates inference results based on the second execution master file and test data.

[0116] In this embodiment of the invention, the server pushes the second execution master file and test data to the electronic device via an ADB tool. The electronic device includes real backend hardware. The electronic device generates inference results based on the second execution master file and test data, and transmits the inference results to the server. The server outputs the inference results to the result collector and executes them. Figure 1Steps 103 and 104 shown enable the simultaneous development of backend hardware and algorithm design, algorithm model development, and hardware logic development. Backend test results allow technicians to easily identify logic design defects and provide auxiliary and guiding modification schemes for logic designers, reducing the occurrence of large accuracy errors in network model inference results due to backend design defects. It can collect the backend performance in the inference process in real time, analyze and summarize the numerical deviations of each layer in the inference process, and analyze the performance of each type of backend operator.

[0117] This invention provides a method for obtaining parameter data packets, network model files, and test data; generating inference results based on the parameter data packets, network model files, and test data; generating baseline data based on the network model files and test data; and generating backend test results based on the baseline data and inference results. This allows for real-time collection of backend performance during inference, analysis and summarization of computational deviations at each layer, and analysis of the performance of each type of backend operator. It enables real-time acquisition of backend operator test results, allowing for the determination of the gap between the backend operator's output accuracy and the baseline numerical accuracy. Computer equipment is used to evaluate the inference performance of the current backend operator design, providing guidance for modifications during backend operator design. When designers develop deep learning solutions on third-party backends, it provides a method for quickly aligning baseline accuracy, achieving decoupling of backend devices. It enables rapid testing of network model inference accuracy using a single framework. It provides comprehensive support for hardware, software, and algorithm designers at different stages, covering logic design, backend operator design, software stack design, and algorithm design, meeting the development needs of designers in different roles. Furthermore, it allows some later-stage work, such as algorithm and software stack design, to be moved to the same stage as logic design and backend operator design, significantly reducing the development cycle and improving efficiency.

[0118] Figure 10 This is a schematic diagram of a backend testing device provided in an embodiment of the present invention, as shown below. Figure 10 As shown, the device includes: an acquisition module 11, a first generation module 12, a second generation module 13, and a third generation module 14. The acquisition module 11 is connected to the first generation module 12, the first generation module 12 is connected to the second generation module 13, and the second generation module 13 is connected to the third generation module 14.

[0119] The acquisition module 11 is used to acquire parameter data packets, network model files, and test data; the first generation module 12 is used to generate inference results based on the parameter data packets, network model files, and test data; the second generation module 13 is used to generate baseline data based on the network model files and test data; and the third generation module 14 is used to generate backend test results based on the baseline data and inference results.

[0120] In this embodiment of the invention, the parameter data package includes first test environment parameters. The first generation module 12 is specifically used to generate a first inference graph based on a network model file in a first backend environment corresponding to the first test environment parameters; determine whether to update the first inference graph based on the parameter data package; if it is determined that the first inference graph should be updated, generate a second inference graph based on the parameter data package and the first inference graph; generate a first execution queue based on the second inference graph; and generate an inference result based on the test data and the first execution queue.

[0121] In this embodiment of the invention, the first generation module 12 is specifically used to determine whether the parameter data packet includes a matching backend replacement identifier; if it is determined that the parameter data packet includes a matching backend replacement identifier, then the first inference graph is updated.

[0122] In this embodiment of the invention, the parameter data package includes second test environment parameters. The first generation module 12 is specifically used to generate a third inference graph based on the network model file in the second backend environment corresponding to the second test environment parameters; determine whether to update the third inference graph based on the parameter data package; if it is determined that the third inference graph should be updated, generate a fourth inference graph based on the parameter data package and the third inference graph; generate a first sequence and a first execution file based on the fourth inference graph; and generate an inference result based on the first sequence, the first execution file, and the test data.

[0123] In this embodiment of the invention, the fourth inference graph includes at least one subgraph, and the first execution file includes at least one first execution file; the first generation module 12 is specifically used to compile at least one subgraph to generate a first execution file corresponding to each subgraph; and to generate a first sequence according to the order in which the subgraphs are compiled.

[0124] In this embodiment of the invention, the parameter data package includes third test environment parameters, and the first generation module 12 is specifically used to generate inference results based on the network model file and test data in the third backend environment corresponding to the third test environment parameters.

[0125] In this embodiment of the invention, the parameter data package includes fourth test environment parameters. The first generation module 12 is specifically used to obtain a backend configuration parameter file in the fourth backend environment corresponding to the fourth test environment parameters; generate a second execution master file based on the backend configuration parameter file and the network model file; and generate inference results based on the second execution master file and the test data.

[0126] This invention provides a backend testing device. A computer device acquires parameter data packets, network model files, and test data; generates inference results based on the parameter data packets, network model files, and test data; generates baseline data based on the network model files and test data; and generates backend test results based on the baseline data and the inference results. This allows the computer device to access the hardware backend during model training based on the backend setting parameters included in the parameter data packets, enabling real-time verification of the numerical stability of the trained model on the backend, thereby quickly testing the stability of the backend's inference values. It achieves decoupling of the backend devices; it can access backends at different stages, achieving real-time testing of software simulation hardware, principle verification hardware, or real backend hardware; it can also quickly verify the performance of existing backends on different network models; and the backend test results allow technicians to easily identify logical design flaws in the backend, providing guidance for later modifications and reducing the development cycle.

[0127] This invention provides a storage medium that includes a stored program. When the program runs, it controls the device where the storage medium is located to execute the steps of the embodiments of the backend testing method described above. For a detailed description, please refer to the embodiments of the backend testing method described above.

[0128] This invention provides a computer device including a memory and a processor. The memory is used to store information including program instructions, and the processor is used to control the execution of the program instructions. When the program instructions are loaded and executed by the processor, they implement the steps of the embodiments of the above-described backend testing method. For a detailed description, please refer to the embodiments of the above-described backend testing method.

[0129] Figure 11 This is a schematic diagram of a computer device provided in an embodiment of the present invention. Figure 11 As shown, the computer device 30 in this embodiment includes a processor 31, a memory 32, and a computer program 33 stored in the memory 32 and executable on the processor 31. When the computer program 33 is executed by the processor 31, it implements the testing method described in the embodiment; to avoid repetition, it will not be described in detail here. Alternatively, when the computer program is executed by the processor 31, it implements the functions of each model / unit in the back-end testing device described in the embodiment; to avoid repetition, it will not be described in detail here.

[0130] Computer device 30 includes, but is not limited to, processor 31 and memory 32. Those skilled in the art will understand that... Figure 11 This is merely an example of computer device 30 and does not constitute a limitation on computer device 30. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device 30 may also include input / output devices, network access devices, buses, etc.

[0131] The processor 31 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0132] The memory 32 can be an internal storage unit of the computer device 30, such as a hard disk or RAM of the computer device 30. The memory 32 can also be an external storage device of the computer device 30, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device 30. Furthermore, the memory 32 can include both internal and external storage units of the computer device 30. The memory 32 is used to store computer programs and other programs and data required by the computer device 30. The memory 32 can also be used to temporarily store data that has been output or will be output.

[0133] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0134] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0135] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0136] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0137] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0138] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A backend testing method, characterized in that, include: Obtain parameter data packets, network model files, and test data; Inference results are generated based on the parameter data packet, the network model file, and the test data. Baseline data is generated based on the network model file and the test data; Generate backend test results based on the baseline data and the inference results; Wherein, if the parameter data packet includes first test environment parameters, then the step of generating inference results based on the parameter data packet, the network model file, and the test data includes: In the first backend environment corresponding to the first test environment parameters, a first inference graph is generated based on the network model file; Determine whether to update the first inference graph based on the parameter data packet; If it is determined that the first inference graph needs to be updated, then a second inference graph is generated based on the parameter data packet and the first inference graph; A first execution queue is generated based on the second inference graph; The inference result is generated based on the test data and the first execution queue.

2. The method according to claim 1, characterized in that, The step of determining whether to update the first inference graph based on the parameter data packet includes: Determine whether the parameter data packet includes a matching backend replacement identifier; If it is determined that the parameter data packet includes the matching backend replacement identifier, then the first inference graph is updated.

3. The method according to claim 1, characterized in that, If the parameter data packet includes second test environment parameters, then generating the inference result based on the parameter data packet, the network model file, and the test data includes: In the second backend environment corresponding to the second test environment parameters, a third inference graph is generated based on the network model file; Determine whether to update the third inference graph based on the parameter data packet; If it is determined that the third inference graph needs to be updated, then a fourth inference graph is generated based on the parameter data packet and the third inference graph. Generate a first sequence and a first execution master file based on the fourth reasoning graph; The inference result is generated based on the first sequence, the first execution master file, and the test data.

4. The method according to claim 3, characterized in that, The fourth inference graph includes at least one subgraph, and the first execution master file includes at least one first execution file; The step of generating the first sequence and the first execution master file based on the fourth reasoning graph includes: Compile at least one of the subgraphs to generate a first executable file corresponding to each subgraph; The first order is generated according to the order in which the subgraphs are compiled.

5. The method according to claim 1, characterized in that, If the parameter data packet includes third test environment parameters, then generating the inference result based on the parameter data packet, the network model file, and the test data includes: In the third backend environment corresponding to the third test environment parameters, inference results are generated based on the network model file and the test data.

6. The method according to claim 1, characterized in that, If the parameter data packet includes a fourth test environment parameter, then generating the inference result based on the parameter data packet, the network model file, and the test data includes: In the fourth backend environment corresponding to the fourth test environment parameter, obtain the backend configuration parameter file; A second execution master file is generated based on the backend configuration parameter file and the network model file; The inference result is generated based on the second execution master file and the test data.

7. A backend testing device, characterized in that, include: The acquisition module is used to acquire parameter data packets, network model files, and test data. The first generation module is used to generate inference results based on the parameter data packet, the network model file, and the test data; The second generation module is used to generate baseline data based on the network model file and the test data; The third generation module is used to generate backend test results based on the baseline data and the inference results; Wherein, if the parameter data packet includes first test environment parameters, then the step of generating inference results based on the parameter data packet, the network model file, and the test data includes: In the first backend environment corresponding to the first test environment parameters, a first inference graph is generated based on the network model file; Determine whether to update the first inference graph based on the parameter data packet; If it is determined that the first inference graph needs to be updated, then a second inference graph is generated based on the parameter data packet and the first inference graph; A first execution queue is generated based on the second inference graph; The inference result is generated based on the test data and the first execution queue.

8. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device where the storage medium is located to perform the backend testing method according to any one of claims 1 to 6.

9. A computer device comprising a memory and a processor, the memory for storing information including program instructions, and the processor for controlling the execution of the program instructions, characterized in that, When the program instructions are loaded and executed by the processor, they implement the backend testing method steps of any one of claims 1 to 6.

Images