Operator precision verification method and device, server and medium

Abstract

The invention provides an operator precision verification method and device, a server and a medium, and the method comprises the steps: carrying out the reasoning of a model based on model input, and obtaining the standard output of each operator in the model in reasoning; aiming at each operator, taking the standard output of the previous operator of the operator in the model as the test input of the operator, and obtaining the test output of each operator under the test input; and based on the standard output and the test output of each operator, carrying out precision verification on each operator. According to the method, the device, the server and the medium provided by the invention, the error of each operator is determined based on the standard output and the test output in the comparison reasoning process, and the accuracy and efficiency of operator precision verification can be effectively improved. The reliable standard output is generated and used as the test input, so that the data transmission continuity in the verification process is ensured, the problem of accumulative errors is avoided, and automatic high-efficiency precision verification is realized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an operator accuracy verification method, device, server and medium. Background Art

[0002] In the field of artificial intelligence, operator accuracy verification is the core link to ensure that neural network models maintain high performance and accuracy in different application scenarios. The accuracy of the operator directly affects the overall performance of the model.

[0003] The current operator accuracy verification method usually relies on manually written test scripts to verify the output of each operator one by one. This method is labor-intensive and inefficient, especially when the model contains a large number of operators, developers need to invest a lot of time and energy in debugging and verification. In addition, due to the large number and high complexity of operators, the manual verification process will also slow down the model adaptation process, making it difficult for the model to quickly adapt to different software and hardware environments, seriously affecting the deployment efficiency of the model. Therefore, the existing technology urgently needs to be improved in operator accuracy verification and model adaptation efficiency to meet the increasingly complex model development needs. Summary of the invention

[0004] The present invention provides an operator accuracy verification method, device, server and medium, which are used to solve the defects of low efficiency of operator accuracy verification and model adaptation in related technologies.

[0005] The present invention provides an operator accuracy verification method, comprising the following steps: Based on the model input, the model is inferred to obtain the standard output of each operator in the model in the inference; For each operator, use the standard output of the previous operator in the model as the test input of the operator, and obtain the test output of each operator under the test input; Based on the standard output and the test output of each operator, the accuracy of each operator is verified.

[0006] According to an operator accuracy verification method provided by the present invention, the acquisition of each operator in the model includes: Based on the operator optimization strategy, the model is segmented into operator graphs to obtain each operator graph included in the model, where each operator graph corresponds to an operator.

[0007] According to an operator accuracy verification method provided by the present invention, the operator optimization strategy includes at least one of an operator fusion strategy and an operator replacement strategy, the operator fusion strategy is used to fuse to obtain a fused operator that can be tested, and the operator replacement strategy is used to replace to obtain a single operator that can be tested.

[0008] According to an operator accuracy verification method provided by the present invention, for each operator, the standard output of the previous operator in the model is used as the test input of the operator, including: For each operator except the first operator in the model, the standard output of the previous operator in the model is used as the test input of the operator; And, using the model input as a test input of the first operator.

[0009] According to an operator accuracy verification method provided by the present invention, obtaining the test output of each operator under the test input includes: The test input of each operator is sent to the device, and the test output obtained by the device testing each operator based on the test input of each operator is received.

[0010] According to an operator precision verification method provided by the present invention, there are at least two model inputs, and at least one of the input data range, the number of input elements, and the input data type of the at least two model inputs is different.

[0011] According to an operator accuracy verification method provided by the present invention, the accuracy verification of each operator is performed based on the standard output and test output of each operator, including: Based on the standard output and the test output of each operator under at least two model inputs, performing error calculation on each operator to obtain an error value of each operator; Based on the error value of each operator and the error threshold of each operator, the accuracy verification result of each operator is determined; the error threshold is determined based on the historical error value of the operator.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an operator accuracy verification method as described above is implemented.

[0013] The present invention also provides a server, comprising a host and a device, wherein the device comprises at least one execution unit; The host is used to execute the operator precision verification method; the device is used to control the at least one execution unit to perform operator testing to obtain a test output, and the test output is used for executing the operator precision verification method.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the operator accuracy verification method as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned operator accuracy verification methods.

[0016] The operator precision verification method, device, server and medium provided by the present invention can effectively improve the accuracy and efficiency of operator precision verification by determining the error of each operator based on the standard output generated by comparative reasoning and the test output generated by the test. Among them, by generating a reliable standard output and using it as the test input of the next operator, the cumulative error problem caused by continuing to use the test output of the previous operator as the test input of the subsequent operator in the test of multiple operators is effectively avoided, and automated and efficient precision verification is achieved. The above-mentioned precision verification process not only reduces manual intervention, but also can adapt to various model structures, thereby greatly improving the adaptation and application capabilities of deep learning models. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic diagram of the hardware structure of the operator accuracy verification provided by the present invention.

[0019] Figure 2 This is one of the flow charts of the operator accuracy verification method provided by the present invention.

[0020] Figure 3 This is the second flow chart of the operator accuracy verification method provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the operator accuracy verification device provided by the present invention.

[0022] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention.

[0023] Figure 6 It is a structural schematic diagram of the server provided by the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] In the field of artificial intelligence, operators are the basic building blocks used in neural networks to perform various mathematical operations and data processing. Each operator is responsible for completing a specific computing task, such as matrix multiplication, convolution operation, activation function application, etc. The accuracy of the operator directly affects the calculation results and performance of the entire neural network model. Therefore, ensuring that the operator can calculate correctly under different input conditions is the key to ensuring the stability and accuracy of the neural network model.

[0026] In a neural network model, an operator usually receives one or more input data, performs specific operations, and outputs results. This output result usually exists in the form of a tensor, where each individual value or data point is called an "element". For example, if the output of an operator is a matrix array with 3 rows and 4 columns, then the output of the operator contains 12 elements. Each operation of the operator will generate an output tensor containing multiple elements. Therefore, verifying the accuracy of these elements and ensuring that the calculation results of each element meet expectations is the core task of operator verification.

[0027] At present, the verification of operators in related technologies is usually carried out by manually writing test scripts. Specifically, developers will extract the operators in the model one by one according to the structure of the neural network model, and write corresponding test scripts for each operator. These test scripts will define a set of input data, call the operator to perform calculations, and generate output results. Then, the actual output of the operator is compared with the expected "golden data" to determine whether the calculation accuracy of the operator meets the requirements. Golden data is reference output data generated by high-precision or standardized methods, which can serve as a benchmark for the correctness and accuracy of the operator. The entire verification process is usually carried out operator by operator to ensure that each operator in the neural network model can generate accurate results under the specified input conditions. The advantage of this verification method is that it can carefully cover all operators in the model to ensure that the function of each operator is fully verified.

[0028] However, the manual verification method of writing test scripts is labor-intensive and cumbersome, resulting in low overall efficiency. As the complexity of the model increases, the manual verification process will become more time-consuming, slowing down the model adaptation process, making it difficult for the model to quickly adapt to different software and hardware environments, seriously affecting the development and deployment efficiency of deep learning models.

[0029] In view of the above problems, an embodiment of the present invention provides an operator accuracy verification method. The operator accuracy verification method provided by the embodiment of the present invention can be based on Figure 1 The hardware structure for operator accuracy verification is shown in FIG. Figure 1 In the present invention, the operator accuracy verification method can be implemented based on the device and the host. The device and the host can communicate and exchange data.

[0030] Among them, the device can be an artificial intelligence chip, such as GPU (Graphics Processing Unit), GPGPU (General-Purpose Graphics Processing Unit), TPU (Tensor Processing Unit), etc. A device can include at least one execution unit (Execution Unit, EU), and the execution unit can be used to perform operator operations in the device; the host can be a central processing unit or computer connected to the device, such as CPU (Central Processing Unit), microprocessor, etc. Both the host and the device can receive, process and respond to information and requests from each other.

[0031] In the operator accuracy verification method, the test input for operator accuracy verification can be generated by the host and sent to the device, and the operation of the target operator can be implemented at the execution unit inside the device. In addition, after obtaining the test output, the device can send the test output to the host, and the host will perform error calculation to achieve operator accuracy verification.

[0032] Figure 2 This is one of the flow charts of the operator accuracy verification method provided by the present invention, such as Figure 2 As shown, the operator accuracy verification method can be applied to the host in the above hardware structure, and the operator accuracy verification method may include the following steps: Step 210, based on the model input, the model is inferred to obtain the standard output of each operator in the model in the inference.

[0033] Specifically, the input of the model usually includes a set of preprocessed and formatted standardized data, which may be images, text, audio, or other types of data, depending on the application field of the model. For example, in image recognition tasks, the input data may come from a pre-collected image dataset, or images collected in real time; in natural language processing tasks, the input data may be text data from articles, conversation records, or posts on social media, etc. These text data will be encoded or otherwise processed into a format that the model can understand; in audio processing tasks, the input data can be collected audio or recording files, or it may come from a preprocessed audio dataset. After data collection or file reading, the preprocessed input data will be passed to the model through the input layer of the model as the basis for subsequent calculations.

[0034] Based on the model input, the model processes the input data layer by layer according to the weights and bias parameters learned during its training process. In this process, the neural network model will traverse each operator in turn. Each operator is responsible for performing specific computational tasks, such as convolution, pooling, matrix multiplication, and activation function application.

[0035] In inference calculation, each operator in the model will generate corresponding output data. Since these outputs are the expected correct results produced by idealized calculations according to predetermined operator operation rules under known, standardized and preprocessed input data conditions, based on the weights and parameters learned during the model training process, these output data are called standard outputs, which can also be understood as Golden Data. Standard output is the correct result that the model should produce after idealized calculation under the current model input conditions. The standard output of each operator will be recorded as a benchmark for subsequent accuracy verification. This process ensures that all operators in the model have undergone rigorous inference calculations and provides reliable reference data for subsequent verification steps.

[0036] It should be noted that model reasoning is calculated based on the host, that is, a central processing unit or computer to which the device is connected.

[0037] Step 220: For each operator, use the standard output of the previous operator in the model as the test input of the operator, and obtain the test output of each operator under the test input.

[0038] Specifically, for each operator, the standard output of the previous operator is used as the test input of the operator for inference testing, thereby obtaining the test output of the operator. The test output refers to the actual calculation result generated by the operator under the condition that the standard output of the previous operator is used as input. By comparing these test outputs with the corresponding standard outputs, the calculation accuracy of each operator can be evaluated. Therefore, only one complete model inference in step 210 is required to simultaneously obtain the standard output of each operator and the test input of the next operator, thereby avoiding repeated calculations and improving verification efficiency.

[0039] Although the input of each operator in the inference process and the test process is the output of the previous layer on the surface, there are important differences in the nature and role of the input. In the inference process, the overall model performs calculations step by step according to the framework level, relying on the output of each layer of operators calculated in real time based on the model's weights, biases, and input data.

[0040] During the test, the focus is on verifying each operator one by one. The input of each operator is the standard output of the previous layer, that is, the correct result that should be produced under ideal conditions. This standardized input ensures that each operator is independent of the error of the previous operator during testing, avoiding the accumulation of errors. By comparing the actual output and standard output of each operator, the calculation accuracy of each operator can be accurately verified to ensure that its performance meets expectations, thereby effectively locating and verifying whether the performance of a single operator is accurate.

[0041] The significance of the testing process is that it goes deep into the execution details of each operator, rather than focusing on the operation of the entire framework like reasoning. The testing process ensures the overall accuracy of the model by independently verifying the computational performance of each operator by avoiding error propagation and accumulation. By comparing the standard output with the test output, the test can effectively evaluate the computational accuracy of each operator and help detect and correct problems in the model. In addition, the test provides independent verification conditions for each operator, which facilitates the rapid location of problems and targeted optimization of specific operators, ultimately improving the overall performance of the model.

[0042] It should be noted that the model inference is performed by the host, while the test of each operator is performed on the device, that is, the host can only obtain the test output transmitted by the device and does not participate in the test.

[0043] The reuse of standard outputs greatly simplifies the verification process. In a complete model reasoning process, the standard output and test output of each operator can be obtained. Because the standard output is directly used as the test input of the next operator, there is no need to perform multiple independent reasoning and calculation operations. This not only reduces the consumption of computing resources, but also shortens the verification time and improves the efficiency of the entire verification process. At the same time, the verification method based on standard output reuse ensures the coherence and consistency of the calculation results of each operator in the entire model reasoning chain.

[0044] Step 230: Verify the accuracy of each operator based on the standard output and test output of each operator.

[0045] Specifically, after obtaining the standard output and test output of each operator, the two outputs can be compared. Here, the standard output represents the correct result that the operator should produce under ideal conditions, while the test output is the result generated during the actual reasoning process. For each operator, by comparing the difference between the standard data and the test output, the error index of the operator can be calculated, and based on the error index of the operator, it can be judged whether the operator itself meets the accuracy requirements, thereby achieving accuracy verification.

[0046] Among them, calculating the error index can summarize the accuracy performance of the operator. The error indicators here can include mean square error (MSE), mean absolute error (MAE), maximum error, and L2 norm (L2Norm). The mean square error reflects the overall error level by squaring the differences of each element and averaging them, and is particularly sensitive to large errors. The mean absolute error provides an average level of error and intuitively shows the accuracy of the output, while the maximum error helps identify possible outliers or extreme errors. The L2 norm calculates the square root of the sum of the squares of all element differences, providing a measure of the overall error, which is particularly suitable for measuring the total amount of error.

[0047] By comparing these error indicators with the preset error threshold, we can determine whether the calculation accuracy of the operator meets the requirements. If the error is within the error threshold, the accuracy verification result of the operator is considered passed; if it exceeds the error threshold, it indicates that the operator has a large calculation error and may require further analysis and adjustment. At this point, the developer may need to optimize the implementation of the operator or reset related parameters to ensure that the accuracy requirements are met.

[0048] The method provided by the embodiment of the present invention determines the error of each operator based on the standard output generated by the comparative reasoning and the test output generated by the test, which can effectively improve the accuracy and efficiency of the operator precision verification. Among them, by generating a reliable standard output and using it as the test input of the next operator, the cumulative error problem caused by continuing to use the test output of the previous operator as the test input of the subsequent operator in the test of multiple operators is effectively avoided, and automated and efficient precision verification is achieved. The above-mentioned precision verification process not only reduces manual intervention, but also can adapt to various model structures, thereby greatly improving the adaptation and application capabilities of the deep learning model.

[0049] Based on the above embodiment, obtaining each operator in the model includes: Based on the operator optimization strategy, the model is segmented into operator graphs to obtain each operator graph included in the model, where each operator graph corresponds to an operator.

[0050] Specifically, the operator graph of the entire model can be segmented based on a specific operator optimization strategy, so that the segmented operator graph corresponds to the operator actually used for accuracy verification, thereby realizing operator segmentation in the model.

[0051] In this process, the complex computational graph of the model itself is decomposed into multiple smaller and independent operator graphs. Each operator graph can represent a specific operator in the model, and by applying optimization strategies, the original operators in the model can be optimized during the operator graph segmentation process. These optimizations may include operator fusion, operator replacement, etc. Through the above optimization, the resulting operator graph can be more suitable for efficient test calculation processes.

[0052] In the actual testing process, the operators corresponding to the operator graphs obtained by segmentation based on the operator optimization strategy have brought significant advantages. Specifically, these operators can more efficiently utilize hardware resources during the testing process and reduce the time and energy consumption required for inference calculations. This optimization significantly reduces resource consumption and improves test efficiency in large-scale testing, thereby accelerating the overall progress of operator verification.

[0053] At the same time, in the embodiment of the present invention, this optimization method based on operator graph segmentation provides strong support for the development and deployment of deep learning models. By segmenting operators, developers can debug and verify models more efficiently, reduce development cycles, and ensure the adaptability of models in different environments.

[0054] The adaptability of the model referred to here specifically refers to the ability of the deep learning model to run stably and correctly under different hardware, software platforms and various model input conditions, and output consistent and accurate results. In deep learning, each neural network model consists of multiple network layers, and each network layer can be regarded as an operator. In order for the neural network model to run correctly on a specific platform, it is necessary to ensure that each operator can be executed correctly and the calculation results are accurate. Good adaptability is crucial for the deployment of the model. It enables the model to adapt to a variety of operating environments, reduce performance fluctuations and calculation errors caused by platform differences, and thus improve the versatility, stability and application effect of the model. Based on the above embodiment, the operator optimization strategy includes at least one of an operator fusion strategy and an operator replacement strategy, the operator fusion strategy is used to fuse and obtain a fusion operator available for testing, and the operator replacement strategy is used to replace a single operator available for testing.

[0055] Among them, the operator optimization strategy includes at least one of the operator fusion strategy and the operator replacement strategy, which are used to improve the efficiency and effect of operator precision verification. By optimizing the calculation method and structure of the operator, the operator fusion strategy and the operator replacement strategy ensure the accuracy and efficiency of the precision verification process.

[0056] Specifically, the main function of the operator fusion strategy is to merge multiple continuously executed operators into a fused operator. This fusion can reduce unnecessary intermediate calculation steps and data transfer between operators, thereby simplifying the calculation process. In the process of operator accuracy verification, the fused operator can perform calculations with higher efficiency, reducing the error points that may be introduced in the verification process. By concentrating the originally scattered calculations into a fused operator for processing, the calculation path in the verification process is simpler, which makes the verification results more consistent and reliable. The fused operator can effectively avoid the accuracy loss caused by multiple data transfers, thereby ensuring that the output results of the operator can be closer to the theoretical standard output in the accuracy verification. At the same time, since the fused operator is tested and available, it can be supported by hardware in the test link, thereby ensuring the feasibility of operator accuracy verification.

[0057] The operator replacement strategy improves the calculation accuracy and efficiency of the operator by replacing the original operator with a more efficient and optimized operator. During the operator accuracy verification process, the replaced operator can provide more accurate calculation results and reduce errors in the verification process because the calculation path or algorithm is optimized. Especially when verifying operators that require extremely high calculation accuracy, the replacement strategy can select operators with better numerical stability and smaller errors, thereby improving the credibility of accuracy verification. At the same time, the replaced operator can be supported by existing hardware and run smoothly, so it is testable in the testing phase, ensuring the feasibility of operator accuracy verification.

[0058] Based on the above embodiment, in step 220, for each operator, using the standard output of the previous operator in the model as the test input of the operator includes: For each operator except the first operator in the model, the standard output of the previous operator in the model is used as the test input of the operator; And, using the model input as a test input of the first operator.

[0059] Specifically, for each operator in the model, except for the first operator in the model, all subsequent operators will use the standard output of the previous operator as their test input. This means that during the test process, the test calculation of each operator will be based on the standard output of its previous operator, rather than the test output of the previous operator. This method avoids the error accumulation problem that may be caused by the test output of the previous operator, ensures that the test input of each operator is a verified ideal result, and thus improves the test accuracy of each operator.

[0060] For the first operator in the model, since it is at the starting position of the model and there is no preceding operator to provide standard output for it, the test input of the first operator directly uses the model input. The model input is usually a preprocessed external input, such as an image, text, or other type of input signal. After entering the model, these data are first passed to the first operator for processing. By directly using the model input in step 210 as the test input of the first operator, it can be ensured that the model can perform accurate calculations based on actual input data in the initial stage of reasoning, thereby laying the foundation for the accuracy verification of subsequent operators.

[0061] This standard output reuse design not only ensures the consistency of the input conditions of each operator with the actual operating environment, but also can effectively evaluate the accuracy performance of the entire model through layer-by-layer transmission and verification.

[0062] Based on the above embodiment, in step 220, obtaining the test output of each operator under the test input includes: The test input of each operator is sent to the device, and the test output obtained by the device testing each operator based on the test input of each operator is received.

[0063] The process of calculating the test output of each operator under the test input depends on Figure 1 The hardware structure of operator precision verification shown in FIG. The hardware structure includes a host and a device connected to the host, and the device contains multiple execution units.

[0064] In the specific implementation, the host sends the test input of each operator to the execution unit in the device. The execution unit in the device is a hardware module specially designed to handle complex computing tasks, such as Figure 1 As shown, each execution unit can receive and process data independently. When the host sends test inputs to the device, multiple execution units in the device receive these test inputs in parallel or sequentially and start computing each operator.

[0065] After receiving the test input, each execution unit will process the input data according to the calculation logic of the operator and generate the corresponding test output. The design of these execution units enables them to efficiently process the calculation tasks in the deep learning model, ensuring that accurate test output results are generated in a short time.

[0066] Once the execution unit completes the calculation, the device transmits the test output of each operator back to the host. After receiving these test outputs, the host compares them with the standard output to evaluate the accuracy and performance of each operator. With this architecture, the entire operator accuracy verification process is implemented efficiently. The parallel processing capabilities of multiple execution units within the device significantly improve the efficiency of the verification process, while the two-way communication between the host and the device ensures the reliability and speed of data transmission.

[0067] Through the close collaboration between the host and the device, the operator accuracy verification process can be automated and made more efficient. The host is responsible for managing and sending test input data, while the execution unit inside the device focuses on efficient computing tasks, ensuring that each operator can quickly and accurately generate test outputs after receiving the test input. Efficient data transmission and two-way communication between the host and the device not only improve the efficiency of the entire verification process, but also ensure the reliability of data processing and the accuracy of the results. Through this coordinated verification process, it can ensure that the deep learning model still maintains a high degree of accuracy and stability in a variety of complex scenarios, providing reliable guarantees for model optimization and actual deployment.

[0068] Based on the above embodiment, in the operator precision verification method, there are at least two model inputs, and at least one of the input data range, the number of input elements, and the input data type of the at least two model inputs is different.

[0069] In the embodiment of the present invention, the model has at least two inputs, and these inputs differ in multiple aspects to fully verify the performance and accuracy of the operator. These differences may include the range of input data, the number of input elements, and the type of input data.

[0070] Specifically, different model inputs may cover data values ​​from a small range to a large range to test the stability of the operator when processing data in different ranges. For example, one model input may contain a relatively concentrated number of values, while another model input may contain a wider range of values ​​to ensure that the operator can maintain high-precision calculation results when facing various data sizes.

[0071] In addition, the difference in the number of data elements input to different models is used to evaluate the performance of operators when processing data sets of different sizes. By setting the number of data elements to change, the processing capabilities of operators under small-scale and large-scale data conditions can be observed to verify whether they can provide accurate output results while maintaining efficiency.

[0072] At the same time, different input data types, such as floating point numbers, integers, or other data formats, are also used to test the adaptability of operators. This difference can help verify whether the operator can still maintain consistent computing performance and accuracy when processing data in various formats.

[0073] By designing these differentiated model inputs, operator accuracy verification can comprehensively test the performance of operators and ensure their reliability and stability under different conditions. This multi-dimensional verification method provides rich data support for operator optimization and improvement, helping to achieve higher performance standards in practical applications.

[0074] Based on the above embodiment, the accuracy verification of each operator based on the standard output and the test output of each operator includes: Based on the standard output and the test output of each operator under at least two model inputs, performing error calculation on each operator to obtain an error value of each operator; Based on the error value of each operator and the error threshold of each operator, the accuracy verification result of each operator is determined; the error threshold is determined based on the historical error value of the operator.

[0075] First, the error value of each operator is obtained by calculating the error of the standard output and test output of each operator under at least two different model inputs. This process is to comprehensively evaluate the accuracy performance of the operator. Since different model inputs may have different data ranges, number of elements, or data types, each operator may exhibit different computing characteristics under these diverse input conditions. By comparing the outputs under these input conditions one by one and calculating the errors, we can have a deeper understanding of the accuracy and stability of the operator.

[0076] In the specific error calculation process, the test output of each operator is compared element by element with its corresponding standard output to accurately calculate the error value. The error value reflects the degree of deviation between the operator in the actual calculation and the ideal result. There are various ways to calculate the error value, such as using mean square error, absolute error or L2 norm, which effectively quantify the size of the error and provide a reliable basis for subsequent accuracy evaluation.

[0077] By comparing the error value in the current test with the preset error threshold, it is possible to accurately determine whether the accuracy of each operator meets the requirements. If the error value of an operator is lower than or equal to the error threshold, it means that the performance of the operator under the current test conditions is in line with expectations and the accuracy verification has passed, which means that the calculation results of the operator in the model are accurate and reliable; if the error value exceeds the threshold, it indicates that the operator may have accuracy problems and further analysis and optimization are required to ensure that its performance in actual applications meets the requirements.

[0078] The error threshold is determined by statistically analyzing the error performance of the operator in historical tests. It represents the maximum error range that the operator can tolerate in actual applications. This error threshold not only takes into account the historical performance of the operator under different input conditions, but may also be combined with the requirements and standards in actual applications to ensure that its setting is reasonable and meets actual needs.

[0079] The operator accuracy verification device provided by the present invention is described below. The operator accuracy verification device described below and the operator accuracy verification method described above can be referred to each other.

[0080] Based on the above embodiments, Figure 3 This is the second flow chart of the operator accuracy verification method provided by the present invention. Figure 3 As shown, the method includes: Step 310, obtain the model file and split the model operator graph. In this step, first obtain the model file of the neural network, which is usually a pre-trained model, such as a model file in ONNX format. After obtaining the model file, the complete model file is split into operator graphs. Operator graph splitting refers to decomposing the calculation graph of the entire neural network into multiple smaller subgraphs, each subgraph representing one or more operators. Through this splitting, each operator in the model can be more effectively managed and optimized, laying the foundation for the subsequent verification process. This step ensures that the operators in the model can be verified individually and prepared for generating standard outputs.

[0081] Step 320, generate the standard output of the operator based on the topological structure. Based on the operator graph segmented in step 310, generate the standard output of each operator. In this step, based on the topological structure of the model, reasoning calculations are performed on each operator graph step by step to generate the standard output of each operator. These standard data are very critical because they provide a standard with which subsequent operator outputs will be compared to determine the accuracy of the operator.

[0082] Step 330, traverse all operator graphs, call operator implementations and obtain output data. In this step, each operator graph in the model is traversed. At this point, the specific implementation of the operator is called, and each operator is run using the previously generated test input. The operator implementation can be executed by hardware or simulators in the actual deployment environment to obtain test output. The test output data represents the performance of the operator in the actual operating environment and will serve as an important basis for accuracy verification.

[0083] Step 340, compare the operator output data with the standard data of the corresponding operator to verify the accuracy. In this step, the test output generated by each operator is compared with the standard data obtained in step 320 to calculate the error value. This comparison process usually involves a variety of error calculation methods, such as mean square error, L2 norm, etc., to quantify the accuracy of the operator. If the error value is within the preset threshold range, the accuracy verification of the operator passes; if the error exceeds the threshold, it means that there is a deviation in the calculation result of the operator, and further optimization and adjustment are required. In this way, it can be ensured that each operator can maintain high-precision calculation output in practical applications.

[0084] It should be noted that the operator accuracy verification method provided in the above embodiments of the present invention can be used for operator accuracy verification in various models on artificial intelligence chips, for example, it can be applied to operator accuracy verification of different types of data processing models such as speech, image text, and video. The applicable scenarios of various embodiments of the present invention depend on the type of model used for operator accuracy verification in the present invention and its design purpose.

[0085] Specifically, for speech models such as speech recognition and speech synthesis, a speech data set containing various languages, accents, and environmental noise can be constructed as the model input of the speech model to achieve accuracy verification for each operator in the speech model, thereby ensuring the accuracy of the artificial intelligence chip in speech processing through various embodiments of the present invention. In addition, for speech models, which may include common operators such as LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit), Attention (attention mechanism) and Convolution (convolution), the accuracy verification process of these operators can be achieved through the above-mentioned operator accuracy verification method.

[0086] For computer vision models such as image classification, target detection, and image segmentation, an image test set containing a variety of objects, scenes, and lighting conditions can be designed as the model input of the computer vision model to achieve accuracy verification for each operator in the computer vision model, thereby ensuring the accuracy of the artificial intelligence chip in computer vision processing through the various embodiments of the present invention. For computer vision models, commonly used operators include various Convolution, Pooling, BatchNorm (Batch Normalization), and FullyConnected operators. The accuracy verification process of these operators can be achieved through the above-mentioned operator accuracy verification method.

[0087] For natural language processing models (such as machine translation, text generation, etc.), text datasets including multi-language and multi-domain can be used as model input. Common operators used in language neural network models include Attention, FFN (Feed Forward Network), RMS Norm (Root Mean Square Normalization) and Linear (Linear Transformation), etc. The accuracy verification of these operators can also be performed through the method of the present invention to ensure the accuracy of the language processing model.

[0088] Figure 4 is a schematic diagram of the structure of the operator accuracy verification device provided by the present invention, such as Figure 4 As shown, the device comprises: The standard output generating unit 410 is used to perform reasoning on the model based on the model input, and obtain the standard output of each operator in the model in the reasoning.

[0089] The test output generating unit 420 is used to use the standard output of the previous operator in the model as the test input of each operator, and obtain the test output of each operator under the test input.

[0090] The accuracy verification unit 430 is used to verify the accuracy of each operator based on the standard output and test output of each operator.

[0091] The operator precision verification device provided by the present invention determines the error of each operator based on the standard output generated by the comparative reasoning and the test output generated by the test, which can effectively improve the accuracy and efficiency of the operator precision verification. Among them, by generating a reliable standard output and using it as the test input of the next operator, the cumulative error problem caused by continuing to use the test output of the previous operator as the test input of the subsequent operator in the test of multiple operators is effectively avoided, and automated and efficient precision verification is achieved. The above-mentioned precision verification process not only reduces manual intervention, but also can adapt to various model structures, thereby greatly improving the adaptation and application capabilities of deep learning models.

[0092] Based on any of the above embodiments, the standard output generation unit is specifically used for: Based on the operator optimization strategy, the model is segmented into operator graphs to obtain each operator graph included in the model, where each operator graph corresponds to an operator.

[0093] Based on any of the above embodiments, in the standard output generation unit: The operator optimization strategy includes at least one of an operator fusion strategy and an operator replacement strategy. The operator fusion strategy is used to fuse to obtain a fused operator that is available for testing, and the operator replacement strategy is used to replace to obtain a single operator that is available for testing.

[0094] Based on any of the above embodiments, the test output generation unit is specifically used for: For each operator except the first operator in the model, the standard output of the previous operator in the model is used as the test input of the operator; And, using the model input as a test input of the first operator.

[0095] The test input of each operator is sent to the device, and the test output obtained by the device testing each operator based on the test input of each operator is received.

[0096] Based on any of the above embodiments, in the test output generation unit: There are at least two model inputs, and at least one of the input data range, the number of input elements, and the input data type of the at least two model inputs is different.

[0097] Based on any of the above embodiments, the accuracy verification unit is specifically used for: Based on the standard output and the test output of each operator under at least two model inputs, performing error calculation on each operator to obtain an error value of each operator; Based on the error value of each operator and the error threshold of each operator, the accuracy verification result of each operator is determined; the error threshold is determined based on the historical error value of the operator.

[0098] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the operator precision verification method, which includes: Based on the model input, the model is inferred to obtain the standard output of each operator in the model in the inference; For each operator, use the standard output of the previous operator in the model as the test input of the operator, and obtain the test output of each operator under the test input; Based on the standard output and the test output of each operator, the accuracy of each operator is verified.

[0099] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0100] Based on any of the above embodiments, Figure 6 Schematic diagram of the structure of the server provided by the present invention. Figure 6 As shown, the above operator accuracy verification method can be executed based on a server. Here, the server includes a host and a device, and the device includes at least one execution unit; The host is used to obtain a test input of a target operator and send the test input to the device; the device is used to control at least one of the execution units to operate the target operator based on the test input to obtain a test output of the target operator; The host is also used to execute the operator accuracy verification method as described above.

[0101] Specifically, the server may include a device and a host, and communication and data exchange may be performed between the device and the host.

[0102] Among them, the device can be an artificial intelligence chip, such as GPU, GPGPU, TPU, etc. A device can include at least one execution unit, and within the device, the execution unit can be used to perform operator operations; the host can be a central processing unit or computer connected to the device, such as a CPU, microprocessor, etc. Both the host and the device can receive, process and respond to information and requests from each other.

[0103] In the process of verifying the operator accuracy based on the server, the host can obtain the test input of the target operator. Here, for the host, the test input of the target operator can be randomly generated by the host or read from a pre-stored test input, which is not specifically limited in the embodiment of the present invention.

[0104] After the host obtains the test input, it can send the test input to the device, so that the device obtains the test input of the target operator.

[0105] After the device obtains the test input of the target operator, it can control the execution unit inside the device to run the target operator based on the test input. That is, at the execution unit, the test input is used as the input data of the target operator to perform operations on the target operator. In this way, the device can obtain the operation result of the execution unit, that is, the test output of the target operator.

[0106] Subsequently, the device may send the running result of the execution unit, that is, the test output of the target operator, to the host, whereby the host receives the test output of the target operator.

[0107] In addition, the host can also run the target operator based on the test input and use the result of the host's own operation as the standard output. Alternatively, the host can obtain the pre-stored standard output, thereby the device obtains the standard output of the target operator.

[0108] At this point, the host obtains the test output and standard output of the target operator. Subsequently, the host can determine the overall error of the target operator based on the difference between each element in the test output and the standard output, and determine the accuracy verification result of the target operator based on the overall error and the error threshold of the target operator.

[0109] In the server provided by the embodiment of the present invention, through the interaction between the host and the device, the error of each operator is determined based on the standard output and the test output in the comparison reasoning process, which can effectively improve the accuracy and efficiency of the operator precision verification. By generating reliable standard output and using it as test input, the consistency of data transmission in the verification process is ensured, the cumulative error problem is avoided, and automated precision verification is achieved. This not only reduces manual intervention, but also ensures the stability and accuracy of the model under different input conditions, thereby greatly improving the adaptation and application capabilities of the deep learning model.

[0110] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the operator accuracy verification method provided by the above methods, the method comprising: Based on the model input, the model is inferred to obtain the standard output of each operator in the model in the inference; For each operator, use the standard output of the previous operator in the model as the test input of the operator, and obtain the test output of each operator under the test input; Based on the standard output and the test output of each operator, the accuracy of each operator is verified.

[0111] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the operator accuracy verification method provided by the above methods, the method comprising: Based on the model input, the model is inferred to obtain the standard output of each operator in the model in the inference; For each operator, use the standard output of the previous operator in the model as the test input of the operator, and obtain the test output of each operator under the test input; Based on the standard output and the test output of each operator, the accuracy of each operator is verified.

[0112] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for verifying operator accuracy, characterized in that: include: Based on the model input, the model is inferred to obtain the standard output of each operator in the model in the inference; For each operator, use the standard output of the previous operator in the model as the test input of the operator, and obtain the test output of each operator under the test input; Based on the standard output and the test output of each operator, the accuracy of each operator is verified.

2. The operator accuracy verification method according to claim 1, characterized in that: The acquisition of each operator in the model includes: Based on the operator optimization strategy, the model is segmented into operator graphs to obtain each operator graph included in the model, where each operator graph corresponds to an operator.

3. The operator accuracy verification method according to claim 2, characterized in that: The operator optimization strategy includes at least one of an operator fusion strategy and an operator replacement strategy. The operator fusion strategy is used to fuse to obtain a fused operator that is available for testing, and the operator replacement strategy is used to replace to obtain a single operator that is available for testing.

4. The operator accuracy verification method according to claim 1, characterized in that: For each operator, using the standard output of the previous operator in the model as the test input of the operator includes: For each operator except the first operator in the model, the standard output of the previous operator in the model is used as the test input of the operator; And, using the model input as a test input of the first operator.

5. The operator accuracy verification method according to claim 1, characterized in that: The obtaining of the test output of each operator under the test input includes: The test input of each operator is sent to the device, and the test output obtained by the device testing each operator based on the test input of each operator is received.

6. The operator accuracy verification method according to any one of claims 1 to 5, characterized in that: There are at least two model inputs, and at least one of the input data range, the number of input elements, and the input data type of the at least two model inputs is different.

7. The operator accuracy verification method according to claim 6, characterized in that: The performing accuracy verification on each operator based on the standard output and the test output of each operator includes: Based on the standard output and the test output of each operator under at least two model inputs, performing error calculation on each operator to obtain an error value of each operator; Based on the error value of each operator and the error threshold of each operator, the accuracy verification result of each operator is determined; the error threshold is determined based on the historical error value of the operator.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the operator accuracy verification method as described in any one of claims 1 to 7 is implemented.

9. A server, characterized in that: The device comprises a host and a device, wherein the device comprises at least one execution unit; The host is used to execute the operator accuracy verification method as described in any one of claims 1 to 7, and the device is used to control the at least one execution unit to perform an operator test to obtain a test output, and the test output is used for the execution of the operator accuracy verification method.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the operator accuracy verification method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the operator accuracy verification method according to any one of claims 1 to 7 is implemented.

Images