Calculation verification method and device of operator, electronic equipment, storage medium and program product

Abstract

The invention relates to an operator calculation verification method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring execution information of a target operator through a distribution interface; in a reference computing environment, executing the target operator based on the execution information to obtain a reference result corresponding to the target operator; on a verification hardware platform, executing the target operator based on the execution information to obtain an actual calculation result of the target operator on the verification hardware platform; and comparing the reference result with the actual calculation result to obtain a comparison result corresponding to the target operator on the verification hardware platform.

Description

Technical Field

[0001] The present disclosure relates to the field of verification technology, and in particular to a calculation verification method of an operator, a calculation verification device of an operator, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In computer science, an operator can refer to a function or method that performs a certain calculation or operation. In deep learning frameworks (such as TensorFlow, PyTorch, etc.), operators can refer to functions used to perform various mathematical operations (such as addition, multiplication, convolution, etc.).

[0003] Computational verification refers to the process of ensuring the correctness and accuracy of computational results. In deep learning, due to the increase in model complexity and data volume, computational errors may have a serious impact on the results.

[0004] Computational verification of operators can refer to verifying whether the operators used in a specific computing framework or environment execute as expected and produce correct results. Computational verification of operators is crucial to ensure the correctness, reliability, and interpretability of deep learning models. Summary of the invention

[0005] The present disclosure provides a computational verification technical solution for an operator.

[0006] According to one aspect of the present disclosure, a method for calculating and verifying an operator is provided, comprising:

[0007] Obtain the execution information of the target operator through the distribution interface;

[0008] In a reference computing environment, executing the target operator based on the execution information to obtain a reference result corresponding to the target operator;

[0009] On the verification hardware platform, executing the target operator based on the execution information to obtain an actual calculation result of the target operator on the verification hardware platform;

[0010] The reference result is compared with the actual calculation result to obtain a comparison result corresponding to the target operator on the verification hardware platform.

[0011] In a possible implementation manner, after obtaining the comparison result corresponding to the target operator on the verification hardware platform, the method further includes:

[0012] Based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms, a calculation verification report corresponding to the target operator is generated.

[0013] In a possible implementation manner, among the multiple verification hardware platforms, at least two verification hardware platforms support different calculation precisions.

[0014] In a possible implementation, the method includes:

[0015] The target operator and the execution information are sent to the reference computing environment and the verification hardware platform through the distribution interface.

[0016] In a possible implementation, before acquiring the execution information of the target operator through the distribution interface, the method further includes:

[0017] Create abstraction layers;

[0018] In the abstract layer, the interface method of the distribution interface is defined, wherein the interface method of the distribution interface can be recognized and called by different hardware platforms.

[0019] In a possible implementation, generating a calculation verification report corresponding to the target operator based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms includes:

[0020] Based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms, analyzing causes of differences between the reference result and multiple actual calculation results corresponding to the multiple verification hardware platforms;

[0021] Based on the cause of the difference, a calculation verification report corresponding to the target operator is generated.

[0022] In a possible implementation, generating a calculation verification report corresponding to the target operator based on the cause of the difference includes:

[0023] Generate optimization suggestion information based on the causes of the differences;

[0024] Based on the cause of the difference and the optimization suggestion information, a calculation verification report corresponding to the target operator is generated.

[0025] In a possible implementation manner, the execution information includes input data and execution parameters.

[0026] In a possible implementation, the method further includes:

[0027] In response to the target operator being a computationally intensive operator for GPU execution, a CPU simulation environment is determined as the reference computing environment, and a GPU is determined as the verification hardware platform.

[0028] According to one aspect of the present disclosure, there is provided a computation verification device for an operator, comprising:

[0029] The acquisition module is used to obtain the execution information of the target operator through the distribution interface;

[0030] A first execution module, configured to execute the target operator based on the execution information in a reference computing environment to obtain a reference result corresponding to the target operator;

[0031] A second execution module is used to execute the target operator on the verification hardware platform based on the execution information to obtain an actual calculation result of the target operator on the verification hardware platform;

[0032] The comparison module is used to compare the reference result with the actual calculation result to obtain the comparison result corresponding to the target operator on the verification hardware platform.

[0033] In a possible implementation manner, the device further includes:

[0034] A generation module is used to generate a calculation verification report corresponding to the target operator based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms.

[0035] In a possible implementation manner, among the multiple verification hardware platforms, at least two verification hardware platforms support different calculation precisions.

[0036] In a possible implementation, the device includes:

[0037] A sending module is used to send the target operator and the execution information to the reference computing environment and the verification hardware platform through the distribution interface.

[0038] In a possible implementation manner, the device further includes:

[0039] Create modules to create abstraction layers;

[0040] A definition module is used to define the interface method of the distribution interface in the abstract layer, wherein the interface method of the distribution interface can be recognized and called by different hardware platforms.

[0041] In a possible implementation, the generating module is used to:

[0042] Based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms, analyzing causes of differences between the reference result and multiple actual calculation results corresponding to the multiple verification hardware platforms;

[0043] Based on the cause of the difference, a calculation verification report corresponding to the target operator is generated.

[0044] In a possible implementation, the generating module is used to:

[0045] Generate optimization suggestion information based on the causes of the differences;

[0046] Based on the cause of the difference and the optimization suggestion information, a calculation verification report corresponding to the target operator is generated.

[0047] In a possible implementation manner, the execution information includes input data and execution parameters.

[0048] In a possible implementation manner, the device further includes:

[0049] The determination module is used to determine the CPU simulation environment as the reference computing environment and the GPU as the verification hardware platform in response to the target operator being a computing-intensive operator for GPU execution.

[0050] According to one aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to call the executable instructions stored in the memory to execute the above method.

[0051] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.

[0052] According to one aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes the above method.

[0053] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0054] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0056] Figure 1A flowchart of a method for verifying the calculation of an operator provided by an embodiment of the present disclosure is shown.

[0057] Figure 2 A schematic diagram of a calculation verification method of an operator provided in an embodiment of the present disclosure is shown.

[0058] Figure 3 A block diagram of a computation verification device for an operator provided by an embodiment of the present disclosure is shown.

[0059] Figure 4 A block diagram of an electronic device 1900 provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0060] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0061] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0062] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0063] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0064] In related technologies, the computational verification methods of operators are often disconnected from the operator distribution mechanism within the deep learning framework. This is mainly reflected in unit tests based on predefined test cases, which only verify the accuracy of specific operators, and the test cases are usually static, covering only a limited set of preset input data. Therefore, this verification method may not be able to fully capture the various data variations that deep learning models may encounter in real environments.

[0065] In addition, another common verification method is to output the calculation results of the operator to a log file and then analyze it manually. However, this method lacks the assistance of automated tools. When the scale of deep learning models increases and the number of operators surges, manual analysis will become time-consuming and error-prone.

[0066] Deep learning frameworks usually contain a core distribution mechanism that dynamically maps high-level operator calls to the corresponding underlying implementations based on the characteristics of the input data (such as the data type of the input data or the device it is on) and the type of operation (such as forward calculation or reverse derivation). This distribution mechanism greatly improves the portability of the code, allowing developers to write device-independent code, while the deep learning framework is responsible for selecting the most suitable computing core to execute on the back end. However, although this distribution mechanism plays an important role in deep learning frameworks, it has not been fully utilized in computational verification methods in related technologies.

[0067] Specifically, operator calculation verification methods in related technologies are often performed in the early stages of deep learning model development rather than in actual operating environments. Therefore, these methods may not be able to fully evaluate the performance of deep learning models under a variety of operating conditions. This may lead to calculation accuracy issues when deep learning models are deployed in actual hardware environments due to certain specific data or operating conditions that are not fully considered.

[0068] In order to solve technical problems similar to those described above, an embodiment of the present disclosure provides a method for verifying the calculation of an operator, which obtains the execution information of a target operator through a distribution interface, executes the target operator based on the execution information in a reference computing environment, and obtains a reference result corresponding to the target operator, executes the target operator based on the execution information on a verification hardware platform, and obtains an actual calculation result of the target operator on the verification hardware platform, and compares the reference result with the actual calculation result to obtain a comparison result corresponding to the target operator on the verification hardware platform, thereby dynamically capturing and verifying the operator in an actual operating environment by utilizing the distribution mechanism within a deep learning framework.

[0069] In the related art, static unit testing is difficult to cover various data scenarios that deep learning models may encounter in actual applications, which limits the comprehensiveness and reliability of verification. In the disclosed embodiment, the operator distribution mechanism built into the deep learning framework is used for dynamic capture and verification, so that the verification process covers various actual operating conditions that deep learning models may encounter, including those that cannot be foreseen in static testing, that is, it is possible to test various data variations and operating conditions that may occur in real application scenarios. In addition, related technologies are usually verified in the early stages of model development, and operators are rarely fully tested in the actual operating environment, so it is impossible to ensure that the calculation accuracy is maintained under all operating conditions. In the disclosed embodiment, by dynamically inserting verification logic when the model is actually running, the operator is allowed to be monitored and verified in real time during the execution of the model. This dynamic verification ensures that the performance of the operator in actual operation can be accurately captured and evaluated. Therefore, the embodiments of the present disclosure can realize dynamic verification, improve the coverage of operator calculation verification, improve the comprehensiveness and reliability of operator calculation verification, and enable the calculation accuracy of operators in the deep learning model to be fully verified before actual deployment, which helps the deep learning model to better adapt to various changes in the actual environment and reduce the risk of encountering problems in actual applications.

[0070] The calculation verification method of operators in the related art usually requires a lot of manual intervention, whether it is writing test cases or analyzing log files, it is time-consuming and prone to errors, especially when dealing with complex models. In the embodiments of the present disclosure, operator verification can be performed automatically, including automatically capturing the execution information of the operator through the distribution interface, automatically redirecting through the distribution interface, and automatically capturing, recording and comparing the calculation results of the operator, thereby reducing manual intervention, improving the efficiency of operator calculation verification, and greatly shortening the verification cycle, especially in large-scale deep learning models. Therefore, the embodiments of the present disclosure realize the automation and standardization of operator calculation verification, significantly improve the efficiency of operator calculation verification, and can effectively cope with the growing number of operators and model complexity in deep learning models.

[0071] In addition, since the embodiments of the present disclosure can capture the calculation results instantly when the operator is executed without waiting for the entire deep learning model to run to completion, the verification efficiency can be further improved.

[0072] The operator calculation verification tool in the related art is hardware-dependent and can only run on a specific hardware or software platform, which limits its application in different operating environments. The operator calculation verification method provided by the embodiment of the present disclosure is independent of a specific hardware or software platform and can run across platforms. The operator calculation verification tool implemented by the operator calculation verification method provided by the embodiment of the present disclosure can be used in various computing environments including CPU and GPU, thereby improving the applicability and flexibility of the tool.

[0073] By adopting the embodiments of the present disclosure, it is possible to verify the computational accuracy problems that may occur when operators are running on different hardware platforms, so that potential errors and defects can be discovered and repaired in a timely manner, which is beneficial to ensuring the stability and reliability of the entire deep learning model. It can provide strong technical support for the development and optimization of deep learning models and accelerate the entire process of deep learning models from research and development to deployment.

[0074] The calculation verification method of the operator provided by the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0075] Figure 1 A flow chart of a calculation verification method for an operator provided in an embodiment of the present disclosure is shown. In one possible implementation, the execution subject of the calculation verification method for the operator may be a calculation verification device for the operator. For example, the calculation verification method for the operator may be executed by a terminal device or a server or other electronic device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device or a wearable device, etc. In some possible implementations, the calculation verification method for the operator may be implemented by a processor calling computer-readable instructions stored in a memory. For example Figure 1 As shown, the calculation verification method of the operator includes steps S11 to S14.

[0076] In step S11, the execution information of the target operator is obtained through the distribution interface.

[0077] In step S12, in a reference computing environment, the target operator is executed based on the execution information to obtain a reference result corresponding to the target operator.

[0078] In step S13, on the verification hardware platform, the target operator is executed based on the execution information to obtain an actual calculation result of the target operator on the verification hardware platform.

[0079] In step S14, the reference result is compared with the actual calculation result to obtain a comparison result corresponding to the target operator on the verification hardware platform.

[0080] In the embodiments of the present disclosure, the distribution interface may be an interface in the deep learning framework that is independent of the hardware platform and is used to realize the capture and redirection of operators. Among them, capturing an operator may include obtaining the execution information of the operator. Redirecting an operator may mean sending the operator and its execution information to a specified computing environment or hardware platform. By adopting the distribution interface, the same operator can be executed on different hardware platforms, so that the same operator can be verified based on different hardware platforms.

[0081] In some application scenarios, the distribution interface may also be called a public distribution interface, etc., which is not limited here.

[0082] In a possible implementation, before obtaining the execution information of the target operator through the distribution interface, the method further includes: creating an abstract layer; in the abstract layer, defining the interface method of the distribution interface, wherein the interface method of the distribution interface can be recognized and called by different hardware platforms.

[0083] In this implementation, before implementing the distribution interface, an abstraction layer can be created. The abstraction layer can be used to provide a unified interface specification for different hardware platforms, so that the software on these hardware platforms can interact with the hardware in a standardized way. In this implementation, the interface methods of the distribution interface can be defined in the abstraction layer. These interface methods can be specific functions or methods, which describe the standard input and output formats, data types, execution parameters, etc. required for operator execution. These interface methods can be designed to be recognized and called by different hardware platforms. By defining the interface methods of the distribution interface in the abstraction layer, cross-platform verification of operator calculations can be made possible. That is, no matter on which hardware platform, as long as the hardware platform can recognize and call these interface methods, operator calculations can be performed, thereby enabling cross-platform verification of operator calculations.

[0084] In one possible implementation, a hardware abstraction layer (HAL) can be developed when building a distribution interface. The HAL can hide the details of specific hardware and provide a unified access interface to the upper layer. In addition, the HAL can abstract the execution details of the operator so that the upper layer application does not need to care about the specific implementation of the underlying hardware.

[0085] In a deep learning model, an operator may refer to a function or method that performs a mathematical operation, such as matrix multiplication, convolution operation, etc. In the embodiments of the present disclosure, a target operator may represent any operator to be verified by computation. For example, the target operator may be any operator in a deep learning model.

[0086] In the embodiment of the present disclosure, the execution information of the target operator may be captured through the distribution interface, wherein the execution information of the target operator may represent any information related to the execution of the target operator.

[0087] In a possible implementation manner, the execution information includes input data and execution parameters.

[0088] In this implementation, the execution information of the target operator may include input data of the target operator and execution parameters of the target operator. The execution parameters of the target operator may represent control parameters required when the target operator is executed, and may be used to ensure that the target operator can be correctly executed according to predefined rules and requirements.

[0089] As an example of this implementation, the execution parameters may include at least part of row parameters, column parameters, matrix row number parameters, character string parameters, and the like.

[0090] The row-wise parameter may be an integer parameter that specifies that the matrix operation should be processed row-wise. For example, in a matrix transpose operation, the row-wise parameter may instruct the algorithm to keep the rows of the matrix unchanged and convert the columns to rows.

[0091] The column-wise parameter can be an integer parameter that specifies that the matrix operation should be processed column-wise. In contrast to the row-wise parameter, the column-wise parameter instructs the algorithm to calculate or transform by column in the matrix operation.

[0092] The matrix row number parameter may be an integer parameter that specifies the number of rows in the matrix. The execution parameters may include the matrix row number parameter when it is necessary to determine the dimensions of the matrix and perform certain matrix operations (such as matrix multiplication, matrix slicing, etc.).

[0093] In one example, the string parameter may include an operation mode parameter. The operation mode parameter may be used to specify the type of operation to be performed on a matrix or tensor. For example, it may be an operation name (such as "transpose" for transposition or "multiply" for multiplication).

[0094] Of course, the execution parameters may also include other string parameters, for example, string parameters for providing additional operation details or specifying a specific algorithm variant, such as "convolution-type" which may specify the type of convolution operation (such as "valid" or "same").

[0095] In this implementation, during the operator verification process, the execution parameters will be passed to the target operator together with the input data through the distribution interface so that it can be executed on the reference computing environment and verification hardware platform and the accuracy of its calculation results can be verified.

[0096] In the embodiments of the present disclosure, a computing environment may represent a combination of hardware, software, and network resources that support data processing, storage, transmission, and application execution. A reference computing environment may refer to a computing environment that has been standardized and optimized to ensure consistency and repeatability of computing tasks. In a reference computing environment, hardware and software resources are carefully selected and configured to meet the requirements of a specific type of computing task.

[0097] In one possible implementation, the reference computing environment may include a server equipped with a high-performance CPU (Central Processing Unit) that supports high-precision floating-point operations to ensure more accurate results when performing numerical calculations. In addition, the reference computing environment may also include a matching operating system, mathematical libraries, and application software to fully utilize the high-performance characteristics of the CPU. This configuration enables the reference computing environment to handle tasks that require high precision.

[0098] In the embodiments of the present disclosure, the verification hardware platform may be various hardware devices or systems for executing and verifying operator calculations. Different verification hardware platforms may have different computing capabilities and architectures, and each verification hardware platform may support the execution of operators in a deep learning model.

[0099] In the embodiments of the present disclosure, different verification hardware platforms may have different computing capabilities. For example, a CPU is suitable for multi-threaded tasks, while a GPU (Graphics Processing Unit) is good at parallel processing.

[0100] In one possible implementation, in addition to general-purpose hardware such as CPU and GPU, the verification hardware platform may also include specialized hardware, such as TPU (Tensor Processing Unit) or FPGA (Field-Programmable Gate Array) for deep learning.

[0101] In one possible implementation, different verification hardware platforms may support different levels of numerical precision, such as single precision (FP32), double precision (FP64), or half precision (FP16).

[0102] In a possible implementation manner, among the multiple verification hardware platforms, at least two verification hardware platforms support different calculation precisions.

[0103] In this implementation, among the multiple verification hardware platforms, at least two verification hardware platforms support different calculation precisions, thereby improving the comprehensiveness and adaptability of operator calculation verification. By verifying the target operator on verification hardware platforms with different calculation precisions, it can be ensured that the target operator can meet the expected accuracy requirements in various hardware environments, thereby enhancing the reliability and stability of the deep learning model. In addition, this implementation can detect and solve the calculation inconsistency problems caused by hardware differences, which helps developers optimize operator performance and improve the execution efficiency and accuracy of deep learning models on different devices.

[0104] In one possible implementation, the hardware platform on which the deep learning model to which the target operator belongs is to be deployed can be determined as the verification hardware platform, so that the target operator can be verified in the actual operating environment of the deep learning model.

[0105] In a possible implementation, the hardware platform with the highest priority can be determined as the verification hardware platform from a preset hardware platform list according to the priority of each hardware platform, so that the calculation accuracy of the target operator on the hardware platform with higher priority can be verified preferentially.

[0106] This priority-driven approach can effectively manage and optimize the verification process of operators on multiple hardware platforms, ensuring that the operator's computational accuracy and performance are prioritized in the most critical hardware environments. This helps improve the efficiency of verification work and ensures that operators can achieve the expected performance standards on the most important hardware platforms before actual deployment.

[0107] In a possible implementation, the method further includes: in response to the target operator being a computationally intensive operator for GPU execution, determining a CPU simulation environment as the reference computing environment, and determining a GPU as the verification hardware platform.

[0108] In this implementation, the target operator can be identified to determine whether the target operator is a computationally intensive operator that needs to be executed on the GPU. In the case where the target operator is a computationally intensive operator that needs to be executed on the GPU, the CPU simulation environment can be used as a reference computing environment, for example, a CPU simulation environment that supports high-precision floating-point operations can be used as a reference computing environment, and the GPU can be determined as a verification hardware platform.

[0109] The implementation method can generate high-precision reference results in a well-controlled CPU simulation environment, and then compare them with the actual calculation results on the GPU. This implementation method allows developers to evaluate the performance and accuracy of GPU operators in a stable and predictable environment, while ensuring the consistency and reliability of the verification process. Through this implementation method, computational errors that may occur during GPU execution can be discovered and corrected in a timely manner, thereby improving the overall quality and performance of deep learning models.

[0110] In another possible implementation, when the target operator is an operator for running on a dedicated AI (Artificial Intelligence) accelerator, the verification hardware platform may be an AI accelerator.

[0111] In a possible implementation, the verification parameters can be adjusted according to the performance characteristic information of the verification hardware platform and the requirement information of the target operator. The performance characteristic information of the verification hardware platform may include at least part of the processing capability, memory bandwidth, and computational accuracy. The requirement information of the target operator may include at least part of the accuracy requirements, computational complexity, data dependency, etc. of the target operator. Based on the performance characteristic information of the verification hardware platform and the requirement information of the target operator, as well as the preset verification parameter adjustment strategy, a suitable accuracy level can be selected, the size of the test data set can be adjusted, a reasonable error tolerance can be set, and so on.

[0112] In this implementation, by automatically adjusting verification parameters according to the performance characteristic information of the verification hardware platform and the requirement information of the target operator, the verification process can be optimized and manual intervention can be reduced.

[0113] In a possible implementation, the method includes: sending the target operator and the execution information to the reference computing environment and the verification hardware platform through the distribution interface.

[0114] In this implementation, the distribution interface can be used to dynamically send the target operator and its execution information to different computing environments and hardware platforms. Through the distribution interface, the function of the target operator can be verified on multiple different hardware platforms, thereby ensuring the compatibility of the operator and its performance in different environments.

[0115] This implementation provides an efficient, flexible and scalable way to verify and test operators in deep learning models. Through the distribution interface, the execution and verification process can be automated in different hardware and computing environments, which can quickly identify and resolve hardware dependency issues and ensure that operators can achieve the expected performance and accuracy on various verification hardware platforms.

[0116] In an embodiment of the present disclosure, the target operator may be executed in a reference computing environment based on the execution information to obtain a reference result corresponding to the target operator. The reference result may represent an output result obtained when the target operator is executed in the reference computing environment. The reference result may be used as a benchmark in the verification process to compare with the results obtained by executing the same operator on other hardware platforms.

[0117] In a possible implementation, the target operator can be automatically executed in a reference computing environment based on the execution information by a verification module to obtain a reference result corresponding to the target operator. In this implementation, the target operator can be effectively verified on various hardware platforms through the collaborative work of the distribution interface and the verification module.

[0118] In an embodiment of the present disclosure, the target operator can be executed on the verification hardware platform based on the execution information to obtain the actual calculation result of the target operator on the verification hardware platform. Among them, the actual calculation result can represent the output result obtained after executing the target operator on the verification hardware platform. The actual calculation result can reflect the actual performance of the target operator on a specific hardware, including calculation accuracy, performance, etc. The actual calculation result may depend on the characteristics of the verification hardware platform, such as processing power, memory bandwidth, calculation accuracy, etc. The actual calculation result can be used to compare with the reference result to verify the accuracy and consistency of the target operator on different hardware platforms. Due to hardware differences, the actual calculation results may differ slightly from the reference results, and these differences can be evaluated and analyzed during the verification process.

[0119] In a possible implementation, during the verification process, abnormal situations can be handled, such as data type mismatch, hardware execution error, etc., so as to improve the robustness of the verification process.

[0120] In an embodiment of the present disclosure, after obtaining a reference result corresponding to a target operator and an actual calculation result of the target operator on a verification hardware platform, the reference result can be automatically compared with the actual calculation result to obtain a comparison result corresponding to the target operator on the verification hardware platform. Among them, the comparison result corresponding to the target operator on any verification hardware platform can represent a comparison result between the actual calculation result of the target operator on the verification hardware platform and the reference result. For example, the comparison result can refer to summary data obtained by comparing the difference between the reference result obtained in the reference computing environment and the actual calculation result obtained on the verification hardware platform.

[0121] In a possible implementation, the comparison result may include all difference data between the reference result and the actual calculation result. The difference data may refer to the specific data of all differences between the reference result and the actual calculation result, including but not limited to numerical differences, precision errors, performance index differences, etc.

[0122] In another possible implementation, the comparison result may include difference data between the reference result and the actual calculation result that exceeds the allowable error range. The allowable error range may represent a predefined threshold value for determining whether the difference between the results is within an acceptable range. In this implementation, during the comparison process, if it is found that the difference between the actual calculation result and the reference result exceeds the allowable error range, the difference data may be marked and recorded. The allowable error range may be configured as required to accommodate different verification requirements and environments.

[0123] In some application scenarios, the allowable error range may also be referred to as error tolerance, etc., which is not limited here.

[0124] In a possible implementation, after obtaining the comparison result corresponding to the target operator on the verification hardware platform, the comparison result may be recorded.

[0125] In a possible implementation, after obtaining the comparison result corresponding to the target operator on the verification hardware platform, the method further includes: generating a calculation verification report corresponding to the target operator based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms.

[0126] In this implementation, the target operator is executed on multiple verification hardware platforms, and a comparison result is obtained on each verification hardware platform. These comparison results reflect the execution accuracy of the target operator on the verification hardware platform.

[0127] In this implementation, multiple comparison results corresponding to the target operator on multiple verification hardware platforms can be summarized and analyzed. In addition, a calculation verification report can be generated based on the results of the summary analysis. This calculation verification report can detail the performance of the target operator on all verification hardware platforms, including their calculation accuracy and performance.

[0128] As an example of this implementation, a calculation verification report corresponding to the target operator can be generated by a result reporting tool based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms.

[0129] In this implementation, by generating comprehensive calculation verification reports, developers can gain in-depth insights into operator performance and accuracy, which helps identify possible problems in cross-platform execution and guides them to make necessary optimizations. In addition, this implementation helps accelerate the model development cycle and improve the quality of models deployed to production environments. In addition, through automated processes and detailed calculation verification reports, it can reduce the workload of developers and improve user experience, especially when dealing with complex problems.

[0130] In a possible implementation, the generating of a calculation verification report corresponding to the target operator based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms includes: analyzing causes of differences between the reference results and multiple actual calculation results corresponding to the multiple verification hardware platforms based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms; and generating a calculation verification report corresponding to the target operator based on the causes of the differences.

[0131] In this implementation, based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms, the differences between the reference results and the actual calculation results on each verification hardware platform can be analyzed, for example, including identifying numerical deviations, performance differences, or any other behaviors that deviate from expectations.

[0132] In this implementation, the causes of the differences can be automatically analyzed. For example, the causes of the differences can involve at least part of hardware characteristics, execution environment, operator implementation details, data representation accuracy, computing resource limitations, etc.

[0133] In this implementation, a computation verification report corresponding to the target operator can be generated based on the causes of the differences obtained through automatic analysis. In this implementation, the computation verification report can record in detail the performance of the target operator on each verification hardware platform, the specific details of the differences, and possible causes of the differences.

[0134] This implementation method automatically analyzes the causes of differences and provides developers with possible causes of differences, thereby helping developers to optimize operator performance and accuracy in a targeted manner. This implementation method accelerates the problem diagnosis and resolution process and improves the efficiency and success rate of deep learning model deployment.

[0135] In a possible implementation, generating a calculation verification report corresponding to the target operator based on the cause of the difference includes: generating optimization suggestion information based on the cause of the difference; generating a calculation verification report corresponding to the target operator based on the cause of the difference and the optimization suggestion information.

[0136] In this implementation, the calculation verification report may include the cause of the difference and the optimization suggestion information.

[0137] In this implementation, targeted optimization suggestion information can be automatically generated based on the cause of the difference. The optimization suggestion information can be used by developers to improve the performance of the target operator on the hardware platform. In this implementation, a comprehensive calculation verification report can be generated by combining the cause of the difference and the optimization suggestion information. In this implementation, the calculation verification report not only records the problem, but also provides a way to solve the problem.

[0138] This implementation provides a closed-loop feedback mechanism that not only identifies the problem, but also provides specific solutions to the problem. This comprehensive calculation verification report can help developers optimize and adjust operators more efficiently to adapt to different hardware platforms. Through the optimization suggestions in the calculation verification report, the calculation accuracy, performance and reliability of the operator can be improved, and future development and maintenance costs can be reduced.

[0139] In another possible implementation, the generating a calculation verification report corresponding to the target operator based on the cause of the difference includes: generating optimization suggestion information based on the cause of the difference; and generating a calculation verification report corresponding to the target operator based on the optimization suggestion information. In this implementation, the calculation verification report may include the optimization suggestion information.

[0140] In another possible implementation, after obtaining the comparison result corresponding to the target operator on the verification hardware platform, the method further includes: generating a calculation verification report of the target operator for the verification hardware platform based on the comparison result corresponding to the target operator on the verification hardware platform.

[0141] In a possible implementation, a log corresponding to the verification process may be recorded, wherein the detail level of the log is configurable to adapt to different verification requirements and environments.

[0142] In a possible implementation, the operator calculation verification method provided by the embodiment of the present disclosure can be used to implement an operator calculation verification tool based on the framework public distribution interface.

[0143] As an example of this implementation, the operator calculation verification tool based on the framework public distribution interface is extensible and configurable, and can be expanded and configured according to user needs.

[0144] The operator calculation verification method provided by the embodiment of the present disclosure involves real-time monitoring and analysis of the calculation accuracy of the model operator running in the deep learning framework. Therefore, the operator calculation verification method provided by the embodiment of the present disclosure can be applied to the runtime environment technology field of the deep learning framework.

[0145] The operator calculation verification method provided in the embodiments of the present disclosure is suitable for optimizing the execution of operators on different hardware platforms, including but not limited to CPU, GPU and dedicated AI accelerators. Therefore, the operator calculation verification method provided in the embodiments of the present disclosure can be applied to the field of artificial intelligence hardware optimization technology.

[0146] The operator calculation verification method provided by the embodiment of the present disclosure provides an automated testing process. Therefore, the operator calculation verification method provided by the embodiment of the present disclosure can be applied to the field of software testing technology, especially when performing quality assurance and performance monitoring.

[0147] The operator calculation verification method provided by the embodiment of the present disclosure supports operator accuracy verification across different platforms, so the operator calculation verification method provided by the embodiment of the present disclosure is also related to the field of cross-platform software development technology.

[0148] The operator calculation verification method provided in the embodiment of the present disclosure involves understanding and optimizing operator calculations on different hardware architectures. Therefore, the operator calculation verification method provided in the embodiment of the present disclosure is also related to the technical field of computer architecture.

[0149] The operator calculation verification method provided in the embodiment of the present disclosure can help to deploy AI models more safely by ensuring the calculation accuracy of the operator. Therefore, the operator calculation verification method provided in the embodiment of the present disclosure is also related to AI model deployment strategies and practices.

[0150] Although the above technical fields are used as examples to introduce the technical fields to which the calculation verification method of the operator provided by the embodiment of the present disclosure can be applied, those skilled in the art will understand that the present disclosure should not be limited thereto. Those skilled in the art can apply the calculation verification method of the operator provided by the embodiment of the present disclosure to different technical fields according to the actual application scenario requirements.

[0151] The calculation verification method of the operator provided by the embodiment of the present disclosure is explained below through a specific application scenario. Figure 2 A schematic diagram of a calculation verification method of an operator provided by an embodiment of the present disclosure is shown. In this application scenario, first, a distribution interface independent of the hardware platform can be constructed.

[0152] After the deep learning model is started, the execution information of the target operator can be captured through the distribution interface. The execution information of the target operator may include the input data of the target operator and the execution parameters of the target operator. The target operator and the execution information may be sent to the reference computing environment and the verification hardware platform through the distribution interface. The reference computing environment may be a computing environment that supports high-precision settings.

[0153] The target operator can be automatically executed in a reference computing environment based on the execution information by the verification module to obtain a reference result corresponding to the target operator. The target operator can be executed on a verification hardware platform based on the execution information to obtain an actual calculation result of the target operator on the verification hardware platform.

[0154] After obtaining the reference result corresponding to the target operator and the actual calculation result of the target operator on the verification hardware platform, the reference result can be automatically compared with the actual calculation result to determine whether the reference result is consistent with the actual calculation result. If the reference result is consistent with the actual calculation result, the verification success information can be output. If the reference result is inconsistent with the actual calculation result, the difference information can be recorded, the cause of the difference can be automatically analyzed, and a calculation verification report can be generated, and optimization suggestions can be provided in the calculation verification report.

[0155] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0156] In addition, the present disclosure also provides an operator calculation verification device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement any operator calculation verification method provided by the present disclosure. The corresponding technical solutions and technical effects can be found in the corresponding records in the method part and will not be repeated here.

[0157] Figure 3 A block diagram of a computation verification device for an operator provided by an embodiment of the present disclosure is shown. Figure 3 As shown, the calculation verification device of the operator includes:

[0158] An acquisition module 31 is used to acquire the execution information of the target operator through a distribution interface;

[0159] A first execution module 32, configured to execute the target operator based on the execution information in a reference computing environment to obtain a reference result corresponding to the target operator;

[0160] A second execution module 33 is used to execute the target operator on the verification hardware platform based on the execution information to obtain an actual calculation result of the target operator on the verification hardware platform;

[0161] The comparison module 34 is used to compare the reference result with the actual calculation result to obtain the comparison result corresponding to the target operator on the verification hardware platform.

[0162] In a possible implementation manner, the device further includes:

[0163] A generation module is used to generate a calculation verification report corresponding to the target operator based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms.

[0164] In a possible implementation manner, among the multiple verification hardware platforms, at least two verification hardware platforms support different calculation precisions.

[0165] In a possible implementation, the device includes:

[0166] A sending module is used to send the target operator and the execution information to the reference computing environment and the verification hardware platform through the distribution interface.

[0167] In a possible implementation manner, the device further includes:

[0168] Create modules to create abstraction layers;

[0169] A definition module is used to define the interface method of the distribution interface in the abstract layer, wherein the interface method of the distribution interface can be recognized and called by different hardware platforms.

[0170] In a possible implementation, the generating module is used to:

[0171] Based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms, analyzing causes of differences between the reference result and multiple actual calculation results corresponding to the multiple verification hardware platforms;

[0172] Based on the cause of the difference, a calculation verification report corresponding to the target operator is generated.

[0173] In a possible implementation, the generating module is used to:

[0174] Generate optimization suggestion information based on the causes of the differences;

[0175] Based on the cause of the difference and the optimization suggestion information, a calculation verification report corresponding to the target operator is generated.

[0176] In a possible implementation manner, the execution information includes input data and execution parameters.

[0177] In a possible implementation manner, the device further includes:

[0178] The determination module is used to determine the CPU simulation environment as the reference computing environment and the GPU as the verification hardware platform in response to the target operator being a computing-intensive operator for GPU execution.

[0179] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. Its specific implementation and technical effects can refer to the description of the above method embodiments. For the sake of brevity, they will not be repeated here.

[0180] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may be a volatile computer-readable storage medium.

[0181] The embodiment of the present disclosure further provides a computer program, including a computer-readable code. When the computer-readable code is executed in an electronic device, a processor in the electronic device executes the above method.

[0182] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes the above method.

[0183] An embodiment of the present disclosure also provides an electronic device, comprising: one or more processors; and a memory for storing executable instructions; wherein the one or more processors are configured to call the executable instructions stored in the memory to execute the above method.

[0184] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0185] Figure 4 1 is a block diagram of an electronic device 1900 provided in an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal device. Figure 4, the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0186] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (MacOS X TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or similar.

[0187] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0188] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0189] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0190] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0191] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0192] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0193] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0194] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0195] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0196] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0197] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0198] If the technical solution of the embodiments of the present disclosure involves personal information, the product using the technical solution of the embodiments of the present disclosure has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of the embodiments of the present disclosure involves sensitive personal information, the product using the technical solution of the embodiments of the present disclosure has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to collect his or her personal information; or on the device for processing personal information, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0199] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for verifying the calculation of an operator, characterized in that: include: Obtain the execution information of the target operator through the distribution interface; In a reference computing environment, executing the target operator based on the execution information to obtain a reference result corresponding to the target operator; On the verification hardware platform, executing the target operator based on the execution information to obtain an actual calculation result of the target operator on the verification hardware platform; The reference result is compared with the actual calculation result to obtain a comparison result corresponding to the target operator on the verification hardware platform.

2. The method according to claim 1, characterized in that: After obtaining the comparison result corresponding to the target operator on the verification hardware platform, the method further includes: Based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms, a calculation verification report corresponding to the target operator is generated.

3. The method according to claim 2, characterized in that Among the multiple verification hardware platforms, at least two verification hardware platforms support different calculation precisions.

4. The method according to claim 1, characterized in that: The method comprises: The target operator and the execution information are sent to the reference computing environment and the verification hardware platform through the distribution interface.

5. The method according to any one of claims 1 to 4, characterized in that: Before acquiring the execution information of the target operator through the distribution interface, the method further includes: Create abstraction layers; In the abstract layer, the interface method of the distribution interface is defined, wherein the interface method of the distribution interface can be recognized and called by different hardware platforms.

6. The method according to claim 2 or 3, characterized in that: The generating a calculation verification report corresponding to the target operator based on the multiple comparison results corresponding to the target operator on multiple verification hardware platforms includes: Based on multiple comparison results corresponding to the target operator on multiple verification hardware platforms, analyzing causes of differences between the reference result and multiple actual calculation results corresponding to the multiple verification hardware platforms; Based on the cause of the difference, a calculation verification report corresponding to the target operator is generated.

7. The method according to claim 6, characterized in that The generating a calculation verification report corresponding to the target operator based on the cause of the difference includes: Generate optimization suggestion information based on the causes of the differences; Based on the cause of the difference and the optimization suggestion information, a calculation verification report corresponding to the target operator is generated.

8. The method according to any one of claims 1 to 4, characterized in that The execution information includes input data and execution parameters.

9. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: In response to the target operator being a computationally intensive operator for GPU execution, a CPU simulation environment is determined as the reference computing environment, and a GPU is determined as the verification hardware platform.

10. A computation verification device for an operator, characterized in that: include: The acquisition module is used to obtain the execution information of the target operator through the distribution interface; A first execution module, configured to execute the target operator based on the execution information in a reference computing environment to obtain a reference result corresponding to the target operator; A second execution module is used to execute the target operator on the verification hardware platform based on the execution information to obtain an actual calculation result of the target operator on the verification hardware platform; The comparison module is used to compare the reference result with the actual calculation result to obtain the comparison result corresponding to the target operator on the verification hardware platform.

11. An electronic device, characterized in that: include: one or more processors; a memory for storing executable instructions; The one or more processors are configured to call the executable instructions stored in the memory to execute the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product comprising computer readable code, or a non-volatile computer readable storage medium carrying computer readable code, characterized in that: When the computer readable code is executed in an electronic device, a processor in the electronic device executes the method according to any one of claims 1 to 9.

Images