Operator verification method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to an operator verification method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: constructing a first calculation graph corresponding to a target deep learning model; analyzing the first calculation graph, and determining a key operator in the target deep learning model; inserting a first verification node corresponding to the key operator into the output end of the key operator to obtain a reconstructed second computational graph; executing the second calculation graph in a first specified calculation environment, and obtaining first output data of the key operator in the first specified calculation environment through the first verification node; and verifying the key operator through the first verification node based on the first output data.

Description

Technical Field

[0001] The present disclosure relates to the field of verification technology, and in particular to an operator verification method, an operator verification device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In the field of deep learning technology, an operator can refer to a function or a set of functions that perform a specific mathematical operation, and operators are the basic building blocks for building deep learning models. Each operator corresponds to one or more mathematical operations, which are responsible for processing input data and generating output data. These operations can be linear, such as matrix multiplication, or nonlinear, such as activation functions.

[0003] The calculation accuracy of the operator is directly related to the performance and reliability of the entire deep learning model. Therefore, it is of great significance to verify the calculation accuracy of the operators in the deep learning model. Summary of the invention

[0004] The present disclosure provides an operator verification technical solution.

[0005] According to one aspect of the present disclosure, there is provided an operator verification method, comprising:

[0006] Construct a first computational graph corresponding to the target deep learning model;

[0007] Parsing the first computational graph to determine key operators in the target deep learning model;

[0008] Inserting a first verification node corresponding to the key operator at the output end of the key operator to obtain a reconstructed second calculation graph;

[0009] Execute the second computation graph in a first specified computing environment, and obtain first output data of the key operator in the first specified computing environment through the first verification node;

[0010] The key operator is verified by the first verification node based on the first output data.

[0011] In a possible implementation manner, verifying the key operator based on the first output data by the first verification node includes:

[0012] Through the first verification node, at least two output data of the key operator in at least two computing environments are compared to obtain a comparison result corresponding to the key operator, wherein the at least two computing environments include the first designated computing environment, and the at least two output data include the first output data.

[0013] In a possible implementation, the at least two computing environments further include a standard computing environment, and the at least two items of output data further include reference output data of the key operator in the standard computing environment;

[0014] After determining the key operator in the target deep learning model and before executing the second computation graph in the first designated computing environment, the method further includes: inserting a second verification node corresponding to the key operator at an input end of the key operator;

[0015] The method further includes: obtaining, through the second verification node, first input data of the key operator in the first specified computing environment; transmitting the first input data to the first verification node through the second verification node; and executing the key operator based on the first input data in the standard computing environment through the first verification node to obtain the reference output data;

[0016] The comparing at least two output data of the key operator in at least two computing environments through the first verification node to obtain the comparison result corresponding to the key operator includes: comparing the first output data with the reference output data through the first verification node to obtain the comparison result corresponding to the key operator.

[0017] In a possible implementation, the at least two computing environments include at least two designated computing environments;

[0018] The step of comparing at least two output data of the key operator in at least two computing environments by the first verification node to obtain a comparison result corresponding to the key operator includes:

[0019] At least two output data of the key operator in at least two specified computing environments are compared through the first verification node to obtain a comparison result corresponding to the key operator.

[0020] In a possible implementation, inserting a first verification node corresponding to the key operator at the output end of the key operator includes:

[0021] Bind the first verification node corresponding to the key operator to the output data of the key operator.

[0022] In a possible implementation manner, binding the first verification node corresponding to the key operator with the output data of the key operator includes:

[0023] Creating a copy corresponding to the output data stream of the key operator;

[0024] The copy is delivered to a first verification node corresponding to the key operator.

[0025] In a possible implementation manner, binding the first verification node corresponding to the key operator with the output data of the key operator includes:

[0026] The first verification node corresponding to the key operator is set to reference the output data of the key operator.

[0027] In a possible implementation, after determining the key operator in the target deep learning model and before inserting the first verification node corresponding to the key operator at the output end of the key operator, the method further includes:

[0028] According to the type of the key operator, a first verification node corresponding to the key operator is generated.

[0029] In a possible implementation, parsing the first computation graph to determine key operators in the target deep learning model includes:

[0030] Based on graph theory algorithms and data flow analysis, the first computational graph is parsed to determine the key operators of the target deep learning model.

[0031] In a possible implementation manner, after obtaining the comparison result corresponding to the key operator, the method further includes:

[0032] Generate a verification report corresponding to the key operator according to the comparison result corresponding to the key operator;

[0033] The verification report corresponding to the key operator includes at least part of the following information: information about the computing environment for executing the key operator, input data of the key operator, and output data of the key operator.

[0034] According to one aspect of the present disclosure, there is provided an operator verification device, comprising:

[0035] A construction module, used to construct a first computational graph corresponding to a target deep learning model;

[0036] A parsing module, used to parse the first computational graph to determine key operators in the target deep learning model;

[0037] A reconstruction module, used for inserting a first verification node corresponding to the key operator at the output end of the key operator to obtain a reconstructed second calculation graph;

[0038] An execution module, configured to execute the second computation graph in a first specified computing environment, and obtain first output data of the key operator in the first specified computing environment through the first verification node;

[0039] A verification module is used to verify the key operator based on the first output data through the first verification node.

[0040] In a possible implementation, the verification module is used to:

[0041] Through the first verification node, at least two output data of the key operator in at least two computing environments are compared to obtain a comparison result corresponding to the key operator, wherein the at least two computing environments include the first designated computing environment, and the at least two output data include the first output data.

[0042] In a possible implementation, the at least two computing environments further include a standard computing environment, and the at least two items of output data further include reference output data of the key operator in the standard computing environment;

[0043] The device also includes:

[0044] An insertion module, used for inserting a second verification node corresponding to the key operator at the input end of the key operator;

[0045] A reference obtaining module, configured to obtain, through the second verification node, first input data of the key operator in the first specified computing environment; transmit the first input data to the first verification node through the second verification node; and execute the key operator based on the first input data in the standard computing environment through the first verification node to obtain the reference output data;

[0046] The verification module is used to compare the first output data with the reference output data through the first verification node to obtain a comparison result corresponding to the key operator.

[0047] In a possible implementation, the at least two computing environments include at least two designated computing environments;

[0048] The verification module is used to:

[0049] At least two output data of the key operator in at least two specified computing environments are compared through the first verification node to obtain a comparison result corresponding to the key operator.

[0050] In a possible implementation, the reconstruction module is used to:

[0051] Bind the first verification node corresponding to the key operator to the output data of the key operator.

[0052] In a possible implementation, the reconstruction module is used to:

[0053] Creating a copy corresponding to the output data stream of the key operator;

[0054] The copy is delivered to a first verification node corresponding to the key operator.

[0055] In a possible implementation, the reconstruction module is used to:

[0056] The first verification node corresponding to the key operator is set to reference the output data of the key operator.

[0057] In a possible implementation manner, the device further includes:

[0058] The first generating module is used to generate a first verification node corresponding to the key operator according to the type of the key operator.

[0059] In a possible implementation, the parsing module is used to:

[0060] Based on graph theory algorithms and data flow analysis, the first computational graph is parsed to determine the key operators of the target deep learning model.

[0061] In a possible implementation manner, the device further includes:

[0062] A second generating module, used to generate a verification report corresponding to the key operator according to the comparison result corresponding to the key operator;

[0063] The verification report corresponding to the key operator includes at least part of the following information: information about the computing environment for executing the key operator, input data of the key operator, and output data of the key operator.

[0064] According to one aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; a memory for storing executable instructions; wherein the one or more processors are configured to call the executable instructions stored in the memory to execute the above method.

[0065] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.

[0066] According to one aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes the above method.

[0067] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0068] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0070] Figure 1 A flowchart of an operator verification method provided by an embodiment of the present disclosure is shown.

[0071] Figure 2 A schematic diagram of an operator verification method provided by a disclosed embodiment is shown.

[0072] Figure 3 A block diagram of an operator verification device provided by an embodiment of the present disclosure is shown.

[0073] Figure 4 A block diagram of an electronic device 1900 provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0074] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0075] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0076] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0077] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0078] In the related art, the computational accuracy of operators in deep learning models is verified mainly through two methods: operator-level unit testing and end-to-end model accuracy verification.

[0079] Among them, operator-level unit testing is a traditional verification method that focuses on testing the functionality of a single operator. Developers define a series of test cases during the operator development phase that cover the behavior of the operator on a specific input data set to ensure that the operator can produce the expected output. For example, operations such as convolution and pooling are tested on standard data sets to verify their correctness. However, this approach has limitations. It cannot capture the dynamic behavior of the operator when it interacts with other operators in the actual model, and the impact of complex data flows on computational accuracy.

[0080] End-to-end model accuracy verification focuses on the execution process of the entire model. Using specific tools, the input and output of each operator in the model can be tracked and recorded, and the model can be repeatedly executed on different computing platforms through scripts to compare and verify the consistency of the results. These tools usually use the hook or callback functions provided by the deep learning framework to insert verification code. However, the implementation of this method requires developers to have a deep understanding of the model execution framework and may require significant modifications to the model or framework code.

[0081] Although relevant technologies can verify the computational accuracy of operators in deep learning models to a certain extent, they have some obvious shortcomings:

[0082] Incomplete test coverage: Although unit tests can verify the correctness of operators under specific inputs, they usually cannot cover all data scenarios and complex interactions that operators may encounter in actual models. This may cause some potential accuracy issues to be overlooked, thus affecting the overall performance and reliability of the model.

[0083] Lack of flexibility: End-to-end verification tools in related technologies often require significant modifications to the model or framework code, which limits the adaptability and flexibility of the tools. This reliance on code modification is not conducive to rapid iteration and updating of the model, and increases the complexity of verification work.

[0084] Low cross-platform verification efficiency: Repeatedly executing the entire model for verification on different hardware and computing environments is very time-consuming and difficult to automate, especially for large-scale models. This limits the efficiency of the verification process and makes it difficult to meet the rapid development needs of the deep learning field.

[0085] Low degree of automation: The end-to-end model accuracy verification process involves a lot of manual operations, such as setting up the test environment, recording and comparing calculation results, etc. The tediousness of this manual operation makes the verification process prone to errors and difficult to adapt to the needs of automation.

[0086] These shortcomings limit the effectiveness and efficiency of operator computational accuracy verification and affect the development and optimization process of deep learning models.

[0087] In order to solve technical problems similar to those described above, the embodiment of the present disclosure adopts an operator verification method, which constructs a first computing graph corresponding to a target deep learning model, parses the first computing graph, determines the key operator in the target deep learning model, inserts a first verification node corresponding to the key operator at the output end of the key operator, obtains a reconstructed second computing graph, executes the second computing graph in a first specified computing environment, obtains the first output data of the key operator in the first specified computing environment through the first verification node, and verifies the key operator based on the first output data through the first verification node. In this way, by intelligently reconstructing the computing graph, verification nodes can be dynamically inserted into the computing graph without changing the original architecture of the deep learning model, thereby realizing the accuracy verification of the key operators in the deep learning model in different computing environments.

[0088] In the related art, static unit testing is difficult to cover various data scenarios that deep learning models may encounter in actual applications, which limits the comprehensiveness and reliability of verification. In the embodiments of the present disclosure, by intelligently analyzing and reconstructing the calculation graph, it is possible to fully cover the key operators in the deep learning model and verify the performance of each key operator in a complex data stream.

[0089] The operator verification method provided by the embodiment of the present disclosure does not rely on model and framework modifications and can be seamlessly integrated into existing development and testing processes, thereby enabling flexibility in operator verification.

[0090] The operator verification method provided by the embodiments of the present disclosure is not restricted by a specific hardware platform or deep learning framework, has good cross-platform compatibility, and is easy to deploy and use in different computing environments.

[0091] Compared with the end-to-end test in the related art, the embodiment of the present disclosure reduces the need for repeated execution of the deep learning model by inserting verification nodes in the computational graph, which can greatly improve the verification efficiency and is of great significance for the verification of large-scale and complex models.

[0092] The disclosed embodiments can greatly improve the automation level of operator calculation verification, reduce manual intervention, and significantly improve the efficiency and accuracy of operator verification.

[0093] In summary, the embodiments of the present disclosure can effectively improve the comprehensiveness, flexibility, efficiency and automation of operator precision verification in deep learning models, and can significantly improve the reliability and stability of deep learning models in diversified computing environments, thereby promoting the rapid development and application of deep learning models.

[0094] The operator verification method provided by the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0095] Figure 1 A flow chart of an operator verification method provided by an embodiment of the present disclosure is shown. In one possible implementation, the execution subject of the operator verification method may be an operator verification apparatus. For example, the operator verification method may be executed by a terminal device or a server or other electronic device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device or a wearable device, etc. In some possible implementations, the operator verification method may be implemented by a processor calling computer-readable instructions stored in a memory. For example Figure 1 As shown, the operator verification method includes steps S11 to S15.

[0096] In step S11, a first computational graph corresponding to the target deep learning model is constructed.

[0097] In step S12, the first computational graph is parsed to determine key operators in the target deep learning model.

[0098] In step S13, a first verification node corresponding to the key operator is inserted into the output end of the key operator to obtain a reconstructed second calculation graph.

[0099] In step S14, the second computation graph is executed in a first designated computing environment, and first output data of the key operator in the first designated computing environment is obtained through the first verification node;

[0100] In step S15, the key operator is verified by the first verification node based on the first output data.

[0101] The operator verification method provided by the embodiment of the present disclosure can be used to verify the calculation accuracy of various types of operators. In the deep learning model, common operator types may include linear operators, nonlinear activation operators, pooling operators, normalization operators, dropout operators, connection operators, regularization operators, etc. For example, linear operators may include weight matrix multiplication in a fully connected layer, convolution operations in a convolutional layer, etc. Nonlinear activation operators may include ReLU (Rectified Linear Unit, Corrected Linear Unit), Sigmoid, Tanh, etc., which introduce nonlinear characteristics and allow neural networks to learn complex patterns. Pooling operators may include maximum pooling (Max Pooling) and average pooling (Average Pooling) to reduce the spatial dimension of feature maps and extract important features. Normalization operators may include batch normalization (BatchNormalization) to adjust and scale activation outputs to accelerate the training process and improve model stability. The dropout operator can be used to randomly discard some network connections during training to prevent overfitting. The connection operator may include a skip connection or a residual connection, which is used to establish a direct path in the network to help the gradient flow. The regularization operator may include L1 and L2 regularization, which is used to penalize the model complexity and prevent overfitting.

[0102] In the embodiment of the present disclosure, the target deep learning model may be any deep learning model for which operator verification is to be performed. In the embodiment of the present disclosure, some operators in the target deep learning model may be verified.

[0103] In the disclosed embodiment, after determining the target deep learning model, a first computational graph corresponding to the target deep learning model can be constructed based on the code of the target deep learning model. That is, the first computational graph can be constructed from the code of the target deep learning model. Among them, the computational graph is a data structure that can represent the operations and data dependencies in the target deep learning model in the form of nodes (representing operators) and edges (representing data flows).

[0104] In the disclosed embodiment, after constructing the first computation graph corresponding to the target deep learning model, the first computation graph can be parsed to determine the key operators in the target deep learning model. By parsing the first computation graph to determine the key operators in the target deep learning model, it is possible to determine which operators in the target deep learning model are the key objects in the verification process, thereby providing decision support for subsequent verification node insertion.

[0105] In one possible implementation, an intelligent analysis tool may be used to parse the first computational graph and automatically identify key operators in the target deep learning model.

[0106] In one possible implementation, parsing the first computational graph to determine key operators in the target deep learning model includes: parsing the first computational graph based on graph theory algorithms and data flow analysis to determine key operators in the target deep learning model.

[0107] Among them, graph theory algorithms can be a class of algorithms used to analyze and operate graph structured data. Graph theory algorithms may include depth-first search (DFS) algorithms and / or breadth-first search (BFS) algorithms. Among them, the depth-first search algorithm can start from any node in the computational graph and search the branches of the graph as deeply as possible. The depth-first search algorithm helps to identify long paths and deep structures in the graph, and can reveal key operators that affect the final output in the deep network. The breadth-first search algorithm can start from a node in the computational graph and traverse all reachable nodes in the graph layer by layer. The depth-first search algorithm helps to identify the width in the graph and the hierarchical relationship of the nodes, and can find operators that play a hub role in the network.

[0108] Data flow analysis is a program analysis technique used to determine the flow and usage of data in a program. Data flow analysis can help identify which operators' outputs have a significant impact on the final results of the target deep learning model, and how data flows in the target deep learning model. Data flow analysis can focus on the definition and use of variables, as well as the dependencies between them. Through data flow analysis technology, the flow path of data in the computational graph can be traced, and the output of which operators has a significant impact on downstream nodes can be identified, thereby determining the key operators.

[0109] Among them, the process of parsing the computational graph to determine the key operators does not depend on a specific deep learning framework, that is, the method is universal and can be applied to different deep learning frameworks and model structures.

[0110] In this implementation, the first computational graph is parsed based on graph theory algorithms and data flow analysis to determine the key operators of the target deep learning model, thereby automatically analyzing and identifying the key operators and their data flows in the target deep learning model, providing a basis for subsequent reconstruction of the computational graph. This implementation does not require cumbersome manual settings and analysis processes, and improves the efficiency of operator verification.

[0111] In an embodiment of the present disclosure, after the key operator is identified, the calculation graph can be automatically reconstructed, and the corresponding first verification node is respectively inserted into the output end of each key operator to obtain a reconstructed second calculation graph. Among them, the first verification node corresponding to any key operator can represent the verification node of the output end of the key operator. After the first verification node is inserted into the output end of the key operator, the data that originally flows directly to the next operator will be captured by the first verification node, that is, the first verification node can be used to capture the output data of the key operator and verify based on the output data of the key operator. As part of the second calculation graph, the first verification node can be automatically activated and perform verification tasks when the key operator is executed.

[0112] In one possible implementation, after determining the key operator in the target deep learning model and before inserting the first verification node corresponding to the key operator at the output end of the key operator, the method further includes: generating the first verification node corresponding to the key operator according to the type of the key operator.

[0113] In this implementation, for any key operator, a first verification node corresponding to the key operator can be automatically generated according to the type of the key operator.

[0114] In this implementation, for any key operator, the type of the key operator may be identified first. For example, the key operator may be different types of operations such as convolution, pooling, activation function, and fully connected layer.

[0115] Different types of key operators can correspond to different verification strategies. Among them, the verification strategy can be used to determine how to verify whether the output of the key operator is correct. For example, for the convolution operator, the verification strategy can include checking the correct application of weights and biases, and the correct calculation of the convolution kernel sliding window.

[0116] After determining the type of the key operator and the corresponding verification strategy, the corresponding first verification node can be automatically generated according to the type of the key operator and the corresponding verification strategy. The first verification node can be used to capture the output data of the key operator and verify the output data of the key operator according to the predefined verification strategy.

[0117] In addition, the first verification node may also be configured with appropriate parameters for verification. These parameters may include expected output values, tolerance ranges, comparison functions, etc. For example, for floating point operations, a relative tolerance and an absolute tolerance may be set to determine whether two floating point numbers are close enough.

[0118] In this implementation, by automatically generating the first verification node corresponding to the key operator, the efficiency and accuracy of operator precision verification in the deep learning model can be significantly improved. This implementation reduces manual intervention and can reduce the probability of error.

[0119] In a possible implementation, the method further includes: inserting a second verification node corresponding to the key operator at the input end of the key operator. The second verification node corresponding to any key operator can represent the verification node at the input end of the key operator. The second verification node can be used to obtain the input data of the key operator, and can transmit the input data of the key operator to the verification node at the output end of the same key operator (i.e., the first verification node corresponding to the key operator), so that the first verification node corresponding to the key operator verifies the key operator based on the input data and output data of the key operator.

[0120] In the disclosed embodiment, after the verification node (the first verification node, or the first verification node and the second verification node) is inserted into the computation graph, the topological sorting of the computation graph can also be adjusted, that is, the execution order of the operators in the computation graph can be rearranged to adapt to the newly inserted verification node and maintain the correctness of the data flow and computational dependency. The adjusted topological sorting will maintain the original data dependencies in the first computation graph, ensuring that the required inputs of each operator have been calculated before execution. In graph theory and computation graphs, topological sorting is a process of linearly sorting the vertices of a directed acyclic graph (DAG) so that for any directed edge from vertex uu to vertex vv, vertex uu is in front of vertex vv. This sorting reflects the dependencies of nodes (such as tasks or computational steps) in the graph, ensuring that before any node is executed, the nodes it depends on have been executed.

[0121] In a possible implementation manner, inserting the first verification node corresponding to the key operator at the output end of the key operator includes: binding the first verification node corresponding to the key operator to output data of the key operator.

[0122] In this implementation, the first verification node corresponding to the key operator is bound to the output data of the key operator so that the first verification node receives the output data of the key operator and verifies it.

[0123] In a possible implementation, binding the first verification node corresponding to the key operator with the output data of the key operator includes: creating a copy corresponding to the output data stream of the key operator; and transferring the copy to the first verification node corresponding to the key operator.

[0124] In this implementation, in order to enable the first verification node to receive the output data of the corresponding key operator, a copy of the output data stream of the key operator can be created. That is, the output data of the key operator will be copied and sent to the first verification node, while the original data stream continues to flow to the next operator in the computation graph. The advantage of this is that it will not affect the execution process of the original computation graph.

[0125] This implementation allows the first verification node to independently receive and verify the output data of key operators without interfering with the main computational process. This is critical for achieving automatic verification of operator accuracy in deep learning models, as it provides a mechanism to perform verification immediately after the operator is executed without affecting the overall performance or behavior of the deep learning model. In addition, this implementation helps to maintain the flexibility and scalability of the computational graph, as the first verification node can be easily inserted or removed without disrupting existing computational dependencies. This is very useful for developing and maintaining complex deep learning models.

[0126] In another possible implementation manner, binding the first verification node corresponding to the key operator with the output data of the key operator includes: setting the first verification node corresponding to the key operator to reference the output data of the key operator.

[0127] In this implementation, the first verification node can directly reference the original output data of the corresponding key operator. By referencing the original output data of the key operator by the first verification node, the additional computing and memory overhead caused by creating a copy can be avoided. This is especially important when processing large-scale data or in a resource-constrained environment. Among them, the reference mechanism requires that the original output data of the key operator remains unchanged during the verification process, which ensures that the first verification node can access data that is completely consistent with the output data of the key operator. In addition, the reference mechanism allows the first verification node to access the output data of the key operator immediately after the key operator is executed and start the verification process, which helps to achieve fast and efficient verification. In addition, the use of references allows the first verification node to be dynamically inserted during the runtime of the computation graph without modifying the structure of the computation graph or reallocating the data flow.

[0128] In the disclosed embodiment, after obtaining the reconstructed second computation graph, the second computation graph can be executed in the first specified computing environment, and the first output data of the key operator in the first specified computing environment can be obtained through the first verification node.

[0129] The computing environment may represent a combination of hardware, software, and network resources that support data processing, storage, transmission, and application execution. The specified computing environment may represent a computing environment specified by the user for verifying key operators in the target deep learning model. Different specified computing environments may have different computing capabilities and architectures, and each specified computing environment can support the execution of operators in the deep learning model.

[0130] In the embodiments of the present disclosure, different designated computing environments may include different hardware, such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). Thus, different designated computing environments may have different computing capabilities, for example, a CPU is suitable for multi-threaded tasks, while a GPU is good at parallel processing.

[0131] In addition to general-purpose hardware such as CPUs and GPUs, a given computing environment can also include specialized hardware, such as TPUs (Tensor Processing Units) or FPGAs (Field-Programmable Gate Arrays) for deep learning.

[0132] Different specified computing environments may also support different levels of numerical precision, such as single precision (FP32), double precision (FP64), or half precision (FP16).

[0133] In the embodiments of the present disclosure, for any key operator, the first output data may represent the output data of the key operator in the first specified computing environment. Similarly, when the second computing graph is executed in the second specified computing environment, the output data of the key operator in the second specified computing environment may be referred to as the second output data; when the second computing graph is executed in the third specified computing environment, the output data of the key operator in the third specified computing environment may be referred to as the third output data; and so on.

[0134] In the embodiment of the present disclosure, after the first verification node obtains the first output data of the key operator in the first designated computing environment, the key operator may be verified based on the first output data.

[0135] In a possible implementation, the first verification node may compare the first output data with the expected output value corresponding to the key operator to obtain a verification result corresponding to the key operator.

[0136] Among them, the expected output value corresponding to the key operator can be the output value of the key operator when the target deep learning model is executed in a standard computing environment in advance.

[0137] Among them, a standard computing environment can refer to a computing environment that has been standardized and optimized to ensure the consistency and repeatability of computing tasks. In a standard computing environment, hardware and software resources are carefully selected and configured to meet the needs of a specific type of computing task.

[0138] For example, a standard computing environment can include a server equipped with a high-performance CPU that supports high-precision floating-point operations to ensure more accurate results when performing numerical calculations. In addition, a standard computing environment can also include a matching operating system, mathematical libraries, and application software to fully utilize the high-performance characteristics of the CPU. This configuration enables the standard computing environment to handle tasks that require high precision.

[0139] In this implementation, the first verification node can pre-store the expected output values ​​corresponding to the corresponding key operators, so that when the second calculation graph is executed in each specified computing environment, the calculation accuracy of the key operator can be verified based on the pre-stored expected output values ​​and the output data of the key operator.

[0140] In a possible implementation, a central verification logic module may be provided. The central verification logic module may be connected to each first verification node, responsible for managing and executing the comparison operation of each first verification node, and realizing the integration of verification logic.

[0141] In another possible implementation, the verifying of the key operator based on the first output data by the first verification node includes: comparing at least two output data of the key operator in at least two computing environments by the first verification node to obtain a comparison result corresponding to the key operator, wherein the at least two computing environments include the first designated computing environment, and the at least two output data include the first output data.

[0142] In this implementation, any two computing environments of the at least two computing environments are different from each other. These computing environments may be platforms with different hardware, or may be software environments with different optimization levels.

[0143] In this implementation, the key operator will be executed in at least two different computing environments respectively, and at least two output data will be obtained. The first verification node can collect the output data of the same key operator in different computing environments, and compare the at least two output data of the key operator in at least two computing environments. This comparison can be quantitative, such as by comparing the degree of proximity of the values, or qualitative, such as by comparing the pattern or distribution of the output data.

[0144] In this implementation, the first verification node can be used to implement cross-platform accuracy verification for the same key operator, thereby improving the comprehensiveness and accuracy of operator accuracy verification. This implementation provides an efficient and automated way to verify the execution accuracy of key operators in different computing environments. Through cross-platform comparisons, developers can quickly identify and resolve accuracy issues caused by hardware differences or software implementation differences. This implementation not only improves the efficiency of the verification process and reduces manual intervention, but also enhances the reliability and stability of deep learning models, ensuring that deep learning models can provide consistent performance on different platforms.

[0145] In a possible implementation, the at least two computing environments also include a standard computing environment, and the at least two output data also include reference output data of the key operator in the standard computing environment; the comparing the at least two output data of the key operator in the at least two computing environments through the first verification node to obtain the comparison result corresponding to the key operator includes: comparing the first output data with the reference output data through the first verification node to obtain the comparison result corresponding to the key operator.

[0146] In this implementation, the first verification node executes the key operator in a standard computing environment to generate reference output data. The reference output data may represent output data obtained by executing the key operator in the standard computing environment. The reference output data may be used as a benchmark in the verification process.

[0147] The first verification node may compare the first output data with the reference output data to evaluate the execution accuracy of the key operator.

[0148] In this implementation, by introducing a standard computing environment and reference output data, a clear benchmark is provided for the accuracy verification of key operators. This implementation can more accurately identify and quantify execution differences in different computing environments, thereby helping developers quickly locate problems and take appropriate measures. In addition, by directly comparing with reference data, the performance of key operators can be evaluated more objectively, enhancing the reliability and effectiveness of the verification process. This implementation not only helps to improve the cross-platform compatibility and stability of deep learning models, but also provides strong support for the optimization and maintenance of deep learning models, further promoting the development and application of deep learning technology.

[0149] In a possible implementation, the at least two computing environments also include a standard computing environment, and the at least two output data also include reference output data of the key operator in the standard computing environment; after determining the key operator in the target deep learning model and before executing the second computing graph in the first designated computing environment, the method also includes: inserting a second verification node corresponding to the key operator at the input end of the key operator; the method also includes: obtaining the first input data of the key operator in the first designated computing environment through the second verification node; transmitting the first input data to the first verification node through the second verification node; executing the key operator based on the first input data in the standard computing environment through the first verification node to obtain the reference output data; comparing the at least two output data of the key operator in at least two computing environments through the first verification node to obtain the comparison result corresponding to the key operator, including: comparing the first output data with the reference output data through the first verification node to obtain the comparison result corresponding to the key operator.

[0150] In this implementation, verification nodes can be inserted into the input and output ends of the key operator, respectively. The second verification node at the input end of the key operator can obtain the input data of the key operator in the specified computing environment, and transmit the input data of the key operator in the specified computing environment to the first verification node at the output end of the key operator. The first verification node can obtain the output data of the key operator in the specified computing environment. In addition, the first verification node can execute the key operator in the standard computing environment based on the input data of the key operator in the specified computing environment to obtain the reference output data corresponding to the key operator. The output data of the key operator in the specified computing environment is compared with the reference output data corresponding to the key operator, and the comparison result corresponding to the key operator can be obtained.

[0151] In this implementation, by inserting a second verification node at the input end of the key operator, the consistency of the input data of the key operator executed in the standard computing environment and the input data of the key operator executed in the specified computing environment is ensured, so that the subsequent output data comparison is more reliable. This implementation tests the behavior of the key operator with the same input conditions in different computing environments, so as to more accurately evaluate and compare the execution accuracy of the key operator.

[0152] In one possible implementation, the at least two computing environments include at least two designated computing environments; and the comparing, through the first verification node, at least two output data of the key operator in the at least two computing environments to obtain the comparison result corresponding to the key operator includes: comparing, through the first verification node, at least two output data of the key operator in the at least two designated computing environments to obtain the comparison result corresponding to the key operator.

[0153] In this implementation, first, at least two different designated computing environments may be determined, and these designated computing environments may include different hardware platforms, operating systems, versions of deep learning frameworks or libraries, etc. The second computing graph may be executed in each designated computing environment, and the output data of the key operators may be collected. Furthermore, the output data of the key operators collected in the at least two designated computing environments by the first verification node may be compared.

[0154] In this implementation, the first verification node can be inserted only at the output end of the key operator, without inserting the second verification node at the input end of the key operator. In this implementation, the second computation graph can be executed in at least two designated computing environments respectively, so that the first verification node corresponding to any key operator can obtain the output data of the key operator from different designated computing environments. The first verification node compares the output data of the key operator from different designated computing environments, and can obtain the comparison result corresponding to the key operator.

[0155] This implementation provides a systematic and automated cross-platform operator accuracy verification process. By executing the second computational graph in multiple specified computing environments and comparing the outputs of key operators, possible execution differences between different platforms can be effectively identified and quantified. This implementation helps ensure the consistency and reliability of deep learning models in different deployment environments, thereby enhancing the portability and robustness of deep learning models.

[0156] In another possible implementation, the at least two computing environments also include a second designated computing environment, and the at least two output data also include the second output data of the key operator in the second designated computing environment; the comparing the at least two output data of the key operator in the at least two computing environments through the first verification node to obtain the comparison result corresponding to the key operator includes: comparing the first output data and the second output data through the first verification node to obtain the comparison result corresponding to the key operator.

[0157] In a possible implementation, after obtaining the comparison result corresponding to the key operator, the method further includes: generating a verification report corresponding to the key operator according to the comparison result corresponding to the key operator; wherein the verification report corresponding to the key operator includes at least part of the following information: information about the computing environment for executing the key operator, input data of the key operator, and output data of the key operator.

[0158] It provides an automatic result analysis function, which can help developers understand the reasons for inconsistent operator execution and provide clear guidance for optimizing and improving the accuracy of deep learning models.

[0159] In a possible implementation, generating a verification report corresponding to the key operator according to the comparison result corresponding to the key operator includes: in response to determining that the key operator does not meet a preset accuracy condition according to the comparison result corresponding to the key operator, generating a verification report corresponding to the key operator.

[0160] In this implementation, a preset accuracy condition can be set for the key operator, and the preset accuracy condition can include standards for accuracy, stability, consistency, etc. of the output data. According to the comparison result corresponding to the key operator, it can be evaluated whether the output data of the key operator meets the preset accuracy condition. If the output of the key operator does not meet the preset accuracy condition, a verification report corresponding to the operator can be automatically generated.

[0161] This implementation helps reduce the workload of manual analysis and makes the verification process more efficient and reliable. This has important practical significance for ensuring the quality and performance of deep learning models and accelerating the development and deployment of models.

[0162] In a possible implementation, generating a verification report corresponding to the key operator according to the comparison result corresponding to the key operator includes: analyzing the cause of the difference corresponding to the key operator based on the comparison result corresponding to the key operator; and generating a verification report corresponding to the key operator based on the cause of the difference.

[0163] In this implementation, the verification report corresponding to the key operator can include not only the detailed execution information of the key operator, but also the possible cause analysis, which provides developers with targeted information to help them locate problems and optimize models.

[0164] In a possible implementation, generating a verification report corresponding to the key operator based on the cause of the difference includes: generating optimization suggestion information based on the cause of the difference; generating a verification report corresponding to the key operator based on the cause of the difference and the optimization suggestion information.

[0165] In this implementation, the verification report corresponding to the key operator can include not only the detailed execution information of the key operator, but also the optimization suggestion information. This implementation helps to reduce the difficulty of deep learning model optimization.

[0166] In a possible implementation, at least one of the structure of the verification node, the data binding method, the topology of the computational graph, etc. can be optimized and adjusted according to the verification results of the key operators to improve the efficiency and accuracy of the verification.

[0167] In a possible implementation, a first computational graph corresponding to a target deep learning model may be constructed by a graph construction module. The first computational graph may be parsed by a graph parsing tool to determine key operators in the target deep learning model. A verification node corresponding to a key operator may be generated by a verification node generator. The output data of the same key operator from different computing environments may be compared by a result comparison module to obtain a comparison result corresponding to the key operator. The result analysis tool may analyze the cause of the difference corresponding to the key operator based on the comparison result corresponding to the key operator, and generate a verification report corresponding to the key operator based on the cause of the difference and the optimization suggestion information. A verification report corresponding to the key operator may be generated by a report generator based on the cause of the difference and the optimization suggestion information.

[0168] In this implementation, these tools and modules work together to provide a comprehensive, end-to-end accuracy verification solution for deep learning models.

[0169] The operator verification method provided in the embodiments of the present disclosure can be applied to technical fields such as verification of computer vision and verification of machine learning algorithms, and is not limited here.

[0170] The following describes the operator verification method provided by the embodiment of the present disclosure through a specific application scenario. Figure 2 A schematic diagram of an operator verification method provided by a disclosed embodiment is shown. In this application scenario, operator calculation verification can be automatically performed under the TensorFlow framework, and specific attention can be paid to the calculation graph characteristics and operations unique to TensorFlow.

[0171] 1. Computational Graph Construction and Intelligent Analysis

[0172] In this application scenario, after determining the target deep learning model, the first computational graph corresponding to the target deep learning model can be constructed. In TensorFlow, the target deep learning model can be represented as a first computational graph (tf.Graph), which can contain all operators (operations) and tensors (data) of the target deep learning model. For example, the first computational graph of the target deep learning model can be obtained through TensorFlow's API (Application Programming Interface), and graph analysis techniques, such as TensorFlow's GraphDef API, can be used to identify key operators in the first computational graph, where key operators may have a greater impact on the final accuracy of the target deep learning model.

[0173] 2. Verification Node Insertion and Computation Graph Reconstruction

[0174] According to the analysis results of the first calculation graph, for each key operator, verification nodes can be inserted before and after it to obtain a second calculation graph. Among them, the second verification node at the input end of the key operator can be used to capture the input data of the key operator, and transmit the input data of the key operator to the first verification node at the output end of the same key operator. The first verification node can be used to capture the output data of the key operator, and verify the key operator based on the input data and output data of the key operator. Among them, the insertion of the verification node can be realized through TensorFlow's tf.custom_gradient to capture its input data and output data when the key operator is executed. At the same time, control flow operations such as tf.cond can also be used to selectively execute additional logic according to the verification results, such as logging or adjusting subsequent calculation paths.

[0175] 3. Automated cross-platform result verification

[0176] During the training or reasoning of the target deep learning model, the second computational graph can be executed in multiple computing environments. TensorFlow provides the tf.device context manager, which allows developers to explicitly specify the execution device of the operator, making it easier to reproduce the execution of the operator on different devices and collect the results.

[0177] 4. Comparison of results and analysis of differences

[0178] The output data of key operators collected from different computing environments can be compared through numerical comparison operations provided by TensorFlow (such as tf.math.reduce_all(tf.math.is_close(...))) to automatically identify precision differences that may occur when executing in different computing environments.

[0179] 5. Report generation and problem location

[0180] If there is no difference in accuracy between key operators in different computing environments, verification success information can be output.

[0181] Once the accuracy of the output data of key operators is found to be different in different computing environments, the logging function of TensorFlow (such as tf.summary) can be used to automatically generate a detailed verification report. The verification report can include the type of key operators, information about the computing environment, information about the difference in input and output data, the cause of the difference, optimization suggestions, etc., to help developers quickly locate and solve problems.

[0182] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0183] In addition, the present disclosure also provides an operator verification device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement any operator verification method provided by the present disclosure. The corresponding technical solutions and technical effects can be found in the corresponding records in the method part and will not be repeated here.

[0184] Figure 3 FIG. 2 shows a block diagram of an operator verification device provided by an embodiment of the present disclosure. Figure 3 As shown, the operator verification device includes:

[0185] A construction module 31 is used to construct a first computational graph corresponding to a target deep learning model;

[0186] An analysis module 32, used to analyze the first computation graph and determine key operators in the target deep learning model;

[0187] A reconstruction module 33, used for inserting a first verification node corresponding to the key operator at the output end of the key operator to obtain a reconstructed second calculation graph;

[0188] An execution module 34 is used to execute the second calculation graph in a first specified computing environment, and obtain the first output data of the key operator in the first specified computing environment through the first verification node;

[0189] The verification module 35 is used to verify the key operator based on the first output data through the first verification node.

[0190] In a possible implementation, the verification module 35 is used to:

[0191] Through the first verification node, at least two output data of the key operator in at least two computing environments are compared to obtain a comparison result corresponding to the key operator, wherein the at least two computing environments include the first designated computing environment, and the at least two output data include the first output data.

[0192] In a possible implementation, the at least two computing environments further include a standard computing environment, and the at least two items of output data further include reference output data of the key operator in the standard computing environment;

[0193] The device also includes:

[0194] An insertion module, used for inserting a second verification node corresponding to the key operator at the input end of the key operator;

[0195] A reference obtaining module, configured to obtain, through the second verification node, first input data of the key operator in the first specified computing environment; transmit the first input data to the first verification node through the second verification node; and execute the key operator based on the first input data in the standard computing environment through the first verification node to obtain the reference output data;

[0196] The verification module 35 is used to compare the first output data with the reference output data through the first verification node to obtain a comparison result corresponding to the key operator.

[0197] In a possible implementation, the at least two computing environments include at least two designated computing environments;

[0198] The verification module 35 is used for:

[0199] At least two output data of the key operator in at least two specified computing environments are compared through the first verification node to obtain a comparison result corresponding to the key operator.

[0200] In a possible implementation, the reconstruction module 33 is used to:

[0201] Bind the first verification node corresponding to the key operator to the output data of the key operator.

[0202] In a possible implementation, the reconstruction module 33 is used to:

[0203] Creating a copy corresponding to the output data stream of the key operator;

[0204] The copy is delivered to a first verification node corresponding to the key operator.

[0205] In a possible implementation, the reconstruction module 33 is used to:

[0206] The first verification node corresponding to the key operator is set to reference the output data of the key operator.

[0207] In a possible implementation manner, the device further includes:

[0208] The first generating module is used to generate a first verification node corresponding to the key operator according to the type of the key operator.

[0209] In a possible implementation, the parsing module 32 is used to:

[0210] Based on graph theory algorithms and data flow analysis, the first computational graph is parsed to determine the key operators of the target deep learning model.

[0211] In a possible implementation manner, the device further includes:

[0212] A second generating module, used to generate a verification report corresponding to the key operator according to the comparison result corresponding to the key operator;

[0213] The verification report corresponding to the key operator includes at least part of the following information: information about the computing environment for executing the key operator, input data of the key operator, and output data of the key operator.

[0214] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. Its specific implementation and technical effects can refer to the description of the above method embodiments. For the sake of brevity, they will not be repeated here.

[0215] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may be a volatile computer-readable storage medium.

[0216] The embodiment of the present disclosure further provides a computer program, including a computer-readable code. When the computer-readable code is executed in an electronic device, a processor in the electronic device executes the above method.

[0217] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in an electronic device, a processor in the electronic device executes the above method.

[0218] An embodiment of the present disclosure also provides an electronic device, comprising: one or more processors; and a memory for storing executable instructions; wherein the one or more processors are configured to call the executable instructions stored in the memory to execute the above method.

[0219] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0220] Figure 4 1 is a block diagram of an electronic device 1900 provided in an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server or a terminal. Figure 4 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0221] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (MacOS X TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or similar.

[0222] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0223] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0224] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0225] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0226] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0227] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0228] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0229] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0230] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0231] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0232] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0233] If the technical solution of the embodiments of the present disclosure involves personal information, the product using the technical solution of the embodiments of the present disclosure has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of the embodiments of the present disclosure involves sensitive personal information, the product using the technical solution of the embodiments of the present disclosure has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to collect his or her personal information; or on the device for processing personal information, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0234] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An operator verification method, characterized in that: include: Construct a first computational graph corresponding to the target deep learning model; Parsing the first computational graph to determine key operators in the target deep learning model; Inserting a first verification node corresponding to the key operator at the output end of the key operator to obtain a reconstructed second calculation graph; Execute the second computation graph in a first specified computing environment, and obtain first output data of the key operator in the first specified computing environment through the first verification node; The key operator is verified by the first verification node based on the first output data.

2. The method according to claim 1, characterized in that The verifying the key operator based on the first output data by the first verification node includes: Through the first verification node, at least two output data of the key operator in at least two computing environments are compared to obtain a comparison result corresponding to the key operator, wherein the at least two computing environments include the first designated computing environment, and the at least two output data include the first output data.

3. The method according to claim 2, characterized in that The at least two computing environments further include a standard computing environment, and the at least two output data further include reference output data of the key operator in the standard computing environment; After determining the key operator in the target deep learning model and before executing the second computation graph in the first designated computing environment, the method further includes: inserting a second verification node corresponding to the key operator at an input end of the key operator; The method further includes: obtaining, through the second verification node, first input data of the key operator in the first specified computing environment; transmitting the first input data to the first verification node through the second verification node; and executing the key operator based on the first input data in the standard computing environment through the first verification node to obtain the reference output data; The comparing at least two output data of the key operator in at least two computing environments through the first verification node to obtain the comparison result corresponding to the key operator includes: comparing the first output data with the reference output data through the first verification node to obtain the comparison result corresponding to the key operator.

4. The method according to claim 2, characterized in that: The at least two computing environments include at least two designated computing environments; The step of comparing at least two output data of the key operator in at least two computing environments by the first verification node to obtain a comparison result corresponding to the key operator includes: At least two output data of the key operator in at least two specified computing environments are compared through the first verification node to obtain a comparison result corresponding to the key operator.

5. The method according to any one of claims 1 to 4, characterized in that: The inserting a first verification node corresponding to the key operator at the output end of the key operator includes: Bind the first verification node corresponding to the key operator to the output data of the key operator.

6. The method according to claim 5, characterized in that The binding the first verification node corresponding to the key operator with the output data of the key operator includes: Creating a copy corresponding to the output data stream of the key operator; The copy is delivered to a first verification node corresponding to the key operator.

7. The method according to claim 5, characterized in that The binding the first verification node corresponding to the key operator with the output data of the key operator includes: The first verification node corresponding to the key operator is set to reference the output data of the key operator.

8. The method according to any one of claims 1 to 4, characterized in that After determining the key operator in the target deep learning model and before inserting the first verification node corresponding to the key operator at the output end of the key operator, the method further includes: According to the type of the key operator, a first verification node corresponding to the key operator is generated.

9. The method according to any one of claims 1 to 4, characterized in that: The parsing the first computation graph to determine the key operators in the target deep learning model includes: Based on graph theory algorithms and data flow analysis, the first computational graph is parsed to determine the key operators of the target deep learning model.

10. The method according to any one of claims 2 to 4, characterized in that After obtaining the comparison result corresponding to the key operator, the method further includes: Generate a verification report corresponding to the key operator according to the comparison result corresponding to the key operator; The verification report corresponding to the key operator includes at least part of the following information: information about the computing environment for executing the key operator, input data of the key operator, and output data of the key operator.

11. An operator verification device, characterized in that: include: A construction module, used to construct a first computational graph corresponding to a target deep learning model; A parsing module, used to parse the first computational graph to determine key operators in the target deep learning model; A reconstruction module, used for inserting a first verification node corresponding to the key operator at the output end of the key operator to obtain a reconstructed second calculation graph; An execution module, configured to execute the second computation graph in a first specified computing environment, and obtain first output data of the key operator in the first specified computing environment through the first verification node; A verification module is used to verify the key operator based on the first output data through the first verification node.

12. An electronic device, characterized in that: include: one or more processors; a memory for storing executable instructions; The one or more processors are configured to call the executable instructions stored in the memory to execute the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising computer readable code, or a non-computer program product carrying computer readable code A volatile computer-readable storage medium, characterized in that When the computer readable code is executed in an electronic device, The processor in the electronic device executes the method according to any one of claims 1 to 10.

Images