Model precision alignment test method and system based on code block and electronic equipment

Abstract

The invention provides a model precision alignment testing method and system based on code blocks and electronic equipment, and relates to the technical field of deep learning testing. The method comprises the following steps: delimiting an execution boundary for a to-be-tested code block through a context manager and monitoring the execution boundary; executing the code block under monitoring based on input data in a reference calculation record prepared for the code block, and automatically capturing a generated forward data set; the forward data set at least comprises an input tensor input into the code block and an intermediate tensor generated in the code block; after the reverse gradient calculation is completed, automatically capturing a gradient data set corresponding to the forward data set; and finally, comparing the captured forward data set and / or gradient data set with corresponding data in the reference calculation record to generate a precision alignment test report. According to the method, fine-grained and low-intrusive precision testing of the calculation process of any code snippets in the deep learning model is realized, and the problems that the testing granularity is too coarse and the flexibility is poor in a traditional method are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning testing, and in particular to a model precision alignment testing method and system based on code blocks and an electronic device. BACKGROUND

[0002] In the development and optimization process of a deep learning model, it is crucial to ensure the consistency of the calculation results of the model on different hardware platforms, computing frameworks or implementation versions (i.e., precision alignment).

[0003] Currently, traditional precision alignment testing methods usually rely on Module Hook technology: first, forward and backward hooks are registered for the target module in the reference framework, and the input and output data of the entire module are captured through the hooks; then the module-level data is manually saved; then the same operation is repeated for the same module in the target framework; finally, the input and output results of the module under the two frameworks are compared. However, this method limits the granularity of testing and verification to the level of complete neural network modules, and cannot penetrate the module boundary to monitor the internal specific calculation steps, making it difficult to accurately locate the problem source when the precision is abnormal; and it cannot perform targeted testing on user-defined complex computing logic segments or continuous operations across modules.

[0004] Therefore, how to provide a precision alignment testing scheme that can perform fine-grained and low-intrusive precision alignment testing on any code segment within a deep learning model has become a technical problem to be solved in the field. SUMMARY

[0005] The present application provides a model precision alignment testing method and system based on code blocks and an electronic device to solve the problem of the existing precision alignment method based on module hooks, which has coarse granularity, poor flexibility, cannot obtain intermediate calculation process data, and cannot support complex logic or cross-module operation testing.

[0006] To solve the above technical problems, the present application provides the following technical solutions: The application provides a code block-based model precision alignment test method, which comprises the following steps: defining an execution boundary for a code block to be tested and monitoring the code block in an execution environment of a target deep learning program by a context manager, the code block being an arbitrary code segment containing a series of continuous deep learning computing operations and being customized by a user according to test requirements; executing the code block in the environment monitored by the context manager based on input data in a reference computing record of the code block, automatically capturing all forward computing data generated in the execution process to form a forward data set, the set at least containing an initial tensor input to the code block and one or more intermediate result tensors generated in each computing step inside the code block; after completion of reverse propagation computing of a loss function associated with the code block, automatically searching and capturing the corresponding gradient of each tensor in the forward data set calculated in the reverse propagation process based on the forward data set to form a gradient data set; and finally, comparing the forward data set and / or the gradient data set with corresponding data in the reference computing record to generate a precision alignment test report.

[0007] According to the code block-based model precision alignment test method provided by the application, the final output tensor of the code block is also captured in the process of automatically capturing the forward data set. Accordingly, the subsequently captured gradient data set will include not only the gradients corresponding to the input tensor and the intermediate tensor, but also the gradient tensor corresponding to the output tensor.

[0008] According to the code block-based model precision alignment test method provided by the application, the process of executing the code block in the monitoring environment and capturing the forward data set comprises the following steps: when the program flow enters the jurisdiction range of the context manager, immediately registering a forward computing listening hook on the computation graph of the deep learning framework; during the subsequent execution of the user-defined code block, the listening hook will intercept and record all tensor operation operations and the data results generated thereby in real time, so as to build the forward data set; when the code block is executed and the program exits the context manager, all the forward data collected is arranged and serialized, and is temporarily stored in the cache, for the subsequent reverse gradient capture.

[0009] According to the code block-based model precision alignment test method provided by the application, in the process of recording the tensor operation by the listening hook, the computing dependency graph between different tensors is analyzed and maintained at the same time, so as to ensure that each tensor in the forward data set can be correctly associated, laying a foundation for the subsequent accurate matching and capturing of the corresponding reverse gradient.

[0010] According to the code block-based model precision alignment test method provided by the application, the trigger timing of capturing the gradient data set is after the loss function reverse propagation process related to the code block calculation is completely ended, the reverse propagation calling event of the framework is listened to, and the gradient capture process is automatically started when it is confirmed that the entire reverse calculation graph has been executed.

[0011] According to the code block-based model precision alignment test method provided by the application, the specific operation of automatically capturing the gradient data set comprises: traversing the previously stored forward data set, accessing each tensor object recorded therein one by one, for each forward tensor, querying whether the gradient has been calculated in the current calculation graph, if there is, collecting the gradient value, and finally collecting the gradient data set corresponding to each forward tensor through traversing all forward tensors.

[0012] According to the code block-based model precision alignment test method provided by the application, the input data in the reference calculation record based on the code block is executed, and the code block comprises the following steps: loading the pre-recorded reference calculation record from the persistent storage, extracting the input tensor data therein; in the test execution environment, the input tensor is dynamically injected into the execution environment of the code block to be tested; then, based on the injected input tensor, the code block is executed.

[0013] According to the code block-based model precision alignment test method provided by the application, the reference calculation record can come from different test scenarios. For example, it can be data recorded when the same code runs in another different deep learning framework, or it can be data recorded when the same framework is used but different calculation configurations are used.

[0014] According to the code block-based model precision alignment test method provided by the application, the reference calculation record is generated by the following steps: executing the code block and capturing the reference forward data set and the reference gradient data set in the reference execution environment; associating and saving the reference forward data set and the reference gradient data set to form the reference calculation record.

[0015] The application provides a code block-based model precision alignment test system, which comprises a context management unit configured to define an execution boundary for a code block to be tested and to monitor the code block in an execution environment of a target deep learning program; wherein the code block is a code segment containing continuous deep learning computing operations and is customized by a user; a code block execution unit configured to execute the code block under the monitoring of the context management unit based on input data in a reference computing record of the code block; a forward capturing unit configured to automatically capture a forward data set generated in the execution process; the forward data set at least comprises an input tensor input to the code block and at least one intermediate tensor generated in the internal computation of the code block; a backward capturing unit configured to automatically capture a gradient data set corresponding to the tensors in the forward data set based on the forward data set after the completion of the backward gradient computation associated with the code block; and a precision comparison unit configured to compare the forward data set and / or the gradient data set with corresponding data in the reference computing record and generate a precision alignment test report.

[0016] According to the code block-based model precision alignment test system provided by the application, the context management unit is generally implemented by using a with statement interface in a programming language. When the user performs the test, the user only needs to wrap the code block to be monitored in a specific with statement block, so that the whole test process can be automatically started without modifying the internal logic of the code block.

[0017] According to the code block-based model precision alignment test system provided by the application, the forward capturing unit further comprises a hook registration subunit and a data caching subunit. The hook registration subunit is responsible for mounting specific hook functions to the deep learning framework when the monitoring context is activated, and the hook functions can intercept all tensor operations performed by the framework. The data caching subunit receives operation data through the hook functions and temporarily stores the operation data in an efficient data structure to form the forward data set.

[0018] According to the code block-based model precision alignment test system provided by the application, the system further comprises a data loading unit configured to load the reference computing record from a persistent storage; and the code block execution unit is specifically configured to inject the input tensor in the reference computing record into the execution environment of the code block and execute the code block under the monitoring of the context management unit based on the input tensor.

[0019] The application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the steps of the code block-based model precision alignment test method according to any one of the above embodiments when executing the program.

[0020] The application provides a computer readable storage medium, which stores a computer program, and the program realizes the steps of the code block-based model precision alignment test method when executed by a processor.

[0021] The code block-based model precision alignment test method, system and electronic equipment provided by the application allow a user to freely define any code segment as a test unit by introducing a context manager, realize a leap in monitoring precision from a module level of coarse granularity to a code block level of fine granularity, and enhance the flexibility of the test; the two-stage capturing mechanism is adopted, the input, intermediate and output tensors are completely recorded through a listening hook during forward calculation, and the corresponding gradient is automatically collected after the backward propagation is completed, so that the intermediate calculation process data that cannot be obtained by a traditional method is obtained, which makes it possible to accurately locate the root cause of the precision deviation; through the dynamic data injection and automatic comparison process, the precision alignment verification under different configurations of the same framework or across deep learning frameworks is supported, the test difficulty of complex calculation logic and cross-module operation is effectively solved, so that the invasiveness of the user code is low, and the user only needs to simply wrap to realize comprehensive testing, and the efficiency and reliability of the deep learning model development and debugging are improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the application or the related art, the following will briefly introduce the drawings needed to be used in the embodiments or the related art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0023] Figure 1 is a flowchart of a code block-based model precision alignment test method provided by the application.

[0024] Figure 2 is a schematic diagram of a precision alignment test process across deep learning frameworks provided by the application.

[0025] Figure 3 is a structural schematic diagram of a code block-based model precision alignment test system provided by the application.

[0026] Figure 4 is a structural schematic diagram of an electronic device provided by the application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor belong to the scope of protection of the present application.

[0028] The development and training of deep learning models usually involve precision alignment verification between reference frameworks (such as PyTorch, TensorFlow, etc.) and target frameworks (such as DeepSpeed, Megatron-LM, etc.).

[0029] In this process, the purpose of precision alignment is to ensure that the forward calculation output and the backward gradient calculation of the model remain consistent under different frameworks or different implementations, so as to guarantee the correctness and reproducibility of model training.

[0030] At the level of traditional precision alignment methods, the module hook technology is usually used. This technology is based on the hook mechanism provided by the deep learning framework, which allows users to register callback functions at the input, output or intermediate layer of the model component. When the model performs forward propagation or backward propagation, the registered hook is triggered, so as to capture and save the input tensor, output tensor or gradient data of the specified module. These captured data are then used as a reference benchmark, and after the same calculation is performed in the target framework, layer-by-layer or module-by-module data comparison is performed to verify the calculation consistency.

[0031] However, the above-mentioned module hook technology has the following obvious defects: Firstly, its monitoring granularity is limited to the complete module level, and it cannot perform isolated testing on user-defined, continuous code segments (i.e. code blocks) inside the module. When the module contains multiple calculation steps or complex control flow, if there is a precision deviation, the traditional method is difficult to accurately locate to the specific calculation step or code line.

[0032] Secondly, the traditional hook can usually only capture the input and final output of the module, and cannot obtain the tensor data generated in the intermediate calculation process, losing the key information for analyzing the precision loss propagation path. Furthermore, this method lacks flexibility, and for calculation logic spanning multiple modules or custom non-standard calculation processes, it is often difficult to effectively deploy hooks or perform complete data capture.

[0033] In addition, the whole process usually involves a lot of manual operations, including registering hooks in different frameworks, saving data and performing subsequent comparison, which has low automation and low efficiency.

[0034] In order to optimize the memory access performance, the application provides a code block-based model precision alignment test method, system and electronic equipment. The context manager is used to demarcate the code block to be monitored, and all forward intermediate results and reverse gradients are captured, so as to realize the precision alignment from the module level to the code block level, and realize the fine-grained, low-invasive and full-precision monitoring of the calculation process of any code segment in the deep learning model.

[0035] Figure 1 is a flowchart of a code block-based model precision alignment test method provided by the application. As shown in Figure 1 , the method comprises the following steps: S101, through a context manager, demarcating an execution boundary for a code block to be tested in an execution environment of a target deep learning program and monitoring.

[0036] The target deep learning program can be a model training program or an inference program developed based on a deep learning framework such as PyTorch, DeepSpeed or Megatron-LM. For example, a convolutional neural network training program for image classification, a Transformer model inference program for natural language processing.

[0037] In the embodiment of the application, the code block is a code segment containing continuous deep learning calculation operations, which is customized by the user.

[0038] In the embodiment of the application, the context manager refers to a program component capable of life cycle management and control of the entire execution process of the target code block. Its core function is to realize non-invasive monitoring of the input, execution process and output of the code block without modifying the internal logic of the code block, and support functions such as forward data capture and reverse gradient association.

[0039] Illustratively, the above-mentioned demarcation of the execution boundary refers to the use of the runtime mechanism of the programming language to clearly define the start and end positions of a piece of code, and to ensure that all related operations executed within this boundary are independently identified and controlled.

[0040] In the embodiment of the application, the context manager is implemented through the syntax features of the programming language. The monitoring function can be enabled by simply wrapping the code block to be tested with the with statement, without the need to modify the calculation process or parameter configuration inside the code block.

[0041] Specifically, the with statement defines a scope range at the syntax level, and this range constitutes the execution boundary of the code block to be monitored. When the program execution flow enters the with block, it is considered to enter the boundary; when the with block is exited, it is considered to leave the boundary.

[0042] In actual execution, the workflow of the context manager is divided into three stages: entering context stage, code block execution stage, and exiting context stage.

[0043] In the entering context stage, the current deep learning framework's computation graph state (e.g., the computation graph construction state in the PyTorch framework) is recorded, data capture hooks are registered (for listening to tensor operations), and data storage structures (such as dictionary-type cache structures) are initialized, while the reference relationship of input tensors is saved to ensure the accuracy of subsequent gradient association.

[0044] In the code block execution stage, all tensor-related computation operations are monitored in real time, and the operations are bound to the target code block to avoid confusion with other code logic tensor operations.

[0045] In the exiting context stage, all output data generated by the code block execution is collected, the intermediate calculation results are sorted and serialized to temporary storage media (such as memory cache or local temporary files), and the reverse gradient data is supplemented and associated.

[0046] It should be noted that the serialization here refers to converting tensor data, operation types, and other information into a structured format (such as JSON format, binary format) that can be stored, facilitating subsequent data loading and comparison. In the subsequent steps, these serialized data will be associated with gradient data to form complete calculation records.

[0047] S102, based on the input data in the reference calculation record of the code block, the code block is executed under the monitoring of the context manager, and the forward data set generated in the execution process is automatically captured.

[0048] In the embodiments of the present application, the forward data set at least includes input tensors input to the code block and at least one intermediate tensor generated by the internal calculation of the code block.

[0049] Optionally, when the forward data set generated in the execution process is automatically captured, the output tensor of the code block can also be captured to form a complete forward data link of input-intermediate-output.

[0050] Illustratively, the input tensor refers to the data tensor injected from the reference calculation record for the current test. For example, in an image classification model, the three-dimensional tensor corresponding to the input 256x256 pixel RGB image (shape [1,3,256,256]) is input.

[0051] The intermediate tensor refers to the temporary result tensor generated by each calculation step in the code block. For example, the feature map tensor obtained after convolution operation, the output tensor processed by the activation function.

[0052] The output tensor refers to the final result tensor output after the code block is executed. For example, the probability distribution tensor of each category in the image classification model (shape [1, 1000]).

[0053] Specifically, the capture of the forward data set is implemented through the forward propagation hook registered by the context manager, which can listen to the call of each tensor operation in the code block, record the type, input and output tensors, and operation sequence of the operation, and ensure that all forward data related to the code block are captured completely.

[0054] For example, if the code block contains consecutive operations of convolution operation, batch normalization, ReLU activation, and pooling operation, the forward propagation hook can capture the input tensor (original feature map) and output tensor (convolution feature map) of the convolution operation, the intermediate tensor after batch normalization, the intermediate tensor after ReLU activation, and the output tensor after pooling operation in sequence, while recording the type identifier (such as conv2d, batch_norm, relu, max_pool2d) and operation sequence of each operation.

[0055] In this way, the present application provides comprehensive basic data for subsequent gradient data association and precision comparison by completely capturing the forward data set, solving the defect that the traditional method cannot obtain the intermediate calculation result.

[0056] In some embodiments, the forward calculation listening hook can be registered when entering the context manager.

[0057] For example, the forward calculation listening hook is a callback function implemented based on the interface provided by the deep learning framework. For example, the register_forward_hook interface in the PyTorch framework, after the hook is registered, it will automatically trigger when each tensor operation in the code block is executed.

[0058] Specifically, when registering the forward calculation listening hook, the trigger logic of the hook is specified: Before the tensor operation is executed, record the input tensor reference and operation configuration parameters (such as convolution kernel size, step, etc.).

[0059] After the tensor operation is executed, record the output tensor, and store the corresponding relationship between the input, operation type, and output in the preset data cache structure.

[0060] It should be noted that the registration process of the hook is automatically completed by the context manager, without the need to manually write hook-related code.

[0061] Further, during the execution of the code block, all tensor operations and their results are recorded through the forward calculation listening hook to form the forward data set.

[0062] Exemplarily, the tensor operation includes arithmetic operations of tensors (such as addition, multiplication), built-in operator operations in a deep learning framework (such as convolution, pooling, activation function, matrix multiplication), and all other calculation behaviors related to tensors; and the recorded results include numerical information of tensors, shapes, data types (such as float32, float64), and operation timestamps.

[0063] Specifically, for each tensor operation, the monitoring hook can store the combined information of operation identification-input tensor list-output tensor-operation parameter-timestamp into the data cache structure.

[0064] The operation identification is composed of a framework operator name and a unique serial number (such as “conv2d_001” and “matmul_002”), which ensures the uniqueness of each operation.

[0065] For example, in the matrix multiplication operation, the shapes of the input two tensors (such as [128, 64] and [64, 256]), the shape of the output tensor (such as [128, 256]), the data type (such as float32), and the timestamp of the operation execution (such as “2025-XX-XX 18:49:33”) are recorded.

[0066] Optionally, when recording all tensor operations, the calculation dependency relationship between tensors can also be maintained.

[0067] Exemplarily, the calculation dependency relationship refers to the generation association between tensors, that is, which input tensors generate a tensor through which operation. For example, the intermediate tensor C is generated by the input tensor A and the tensor B through the matrix multiplication operation, and the dependency relationship is recorded as “A→matmul→C” and “B→matmul→C”.

[0068] Specifically, a unique identification (such as “tensor_001” and “tensor_002”) is assigned to each tensor, and when recording the operation information, the mapping relationship between the input tensor identification and the output tensor identification is recorded synchronously to form a dependency relationship chain.

[0069] For example, the input tensor of the code block is tensor_001, tensor_002 is generated through the convolution operation, tensor_003 is generated through the activation function, and the output tensor tensor_004 is generated through the pooling operation, and the dependency relationship chain is “tensor_001→conv2d→tensor_002→relu→tensor_003→max_pool2d→tensor_004”.

[0070] Thus, the present application can accurately locate the link of precision difference by maintaining the calculation dependency between tensors, and when subsequent comparison finds that the precision is inconsistent, it can trace back to the specific tensor operation along the dependency chain, improving the problem troubleshooting efficiency.

[0071] Further, when exiting the context manager, the forward data set is sorted and temporarily stored.

[0072] For example, the sorting process includes integrating the scattered information in the data cache structure according to the operation order, removing redundant data (such as repeated tensor references), and serializing the tensor data.

[0073] The temporary storage location can be a cache dictionary in memory, a temporary file (such as a.tmp format file) in the local file system, or a database; the temporary storage method is adaptively selected according to the data size, small data size is preferentially stored in memory to improve efficiency, and large data size is stored in files or databases to avoid memory overflow.

[0074] Specifically, the sorted forward data set can be stored in a structured format. For example, in a JSON format file, there are five core fields: code block identifier, iteration number, operation list, tensor list, and dependency relationship.

[0075] The code block identifier is used to distinguish different code blocks to be tested (such as code_block_001); the iteration number is used to identify the current iteration round of model training or inference (such as the 100th iteration); the operation list is used to record the detailed information of each operation; the tensor list is used to record the numerical value, shape, data type and other information of all tensors; and the dependency relationship is used to record the generation association between tensors.

[0076] Thus, the present application sorts and temporarily stores the forward data set when exiting the context, ensuring the integrity and structure of the forward data, laying a foundation for subsequent backward gradient capture and data comparison, and avoiding the problem of data loss or format confusion.

[0077] S103、After the backward gradient calculation associated with the code block is completed, the gradient data set corresponding to each tensor in the forward data set is automatically captured based on the forward data set.

[0078] Optionally, the above-mentioned capturing of the gradient data set can be triggered by the backward propagation call of the loss function, and is performed after the backward computation graph is completed.

[0079] Exemplarily, the loss function is a function for measuring the difference between the model prediction result and the true label (such as a cross-entropy loss function, a mean square error loss function); the back propagation is a process of calculating the gradient of each tensor in reverse through the output value of the loss function; the gradient data reflects the influence degree of each tensor on the loss function; and the reverse computation graph is a computation graph structure built by the deep learning framework for gradient calculation, which contains the path and operation logic of gradient transmission.

[0080] Specifically, the context manager monitors the back propagation call of the loss function (such as the loss.backward() function call in the PyTorch framework), and when detecting that the call is triggered, waits for the reverse computation graph to complete all gradient calculations, ensures that all gradients related to the forward tensor have been generated; at the same time, the integrity of the reverse computation graph is verified, and whether there is a tensor whose gradient has not been calculated (such as the missing gradient caused by the broken computation graph) is checked, and if there is an integrity problem, an exception prompt is triggered to ensure the validity of the gradient data.

[0081] In this way, the present application triggers gradient capture by back propagation call and verifies the integrity of the computation graph, ensuring the correspondence between the gradient data and the forward data, and avoiding the problem of incomplete or invalid gradient capture.

[0082] In some embodiments, all tensors contained in the forward data set can be traversed, and the gradient values calculated in the back propagation of each tensor can be collected to form a gradient data set.

[0083] Exemplarily, the gradient values in the gradient data set correspond one-to-one to the tensors in the forward data set, including the gradient of the input tensor, the gradient of the intermediate tensor, and the gradient of the output tensor, and each gradient value exists in the form of a tensor, whose shape is consistent with that of the corresponding forward tensor.

[0084] Specifically, the traversal process is performed in the order of the tensor list in the forward data set, and the corresponding gradient data is associated through the unique identifier of the tensor.

[0085] For example, for the input tensor, the gradient tensor corresponding to its.grad attribute is directly obtained; for the intermediate tensor, its gradient tensor is traced through the dependency relationship of the reverse computation graph; and for the output tensor, its corresponding gradient tensor (usually the gradient of the output tensor with respect to the loss function) is obtained.

[0086] Further, in the collection process, the dimension matching (such as whether the shape of the gradient tensor is consistent with that of the forward tensor) and the association validity (such as whether there is a forward tensor without corresponding gradient) of the gradient data and the forward data can also be verified, and after the verification, the unique identifier of the gradient data and the forward data is bound to form an association record of “tensor identifier-forward data-gradient data”.

[0087] For example, the input tensor tensor_001 in the forward data set (shape [1, 3, 256, 256]) corresponds to a gradient tensor shape of [1, 3, 256, 256], and the intermediate tensor tensor_002 (shape [1, 64, 128, 128]) corresponds to a gradient tensor shape of [1, 64, 128, 128]. After collection, these gradient tensors are bound to the corresponding tensor identifiers to form a gradient data set.

[0088] In this way, the present application collects gradient data by traversing forward tensors, and verifies dimension matching and correlation effectiveness, ensuring the integrity and accuracy of the gradient data set, and providing reliable data for subsequent cross-framework or cross-configuration gradient precision comparison.

[0089] S104, compare the forward data set and / or the gradient data set with the corresponding data in the reference calculation record to generate a precision alignment test report.

[0090] For example, the comparison method includes element-by-element comparison and statistical level comparison, the comparison object includes the numerical difference of the tensor, the shape consistency, and the data type consistency, and the comparison result is represented by a difference value (such as absolute error, relative error) or a consistency identifier (such as consistent, inconsistent); only the forward data set, only the gradient data set, or both can be compared according to the test requirements.

[0091] Specifically, the element-by-element comparison refers to calculating the difference value (such as absolute error = | new tensor element value - reference tensor element value |) for each element of the two corresponding tensors, and calculating the maximum error, average error, and error distribution of all elements; the statistical level comparison refers to comparing the difference of the statistical characteristics (such as mean, variance, maximum, minimum) of the tensors.

[0092] For example, the element-by-element absolute error of the output tensor in the forward data set captured in the current test and the output tensor in the reference calculation record is compared, and if the maximum absolute error is less than a preset threshold (such as 1e-5), it is determined that the forward precision is consistent.

[0093] For example, the mean difference of the gradient tensors is compared, and if the relative error of the mean is less than a preset threshold (such as 1e-4), it is determined that the gradient precision is consistent.

[0094] For example, for precision verification across code blocks, the precision difference of the associated tensors between different code blocks can also be compared based on the dependency relationship chain, for example, the output tensor of code block A is used as the input tensor of code block B, and the output tensor precision of code block A in the reference environment and the new environment is compared, and the input tensor gradient precision of code block B is compared.

[0095] Therefore, the application can accurately evaluate the precision consistency between different frameworks, different configurations or different code blocks through multi-dimensional and refined comparison methods, solve the problem of single comparison dimension and inaccurate precision evaluation of traditional methods, and accurately locate the source of precision difference based on the comparison of the dependency relationship chain.

[0096] In the model precision alignment test method based on code blocks provided by the application, the context manager is used to define the execution boundary of the code block to be tested and to monitor it, breaking the boundary of traditional module-level monitoring, so that the test granularity can be flexibly and accurately defined by the user to any continuous operation segment; under this monitoring, based on the input data in the reference calculation record of the code block, the forward data set including the input and internal intermediate tensor is executed and automatically captured, and the internal state of the calculation process is recorded; finally, based on the captured forward data set, the corresponding gradient data set is captured after reverse calculation, realizing complete mapping and association of forward calculation and reverse gradient at the same fine granularity, thereby realizing fine-grained and low-invasive precision testing of the calculation process of any code segment in the deep learning model.

[0097] Optionally, in S102, executing the code block under the monitoring of the context manager can include: first loading the reference calculation record from the persistent storage, and injecting the input tensor therein into the execution environment of the code block, and further executing the code block under the monitoring of the context manager based on the injected input tensor.

[0098] In the embodiment of the application, the reference calculation record is a data record obtained by executing automatic testing on the same code block under different deep learning frameworks or different configurations of the same framework.

[0099] For example, the calculation record obtained under the DeepSpeed framework can be used as the reference calculation record for comparison with the test result under the Megatron-LM framework, or the calculation record under the FP32 precision configuration of the same framework can be used as the reference for comparison with the result under the FP16 precision configuration.

[0100] Illustratively, when loading the reference calculation record, the target record is quickly located through the index of the persistent storage according to the query condition (such as code block identifier, iteration number, framework type) specified by the test target, and then the forward data and gradient data in the record are deserialized to restore them to tensor format and structured information.

[0101] For example, if the test target is to verify the precision consistency of the Megatron-LM framework and the DeepSpeed framework, the reference calculation record is the calculation record of the same code block and the same iteration number under the DeepSpeed framework, and the input tensor therein is extracted as the injection data after loading.

[0102] For another example, if the test target is to verify the consistency of different precision configurations under the same framework, the reference calculation record is the calculation record under the FP32 precision configuration, and the input tensor thereof is extracted for injection after being loaded.

[0103] Optionally, the input tensor can be injected into the execution environment of the code block by replacing the original input variable value in the execution environment of the code block before execution of the code block.

[0104] For example, the execution environment of the code block refers to the variable space (such as the local variable space and the global variable space of Python) of the code block at runtime; the original input variable refers to a variable defined in the code block that receives input data (such as x in input_data); and the injection process refers to assigning the input tensor in the reference calculation record to the original input variable to overwrite the original value thereof.

[0105] Specifically, the context manager identifies the original input variable name of the code block (which can be specified by the user or automatically identified through syntax parsing) before execution of the code block, and then converts the loaded reference input tensor into a data type supported by the current execution environment (such as adapting the tensor type of the current framework) and assigns it to the original input variable.

[0106] For example, the original input variable of the code block is x, and the input tensor in the reference calculation record is a float32 tensor under the DeepSpeed framework. In the execution environment of the Megatron-LM framework, the tensor is converted into a tensor format supported by the Megatron-LM and assigned to the variable x to achieve dynamic injection of the input tensor.

[0107] In this way, the present application ensures the consistency of input data under different test scenarios by dynamically injecting the reference input tensor, avoids distortion of the precision comparison result caused by input differences, and maintains the convenience of testing without the need for manual modification of the input variable by the user.

[0108] Optionally, the above-mentioned reference calculation record can be obtained by executing the code block in the reference execution environment.

[0109] In some embodiments, the code block can be executed in the reference execution environment and the reference forward data set and the reference gradient data set can be captured, further associated and saved to form the reference calculation record.

[0110] Exemplarily, the association process binds each forward tensor with its corresponding gradient tensor and operation information based on the unique identification and operation dependency chain of the tensor, forming a complete calculation record; the saved calculation record adopts a structured persistent storage format (such as Parquet format, SQLite database), supports fast query and loading, and at the same time, an index is established to improve data retrieval efficiency.

[0111] Specifically, the calculation record contains four parts of core identification information, forward data details, gradient data details, and dependency relationship information.

[0112] Among them, the core identification information includes code block identification, iteration number, framework type (such as DeepSpeed, Megatron-LM), and timestamp; the forward data details include the numerical value, shape, data type, and operation list of all forward tensors; the gradient data details include the numerical value, shape, and data type of all gradient tensors; and the dependency relationship information includes the dependency chain between tensors.

[0113] Specifically, the established index includes code block identification index, iteration number index, and timestamp index. For example, the corresponding calculation record can be quickly queried and loaded through the code block identification and iteration number. For example, the calculation record of code block code_block_001 in the 100th iteration under the DeepSpeed framework can be directly located and loaded through the index.

[0114] It should be noted that the reference execution environment can be any deep learning framework or configuration. The process of generating the reference calculation record is completely consistent with the principle of capturing the forward and gradient data set described in the foregoing steps S101 to S103, and the difference lies in the execution purpose and subsequent use of generating reference data.

[0115] In order to more intuitively show the whole process of the above data capture, storage, loading and comparison, the embodiment of the application also provides a precision alignment test process across deep learning frameworks.

[0116] Figure 2 is a schematic diagram of a precision alignment test process across deep learning frameworks provided by the application.

[0117] As shown in Figure 2 , the whole process involves two different deep learning frameworks or running environments, namely the first framework NV and the second framework BR. The process mainly includes four core stages of data capture (Dump), data loading (Load), data storage (Save) and result comparison (Comparison).

[0118] Exemplarily, in the data capture (Dump) stage, the target code block is first executed in the first framework NV.

[0119] Specifically, in the forward capture dump mode (capture_forward_dump), the code block performs forward computation under the monitoring of the context manager, and captures a forward data set; subsequently, after the completion of the backward propagation gradient computation associated with the code block, in the backward capture dump mode (capture_backward_dump), a gradient data set corresponding to the forward data is captured.

[0120] Then, the captured forward data set and gradient data set are associated and serialized, and stored as a structured reference data.pt file , which serves as reference data for subsequent precision comparison.

[0121] Exemplarily, in the data loading (Load) and result comparison (Comparison) stages, the second framework (BR) environment is switched to, in which the reference data.pt file generated in the foregoing steps is first loaded from the persistent storage.

[0122] Specifically, in the forward capture load mode (capture_forward_load), the input tensor in the reference data is dynamically injected into the current execution environment, and then the code block performs forward computation and captures new forward data; similarly, after the completion of the backward propagation gradient computation, the new gradient data is captured in the backward capture load mode (capture_backward_load), and a to-be-tested data.pt file (such as file) under the second framework is generated.

[0123] Finally, through the result comparison (Comparison) module, the reference data.pt file from the first framework and the to-be-tested data.pt file from the second framework are compared and analyzed item by item, and a precision difference report is generated.

[0124] In this way, the present application realizes a complete closed loop of generating reference data in one framework or environment and automatically testing and verifying in another framework or environment, and is suitable for precision regression testing in scenarios such as model migration, framework switching, hardware adaptation, or operator optimization.

[0125] The code block-based model precision alignment test system provided by the present application is described below, and the code block-based model precision alignment test system described below can be mutually referred to the code block-based model precision alignment test method described above.

[0126] Figure 3A structural schematic diagram of a code block-based model precision alignment test system is provided for an embodiment of the present application. The code block-based model precision alignment test system comprises a context management unit 310, a code block execution unit 320, a forward capturing unit 330, a backward capturing unit 340, and a precision comparison unit 350.

[0127] The context management unit 310 is configured to define an execution boundary for the code block to be tested and perform monitoring in an execution environment of a target deep learning program; the code block is a code segment containing continuous deep learning computing operations and is customized by a user; the code block execution unit 320 is configured to execute the code block under the monitoring of the context management unit based on input data in a reference computing record of the code block; the forward capturing unit 330 is configured to automatically capture a forward data set generated in the execution process; the forward data set at least includes an input tensor input to the code block and at least one intermediate tensor generated in the internal computation of the code block; the backward capturing unit 340 is configured to automatically capture a gradient data set corresponding to the tensors in the forward data set based on the forward data set after the completion of the backward gradient calculation associated with the code block; and the precision comparison unit 350 is configured to compare the forward data set and / or the gradient data set with corresponding data in the reference computing record to generate a precision alignment test report.

[0128] In some embodiments, the context management unit 310 is implemented through a with statement interface in a programming language, and the automatic test is started by wrapping the code block in a context management range defined by the with statement.

[0129] In some embodiments, the forward capturing unit 330 comprises a hook registration subunit and a data caching subunit; the hook registration subunit is configured to mount a hook function for intercepting tensor operations when the execution environment is activated; and the data caching subunit is configured to collect and temporarily store the forward data set through the hook function.

[0130] In some embodiments, the code block-based model precision alignment test system further comprises a data loading unit configured to load the reference computing record from a persistent storage; and the code block execution unit 320 is specifically configured to inject the input tensor in the reference computing record into the execution environment of the code block, and execute the code block under the monitoring of the context management unit based on the input tensor.

[0131] In the code block-based model precision alignment test system provided by the application, a temporary monitoring context is created and managed by a context management unit through a programming interface (such as a with statement) in a runtime environment, so that any custom code segment can be defined as an independent monitoring object in a very low-intrusive manner; in this context, a forward capture unit intercepts and records all tensor operations through a hooked function, and deeply accesses the intermediate calculation state that cannot be accessed by traditional methods to form a fine-grained forward data set; a backward capture unit automatically associates and captures the corresponding gradient based on the accurate data after the backward propagation is completed, ensuring that the forward calculation and the backward gradient form a complete and traceable data pair at the same fine granularity, thereby realizing fine-grained and low-intrusive precision testing of the calculation process of any code segment in a deep learning model.

[0132] Figure 4 An example of an entity structure diagram of an electronic device is shown in Figure 4 The electronic device can include one or more processors 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can invoke logical instructions in the memory 430 to execute a code block-based model precision alignment test method, which includes: defining an execution boundary for a code block to be tested by a context manager and monitoring; based on the input data in the reference calculation record prepared for the code block, executing the code block under monitoring and automatically capturing the generated forward data set; the forward data set at least includes the input tensor input to the code block and the intermediate tensor generated therein; after the reverse gradient calculation is completed, the corresponding gradient data set of the forward data set is automatically captured; finally, the captured forward data set and / or gradient data set is compared with the corresponding data in the reference calculation record to generate a precision alignment test report.

[0133] In addition, the logic instructions in the memory 430 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0134] In another aspect, the present application also provides a computer program product, which includes a computer program, the computer program can be stored on a computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the code block based model precision alignment test method provided in the above embodiments. The method includes: defining an execution boundary for a code block to be tested and monitoring by a context manager; executing the code block under monitoring based on input data in a reference calculation record prepared for the code block, and automatically capturing a generated forward data set; the forward data set at least includes an input tensor input to the code block and an intermediate tensor generated inside; after the reverse gradient calculation is completed, the corresponding gradient data set of the forward data set is automatically captured; finally, the captured forward data set and / or gradient data set are compared with the corresponding data in the reference calculation record to generate a precision alignment test report.

[0135] In another aspect, the present application also provides a computer readable storage medium, which stores a computer program, the computer program is executed by a processor to implement the code block based model precision alignment test method provided in the above embodiments. The method includes: defining an execution boundary for a code block to be tested and monitoring by a context manager; executing the code block under monitoring based on input data in a reference calculation record prepared for the code block, and automatically capturing a generated forward data set; the forward data set at least includes an input tensor input to the code block and an intermediate tensor generated inside; after the reverse gradient calculation is completed, the corresponding gradient data set of the forward data set is automatically captured; finally, the captured forward data set and / or gradient data set are compared with the corresponding data in the reference calculation record to generate a precision alignment test report.

[0136] It should be noted that the computer readable storage medium can be a non-transitory computer readable storage medium or a transitory computer readable storage medium, which is not limited here.

[0137] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0138] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0139] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A code block based model precision alignment test method, characterized by, The method comprises: By a context manager, defining execution boundaries and monitoring for a code block to be tested in an execution environment of a target deep learning program; wherein the code block is a code segment containing continuous deep learning computing operations and is user-defined; Based on input data in a reference computing record of the code block, executing the code block under the monitoring of the context manager, and automatically capturing a forward data set generated in the execution process; the forward data set at least includes an input tensor input to the code block and at least one intermediate tensor generated by internal computing of the code block; After completion of backward gradient calculation associated with the code block, based on the forward data set, automatically capturing a gradient data set corresponding to the tensors in the forward data set respectively; Comparing the forward data set and / or the gradient data set with corresponding data in the reference computing record to generate an accuracy alignment test report.

2. The method of claim 1, wherein, When the forward data set generated in the execution process is automatically captured, it further comprises: Capturing the output tensor of the code block; the gradient data set includes gradient tensors corresponding to the input tensor, the at least one intermediate tensor and the output tensor.

3. The method according to claim 1 or 2, characterized in that, The execution of the code block under the monitoring of the context manager and the automatic capture of the forward data set generated in the execution process comprise: When entering the context manager, register a forward calculation listening hook; During the execution of the code block, record all tensor operations and their results through the forward calculation listening hook to form the forward data set; When exiting the context manager, arrange and temporarily store the forward data set.

4. The method of claim 3, wherein, When recording all tensor operations, it further comprises: Maintain the calculation dependency relationship between tensors.

5. The method of claim 1, wherein, After completion of the backward calculation graph, the automatic capture of the gradient data set corresponding to the tensors in the forward data set respectively based on the forward data set after completion of the backward gradient calculation associated with the code block comprises: Triggered by the backward propagation call of the loss function, and after the completion of the backward calculation graph, the automatic capture of the gradient data set corresponding to the tensors in the forward data set respectively.

6. The method of claim 5, wherein, The automatic capture of the gradient data set corresponding to the tensors in the forward data set respectively comprises: Traverse all tensors contained in the forward data set; Collect the gradient values calculated in the backward propagation of each tensor to form the gradient data set.

7. The method of claim 1, wherein, The execution of the code block under the monitoring of the context manager based on the input data in the reference computing record of the code block comprises: Load the reference computing record from the persistent storage and inject the input tensor therein into the execution environment of the code block; Based on the injected input tensor, execute the code block under the monitoring of the context manager.

8. The method of claim 1, wherein, The reference computing record is a data record obtained by performing automatic testing on the same code block under different deep learning frameworks or different configurations of the same framework.

9. The method of claim 1, wherein, The reference computing record is generated by the following steps: In a reference execution environment, execute the code block and capture a reference forward data set and a reference gradient data set; The reference forward data set is associated with the reference gradient data set and saved to form the reference computation record.

10. A code block based model precision alignment testing system, comprising: The system comprises: a context management unit configured to define an execution boundary for a code block to be tested and to monitor the code block in an execution environment of a target deep learning program, wherein the code block is a code segment containing continuous deep learning computation operations and is customized by a user; a code block execution unit configured to execute the code block under the monitoring of the context management unit based on input data in a reference computation record of the code block; a forward capturing unit configured to automatically capture a forward data set generated in an execution process, wherein the forward data set at least includes an input tensor input to the code block and at least one intermediate tensor generated in an internal computation of the code block; a backward capturing unit configured to automatically capture a gradient data set corresponding to the tensors in the forward data set based on the forward data set after a backward gradient computation associated with the code block is completed; a precision comparison unit configured to compare the forward data set and / or the gradient data set with corresponding data in the reference computation record to generate a precision alignment test report.

11. The system of claim 10, wherein, The context management unit is implemented through a with statement interface in a programming language, and the automated testing is started by wrapping the code block in a context management range defined by the with statement.

12. The system of claim 10 or 11, wherein, The forward capturing unit comprises a hook registration subunit and a data caching subunit. The hook registration subunit is configured to mount a hook function for intercepting tensor operations when the execution environment is activated. The data caching subunit is configured to collect and temporarily store the forward data set through the hook function.

13. The system of claim 10, wherein, The system further comprises: a data loading unit configured to load a reference computation record from a persistent storage; The code block execution unit is specifically configured to inject an input tensor in the reference computation record into an execution environment of the code block, and execute the code block under the monitoring of the context management unit based on the input tensor.

14. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the method in any one of claims 1 to 9.

15. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the method in any one of claims 1 to 9.

Images