Precision loss positioning method, device and equipment

Abstract

The invention provides a precision loss positioning method, device and equipment. The method comprises the following steps: acquiring test data and a network model of a target deep learning model in a conversion process; wherein the network model comprises a first model of a first framework and a second model of a second framework, and the target deep learning model comprises a plurality of neural network modules; respectively inputting the test data into the first model and the second model to obtain intermediate layer output results respectively corresponding to the first model and the second model; determining an abnormal module from the plurality of neural network modules based on intermediate layer output results corresponding to the first model and the second model; wherein the exception module comprises a plurality of variables; and positioning an abnormal position of the first model according to the first variable information of the plurality of variables when the abnormal module operates on the first frame and the second variable information of the plurality of variables when the abnormal module operates on the second frame. The abnormal position in the deep learning model is automatically and quickly positioned, and the model debugging efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of deep learning technology, and in particular to a method, device and equipment for locating precision loss. Background Art

[0002] When migrating deep learning models from one framework to another, such as from a training framework like PyTorch to an inference engine like TensorRT, model accuracy may be lost. However, this loss of accuracy can significantly degrade the model's performance in production environments. For example, in autonomous driving scenarios, insufficient quantization accuracy in object detection models can reduce obstacle recognition accuracy, posing a safety hazard. In medical imaging diagnosis, even slight deviations in accuracy can directly impact lesion localization.

[0003] Therefore, accurately and efficiently locating the source of accuracy loss is a crucial step in the model migration process. Only by quickly identifying the root cause can targeted remediation measures be implemented to ensure the reliability and performance stability of the model in real-world applications. Summary of the Invention

[0004] The present disclosure aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] To this end, the purpose of the present disclosure is to propose a precision loss positioning method, device and equipment, which can automatically and quickly locate abnormal positions in deep learning models without the need for manual layer-by-layer checking of model codes or blind debugging of model parameters, thereby helping to shorten the model debugging cycle, improve model debugging efficiency, and effectively reduce dependence on relevant debugging personnel and reduce labor costs.

[0006] To achieve the above-mentioned purpose, the precision loss positioning method proposed in the first embodiment of the present disclosure includes: Acquire test data and a network model of a target deep learning model during the conversion process; wherein the network model includes a first model of a first framework and a second model of a second framework, and the target deep learning model includes multiple neural network modules; Inputting the test data into the first model to obtain a first intermediate layer output result of the first model, and inputting the test data into the second model to obtain a second intermediate layer output result of the second model; Based on the output result of the first intermediate layer and the output result of the second intermediate layer, determining an abnormal module from the multiple neural network modules; wherein the abnormal module includes multiple variables; The abnormal position of the first model is located according to the first variable information of the multiple variables when the abnormal module runs on the first framework and the second variable information of the multiple variables when the abnormal module runs on the second framework.

[0007] To achieve the above-mentioned purpose, the second embodiment of the present disclosure provides a precision loss positioning device, comprising: A first acquisition module is configured to acquire test data and a network model of a target deep learning model during the conversion process; wherein the network model includes a first model of a first framework and a second model of a second framework, and the target deep learning model includes multiple neural network modules; an input module, configured to input the test data into the first model to obtain a first intermediate layer output result of the first model, and input the test data into the second model to obtain a second intermediate layer output result of the second model; a determination module, configured to determine an abnormal module from the plurality of neural network modules based on the output result of the first intermediate layer and the output result of the second intermediate layer; wherein the abnormal module includes a plurality of variables; A positioning module is used to locate the abnormal position of the first model according to the first variable information of the multiple variables when the abnormal module runs on the first framework and the second variable information of the multiple variables when the abnormal module runs on the second framework.

[0008] The electronic device proposed in the third embodiment of the present disclosure includes: a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements the accuracy loss positioning method proposed in the first embodiment of the present disclosure.

[0009] The fourth embodiment of the present disclosure proposes a non-temporary computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the accuracy loss positioning method proposed in the first embodiment of the present disclosure is implemented.

[0010] The fifth embodiment of the present disclosure proposes a computer program product. When the instructions in the computer program product are executed by a processor, the accuracy loss positioning method proposed in the first embodiment of the present disclosure is executed.

[0011] The accuracy loss positioning method, device and equipment provided by the present disclosure obtain test data and the network model of the target deep learning model during the conversion process; wherein the network model includes a first model of a first framework and a second model of a second framework, and the target deep learning model includes multiple neural network modules; input the test data into the first model to obtain the first intermediate layer output result of the first model, and input the test data into the second model to obtain the second intermediate layer output result of the second model; based on the first intermediate layer output result and the second intermediate layer output result, determine the abnormal module from the multiple neural network modules; wherein the abnormal module includes multiple variables; according to the first variable information of the multiple variables when the abnormal module runs on the first framework and the second variable information of the multiple variables when the abnormal module runs on the second framework, locate the abnormal position of the first model. In this way, the abnormal position in the deep learning model can be automatically and quickly located without manually checking the model code layer by layer or blindly debugging the model parameters, effectively shortening the model debugging cycle, thereby effectively improving the model debugging efficiency, and effectively reducing the dependence on relevant debugging personnel and reducing labor costs. As a result, the abnormal positions in the deep learning model can be located automatically and quickly without manually checking the model code layer by layer or blindly debugging the model parameters. This helps shorten the model debugging cycle, improve model debugging efficiency, and effectively reduce dependence on relevant debugging personnel, reducing labor costs.

[0012] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and / or additional aspects and advantages of the present disclosure will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which: Figure 1 1 is a flow chart of a precision loss location method proposed in an embodiment of the present disclosure; Figure 2 is a flow chart of a precision loss positioning method proposed in another embodiment of the present disclosure; Figure 3 1 is a flowchart of the process of locating the accuracy loss of deep learning model conversion proposed in the present disclosure; Figure 4 It is a structural diagram of a precision loss positioning device proposed in one embodiment of the present disclosure; Figure 5 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0014] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present disclosure and are not to be construed as limiting the present disclosure. On the contrary, the embodiments of the present disclosure include all variations, modifications, and equivalents that fall within the spirit and scope of the appended claims.

[0015] The present disclosure provides a method, device and apparatus for locating precision loss.

[0016] Figure 1 1 is a flow chart of a precision loss positioning method proposed in an embodiment of the present disclosure.

[0017] It should be noted that the executor of the precision loss positioning method of this embodiment is a precision loss positioning device, which can be implemented by software and / or hardware. The device can be configured in an electronic device, which may include but is not limited to a terminal, a server, etc.

[0018] like Figure 1 As shown, the precision loss positioning method includes: Step S101: Acquire test data and the network model of the target deep learning model during the conversion process.

[0019] Among them, the target deep learning model may include multiple neural network modules (also called neural network layers).

[0020] The network model may include a first model of a first framework and a second model of a second framework.

[0021] It should be noted that the first framework and the second framework are different frameworks. The first model can be a network model built and trained on the first framework for the target deep learning model, and the second model can be a network model built on the second framework for the target deep learning model. The first framework can be, but is not limited to, PyTorch, and the second framework can be, but is not limited to, TensorRT.

[0022] The test data can be used to test the target deep learning model, and this disclosure does not limit the amount of test data.

[0023] Step S102: input the test data into the first model to obtain the first intermediate layer output result of the first model, and input the test data into the second model to obtain the second intermediate layer output result of the second model.

[0024] Optionally, in some embodiments, the first intermediate layer output result and the second intermediate layer output result may both include module information corresponding to the corresponding output results, and the module information may be used to indicate the neural network module to which the corresponding output result belongs.

[0025] In an embodiment of the present disclosure, test data can be input into the first model and the second model respectively, so that the first model runs on the first frame and the second model runs on the second frame, thereby obtaining the intermediate layer output results of each neural network module in the first model (referred to as the first intermediate layer output results in the present disclosure), and the intermediate layer output results of each neural network module in the second model (the second intermediate layer output results).

[0026] Among them, it should be noted that the intermediate layer output result of each neural network module when running on the corresponding framework can be a multi-dimensional value or multiple values, or it can be a scalar (that is, a single value). Therefore, when the intermediate layer output result when the neural network module runs on the corresponding framework is a multi-dimensional value or multiple values, the first coefficient corresponding to the neural network module is also multiple. When the intermediate layer output result when the neural network module runs on the corresponding framework is a single value, the first coefficient corresponding to the neural network module is also single. That is to say, the number of the first coefficients of the neural network module is related to the amount of data included in the intermediate layer output result when the neural network module runs on the corresponding framework.

[0027] Step S103: Based on the output results of the first intermediate layer and the second intermediate layer, determine an abnormal module from multiple neural network modules; wherein the abnormal module includes multiple variables.

[0028] It should be noted that the present disclosure does not impose any restrictions on the input and output types of variables, that is, variables can be either input variables or output variables.

[0029] Optionally, in some embodiments, for any neural network module, a first coefficient corresponding to the neural network module is determined based on the difference between the first intermediate layer output result when the neural network module is running on a first framework and the second intermediate layer output result when the neural network module is running on a second framework; and based on the first coefficient corresponding to each neural network module, an abnormal module is determined from multiple neural network modules.

[0030] That is, for any neural network module, the first coefficient corresponding to the neural network module is determined based on the difference between the intermediate layer output results when the neural network module runs on the first framework and the second framework respectively. Furthermore, based on the first coefficient corresponding to each neural network module, the abnormal module can be determined from multiple neural network modules.

[0031] As an example, for any neural network module, the absolute error is determined based on the difference between the intermediate layer output result when the neural network module runs on the first framework and the intermediate layer output result when it runs on the second framework, and the absolute error is determined as the first coefficient corresponding to the neural network module; furthermore, the neural network module whose first coefficient is greater than the corresponding set threshold can be determined as an abnormal module.

[0032] The threshold value may be pre-set, and the present disclosure does not limit the value of the threshold value. It should also be noted that different first coefficients may have the same threshold value or different threshold values, and the present disclosure does not limit this.

[0033] It is understood that each neural network module has a corresponding execution order (also known as a run order). Therefore, in one possible implementation of the disclosed embodiment, the first coefficients corresponding to each neural network module can be analyzed sequentially according to the execution order of the multiple neural network modules, and a target coefficient can be determined from the multiple first coefficients; based on the target coefficient, an abnormal module can be identified.

[0034] As an example, the first coefficient of any neural network module can be compared with a set threshold corresponding to the first coefficient according to the execution order of multiple neural network modules, and the first coefficient among the first coefficients corresponding to the multiple neural network modules that is greater than the corresponding set threshold is determined as the target coefficient, and the neural network module to which the first coefficient belongs is determined as the abnormal module. In other words, the first neural network module among the multiple neural network modules whose first coefficient is greater than the corresponding set threshold is determined as the abnormal module. For example, assuming there are three neural network modules, namely module one, module two, and module three, and the execution order is module one → module two → module three, the first coefficients corresponding to module one include coefficient one and coefficient two, the first coefficients corresponding to module two include coefficient three, and the first coefficients corresponding to module three include coefficient four, coefficient five, and coefficient six. The first coefficient corresponding to module one is first analyzed. Assuming that coefficient one and coefficient two are not greater than the corresponding set threshold, the first coefficient corresponding to module two is then analyzed. Assuming that coefficient three is greater than the corresponding set threshold, module two is the first neural network module among the three neural network modules whose first coefficient is greater than the corresponding set threshold, and module two is determined as the abnormal module.

[0035] Step S104 , locating the abnormal position of the first model according to the first variable information of the multiple variables when the abnormal module runs on the first frame and the second variable information of the multiple variables when the abnormal module runs on the second frame.

[0036] Optionally, in some embodiments, the exception module may include multiple sub-steps, wherein the sub-steps may be used to instruct to perform corresponding mathematical or logical operations on corresponding input data.

[0037] As an example, assuming that the first framework is PyTorch and the second framework is TensorRT, you can use a tracing function (such as the sys.settrace function) to trace and capture the first variable information of multiple variables when the exception module is running on PyTorch, and you can call the TensorRT framework's command line tool trtexec to print the second variable information of multiple variables when the exception module is running on TensorRT.

[0038] The accuracy loss positioning method of the embodiment of the present disclosure obtains test data and a network model of a target deep learning model during the conversion process; wherein the network model includes a first model of a first framework and a second model of a second framework, and the target deep learning model includes multiple neural network modules; the test data is input into the first model to obtain the first intermediate layer output result of the first model, and the test data is input into the second model to obtain the second intermediate layer output result of the second model; based on the first intermediate layer output result and the second intermediate layer output result, an abnormal module is determined from multiple neural network modules; wherein the abnormal module includes multiple variables; according to the first variable information of the multiple variables when the abnormal module is running on the first framework and the second variable information of the multiple variables when the abnormal module is running on the second framework, the abnormal position of the first model is located. In this way, the abnormal position in the deep learning model can be automatically and quickly located without manually checking the model code layer by layer or blindly debugging the model parameters, thereby helping to shorten the model debugging cycle, improve the model debugging efficiency, and effectively reduce the dependence on relevant debugging personnel and reduce labor costs.

[0039] Figure 2 It is a flowchart of a precision loss positioning method proposed in another embodiment of the present disclosure.

[0040] like Figure 2 As shown, the precision loss positioning method includes: Step S201: Acquire test data and the network model of the target deep learning model during the conversion process.

[0041] Step S202: input the test data into the first model to obtain the first intermediate layer output result of the first model, and input the test data into the second model to obtain the second intermediate layer output result of the second model.

[0042] It should be noted that the execution process of step S201 to step S202 can refer to the execution process of any embodiment of the present disclosure, and will not be repeated here.

[0043] Optionally, in some embodiments, a forward hook may be registered for each neural network module in the first model; based on the test data, the registered first model is run in the first framework to obtain a first intermediate layer output result.

[0044] That is to say, a forward hook is registered for each neural network module in the first model, and the test data is input into the registered first model so that the registered first model runs on the first framework to obtain the first intermediate layer output result of the first model.

[0045] It should be noted that by registering a forward hook for each neural network module in the first model, the corresponding intermediate layer output results are captured from the corresponding neural network module through the forward hook tool.

[0046] Optionally, in some embodiments, the first model is formatted to obtain an intermediate representation model; based on the intermediate layer output results of the corresponding neural network module in the first model, the output items of the intermediate representation model are expanded to obtain an expanded intermediate representation model; based on the test data, the expanded intermediate representation model as the second model is run on the second framework to obtain the second intermediate output layer information.

[0047] As an example, assuming that the first framework is PyTorch and the second framework is TensorRT, the trained model (referred to as the first model in this disclosure) can be converted into an ONNX model running on the ONNX (Open Neural Network Exchange) framework, and the model is determined as the intermediate representation model; according to the output item name corresponding to the intermediate layer output result of the corresponding neural network module in the first model, the ONNX model code is modified, and the output item name corresponding to the intermediate layer output result of the corresponding neural network module in the first model is added as the output item or output node of the ONNX model to obtain an extended intermediate representation model, and the extended intermediate representation model is used as the second model; the test data is input as the extended intermediate representation model of the second model, so that the second model runs on the second framework TensorRT to obtain the second intermediate output layer information.

[0048] Step S203: Based on the output results of the first intermediate layer and the second intermediate layer, determine an abnormal module from multiple neural network modules; wherein the abnormal module includes multiple variables.

[0049] It should be noted that the execution process of step S203 can refer to the execution process of any embodiment of the present disclosure, and will not be described in detail here.

[0050] In the embodiment of the present disclosure, the exception module may include multiple sub-steps.

[0051] Step S204: for any variable, compare the first variable information and the second variable information corresponding to the variable to determine the difference of the variable.

[0052] The first variable information and the second variable information may include variable values of corresponding variables.

[0053] As an example, for any variable, the absolute error between the variable value in the first variable information and the variable value in the second variable information corresponding to the variable is calculated, and the absolute error is determined as the difference in the variable.

[0054] Step S205: determining a target sub-step from the plurality of sub-steps according to the difference between the variables.

[0055] As an example, the difference values of the variables belonging to each sub-step can be analyzed in sequence according to the execution order of multiple sub-steps, and the sub-step to which the variable whose difference value of the first variable appears in the multiple sub-steps is greater than the corresponding set threshold belongs is determined as the target sub-step.

[0056] As another example, a target variable can be determined from multiple variables based on the difference between the variables, and the sub-step to which the target variable belongs (or matches, or is associated) can be determined as the target sub-step. For example, a variable whose difference between the variables is greater than a set threshold can be determined as the target variable, and the sub-step to which the variable belongs can then be determined as the target sub-step.

[0057] As another example, for any sub-step, the target coefficient of the sub-step can be determined based on the difference of at least one variable associated with or involved in the sub-step. Then, the error change rate of the target coefficient of any sub-step can be determined in sequence according to the execution order of each sub-step; based on the error change rate corresponding to each sub-step, the target sub-step is determined from multiple sub-steps, and the target sub-step is determined as an abnormal position.

[0058] As an example, for any sub-step, a target value can be determined from the difference of at least one variable associated with or involved in the sub-step, for example, the maximum value of the difference of at least one variable associated with or involved in the sub-step is determined as the target value; then, based on the target value, the target coefficient of the sub-step is determined, for example, the relative error of the variable is determined by the ratio of the target value to the variable value of the corresponding variable in the first variable information, and the relative error is determined as the target coefficient of the sub-step; further, the error change rate of the target coefficient of any sub-step can be determined according to the execution order of each sub-step, for example, for any sub-step, based on the target coefficient of the previous sub-step of the sub-step and the target coefficient of the sub-step, the error change rate of the sub-step can be determined according to the following formula: ; (1) Among them, t represents the execution order of the sub-step, t-1 represents the execution order of the previous step of the sub-step, Mt represents the target coefficient of the sub-step, Mt-1 represents the target coefficient of the previous sub-step of the sub-step, and Ct represents the error change rate of the sub-step.

[0059] Finally, based on the error change rate of each sub-step, the target sub-step can be determined from multiple sub-steps. For example, the sub-step with the largest error change rate can be determined as the target sub-step; or, according to the execution order of each sub-step, the first sub-step among the multiple sub-steps whose error change rate is greater than the corresponding set change rate threshold can be determined as the target sub-step; for example, according to the execution order of each sub-step, the first sub-step among the multiple sub-steps whose associated variable difference is greater than the corresponding set difference threshold and whose error change rate is greater than the corresponding set change rate threshold can be determined as the target sub-step.

[0060] The difference threshold value may be pre-set, and the present disclosure does not limit its value. It should also be noted that the difference threshold values of different variables may be the same or different, and the present disclosure does not limit this.

[0061] The change rate threshold may be preset, and the present disclosure does not impose any restrictions on its value.

[0062] Step S206: Based on the target sub-step, determine the abnormal location.

[0063] As an example, the target sub-step may be determined as an abnormal location.

[0064] Optionally, in some embodiments, at least one variable associated with the abnormal position can be analyzed to determine a target error pattern corresponding to the abnormal position from multiple preset error patterns; obtain a repair rule that matches the target error pattern; based on the repair rule, generate repair prompt information, and send the prompt information to the user.

[0065] The preset error mode may include, but is not limited to: all-zero output, gradual error accumulation, sudden large error, etc., and the present disclosure does not impose any restrictions on this.

[0066] The repair rule may be used to indicate a method or logic for repairing the corresponding abnormal location.

[0067] As an example, when the variable values of all the variables belonging to the output type in at least one variable associated with the target sub-step or the abnormal position are all 0, the target error mode corresponding to the abnormal position is determined to be an all-zero output; when the error change rate of the sub-step corresponding to the abnormal position is determined based on at least one variable associated with the sub-step corresponding to the abnormal position, and the error value change rate of at least one sub-step adjacent to the execution order of the abnormal position is obtained, when the error change rate is analyzed according to the execution order of the sub-step corresponding to the abnormal position and at least one sub-step adjacent to the execution order of the abnormal position, when the error change rate gradually increases with the above execution order, the target error mode corresponding to the abnormal position is determined to be a gradual error accumulation, etc., which will not be repeated. Furthermore, based on the target error mode, the corresponding abnormal cause can be queried, and based on the abnormal cause, the repair rule corresponding to the abnormal cause can be determined; finally, prompt information can be generated and sent according to the repair rule, such as prompting the user of the abnormal position of the first model, the target error mode, the method or manner used to repair the first model, etc.

[0068] It should be noted that the above method for determining the target error pattern corresponding to the abnormal position is only exemplary. In practical applications, other methods may be used, and the present disclosure does not impose any limitation on this.

[0069] It can be understood that after the abnormal position in the first model is repaired, the repaired first model can be tested. Optionally, in some embodiments, after the abnormal position in the first model is repaired, the repaired first model can be backtested and analyzed to evaluate the actual effect of the accuracy recovery of the repaired first model.

[0070] The precision loss location method of the disclosed embodiment compares the first variable information and the second variable information corresponding to any variable to determine the difference in the variable. Based on the difference in each variable, a target sub-step is determined from multiple sub-steps. Based on the target sub-step, the location of the anomaly is determined. This allows for fine-grained screening of sub-steps in an anomaly module and rapid location of the anomaly.

[0071] In order to clearly illustrate the accuracy loss positioning method disclosed in the present invention, a detailed description is given below with reference to examples.

[0072] As an example, the accuracy loss location method of the present disclosure is applied to the accuracy loss location process when the target deep learning model is deployed from the training framework (such as PyTorch) (referred to as the first framework in this disclosure) to the inference framework (such as TensorRT) (referred to as the second framework in this disclosure), as shown in FIG. Figure 3 As shown in Figure 2, the deep learning model conversion accuracy loss location process can include the following steps: 1. Module-level coarse-grained positioning The implementation process may include the following steps: 1.1 Dynamically register forward hooks In the trained target deep learning model of the training framework (referred to as the first model in this disclosure), forward hooks are registered for each neural network layer (referred to as a neural network module in this disclosure) in the trained target deep learning model; test data is input into the trained target deep learning model so that the trained target deep learning model can be run on the training framework, and the intermediate layer output results of each neural network layer in the trained target deep learning model (referred to as the first intermediate layer output results in this disclosure) are automatically captured through the forward hooks registered in any neural network layer.

[0073] 1.2 Expand ONNX output nodes The trained target deep learning model is converted into ONNX format to obtain an ONNX model (referred to as an intermediate representation model in this disclosure). The output items corresponding to the output results of the intermediate layers of each neural network layer in the trained target deep learning model are added to the output nodes of the ONNX model to obtain a modified ONNX model. The modified ONNX model is loaded into the inference framework to obtain a deployed model (referred to as the second model in this disclosure).

[0074] The test data is input into the deployed model so that the deployed model can be run on the inference framework to obtain the intermediate layer output results of each neural network layer in the deployed model (referred to as the second intermediate layer output results in this disclosure).

[0075] 1.3 Automation comparison: For any neural network layer, the script automatically compares the intermediate output results of the neural network layer in the training framework and the intermediate output results in the inference framework, and calculates the absolute error between the intermediate output results of the neural network layer in the training framework and the intermediate output results in the inference framework. Then, the maximum absolute error of the neural network is determined from the absolute error between the intermediate output results of the neural network layer in the training framework and the intermediate output results of the inference framework (referred to as the first coefficient in this disclosure). Optionally, an error heat map can be generated based on the maximum absolute error of each neural network layer to indicate the absolute error of the corresponding neural network layer, thereby visually showing the error accumulation path to the user.

[0076] Finally, each neural network layer can be compared with the corresponding set error threshold in the order of execution of each neural network layer, and the neural network layer ranked first with the maximum absolute error greater than the corresponding set error threshold (referred to as the set threshold in this disclosure) is determined as an abnormal module.

[0077] In this way, the neural network module that causes accuracy anomalies can be quickly located.

[0078] 2. Operation-level fine-grained tracing The implementation process may include the following steps: 2.1 Dynamic Tracking Use the sys.settrace tracing function to track the forward computation process of the abnormal module when it runs on the training framework, and record the variable values of the variables in each operation step (referred to as sub-steps in this disclosure); The variables may be input variables or output variables, which is not limited in the present disclosure.

[0079] 2.2 Variable Snapshot Comparison You can call the TensorRT framework's command-line tool trtexec to print the variable values of each operation step when the exception module is running on TensorRT; The variable values of the variables in each operation step when the abnormal module is running on the training framework are aligned and compared with the variable values of the variables in each operation step when the abnormal module is running on TensorRT to determine the relative error and absolute error of each variable; finally, based on the relative error and absolute error of the variables in each operation step, the operation step with a sudden error is identified from each operation step, and the operation step with the sudden error is located as the abnormal step that caused the error (referred to as the abnormal position in this disclosure).

[0080] 3. Root cause pattern matching and repair 3.1 Error pattern matching: Using a rule engine, based on the variable values and / or variable states of the variables in the abnormal step, a target error pattern that matches the abnormal step is determined from multiple preset error patterns (such as all-zero output, gradual error accumulation, sudden large error, etc.). Based on the target error pattern, the abnormal cause that matches the target error pattern is obtained (such as weight not loaded, low-precision overflow, etc.); 3.2 Root Cause Repair: Based on the cause of the abnormality, the repair suggestion corresponding to the abnormal cause (referred to as a repair rule in this disclosure) is determined (such as enabling FP32 mode, adjusting quantization parameters, etc.), and based on the abnormal step, the target error pattern and abnormal cause of the abnormal step, and the repair suggestion, a prompt message can be generated and sent to remind the relevant user of the abnormal step in the trained target deep learning model where the abnormality occurred, the target error pattern and abnormal cause of the abnormal step, and the repair suggestion. In this way, the relevant user can quickly locate the abnormal step in the trained target deep learning model where the abnormality occurred, understand the error pattern and abnormal cause of the abnormal step, and repair the abnormal step in the trained target deep learning model based on the repair suggestion.

[0081] 3.3 Automated Backtesting Verification: After repairing the abnormal steps in the trained target deep learning model, a backtest analysis can be performed on the repaired trained target deep learning model to evaluate the actual effect of the accuracy recovery of the repaired trained target deep learning model. If the accuracy of the target deep learning model is confirmed to be restored, the repair is determined to be completed. If the accuracy of the target deep learning model is confirmed to be restored, the repair and backtest analysis will continue.

[0082] The accuracy loss positioning method disclosed in this disclosure has the following advantages: 1. A three-level progressive positioning system (module level → operation level → root cause level) is adopted. At the module level, dynamic hooks are used to automatically compare the output of the intermediate layer to achieve coarse-grained screening. At the operation level, a tracking function is used to implement line-level code tracking to achieve fine-grained positioning. At the root cause level, error pattern matching based on the rule engine is used to achieve causal association.

[0083] 2. Dynamic code tracing technology is used to apply the sys.settrace system-level debugging tool to deep learning model conversion scenarios, enabling accurate capture of row-level variable snapshots during the forward computation process. A dynamic intermediate layer output expansion mechanism is proposed to automatically insert output nodes during the ONNX model export phase, thereby avoiding the tedious manual modification of the model structure. This method has a zero-intrusive code feature and can achieve runtime monitoring without any modification to the original model code, effectively avoiding the drawbacks of manually adding print statements or decorators.

[0084] 3. Adopt a closed-loop diagnosis and repair system, build a rule library containing 18 common error patterns, develop an automated repair and verification engine, and achieve a closed loop of the entire "locate → repair → backtest" process, effectively improving debugging efficiency.

[0085] Figure 4 Schematic diagram of the structure of the precision loss positioning device proposed in one embodiment of the present disclosure.

[0086] like Figure 4 As shown, the precision loss positioning device 400 includes: The first acquisition module 401 is used to acquire test data and the network model of the target deep learning model during the conversion process; wherein the network model includes a first model of the first framework and a second model of the second framework, and the target deep learning model includes multiple neural network modules.

[0087] The input module 402 is used to input the test data into the first model to obtain the first intermediate layer output result of the first model, and input the test data into the second model to obtain the second intermediate layer output result of the second model.

[0088] The determination module 403 is used to determine an abnormal module from multiple neural network modules based on the output results of the first intermediate layer and the output results of the second intermediate layer; wherein the abnormal module includes multiple variables.

[0089] The positioning module 404 is used to locate the abnormal position of the first model according to the first variable information of the multiple variables when the abnormal module runs on the first frame and the second variable information of the multiple variables when the abnormal module runs on the second frame.

[0090] In a possible implementation of the embodiment of the present disclosure, the determination module 403 is used to: for any neural network module, determine the first coefficient corresponding to the neural network module according to the difference between the first intermediate layer output result when the neural network module is running on the first framework and the second intermediate layer output result when the neural network module is running on the second framework; based on the first coefficient corresponding to each neural network module, determine the abnormal module from multiple neural network modules.

[0091] In a possible implementation of the embodiment of the present disclosure, the determination module 403 is used to: analyze the first coefficients corresponding to each neural network module in turn according to the execution order of multiple neural network modules, and determine the target coefficient from the multiple first coefficients; based on the target coefficient, determine the abnormal module.

[0092] In a possible implementation of the embodiment of the present disclosure, the input module 402 is used to: register a forward hook for each neural network module in the first model; and run the registered first model in the first framework based on the test data to obtain a first intermediate layer output result.

[0093] In a possible implementation of the embodiment of the present disclosure, the input module 402 is used to: convert the format of the first model to obtain an intermediate representation model; expand the output items of the intermediate representation model based on the intermediate layer output results of the corresponding neural network module in the first model to obtain an expanded intermediate representation model; and run the expanded intermediate representation model as the second model on the second framework based on the test data to obtain second intermediate output layer information.

[0094] In a possible implementation of the embodiment of the present disclosure, the abnormality module includes multiple sub-steps; the positioning module 404 is used to: for any variable, compare the first variable information and the second variable information corresponding to the variable to determine the difference between the variables; determine the target sub-step from the multiple sub-steps based on the difference between each variable; and determine the abnormality location based on the target sub-step.

[0095] In a possible implementation of the embodiment of the present disclosure, the positioning module 404 is used to: determine, for any sub-step, a target coefficient of the sub-step based on the difference of at least one variable associated with the sub-step; determine the error change rate of the target coefficient of any sub-step in sequence according to the execution order of each sub-step; and determine a target sub-step from multiple sub-steps based on the error change rate corresponding to each sub-step.

[0096] In a possible implementation of the embodiment of the present disclosure, the precision loss positioning device 400 may further include: The analysis module is configured to analyze at least one variable associated with the abnormal position to determine a target error pattern corresponding to the abnormal position from a preset error pattern.

[0097] The second acquisition module is used to acquire a repair rule that matches the target error pattern.

[0098] The processing module is used to generate prompt information based on the repair rules and send the prompt information to the user.

[0099] It should be noted that the above explanation of the precision loss positioning method is also applicable to the precision loss positioning device of this embodiment and will not be repeated here.

[0100] The precision loss positioning device of the embodiment of the present disclosure obtains test data and a network model of a target deep learning model during the conversion process; wherein the network model includes a first model of a first framework and a second model of a second framework, and the target deep learning model includes multiple neural network modules; inputs the test data into the first model to obtain the first intermediate layer output result of the first model, and inputs the test data into the second model to obtain the second intermediate layer output result of the second model; based on the first intermediate layer output result and the second intermediate layer output result, an abnormal module is determined from multiple neural network modules; wherein the abnormal module includes multiple variables; according to the first variable information of the multiple variables when the abnormal module runs on the first framework and the second variable information of the multiple variables when the abnormal module runs on the second framework, the abnormal position of the first model is located. In this way, the abnormal position in the deep learning model can be automatically and quickly located without manually checking the model code layer by layer or blindly debugging the model parameters, thereby helping to shorten the model debugging cycle, improve the model debugging efficiency, and effectively reduce the dependence on relevant debugging personnel and reduce labor costs.

[0101] Figure 5 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Figure 5 The electronic device 12 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0102] like Figure 5As shown, electronic device 12 is implemented as a general-purpose computing device. Components of electronic device 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and a bus 18 that connects various system components (including system memory 28 and processing unit 16).

[0103] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnection (PCI) bus.

[0104] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0105] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 5 Not shown, usually called a "hard drive").

[0106] although Figure 5Although not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a floppy disk) and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a Compact Disc Read Only Memory (CD-ROM), a Digital Video Disc Read Only Memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present disclosure.

[0107] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally implement the functions and / or methodologies described in the embodiments of the present disclosure.

[0108] The electronic device 12 can also communicate with one or more external devices 14 (e.g., a keyboard, pointing device, display 24, etc.), one or more devices that enable human interaction with the electronic device 12, and / or any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). This communication can occur via an input / output (I / O) interface 22. Furthermore, the electronic device 12 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via the bus 18. It should be understood that, although not shown, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0109] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28 , such as implementing the accuracy loss positioning method mentioned in the above embodiment.

[0110] In order to implement the above embodiments, the present disclosure further proposes a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the accuracy loss location method proposed in the above embodiments of the present disclosure.

[0111] In order to implement the above embodiments, the present disclosure further proposes a computer program product. When an instruction processor in the computer program product executes, the accuracy loss location method proposed in the above embodiments of the present disclosure is executed.

[0112] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow from the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0113] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0114] It should be noted that, in the description of this disclosure, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In addition, in the description of this disclosure, unless otherwise specified, the meaning of "plurality" is two or more.

[0115] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0116] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0117] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0118] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0119] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0120] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0121] Although the embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are illustrative and are not to be construed as limitations on the present disclosure. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present disclosure.

Claims

1. A precision loss positioning method, characterized in that: The method comprises: Acquire test data and a network model of a target deep learning model during the conversion process; wherein the network model includes a first model of a first framework and a second model of a second framework, and the target deep learning model includes multiple neural network modules; Inputting the test data into the first model to obtain a first intermediate layer output result of the first model, and inputting the test data into the second model to obtain a second intermediate layer output result of the second model; Based on the output result of the first intermediate layer and the output result of the second intermediate layer, determining an abnormal module from the multiple neural network modules; wherein the abnormal module includes multiple variables; The abnormal position of the first model is located according to the first variable information of the multiple variables when the abnormal module runs on the first framework and the second variable information of the multiple variables when the abnormal module runs on the second framework.

2. The method according to claim 1, characterized in that The determining an abnormal module from the plurality of neural network modules based on the output result of the first intermediate layer and the output result of the second intermediate layer includes: For any of the neural network modules, determining a first coefficient corresponding to the neural network module according to a difference between a first intermediate layer output result when the neural network module is run on the first framework and a second intermediate layer output result when the neural network module is run on the second framework; Based on the first coefficient corresponding to each of the neural network modules, the abnormal module is determined from the multiple neural network modules.

3. The method according to claim 2, characterized in that The determining the abnormal module from the plurality of neural network modules based on the first coefficient corresponding to each of the neural network modules includes: Analyzing the first coefficients corresponding to the neural network modules in sequence according to the execution order of the plurality of neural network modules, and determining a target coefficient from the plurality of first coefficients; Based on the target coefficient, the abnormal module is determined.

4. The method according to claim 1, wherein Inputting the test data into the first model to obtain a first intermediate layer output result of the first model includes: registering a forward hook for each of the neural network modules in the first model; Based on the test data, the registered first model is run in the first framework to obtain the first intermediate layer output result.

5. The method according to claim 4, characterized in that Inputting the test data into the second model to obtain a second intermediate layer output result of the second model includes: Converting the first model into a format to obtain an intermediate representation model; Expanding the output items of the intermediate representation model based on the intermediate layer output results of the corresponding neural network module in the first model to obtain an expanded intermediate representation model; Based on the test data, the expanded intermediate representation model as the second model is run on the second framework to obtain the second intermediate output layer information.

6. The method according to claim 1, characterized in that The abnormal module includes multiple sub-steps; locating the abnormal position of the first model according to the first variable information of the multiple variables when the abnormal module is running on the first framework and the second variable information of the multiple variables when the abnormal module is running on the second framework, including: For any of the variables, comparing the first variable information and the second variable information corresponding to the variable to determine a difference in the variable; determining a target sub-step from the plurality of sub-steps according to the difference between the variables; Based on the target sub-step, the abnormality location is determined.

7. The method according to claim 6, characterized in that Determining a target sub-step from the plurality of sub-steps according to the difference between the variables includes: For any of the sub-steps, determining a target coefficient of the sub-step according to a difference value of at least one of the variables associated with the sub-step; Determining the error change rate of the target coefficient of any of the sub-steps in sequence according to the execution order of the sub-steps; The target sub-step is determined from the multiple sub-steps according to the error change rate corresponding to each sub-step.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: analyzing at least one of the variables associated with the abnormal position to determine a target error pattern corresponding to the abnormal position from preset error patterns; Obtaining a repair rule that matches the target error pattern; Based on the repair rule, prompt information is generated and sent to the user.

9. A precision loss positioning device, characterized in that: The device comprises: A first acquisition module is configured to acquire test data and a network model of a target deep learning model during the conversion process; wherein the network model includes a first model of a first framework and a second model of a second framework, and the target deep learning model includes multiple neural network modules; an input module, configured to input the test data into the first model to obtain a first intermediate layer output result of the first model, and input the test data into the second model to obtain a second intermediate layer output result of the second model; a determination module, configured to determine an abnormal module from the plurality of neural network modules based on the output result of the first intermediate layer and the output result of the second intermediate layer; wherein the abnormal module includes a plurality of variables; A positioning module is used to locate the abnormal position of the first model according to the first variable information of the multiple variables when the abnormal module runs on the first framework and the second variable information of the multiple variables when the abnormal module runs on the second framework.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images