An agent-based cross-framework model inference precision anomaly positioning and repairing method and system

Abstract

The application relates to an agent-based cross-framework model reasoning precision anomaly positioning and repairing method and system, which comprises the following steps: loading a target model in a reasoning framework, injecting a hook function to capture operator metadata and output tensors, and generating a reasoning log; based on the reasoning log, performing cross-framework alignment mapping to establish a mapping relationship; performing numerical comparison of output tensors on the target layer and the reference layer with the established mapping relationship, calculating the numerical difference, and marking an abnormal layer when the numerical difference exceeds a preset abnormal threshold; calling a debugging tool to debug, analyze and locate the specific implementation difference causing the numerical difference; inputting the implementation difference into a code generation model to generate a repair code; and replacing the repair code into the target model and repeatedly executing until the numerical difference abnormal threshold is reached; and the application provides a closed-loop solution capable of automatically completing cross-framework network layer mapping, abnormal layer positioning, difference root analysis, repair code generation and iterative verification.

Description

Technical Field

[0001] This invention relates to the fields of computer technology and artificial intelligence, and more particularly to a debugging and cross-framework transfer technology for deep learning models. More specifically, it relates to a method and system for locating and repairing anomalies in cross-framework model inference accuracy based on intelligent agents. Background Technology

[0002] In recent years, deep learning models, represented by Large Language Models (LLM) and Large Visual Models, have made groundbreaking progress in various fields. These models are typically trained using mainstream deep learning frameworks such as PyTorch and TensorFlow. However, in practical industrial applications, the training environment and production deployment environment of a model are often separated. For example, a model may be developed and trained in the PyTorch framework, which has a rich ecosystem and flexibility, but in order to pursue the ultimate inference performance or to adapt to specific hardware (such as NPU, TPU), it needs to be migrated to another high-performance inference framework (such as TensorFlow, TensorRT, or a self-developed framework).

[0003] This cross-framework migration and deployment process was not without its challenges, one of the most difficult being the loss of inference accuracy. Although both frameworks aim to achieve the same functionality, their specific implementations of underlying operators (such as Conv2d, LayerNorm, and Attention) may have subtle but crucial differences. These differences might manifest in the following ways:

[0004] 1. Different Padding Strategies: In convolution operations, the calculation details of the padding methods for boundary pixels (such as SAME, VALID) may not be completely consistent in different frameworks, resulting in slight deviations in the size or value of the output feature map.

[0005] 2. Differences in numerical calculation accuracy: Different frameworks may have different rounding and calculation rules for floating-point numbers (such as FP32, FP16, BF16). These tiny errors will accumulate and amplify layer by layer in deep models containing hundreds or even thousands of network layers, especially in autoregressive Transformer models, eventually leading to significant deviations in inference results.

[0006] 3. Inconsistent default parameters: Some operators may have default values ​​for certain parameters in one framework, while they need to be explicitly specified in another. For example, the Dropout layer is automatically disabled in PyTorch's .eval() mode, but in some inference frameworks, it may be necessary to explicitly pass the is_training=False flag;

[0007] 4. Differences in operator fusion strategies: In order to optimize performance, inference frameworks usually perform operator fusion. However, different frameworks have different fusion strategies and capabilities, which may also introduce equivalent transformations in computational logic, resulting in numerical differences.

[0008] Existing technologies attempt to solve this problem, but have many shortcomings:

[0009] 1. Manual debugging is inefficient and lacks scalability: The most primitive method is for engineers to manually execute the model layer by layer in two frameworks, print and compare the output tensors of each layer. For modern large models with hundreds of layers, this method is time-consuming and labor-intensive, like finding a needle in a haystack, with debugging cycles that can last for weeks, and it relies entirely on the engineer's personal experience, making it impossible to scale.

[0010] 2. Insensitivity to Traditional Statistical Metrics: Some methods attempt to use statistical metrics such as Kullback-Leibler divergence to measure the similarity between the distributions of two output tensors. However, Kullback-Leibler divergence measures the difference in the overall distribution and is not sensitive enough to deterministic discrepancies, which, although numerically small, can be amplified in subsequent layers. For example, two tensors may have a small Kullback-Leibler divergence, but a numerical deviation at a key point could lead to completely different branches in subsequent calculations.

[0011] 3. Limitations of Model Conversion Tools: Model conversion tools such as ONNX (Open Neural Network Exchange) aim to provide a universal intermediate representation (IR) to bridge different frameworks. However, these tools mainly address the conversion of model structure and weights, and are generally powerless to handle the inherent semantic differences in operator implementations. The converted model may still suffer from accuracy degradation, and these tools themselves lack robust debugging and automatic repair mechanisms.

[0012] 4. General code repair techniques lack domain specificity: Although general automatic code repair techniques based on Abstract Syntax Tree (AST) alignment or large language models exist, they are not designed for the specific scenario of cross-framework transfer in deep learning. They cannot understand concepts in the fields of operators, tensors, and dimensions, nor can they integrate dynamic runtime information (such as tensor values) for comprehensive judgment and repair. Summary of the Invention

[0013] To address the shortcomings and problems in existing technologies, the main objective of this invention is to provide a method and system for locating and repairing inference accuracy anomalies in cross-framework models based on intelligent agents. This aims to solve the technical problem in existing technologies where differences in operator implementation lead to decreased inference accuracy when deep learning models are deployed across frameworks, and the lack of automated, efficient, and high-precision location and repair methods. Specifically, this invention strives to provide a closed-loop solution capable of automatically completing cross-framework network layer mapping, anomaly layer location, root cause analysis of discrepancies, repair code generation, and iterative verification.

[0014] To achieve the above objectives, the core technical solution adopted by this invention is as follows: This invention provides a method for locating and repairing anomalies in cross-framework model inference accuracy based on intelligent agents. This method is driven by an intelligent agent program and follows a logical closed loop of "mapping-comparison-location-repair-verification". The method includes the following steps:

[0015] First, target models with identical structures and weights are loaded into both the first inference framework (target environment) and the second inference framework (baseline environment). By injecting hook functions into the execution logic of the two frameworks, detailed information of each network layer (operator) is captured in real time and non-intrusively during forward inference. This includes metadata such as operator type, dimensions of input / output tensors, number and type of parameters, and the actual output tensor of that layer. This information is then formatted into a structured inference log.

[0016] Next, based on the collected inference logs, cross-framework network layer alignment mapping is performed. This step aims to accurately find the corresponding layer in the second frame for each layer in the first frame. To balance accuracy and robustness, a dual mapping strategy is adopted. First, a compact and unique hash signature is generated by serializing and hashing the key metadata (operator type, input / output dimensions, number of parameters, parameter type) of each layer. If the hash signature of a layer in the first frame (target layer) is completely identical to that of a layer in the second frame (reference layer), they are considered to be the same layer, and an exact mapping is established. For cases where hash signatures do not match due to naming or subtle attribute differences, a fuzzy mapping strategy is initiated. This strategy calculates the cosine similarity between the input dimension vector and the output dimension vector of the target layer and the reference layer. If the similarity score is higher than a preset threshold (e.g., 0.95), and their output dimensions are strictly equal, they are also considered to be corresponding layers, and a fuzzy mapping is established.

[0017] Then, for all layer pairs that have successfully established a mapping relationship, an element-wise numerical comparison of the output tensors is performed. The relative and absolute differences between the target layer's output tensor and the reference layer's output tensor are calculated. These differences are compared with a preset, tolerable floating-point error range (i.e., anomaly threshold, such as 1e-4). Once a layer pair's numerical difference exceeds this threshold, the target layer is marked as an "anomaly layer," indicating that the source of the precision deviation is likely there.

[0018] Subsequently, a deep difference analysis process is initiated for the marked anomalous layers. First, for example, using debugging tools such as Python's PDB (Python Debugger), a breakpoint is dynamically inserted at the corresponding source code location of the anomalous layer. The inference process is re-executed; when the breakpoint is reached, the program pauses, capturing the most accurate runtime context information, including the precise input tensor entering the layer, the weight parameters loaded for that layer, and any intermediate computation results generated within the operators. Simultaneously, this method parses the operator implementation source code of the target layer in the first frame and the implementation source code of the reference layer in the second frame, converting them into Abstract Syntax Trees (ASTs). An AST is a data structure that can represent the syntactic structure of code. By comparing these two ASTs, the root causes of numerical differences can be precisely identified at the code logic level; for example, one frame uses floor rounding while the other uses round rounding, or one frame's convolution operation includes a bias term while the other does not.

[0019] Next, the results of the deep analysis—namely, the specific implementation differences identified (e.g., "missing training=False parameter")—along with the operator's description information (name, type, etc.) and a reference code snippet, are fed into a large, pre-trained model specifically designed for code generation. This model (e.g., CodeT5 or a similar model based on the Transformer architecture and fine-tuned using a massive codebase) understands the intent of the difference report and generates a code snippet to fix the problem according to the syntax of the target framework (the first inference framework).

[0020] Finally, the generated repair code is dynamically applied to the running target model using a "hot-swapping" technique, replacing the old, problematic code logic with the new code logic without restarting the entire program. Then, the aforementioned numerical comparison step is immediately triggered to re-verify the accuracy of the repaired layer. If the numerical difference falls below a threshold, the repair is successful, the agent will solidify the modification, and continue checking the next anomalous layer. If verification fails, the generated code is incorrect, the agent will perform a rollback operation, reverting the modification, and feeding back the failure information to the code generation model, instructing it to attempt to generate an alternative repair solution. This "repair-verify-rollback / confirm" process continues iterating until the accuracy of all anomalous layers in the model is aligned. Furthermore, the cross-framework alignment mapping steps include:

[0021] Extract the preset attributes of each layer of operators. The preset attributes include operator type, input dimension, output dimension, number of parameters, and parameter type.

[0022] A unique hash signature is generated for each layer based on preset attributes. If the hash signatures of the target layer and the reference layer are consistent, an exact mapping is established.

[0023] If the hash signatures are inconsistent, the cosine similarity between the input dimension vector and the output dimension vector of the target layer and the reference layer is calculated. When the cosine similarity is higher than the preset similarity threshold and the output dimensions are consistent, a fuzzy mapping is established.

[0024] Furthermore, the hash signature is generated by performing hash operations on serialized data with different operator types, input dimensions, output dimensions, number of parameters, and parameter types; and the similarity threshold is dynamically adjusted according to the depth of the network layer, with the similarity threshold for deeper networks being higher than that for shallower networks.

[0025] Furthermore, the steps to analyze and pinpoint the specific implementation differences that lead to the numerical discrepancies include:

[0026] Dynamic breakpoints are inserted at the anomaly layer to capture its runtime context information during the inference process. The runtime context information includes the input tensor, weight parameters, and intermediate calculation results.

[0027] The operator implementation code of the target layer in the first inference framework and the operator implementation code of the reference layer in the second inference framework are analyzed to generate the corresponding first abstract syntax tree and second abstract syntax tree respectively;

[0028] By comparing the first and second abstract syntax trees, structural differences in parameter initialization, computation order, or numerical precision handling are identified as specific implementation differences.

[0029] Furthermore, the code generation model is a model based on a pre-trained Transformer architecture and fine-tuned using code corpus; the iterative verification step also includes: if the verification fails, the hot replacement operation is rolled back, and the code generation model is instructed to generate alternative repair code based on historical repair records.

[0030] Furthermore, the present invention also provides a system for implementing the above method, comprising:

[0031] The log capture module is used to capture operator metadata and output tensors of each network layer in real time through injected hook functions within the same target model running in the first inference framework and the second inference framework respectively, and generate structured inference logs.

[0032] The cross-frame mapping module is used to align and map the network layers of the first inference frame and the second inference frame based on the structured inference log, and to establish the mapping relationship between the target layer and the reference layer.

[0033] The numerical comparison module is used to compare the output tensors of the target layer and the reference layer with the established mapping relationship, calculate their numerical differences, and output an abnormal layer label when the numerical difference exceeds a preset abnormal threshold.

[0034] The difference analysis module is used to call the debugging tool to perform dynamic breakpoint debugging on the exception layer after receiving the exception layer flag, and analyze and locate the specific implementation differences that cause the numerical differences by comparing the abstract syntax tree.

[0035] The code repair module is used to input implementation differences into a code generation model and generate repair code for repairing the target layer.

[0036] The iterative verification module is used to hot-swap the repair code into the target model and trigger the numerical comparison module to re-verify until the numerical difference is lower than the anomaly threshold.

[0037] Preferably, the cross-frame mapping module includes:

[0038] The attribute extraction unit is used to extract the preset attributes of each layer of operators. The preset attributes include operator type, input dimension, output dimension, number of parameters and parameter type.

[0039] The precise mapping unit is used to generate a unique hash signature for each layer based on preset attributes, and to establish a precise mapping when the hash signatures of the target layer and the reference layer are detected to be consistent.

[0040] The fuzzy mapping unit is used to calculate the cosine similarity between the input dimension vector and the output dimension vector of the target layer and the reference layer when the hash signatures are inconsistent, and to establish a fuzzy mapping when the cosine similarity is higher than a preset similarity threshold and the output dimensions are consistent.

[0041] Preferably, the precise mapping unit generates a hash signature by performing a hash function on serialized data of operator type, input dimension, output dimension, number of parameters, and parameter type; the fuzzy mapping unit is configured to read from memory and apply different similarity thresholds based on the depth information of the network layer.

[0042] Preferably, the difference analysis module includes:

[0043] The dynamic debugging unit is used to insert dynamic breakpoints at the code locations corresponding to the exception layer and capture runtime context information during the inference process, including input tensors, weight parameters, and intermediate calculation results.

[0044] Abstract syntax tree generation unit is used to parse the operator implementation code of the target layer and the reference layer, and generate the corresponding first abstract syntax tree and second abstract syntax tree respectively;

[0045] The syntax tree comparison unit is used to compare the first abstract syntax tree with the second abstract syntax tree, identify and output structural differences in parameter initialization, computation order or numerical precision processing.

[0046] Preferably, the code repair module includes a code generation model based on a pre-trained Transformer architecture and fine-tuned using code corpus; the iterative verification module also includes a rollback controller, which is used to undo the hot replacement operation of the repair code when verification fails, and send a regeneration instruction containing historical failure information to the code repair module.

[0047] Compared with the prior art, the beneficial effects of this invention are significant and multifaceted:

[0048] 1. Improved debugging efficiency: This invention transforms the traditionally time-consuming, cross-framework model debugging process into a fully automated, agent-driven workflow. It can complete the accuracy alignment of complex, large models containing hundreds of network layers within hours. This significantly shortens the model deployment cycle and accelerates the iteration speed of AI applications.

[0049] 2. Precise localization and traceable root cause: By combining a dual mapping strategy based on hashing and cosine similarity, network layers across frameworks can be accurately and robustly aligned. Furthermore, by setting dynamic breakpoints at the exception layer to capture runtime context and combining this with AST-level code semantic comparison, it is possible to penetrate the surface of phenomena and reach the root cause at the code implementation level that leads to accuracy deviations, avoiding blind guessing.

[0050] 3. High sensitivity and extremely low false alarm rate: This invention employs tensor-level element-by-element numerical comparison (relative and absolute differences), exhibiting extremely high sensitivity to minute numerical deviations that might be ignored by traditional statistical indicators but are amplified in subsequent calculations. Experimental data shows that the false alarm rate can be controlled below 0.1%, ensuring the effectiveness of the analysis.

[0051] 4. Closed-Loop Automation and Intelligent Repair: This invention constructs a complete automated closed loop from "mapping," "comparison," "location," "repair," to "verification." In particular, the introduction of a finely tuned large-scale code generation model ensures that the repair solution is no longer a preset rule, but rather intelligently generated code based on specific difference analysis reports, offering greater flexibility and accuracy. Iterative verification and rollback mechanisms guarantee the stability and reliability of the entire repair process.

[0052] 5. Strong universality and compatibility: The overall methodology of this invention does not depend on a specific deep learning framework. Its core modules (such as log capture, mapping, alignment, and repair) can be configured to adapt to different mainstream frameworks (such as PyTorch, TensorFlow, PaddlePaddle, etc.). The solution supports automatic conversion and repair of more than 20 common operators (such as Conv, LayerNorm, Attention, LSTM, etc.), demonstrating good versatility and scalability. Attached Figure Description

[0053] To enable those skilled in the art to more clearly and comprehensively understand the technical solutions of the present invention, preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the accompanying drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the accompanying drawings:

[0054] Figure 1 This is a functional block diagram of an agent-based cross-frame model inference accuracy anomaly localization and repair system according to an embodiment of the present invention.

[0055] Figure 2 This is a detailed flowchart of a method for locating and repairing anomalies in cross-frame model inference accuracy based on an agent, according to an embodiment of the present invention.

[0056] In the above figures, the same reference numerals denote parts that have the same or similar functions. The names of the parts represented by each reference numeral are as follows:

[0057] 100: System Repair; 110: Log Capture Module; 120: Cross-Framework Mapping Module; 121: Attribute Extraction Unit; 122: Precise Mapping Unit; 123: Fuzzy Mapping Unit; 130: Numerical Comparison Module; 140: Difference Analysis Module; 141: Dynamic Debugging Unit; 142: Abstract Syntax Tree Generation Unit; 143: Syntax Tree Comparison Unit; 150: Code Repair Module; 160: Iterative Verification Module. Detailed Implementation

[0058] The present invention will now be described in detail with reference to the accompanying drawings:

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, several preferred embodiments of the invention will be described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are merely for explaining the invention and do not constitute any limitation on its scope of protection. Any modifications, equivalent substitutions, or improvements made based on the spirit and principles of this invention should be included within the scope of protection of this invention.

[0060] Example 1: This example details a complete implementation of anomaly localization and repair for inference accuracy in a cross-frame model based on intelligent agents. (Refer to...) Figure 1 and Figure 2 For example, this solution is executed by a repair system 100 deployed on a computing device (such as a server or workstation). This computing device typically includes a high-performance processor (CPU / GPU), large-capacity memory, storage devices, and network interfaces, and comes pre-installed with necessary software such as a Python environment, target and reference deep learning frameworks, etc. The entire process is scheduled by an agent program acting as the overall controller.

[0061] Step S110: Load the model and capture logs

[0062] The first step of this method is to load the same target model in the first inference framework and the second inference framework, and to capture the operator metadata and output tensors of each network layer in real time through the injected hook function (S110) to generate a structured inference log.

[0063] Specifically, assume the first inference framework is TensorFlow for production deployment, and the second inference framework is PyTorch for prototyping. The target model is a pre-trained Visual Transformer (ViT) model with hundreds of layers, trained in PyTorch. The agent first converts the PyTorch model's weight file (.pt or .pth) to a TensorFlow-compatible format (such as .pb or HDF5). Then, the model structure and the converted weights are loaded into both frameworks respectively, ensuring that the parameters of the two models are completely identical in the initial state.

[0064] To capture runtime information for each layer, the agent utilizes hook mechanisms provided by two frameworks. In PyTorch, this can be achieved using `register_forward_hook`; in TensorFlow, it can be achieved using `tf.GradientTape` or by embedding callback logic in a custom layer. The agent registers a forward propagation hook for each network layer (`torch.nn.Module` or `tf.keras.layers.Layer`) in the model. When the model performs a complete forward inference, the corresponding hook function is triggered whenever data flows through a layer.

[0065] This hook function is designed to extract and log the following information:

[0066] Layer identifier: The unique name or index of the layer, such as encoder.layer.0.attention.

[0067] Operator metadata:

[0068] Operator type (OpType): For example, Conv2d, LayerNorm, GELU.

[0069] Input dimension (InDim): The shape of the input tensor received by this layer, for example (Batch, SeqLen, HiddenSize).

[0070] Output Dimension (OutDim): The shape of the output tensor produced by this layer.

[0071] Number of parameters (ParamNum): The total number of trainable parameters (weights and biases) contained in this layer.

[0072] Parameter type (ParamType): The data type of the parameter, such as float32, float16.

[0073] Output Tensor: The actual output tensor data obtained after forward computation of this layer. To save storage, it is usually transferred from GPU memory to CPU memory and converted to NumPy array format.

[0074] All this information is organized into a structured log entry, such as a JSON object, and appended to a log file. A complete forward inference operation generates one such log entry for each layer, ultimately resulting in two complete inference logs, one for TensorFlow and one for PyTorch. The value of this step lies in its ability to provide comprehensive and detailed "ground truth" data for subsequent analysis in a non-intrusive and automated manner.

[0075] Step S120: Cross-frame alignment mapping

[0076] The second step of the method is to perform cross-frame alignment mapping (S120) on the network layers of the first inference framework and the second inference framework based on the structured inference log, and establish the mapping relationship between the target layer and the reference layer.

[0077] After obtaining the inference logs from both frameworks, the cross-framework mapping module 120 begins operation. Its goal is to solve the problem of "which layer in the PyTorch log corresponds to the conv2d_1 layer in the TensorFlow log?" Since different frameworks have different automatic naming mechanisms, directly matching by layer name is unreliable. Therefore, this invention employs a more robust dual mapping strategy.

[0078] In a preferred embodiment, this step is specifically completed collaboratively by the attribute extraction unit 121, the precise mapping unit 122, and the fuzzy mapping unit 123.

[0079] First, the attribute extraction unit 121 reads log entries one by one from the two log files and extracts the preset key attributes, namely the aforementioned operator type, input dimension, output dimension, number of parameters and parameter type.

[0080] Next, the precise mapping unit 122 performs a precise mapping attempt for each layer. It extracts five key attributes, serializes them into strings in a fixed order (e.g., OpType||InDim||OutDim||ParamNum||ParamType), and then inputs this serialization into a fast hash function (e.g., SHA-256) to generate a unique hash signature for each layer. The advantage of hash signatures is that even the slightest difference in the original attributes will result in drastically different hash values. The precise mapping unit 122 iterates through all layers in TensorFlow (the target framework), using their hash signatures to search for an identical hash signature in all layers of PyTorch (the reference framework). If found, the two layers are considered equivalent, a precise mapping relationship is immediately established, and both layers are removed from the list of layers to be mapped. This method can quickly and accurately handle the vast majority of layers with completely identical structures, making it extremely efficient.

[0081] However, in some cases, hash signatures may be inconsistent. For example, a framework might introduce an extra internal variable during implementation that does not affect the computational logic, causing a slight change in the number of parameters (ParamNum). In this case, the fuzzy mapping unit 123 intervenes. For all remaining unmapped layers, it employs a more relaxed but still structurally similar fuzzy mapping strategy. It calculates the cosine similarity between the input and output dimension vectors of a target layer and a reference layer. A dimension vector is a one-dimensional vector flattened from a tensor shape, such as (1, 512, 768). Cosine similarity measures the consistency in the direction of two vectors and is well-suited for judging the similarity of dimensional structures. If the calculated similarity score Scos is higher than a preset similarity threshold α, and their output dimensions OutDim are strictly equal (a crucial prerequisite for ensuring the validity of subsequent numerical comparisons), then the fuzzy mapping unit 123 determines that they correspond and establishes a fuzzy mapping.

[0082] As a more specific implementation, the similarity threshold α can also be dynamically adjusted according to the depth of the network layers. For example, the agent can pre-determine a strategy: for the first 1 / 3 of the shallow network in the model, whose main function is basic feature extraction and whose structure allows for some flexibility, α can be set to a relatively loose value, such as 0.85; while for the last 1 / 3 of the deep network in the model, which deals with highly abstract semantic features, any structural deviation may have a significant impact on the final result, thus requiring stricter matching, and α should be set to a higher value, such as 0.95. This dynamic threshold strategy further improves the accuracy and flexibility of the mapping.

[0083] Step S130: Numerical Comparison and Anomaly Triggering

[0084] The third step of the method is to perform a numerical comparison of the output tensors of the target layer and the reference layer that have established a mapping relationship (S130), calculate their numerical differences, and mark the target layer as an abnormal layer when the numerical differences exceed a preset abnormal threshold.

[0085] This step is performed by the numerical comparison module 130. It iterates through all the mapping pairs generated in the previous step (including exact and fuzzy mappings). For each pair, it reads the output tensor T_target of the target layer (TensorFlow) and the output tensor T_ref of the reference layer (PyTorch) from the log. Then, it calculates the numerical difference between the two tensors. For a comprehensive evaluation, two metrics are typically calculated:

[0086] Relative difference (Δrel): sum(abs(T_target - T_ref)) / sum(abs(T_ref)). This metric measures the difference relative to the original value and is more equitable for tensors with larger values.

[0087] Absolute difference (Δabs): sum(abs(T_target - T_ref)). This metric measures the total amount of difference and is more sensitive to tensors with values ​​close to zero.

[0088] The numerical comparison module 130 compares the calculated Δrel and Δabs with globally configured empirical values, namely the anomaly thresholds ε_rel and ε_abs (e.g., ε_rel is set to 1e-4 and ε_abs to 1e-5). These thresholds represent tolerable normal computational noise introduced by factors such as different hardware floating-point units. If Δrel > ε_rel or Δabs > ε_abs, it indicates a significant difference in the outputs of these two layers that exceeds the normal range. In this case, the numerical comparison module 130 marks the target layer (a layer in TensorFlow) as an "anomaly layer" and passes its information (layer name, difference value, etc.) to the next stage, the difference analysis module 140.

[0089] Step S140: In-depth difference analysis

[0090] The fourth step of this method is to call a debugging tool to perform dynamic breakpoint debugging for the exception layer and combine it with abstract syntax tree comparison (S140) to analyze and locate the specific implementation differences that cause the numerical differences.

[0091] Once the difference analysis module 140 receives the anomaly layer marker, it initiates a thorough root cause investigation process. This process aims to answer the question, "Why is there such a large difference in the output of this layer?"

[0092] In a preferred embodiment, this step is performed collaboratively by the dynamic debugging unit 141, the abstract syntax tree generation unit 142, and the syntax tree comparison unit 143.

[0093] First, the dynamic debugging unit 141 uses Python's pdb library (or a similar debugging tool) to locate the entry point of the forward propagation function (call method) of the exception layer in the TensorFlow model source code and dynamically inserts a breakpoint there. Then, it re-initiates the model's forward inference. When the code execution reaches the breakpoint, the program pauses. At this point, the dynamic debugging unit 141 can fully access all variables within the function's scope, thereby capturing the most accurate runtime context information, including:

[0094] Input tensor X: The raw input tensor that enters this layer without any processing.

[0095] Weight parameter W: The weights and biases used in this layer.

[0096] Intermediate calculation result Y_mid: Inside the operator, the final output is the result of any intermediate steps before the final output is produced.

[0097] This information is crucial for understanding the computation process, and it is saved for subsequent analysis.

[0098] Meanwhile, the Abstract Syntax Tree (AST) generation unit 142 begins operation. It reads the source code (.py file) of the operator implementation for the exception layer in TensorFlow, and the source code of the corresponding reference layer in PyTorch. Using Python's built-in `ast` library, it parses these two source code texts into a tree-like data structure, namely an Abstract Syntax Tree (AST). The AST discards non-syntactic elements such as spaces and comments, retaining only the core program structure, such as variable assignment, function calls, control flow (if / for), and arithmetic operations.

[0099] Finally, the syntax tree comparison unit 143 performs a deep comparison of the generated first AST (TensorFlow) and second AST (PyTorch). It doesn't just perform a simple text comparison, but a structural comparison node by node and attribute by attribute. This comparison can uncover very subtle but crucial semantic differences. For example:

[0100] Differences in parameter initialization: It might find that PyTorch's Conv2d layer defaults to bias=True, while TensorFlow's Conv2D layer has use_bias=False under a certain configuration. On the AST, this manifests as a node having missing or different attributes.

[0101] Differences in computation order: It might find that one frame performs ReLU activation before dropout, while another frame performs it in the reverse order. In an AST, this manifests as a difference in the order of subtrees.

[0102] Differences in numerical precision handling: It may find that one framework uses tf.cast(x,tf.float32) to ensure precision when performing division operations, while another framework performs the calculation directly on FP16.

[0103] The syntax tree comparison unit 143 integrates all identified structural differences into a detailed difference analysis report. This report, along with runtime context information captured from the dynamic debugging unit 141, is sent to the code repair module 150.

[0104] Step S150: Smart Code Repair

[0105] The fifth step of the method is to input the implementation differences into a code generation model (S150) to generate repair code for repairing the target layer.

[0106] This step is performed by the code repair module 150, the core of which is a powerful code generation model. As a preferred implementation, this code generation model is based on a pre-trained Transformer architecture (such as T5, GPT, etc.) and fine-tuned using a massive corpus of publicly available code (such as Python code on GitHub). This fine-tuning makes it particularly adept at understanding the context and syntax of code, and capable of generating high-quality code snippets based on instructions.

[0107] The code repair module 150 integrates the information received from the difference analysis module 140 into a structured prompt, which is then input into the code generation model. This prompt typically includes:

[0108] Context: "You are an expert at fixing accuracy issues in TensorFlow models."

[0109] Operator information: "The target operator is tf.keras.layers.Conv2D, and the layer name is block1_conv1."

[0110] Difference description: "Compared to the PyTorch reference implementation, AST analysis revealed that the current implementation lacks an explicit setting for the padding strategy, while the reference implementation uses padding='same'."

[0111] Reference code: (Optional) Provide the corresponding code snippet in PyTorch.

[0112] Command: "Please generate the repaired TensorFlow code snippet".

[0113] After understanding this hint, the code generation model will generate a piece of fix code that conforms to TensorFlow syntax. For example, it might output: tf.keras.layers.Conv2D(..., padding='same', ...).

[0114] Step S160: Iterative verification and loop closure

[0115] The sixth step of the method is to hot-replace the repair code into the target model and repeat the numerical comparison step (S130) for iterative verification (S160) until the numerical difference is lower than the anomaly threshold.

[0116] This step is performed by the iterative verification module 160, which is responsible for closing the entire process into a loop.

[0117] First, it receives the repair code snippet generated by the code repair module 150. Then, it performs hot-swapping. In dynamic languages ​​like Python, this can be achieved by directly modifying the methods or properties of an object at runtime. For example, an agent can dynamically create a new Conv2D layer instance that uses the repaired code and then replace the old instance in the model with it.

[0118] After the replacement is completed, the iterative verification module 160 immediately triggers the numerical comparison module 130, but this time the comparison is limited to the one anomalous layer that was just repaired. The model re-executes the local forward inference starting from that layer and calculates the new numerical differences Δrel' and Δabs'.

[0119] The next step is to make a judgment:

[0120] If both Δrel' and Δabs' are below the anomaly threshold, the repair is considered successful. The iterative verification module 160 records this successful repair (including old code, new code, precision changes, etc.) and solidifies the modification. Then, the agent continues to process the next anomaly from the anomaly layer list.

[0121] If the numerical difference still exceeds the limit, or even becomes larger, it indicates that the generated code is incorrect or incomplete. In this case, as a preferred implementation, the iterative verification module 160 will perform a rollback operation, undoing the previous hot replacement and restoring the model to its pre-repair state. Simultaneously, it will send a regeneration instruction containing historical failure information to the code repair module 150. This instruction might include: "The last attempt to add padding='same' failed. Please analyze whether there are other differences, or try another repair method." Upon receiving this instruction, the code generation model will combine the experience of failures to generate an alternative repair solution.

[0122] This "repair-verify-rollback / confirm" cycle continues until the anomaly layer is successfully repaired or the preset maximum number of attempts is reached. Once all anomalies have been handled, the entire automated repair process ends. The agent ultimately generates a complete repair log report, detailing every code change, variation in accuracy metrics, and potential impact on inference latency, providing developers with comprehensive decision-making support.

[0123] Example 2: This example provides an agent-based cross-frame model inference accuracy anomaly localization and repair system 100 that implements the above method. (Refer to...) Figure 1 The system can be one or more servers connected via a network in terms of hardware, and its core is a series of software modules running on a processor.

[0124] The system includes:

[0125] A log capture module 110 is implemented as a set of hook functions that can be injected into a first inference framework (such as TensorFlow) and a second inference framework (such as PyTorch). This module is passively triggered during model inference and is responsible for capturing the operator metadata and output tensors of each network layer, and storing this information in a structured log storage area (such as a Redis database or a local JSON file) after formatting.

[0126] A cross-frame mapping module 120 is activated after logs from both frames have been generated. This module is further divided into:

[0127] Attribute extraction unit 121: Responsible for reading data from the log storage area and parsing out the key attributes of each layer.

[0128] Precise mapping unit 122: It has a built-in hash function (such as SHA-256) to generate a hash signature for each layer and establish a precise mapping by comparing the hash signatures.

[0129] Fuzzy Mapping Unit 123: Contains an algorithm implementation for calculating the cosine similarity of vectors. It reads a dynamic similarity threshold α from a configuration file and establishes a fuzzy mapping by calculating dimensional similarity when precise mapping fails.

[0130] A numerical comparison module 130 extracts corresponding output tensor pairs from the log storage area based on the mapping table output by the cross-frame mapping module 120. This module implements algorithms for calculating relative and absolute differences, compares them with a preset anomaly threshold ε, and finally outputs a list of anomaly layers.

[0131] A differential analysis module 140 is the system's "diagnostic core." It receives a list of anomaly layers and performs in-depth analysis on each. This module internally includes:

[0132] Dynamic debugging unit 141: It interacts with the target framework's debugging interface (such as pdb), and can set breakpoints at specified lines of code and capture runtime variables.

[0133] Abstract Syntax Tree Generation Unit 142: It has a built-in interface to Python's AST library, which can convert source code files or strings into AST objects.

[0134] Syntax Tree Comparison Unit 143: It implements a recursive tree traversal and comparison algorithm, which can identify structural differences between two ASTs and generate a structured difference report.

[0135] A code-fixing module (150) is the system's "surgical tool." At its core is an API client that encapsulates a large code generation model. This module is responsible for formatting the difference analysis report into model-friendly suggestions, sending requests, and receiving the returned code-fixing snippets.

[0136] An iterative verification module 160 serves as the system's "closed-loop controller." It is responsible for performing hot code replacements and coordinating the verification by the numerical comparison module 130. It also includes a rollback controller to undo changes in case of verification failure and manages interactions with the code repair module 150 to attempt alternatives.

[0137] Throughout the system, a top-level agent program is responsible for scheduling the execution order of these modules, managing data flow, and recording the final repair report, thereby achieving end-to-end automation. Finally, it should be emphasized that the embodiments described above are merely illustrative of the technical concept and preferred implementation of the present invention, and are not intended to exhaustively describe or limit the scope of protection of the present invention. Any person skilled in the art, after understanding the spirit and core technical solutions of the present invention, may make various modifications, equivalent substitutions, or improvements based on the content disclosed in the present invention without departing from its basic principles. These obvious variations or substitutions should all be considered to be included within the scope of protection claimed by the present invention.

Claims

1. A method for locating and repairing inference accuracy anomalies in cross-frame model based on intelligent agents, characterized in that, Includes the following steps: The same target model is loaded in the first and second inference frameworks, and the operator metadata and output tensors of each network layer are captured in real time through the injected hook function to generate a structured inference log. Based on the structured inference log, cross-frame alignment mapping is performed on the network layers of the first inference framework and the second inference framework to establish a mapping relationship between the target layer and the reference layer. For the target layer and the reference layer with the established mapping relationship, the output tensor values ​​are compared and their numerical differences are calculated. When the numerical difference exceeds a preset abnormal threshold, the target layer is marked as an abnormal layer. For the aforementioned exception layer, the debugging tool is invoked to perform dynamic breakpoint debugging and combined with abstract syntax tree comparison to analyze and locate the specific implementation differences that cause the numerical differences; The implementation differences are input into a code generation model to generate repair code for fixing the target layer. The repair code is hot-replaced into the target model, and the numerical comparison step is repeated for iterative verification until the numerical difference is lower than the anomaly threshold. The steps of analyzing and locating the specific implementation differences that cause the numerical differences include: Dynamic breakpoints are inserted at the anomaly layer to capture its runtime context information during the inference process. The runtime context information includes the input tensor, weight parameters, and intermediate calculation results. The operator implementation code of the target layer in the first inference framework and the operator implementation code of the reference layer in the second inference framework are parsed to generate corresponding first abstract syntax trees and second abstract syntax trees, respectively. By comparing the first abstract syntax tree and the second abstract syntax tree, structural differences in parameter initialization, calculation order, or numerical precision processing are identified as the specific implementation differences.

2. The method for anomaly localization and repair of inference accuracy in a cross-frame model based on an agent, as described in claim 1, is characterized in that: The steps of the cross-frame alignment mapping include: Extract the preset attributes of each layer of operators, including operator type, input dimension, output dimension, number of parameters, and parameter type; A unique hash signature is generated for each layer based on the preset attributes. If the hash signatures of the target layer and the reference layer are consistent, an exact mapping is established. If the hash signatures are inconsistent, the cosine similarity between the input dimension vector and the output dimension vector of the target layer and the reference layer is calculated. When the cosine similarity is higher than the preset similarity threshold and the output dimensions are consistent, a fuzzy mapping is established.

3. The method for locating and repairing inference accuracy anomalies in a cross-frame model based on agents according to claim 2, characterized in that: The hash signature is generated by performing a hash operation on serialized data with operator type, input dimension, output dimension, number of parameters, and parameter type; and the similarity threshold is dynamically adjusted according to the depth of the network layer, with the similarity threshold corresponding to the deep network being higher than that corresponding to the shallow network.

4. The method for anomaly localization and repair of inference accuracy in a cross-frame model based on an agent, as described in claim 1, is characterized in that: The code generation model is a model obtained by fine-tuning a pre-trained Transformer architecture using code corpus; the iterative verification step further includes: if the verification fails, rolling back the hot replacement operation and instructing the code generation model to generate alternative repair code based on historical repair records.

5. A system for locating and repairing anomalies in inference accuracy of cross-frame models based on intelligent agents, characterized in that: include: The log capture module (110) is used to capture the operator metadata and output tensors of each network layer in real time through the injected hook function in the same target model running in the first inference framework and the second inference framework respectively, and generate structured inference logs. The cross-frame mapping module (120) is used to perform alignment mapping on the network layers of the first inference framework and the second inference framework based on the structured inference log, and to establish a mapping relationship between the target layer and the reference layer. The numerical comparison module (130) is used to perform numerical comparison of the output tensors of the target layer and the reference layer with the established mapping relationship, calculate their numerical differences, and output an abnormal layer mark when the numerical difference exceeds a preset abnormal threshold. The difference analysis module (140) is used to call the debugging tool to perform dynamic breakpoint debugging on the abnormal layer after receiving the abnormal layer mark, and to analyze and locate the specific implementation differences that cause the numerical difference by comparing the abstract syntax tree. The code repair module (150) is used to input the implementation differences into a code generation model to generate repair code for repairing the target layer; The iterative verification module (160) is used to hot-replace the repair code into the target model and trigger the numerical comparison module to re-verify until the numerical difference is lower than the anomaly threshold. The difference analysis module (140) includes: The dynamic debugging unit (141) is used to insert dynamic breakpoints at the code positions corresponding to the exception layer and capture runtime context information during the inference process, including input tensors, weight parameters, and intermediate calculation results. Abstract syntax tree generation unit (142) is used to parse the operator implementation code of the target layer and the reference layer, and generate the corresponding first abstract syntax tree and second abstract syntax tree respectively; The syntax tree comparison unit (143) is used to compare the first abstract syntax tree with the second abstract syntax tree, identify and output structural differences in parameter initialization, calculation order or numerical precision processing.

6. The system for locating and repairing anomalies in inference accuracy of a cross-frame model based on an agent, as described in claim 5, is characterized in that: The cross-frame mapping module (120) includes: The attribute extraction unit (121) is used to extract the preset attributes of each layer operator. The preset attributes include operator type, input dimension, output dimension, number of parameters and parameter type. The precise mapping unit (122) is used to generate a unique hash signature for each layer based on the preset attribute, and to establish a precise mapping when the hash signatures of the target layer and the reference layer are detected to be consistent. The fuzzy mapping unit (123) is used to calculate the cosine similarity between the input dimension vector and the output dimension vector of the target layer and the reference layer when the hash signatures are inconsistent, and to establish a fuzzy mapping when the cosine similarity is higher than a preset similarity threshold and the output dimensions are consistent.

7. The system for locating and repairing inference accuracy anomalies in a cross-frame model based on an agent, as described in claim 6, is characterized in that: The precise mapping unit (122) generates the hash signature by performing a hash function on serialized data of operator type, input dimension, output dimension, number of parameters and parameter type; the fuzzy mapping unit (123) is configured to read from memory and apply different similarity thresholds based on the depth information of the network layer.

8. The system for locating and repairing anomalies in inference accuracy of a cross-frame model based on an agent, as described in claim 5, is characterized in that: The code repair module (150) includes a code generation model based on a pre-trained Transformer architecture and fine-tuned using code corpus; the iterative verification module (160) also includes a rollback controller, which is used to undo the hot replacement operation of the repair code when verification fails, and send a regeneration instruction containing historical failure information to the code repair module.

Images