Method, apparatus, device and medium for link insight based on uniform tracking identifier

Abstract

The application relates to the technical field of software monitoring, and discloses a link insight method, device, equipment and medium based on unified tracking identification, which comprises the following steps: collecting application semantic data and infrastructure operation data through an application layer probe and a kernel layer probe respectively; associating the two types of data based on unified tracking identification and time sequence information to generate full-link tracking data; organizing sessions and workflows, determining calling relationships and context transmission relationships, and generating dynamic calling path data; performing abnormal positioning, root cause tracing, cost analysis, quality analysis and performance analysis based on the dynamic calling path data to generate link insight results. The application can be applied to business scenarios such as financial technology and medical health, application semantic data and infrastructure operation data are collected through mixed probes, association is established based on unified tracking identification and time sequence information, a calling link that penetrates application operation and infrastructure operation processes is formed, and the link tracking integrity and fault root cause positioning accuracy are improved.

Description

Technical Field

[0001] This invention relates to the field of software monitoring technology, and in particular to a link insight method, apparatus, device and medium based on unified tracking identifiers. Background Technology

[0002] Currently, the observability of large-scale model applications typically involves setting up observation systems for the application invocation process and the infrastructure operation process, respectively. The application side records business semantic information such as model invocation, prompt content processing, vector retrieval, tool invocation, and output generation. The infrastructure side records operational information such as resource scheduling, storage and transmission, cross-node communication, and input / output access. Because the two types of observation data differ in recording objects, time granularity, and identification methods, application operation records and underlying operation records are often scattered across different tracing paths and monitoring views during multi-round sessions, workflow branches, call retries, and continuous context transmission. This necessitates manual cross-layer comparison and correlation.

[0003] In the fintech business, large-scale models are increasingly being applied to credit review, insurance underwriting, claims processing, fraud analysis, investment advisory, financial compliance review, and customer service. A single financial transaction may involve continuous processes such as customer information retrieval, financial knowledge inquiry, risk data processing, external tool invocation, model inference, and result verification, spanning multiple service nodes and processing resources. When credit review responses are delayed, claims processes are interrupted, or fraud analysis encounters anomalies, business session records, model invocation records, tool invocation records, and resource scheduling records are typically presented separately. Operations personnel need to repeatedly verify the invocation time, node status, and context changes across different data sources to reconstruct the invocation process before and after the anomaly occurred.

[0004] In the healthcare field, large-scale models are increasingly being applied to intelligent triage, medical record organization, medical knowledge retrieval, examination report interpretation, medication review, and clinical auxiliary analysis. These operations typically require continuous execution of information retrieval, medical knowledge retrieval, professional tool processing, and model generation within a patient session, while transmitting symptom descriptions, examination results, and diagnostic context between different processing nodes. When medical retrieval is delayed, medical record processing is interrupted, or model generation status is abnormal, patient sessions, medical workflows, model call semantics, and underlying operational status are usually distributed across different observation records. Continuously reconstructing the retrieval nodes, tool nodes, model nodes, and processing resource nodes involved in the anomalies requires comparison of data from multiple sources. Summary of the Invention

[0005] The main objective of this invention is to provide a link insight method, apparatus, device, and storage medium based on a unified tracking identifier, aiming to solve the technical problem in the prior art where application layer business semantic tracking and infrastructure layer tracking are disconnected, making it difficult to reconstruct the complete link of model calls and thus making it difficult to locate the root cause of the fault.

[0006] To achieve the above objectives, the present invention provides a link insight method based on a unified tracking identifier, comprising: Application semantic data is generated by collecting application semantic data during the model call process through application layer probes, and infrastructure operation data is generated by collecting infrastructure operation data through kernel layer probes. Obtain a unified tracking identifier and timing information, and associate the application semantic tracking data with the infrastructure operation tracking data based on the unified tracking identifier and timing information to generate end-to-end tracking data; Based on the end-to-end tracing data, sessions and workflows are organized to determine call relationships and context passing relationships, and dynamic call path data is generated based on the call relationships and context passing relationships. Based on the dynamic call path data, anomaly location and root cause tracing are performed to generate root cause analysis data. Based on the dynamic call path data, call cost analysis, output quality analysis, and runtime performance analysis are performed to generate cost analysis data, quality analysis data, and performance analysis data. Link insight results are generated based on the root cause analysis data, the cost analysis data, the quality analysis data, and the performance analysis data.

[0007] Furthermore, to achieve the above objectives, the present invention provides a link insight device based on a unified tracking identifier, comprising: The hybrid probe acquisition module is used to collect application semantic data during the model call process through application layer probes to generate application semantic tracing data, and to collect infrastructure operation data through kernel layer probes to generate infrastructure operation tracing data. The end-to-end association module is used to obtain unified tracking identifiers and time-series information, and associate the application semantic tracking data with the infrastructure operation tracking data based on the unified tracking identifiers and time-series information to generate end-to-end tracking data. The call path construction module is used to organize sessions and workflows based on the end-to-end tracing data, determine call relationships and context passing relationships, and generate dynamic call path data based on the call relationships and context passing relationships. The root cause analysis module is used to locate anomalies and trace root causes based on the dynamic call path data, and generate root cause analysis data. The multidimensional analysis module is used to perform call cost analysis, output quality analysis, and runtime performance analysis based on the dynamic call path data, and generate cost analysis data, quality analysis data, and performance analysis data. The link insight module is used to generate link insight results based on the root cause analysis data, the cost analysis data, the quality analysis data, and the performance analysis data.

[0008] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a link insight program based on a unified tracing identifier stored in the memory and executable on the processor, wherein when the link insight program based on a unified tracing identifier is executed by the processor, it implements the steps of the link insight method based on a unified tracing identifier as described above.

[0009] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a link insight program based on a unified tracing identifier, wherein the link insight program based on a unified tracing identifier, when executed by a processor, implements the steps of the link insight method based on a unified tracing identifier as described above.

[0010] Beneficial Effects: This invention relates to the field of software monitoring technology and discloses a link insight method, apparatus, device, and medium based on a unified tracing identifier. The method includes: collecting application semantic data and infrastructure operation data through application-layer probes and kernel-layer probes respectively, generating corresponding tracing data; associating the two types of tracing data based on a unified tracing identifier and timing information to generate full-link tracing data; organizing sessions and workflows based on the full-link tracing data, determining call relationships and context passing relationships, and generating dynamic call path data; and performing anomaly localization, root cause tracing, call cost analysis, output quality analysis, and operational performance analysis based on the dynamic call path data to generate link insight results. This invention can be applied to business scenarios such as fintech and healthcare. By acquiring application semantic data and infrastructure operation data through hybrid probes and establishing associations using a unified tracing identifier and timing information, a dynamic call path is formed that connects application operations and infrastructure operation processes. This allows anomaly propagation, root cause nodes, call costs, output quality, and operational performance to be analyzed within the same link, thereby improving the completeness of link tracing and the accuracy of fault root cause localization. Attached Figure Description

[0011] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a schematic diagram of an application environment for a link insight method based on a unified tracking identifier according to an embodiment of the present invention; Figure 2This is a flowchart illustrating an embodiment of the link insight method based on unified tracking identifiers of the present invention; Figure 3 for Figure 2 A detailed flowchart of step S10; Figure 4 for Figure 2 A detailed flowchart of step S20; Figure 5 for Figure 2 A detailed flowchart of step S30; Figure 6 for Figure 2 A detailed flowchart of step S40; Figure 7 for Figure 2 A detailed flowchart of step S50; Figure 8 for Figure 2 A detailed flowchart of step S60; Figure 9 This is a schematic diagram of the functional modules of a preferred embodiment of the link insight device based on unified tracking identifier of the present invention; Figure 10 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 11 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0012] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0013] The link insight method based on unified tracking identifiers provided in this invention can be applied to, for example... Figure 1In this application environment, the client communicates with the server via a network. The server can use application-layer probes and kernel-layer probes to collect application semantic data and infrastructure operation data respectively, generating corresponding tracing data. Based on a unified tracing identifier and timing information, the two types of tracing data are correlated to generate end-to-end tracing data. Based on the end-to-end tracing data, sessions and workflows are organized to determine call relationships and context passing relationships, generating dynamic call path data. Based on the dynamic call path data, anomaly localization, root cause tracing, call cost analysis, output quality analysis, and operational performance analysis are performed to generate link insight results. This invention can be applied to business scenarios such as fintech and healthcare. By acquiring application semantic data and infrastructure operation data through hybrid probes and establishing correlations using unified tracing identifiers and timing information, a dynamic call path is formed that connects application operations and infrastructure operation processes. This allows anomaly propagation, root cause nodes, call costs, output quality, and operational performance to be analyzed within the same link, thereby improving the completeness of link tracing and the accuracy of fault root cause localization. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The present invention will now be described in detail through specific embodiments.

[0014] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the link insight method based on a unified tracking identifier provided by the present invention. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0015] like Figure 2 As shown, the link insight method based on unified tracking identifier proposed in this invention includes the following steps: S10: Application semantic data is collected during the model call process through application layer probes to generate application semantic tracking data, and infrastructure operation data is collected through kernel layer probes to generate infrastructure operation tracking data. In this embodiment, application layer probes are set at model access, prompt content organization, retrieval, tool invocation, or output reception locations. They can collect invocation events using code embedding, function interception, runtime instrumentation, proxy forwarding, or callback listening. Application semantic data is used to record operation states with business meaning, and may include operation type, invocation identifier, time information, input summary, output summary, invocation status, model identifier, and generation unit consumption. When sensitive content is involved, field anonymization, hash conversion, or content summary retention of tracking information can be used. Collected events are merged according to invocation identifiers and organized based on time information to generate application semantic tracking data.

[0016] Kernel-level probes are positioned at kernel event locations corresponding to resource scheduling, memory access, device tasks, network communication, storage read / write, or process switching. They can collect infrastructure operational data through kernel tracepoints, performance event interfaces, system call interceptions, driver callbacks, or device telemetry interfaces. Infrastructure operational data may include resource operation type, event time, duration, process information, thread information, running node information, and resource object information. Collected events are grouped by resource object, process, or thread and organized according to event time to generate infrastructure operational tracing data.

[0017] Where modification of the model call code is permitted, application-layer probes are set up in model access functions, retrieval functions, and tool call functions using a software development kit (SDK). Events are written at the start, return, abnormal exit, or timeout of a function call, and application semantic tracing data is generated in batches via a memory queue. Kernel-layer probes are attached to process scheduling, network transmission and reception, file access, and device driver event locations. The collected kernel events are written to a buffer and converted into infrastructure runtime tracing data. When the call volume is large, the content digest length can be shortened, while retaining a more complete record of abnormal calls.

[0018] When application code cannot be modified, a model gateway or runtime proxy is used to intercept model requests, model responses, streaming data, and tool call messages to generate application semantic data. Node-level data acquisition programs are used to monitor kernel events and acceleration device events, and infrastructure runtime data is aggregated based on process, thread, or container information. When multiple runtime nodes participate in model inference, event buffers can be set separately, and the time fields and resource operation fields of each node can be unified.

[0019] In the fintech business field, when the model is used for insurance claims review, the application layer probe can collect the operation type, time information and call status of material parsing, policy clause retrieval, risk verification tool calls and model output generation; the kernel layer probe can collect acceleration device tasks, memory transfer, database access and file read events during model inference.

[0020] In the healthcare business field, when the model is used for health consultation, the application layer probe can collect operation events corresponding to consultation content parsing, health information retrieval, medical knowledge retrieval, and response generation; the kernel layer probe can collect processor scheduling, accelerated device operation, storage retrieval, and network communication events.

[0021] This embodiment records business operation events during model invocation through application layer probes and resource events related to computation, storage, communication, and input / output through kernel layer probes, thereby forming continuous and structured tracking data for application operation status and infrastructure operation status, thus improving the integrity of model invocation observation data.

[0022] S20, obtain unified tracking identifier and timing information, and associate the application semantic tracking data with the infrastructure operation tracking data based on the unified tracking identifier and timing information to generate full-link tracking data; In this embodiment, a unified tracking identifier is used to distinguish different model call requests. It can be generated when a request enters the model application and is passed along with the call context during model access, retrieval, tool invocation, and cross-process communication. Operation records in application semantic tracking data are written to the unified tracking identifier, and infrastructure operation tracking data can be written to the same identifier based on the identifier mapping information maintained by processes, threads, or running nodes, thereby grouping two types of records belonging to the same model call into the same data set.

[0023] Timing information includes the start time, end time, duration, and order of each operation. When different running nodes use different timing sources, the time field can be corrected based on the clock offset, or the local monotonic clock can be converted to a unified time base. Data corresponding to the same model call is filtered according to a unified tracking identifier, and then a correspondence is established between application semantic operations and concurrent resource scheduling, data transmission, network communication, and storage access records based on overlapping time intervals, time inclusion, or adjacent time relationships. The association results retain the application operation content, infrastructure operating status, time relationships, and cross-layer correspondences, generating complete tracking data.

[0024] When the model request context can be passed across components, a tracing identifier generated at the request entry point is used to mark each application operation, and this tracing identifier is written to the process or thread context mapping table. After a kernel event occurs, the mapping table is queried based on the process identifier and thread identifier to supplement the infrastructure operation record with tracing identifiers. The two types of records are grouped according to the tracing identifier, and time interval matching is completed based on the start and end times.

[0025] When model inference spans multiple runtime nodes, node time calibration information is used to correct time discrepancies in the records of each node, and a tracking identifier is carried in the request message. When some infrastructure events cannot directly carry tracking identifiers, the corresponding model call is determined based on the runtime node, process, thread, and event time window. When multiple application operations exist within a time window, the infrastructure runtime records are associated with application operations that are closer in time range or have a clearer inclusion relationship.

[0026] In the field of fintech business, when the insurance claims model performs material analysis, policy retrieval and model inference in sequence, each application operation carries the same tracking identifier. The computing resource scheduling, database access and file reading records are associated with the corresponding operations according to the process, thread and time interval, thus forming complete tracking data for the same claims request.

[0027] In the healthcare business, when a health consultation model completes health information retrieval and response generation across nodes, each node transmits a unified tracking identifier and corrects for node time deviations. Network communication records generated during medical knowledge retrieval and acceleration device operation records generated during model inference are associated with corresponding application operations according to the tracking identifier and time interval.

[0028] This embodiment aggregates application semantic tracking data and infrastructure operation tracking data generated from the same model call by using a unified tracking identifier, and uses time sequence information to correct the recording order and establish cross-layer correspondence, so that data scattered at different collection levels can form continuous and verifiable complete tracking data, thereby reducing the correspondence deviation between application operation and resource operation records.

[0029] S30, organize sessions and workflows based on the full-link tracing data, determine call relationships and context passing relationships, and generate dynamic call path data based on the call relationships and context passing relationships; In this embodiment, operation records in the complete tracking data are aggregated according to session identifiers, request identifiers, and time boundaries. Multiple rounds of input, model calls, retrieval operations, and tool calls within the same interaction cycle are organized into session node groups. Session node groups are further subdivided according to workflow identifiers, task identifiers, or operation dependencies, ensuring that operation records undertaking the same processing task are grouped into the same workflow node group. Call relationships are determined based on node identifiers, parent references, trigger records, and time sequence, used to represent initiation, branching, retry, and return relationships between nodes. Context passing relationships are established based on the source node, receiving node, field version, and passing direction of the context field, used to record the flow of prompts, search results, tool return values, and intermediate states between different nodes. All nodes are arranged in chronological order and connected through call relationships and context passing relationships, forming dynamic call path data that includes node states, execution order, and branch changes.

[0030] When the calling framework can pass session identifiers and workflow identifiers, the corresponding identifiers are written using request headers, call context, or message attributes, and each operation record is grouped according to the identifier. Node identifiers and parent node identifiers are used to establish the call direction. The context field records the source node and version information before sending and the receiving node after receiving, thus forming a context passing relationship. When a model request branches or retryes, a branch identifier or retry identifier is written for the newly added node, and the newly added node is continued to the original call position.

[0031] When some components lack complete identifiers, record matching is performed using request identifiers, process information, operation time ranges, and message digests. Operations with adjacent times and sequential input-output relationships are grouped into the same session or workflow. When the context content is large, changes are recorded using field digests, version numbers, and content fingerprints, eliminating the need to repeatedly save complete content. When there are many concurrent workflows, node sets are maintained separately according to workflow identifiers, and node connections are established only when there is a call or context passing, reducing cross-associations between unrelated records.

[0032] In the fintech business, when insurance claims models process multiple rounds of supplementary materials for the same claim application, they can aggregate material parsing, policy retrieval, risk verification, and model generation records according to the claims session identifier, and organize each node according to the claims review workflow. Policy liability information, material summaries, and verification results are recorded and transmitted according to the source node, receiving node, and version information, forming dynamic call path data corresponding to the claim request.

[0033] In the healthcare business field, when a health consultation model receives symptom descriptions, examination summaries, and medication information, it can group multiple rounds of consultation records into a single session and divide health record queries, medical knowledge retrieval, medication verification, and response generation into corresponding workflows. Symptom information, search results, and medication verification results are transmitted along each node, forming dynamic call path data corresponding to the health consultation process.

[0034] This embodiment aggregates scattered operation records through sessions and workflows, and connects each processing node using call relationships and context passing relationships. This enables multi-round interactions, branch calls, retry calls, and context transitions to be presented continuously with dynamic call path data, thereby improving the integrity of the restoration of complex model call processes.

[0035] S40, based on the dynamic call path data, perform anomaly location and root cause tracing to generate root cause analysis data; In this embodiment, the call status, duration, retry status, resource usage status, and time information of each node are extracted from the dynamic call path data and compared with the normal state range of the corresponding node. Nodes exceeding preset judgment conditions are identified as abnormal nodes. The time of occurrence of the abnormality, the type of abnormality, and the degree of deviation together constitute the abnormality localization result. Root cause tracing extends upstream along the call relationship and context propagation relationship between nodes. Combining the time of occurrence of the abnormality, it is determined whether there is a sequential relationship between the abnormal nodes. The abnormal nodes that can be continuously connected are arranged according to the propagation direction to form the abnormality propagation path. The root cause node is determined from the abnormal propagation path. It usually corresponds to the node where the abnormality occurred earlier, can connect to multiple abnormal nodes downstream, and whose abnormal state matches the changes of subsequent nodes. Root cause analysis data may include abnormal propagation path, root cause node, abnormal state of each node, and propagation relationship between nodes.

[0036] Assuming each node has a stable operating baseline, the normal range is determined by the historical distribution of node duration, call status, retries, and resource usage. When the current node's data exceeds the corresponding range, the node is marked as an abnormal node, and related nodes are filtered upstream according to the time of the abnormality and the direction of the call. When multiple upstream abnormal nodes exist, the propagation order is determined based on time proximity, call dependency strength, and context propagation consistency to generate an abnormal propagation path.

[0037] When node operating status fluctuates significantly, a sliding time window is used to update the node baseline, and weights are assigned to the duration deviation, call failure status, and resource usage deviation. Abnormal nodes are ranked according to their overall deviation degree and upstream / downstream connectivity, and upstream nodes that can explain downstream abnormal changes are selected as root cause nodes. When multiple anomaly sources appear in parallel branches, multiple anomaly propagation paths can be generated, and a corresponding root cause node is determined for each path.

[0038] In the fintech business, when insurance claims models experience delays in review responses, abnormal nodes can be identified based on the duration and call status of the document parsing, policy retrieval, risk verification, and model generation nodes. If the policy retrieval node times out early, and subsequent risk verification and model generation nodes continuously experience waiting times, the relevant nodes can be organized into an abnormal propagation path, and the policy retrieval node can be identified as the root cause node.

[0039] In the healthcare business, when a health consultation model exhibits abnormal response generation, the state changes of nodes related to health information retrieval, medical knowledge inquiry, medication verification, and model generation can be analyzed. If the medical knowledge inquiry node returns abnormal content and this is passed on through context to affect subsequent generation nodes, an abnormal propagation path can be formed along the relevant nodes, and the medical knowledge inquiry node can be identified as the root cause node.

[0040] This embodiment identifies anomalies in the node states of dynamic call paths and traces upstream by combining call relationships, context passing relationships, and time relationships. It can organize scattered abnormal nodes into anomaly propagation paths with propagation directions and identify root cause nodes from them, thereby improving the accuracy of anomaly location and root cause judgment in complex model call processes.

[0041] S50, based on the dynamic call path data, perform call cost analysis, output quality analysis and runtime performance analysis, and generate cost analysis data, quality analysis data and performance analysis data; In this embodiment, the dynamic call path data includes the model type, number of calls, generation unit consumption, resource usage, input content, output content, stage time, and inter-node relationships for each call node. Call cost analysis aggregates the consumption generated by each node according to the conversion parameters corresponding to model consumption and resource consumption, generating cost analysis data that can distinguish between model call consumption and resource usage consumption. Output quality analysis judges the degree of matching between input and output content, the consistency between output and reference content, content completeness, and security anomalies, organizing the judgment results into quality analysis data. Operational performance analysis determines the duration and resource distribution of each operational stage based on the node's start time, end time, waiting time, and resource usage status, generating performance analysis data. All three types of analysis retain the associated information of the corresponding call nodes and call processes, enabling the analysis results to reflect the cost, quality, and operational status of specific nodes.

[0042] When the model type and resource configuration are relatively fixed, a pre-configured conversion table is used to record the consumption parameters corresponding to different models, computing resources, and storage resources. The number of generated units, resource usage, and time occupied are collected item by item according to the calling node. Output quality is judged by input-output field matching, reference content comparison, and sensitive content detection. Running performance is analyzed by stage time consumption statistics and resource usage interval statistics.

[0043] In scenarios involving parallel calls to multiple models and dynamic changes in resource configuration, a conversion parameter table with version identifiers is used to match model versions and resource types, and the consumption of each node is summarized according to the workflow stage. When the output content is long, it is compared in segments according to paragraphs or semantic units, and the quality results are summarized. When the call process includes parallel branches, the duration and resource consumption of each branch are calculated separately, and performance analysis data is generated according to the branch relationship.

[0044] In the field of fintech business, when the insurance claims model calls the policy retrieval, risk verification and text generation components, it can count the consumption and resource usage of the generation units at each node, judge the consistency between the review results and the policy terms and claims materials, and analyze the duration of each node to form corresponding cost analysis data, quality analysis data and performance analysis data.

[0045] In the healthcare business field, when the health consultation model calls the components for health information retrieval, medical knowledge query and response generation, it can collect the consumption of each node, determine the degree of matching between the response content and the consultation information and medical reference content, and statistically analyze the time consumption and resource usage status of the retrieval and generation stages, forming three types of corresponding analytical data.

[0046] This embodiment classifies and analyzes the model consumption, resource consumption, input and output content, and runtime sequence in the dynamic call path data, which can generate cost analysis data, quality analysis data, and performance analysis data respectively. This allows the consumption, output status, and running efficiency of each call node to be presented in a structured manner, improving the data analysis completeness of the model call process.

[0047] S60, Generate link insight results based on the root cause analysis data, the cost analysis data, the quality analysis data, and the performance analysis data.

[0048] In this embodiment, root cause analysis data is used to characterize the anomaly propagation path, root cause nodes, and node anomaly states; cost analysis data is used to characterize the consumption generated by model calls and resource usage; quality analysis data is used to characterize the matching status and security anomalies between output content and input / reference content; and performance analysis data is used to characterize the duration and resource usage status of each call phase. These four types of data are merged according to a unified call identifier, node identifier, or time information, establishing a connection between the analysis content corresponding to the same model call and the same call node. The merged data is then used to form insight items according to the root cause, cost, quality, and performance dimensions. Each insight item is configured to its corresponding call node, and the insight items are organized according to the node arrangement relationship in the call path to generate insight results containing the anomaly source, consumption, output status, and running status. When some nodes lack data for a certain dimension, other already formed insight items are retained, without affecting the data organization of the corresponding node.

[0049] When various analytical data use a fixed field structure, a unified call identifier is used to establish a call-level data set, and a node identifier is used to establish a node-level data set. Root cause analysis data is written to the data set corresponding to the abnormal node, while cost analysis data, quality analysis data, and performance analysis data are written to their respective sets according to the node identifier. Then, various insight items are organized through a preset output template to generate structured insight results.

[0050] As various analytical data are continuously generated, an incremental merging approach is used to maintain the data status of the calling nodes. Upon the arrival of new root cause, cost, quality, or performance data, the insights for the corresponding nodes are updated based on the unified calling identifier and node identifier, and the node order is reorganized according to the current calling path. When parallel branches exist, the insights in each branch are organized separately; when missing data exists, only existing analytical dimensions are output, and missing dimension markers are recorded.

[0051] In the fintech business, when the insurance claims review process includes material analysis, policy retrieval, risk verification, and model generation, the anomaly propagation path, the call cost of each node, the quality of the review results, and the performance of each stage can be merged into the corresponding node to generate insight results that can reflect the source of the anomaly, resource consumption, and the status of the review output.

[0052] In the healthcare business field, when the health consultation process includes health information retrieval, medical knowledge inquiry, medication verification and response generation, root cause analysis data, call consumption, response quality and operational performance can be organized to the corresponding nodes to generate insight results that can present abnormal nodes, content status and stage operation status.

[0053] This embodiment integrates root cause, cost, quality, and performance analysis data by unifying call identifiers, node identifiers, and time information, and organizes the analysis content of each dimension to the corresponding call node. This can form insight results that simultaneously reflect the source of the anomaly, call consumption, output status, and running status, thereby improving the completeness of the correlation between multi-dimensional analysis data and the clarity of the call process presentation.

[0054] In one embodiment, such as Figure 3 As shown, step S10 above includes: S101, set up application layer probes at the model call entry, prompt content processing entry, vector retrieval entry, tool call entry and streaming generation entry, and record the application semantic operation identifier, parent application semantic operation identifier, application operation type, application start time, application end time, input summary, output summary, call status, running node identifier, process identifier and thread identifier of each application semantic operation through the application layer probes, and record the model identifier and generation unit consumption corresponding to the model call operation to form application semantic data in the model call process; S102, based on the application semantic operation identifier, merge the application semantic data corresponding to the same application semantic operation to generate application semantic operation records, and organize each application semantic operation record according to the parent application semantic operation identifier, the application start time and the application end time to generate application semantic tracking data. S103, set kernel layer probes at kernel event mount points corresponding to resource scheduling, storage transmission, cross-node communication, process switching and input / output access, and record infrastructure operation identifier, infrastructure operation type, infrastructure start time, infrastructure end time, process identifier, thread identifier, running node identifier and resource object identifier of each infrastructure operation through the kernel layer probes to form infrastructure operation data; S104, based on the infrastructure operation identifier, merge the infrastructure operation data corresponding to the same infrastructure operation operation, generate infrastructure operation record, and organize each infrastructure operation record according to the process identifier, the thread identifier, the running node identifier, the resource object identifier, the infrastructure start time and the infrastructure end time to generate infrastructure operation tracking data.

[0055] In this embodiment, application-layer probes can be set at function entry points, call interceptors, or callback locations in the model access interface, prompt content rendering component, vector retrieval component, tool scheduling component, and streaming output component. The model call entry point is used to capture model request initiation, return, exception, and timeout events; the prompt content processing entry point is used to record template loading, variable population, and prompt content generation events; the vector retrieval entry point is used to record retrieval requests, retrieval result returns, and retrieval status; the tool call entry point is used to record tool selection, parameter passing, and tool return; and the streaming generation entry point is used to record the start of streaming output, data fragment return, and generation end. Probes can intercept write events through code embedding, function wrapping, runtime instrumentation, or proxies, avoiding changes to the original call logic of each business operation.

[0056] Application semantic operation identifiers are used to aggregate events generated during the start, end, exception, or segmented return of the same operation. These identifiers can be generated when the operation is created and remain unchanged throughout the operation's lifespan. Parent application semantic operation identifiers are derived from the current call context and are used to represent the hierarchical relationship between model calls, prompt content processing, vector retrieval, tool calls, and streaming generation. Application operation types are used to distinguish different semantic operations; the application start time and application end time are recorded at the operation entry and exit positions, respectively; input and output summaries can be generated through field extraction, content truncation, hash transformation, or structured compression; the call status is determined based on normal return, abnormal exit, timeout, or cancellation. Run node identifiers, process identifiers, and thread identifiers are read from the current runtime environment and are used to record the actual location where the operation occurred. Model call operations also record the model identifier and generation unit consumption. The model identifier can consist of the model name, version, or deployment instance marker, and the generation unit consumption can be determined based on input generation units, output generation units, or a combination of both.

[0057] The same application semantic operation may generate start events, end events, streaming fragment events, and exception events. These events are grouped into application semantic operation records based on the application semantic operation identifier. During the merging process, start time, end time, call status, and input / output summary can be added according to the event type, and duplicate events are deduplicated. A parent-child hierarchy is established based on the parent application semantic operation identifier, and the time range and nesting relationship of each record are determined based on the application start time and application end time, thereby forming application semantic tracking data that includes operation content, hierarchical relationships, and time relationships.

[0058] Kernel-level probes can be attached to kernel trace points, performance event locations, system call entry points, or driver callback locations corresponding to resource scheduling, storage transfer, cross-node communication, process switching, and I / O access. Resource scheduling records task waiting, scheduling, and running events; storage transfer records data movement between memory, device memory, or storage media; cross-node communication records connection establishment, data transmission, and data reception; process switching records thread or process state transitions; and I / O access records file, block device, or network I / O events. When an event occurs, the probe writes the infrastructure operation identifier, operation type, start time, end time, process identifier, thread identifier, running node identifier, and resource object identifier. The resource object identifier can represent a processor, acceleration device, memory region, network connection, storage volume, file object, or I / O channel.

[0059] Infrastructure operation processes may consist of multiple kernel events. Start, end, and status events corresponding to the same operation are merged based on the infrastructure operation identifier to generate an infrastructure operation record. Process and thread identifiers are used to determine the running entity to which the operation belongs, running node identifiers are used to distinguish different nodes, and resource object identifiers are used to distinguish specific resources. Records are arranged according to the infrastructure start and end times, and grouped by process, thread, node, and resource object to form infrastructure operation tracing data that reflects resource scheduling, data transmission, communication, and input / output processes.

[0060] This embodiment sets probes at application semantic operation locations and kernel runtime event locations respectively, which can synchronously record the business operation status of model calls and the underlying resource runtime status. By merging and dispersing events by operation identifiers and organizing records by combining parent identifiers, time information and runtime environment identifiers, event duplication, unclear attribution and missing time relationships can be reduced, thereby improving the completeness, structure and recording accuracy of model call observation data.

[0061] In one embodiment, such as Figure 4 As shown, step S20 above includes: S201, obtain a unified tracking identifier from the model call context, obtain application semantic operation records, parent-child hierarchical relationships between application semantic operation records, application start time, application end time and application running association information from the application semantic tracking data, and obtain infrastructure running operation records, infrastructure start time, infrastructure end time and infrastructure running association information from the infrastructure running tracking data. The application running association information and the infrastructure running association information both include running node identifier, process identifier and thread identifier. S202, determine the application semantic time interval based on the application start time and the application end time, determine the infrastructure operation time interval based on the infrastructure start time and the infrastructure end time, and generate timing information based on the application semantic time interval and the infrastructure operation time interval; S203, associate the unified tracking identifier with the application semantic operation record corresponding to the model call context, and associate the unified tracking identifier with other application semantic operation records in the same model call process level by level along the parent-child hierarchy; S204, based on the running node identifier, process identifier and thread identifier respectively in the application running association information and the infrastructure running association information, filter the infrastructure running operation records that correspond to the application semantic operation records that have been associated with the unified tracking identifier from the infrastructure running operation records, and associate the unified tracking identifier with the filtered infrastructure running operation records; S205, based on the unified tracking identifier, filter application semantic operation records and infrastructure operation records belonging to the same model call process, and according to the time sequence information, associate the infrastructure operation operation records whose infrastructure operation time interval is within the application semantic time interval with the application semantic operation record corresponding to the application semantic time interval with the shortest duration, and generate a cross-layer mapping relationship. S206, associating the application semantic operation record and the infrastructure operation record according to the cross-layer mapping relationship, and arranging the application semantic operation record and the infrastructure operation record according to the time sequence information to generate full-link tracing data.

[0062] In this embodiment, the unified tracking identifier is extracted from the model invocation context and used to distinguish and aggregate application semantic operation records and infrastructure operation records generated from the same model invocation. The model invocation context can consist of request context, thread context, process context, or cross-service passed fields, and the identifier remains consistent during model access, retrieval, tool invocation, and output generation. The application semantic operation record contains the application operation content and runtime attributes, and the parent-child hierarchy between records is used to represent the inclusion or invocation relationship between the model invocation and its internal operations. The application start time and application end time are used to describe the effective duration of the application semantic operation. The runtime node identifier, process identifier, and thread identifier in the application runtime association information are used to indicate the actual runtime location where the application semantic operation occurred.

[0063] Infrastructure operation logs are used to store operational events generated by resource scheduling, data transmission, communication, and input / output access. The infrastructure start and end times describe the duration of infrastructure operations. Infrastructure operation association information uses running node identifiers, process identifiers, and thread identifiers to record the node, process, and thread to which the operational event belongs, enabling comparable operational attributes between application semantic operation logs and infrastructure operation logs.

[0064] The application start time and application end time are combined to form the application semantic time interval, and the infrastructure start time and infrastructure end time are combined to form the infrastructure operation time interval. Time sequence information includes the start position, end position, duration, inclusion relationship, and arrangement relationship of each time interval. When different operating nodes use different timing sources, the time fields can be corrected based on clock offset, or the local time can be converted to a unified time base to reduce the impact of time deviations between nodes on record matching.

[0065] The model invocation context and application semantic operation record can be associated through request identifiers, invocation entry identifiers, or context storage locations, and a unified tracking identifier is written to the corresponding application semantic operation record. After the parent application semantic operation record completes the identifier association, the unified tracking identifier is passed down level by level along the parent-child hierarchy to the child operation records, so that the prompt content processing, vector retrieval, tool invocation, and output generation records within the same model invocation have the same identifier. When nested invocations or branch invocations exist, each child record retains its own operation identifier while inheriting the unified tracking identifier of its parent model invocation.

[0066] Application semantic operation records associated with a unified tracking identifier are used as the screening criterion. The application runtime association information is compared with the runtime node identifier, process identifier, and thread identifier in the infrastructure runtime association information. Infrastructure runtime operation records with consistent identifiers or satisfying preset mapping relationships are identified as candidate records. When a thread switches, the new thread affiliation can be determined based on the intra-process thread mapping information; when a model call runs across processes, process mapping can be established based on the inter-process context passing records; when running across nodes, record screening can be completed based on the unified tracking identifier and runtime node mapping information carried in the request message. The filtered infrastructure runtime operation records are associated with the same unified tracking identifier, thereby determining the model call process to which the two types of records belong.

[0067] Application semantic operation records and infrastructure operation records are collected separately according to a unified tracking identifier, and then time interval matching is performed based on time sequence information. If an infrastructure operation time interval falls within an application semantic time interval, it indicates that the corresponding resource operation occurred during the duration of the application semantic operation. When an infrastructure operation time interval falls within multiple nested application semantic time intervals, the application semantic time interval with the shortest duration is selected to classify resource operations into more granular application semantic operations, reducing matching bias caused by resource operations being classified as broader upper-level operations. Records not fully included but overlapping with application semantic time intervals can be assigned based on the overlap duration, start time distance, or end time distance.

[0068] The cross-layer mapping relationship records the identification relationships, operational attribute relationships, and time interval relationships between application semantic operation records and infrastructure operation records. Based on the cross-layer mapping relationship, application records such as model calls, prompt content processing, vector retrieval, tool calls, or output generation are combined with resource scheduling, storage and transmission, communication, and input / output records that occur during their duration. The combined records are arranged according to time sequence information and retain a unified tracking identifier, application operation content, infrastructure operation status, and mapping relationship between the two types of records, generating full-process tracking data covering the same model call application operation and resource operation process.

[0069] This embodiment aggregates two types of tracking records generated from the same model call by using a unified tracking identifier, and filters the running affiliation by using running node identifier, process identifier, and thread identifier. Then, it establishes a cross-layer mapping based on the inclusion relationship of time intervals and duration. This can reduce mismatches and omissions between application semantic operations and infrastructure operation operations, and organize records scattered across different collection levels according to the same model call and actual occurrence order, thereby improving the integrity and correlation accuracy of the entire tracking data.

[0070] In one embodiment, such as Figure 5 As shown, step S30 above includes: S301, extract the unified tracking identifier, session identifier, workflow identifier, node identifier, parent node identifier, branch identifier, retry identifier, business context field, field source node identifier, context version identifier, and timing information from the end-to-end tracking data, and merge them according to the node identifier to generate an execution node record associated with the unified tracking identifier; S302, the execution node records are collected according to the session identifier to generate a session node group, and each session node group is divided according to the workflow identifier to generate a workflow node group; S303, within each workflow node group, the parent-child call relationship is determined based on the node identifier and the parent node identifier, the branch call relationship is determined based on the branch identifier, the retry call relationship is determined based on the retry identifier, and the parent-child call relationship, the branch call relationship, and the retry call relationship are combined into a call relationship; S304, taking the execution node record corresponding to the field source node identifier as the source node, determine the transmission direction and receiving node of the business context field along the calling relationship, associate the business context field from the source node to the receiving node according to the transmission direction, and determine the version succession order of the business context field corresponding to the receiving node according to the context version identifier, and generate a context transmission relationship. S305, arrange the execution node records in each workflow node group according to the timing information, and organize the execution node records based on the call relationship and the context passing relationship to generate dynamic call path data associated with the unified tracking identifier.

[0071] In this embodiment, a unified tracing identifier, session identifier, workflow identifier, and fields describing node relationships and context flow are read from the complete tracing data. The unified tracing identifier distinguishes different model call processes and ensures that each node record belongs to the same call. The session identifier aggregates multiple rounds of requests and responses generated during the same interaction, and the workflow identifier distinguishes the execution processes performing different tasks within the same session. The node identifier uniquely identifies running nodes such as model calls, retrieval, tool calls, conditional judgments, or result generation; the parent node identifier points to the parent node that directly triggered the current node. The branch identifier distinguishes parallel branches, conditional branches, or routing branches, and the retry identifier marks repeated calls triggered by failure, timeout, or return status.

[0072] The start records, end records, call status, input / output information, and resource operation information generated by the same node are merged according to the node identifier, and the merge result is saved as an execution node record. The execution node record retains a unified tracking identifier, so that different nodes can still trace back to the same model call in subsequent aggregation processes. Nodes with the same identifier but discontinuous record times or different retry identifiers can be formed into different execution node records to avoid merging multiple runs into a single run.

[0073] Execution node records are grouped according to session identifiers, forming session node groups from model calls, retrievals, and tool calls within the same multi-round interaction. Session node groups are then further divided according to workflow identifiers, forming workflow node groups for different tasks, different agent execution processes, or different business branches. A session can include multiple workflow node groups, and a workflow node group can include multiple execution node records for sequential execution, parallel execution, branch execution, and retry execution.

[0074] Within a workflow node group, parent node identifiers are matched with node identifiers to determine the parent-child call relationship between superior and subordinate nodes. Nodes with the same branch identifier or triggered by the same branch are organized into branch call relationships, and nodes with the same retry node and continuously changing retry identifiers are organized into retry call relationships. Parent-child call relationships describe the direct call direction, branch call relationships describe the branching and merging of the call process, and retry call relationships describe multiple runs of the same operation. These three types of relationships are combined to form call relationships, which represent the connection method between nodes within the workflow node group.

[0075] Business context fields may include prompt fragments, search results, tool return values, intermediate states, routing results, or control parameters. The field source node identifier is used to determine the execution node record that generated the business context field. Using the corresponding execution node record as the source node, the downstream node capable of receiving the field is found along the call relationship, and the transmission direction is determined based on the node's call direction. After the business context field is associated with the receiving node, the version succession order of the field among different nodes is determined based on the context version identifier. When a field is updated, appended, or replaced during transmission, a new context version identifier is written to the updated field, and the succession relationship between the previous and subsequent versions is recorded, thereby generating the context transmission relationship.

[0076] Timing information is used to determine the start time, end time, duration, and order of execution node records. Execution node records in each workflow node group are arranged according to timing information, and the operational connections between nodes are described through call relationships, while the flow of business data between nodes is described through context passing relationships. The arranged node records and these two types of relationships together constitute dynamic call path data, which is associated with a unified tracking identifier, enabling continuous recording of session structure, workflow structure, node execution order, branch changes, retry processes, and context version changes within the same model call.

[0077] This embodiment hierarchically aggregates execution node records through session identifiers and workflow identifiers, and establishes various call relationships using node identifiers, parent node identifiers, branch identifiers, and retry identifiers. Then, based on the field source node identifier and context version identifier, it records the transmission direction and version succession order of the business context. This can organize scattered node records into dynamic call path data with a clear running order, call structure, and data flow relationship, thereby improving the completeness of the restoration of complex model call processes and the accuracy of node relationship identification.

[0078] In one embodiment, such as Figure 6 As shown, step S40 above includes: S401, determine the execution node from the dynamic call path data, and extract the unified tracking identifier and the node identifier, call relationship, context passing relationship, node start time, node end time, call status, retry count and resource usage corresponding to each execution node from the dynamic call path data. Determine the node duration based on the node start time and node end time, and generate node running characteristic data. S402, obtain historical dynamic call path data, extract historical normal operation feature data corresponding to each node identifier from the historical dynamic call path data, determine node duration deviation, call status difference, retry number deviation and resource usage deviation based on the node operation feature data and the historical normal operation feature data, and generate node abnormal feature data; S403, based on preset anomaly judgment conditions, filter out abnormal execution nodes from the node anomaly feature data, and determine the node start time corresponding to each abnormal execution node as the anomaly start time to generate anomaly location data. S404, using the abnormal execution node in the abnormal location data as the tracing object, filter other abnormal execution nodes whose abnormal start time is no later than the current downstream abnormal execution node along the call relationship upstream, determine the tracing object and the other abnormal execution nodes as candidate abnormal propagation nodes, arrange the candidate abnormal propagation nodes according to the order of the call relationship and the abnormal start time, filter the target context propagation relationship from the context propagation relationship where the propagation start point and the propagation end point are adjacent to two candidate abnormal propagation nodes, and generate an abnormal propagation path based on the candidate abnormal propagation nodes and the target context propagation relationship; S405, from the candidate anomaly propagation nodes included in the anomaly propagation path, select the candidate anomaly propagation node that starts the anomaly earliest and is continuously connected to other candidate anomaly propagation nodes downstream along the target context propagation relationship, determine the root cause node, determine the node identifier corresponding to the root cause node as the root cause node identifier, and generate root cause analysis data based on the anomaly propagation path, the candidate anomaly propagation nodes, the root cause node, the root cause node identifier, the node anomaly feature data corresponding to the root cause node, and the unified tracking identifier.

[0079] In this embodiment, each execution node is identified from the dynamic call path data, and different model call processes are distinguished according to a unified tracking identifier. Node identifiers are used to maintain the correspondence between execution nodes in current and historical data; call relationships are used to indicate the triggering direction and upstream / downstream connections between execution nodes; and context passing relationships are used to indicate the passing direction of business context fields between execution nodes. The node start time and node end time record the time positions when the execution node starts and ends, respectively, and the time difference between the two forms the node duration. Call status can include normal completion, call failure, call timeout, call cancellation, or return an exception; the retry count records the number of times the same operation is run again due to failure, timeout, or unsatisfactory results; resource usage can include processor usage, acceleration resource usage, storage access usage, memory usage, or communication transmission usage. The node duration, call status, retry count, and resource usage are combined according to the node identifier to form node operation characteristic data that reflects the running status of each execution node.

[0080] Historical normal operation characteristic data is extracted from historical dynamic call path data corresponding to the same node identifier. Historical records can be filtered by model version, resource configuration, request type, or time range, excluding records corresponding to call failures, timeouts, and resource anomalies. The duration, call status, number of retries, and resource usage of the filtered nodes are statistically analyzed to form the normal operation range for each execution node. Node duration deviation can be determined based on the difference between the current duration and the historical duration range; call status deviation can be determined based on whether the current call status is consistent with the historical normal call status; retry count deviation can be determined based on the change in the current retry count distribution relative to the historical retry count; resource usage deviation can be determined based on the degree to which the current resource usage exceeds or falls below the historical normal range. These deviations are combined with the corresponding node identifier to generate node anomaly characteristic data.

[0081] Preset anomaly detection criteria can be applied individually to deviations in duration, call status, retry count, and resource usage, or multiple deviations can be combined for judgment. When any deviation meets the corresponding condition, or when the combined result of multiple deviations meets the corresponding condition, the corresponding execution node is filtered as an abnormal execution node. The start time of the abnormal execution node is recorded as the anomaly start time, used to compare the occurrence order of different abnormal execution nodes. The abnormal execution node, the anomaly start time, the node identifier, and the corresponding node anomaly characteristic data together constitute the anomaly location data, ensuring that the anomaly location corresponds to the anomaly state.

[0082] Using the abnormal execution nodes in the anomaly location data as the tracing objects, upstream execution nodes are checked level by level along the call relationship. If the start time of the anomaly in an upstream abnormal execution node is no later than that of the current downstream abnormal execution node, it indicates that the upstream anomaly has the potential to propagate downstream in time. The tracing object and the upstream abnormal execution nodes that meet the time condition are identified as candidate anomaly propagation nodes. When there are multiple upstream branches, candidate anomaly propagation nodes can be screened along each branch separately; when there are retry calls, the corresponding nodes can be retained based on the call relationship between the retry node and the original calling node. Candidate anomaly propagation nodes are arranged according to the call direction and the start time of the anomaly, so that adjacent nodes simultaneously satisfy the upstream-downstream relationship and the time succession relationship.

[0083] Records whose starting and ending points fall on adjacent candidate exception propagation nodes are selected from the context propagation relationships to form the target context propagation relationship. The target context propagation relationship may include the propagation field, field version, source node, receiving node, and propagation direction, used to determine whether the context content generated or modified by the upstream exception node is received by the downstream exception node. Candidate exception propagation nodes and the target context propagation relationship are organized according to the call direction to generate exception propagation paths. When there is only one exception execution node, the exception propagation path can consist of a single candidate exception propagation node; when there are multiple branch propagation processes, multiple exception propagation paths can be formed respectively.

[0084] From the candidate anomaly propagation nodes included in the anomaly propagation path, the node with the earliest anomaly start time is selected, and it is checked whether this node can continuously connect to downstream candidate anomaly propagation nodes along the target context. Candidate anomaly propagation nodes that satisfy the conditions of being earlier in time, continuous propagation, and able to connect to subsequent anomaly nodes are identified as root cause nodes. The node identifier corresponding to the root cause node forms the root cause node identifier, used to uniquely locate the root cause node in the anomaly propagation path. The anomaly propagation path, candidate anomaly propagation nodes, root cause nodes, root cause node identifiers, node anomaly feature data corresponding to the root cause node, and unified tracking identifiers together constitute root cause analysis data, maintaining a structured relationship between the anomaly occurrence location, propagation direction, context propagation process, and root cause nodes.

[0085] This embodiment identifies abnormal nodes and their abnormal start times by comparing the duration, call status, retry count, and resource usage of each execution node with their corresponding historical normal operation characteristics. Based on the call relationship, candidate abnormal propagation nodes with temporal succession are screened, and the abnormal propagation process between adjacent nodes is verified using the context passing relationship, thus forming an abnormal propagation path with a clear propagation direction. Then, the root cause node that occurs earlier and can continuously connect to downstream abnormal nodes is screened from the abnormal propagation path, thereby improving the accuracy of abnormal location, propagation process identification, and root cause judgment.

[0086] In one embodiment, such as Figure 7 As shown, step S50 above includes: S501, extract model identifier, generation unit consumption, resource object identifier, resource usage, resource usage duration, model input content, model output content, retrieval reference content, stage identifier, stage start time and stage end time from the dynamic call path data to generate multidimensional analysis feature data; S502, obtain model consumption conversion information and resource consumption conversion information, determine model consumption conversion parameters from the model consumption conversion information based on the model identifier in the multidimensional analysis feature data, determine resource consumption conversion parameters from the resource consumption conversion information based on the resource object identifier, determine model call cost based on the generation unit consumption and the model consumption conversion parameters, determine resource usage cost based on the resource usage, the resource usage duration and the resource consumption conversion parameters, and perform call cost analysis based on the model call cost and the resource usage cost to generate cost analysis data; S503, based on the model input content and model output content in the multidimensional analysis feature data, determine semantic matching data, based on the model output content and the retrieval reference content, determine factual consistency data, identify security anomaly data from the model output content, and perform output quality analysis based on the semantic matching data, the factual consistency data and the security anomaly data to generate quality analysis data; S504, determine the duration of a stage based on the stage start time and stage end time in the multidimensional analysis feature data, associate the stage duration, resource usage, and resource usage duration with the stage identifier, generate stage performance distribution data, and perform runtime performance analysis based on the stage performance distribution data to generate performance analysis data.

[0087] In this embodiment, model identifiers, generation unit consumption, resource object identifiers, resource usage, resource usage duration, and content and stage fields are read from the dynamic call path data and organized according to node affiliation and time range. The model identifier can consist of the model name, version, deployment instance, or access endpoint, used to match the corresponding consumption conversion information. The generation unit consumption can include the number of input generation units, the number of output generation units, and their combined value. The resource object identifier is used to distinguish processors, accelerator cards, memory, storage space, or communication resources; the resource usage is used to record the occupancy ratio, transmission volume, read / write volume, or memory usage; and the resource usage duration is used to record the duration for which the corresponding resource is invoked or occupied.

[0088] The model input includes prompts, context, and control parameters submitted to the model. The model output includes generated text, structured results, or status information. Reference content is derived from vector retrieval, knowledge retrieval, or external data queries, used to verify the factual content in the model output. Stage identifiers distinguish the operational stages, such as prompt organization, retrieval, tool invocation, model inference, and output generation. Stage start and end times define the time range of each stage. These fields are aggregated according to the corresponding model invocation node and operational stage to form multidimensional analytical feature data.

[0089] Model consumption conversion information records the conversion relationship between different model identifiers and generation unit consumption. Model consumption conversion parameters can be configured separately according to model version, input generation unit type, and output generation unit type. The corresponding parameter is selected based on the model identifier, and the generation unit consumption is converted according to the parameter to determine the model call cost. Resource consumption conversion information records the conversion relationship between different resource objects and resource usage and duration. Resource consumption conversion parameters can be set according to resource type, configuration specifications, or usage range. The corresponding parameter is selected based on the resource object identifier, and the resource usage cost is determined based on resource usage and duration. Model call cost and resource usage cost can be summarized by node, stage, or complete call process to form cost analysis data.

[0090] The model input and output can be converted into semantic representations, and then analyzed using semantic similarity, semantic entailment, target field matching, or task-specific methods. Figure 1 Consistency is determined by semantic matching data. Semantic matching data can record the overall matching degree, as well as whether each objective in the input requirements is responded to in the output content.

[0091] Factual statements in the model's output can be extracted according to semantic units and matched with supporting information in the retrieved reference content. Output facts are recorded as consistent when supported by the reference content, as contradictory when they conflict with the reference content, and as lacking evidence when no supporting content can be found, thus forming factual consistency data. Security anomaly data can be generated through content classification, sensitive field identification, command attack identification, and abnormal output pattern detection. This data is used to record privacy leaks, dangerous content, unauthorized command responses, or abnormal output formats. Semantic matching data, factual consistency data, and security anomaly data are combined according to their corresponding calling nodes to form quality analysis data.

[0092] The duration of a phase is determined by its end and start times. Based on the phase identifier, the phase duration is correlated with resource usage and duration within the same time frame, forming phase performance distribution data. This data records the percentage of time spent, resource usage distribution, waiting time, and performance differences between phases. Based on the duration and resource usage status of each phase, phases with concentrated durations, abnormal resource usage, or increased waiting times can be identified, generating performance analysis data.

[0093] This embodiment categorizes and organizes model identification, generation unit consumption, resource usage, input and output content, retrieval reference content, and stage timing to obtain model call cost, resource usage cost, semantic matching status, factual consistency status, security anomaly status, and stage performance distribution. This allows cost, quality, and performance analyses to be mapped to specific call nodes and running stages, thereby improving the completeness and accuracy of multidimensional analysis results in the model call process.

[0094] In one embodiment, such as Figure 8 As shown, step S60 above includes: S601, extract the abnormal propagation path, candidate abnormal propagation node, root cause node, root cause node identifier, node abnormal feature data and unified tracking identifier from the root cause analysis data; extract the model call cost, resource usage cost, unified tracking identifier and node identifier from the cost analysis data; extract the semantic matching data, fact consistency data, security anomaly data, unified tracking identifier and node identifier from the quality analysis data; and extract the stage performance distribution data, unified tracking identifier and node identifier from the performance analysis data to generate multi-dimensional insight feature data; S602, according to the unified tracking identifier, merge the multi-dimensional insight feature data, associate the root cause node and the node anomaly feature data with the candidate anomaly propagation node in the anomaly propagation path whose node identifier is the same as the root cause node identifier, and according to the node identifier, associate the model call cost, the resource usage cost, the semantic matching data, the fact consistency data, the security anomaly data and the stage performance distribution data with the candidate anomaly propagation node in the anomaly propagation path whose node identifier is the same, to generate multi-dimensional insight associated data; S603, Generate root cause insight items based on the root cause nodes and node anomaly feature data in the multidimensional insight association data, and divide the multidimensional insight association data according to the node identifier to generate node insight association data corresponding to each candidate anomaly propagation node. S604, For each candidate anomaly propagation node, when the corresponding node insight associated data includes model call cost and resource usage cost, generate a cost insight item based on the model call cost and the resource usage cost; S605, when the corresponding node insight associated data includes semantic matching data, factual consistency data, and security anomaly data, a quality insight item is generated based on the semantic matching data, the factual consistency data, and the security anomaly data; S606, When the corresponding node insight associated data includes stage performance distribution data, a performance insight item is generated based on the stage performance distribution data; S607, associate the root cause insight item with the root cause node identifier, associate the cost insight item, the quality insight item and the performance insight item with the node identifier of the corresponding candidate anomaly propagation node, and combine the root cause insight item, the cost insight item, the quality insight item and the performance insight item into insight item data; S608, according to the node identifier associated with each insight item in the insight item data, associate each insight item with a candidate anomaly propagation node with the same node identifier in the anomaly propagation path, organize the insight item data according to the order of each candidate anomaly propagation node in the anomaly propagation path, and generate a link insight result based on the organized insight item data and the anomaly propagation path.

[0095] In this embodiment, the following are read from the root cause analysis data: anomaly propagation path, candidate anomaly propagation nodes, root cause node, root cause node identifier, node anomaly feature data, and unified tracking identifier. The anomaly propagation path stores the propagation direction and arrangement relationship between anomaly nodes. Candidate anomaly propagation nodes constitute the node set in the path. The root cause node identifier is used to locate the root cause node within this node set. The node anomaly feature data is used to record the root cause node's duration deviation, call status difference, retry count deviation, or resource usage deviation. The unified tracking identifier is used to distinguish different model call processes, preventing the analysis content generated by different calls from being mixed and aggregated.

[0096] The cost analysis data retrieves model call costs, resource usage costs, unified tracking identifiers, and node identifiers. The quality analysis data retrieves semantic matching data, factual consistency data, security anomaly data, unified tracking identifiers, and node identifiers. The performance analysis data retrieves stage performance distribution data, unified tracking identifiers, and node identifiers. Model call costs reflect the discounted cost of generation units, and resource usage costs reflect the discounted resource usage and duration. Semantic matching data reflects the matching status between model output and input requirements. Factual consistency data reflects the support or conflict status between model output and retrieved reference content. Security anomaly data records the type and location of anomalies. Stage performance distribution data records the duration, resource usage, and resource usage duration of each operational stage. The combination of these data and identifiers forms multi-dimensional insight feature data.

[0097] Multidimensional insight feature data is merged according to a unified tracking identifier, ensuring that root cause, cost, quality, and performance information belonging to the same model call enters the same dataset. Root cause nodes and node anomaly feature data are associated with candidate anomaly propagation nodes with the same node identifier in the anomaly propagation path through the root cause node identifier. Model call cost, resource usage cost, semantic matching data, factual consistency data, security anomaly data, and stage performance distribution data are associated with candidate anomaly propagation nodes with the same node identifier through their respective carried node identifiers. The association results retain the data source, node affiliation, and anomaly propagation order, forming multidimensional insight associated data.

[0098] Root cause nodes and their anomaly characteristics are organized into root cause insights, which may include root cause node identifiers, anomaly types, anomaly deviations, and anomaly initiation information. Multidimensional insight-related data are categorized by node identifiers; data belonging to the same candidate anomaly propagation node are grouped into the same node insight-related data. Each node insight-related data retains existing content from the cost, quality, and performance dimensions, avoiding cross-combination of analysis results from different nodes.

[0099] When node insights include model call costs and resource usage costs, these two types of cost data are organized according to the calling node to generate cost insight items. When node insights include semantic matching data, factual consistency data, and security anomaly data, these three types of quality data are organized according to data type and anomaly status to generate quality insight items. When node insights include stage performance distribution data, stage identifiers, stage durations, and resource usage status are organized to generate performance insight items. If a node lacks partial analysis data, only insight items that meet the data conditions are generated.

[0100] Root cause insights are associated with root cause node identifiers, while cost, quality, and performance insights are associated with the node identifiers of their respective candidate anomaly propagation nodes. These insights are combined into insight item data, which is then written into candidate anomaly propagation nodes with the same node identifier in the anomaly propagation path, based on the node identifier carried by each insight item. The insight item data is organized according to the order of the candidate anomaly propagation nodes in the anomaly propagation path, ensuring that the anomaly source, node anomaly state, call cost, output quality, and runtime performance are presented continuously in the direction of anomaly propagation, generating insight results.

[0101] This embodiment unifies the tracking identifiers to merge root cause, cost, quality, and performance analysis data generated from the same model call, and uses root cause node identifiers and node identifiers to associate data from different dimensions to corresponding candidate anomaly propagation nodes, thus avoiding cross-call data mixing and cross-node data mismatch. By dividing multidimensional insight-related data by nodes and generating corresponding insight items according to the actual data contained, and then organizing each insight item according to the node order in the anomaly propagation path, the anomaly source, cost status, output quality, and operational performance form insight results with clear node affiliation and propagation order, thereby improving the correlation accuracy of multidimensional analysis data and the completeness of the anomaly process presentation.

[0102] In one embodiment, within the fintech business domain, a large-scale model application can be deployed on an insurance claims review platform to handle vehicle insurance claim acceptance, policy liability inquiry, accident material verification, historical claims retrieval, fraud risk identification, and claims opinion generation. After a user submits an accident description, vehicle information, repair list, and video materials, the model application sequentially executes the following steps: rendering of the prompt content template, policy clause vector retrieval, historical case retrieval, calling the repair price verification tool, calling the anti-fraud model, and generating the claims opinion. Application-layer probes are set at the model call entry point, prompt content processing entry point, vector retrieval entry point, tool call entry point, and streaming generation entry point, recording the operation identifier, parent operation identifier, model name, prompt content summary, retrieval document identifier, generation unit consumption, input / output summary, start time, end time, and call status for each application semantic operation. Kernel-layer probes collect CPU scheduling, GPU kernel tasks, video memory copying, network transmission and reception, vector database access, and file input / output events corresponding to the model inference process, recording running nodes, processes, threads, and resource objects, forming application semantic tracking data and infrastructure operation tracking data, respectively.

[0103] When a claim request enters the platform, a unified tracking identifier is generated and passed between the claim orchestration service, vector retrieval service, anti-fraud service, model service, and tool service, along with the request context. Application semantic records and infrastructure operation records generated by different service nodes are aggregated according to the unified tracking identifier and corrected using a unified time base or clock offset to form comparable time-series information. Running node identifiers, process identifiers, and thread identifiers are used to filter infrastructure operation records belonging to the same claim request. Time intervals are used to associate graphics processor inference, video memory transfer, network communication, and database access records with corresponding prompt content processing, document retrieval, tool execution, or model generation operations. When a video memory copy falls within the time range of both a model call operation and a streaming generation operation, the record can be associated with the shorter-duration streaming generation operation, thus forming complete tracking data covering the claim request entry point, application operation, model inference, and underlying resource operation process.

[0104] When the same complainant submits supplementary repair invoices, accident photos, or liability determination materials, multiple tracking records are grouped into the same claims session based on session identifiers. Workflows are then divided according to tasks such as material review, liability assessment, risk verification, and claims opinion generation. Each execution node retains a node identifier, parent node identifier, branch identifier, and retry identifier to identify sequential calls, conditional branches, parallel verification, and failure retries. For example, if the repair price verification result exceeds a preset range, the workflow branches to the manual review auxiliary node and the historical high-risk case retrieval node; when the vector database times out and is retrieved again, a retry node is generated and its association with the original retrieval node is retained. Policy terms summary, repair price verification results, historical claims records, and anti-fraud scores are passed between different nodes as business context fields. The source of the content and the order of updates are recorded through the field source node identifier and context version identifier, forming dynamic call path data that reflects multiple rounds of interaction, tool calls, branch decisions, and context inheritance processes.

[0105] The platform can save the normal operation characteristics of each node, including the policy retrieval time range, model inference duration, tool call status, retry count, graphics processor usage, and network transmission volume. When the high-percentile latency of a claim review increases significantly, the current node's operation characteristics are compared with historical normal operation characteristics to filter nodes with abnormal duration, call status, retry count, or resource usage, and the start time of the anomaly is recorded. If the vector retrieval node experiences high CPU usage and call timeout early on, and the subsequent anti-fraud node triggers a retry due to lack of retrieval results, and the model generation node runs repeatedly due to incomplete context, an anomaly propagation path is formed along the call relationship and context propagation relationship, and the vector retrieval node is identified as the root cause node. Root cause analysis data can reflect that timeouts occur during the document retrieval stage due to high resource usage of the vector database node, and the anomaly is propagated to the anti-fraud verification and model generation processes through the retrieval result fields.

[0106] Cost analysis determines the model call cost and resource usage cost for each claims review based on model name, number of input generation units, number of output generation units, graphics processor usage time, and storage access volume. It also identifies invalid consumption caused by duplicate content concatenation, invalid document retrieval, repeated tool calls, or cache misses. Output quality analysis performs semantic matching and factual consistency verification between the model-generated claims opinion and user reports, policy terms, and retrieved documents. It also identifies security anomalies such as incorrect policy liability citations, omissions in compensation conditions, leakage of sensitive customer information, and inappropriate risk conclusions. Performance analysis tracks the duration and resource distribution of each stage, including content rendering, vector retrieval, model pre-filling, unit-by-unit generation, and post-processing. It determines that time consumption is concentrated in the data preprocessing, retrieval waiting, model inference, or streaming generation stages, generating cost analysis data, quality analysis data, and performance analysis data.

[0107] Root cause analysis data, cost analysis data, quality analysis data, and performance analysis data are merged according to a unified tracking identifier and associated with specific candidate anomaly propagation nodes in the anomaly propagation path according to node identifiers. Vector retrieval nodes can be associated with resource usage anomalies, retrieval timeouts, resource usage costs, and stage consumption times. Model generation nodes can be associated with generation unit consumption, factual consistency, security anomalies, and performance status of the pre-filling stage and the unit-by-unit generation stage. Each node forms root cause insights, cost insights, quality insights, and performance insights, which are organized into link insight results according to the anomaly propagation order. The survey panel can present the complete process of a claim request from user submission of materials, policy retrieval, risk verification to model generation, while also aggregating anomaly tracking records, resource indicators, cost distribution, quality analysis, and root cause conclusions for R&D, operations, claims review, and risk management personnel to conduct troubleshooting and business analysis.

[0108] In one embodiment, a link insight device based on a unified tracking identifier is provided, which corresponds one-to-one with the link insight method based on a unified tracking identifier in the above embodiments. (Refer to...) Figure 9 , Figure 9 This is a functional module diagram of a preferred embodiment of the link insight device based on unified tracking identifiers of the present invention. The link insight device based on unified tracking identifiers includes: a hybrid probe acquisition module 10, a full-link association module 20, a call path construction module 30, a root cause analysis module 40, a multi-dimensional analysis module 50, and a link insight module 60. Detailed descriptions of each functional module are as follows: The hybrid probe acquisition module 10 is used to collect application semantic data during the model call process through application layer probes to generate application semantic tracking data, and to collect infrastructure operation data through kernel layer probes to generate infrastructure operation tracking data. The end-to-end association module 20 is used to obtain a unified tracking identifier and time sequence information, and associate the application semantic tracking data with the infrastructure operation tracking data based on the unified tracking identifier and the time sequence information to generate end-to-end tracking data. The call path construction module 30 is used to organize sessions and workflows based on the full-link tracing data, determine call relationships and context passing relationships, and generate dynamic call path data based on the call relationships and context passing relationships. Root cause analysis module 40 is used to perform anomaly location and root cause tracing based on the dynamic call path data, and generate root cause analysis data; The multidimensional analysis module 50 is used to perform call cost analysis, output quality analysis, and operational performance analysis based on the dynamic call path data, and generate cost analysis data, quality analysis data, and performance analysis data. Link insight module 60 is used to generate link insight results based on the root cause analysis data, the cost analysis data, the quality analysis data, and the performance analysis data.

[0109] In one embodiment, the hybrid probe acquisition module 10 is specifically used for: Application layer probes are set at the model call entry, prompt content processing entry, vector retrieval entry, tool call entry, and streaming generation entry. The application layer probes record the application semantic operation identifier, parent application semantic operation identifier, application operation type, application start time, application end time, input summary, output summary, call status, running node identifier, process identifier, and thread identifier of each application semantic operation. The model identifier and generation unit consumption corresponding to the model call operation are also recorded to form application semantic data in the model call process. Based on the application semantic operation identifier, the application semantic data corresponding to the same application semantic operation are merged to generate application semantic operation records. Based on the parent application semantic operation identifier, the application start time and the application end time, the application semantic operation records are organized to generate application semantic tracking data. Kernel layer probes are set at kernel event mount points corresponding to resource scheduling, storage transmission, cross-node communication, process switching, and input / output access. The kernel layer probes record the infrastructure operation identifier, infrastructure operation type, infrastructure start time, infrastructure end time, process identifier, thread identifier, running node identifier, and resource object identifier of each infrastructure operation, forming infrastructure operation data. Based on the infrastructure operation identifier, infrastructure operation data corresponding to the same infrastructure operation operation are merged to generate infrastructure operation operation records. Based on the process identifier, the thread identifier, the running node identifier, the resource object identifier, the infrastructure start time, and the infrastructure end time, the infrastructure operation operation records are organized to generate infrastructure operation tracking data.

[0110] In one embodiment, the end-to-end association module 20 is specifically used for: The unified tracking identifier is obtained from the model call context. The application semantic operation record, the parent-child relationship between each application semantic operation record, the application start time, the application end time and the application running association information are obtained from the application semantic tracking data. The infrastructure running operation record, the infrastructure start time, the infrastructure end time and the infrastructure running association information are obtained from the infrastructure running tracking data. The application running association information and the infrastructure running association information both include running node identifier, process identifier and thread identifier. The application semantic time interval is determined based on the application start time and the application end time, the infrastructure operation time interval is determined based on the infrastructure start time and the infrastructure end time, and time sequence information is generated based on the application semantic time interval and the infrastructure operation time interval. The unified tracking identifier is associated with the application semantic operation record corresponding to the model call context, and the unified tracking identifier is associated with other application semantic operation records in the same model call process level by level along the parent-child hierarchy; Based on the running node identifier, process identifier, and thread identifier corresponding to the application running association information and the infrastructure running association information, respectively, the infrastructure running operation records corresponding to the application semantic operation records that have been associated with the unified tracking identifier are selected from the infrastructure running operation records, and the unified tracking identifier is associated with the selected infrastructure running operation records. Based on the unified tracking identifier, application semantic operation records and infrastructure operation records belonging to the same model call process are filtered, and according to the time sequence information, the infrastructure operation operation records whose infrastructure operation time interval is within the application semantic time interval are associated with the application semantic operation records corresponding to the application semantic time interval with the shortest duration, thereby generating a cross-layer mapping relationship. The application semantic operation records and the infrastructure operation records are associated based on the cross-layer mapping relationship, and the application semantic operation records and the infrastructure operation records are arranged according to the time sequence information to generate full-link tracing data.

[0111] In one embodiment, the path construction module 30 is invoked, specifically for: Extract the unified tracking identifier, session identifier, workflow identifier, node identifier, parent node identifier, branch identifier, retry identifier, business context field, field source node identifier, context version identifier, and timing information from the end-to-end tracking data, and merge them according to the node identifier to generate execution node records associated with the unified tracking identifier; The execution node records are aggregated according to the session identifier to generate a session node group, and each session node group is divided according to the workflow identifier to generate a workflow node group. Within each workflow node group, the parent-child call relationship is determined based on the node identifier and the parent node identifier, the branch call relationship is determined based on the branch identifier, and the retry call relationship is determined based on the retry identifier. The parent-child call relationship, the branch call relationship, and the retry call relationship are then combined into a call relationship. Using the execution node record corresponding to the source node identifier of the field as the source node, the transmission direction and receiving node of the business context field are determined along the calling relationship. The business context field is associated from the source node to the receiving node according to the transmission direction. The version succession order of the business context field corresponding to the receiving node is determined according to the context version identifier, and a context transmission relationship is generated. The execution node records in each workflow node group are arranged according to the time sequence information, and the execution node records are organized based on the call relationship and the context passing relationship to generate dynamic call path data associated with the unified tracking identifier.

[0112] In one embodiment, the root cause analysis module 40 is specifically used for: The execution node is determined from the dynamic call path data, and the unified tracking identifier, node identifier, call relationship, context passing relationship, node start time, node end time, call status, retry count and resource usage corresponding to each execution node are extracted from the dynamic call path data. The node duration is determined based on the node start time and node end time, and node running characteristic data is generated. Obtain historical dynamic call path data, extract historical normal operation feature data corresponding to each node identifier from the historical dynamic call path data, determine node duration deviation, call status difference, retry number deviation and resource usage deviation based on the node operation feature data and the historical normal operation feature data, and generate node abnormal feature data; Based on preset anomaly determination conditions, abnormal execution nodes are selected from the node anomaly feature data, and the node start time corresponding to each abnormal execution node is determined as the anomaly start time to generate anomaly location data. Using the abnormal execution nodes in the abnormal location data as the tracing objects, other abnormal execution nodes whose abnormal start time is no later than the current downstream abnormal execution node are filtered upstream along the call relationship. The tracing objects and the other abnormal execution nodes obtained by the filtering are determined as candidate abnormal propagation nodes. The candidate abnormal propagation nodes are arranged in the order of the call relationship and the abnormal start time. Target context propagation relationships with the propagation start point and the propagation end point being two adjacent candidate abnormal propagation nodes are filtered from the context propagation relationship. An abnormal propagation path is generated based on the candidate abnormal propagation nodes and the target context propagation relationship. From the candidate anomaly propagation nodes included in the anomaly propagation path, the candidate anomaly propagation node with the earliest anomaly start time and which is continuously connected to other candidate anomaly propagation nodes downstream along the target context propagation relationship is selected to determine the root cause node. The node identifier corresponding to the root cause node is determined as the root cause node identifier. Root cause analysis data is generated based on the anomaly propagation path, the candidate anomaly propagation nodes, the root cause node, the root cause node identifier, the node anomaly feature data corresponding to the root cause node, and the unified tracking identifier.

[0113] In one embodiment, the multidimensional analysis module 50 is specifically used for: Extract model identifier, generation unit consumption, resource object identifier, resource usage, resource usage duration, model input content, model output content, retrieval reference content, stage identifier, stage start time, and stage end time from the dynamic call path data to generate multidimensional analysis feature data; The system acquires model consumption conversion information and resource consumption conversion information. Based on the model identifier in the multidimensional analysis feature data, it determines the model consumption conversion parameters from the model consumption conversion information. Based on the resource object identifier, it determines the resource consumption conversion parameters from the resource consumption conversion information. Based on the generation unit consumption and the model consumption conversion parameters, it determines the model call cost. Based on the resource usage, the resource usage duration, and the resource consumption conversion parameters, it determines the resource usage cost. Based on the model call cost and the resource usage cost, it performs call cost analysis and generates cost analysis data. Semantic matching data is determined based on the model input content and model output content in the multidimensional analysis feature data. Factual consistency data is determined based on the model output content and the retrieval reference content. Security anomaly data is identified from the model output content. Output quality analysis is performed based on the semantic matching data, the factual consistency data and the security anomaly data to generate quality analysis data. The duration of a stage is determined based on the start and end times of the stage in the multidimensional analysis feature data. The stage duration, resource usage, and resource usage duration are associated with the stage identifier to generate stage performance distribution data. Performance analysis is then performed based on the stage performance distribution data to generate performance analysis data.

[0114] In one embodiment, the link insight module 60 is specifically used for: From the root cause analysis data, we extract the anomaly propagation path, candidate anomaly propagation nodes, root cause nodes, root cause node identifiers, node anomaly feature data, and unified tracking identifiers. From the cost analysis data, we extract the model call cost, resource usage cost, unified tracking identifiers, and node identifiers. From the quality analysis data, we extract the semantic matching data, factual consistency data, security anomaly data, unified tracking identifiers, and node identifiers. From the performance analysis data, we extract the stage performance distribution data, unified tracking identifiers, and node identifiers to generate multi-dimensional insight feature data. The multidimensional insight feature data is merged according to the unified tracking identifier. The root cause node and the node anomaly feature data are associated with the candidate anomaly propagation node in the anomaly propagation path whose node identifier is the same as the root cause node identifier. According to the node identifier, the model call cost, the resource usage cost, the semantic matching data, the fact consistency data, the security anomaly data, and the stage performance distribution data are associated with the candidate anomaly propagation node in the anomaly propagation path whose node identifier is the same, thereby generating multidimensional insight associated data. Root cause insight items are generated based on the root cause nodes and node anomaly feature data in the multidimensional insight associated data, and the multidimensional insight associated data is divided according to the node identifier to generate node insight associated data corresponding to each candidate anomaly propagation node. For each candidate anomaly propagation node, when the corresponding node insight associated data includes model call cost and resource usage cost, a cost insight item is generated based on the model call cost and the resource usage cost; When the corresponding node insight associated data includes semantic matching data, factual consistency data, and security anomaly data, a quality insight item is generated based on the semantic matching data, the factual consistency data, and the security anomaly data; When the corresponding node insight associated data includes stage performance distribution data, a performance insight item is generated based on the stage performance distribution data. Associate the root cause insight item with the root cause node identifier, associate the cost insight item, the quality insight item, and the performance insight item with the node identifier of the corresponding candidate anomaly propagation node, and combine the root cause insight item, the cost insight item, the quality insight item, and the performance insight item into insight item data; According to the node identifier associated with each insight item in the insight item data, each insight item is associated with a candidate anomaly propagation node with the same node identifier in the anomaly propagation path, and the insight item data is organized according to the order of each candidate anomaly propagation node in the anomaly propagation path. Based on the organized insight item data and the anomaly propagation path, a link insight result is generated.

[0115] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements server-side functions or steps of a link insight method based on a unified tracing identifier.

[0116] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements client-side functions or steps of a link insight method based on a unified tracing identifier.

[0117] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Application semantic data is generated by collecting application semantic data during the model call process through application layer probes, and infrastructure operation data is generated by collecting infrastructure operation data through kernel layer probes. Obtain a unified tracking identifier and timing information, and associate the application semantic tracking data with the infrastructure operation tracking data based on the unified tracking identifier and timing information to generate end-to-end tracking data; Based on the end-to-end tracing data, sessions and workflows are organized to determine call relationships and context passing relationships, and dynamic call path data is generated based on the call relationships and context passing relationships. Based on the dynamic call path data, anomaly location and root cause tracing are performed to generate root cause analysis data. Based on the dynamic call path data, call cost analysis, output quality analysis, and runtime performance analysis are performed to generate cost analysis data, quality analysis data, and performance analysis data. Link insight results are generated based on the root cause analysis data, the cost analysis data, the quality analysis data, and the performance analysis data.

[0118] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, and a computer program is stored thereon, which, when executed by a processor, performs the following steps: Application semantic data is generated by collecting application semantic data during the model call process through application layer probes, and infrastructure operation data is generated by collecting infrastructure operation data through kernel layer probes. Obtain a unified tracking identifier and timing information, and associate the application semantic tracking data with the infrastructure operation tracking data based on the unified tracking identifier and timing information to generate end-to-end tracking data; Based on the end-to-end tracing data, sessions and workflows are organized to determine call relationships and context passing relationships, and dynamic call path data is generated based on the call relationships and context passing relationships. Based on the dynamic call path data, anomaly location and root cause tracing are performed to generate root cause analysis data. Based on the dynamic call path data, call cost analysis, output quality analysis, and runtime performance analysis are performed to generate cost analysis data, quality analysis data, and performance analysis data. Link insight results are generated based on the root cause analysis data, the cost analysis data, the quality analysis data, and the performance analysis data.

[0119] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0120] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0122] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0123] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A link insight method based on unified tracking identifiers, characterized in that, Includes the following steps: Application semantic data is generated by collecting application semantic data during the model call process through application layer probes, and infrastructure operation data is generated by collecting infrastructure operation data through kernel layer probes. Obtain a unified tracking identifier and timing information, and associate the application semantic tracking data with the infrastructure operation tracking data based on the unified tracking identifier and timing information to generate end-to-end tracking data; Based on the end-to-end tracing data, sessions and workflows are organized to determine call relationships and context passing relationships, and dynamic call path data is generated based on the call relationships and context passing relationships. Based on the dynamic call path data, anomaly location and root cause tracing are performed to generate root cause analysis data. Based on the dynamic call path data, call cost analysis, output quality analysis, and runtime performance analysis are performed to generate cost analysis data, quality analysis data, and performance analysis data. Link insight results are generated based on the root cause analysis data, the cost analysis data, the quality analysis data, and the performance analysis data.

2. The link insight method based on unified tracking identifier as described in claim 1, characterized in that, Application semantic data is generated by collecting application semantic data during model invocation through application layer probes, and infrastructure operation data is generated by collecting infrastructure operation data through kernel layer probes, including: Application layer probes are set at the model call entry, prompt content processing entry, vector retrieval entry, tool call entry, and streaming generation entry. The application layer probes record the application semantic operation identifier, parent application semantic operation identifier, application operation type, application start time, application end time, input summary, output summary, call status, running node identifier, process identifier, and thread identifier of each application semantic operation. The model identifier and generation unit consumption corresponding to the model call operation are also recorded to form application semantic data in the model call process. Based on the application semantic operation identifier, the application semantic data corresponding to the same application semantic operation are merged to generate application semantic operation records. Based on the parent application semantic operation identifier, the application start time and the application end time, the application semantic operation records are organized to generate application semantic tracking data. Kernel layer probes are set at kernel event mount points corresponding to resource scheduling, storage transmission, cross-node communication, process switching, and input / output access. The kernel layer probes record the infrastructure operation identifier, infrastructure operation type, infrastructure start time, infrastructure end time, process identifier, thread identifier, running node identifier, and resource object identifier of each infrastructure operation, forming infrastructure operation data. Based on the infrastructure operation identifier, infrastructure operation data corresponding to the same infrastructure operation operation are merged to generate infrastructure operation operation records. Based on the process identifier, the thread identifier, the running node identifier, the resource object identifier, the infrastructure start time, and the infrastructure end time, the infrastructure operation operation records are organized to generate infrastructure operation tracking data.

3. The link insight method based on unified tracking identifier as described in claim 1, characterized in that, Obtain a unified tracking identifier and timing information; based on the unified tracking identifier and timing information, associate the application semantic tracking data with the infrastructure operation tracking data to generate end-to-end tracking data, including: The unified tracking identifier is obtained from the model call context. The application semantic operation record, the parent-child relationship between each application semantic operation record, the application start time, the application end time and the application running association information are obtained from the application semantic tracking data. The infrastructure running operation record, the infrastructure start time, the infrastructure end time and the infrastructure running association information are obtained from the infrastructure running tracking data. The application running association information and the infrastructure running association information both include running node identifier, process identifier and thread identifier. The application semantic time interval is determined based on the application start time and the application end time, the infrastructure operation time interval is determined based on the infrastructure start time and the infrastructure end time, and time sequence information is generated based on the application semantic time interval and the infrastructure operation time interval. The unified tracking identifier is associated with the application semantic operation record corresponding to the model call context, and the unified tracking identifier is associated with other application semantic operation records in the same model call process level by level along the parent-child hierarchy; Based on the running node identifier, process identifier, and thread identifier corresponding to the application running association information and the infrastructure running association information, respectively, the infrastructure running operation records corresponding to the application semantic operation records that have been associated with the unified tracking identifier are selected from the infrastructure running operation records, and the unified tracking identifier is associated with the selected infrastructure running operation records. Based on the unified tracking identifier, application semantic operation records and infrastructure operation records belonging to the same model call process are filtered, and according to the time sequence information, the infrastructure operation operation records whose infrastructure operation time interval is within the application semantic time interval are associated with the application semantic operation records corresponding to the application semantic time interval with the shortest duration, thereby generating a cross-layer mapping relationship. The application semantic operation records and the infrastructure operation records are associated based on the cross-layer mapping relationship, and the application semantic operation records and the infrastructure operation records are arranged according to the time sequence information to generate full-link tracing data.

4. The link insight method based on unified tracking identifier as described in claim 1, characterized in that, Based on the end-to-end tracing data, sessions and workflows are organized to determine call relationships and context passing relationships, and dynamic call path data is generated based on the call relationships and context passing relationships, including: Extract the unified tracking identifier, session identifier, workflow identifier, node identifier, parent node identifier, branch identifier, retry identifier, business context field, field source node identifier, context version identifier, and timing information from the end-to-end tracking data, and merge them according to the node identifier to generate execution node records associated with the unified tracking identifier; The execution node records are aggregated according to the session identifier to generate a session node group, and each session node group is divided according to the workflow identifier to generate a workflow node group. Within each workflow node group, the parent-child call relationship is determined based on the node identifier and the parent node identifier, the branch call relationship is determined based on the branch identifier, and the retry call relationship is determined based on the retry identifier. The parent-child call relationship, the branch call relationship, and the retry call relationship are then combined into a call relationship. Using the execution node record corresponding to the source node identifier of the field as the source node, the transmission direction and receiving node of the business context field are determined along the calling relationship. The business context field is associated from the source node to the receiving node according to the transmission direction. The version succession order of the business context field corresponding to the receiving node is determined according to the context version identifier, and a context transmission relationship is generated. The execution node records in each workflow node group are arranged according to the time sequence information, and the execution node records are organized based on the call relationship and the context passing relationship to generate dynamic call path data associated with the unified tracking identifier.

5. The link insight method based on unified tracking identifier as described in claim 1, characterized in that, Based on the dynamic call path data, anomaly localization and root cause tracing are performed to generate root cause analysis data, including: The execution node is determined from the dynamic call path data, and the unified tracking identifier, node identifier, call relationship, context passing relationship, node start time, node end time, call status, retry count and resource usage corresponding to each execution node are extracted from the dynamic call path data. The node duration is determined based on the node start time and node end time, and node running characteristic data is generated. Obtain historical dynamic call path data, extract historical normal operation feature data corresponding to each node identifier from the historical dynamic call path data, determine node duration deviation, call status difference, retry number deviation and resource usage deviation based on the node operation feature data and the historical normal operation feature data, and generate node abnormal feature data; Based on preset anomaly determination conditions, abnormal execution nodes are selected from the node anomaly feature data, and the node start time corresponding to each abnormal execution node is determined as the anomaly start time to generate anomaly location data. Using the abnormal execution nodes in the abnormal location data as the tracing objects, other abnormal execution nodes whose abnormal start time is no later than the current downstream abnormal execution node are filtered upstream along the call relationship. The tracing objects and the other abnormal execution nodes obtained by the filtering are determined as candidate abnormal propagation nodes. The candidate abnormal propagation nodes are arranged in the order of the call relationship and the abnormal start time. Target context propagation relationships with the propagation start point and the propagation end point being two adjacent candidate abnormal propagation nodes are filtered from the context propagation relationship. An abnormal propagation path is generated based on the candidate abnormal propagation nodes and the target context propagation relationship. From the candidate anomaly propagation nodes included in the anomaly propagation path, the candidate anomaly propagation node with the earliest anomaly start time and which is continuously connected to other candidate anomaly propagation nodes downstream along the target context propagation relationship is selected to determine the root cause node. The node identifier corresponding to the root cause node is determined as the root cause node identifier. Root cause analysis data is generated based on the anomaly propagation path, the candidate anomaly propagation nodes, the root cause node, the root cause node identifier, the node anomaly feature data corresponding to the root cause node, and the unified tracking identifier.

6. The link insight method based on unified tracking identifier as described in claim 1, characterized in that, Based on the dynamic call path data, call cost analysis, output quality analysis, and runtime performance analysis are performed to generate cost analysis data, quality analysis data, and performance analysis data, including: Extract model identifier, generation unit consumption, resource object identifier, resource usage, resource usage duration, model input content, model output content, retrieval reference content, stage identifier, stage start time, and stage end time from the dynamic call path data to generate multidimensional analysis feature data; The system acquires model consumption conversion information and resource consumption conversion information. Based on the model identifier in the multidimensional analysis feature data, it determines the model consumption conversion parameters from the model consumption conversion information. Based on the resource object identifier, it determines the resource consumption conversion parameters from the resource consumption conversion information. Based on the generation unit consumption and the model consumption conversion parameters, it determines the model call cost. Based on the resource usage, the resource usage duration, and the resource consumption conversion parameters, it determines the resource usage cost. Based on the model call cost and the resource usage cost, it performs call cost analysis and generates cost analysis data. Semantic matching data is determined based on the model input content and model output content in the multidimensional analysis feature data. Factual consistency data is determined based on the model output content and the retrieval reference content. Security anomaly data is identified from the model output content. Output quality analysis is performed based on the semantic matching data, the factual consistency data and the security anomaly data to generate quality analysis data. The duration of a stage is determined based on the start and end times of the stage in the multidimensional analysis feature data. The stage duration, resource usage, and resource usage duration are associated with the stage identifier to generate stage performance distribution data. Performance analysis is then performed based on the stage performance distribution data to generate performance analysis data.

7. The link insight method based on unified tracking identifier as described in claim 1, characterized in that, Based on the root cause analysis data, cost analysis data, quality analysis data, and performance analysis data, link insight results are generated, including: From the root cause analysis data, we extract the anomaly propagation path, candidate anomaly propagation nodes, root cause nodes, root cause node identifiers, node anomaly feature data, and unified tracking identifiers. From the cost analysis data, we extract the model call cost, resource usage cost, unified tracking identifiers, and node identifiers. From the quality analysis data, we extract the semantic matching data, factual consistency data, security anomaly data, unified tracking identifiers, and node identifiers. From the performance analysis data, we extract the stage performance distribution data, unified tracking identifiers, and node identifiers to generate multi-dimensional insight feature data. The multidimensional insight feature data is merged according to the unified tracking identifier. The root cause node and the node anomaly feature data are associated with the candidate anomaly propagation node in the anomaly propagation path whose node identifier is the same as the root cause node identifier. According to the node identifier, the model call cost, the resource usage cost, the semantic matching data, the fact consistency data, the security anomaly data, and the stage performance distribution data are associated with the candidate anomaly propagation node in the anomaly propagation path whose node identifier is the same, thereby generating multidimensional insight associated data. Root cause insight items are generated based on the root cause nodes and node anomaly feature data in the multidimensional insight associated data, and the multidimensional insight associated data is divided according to the node identifier to generate node insight associated data corresponding to each candidate anomaly propagation node. For each candidate anomaly propagation node, when the corresponding node insight associated data includes model call cost and resource usage cost, a cost insight item is generated based on the model call cost and the resource usage cost; When the corresponding node insight associated data includes semantic matching data, factual consistency data, and security anomaly data, a quality insight item is generated based on the semantic matching data, the factual consistency data, and the security anomaly data; When the corresponding node insight associated data includes stage performance distribution data, a performance insight item is generated based on the stage performance distribution data. Associate the root cause insight item with the root cause node identifier, associate the cost insight item, the quality insight item, and the performance insight item with the node identifier of the corresponding candidate anomaly propagation node, and combine the root cause insight item, the cost insight item, the quality insight item, and the performance insight item into insight item data; According to the node identifier associated with each insight item in the insight item data, each insight item is associated with a candidate anomaly propagation node with the same node identifier in the anomaly propagation path, and the insight item data is organized according to the order of each candidate anomaly propagation node in the anomaly propagation path. Based on the organized insight item data and the anomaly propagation path, a link insight result is generated.

8. A link insight device based on a unified tracking identifier, characterized in that, The link insight device based on unified tracking identifiers includes: The hybrid probe acquisition module is used to collect application semantic data during the model call process through application layer probes to generate application semantic tracing data, and to collect infrastructure operation data through kernel layer probes to generate infrastructure operation tracing data. The end-to-end association module is used to obtain unified tracking identifiers and time-series information, and associate the application semantic tracking data with the infrastructure operation tracking data based on the unified tracking identifiers and time-series information to generate end-to-end tracking data. The call path construction module is used to organize sessions and workflows based on the end-to-end tracing data, determine call relationships and context passing relationships, and generate dynamic call path data based on the call relationships and context passing relationships. The root cause analysis module is used to locate anomalies and trace root causes based on the dynamic call path data, and generate root cause analysis data. The multidimensional analysis module is used to perform call cost analysis, output quality analysis, and runtime performance analysis based on the dynamic call path data, and generate cost analysis data, quality analysis data, and performance analysis data. The link insight module is used to generate link insight results based on the root cause analysis data, the cost analysis data, the quality analysis data, and the performance analysis data.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a link insight program based on a unified tracing identifier stored in the memory and executable on the processor, wherein the link insight program based on a unified tracing identifier, when executed by the processor, implements the steps of the link insight method based on a unified tracing identifier as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a link insight program based on a unified tracing identifier, which, when executed by a processor, implements the steps of the link insight method based on a unified tracing identifier as described in any one of claims 1-7.

Images